communication. community. cognition.
Posts tagged measurement
A Cupful of Wisdom
Jul 12th
Soccer is the most boring thing to watch on television.
- America
The cynics are having a field day with the World Cup final, calling Spain’s last-minute-of-overtime 1-0 victory a snoozefest.
I’m not here to argue with them, but it is important to understand why. More >
Podcast: Play in new window | Download
The Dangers of Automated Analysis
Apr 26th
As online conversations increase about a company or brand, so does the need for an intelligent way to dissect the data and look at meaningful trends. Just be wary that not all divisions of data are necessarily useful.
This morning, while monitoring the online mentions of my company from the weekend, I saw the following messages from a Facebook user. The name is redacted, and I am changing the text considerably, so don’t bother searching.
“Heard the boom, and now a line is down in the neighborhood. Have no idea when power is going to be back up. Help me, Alabama Power!” – Sent at noon, Facebook for Blackberry
“Way to go, geniuses. Crew is here to fix, but they don’t have the right parts and have to come back.” – Sent at 2pm, Facebook for Blackberry
“Power is back, and all is right with the world.” – Sent at 4pm
Fortunately, our conversational traffic is not that large. The business model doesn’t lend itself to massive amounts of in-your-face marketing and messaging – and we’re not in the heavy competitive arena that many retail outlets and national brands must navigate. With a smaller amount of traffic, it’s still at the stage where we can look at the individual messages for context.
The monitoring services that will win out will be the ones that can accurately gauge “sentiment” in meaningful ways. If you have a large spike in traffic or mentions, were they positive or negative? And if so, what can you learn about what those messages have in common?
My example above is overly simplistic, but what if we started pulling in data which told us that people who mentioned us using mobile applications (like Facebook for Blackberry) were 80% more likely to give us negative sentiment? Statistically, it would be a valid comparison to make. But how useful is that?
Primer on Causation
There is an upside in having many people in your organization with access to monitoring tools. The more in-touch they are with buzz and conversation, the more likely they will be to think in terms of your customers. But there’s also a danger, if they don’t know how to properly parse that information.
Without going into a dissertation on regression analysis, here are the outcomes you might get from looking at your raw data:
- Causation - the implication that one event is a necessary condition for the other
- Correlation - the implication that two events occur together with regularity, and might share a common cause
- Coincidence - the observation that two events have occurred together, but there is no proof of correlation or causation.

Not all data are good data
Looking at my simple data points above – yes, we have messages that are negative coming from a mobile device. The non-mobile message is positive. Assuming causation, however, would get you in big trouble. On a large scale, this could lead to an enterprise that alters its allocation for mobile strategy, either more resources than required or not enough. (Maybe even abandoning it entirely, because those people are just too mean!)
If one considers correlation, we might find value in digging a little further, to see what else there might be in common. In this case, we might suspect there is a connection somehow, but it would take more digging.
Writing it off as coincidence might be just as dangerous as committing to causation, because then you’ve entered a mindset where those variables will be dismissed more readily in the future. You might have a different set of circumstances later, and the dots you couldn’t connect today might be the lynchpin of a pattern tomorrow.
The Truth
In my case above, we’d all arrive (I hope) at the more sensible conclusion that the messages sent from the mobile device would be more inclined to be negative, because the user is frustrated by a power outage. Once the user can enjoy a full keyboard and computer, the frustration is gone.
The “Blackberry” connection has more to do with the circumstance (a correlation) than it does with a causation.

Useless data lead to useless charts
The above example is so clear and plain to us now, because we had a common tie for those data points: the same user. But what if we saw a similar phenomenon, where the tweets and updates coming from mobile devices were so sharply negative? And there was an occasional message that was positive, from a non-mobile client? Would that be as easy to parse? It’s not just weather-related events, either. Occasionally you’ll find messages from people who are complaining about a line being too long when they want to pay a bill. Those are coming from mobile devices as well.
Instant analytics are a wonderful thing. But if you let them do your job of critical thinking, they will lead you to some very wrong answers. Context is King.
A Lesson on Worth
Mar 26th
I recently ran across one of those messages about how our nation needs to get its priorities in order, because we pay professional athletes so much more than we do teachers.
On the surface, it seems a difficult point to argue, because obviously teaching is important.
This is where an understanding of marginal utility comes into play.
The Difference is the Difference
We recently put our fantasy baseball league together, and guess who one of the top draft priorities was? A catcher for the Minnesota Twins named Joe Mauer. Statistically he’s as good as anyone in the game, but he’s even more valuable as a catcher because of the scarcity at his position.
The numbers will tell you that Albert Pujols is the best pick in the game, but I might bypass him for Mauer because the next guy down in First Base eligibility is not as far a drop as the next catcher beneath Mauer. So it’s not just a game of raw numbers, it’s how much of a marginal difference you get from the swap.
The difference in value (either in the fantasy game or in the real world) is a function of the difference you bring. Major League players are where they are because there is an appreciable difference between hitting a curve ball one out of every four tries instead of one out of every five. Do it one out of every three and you’ll go to All-Star games and the Hall of Fame.
So, why are teachers paid so much less than professional athletes?
Wait, are they?
Statistics Are Mean
We assume we know what teachers make, because on a year to year basis the answer is “not enough.” Teachers are always asking for raises, and in that regard are no different than the rest of us. However, many of the teachers I interviewed during my time in television news were shocked to find out that a starting TV reporter made far less than a starting teacher – and depending upon the station had to drive their own vehicle to cover the news.
We also assume we know what professional athletes make, because we see the vapor trail of zeros in the headlines of the sports page. And if there are enough zeros, it goes in the business pages as well. The USA Today has a salary calculator for the major sports leagues. Here are the stats for median salary for Major League Baseball teams for 2009.
| New York Yankees | $ 5,200,000 |
| New York Mets | $ 2,612,500 |
| Philadelphia Phillies | $ 2,500,000 |
| Detroit Tigers | $ 2,237,500 |
| Chicago Cubs | $ 2,200,000 |
| Cleveland Indians | $ 1,950,000 |
| Los Angeles Angels | $ 1,800,000 |
| Boston Red Sox | $ 1,625,000 |
| Kansas City Royals | $ 1,600,000 |
| Houston Astros | $ 1,550,000 |
| Arizona Diamondbacks | $ 1,500,000 |
| Baltimore Orioles | $ 1,500,000 |
| Milwaukee Brewers | $ 1,347,500 |
| Tampa Bay Rays | $ 1,290,000 |
| Los Angeles Dodgers | $ 1,250,000 |
| Atlanta Braves | $ 1,237,500 |
| Chicago White Sox | $ 1,112,500 |
| Pittsburgh Pirates | $ 1,062,500 |
| Cincinnati Reds | $ 970,000 |
| St. Louis Cardinals | $ 950,000 |
| Toronto Blue Jays | $ 932,500 |
| Colorado Rockies | $ 800,000 |
| San Francisco Giants | $ 661,250 |
| Texas Rangers | $ 555,000 |
| Minnesota Twins | $ 525,000 |
| Washington Nationals | $ 500,000 |
| Seattle Mariners | $ 480,000 |
| Florida Marlins | $ 470,000 |
| San Diego Padres | $ 466,200 |
| Oakland Athletics | $ 410,000 |
It pays to be a Yankee.
Remember, this isn’t the average (or mean), but the median.
Average indicates you dumped the whole payroll together and divided by the number of people who got the paychecks. Median means you picked the person right in the middle, where half make more and half make less. If you put Bill Gates in a room with 100 teachers, the average salary of the room would be a lot higher, but it wouldn’t affect the median all that much.
Still, the guy in the middle of the pack for the lowly Oakland Athletics makes decent money.
Play around with the calculator. The median salaries for NFL teams ranged from $1.3-million for the New York Giants down to $541K for the St. Louis Rams. In the NBA, the “middle-man” in the New York Knicks pecking order gets over $6-million a year, while the median for the Miami Heat is a cool $1.1-million.
But is this the applicable comparison for educators?
Perception is not reality
Again, you look at the headlines and the dollar amounts and compare that to what you see in the classrooms. Look at the parking lot of your nearest public school, and you’ll not see limousines (but you’ll see better cars than you might think.)
What you don’t see in the comparisons is the salaries of all the professional athletes who never made the major leagues. You don’t see the stats on their average career, which ranges anywhere from 2-5 years for the top sports.
You occasionally run across the profile of the minor-league baseball player who is trucking it in Single-A ball for near-minimum wage. Or a reference to Super Bowl MVP Kurt Warner, who worked stocking shelves at a grocery store for $5.50/hour to supplement his income while in minor-league football.
I won’t do the math. Researchers at Penn State already did. In 2004, the median salary for a professional athlete was $48,310 per year.
That same study, when you looked up public education, median earnings for kindergarten and elementary school teachers was between $41,400 to $45,920, based on location. Same range for teachers at junior high and high schools.
Shocking, but not in the way you thought.
Marginal Truths
There is a distinct difference between the guy who hits .280 and the guy who hits .240. It is a statistical measurement that can be correlated to a team’s chance of success.
Many of the factors that make for successful teachers are harder to quantify, so it would seem they are “punished” for dealing with so many intangibles. But there are a couple of other factors to consider.
How much of that lack of measurement is the fault of the teachers’ unions?
The unions typically fight against any standards-based testing, and I can understand why. When tests are administered across a broad area, some will perform better because of external factors. Kids with rich parents who provide additional tutoring and resources, and kids with two parents at home who care about education tend to do very well in school. Children in districts that are poorer and have more fractured home lives are not going to perform as well.
But who says the measurements have to be about raw performance? Why not measure students at the beginning of a year, and again at the end? And we can see how much improvement particular teachers provide, with apples to apples comparisons.
The unions likely won’t stand for that, either.
To Market
As it is, there is a very limited amount of performance-based incentive for teaching achievement. In some states, you’ll get a certain bump for having a Masters degree, or some type of certification in your subject. But most of the compensation is determined by a ladder system based on seniority.
The unions don’t want that level of disparity, though, because it would lead to the kind of income disparity we see in professional sports. For every Alex Rodriguez or Joe Mauer, there are many has-beens and wanna-bes who toil away for “the love of the game,” or some other homily designed to get their minds off their tiny paychecks.
And when it comes down to it, if you are engaged in a vocation where there are so many intangibles – so many factors of value that defy measurement – is one teacher really that much more valuable than another? Well, yes. If given a choice I wouldn’t want to just be thrown in at random.
But I don’t have a choice. There’s no free market for schools (at least not in Alabama.) And the teachers that excel – that really bring additional value to their profession and to the students they reach – they don’t have a free market either. A free market for teachers would provide the basis and incentive for finding a way to measure those intangibles. It would also mean some would end up as rock-stars (as much as their marginal utility would allow,) and some would end up bagging groceries next to young Kurt Warner.
And deep down, I’m not so sure that’s what they want.


