Posts Tagged ‘bias’


Sports, perception and sample-size bias

New York Rangers goalie Henrik Lundqvist is the Tim Tebow of hockey. Actually, he's even better, having won nine of his last ten. Photo by Robert Kowal.

Apologies in advance, but if you don’t follow sports this post may not make much sense.

This afternoon, the Denver Broncos picked up their sixth win in seven games this season with Tim Tebow starting at quarterback. If you haven’t heard, Tebow has what might be called non-traditional passing mechanics, but as many commentators have noted, he “just wins.” There’s a lot that could be said, and already has been said, about the strange way in which quarterbacks are credited with team success in football, but that’s not really the point of this post. Rather, I want to point out how odd it is that seven games — even seven games that include six wins — can be considered so meaningful in football.

This stretch has turned Denver’s season around, to be sure. They lost four of their first five, but now find themselves tied for the lead in their division. But this is only possible because of the NFL’s relatively tiny schedule. Consider that, for a hockey goaltender — probably the only every-game player in North American major team sports with as much impact as a quarterback — six wins in seven games is barely noticeable; it’s less than a tenth of the season. For a baseball player (where there isn’t such a great analogue, since starting pitchers only go every fifth game), six wins in seven games is a good week. You could win your league’s player of the week award in May and be sent down to the minors in June. For Tebow, six wins in seven games is two months and half the season, and it’s especially significant when one of your divisional rivals (San Diego) is imploding at the same time.

But this is all perception; if we’re trying to think about what this seven-game sample means in terms of predicting the larger population of games that is a player’s career, seven tells us nothing. It doesn’t matter that an NFL season is only 16 games long; seven games don’t provide enough observations to reduce the error to an acceptable level. If we were to look at a proportionally similar number of baseball games — 70 — we’d keep the proportion the same, but reduce the sampling error by examining nominally more cases.

So where does this perception error come from? Is it just the kind of rank innumeracy we see in many contexts? Maybe, but I suspect there’s also an important media effect as well. Sports media — both reporters and game broadcasters — and the sports culture they’re embedded in frequently express hostility toward data-driven strategy. Narratives and tradition rule in sports, and when data contradict them it’s because data can’t possibly figure out the relevant “intangibles.” Noting that a seven-game span isn’t really an illuminating sample gets in the way of a lot of narrative structure.

Filed: Science Is Real || 21:49, December 4 || 1 Comment »


What does bias look like?

When we talk about “bias” or “slant” in reporting (and though those are not really great terms, I use them because I think there’s a decent shared understanding of them), we usually talk about it in political terms. It seems to manifest primarily in ideological terms — liberal bias, conservative bias — and the response of most traditional news organizations to such charges has been to make “balance” an explicit goal in their reporting. Sometimes they achieve this goal, sometimes they don’t, sometimes the reporting is good, sometimes it isn’t, but part of the outcome of the charges and the response is that the idea of bias is salient and understandable for the typical political news consumer.

So, OK, even if the audience might not be very good at understanding or perceiving real bias or slant (particularly outside of the hostile media effect), forces have conspired to make them think about it within the confines of political news. But what about other types of news? When people read or see stories about non-political (or at least non-party political) topics, do they think about or perceive slant in the same way? An important sub-question is whether the audience even perceives multiple “sides” to a story in which traditional political divisions are not and the reporter doesn’t present multiple sides. For example, if a local affiliate runs a story that is largely a recut video news release about some dangerous new germ that requires you to stock up on Purell, does the audience see that as a slanted piece? With a week until the AEJMC deadline, I probably should be thinking about more pressing things, like research I actually have in progress, but this has been nagging me for a few days now.

Filed: Super Special Questions || 10:02, March 25 || 2 Comments »