Update: More comments here.

John “Data Crashes ‘R Us” Lott and his ten research assistants try to come to Limbaugh’s defense:

Whether Rush Limbaugh’s comments on Donovan McNabb were “racist,” there is a general agreement that he was factually wrong, that Limbaugh did not know what he is talking about. Yet, what is the evidence? [snip]

The evidence suggests that Rush is right, though the simplest measures indicate that the difference is not huge. Looking at just the averages, without trying to account for anything else, reveals a ten-percent difference in coverage (with 67 percent of stories on blacks being positive, 61 percent for whites).

The evidence suggests Rush is right? Sadly, No! Actually, the evidence suggests that if you do a good enough job of picking a subset of your data, you can find pretty much any result you want.

In the first section of his article, Lott claims to be examining news coverage of quarterbacks, and it was overall bias that Rush had alleged. But what subset of his data did Lott use when presenting his simple comparison of averages? The one that showed articles about black QBs to be 67% positive, compared to 61% for whites? Lott only used weeks in which these QBs played. Should that matter? Big time.

Lott’s total dataset includes 1346 articles, 1013 about white QBs, 333 [was originally listed as 303, then accidentally for about 20 minutes as 343!*] about black QBs. The “simplest measure” shows that white QBs receive positive comments in 57% of all articles, while black QBs’ positive articles only amount to 53%. Though again a small difference, the result is exactly the opposite of what Lott (and Limbaugh) claimed. [Lott’s selection excludes 50 articles about white QBs that are 70% positive, and 41 articles about black QBs, 30 of which are negative. What a convenient feature of the data this turned out to be.]

When it comes to Lott’s regression, excluding weeks where the QB did not play does make sense, since there is no “objective” measure of player quality that can be used to control for the (alleged) race effect. [In the Eagles’ bye week, Donovan McNabb’s new coverage was 3 positive articles, 23 negative ones.] But we were puzzled by this statement:

In addition, I accounted for average differences in media coverage both in the quarterback’s city and the opponent’s city as well as differences across weeks of the season.

But Lott’s dependent variable, i.e. the one whose variance he wants to explain, is the percentage of articles that are positive about a given QB. Had he used the total number of positive articles, controlling for media coverage would have made sense. Here however we fail to see why. Nor do we see why one would want to control for differences across weeks — the independent variable is size neutral. Moreover, the bias alleged is by the media. Whether a given game is covered by 3 or 8 papers, the bias effect should be found regardless.

So what happens if one runs a regression on Lott’s data without “correcting” for the extent of media coverage? To find out, we put our staff of one to work using EasyReg International, a free econometrics program for regression analysis. We imported Lott’s data, and ran a tobit regression keeping the same dependent variable and using as our independent variables: