During the oral arguments for Shelby County v. Holder, Chief Justice John Roberts quoted some statistics that, according to his interpretation, showed the turnout ratio of minority voters to white voters was worse in Massachusetts than in any other state. This prompted a quick response from the Massachusetts election officials and a more measured one from Nate Silver on FiveThirtyEight. As the Chief Justice may be learning, statistics are tricky things.
The day after the remarks by the Chief Justice the Globe headline was
Chief justice blasted over Mass. voting ‘cheap shot’
Talk about feeling insulted! The nerve to compare us to Mississippi!
“Do you know which state has the worst ratio of white voter turnout to African-American voter turnout?” Roberts asked Donald Verrilli Jr., solicitor general for the Department of Justice, during Wednesday’s arguments.
“I do not know that,” Verrilli answered.
“Massachusetts,” Roberts responded, adding that even Mississippi has a narrower gap.
Roberts later asked if Verrilli knew which state has the greatest disparity in registration. Again, Roberts said it was Massachusetts.
The problem is, Roberts is woefully wrong on those points, according to Massachusetts Secretary of State William F. Galvin, who on Thursday branded Roberts’s assertion a slur and made a declaration of his own. “I’m calling him out,” Galvin said.
Galvin was not alone in his view. Academics and Massachusetts politicians said that Roberts appeared to be misguided. A Supreme Court spokeswoman declined to offer supporting evidence of Roberts’s view, referring a reporter to the court transcript.
On Thursday, Galvin tried to set the record straight. “We have one of the highest voter registrations in the country,” he said, “so this whole effort to make a cheap-shot point at Massachusetts is deceptive.”
Map of Section 5 Covered Jurisdictions (Photo credit: Wikipedia)
So what’s going on here? Trust Nate Silver to explain.
Section 5 of the Voting Rights Act, which is being challenged by Shelby County, Ala., in the case before the court, requires that certain states, counties and townships with a history of racial discrimination get approval (or “pre-clearance”) from the Department of Justice before making changes to their voting laws. But Chief Justice Roberts said that Mississippi, which is covered by Section 5, has the best ratio of African-American to white turnout, while Massachusetts, which is not covered, has the worst, he said.
Chief Justice Roberts’s statistics appear to come from data compiled in 2004 by the Census Bureau, which polls Americans about their voting behavior as part of its Current Population Survey. In 2004, according to the Census Bureau’s survey, the turnout rate among white voting-aged citizens was 60.2 percent in Mississippi, while the turnout rate among African-Americans was higher, 66.8 percent. In Massachusetts, conversely, the Census Bureau reported the white turnout rate at 72.0 percent but the black turnout rate at just 46.5 percent.
As much as it pleases me to see statistical data introduced in the Supreme Court, the act of citing statistical factoids is not the same thing as drawing sound inferences from them. If I were the lawyer defending the Voting Rights Act, I would have responded with two queries to Chief Justice Roberts. First, are Mississippi and Massachusetts representative of a broader trend: do states covered by Section 5 in fact have higher rates of black turnout on a consistent basis? And second, what if anything does this demonstrate about the efficacy of the Voting Rights Act?
Turns out that the Current Population Survey has a very high margin of error.
One reason to be suspicious of the representativeness of Mississippi and Massachusetts is the high margin of error associated with these calculations, as noted by Nina Totenberg of NPR.
Like other polls, the Current Population Survey is subject to sampling error, a result of collecting data among a random subsample of the population rather than everyone in the state. In states like Massachusetts that have low African-American populations, the margin of error can be especially high: it was plus-or-minus 9.6 percentage points in estimating the black turnout rate in 2004, according to the Census Bureau. Even in Mississippi, which has a larger black population, the margin of error was 5.2 percentage points.
The other problem is that the Chief Justice was using 2004 figures when the 2010 numbers had a lower margin of error. So what, if any thing can we conclude.
In the chart below, I have aggregated the 2004 turnout data into two groups of states, based on whether or not they are covered by Section 5. (I ignore states like New York where some counties are subject to Section 5 but others are not.) In the states covered by Section 5, the black turnout rate was 59.2 percent in 2004, while it was 60.8 percent in the states that are not subject to it. The ratio of white-to-black was 1.09 in the states covered by Section 5, but 1.12 in the states that are not covered by it. These differences are not large enough to be meaningful in either a statistical or a practical sense.
So did Chief Justice Roberts misconstrue the data? If he meant to suggest that states covered by Section 5 consistently have better black turnout rates than those that aren’t covered by the statute, then his claim is especially dubious. However, the evidence does support the more modest claim that black turnout is no worse in states covered by Section 5. There don’t seem to be consistent differences in turnout rates based on whether states are covered by Section 5 or not.
The bigger potential flaw with Chief Justice Roberts’s argument is not with the statistics he cites but with the conclusion he draws from them.
And here what Silver thinks we should be asking.
…the fact that black turnout rates are now roughly as high in states covered by Section 5 might be taken as evidence that the Voting Rights Act has been effective. There were huge regional differences in black turnout rates in the early 1960s, before the Voting Rights Act was passed. (In the 1964 election, for example, nonwhite turnout was about 45 percent in the South, but close to 70 percent elsewhere in the country.) These differences have largely evaporated now.
How much of this is because of the Voting Rights Act, as opposed to other voter protections that have been adopted since that time, or other societal changes? And even if the Voting Rights Act has been important in facilitating the changes, how many of the gains might be lost if the Section 5 requirements were dropped now?
To put it nicely, the Chief Justice is using correct statistics to come to not only the incorrect conclusion, but also to ask the wrong questions. Silver concludes
These are difficult questions that the Supreme Court faces. They are questions of causality – and as any good lawyer knows, establishing a chain of causality is often the most difficult chore in a case.
Statistical analysis can inform the answers if applied thoughtfully. But statistics can obscure the truth when they become divorced from the historical, legal and logical context of a case.
We can only hope that some law clerk at the Supreme Court reads FiveThirtyEight and talks to enough Justices. Given all the shenanigans going on in Section 5 covered and not covered states on voting rules, now is not the time to over turn this modest brake insuring voting rights.
Official 2005 photo of Chief Justice John G. Roberts (Photo credit: Wikipedia)
FortLeft 