2016 July 9^th

George Will’s column yesterday was titled, “Sobering evidence of social science statistics.” Will sites the “Coleman Report,” a huge study of over 3 thousand schools and 600 thousand students. Will claims that this report was released by the Johnson administration on the July 4^th weekend in 1964 “hoping to bury it.”  Then Will writes, “From 1938 when the electorate rebuked Franklin Roosevelt for his plan to pack the Supreme Court… Republicans and conservative Democrats prevent(ed) a liberal legislating majority.” Well, Will is half-right; that election ended Roosevelt’s sheer dominance of the nation’s politics. But the Supreme Court “packing” had much less to do with that vote than the increase in the unemployment rate to 20 percent. Except for the Republican politicians, most voters were surely more concerned with the retreat in in the economic recovery than with the Supreme Court. Will, however cannot stand to miss an opportunity to belittle Roosevelt.

The “Coleman Report,” was an investigation mandated by the 1964 Civil Rights Act, (Can Will imagine any such investigation being authorized by the current Republican dominated Congress?) At the time of the report, the received wisdom was that the more money a school had to spend, the better the student’s performance.

Will quotes the report, “One implication stands out above all: That schools bring little to bear on a child’s achievement that is independent of his background and general social context; and that this very lack of an independent effect means that the inequality…carries along to adult life.” Then Will says, “...their baton of brave and useful sociology has passed to Charles Murray…” Murray and Herrnstein were the authors of “The Bell Curve.” This is another effort to diminish the effects of environment and its methodology has been heavily criticized.

So what do critics have to say about Will’s primary pitch, the Coleman Report?
Coleman found poverty and minority status to be more predictive of student achievement than just differences in school funding, a finding frequently distorted to suggest that “research shows school funding doesn’t matter in achievement.” Coleman never said that. Even what he did say is heavily the result of the kind of data analysis he did.

In simplest terms, the statistical procedure of the Coleman Report relies on a problematic stepwise analysis of variance approach, which makes strong assumptions about which factors are fundamental causes of achievement and which are of secondary significance. Coleman assumed that family influences come first, and that school factors are to be introduced into the analysis only after all effects that can be attributed to the family are identified. Accordingly, the first step of the statistical analysis assesses how much of the achievement variation across schools could be attributed to variations in family background factors. Only after these background factors are fully accounted for is the second step taken—a look at the characteristics of the schools that make the biggest difference in determining the variation in student achievement.

This approach privileges family background over any indicators of school resources or peer group relationships, as it implicitly attributes all shared variation to those variables included in the first step of the stepwise modeling. For example, if parental education and teacher experience are both strongly related to achievement, and children from better-educated families attend schools with more-experienced teachers, then it will appear as if teacher experience has little effect while the effect of parental education is magnified. The first step, looking at just the relationship between achievement and parental education, actually incorporates both the direct effect of parental education on achievement and the indirect effect of the more-experienced teachers in their schools. When the analysis gets to the point of adding teacher experience to the explanation of achievement, the only marginal impact will come from the portion of variation in experience that is totally unrelated to family background.

So Mr. Will, is this criticism from some left leaning outfit determined to trash any study that right wingers can present to defend spending as little as possible on education? No, not really; here is the author of this critique and his affiliation.

Eric A. Hanushek is senior fellow at the Hoover Institution at Stanford University and research associate at the National Bureau of Economic Research.