In Which the Argumentum Gets Even More Verbosium

In March, I wrote a blog post about a legal decision rejecting a proposal for integrating Minnesota schools that would have decimated high-performing, culturally affirming schools while leaving majority white schools majority white. In the post, I said that University of Minnesota Professor Myron Orfield, who directs the Institute on Metropolitan Opportunity, had engaged in the fallacy of argumentum verbosium–otherwise known as proof by intimidation–by slinging around impenetrable jargon in a way that glossed over fundamental weaknesses in a report he asserts proves public charter schools underperform their mainline district brethren.

The judges who issued the 93-page opinion were not so arch as I, but also found fault with Orfield’s characterizations.

In the wake of my post, former local education advocate-turned social science grad student Dan Wick submitted a guest post, “Argumentum Verbosium Pt II: On ‘All Things Being Equal,'” dissecting, in accessible English, the report Orfield wrote: “What on earth is regression analysis and how can I use it to state my policy conclusions with infallible certainty?”  

Yesterday Orfield submitted a comment to Wick’s post. Because it is quite long, instead of posting it at the end of Wick’s guest post, I am putting it up as a second guest post. Because Orfield takes some pot-shots at Wick, I’ve offered Wick the chance to reply. And because we just might be creating a central repository of the arguments on this issue, I will also post separately a-point-by-point rebuttal to Orfield’s arguments in the Minnesota case authored by attorney Cindy Lavorato, who argued successfully against the rule. 

Finally, verbosium though it is and tempting though it is, I haven’t touched Orfield’s prose, which follows: 

Beth Hawkin’s blog (“Argumentum Verbosium”) recently included a rant from charter advocate Dan Wick criticizing our work on charter schools in the Twin Cities. Carrying on the practice of the (now defunct) charter advocacy organization where he worked prior to returning to school, Wick’s critique includes innumerable inaccuracies regarding our analysis and the statistical tools that we use.

Our work relies on regression analysis to compare the performance of students in charters to those in traditional neighborhood schools (“traditionals”). Regression analysis is the most commonly used statistically tool for evaluating relationships involving more than two factors. Wick suggests that our analysis which controls for 14 school characteristics cannot be used to support our conclusion – that charter schools provide a poor alternative to the region’s students because students in charter schools under-perform those in traditionals with similar characteristics.

Describing regression analysis as “fancy math,” Wick argues that “in order to make bold claims like Mr. Orfield’s, we need to be 100% sure that we have identified literally ALL of the fundamental differences between the schools that might influence academic performance.” This statement shows a massive misconception about the nature of social scientific inquiry. No science, including the physical sciences, requires 100% certainty. To argue that social scientists who must deal with the complexities of human behavior should meet this hurdle before making policy recommendations is ludicrous. If 100% certainty is the requirement, then social scientists would literally have nothing to provide to any policy discussion. Is this really what Wick is after?

Our analysis controls for the school characteristics most commonly cited by charter advocates and critics alike as the primary determinants of school-wide averages of student performance. Wick argues that they are inadequate because they don’t fully capture differences across different parts of the region. (Cities and suburbs is how he simplistically characterizes the relevant parts of region.) He says that “{w}e would expect urban schools to deal with more psychological and emotional trauma in their student body among other problems. This could lower their test scores when compared to suburban schools.”

Wick characterizes this as an example of “fixed effects,” arguing that “when analyzing student performance across the metro it is important to account for shared experiences between groups of schools that might influence their results systematically. For example, schools across the metro receive different levels of funding. Because suburban schools use property taxes to supplement their funding it gives them a large resource advantage. This advantage is not reflected in the model and results in biased results.”

Needless to say, we are aware of fixed effects. In fact, we tested our model for this problem using the methods most commonly employed to deal with it. First, we included a variable in the analysis to test whether there were city-suburb differences in testing results after controlling for our 14 variables. This variable was not a significant explanator of performance and including it had negligible effects on the measured (negative) charter effect. Second, we included a variable that allowed school poverty to have a different effect in the suburbs than in the central cities to control for the possibility that urban poverty is associated with problems not found to the same degree in the suburbs. The results were again statistically insignificant with no substantive effect on the measured (negative) impact of charter schools.

The implication of these tests is that our list of control variables does in fact capture the most important differences between urban and suburban schools.

The second part of Wick’s statement – that suburban schools have resource advantages over urban schools – is also incorrect. Anyone with even minimal knowledge of school funding in Minnesota knows that state aid more than makes up for any disadvantages the two central cities face with regard to taxing capacities. The Minneapolis and St. Paul school districts both spend significantly more per pupil than any suburban district in the region. This applies to urban charters as well since they receive the equivalent state funding for students who come from the two central cities (the overwhelming majority of their students). Regardless, the empirical research on this issue shows that the relationship between school spending and student performance is very tenuous, if it exists at all. School composition – measured by the kinds of variables that we use in our work – is virtually always found to be much more important.

Finally, Wick states that our model “relies on the assumption that parents have a feasible choice between all of the schools in the metro area through programs like the Choice is Yours.” The model does no such thing. The regression analysis simply compares the performance of students in charter schools to students in traditionals after controlling for a set of school characteristics most likely to affect average school pass rates.

The policy changes we support would include enhancing access to suburban school options for central city students. However, we do not assume that the access now available is adequate to the task. We support regional solutions and recognize that effective regional solutions would require significantly more effort by the state to implement.

Having said this, it is also worth noting that Wick almost certainly under-estimates the degree of access to suburban schools now available to urban students, thereby over-stating the effort needed to improve the current situation. In 2010, 4,732 students open enrolled out of Minneapolis to suburban districts (including Choice is Yours participants). Of these, 67% were non-white and 66% were free and reduced price lunch (FRED) eligible. In comparison, there were 11,153 students in charter schools located in Minneapolis in 2013 and their demographic characteristics were similar to open enrollee and Choice is Yours participants – 71% were FRED eligible and 74% were non-white. The number and composition of these two groups suggests that increasing access to higher performing suburban schools could enhance choices available to urban students without the disadvantages now associated with the charter system – lower student performance and greater segregation on average than the traditional system.

Wick characterizes (and implicitly ridicules) our support of metro solutions and metro integration as “ideological.” We reject this characterization. Our support for integration and metro approaches is based on rigorous analyses by many social scientists which show that students benefit in a number of important ways from integrated educational environments. We base our policy recommendations on hard data and analysis. On the other hand, Wick’s rant includes exactly zero references to any data or rigorous analyses by other social scientists, implying that he in fact is the ideologue.


One thought on “In Which the Argumentum Gets Even More Verbosium

  1. Bill Jones

    Whenever I read something written by Myron I am reminded of the old adage “If you can’t dazzle them with brilliance, baffle them with B… S….” I have no doubt that he is a smart fellow and passionate in his beliefs however his “hardening of the categories” seem to prevent him from looking at any factors which don’t fit into his narrow preconceived set of beliefs.


More Comments