Home > Get Involved > About Us > Mission & Goals > Networking in Action > Networking Archive	Print View

Doug Earl, past president of NEAT - Saint Paul Network of Education Action Team

May 5, 2004

Dr. Bracey began his analysis of school test scores in 1991 with his publication of “Why Can’t They Be Like We Were”.  This publication was in response to the publication of a number of reports, going back to the 1983 “A Nation At Risk”, which portrayed the Public Education System in the United States as dismally failing and getting worse. Through rigorous analysis of the data presented in these critical reports, Dr. Bracey was able to show that apparent trends in lowered student achievement could largely be explained as either broad claims that had no real data to support them, or incorrect application of statistical methods.  In the cases that did show genuine diminishment in student achievement, Dr. Bracey was able to show corresponding demographic and societal changes that were the actual root causes of lowered achievement.  Dr. Bracey followed up this first publication a year later with a publication that was given the title of “The Second Bracey Report on the Condition of Public Education”.  The series has become an annual publication with the Thirteenth Bracey Report being published last fall.

As an example of what turns out to be numerous sweeping statements that have no factual data to support them, Dr. Bracey followed up on a published claim of “High School Seniors Now Achieving at Seventh Grade Level”.  His investigation traced the source of the quote back to an organization associated with providing solutions to the apparent decrease in school quality.  There was no study, or data that could be produced to support the statement.

Misinterpretation of accurate data can also be misleading.  Dr. Bracey presented a chart that correlated increased funding with student achievement.  On first view, one observes that student achievement stays flat despite increased spending.  On further analysis, it is seen that the number of students in the study had increased over the period of the analysis.  When the investment per capita in was plotted against student achievement, it became readily apparent that increased funding had actually improved achievement.

Beyond the unsupported inflammatory statements, and the misinterpreted data, there are, indeed, instances where aggregate test scores have diminished.  Dr. Bracey presented such a case that, on the surface, clearly showed such a trend.  On further investigation, what was shown was that the demographics of the test population had also changed over the period of the investigation.  The influx of greater numbers of students of poverty, into the test population, when averaged in with the reduced proportion of higher-achieving students had, indeed, resulted in a lower aggregate score.  The point that is often missed, is that when the trends of all of the test groups are looked at individually, it is often the case that the achievement of all groups is increasing, even though the aggregate score is not.  This phenomenon is known as “Simpson’s Paradox” and often explains misleading data that is gleaned from populations composed of subgroups of varying indices that change in number with time.

Dr. Bracey convincingly presented the case that, despite much information to the contrary, public schools in the United States, are generally producing higher performing students than they did in the past.  He is, however, concerned that this positive trend will not continue, as schools respond to reduced funding and evermore emphasis on testing by increasing class size, eliminating curricula, and focusing on programs meant specifically to increase test scores.

D. E. Earl - 05/05/04