Sunday, July 3, 2011

Is it fair to compare test scores between public and private schools?



Here's an interesting statistic from this week's release of Michigan Merit Exam data: Thirty-two percent of Michigan's private-school juniors who took the ACT in March tested as fully college-ready compared to 17 percent of public-school juniors.
Clear proof that private schools provide a better education?
Or a misleading barometer of school quality?
There's plenty of private-school advocates who would argue the former, and trumpet state assessment test scores as a sign of the academic superiority of private schools.
The counter argument can be explained with this analogy: It's like looking at the death rates at South Haven Community Hospital versus those at Bronson Methodist Hospital, a comparison made utterly meaningless because one is a facility that deals with routine medical matters and the other is a regional treatment center for critically ill and injured patients. It tells us nothing about whether either South Haven or Bronson are good, mediocre or poor at what they do.
So it is with public and private schools: They are institutions with very different missions, serving very different clientele.
The strength of public schools is their willingness to educate everybody who walks in the door -- the autistic kid, sixth-grader who can't read, the child who just moved here from Mexico, the teen ordered back to school by his probation officer. The challenge for public schools is being all things to all people, which can lead to competing agendas and a struggle to establish priorities. 
The strength of private schools is the ability to create an educational community of like-minded people around a common vision. By virtue of charging tuition, private schools attract families who buy into the school's vision and are highly committed to their children's education, an ideal environment for nurturing academics. But, by no means, are private schools a representative slice of the American K-12 population.
No surprise, there are stark differences between the sociodemographics of public and private schools.
Among the data released this week by the Michigan Department of Education was demographic information on Michigan's Class of 2012. Of the 107,995 public school juniors who took the MME, 36 percent qualify for the subsidized lunch program; 24 percent are minorities, predominately African-American and Hispanic; 9 percent are classified as disabled, 2.5 percent are not proficient in English and 1 percent are classified as "homeless."
The state did not do a collective profile on the 4,597 private-school juniors who took the MME. But I added up the data for the 16 largest schools, which together enroll more than half of the private-school test-takers.
Sixteen percent of those students are minorities, with a heavy representation of Asians. Only nine students among the 2,388 test-takers were classified as "economically disadvantaged" in the reports and only seven were classified as disabled. All of the private-school test-takers were proficient in English, according to the reports, and none were homeless. (Incidentally, it's possible that private schools serve more poor and/or disabled kids than the numbers here reflect, but unlike the public schools, participation in the MME is voluntary and there may be students who didn't take the tests.)
Decades of research indicate that, far more than school quality, raw test scores reflect student sociodemographics -- in the same way that a medical patient's prognosis is more dependent on his or her condition than his or her doctor.

If a school served a high-poverty population and has a number of student who can't speak English, those test scores are inevitably lower than scores for an affluent school that serves middle-class white kids. (While there are a few high-profile exceptions to this in elementary, there's really no notable exceptions among American high schools.) It's no coincidence that the top-performing schools in this region, and elsewhere throughout the country, serve the most affluent communities.

And even affluent suburban schools are at a disadvantage compared to private schools in terms of state assessment tests. As indicated by the statistics cited here, private schools test the cream: Affluent and middle-class white and Asian able-bodied students with highly motivated parents. As testing pools go, you can't get better than that.

Meanwhile, even the most affluent public school has some at-risk students. 

Let's look at Kalamazoo County as an example. Almost 200 private school students took the MME. All but nine were white and all but two were middle-class. There were no disabled students or students who were not proficient in English.

By comparison, Mattawan High School-- the area's top performer among public schools on the MME -- is the most affluent school in this area and has the smallest at-risk population. Still, of their 279 test-takers, 33 qualified for the subsidized lunch program, 18 were non-white, 16 were disabled and a few  met the standard for "homeless." 

And let's look at Kalamazoo's Loy Norrix High School: Of 240 test-takers, 139 were "economically disadvantaged", 131 were non-white, 27 had limited English proficiency, and 29 were disabled. 

Those differences are reflected in the test results: 52 percent of Kalamazoo Christian High School students tested as "fully college ready" in English, math, science and social studies compared to 36 percent of Mattawan students and 15 percent of Norrix students. 

And here's the thing: If you were to switch the K-Christian and Norrix staffs and curriculum tomorrow, it's very unlikely those numbers would change.

Don't believe me? Consider this. 

Norrix's overall passage rate on the MME reading test was 57 percent. But among non-disabled white students, it was 85 percent -- and that's including white students from low-income families. By comparison, the passage rate for Hackett  and Heritage Christian was 86 percent and for K-Christian, it was 89 percent. In a true sociodemographic matchup of middle-class white kids to middle-class white kids -- and we're not even factoring in the motivated-parent piece, which is huge -- Norrix might well have topped the three private schools, at least in that category.  

The point is, private and public school results overlap much more than the raw results indicate and raw test scores are a terribly misleading measure of school quality, because the data is so muddied by sociodemographics. If private or public schools want a true picture of how their school is performing, they need to dig into the data and either compare their school with one that has a similar demographic profile or compare similar types of students.

Julie Mack is a reporter for the Kalamazoo Gazette. Contact her at

No comments:

Post a Comment