PISA: It's Poverty Not Stupid

by Mel Riddle,  National Association of Secondary School Principals

There are three kinds of lies; lies, damn lies, and statistics."--Mark Twain

"There are three kinds of lies; lies, damn lies, and statistics."--Mark Twain

The release of the 2009 PISA results this past week has created quite a stir and has provided ample fodder for public school bashers and doomsayers who further their own philosophical and profit-motivated agendas by painting all public schools as failing. For whatever reason, these so-called experts, many of whom have had little or no actual exposure to public schools, refuse to paint an accurate picture of the state of education.

Secretary of Education, Arne Duncan, should be providing the nation with a proper vision and focus for public education. He knows our challenges all too well. He confirmed that he gets it when he recently wrote me saying, "We must build a culture nationally where great educators ... choose to work with children and communities who need the most help." I believe his message is sincere and heartfelt and it is spot on. However, overstating a problem in order to increase the sense of urgency around school improvement is just as bad as understating the problem.

This week, Duncan had a golden opportunity to use the PISA results to provide focus for our education efforts and to point us in the right direction. Instead, he dug himself deeper into the pseudo-reformers' hole--more charter schools, more reliance on competition and free-market strategies, more testing, more use of test scores to evaluate teachers, more firing of principals and teachers, more closing of low-scoring schools--when he said, "the PISA scores released this past Tuesday were "a massive wake-up call," because the scores show American students holding relatively steady in the middle of the pack of the developed nations taking the international exam.

There is, however, someone who recognizes that the data is being misinterpreted. NEAToday published remarks from National Association of Secondary School Principals Executive Director, Dr. Gerald N. Tirozzi, that have taken "a closer look at how the U.S. reading scores on PISA compared with the rest of the world’s, overlaying it with the statistics on how many of the tested students are in the government’s free and reduced lunch program for students below the poverty line." Tirozzi pointed out, “Once again, we’re reminded that students in poverty require intensive supports to break past a condition that formal schooling alone cannot overcome.” Tirozzi demonstrates the correlation between socio-economic status and reading by presenting the PISA scores in terms of individual American schools and poverty.  While the overall PISA rankings ignore such differences in the tested schools, when groupings based on the rate of free and reduced lunch are created, a direct relationship is established.

Free and Reduced Meal Rate	PISA Score
Schools with < 10%	551
Schools with 10-24.9%	527
Schools with 25-49.9%	502
Schools with 49.9-74.9%	471
Schools with >75%	446
U.S. average	500
OECD average	493

With strong evidence that increased poverty results in lower PISA scores the next question to be asked is what are the poverty rates of the countries being tested? (Listed below are the countries that were tested by PISA along with available poverty rates. Some nations like Korea do not report poverty rates.)

Country	Poverty Rate	PISA Score
Denmark	2.4%	495
Finland	3.4%	536
Norway	3.6%	503
Belgium	6.7%	506
Switzerland	6.8%	501
Czech Republic	7.2%	478
France	7.3%	496
Netherlands	9.0%	508
Germany	10.9%	497
Australia	11.6%	515
Greece	12.4%	483
Hungary	13.1%	494
Austria	13.3%	471
Canada	13.6%	524
Japan	14.3%	520
Poland	14.5%	500
Portugal	15.6%	489
Ireland	15.7%	496
Italy	15.7%	486
United Kingdom	16.2%	494
New Zealand	16.3%	521
United States	21.7%	500

Leveling the playing field

A more accurate assessment of the performance of U.S. students would be obtained by comparing the scores of American schools with comparable poverty rates to those of other countries.

Schools in the United States with less than a 10% poverty rate had a PISA score of 551.  When compared to the ten countries with similar poverty numbers, that score ranked first. 

Country	Poverty Rate	PISA Score
United States	<10%	551
Finland	3.4%	536
Netherlands	9.0%	508
Belgium	6.7%	506
Norway	3.6%	503
Switzerland	6.8%	501
France	7.3%	496
Denmark	2.4%	495
Czech Republic	7.2%	478

In the next category (10-24.9%) the U.S. average of 527 placed first out of the ten comparable nations. 

Country	Poverty Rate	PISA Score
United States	10%-24.9%	527
Canada	13.6%	524
New Zealand	16.3%	521
Japan	14.3%	520
Australia	11.6%	515
Poland	14.5%	500
Germany	10.9%	497
Ireland	15.7%	496
Hungary	13.1%	494
United Kingdom	16.2%	494
Portugal	15.6%	489
Italy	15.7%	486
Greece	12.4%	483
Austria	13.3%	471

For the remaining U.S. schools, their poverty rates over 25% far exceed any other country tested.  However, when the U.S. average of 502 for poverty rates between 25-49.9% is compared with other countries it is still in the upper half of the scores.

Mathematically Speaking

The results of the latest PISA testing should raise serious concerns.  However, the overall ranking of 14th in reading is not the reason to be concerned. The problem is not as much with our educational system as it is with our high poverty rates. The real crisis is the level of poverty in too many of our schools and the relationship between poverty and student achievement. Our lowest achieving schools are the most under-resourced schools with the highest number of disadvantaged students. We cannot treat these schools in the same way that we would schools in more advantaged neighborhoods or we will continue to get the same results. The PISA results point out that the U.S. is not alone in facing the challenge of raising the performance of disadvantaged students.

U.S. % Poverty	Other Countries	PISA Score
U.S. (<10%)		551
	Korea	539
	Finland	536
U.S. (10-24.9%)		527
	Canada	524
	New Zealand	521
	Japan	520
	Australia	515
	Netherlands	508
	Belgium	506
	Norway	503
U.S. (25-49.9%)		502
	Estonia	501
	Switzerland	501
	Poland	500
	Iceland	500
U.S. (Average)		500
	Sweden	497
	Germany	497
	Ireland	496
	France	496
	Denmark	495
	United Kingdom	494
	Hungary	494
	Portugal	489
	Italy	486
	Slovenia	483
	Greece	483
	Spain	481
	Czech Republic	478
	Slovak Republic	477
	Israel	474
	Luxembourg	472
U.S. (50-74.9%)		471
	Austria	471
	Turkey	464
	Chile	449
U.S. (over 75%)		446
	Mexico	425

Additional observations from PISA results:

·      Shanghai, China topped the list with 556 but is not included in this analysis because Shanghai is a city not a country and because only 35% of Chinese students ever enter high school and because "when you spend all your time preparing for tests, and when students are selected based on their test-taking abilities, you get outstanding test scores."

·      Of all the nations participating in the PISA assessment, the U.S. has, by far, the largest number of students living in poverty--21.7%. The next closest nations in terms of poverty levels are the United Kingdom and New Zealand have poverty rates that are 75% of ours.

·      U.S. students in schools with 10% or less poverty are number one country in the world.

·      U.S. students in schools with 10-24.9% poverty are third behind Korea, and Finland.

·      U.S. students in schools with 25-50% poverty are tenth in the world.

·      U.S. students in schools with greater than 50% poverty are near the bottom.

·      There were other surprises. Germany with less than half our poverty, scored below the U.S. as did France with less than a third our poverty and Sweden with a low 3.6% poverty rate.

·      Having recently listened to Sir Michael Barber talk about the amazing progress of the reforms in the United Kingdom, I was absolutely shocked to see that the UK, with 25% less poverty, scored below the U.S. average.

The Real Meaning of PISA: It's Poverty Not Stupid

If the so-called experts would have honestly and responsibly reported the PISA results, we might now be on the road to responsible school improvement instead continuing down the road of "reform de jour."

President Bill Clinton is famous for his campaign slogan, "It's the economy stupid!" When it comes to student achievement and school improvement, it's poverty not stupid! Researchers report that perhaps the only true linear relationship in the social sciences is the relationship between poverty and student performance. While there is no relationship between poverty and ability, the relationship between poverty and achievement is almost foolproof. To deny that poverty is a factor to be overcome as opposed to an excuse is to deny the reality that all educators, human services workers, law enforcement officers, medical professionals and religious clergy know and have known for years.

PISA reports average scores. The problem is that the U.S. is not average. While the U.S. is the top country in global competitiveness, we also have the highest percentage of students living in poverty and, regretfully, poverty impacts test scores.

To Secretary Duncan, poverty is not an issue that educators must address. At least he won't admit it in public. Apparently, he wants to take away all the excuses from teachers and principals. When I met with Duncan, I asked him if he had read the book or seen the movie, Blind Side. He indicated that he had and that he had enjoyed it very much. I reminded him of the pride and sense of accomplishment felt by the teachers in the private school attended by Michael Oher. In their minds, they had performed a miracle. I pointed out that, in high-poverty schools, a Michael Oher is the average student. In schools like ours we have hundreds of students like Michael Oher who depend on our school for everything including food, clothing, and emotional support.

The Bottom Line

School improvement is not an event. It is an ongoing process that has no end. As a principal, parents and community members would repeatedly ask me, "When can we stop our comprehensive school-wide literacy initiative? I would answer, "We will stop emphasizing reading, writing, thinking and speaking when our parents repeatedly complain that their children are reading too fast with comprehension that is too high and when our students' writing skills are so superior that they are regularly winning Pulitzers and other literary awards." Smiles would erupt throughout the audience. They got it. They understood that literacy skills can always be improved and so can our schools.

There are three compelling reasons why we must improve our schools:

1. We have a moral and ethical obligation to provide every student with the best education, the kind of education that we would want for our own children.
2. In a knowledge economy, the country with the best-educated populace will have the highest standard of living.
3. Every dropout as well as every graduate who is not prepared for at least some post-secondary education and training is and will continue to be an economic and social burden on their local community and on this nation for their entire life.

The challenge of ensuring that each and every student is a life-long learner prepared to contribute in a global community is daunting enough. We don't need more hyperbole, particularly from those education insiders who should know better. For those of us who are deeply committed to improving the performance of every student, this rhetoric is counterproductive because it seriously erodes our ability to hire teachers, obtain resources, and gain the confidence and support of our communities.

We count on our leaders to provide focus and direction. Sadly, our education leaders don't trust us enough to tell us the truth. The problem is that we will never solve a problem that our leaders refuse to admit even exists. The comparison of PISA scores by poverty clearly identifies our strengths and challenges as a nation. Our schools with less than 50% poverty) are some of the best in the world. Our extremely high-poverty schools, with over 50% poverty, are among the poorest performing internationally.

Instead of labeling all schools as failing, we must find a way to raise the performance of our students in under-resourced schools. Instead of looking to low-poverty countries like Finland for direction, we should be looking to take what we already know about educating students in high-performing, high-poverty schools like our Breakthrough Schools and scaling up their successes across the nation. We continually look for gold in other countries when, all along, we are sitting on Acres of Diamonds.

Truthfully, you and I know all too well that Secretary Duncan, who led schools in Chicago, is aware of the relationship between poverty and student achievement, but he doesn't trust us enough to tell us the truth. He is afraid that we will use poverty as an excuse and that we will forget about our disadvantaged students. Ironically, by not acknowledging poverty as a challenge to be overcome, Duncan is forgetting about our disadvantaged students. Duncan needs to deliver the message that all our students deserve not only access to an education, but access to an excellent education. He needs to repeatedly remind us that, when it comes to school improvement, it's poverty not stupid.

Posted by Mel Riddile on December 15, 2010 12:13 PM