Understanding Education Indicators: A Practical Primer for Research and Policy
reviewed by Michael W. Apple — February 01, 2013
Among the most powerful assumptions that now guide a good deal of educational reform is the claim that “evidence-based practices” will almost inexorably lead to lasting improvements in educational policy and practice. It should not be necessary to say—but it is even more necessary to say now—that the questions surrounding what actually counts as evidence, how it should be collected, who should collect it, what values are embedded in the questions that are asked, and what purposes the evidence should serve are crucial issues. They need to be openly and continually debated. Yet in our understandable rush to “do something,” to fix what are usually seen as simply technical problems, these questions are often seen as less significant. Or when they are considered, they are dealt with in largely rhetorical ways (see, e.g., Smith, et al., 2004). Issues of efficiency and cost-savings are all too often substituted for educational, ethical, and political substance. Performance pay for teachers, value-added measures, more testing more often, and the list could go on for quite a long time, are accepted relatively uncritically or are mandated with little regard for the questions that these procedures themselves raise or the hidden costs of instituting them. The fact that such policies are based both on some quite questionable empirical evidence and on a number of even more problematic views of what counts as an education worthy of its name seems to be beside the point when decisions are made (see Ravitch, 2010).
The phrase that has been used to describe the impulses and procedures associated with a good deal of these policies and practices is audit cultures (see Leys, 2003). As I show in Educating the “Right” Way (Apple, 2006), such cultures are grounded in both political and epistemological movements that are associated with neoliberal agendas of competition, markets and performance (see also Sandler and Apple, 2011). While couched in neutral language, they carry with them an entire set of ideological assumptions that are decidedly not neutral. Just as importantly, they are much less effective than their proponents are likely to admit publicly.
Let us not be naïve in assuming that, by themselves, the well-deserved criticisms of these tendencies will somehow cause governments and school districts throughout the nation to radically question their commitment to audit cultures. It will take concerted and well-organized intellectual, political, and practical efforts over many years to even slow down the pace of uncritical acceptance of this agenda. Given the fact that so many educators will be confronted with the demand for (limited kinds of) evidence, with increasingly high stakes implications for themselves and for their students and communities, books that clearly and concisely lay out how evidence and indicators are to be interpreted and used or misused are even more important now than ever.
Mike Planty and Devon Carlson’s volume Understanding Education Indicators: A Primer for Research and Policy lives up to the promise of its subtitle. It is exactly what it claims to be. A good deal of what it covers is what I would expect to be dealt with in a well-designed introductory course that combines research in educational policy with the various uses of social statistics and the questions that underpin such uses. While the book could have been strengthened with a more overt focus on the political and ideological content and implications of research and data collection and use, it is very good at what it does.
I say this with some hesitation. I wish that our educational policies and practices were not so driven by very limited understandings of what counts as evidence. And I certainly wish that researchers and policy makers/implementers had a more nuanced grasp of the complex political interests that lie behind the questions that are usually asked about “educational effectiveness.” On the other hand, we do live in a reality in which audit cultures reign supreme, in which dominant forms of educational evaluation are employed rather uncritically, and in which very limited understandings of education are institutionalized in mandates and policies. In situations such as this, it is much better that educators have the technical understandings of what can and cannot be claimed from research and of what kinds of interpretive processes are legitimate in the world of social and educational indicators. Technical understandings may be limited if that is all someone has. But they are still essential in times such as these. Otherwise, “experts for hire” decide both “what works” and what is good or bad in educational policy and practice. That would not be a good outcome to say the least.
Because of this, even when it doesn’t go far enough, I do indeed recommend Planty and Carlson’s book. To give an example of what this recommendation means to me personally, I am about to lend my copy of Understanding Education Indicators to my son, an administrator in a troubled school district. His responsibility is the creation, gathering, and interpretation of evidence about effectiveness. He is deeply committed to social justice inside and outside of education and has many questions about what is taken for granted as “evidence” in educational research, policy, and practice. The pressures on him and on the district in which he works are intense. The fact that I want him to read this book documents my sense that there is some very worthwhile material in it. Read as a set of clear cautions, and as a set of reminders about some of what needs to thought about more rigorously before we make decisions on what we are told are “facts,” it has considerable value.
And yet, I need to remind myself and the reader that all too much of this exists in a world in which what now counts as “evidence” and how it is collected and used goes on in a way that seems more than a little disconnected from the real lives of students and teachers, and even more so from the realities of poverty, the destruction of jobs and communities, resurgent racism, disgraceful rates of incarceration, and a future that looks increasingly bleak. We ask the school to fix all of this and act as if the accumulation of more evidence of certain kinds and more mandates on the social uses of such evidence will solve these problems. As I show elsewhere, this will simply not do (Apple, 2013). When the search for and use of (a much wider range of) “indicators” is organically connected to a larger set of movements to alter the relations among schools, communities, local and regional political economies, and differential power relations, then they can contribute to substantive and lasting progress (Anyon, 2005; Anyon, in press). Without such a thick egalitarian vision, without a clear set of commitments to critical democracy, what seem like gains can actually be losses. With this set of commitments and a long term view of what is required to actually make a lasting difference, successes are possible. The experience of Porto Alegre in Brazil—with its long lasting democratization of educational and larger socio/economic decisions and results--provides an important example that substantiates these claims (Gandin and Apple, 2012; Apple, 2013; see also Wright, 2010).
Thus, while I mean it when I say that books such as Understanding Education Indicators are indeed quite helpful, in saying this I again do not want to be misunderstood. Within their own sphere, these kinds of books provide us with important ways of dealing with the technical and interpretive problems of employing quantitative data in thoughtful ways. And in a field where medical-model research is increasingly seen as the “gold standard,” this is more than a little important. Yet these tools must not be focused upon so heavily that they keep us from asking the equally significant questions with which I began this review.
Of course, if used wisely to help us ask and answer these larger questions, they should be welcomed. But if they remain within the boundaries of neoliberal agendas and audit cultures, they are worrisome. They do not and must not take the place of the building of alternatives and the demand for a more critically democratic education that deals with the realities of inequality in this society. Some of this is happening already. Movements throughout the United States and elsewhere are growing that are committed to a thicker version of educational justice and to a school that respects its teachers, creates a curriculum that responds to the immense diversity of this nation, and treats students and communities as co-responsible subjects of self-formation. Indeed there is a very long history of this, especially in minoritized communities and in women’s and labor movements (see, e.g., Apple, 2013).
This is not only in the past. More currently, teachers and administrators are also taking more activist roles in raising substantive questions about what is happening in education today and about the production and often punitive use of the very limited range of evidence that is imposed through audit cultures. In Seattle for example, in reaction to what they believe is an over-reliance on standardized testing in their schools, the entire faculty at one secondary school has boycotted the standardized tests (Queally, 2013). The pages of journals such as Rethinking Schools are filled with the stories of successful mobilizations, responsive curricula, the creation and use of more diverse forms of evidence, and similar kinds of things. A number of recent books document these movements in powerful ways as well (see, e.g., Lipman, 2011; Watson, 2012).
In the end, where does this leave us? My suggestion is to read Planty and Carlson’s nicely written volume—but to then do two things. First, use it to be more skeptical of the collection and uses of the social and educational data that now are taken as “facts” in education today. Second, and just as importantly, take up the responsibility to ensure that you are just as knowledgeable of the critical traditions and actions that counter the realities and dangers of audit cultures and the severely truncated understandings of the issues of “evidence” that are entailed in such misunderstandings of the complex realities of education associated with them. If your reading of this book helps in these two tasks, then it will be even more worthwhile.
References
Anyon, J. (2005). Radical possibilities: Public policy, urban education, and a new social movement. New York, NY: Routledge.
Anyon, J. (in press). Radical possibilities: Public policy, urban education, and a new social movement, 2nd edition. New York, NY: Routledge.
Apple, M. W. (2006). Educating the “right” way: Markets, standards, God, and inequality, 2nd edition. New York, NY: Routledge.
Apple, M. W. (2013). Can education change society? New York, NY: Routledge.
Gandin, L. A. and Apple, M. W. (2012). Can critical democracy last? Porto Alegre and the struggle over “thick” democracy in education. Journal of Education Policy 27:621-639.
Leys, C. (2003). Market-driven politics: Neoliberal democracy and the public interest. New York, NY: Verso.
Lipman, P. (2011). The new political economy of urban education: Neoliberalism, race, and the right to the city. New York, NY: Routledge.
Queally, J. (2013). Boycott of standardized tests spread as Seattle teachers revolt. Common Dreams, January 14. http://www.commondreams.org/headline/2013/01/12-2, downloaded at 11:20am.
Ravitch, D. (2010). The death and life of the great American school system. New York, NY: Basic Books.
Sandler, J. and Apple, M. W. (2011). A culture of evidence, a politics of objectivity. In Z. Leonardo (Ed.) Handbook of cultural politics and education (pp. 325-340). New York, NY: Routledge.
Smith, M. L., with Miller-Kahn, L., Heinecke, W, and Jarvis, P. (2004). Political spectacle and the fate of American schools. New York, NY: Routledge.
Watson, V. (2012). Learning to liberate: Community-based solutions to the crisis in urban education. New York, NY: Routledge.
Wright, E. O. (2010). Envisioning real utopias. New York, NY: Verso.
| Cite This Article as: Teachers College Record, Date Published: February 01, 2013 http://www.tcrecord.org ID Number: 17012, Date Accessed: 2/5/2013 6:35:30 PM |
No comments:
Post a Comment