Studies Find Charters Vary in Quality, Creativity

By Jackie Zubrzycki, in Education Week

There is one point on which critics and proponents can all agree: There are plenty of good charter schools out there, and plenty of bad ones.Twenty-one years after Minnesota passed the nation's first charter school law, researchers still disagree on whether such independent public schools are any more effective than regular public schools.

Much less is known, however, about how innovative charter schools turned out to be.

Now serving more than 2 million students in 41 states and the District of Columbia, charter schools were conceived to be incubators of innovation. The idea was that this new breed of public school could foster fresh thinking on schooling by freeing educators from many of the regulations that govern—and, some would say, hamstring—regular schools. The quid pro quo was that charters would have to yield student-achievement results as good as or better than those of regular public schools.

Large-scale studies released since 2009 from Mathematica Policy Research, the National Charter School Research Project, and the Center for Research on Education Outcomes, or CREDO, at Stanford University indicate that charters' performance varies widely by operator, by geography, and by school philosophy."There certainly are innovative charter schools," said Kevin G. Welner, the director of the National Education Policy Center at the University of Colorado at Boulder, "but there are others that are not. What takes place in the classroom at a charter school can look the same as any other classroom—and usually does."

"The variation within the sector is much larger than whatever the difference is between it and other sectors," said Brian P. Gill, a researcher with Mathematica, a research group in Princeton, N.J. "I don't think we really know yet what the average effect is."

In 2009, CREDO released an overview of findings from its research on charters in 16 states. The center found that 17 percent of charters were performing better than local public schools, while 37 percent were performing worse. The CREDO researchers also found that students in poverty and English-language learners at charter schools had higher test scores compared with statistically matched students in local public schools.

State-level analysis like that in the CREDO study is important, said Robin Lake, the director of the Center on Reinventing Public Education, or CRPE, at the University of Washington, Bothell, because "each state has taken [charter schools] on very differently."

CREDO's most recent report, released last spring, focused on Pennsylvania, and found that performance there varied widely, with urban schools showing better results. Cyber charter schools—whose students account for 30 percent of the state's charter enrollment—fared particularly poorly.

Networks in Focus

Some researchers have also begun to focus more recently on studying networks of charter schools, in part to discover whether consistent practices and an overall management structure might yield different results.

But Mathematica's January 2012 study of 130 charter management organizations, or CMOs, found that "achievement impacts for individual CMOs are more often positive than negative, but vary substantially in both directions," according to Mr. Gill, one of the authors. The CMO study found that charter schools in Florida and Chicago were improving their students' educational attainment, and that schools that showed positive effects in reading and math also tended to do well in science and social studies. Mr. Gill said examining those factors was an important step toward evaluating schools' performance beyond scores on reading and math tests, which are what most studies up to now had focused on.

A meta-analysis of 40 studies on charter schools published last fall as part of the CRPE's National Charter School Research Project found that charters had a small positive impact on scores in elementary school math and reading and in middle school math, but none on high school or middle school reading scores. Urban charter schools saw greater gains than non urban charters.

A goal for researchers now is to determine what the most successful charter schools and networks might be doing right so that those practices can be disseminated more widely. The Mathematica study of CMOs, for instance, found that schools with teacher coaching and clear behavior policies had higher test scores.But some of the same studies have also shown that when charter schools were successful, they tended to be very successful. For instance, a 2010 report on Mathematica's multiyear study of the best-known "no excuses" school, the Knowledge Is Power Program, found that students who attended KIPP schools for three years showed growth equivalent to 1.2 years of extra instruction in math and .9 years in reading. Researcher Brian Gill, one of the report's authors, said that "in a dozen years of studying education policies and intervention, I've never been involved with a study that showed more consistently positive achievement effects than what we have seen with KIPP."

Several groups of researchers are calling attention to the "no excuses" practices used by successful charter schools like those in the KIPP network as one such ingredient of success, but there is still a desire for more research on student achievement, school funding, and the replicability of successful charter school practices.

A 2011 report from the Massachusetts Institute of Technology on Massachusetts charter schools from Joshua D. Angrist, Parag A. Pathak, and Christopher R. Walters found that urban charters outperformed nonurban charters. Like the Mathematica researchers, the MIT scholars attributed some of the urban charters' success to a "no excuses" model—a "strict disciplinary environment, an emphasis on student behavior and comportment, extended time in school, and an intensive focus on traditional reading and math skills."

However, Christopher A. Lubienski, an education professor at the University of Illinois at Urbana-Champaign, said some of the difference between nonurban and urban schools has to do with the schools' missions. "[Charter] schools in rural areas are often set up as an alternative for at-risk students," Mr. Lubienski said, whereas many urban charter schools—like those using no excuses models—are founded to serve as alternatives to failing urban districts and attract a wide range of students from public school districts.

A 2011 study of New York City charter schools by Harvard University researchers Roland G. Fryer and William Dobbie also connected elements of the no-excuses model to improved academic results.

Some research suggests, though, that the success of a no-excuses exemplar like KIPP could also stem in part from those schools' attrition practices.

As charters are increasingly well-studied and the variation in quality becomes clearer, Mr. Miron said, "there's been an interesting change in the discourse. We don't hear as much about innovation or performance. Instead, we hear that charters promote parent choice.""On average, kids exit KIPP schools at rates similar to other middle schools' nearby in their districts. A difference is they don't tend to be replaced," said Mr. Gill. He said part of KIPP's positive effect may be due to the peer environment rather than innovative practices.

The University of Illinois' Mr. Lubienski concurred.

"Whether charter schools are a better option is now beside the point," he said. "It's the option in and of itself."

Coverage of the education industry and K-12 innovation is supported in part by a grant from the Bill & Melinda Gates Foundation.