The Plural Of Anecdote Is Data

 by Esther Quintero on April 11, 2013

Last week, I attended a Center for American Progress (CAP) discussion, where UC Berkeley professor David Kirp spoke about his research on Union City’s school system, and offered some ideas from his new book, Improbable Scholars: The Rebirth of a Great American School System and a Strategy for America’s Schools.

Kirp’s work and Union City have received a lot of attention in the last month or so, and while most find the story heartening, a few commentators have had more skeptical reactions. True, this is the story of one district in onestate finding success through collaboration and hard work, but research from other disciplines – sociology, business, management, organizational studies – suggests that similar human dynamics can be observed in settings other than schools and school districts. I would like to situate Kirp’s work in this broader framework; that is, among a myriad of studies – case studies, if you will – pointing to the same fundamental phenomena.

Union City is a community with an unemployment rate 60 percent higher than the national average, where three-quarters of public school students live in homes where only Spanish is spoken. About 25 years ago, the school district was in so much trouble that state officials threatened a state takeover. Since then, Union City’s measured performance has improved considerably. In 2011, almost 90 percent of the district’s students graduated from high school, and 60 percent went on to college. The change is large enough to suggest some degree of “real” improvement, and it’s plausible to believe that better school quality had at least something to do with that. So, what was Union City’s school improvement strategy?

Kirp’s remarks (and some of the discussion that followed his presentation at CAP) highlighted the many things that the district appears to be doing well. Workforce stability and a “grow your own” approach to teacher training and recruitment seemed particularly central. As explained in Improbable Scholars:

You won’t find any Teach for America recruits on a mission to reinvent urban education – at Washington, as in every Union City School, almost all the teachers grew up within hailing distance of the community. (…) They attended local colleges like Jersey City State, Saint Peters, and Fairleigh Dickinson; Rutgers embodied the limits of their ambition. Soon after graduating they started teaching in Union City, and there they have stayed for their entire careers. (p. 17)

This notion – achieving good results with ordinary (committed) people – brought to mind the work of Charles O’Reilly and Jeffery Pfeffer, both professors at the Stanford Graduate School of Business.

About a decade ago, O’Reilly and Pfeffer wrote Hidden Value, a book that set out to answer the following question: What’s the most important factor for organizational success in today’s knowledge-based economy? Hint: it’s notattracting and retaining exceptional people.

While that strategy can seem compelling initially, the authors say, smart companies avoid doing what everybody else is doing: fighting an escalating war for talent. Instead, they are focusing on doing something “infinitely more useful and much more difficult to copy”: building organizations that make it possible for ordinary people to perform at their best and achieve extraordinary results.

According to Kirp, that’s exactly what Union City did: make it possible for “ordinary” teachers, who were drawn from the community and did not attend the nation’s top universities, to do their work well and achieve good results.

After all, O’Reilly and Pfeffer explain:

The unfortunate mathematical fact is that only 10 percent of the people are going to be in the top ten percent. So, companies have a choice. They can all chase to suck the same supposed talent. Or, they can (…) build an organization that makes it possible for regular folks to perform as if they were in the top 10 percent. (p.2)

Hidden Value offers many examples of organizations that have achieved extraordinary levels of success “with people who really aren’t that much different or smarter than those working in the competition.”

Throughout the book, the authors illustrate that the classic formulation of performance as a consequence of ability times motivation is not only “inadequate to explain individual performance,” but also “much too simplistic for organizations, where performance is also a consequence of the environment in which individuals work and how they work together.” (p. 9)

In fact, the authors argue:

When systems are designed to closely monitor and control people, those being controlled will soon begin to resent their lack of autonomy and the lack of trust. Thirty years of research in social psychology has documented how increased monitoring can undermine motivation and cause previously engaged people to reduce their effort. (p. 17)

Another example, also from the world of business: In 2010, researchers Chip Heath (also at the Stanford Graduate School of Business) and Dan Heath (Duke University Center for the Advancement of Social Entrepreneurship) published a fascinating book titled Switch: How to Change Things When Change Is Hard. This book is a gem, extremely accessible, well researched, and crammed full of tips and lessons for those interested in organizational dynamics.

Here, I want to focus on one particular story, the story of Brasilata, a $170 million Brazilian firm that produces steel cans. While can manufacturing may sound somewhat mundane, the company “defies the stereotype of a boring, stuck-in-its ways manufacturer” and has, in fact, “one of the best reputations for innovation of any company in Latin America.”

Brasilata’s founders were inspired by the philosophy of Japanese car manufacturers like Honda and Toyota, which empowered their frontline employees to take ownership of their work. For instance, at Toyota, any employee who spotted a defect could stop the assembly line. (p. 156)

So, “how does a can manufacturer become known as an innovator?” Heath and Heath ask. In 1987, Brasilata launched a Japanese-inspired employee-innovation program:

A new identity was the core of the program. Employees of Brasilata became known as “inventors,” and when new employees joined the firm, they were asked to sign an “innovation contract.” This wasn’t simply feel-good language. Top management challenged employees to be on the lookout for potential innovations – ideas for how to create better products, improve production processes, and squeeze costs out of the system. Procedures developed within the factory made it easy for inventors to submit their ideas. (p. 157)

In 2008, employees submitted 134,846 ideas, more than 145 ideas per worker. Many of the suggestions led to the development of new products, and helped to deal with special conditions such as the 2001 energy crisis, which forced the government of Brazil to ration energy, with businesses receiving a strict electricity quota.

The inventors went to work dreaming up power-saving ideas – hundreds of them. Within a few weeks, Brasilata’s energy consumption was reduced by 35 percent, falling below the company’s quota and allowing the company to resell its extra energy. (p. 158)

Another unexpected idea was jointly suggested by two employees: “Eliminate our jobs; they’re not necessary anymore.” The idea was accepted, and the company, which has a no-dismissal policy, found new, more productive uses for the employees.

Heath and Heath argue that to produce change, especially when such change is hard, one of the things you need to do is “grow your people” and “cultivate a sense of identity.” This is also exactly what Union City did. In a way, Union City’s teachers went to work “dreaming up” ideas about how to help their kids:

These kids are our kids. (…) We know them – we know where they come from and what they are going through. We’re here for them. (p. 17)

Some might dismiss these and similar accounts as anecdotal, but, as Raymond Wolfinger (professor emeritus of American Politics at UC Berkeley) once said, “the plural of anecdote is data.”

While any single story or observation (qualitative or quantitative) is insufficient for scientific generalization (or sound policy making), gathering multiple empirical reports (including qualitative reports or stories) and aggregating them properly can produce testable hypotheses and tentative theories about how the world works. So, there is a lot of room between rushing to adopt a set of policies based on the experiences of one district and dismissing these experiences as meaningless – especially when similar patterns from widely different contexts and fields can be observed in support of the same basic premise.

That is, there is every reason to believe that, given the right organizational conditions, a majority of workers (including educators) can and will rise to the occasion.