Highlights
• Finding: Students whose teachers held a master’s degree performed statistically significantly better in both reading and language arts than students whose teachers did not hold a master’s degree.


• Finding: Students whose teachers held an M.S. in Education (M.S.Ed.) with a specialization in Elementary Reading and Literacy (Grades PreK–6) from Walden University performed statistically significantly better in language arts than students whose teachers held a master’s degree from other institutions.


• Finding: Students whose teachers held an M.S. in Education (M.S.Ed.) with a specialization in Elementary Reading and Literacy (Grades PreK–6) from Walden University performed the same statistically in reading as students whose teachers held a master’s degree from other institutions.


• Study data: 4,106 teachers of record for reading; 205,226 student observations from grades 2 through 5; 2004 through 2010


• Outcome measure: Georgia Criterion-Referenced Competency Tests (CRCT) in language arts and reading.

THE STUDY

Arroyo Research Services has been commissioned by Walden University to engage in research partnerships with two large school systems to study the relationship between advanced degrees and teacher effectiveness as measured by student academic performance. In these partnerships, we seek extensive information regarding the effectiveness of master’s degree graduates in order to replicate a teacher effectiveness study conducted in Tacoma, Washington (Arroyo Research Services, 2009), and to answer questions of interest to each participating institution. The 2009 study compared academic outcomes of students whose teachers completed a Walden University Master of Science in Education (M.S.Ed.) with a specialization in Elementary Reading and Literacy (Grades PreK–6) to those of students whose teachers held master’s degrees from other institutions. The study reported herein uses student performance data provided by a large suburban district in Georgia and aims to address three primary questions:

• Do teachers with master’s degrees demonstrate greater teacher effectiveness than teachers without master’s degrees?

• Do teachers with content-specific master’s degrees demonstrate greater teacher effectiveness than teachers with other master’s degrees?

• Does obtaining a master’s degree increase teacher effectiveness?

SAMPlE CHARACTERISTICS

The data used in the study is drawn from district teacher and student performance data that include 2004–2010 student/teacher roster data, districtwide assessment data, teacher hiring data that includes schools attended and degrees earned, teacher and student demographic data, school demographics, and teacher records that allow imputation of degree attainment dates.

Additional degree data has been provided by Walden University and Market Data Retrieval.

The primary outcomes of interest are student performance on the reading and language arts sections of the Georgia Criterion-Referenced Competency Tests (CRCT) in grades 2 through 5 for 2004 through 2010. We standardized all CRCTresults by year, grade level, and subject area by calculating z-scores using state means and standard deviations provided by the district. At the teacher level, demographic data were made available by the district.

The analytic sample included only those students who had one reading teacher of record per year. If a student experienced a teacher change in any given year, that student’s CRCT score for that year was dropped from these analyses. Student performances were also limited to those in classes with a minimum of 13 students and a maximum of 30, effectively trimming the top and bottom 2% of class sizes, thereby focusing primarily on reading taught within multi-subject classrooms. A total of 205,226 student observations from 2004 to 2010 were included. A student could appear in the data set once each year for which they met the criteria.

The analytic sample included all teachers of record for reading during 2004–2010 (n = 4,106) whose students met the criteria above. Because the student record for each year includes information about if and when their teacher obtained a master’s degree, teachers who obtained their degree during the study period could have student performances both before and after they obtained their degree in the same analysis. The teacher sample was primarily female (92.5%) and white (80.3%). The average years of experience was 11, with an average of seven years worked in the district.

RESEARCH METHODOlOgY

This brief addresses the first two research questions above using these analytic strategies: 1) comparison of students’ performance in reading and language arts, as measured by the CRCT, between master’s degree and non-master’s degree teachers, and 2) comparison of students’ performance in reading and language arts, as measured by the CRCT, between teachers with a Walden M.S.Ed. with a specialization in Elementary Reading and Literacy (Grades PreK–6) and teachers with master’s degrees granted from other institutions. Multilevel regression models were fit due to the hierarchical structure of the data. Students’ scores were pooled  across years, by teacher, and schools were treated as fixed effects.

 Each analysis controls for:

• Student demographic characteristics such as gender, race, free and reduced lunch status, English language learner status, special education status, number of days enrolled, and studentprior academic achievement as measured by the CRCT.

• Teacher experience and peer academic performance (effect of classmate academic performance).

• School characteristics, including overall school academic performance and Title I status.

• Grade level and school year at the time of testing.The first analysis compares the performance of all students whose teachers did not have a master’s degree at the time of the CRCT to students whose teachers did have a master’s degree at the time of the CRCT. The second analysis includes only students of teachers who held a master’s degree at the time of the CRCT and compares performances between Walden M.S.Ed. with a specialization in Elementary Reading and Literacy (Grades PreK–6) graduates and teachers with master’s degrees from other institutions.

DETAIlED RESUlTS

We find that holding a master’s degree is associated with higher student outcomes in language arts and/or reading. Additionally, we find that holding a Walden M.S.Ed. with a specialization in Elementary Reading and Literacy (Grades PreK–6) is associated with statistically significantly higher student outcomes in language arts than other master’s degrees. Holding a Walden M.S.Ed. with a specialization in Elementary Reading and Literacy (Grades PreK–6) is not associated with statistically different student outcomes in reading than other master’s degrees. These results suggest that elementary teachers’ attainment of master’s degrees, irrespective of area of study, is associated with improved student achievement in both reading and language arts and that content specific programs are associated with additionally improved student achievement outcomes in language arts.

• Students whose teachers held a master’s degree performed .02 standard deviations higher in both language arts and reading. This is statistically significant at p < .01 for both.

Variables included in the analysis were selected using stepwise regression and other empirical methods to determine which variables influenced the outcomes of interest within the available data set. Additional details about variables and their selection are included in the FAQs. specialization in Elementary Reading and Literacy (Grades

PreK–6) performed .03 standard deviations higher in language arts than students whose teachers held a master’s degree from other institutions. This is statistically significant at p < .05.

• Students whose teachers held a Walden M.S.Ed. with a specialization in Elementary Reading and Literacy (Grades PreK–6) performed the same in reading as students whose teachers held a master’s degree from other institutions.

Notes regarding these outcomes:

• The standard deviation units in which results are reported are regression coefficients of standardized outcome variables and can therefore be read as measures of unique effect, or effect sizes, which are used by researchers to provide comparisons of the magnitude of an effect across studies. 

• Where “statistical significance” tells us the likelihood that the observed difference is due to chance, effect size tells us about the magnitude of the difference. Effect size takes into account the variation in scores within each group. 

• The effect sizes found in this study are higher than most studies have found for master’s degree effects and are in the range of small but meaningful for studies and measures of this type. The National Council on Teacher Quality (NCTQ, 2010), for example, used effect size standards of .15 for a large effect, .06 for a moderate effect, and .009 for a small but significant effect to review 36 studies of teachers with master’s degrees. Of these 36 studies, NCTQ found 16 studies with positive but not significant effects (all with effect sizes between 0 and .006); the remaining studies showed negative results for master’s degrees, with five studies finding small but significant negative effect sizes between -.012 and -.019. Clotfelter, Ladd, & Vigdor (2007) found statistically significant effect sizes for master’s degrees of -.003 and -.007 for reading outcomes using robust methods and a comprehensive data set covering the state of North Carolina.

By comparison, they also found statistically significant effect sizes for National Board Certified Teachers (NBCT) of between .01 and .02 for reading. Within the same study, these effect sizes for NBCTs are approximately 10 times stronger than those found for class size, six times stronger than classroom racial composition, and .4 times as strong as the effect size for parents who dropped out of high school.

• One way to understand the practical significance of these effects is to compare the effect sizes found for master’s degrees to the average difference between current year and prior year student test scores within the study sample. This is helpful because the effect sizes for the master’s degree analyses and the difference between average prior year scores and current year scores both describe group differences achieved within one year using the same testing data. From school years 2004 through 2009, the average growth in standardized scale scores on the CRCT assessment for students in the sample was .046 for reading and .059 for language arts. That is, without controlling for any other factors, students in the sample performed an average of .046 standard deviations higher in the current school year than in the prior school year in reading and .059 standard deviations higher for language arts. This includes students whose teachers had obtained a master’s degree and students whose teachers had not obtained a master’s degree.

By comparison, a .02 standardized mean difference for students whose teachers held a master’s degree is equal to 43% of the average year-to-year growth in reading performance for students in the sample; for language arts, a .02 standardized mean difference is equal to 34% of the average year-to-year growth in language arts performance for students in the sample.

While there is evidence to support content specificity in advanced degrees for classroom teachers, particularly in mathematics and science (Goldhaber & Brewer, 1997; Monk, 1994), this evidence is based primarily on studies of secondary teachers. It is not well understood how important an advanced degree with a content-area specialization (e.g., Elementary

Reading and Literacy [Grades PreK–6]) for a primary-grades teacher may be for improved student outcomes. This study adds to the body of evidence related to the question of whether content-specific degrees have a stronger effect on teacher effectiveness. This study is not designed to establish causality, and it is not our intent to overstate our claims. We do find compelling evidence for a continued investigation of the relation between teachers’ advanced degrees and student outcomes, and we find evidence to support the hypothesis that master’s degrees are associated with teacher effectiveness.

NEXT STEPS

The research to date has focused on determining whether teachers who hold general and content-specific master’s degrees are demonstrably more effective than those who do not when controlling for basic differences in school, teacher and student demographics, and performance. Further research will explore the extent to which teachers who obtain a master’s degree become more effective by doing so. The full study and associated Frequently Asked Questions will be published at arroyoresearchservices.com and WaldenU.edu/Outcomes.