This post originally appeared on the MIT news site and is republished here with permission.
Key points:
- School quality ratings significantly reflect a school's student readiness, not just the school's contribution to learning achievements.
- Key Qualities of a Head Coach
- Addressing the Root Causes of Chronic K-12 Absenteeism
- For more school news, visit eSN's Educational Leadership hub
Are schools that do well on tests very effective or do they primarily enroll students who are already well prepared for success? A study co-authored by MIT academics concludes that widely held school quality ratings reflect their students' preparation and family background as much or more than a school's contribution to learning achievement.
In fact, the study finds that many schools that receive relatively low ratings perform better than these ratings would imply. Conventional grades, research makes clear, are highly correlated with race. Specifically, many published school grades have a very positive correlation with the proportion of white students.
“A school's average results reflect, to some extent, the demographic mix of the population it serves,” says MIT economist Josh Angrist, a Nobel Prize winner who has long analyzed educational outcomes. Angrist is co-author of a recently published paper detailing the results of the study.
The study, which examines Denver and New York City school districts, has the potential to significantly improve the way school quality is measured. Instead of raw aggregate measures such as test scores, the study uses changes in test scores and a statistical adjustment for racial composition to calculate more precise measures of the causal effects that attendance at a particular school has on the advances in student learning. This methodologically sophisticated research relies on the fact that both Denver and New York City assign students to schools in ways that allow researchers to mimic the conditions of a randomized trial.
Documenting a strong correlation between the grading systems currently used and race, the study finds that white and Asian students tend to attend schools with higher ratings, while black and Hispanic students tend to cluster in schools with lower ratings. low.
“Simple measures of school quality, which are based on average school statistics, are invariably highly correlated with race, and those measures tend to be a misleading guide to what you can expect when sending your child to that school.” says Angrist. .
Paper, “Race and the poor measurement of school quality”, appears in the last issue of theAmerican Economic Review: Perspectives. The authors are Angrist, Ford Professor of Economics at MIT; Peter Hull PhD '17, professor of economics at Brown University; Parag Pathak, Class of 1922 Professor of Economics at MIT; and Christopher Walters PhD '13, associate professor of economics at the University of California, Berkeley. Angrist and Pathak are professors in the MIT Department of Economics and co-founders of MIT's Blueprint Labs, a research group that often examines school performance.
The study uses data provided by the Denver and New York City public school districts, where sixth graders apply for spots in select middle schools and the districts use a school assignment system. In these districts, students can choose any school in the district, but some schools are oversubscribed. Under these circumstances, the district uses a random lottery number to determine who gets a seat and where.
Under the lottery within the seating algorithm, groups of similar students randomly attend a variety of different schools. This facilitates comparisons that reveal the causal effects of school attendance on learning gains, as in a randomized clinical trial of the type used in medical research. Using math and English test scores, researchers assessed student progress in Denver from the 2012-2013 through 2018-2019 school years, and in New York City from the 2016-2017 through 2018 school years. 2019.
It turns out that these school assignment systems are mechanisms that some of the researchers have helped build, allowing them to better understand and measure the effects of school assignment.
“An unexpected dividend of our work designing Denver and New York City's centralized choice systems is that we see how students are rationed (distributed) to schools,” Pathak says. “This leads to a research design that can isolate cause and effect.”
Ultimately, the study shows that much of the school-to-school variation in total raw test scores is due to the types of students at a given school. This is a case of what researchers call “selection bias.” In this case, selection bias arises from the fact that more advantaged families tend to prefer the same set of schools.
“The fundamental problem here is selection bias,” says Angrist. “In the case of schools, selection bias has many consequences and is a large part of American life. “Many decision makers, whether families or policymakers, are being misled by a kind of naive interpretation of the data.”
In fact, Pathak observes, the preponderance of more simplistic school grades today (found on many popular websites) not only creates a misleading picture of how much value schools add to students, but has a self-reinforcing effect ( as well-prepared and better-prepared schools) Excluded families increase housing costs near highly rated schools. As the scholars write in the article, “Biased grading schemes direct households to low-minority schools rather than high-quality schools, while penalizing schools that improve the performance of disadvantaged groups.”
The research team hopes their study will lead districts to examine and improve how they measure and report on school quality. To that end, Blueprint Labs is working with the New York City Department of Education to pilot a new grading system later this year. They also plan to do additional work to examine how families respond to different types of information about school quality.
As the researchers propose improving scores in a way they believe is straightforward, taking into account student readiness and improvement, they believe more officials and districts may be interested in updating their measurement practices.
“We are hopeful that the simple regression adjustment we propose will make it relatively easy for school districts to use our measure in practice,” Pathak says.
The research was supported by the Walton Foundation and the National Science Foundation.
Reprinted with permission of MIT News.
