Measuring school quality beyond test scores

Photo by Shubham Sharan on Unsplash.

Challenge

 

The NYC Department of Education’s (NYC DOE) annual reports about school performance focused heavily on test scores and other quantitative performance metrics. While those outcomes are important, they don’t tell readers about other key aspects of student experiences and school quality. And focusing solely on quantitative outcomes gives limited guidance about what changes at the school could bring about better outcomes.

How might we help educators, administrators, and community members gain a more complete and actionable understanding of school performance and quality?


Process

 

The NYC DOE drew on research from the Chicago Consortium for School Research that identified specific school conditions and practices—rigorous instruction, collaborative teachers, supportive environment, effective school leadership, strong family-community ties, and trust—that led to improved student achievement, and organized these elements into a “Framework for Great Schools.” The NYC DOE partnered with researchers to align the NYC School Survey of teachers, students, and parents with these areas. The NYC DOE also identified data from Quality Reviews, qualitative school evaluations by expert reviewers, that reflected these areas.

We turned the Framework for Great Schools concept into concrete data, and analyzed those results to create school ratings in each of the six areas. We engaged stakeholders and got feedback on our approach. We released an internal Framework report for principals and field staff, and conducted trainings and feedback sessions.

We incorporated the Framework for Great Schools into our annual reports, along with quantitative data on school performance, and conducted trainings and feedback sessions. Over time, we have continued to improve how we analyze and share this information. For example, readers can investigate more deeply to learn about the specific survey questions and Quality Review findings in each area.


Outcome

 

Our annual reports on school performance and quality are organized around the Framework for Great Schools, which provide information on school conditions and practices that lead to improved student outcomes. Parents can use these reports to better understand school quality, and educators and administrators can use the reports to identify areas for schools to work on. These reports receive over 200,000 unique page views per year.

Also, our research has found—similar to the Chicago research—that schools strong in the Framework areas are far more likely to help students grow and exceed expected outcomes than school weak in the Framework areas.


My Role

 

I played a major role in planning how the Framework for Great Schools would be incorporated into our reports. I facilitated planning sessions to determine the data points for each of the report sections and decide how they would be presented; conducted feedback sessions with stakeholders (including principals, administrators, union leaders, and parents) and synthesized feedback; developed the method for calculating school ratings in each area; and conducted trainings and presentations about the updated report structure based on the new framework.


Project Artifacts

Snapshot.JPG

School Quality Snapshot screenshot

The School Quality Snapshot is a report on school performance and quality for parents and educators. It is organized around the Framework for Great Schools.

Guide 2.JPG

School Quality Guide screenshot

The School Quality Guide lets readers dig deeper into the data about school conditions and practices. The report is organized around the Framework for Great Schools.

Decisions doc.JPG

Planning document

Excerpt from document outlining key questions about implementing the new framework in our reports.

Feedback 5.png

Principal feedback

Excerpt of summary notes from meetings with principals about the proposed report changes.

Previous
Previous

Experimenting on promising projects

Next
Next

Showing school outcomes in two dimensions