Moving Beyond Single Rating Systems: Alternative State Accountability Frameworks to Promote Continuous Improvement

Several states are asking us: What alternatives are there to grading schools in state accountability systems? We recognize that states want a way to help communicate their work on education quality, effectiveness and accountability. The real question is this: How can states better determine school effectiveness, report on school progress and give data-driven feedback and evidence-based supports to move all schools forward in continuous improvement — ultimately to ensure quality, benchmarking, school improvement and excellence for every child?

With the passage of the Every Student Succeeds Act (ESSA), state leaders are asking important questions on how they can rethink the design of an accountability system that is meaningful to stakeholders, and that advances equity and quality. States can consider how next generation accountability systems focusing on continuous improvement can provide greater transparency, improve learning and support all schools in accelerating growth and closing gaps.

ESSA requires states to assign summative determinations of quality to schools to comply with federal law. States must set, at a minimum, three levels of performance for each indicator in their accountability system and for summative school rating and use this system of measures to identify schools for comprehensive support and targeted support. (To learn more about federal accountability requirements, see the iNACOL issue brief, Rethinking Accountability for Continuous Improvement.)

Balanced Scorecards With Multiple Measures Accountability

A balanced approach to measuring school quality can link multiple measures to each other and to a long-term vision focused on learning and growth. A balanced scorecard can communicate the right information to the appropriate stakeholders and drive more meaningful and timely identification of schools for improvement.

Balanced scorecards are an approach to multiple measures accountability for “defining the links between leading inputs, processes and outcomes and focusing on the importance of managing these components to achieve the organization’s strategic priorities.” (Strategy Focused Schools: An Implementation of the Balanced Scorecard in Provision of Educational Services)

The Problem With Single, Summative School Quality Ratings

In K-12 education, single, summative ratings, such as assigning grades to rank schools in a state, provide little insight into how students are learning and how schools could improve. For example, A-F school ratings assign a single letter grade to a school after averaging several domains of school performance. This letter grade may be easy for stakeholders to understand, yet the paradox is that the oversimplification actually explains little about the degree to which schools are accurately reporting student success. A school with a high grade may be doing well on the most heavily weighted measures, yet this could hide the low performance of certain student subgroups, and schools receiving a lower rating may show significant improvements in student growth and provide mixed messages when in fact the school is heading in the right direction for continuous improvement. This can happen when A-F ratings give significant weight to summative tests. Information from accountability systems should be used for continuous improvement of all schools.

State School Quality Ratings Systems

Under ESSA, states have gained considerably more autonomy over the design of school accountability systems. Some states are creating accountability systems that go beyond single summative ratings to capture multiple measures of school quality, provide stakeholders with timely, relevant school data, and encourage all schools toward continuous improvement. States can also look to international examples of how other countries have developed accountability systems that ensure quality and consistency in measuring school effectiveness, transparent data and reporting systems, and school and system improvement that is ongoing and based on reciprocal accountability.

How do states push for more continuous improvement now that they have additional flexibility? States have much more flexibility to improve their plans to focus on continuous improvement, and we want to highlight future directions in the following examples of states beginning to move in this direction. We are beginning to see some states approach how they use their rating systems differently by focusing on how data is reported and how schools will be determined for robust support for tiers of school improvement.

The Education Commission of the States provides a national overview on how states are currently approaching accountability in their state ESSA plans here. Below are promising examples of how some states and other countries are using data in more balanced ways and are designing accountability systems that promote continuous improvement.

Connecticut’s Multiple Measures Accountability and Data Reporting

Connecticut will use categories 1-5 as part of its “Next Generation Accountability System.” Connecticut schools will receive a report that provides a picture of how the school measures up on the state’s twelve indicators. The data will be easily searchable and will provide a more comprehensive, holistic picture of school performance. Each of the five categories (from top to bottom quartiles of school performance) are used to inform the three tiers of state support (basic, moderate, and intensive support) for continuous school improvement under Connecticut’s State ESSA Plan.

Maine’s Personalized School Support System

Maine will use tiers of support for schools based on performance, with the highest level of support offering coaching and mentoring to teach effective strategies for school turnaround, in addition to increased funding for staff professional development. Additionally, Maine’s Department of Education will provide access to individualized support to all schools and school administrative units to establish improvement goals that aim to improve outcomes for students, with the intention of setting goals that are relevant and appropriate to the needs of individual school communities.

California’s State and Local Accountability Dashboards

California will use a dashboard: The California dashboard on multiple measures of school success provides schools and communities with information on how students are progressing on the state’s 10 indicators of school success. Six of the indicators are common across the state and collected by the state to monitor school performance and outcomes. The other four indicators are designed and collected by local education agencies, and provide district and school leaders information on non-academic factors that could influence student learning outcomes and educator capacity that is valuable for purposes of school support, early intervention and building educator capacity.

Vermont’s Model for Continuous Improvement in School Accountability

Vermont’s State ESSA Plan includes multiple measures on school performance through five state priorities: safe, healthy schools; high quality staffing; investment priorities; academic proficiency; and personalization. These priorities are aligned to Vermont’s Education Quality Standards (EQS), a series of policy guidelines requiring all schools to provide “educational opportunities which are substantially equal in quality, ensure continuous improvement in student performance… and annually report to the community.”

Resources on Vermont’s Accountability and Continuous Improvement Plans

Annual Snapshot: Multiple Measures: This graphic illustrates Vermont’s five core priorities and the indicators for which all schools in the state will collect data. The Vermont Agency of Education will use a rating system for these indicators that emphasizes continuous growth and improvement to meet all student outcomes at “on-target.”

Weighting the Measures: Academic Proficiency: This graphic further breaks down how Vermont will weight each measure to arrive at an overall rating for each of the state’s five core priorities. The state takes into consideration measures for college and career readiness, graduation rates and other post-secondary outcomes. Further, the weights vary slightly depending on specific school characteristics, such as student subgroups.

State Accountability: All Measures: This resource provides insight on how Vermont approaches each of its core accountability principles aligned to proposed reporting measures on student outcomes. The proposed reporting measures provide transparency of data collection and the key accountability questions can help to shape conversations between the state, district and school stakeholders on school progress.

Ensuring School Quality In Alberta, Canada and New Zealand

Alberta, Canada

The province of Alberta, Canada uses the “Accountability Pillar” to:

• Give school boards a consistent way to measure their success and assess progress using a broad spectrum of measures;
• Help identify areas that need improvement and set priorities for the future;
• Provide schools with a wide range of data showing how they are performing; and
• Help schools focus on more than just student achievement, resulting in a more complete approach to accountability and system improvement.

Alberta’s approach to accountability helps to foster continuous improvement and reciprocal accountability. The Alberta Ministry websites states, “As part of the accountability process, school authorities report their Accountability Pillar results to their communities. Sharing the results with their communities allows Albertans to see how their school authority is performing. This ensures that the entire education system is more open and accountable to all Albertans.”

For more information, see the fact sheet: Accountability: Achieving Outcomes, Reporting Results and Using Results for Information Decision-Making in the K-12 Education System.

Also, see the iNACOL blog post: Reflections on Education Ministry Meetings in Alberta, Canada – Positive Insights across North America and Lessons for the USA.

New Zealand

Accountability in New Zealand is also focused on continuous improvement and building educator capacity to ensure every student can succeed.

The New Zealand Education Review Office (ERO) states, “All schools are expected to be involved in an ongoing, cyclical process of evaluation and inquiry for improvement. Through the annual reporting process, they are required to report on the achievement of their students, their priorities for improvement, and the actions they plan to take.”

This video describes New Zealand’s priorities and processes to ensure success for every student.

A resource to learn more about New Zealand’s accountability processes is School Evaluation Indicators: Effective Practice for Improvement and Learner Success.

Next Steps on Rethinking Accountability

States have the opportunity under ESSA to transform accountability with models that advance equity and align to student-centered learning. This shift in responsibility creates the opportunity for states to reimagine new accountability models that align to goals of college and career readiness for all students, and to move to a culture of continuous improvement.

Learn More:

• Learning Policy Institute: Accountability for College and Career Readiness: Developing a New Paradigm
• iNACOL Issue Brief: Rethinking State Accountability to Support Personalized, Competency- Based Learning in K-12 Education
• iNACOL Issue Brief: Current to Future State: Issues and Action Steps for State Policy to Support Personalized, Competency- Based Learning
• iNACOL Issue Brief: Redefining Student Success: Profile of a Graduate