Innovating Towards Next Generation Accountability

I was inspired to read the National Center for Assessment’s recent article calling for new accountability pilots, Why It’s Time for a New Accountability Pilot.

The time is overdue to architect short-term policy changes toward long-term goals to advance education systems change for human thriving. We need to fundamentally rethink and redesign toward next generation accountability in the United States. The federal government needs to change the frame.

America’s current school accountability policy is designed to rank and sort students and schools. Though it was intended to drive continuous educational improvement, it has fallen far from that goal. Policy makers across the country have been engaging in conversations about next generation accountability. In fact, the US Department of Education recently submitted a Request for Information on the Innovative Assessment and Accountability Demonstration Authority in Section 1204 of ESSA. You can see Aurora’s letter in response to that request here.

The Aurora Institute believes that the time is now for the federal government to rethink and innovate its approach to accountability and assessment policy to ensure that every learner has access to meaningful assessments and accountability systems that are designed to improve teaching and learning. These new systems could provide better information to students, families, and communities about what matters most. 

The Problem 

The authors of No Child Left Behind acknowledge that the current accountability system is not producing the results they had hoped for. And it is resulting in unintended consequences, such as narrowing of the curriculum and the focus of schools. The data models conceived in 2001 and formalized through No Child Left Behind have not matured to provide real-time data for every student, every day. The time has come to stop doing the same thing year after year and make room for piloting innovations in next generation accountability.

The current accountability models are not resulting in continuous improvement for schools and students because they are focused on comparability, compliance-based, based on data that comes too late, and top-down.

Old Data Models 

The current accountability models are entrenched in the factory model approach to education and are time-based, rather than learning-based. They are more interested in assessing students on a single path based on age, rather than interest, ability, or growth. As a result, the current accountability and assessment frame prioritizes comparability of students and schools over cumulative validity of the evidence of outcomes. By over-relying on standardized tests, these systems favor comparability over validity and fly in the face of the science of learning. These tests rely on an overly narrow view of what students know and are able to do, prioritizing test item sameness over a comprehensive and multi-faceted framework of balanced assessments, evidence, and student outcomes that matter for well-rounded students. By prioritizing comparability and replicability, the metrics systems are held accountable for are not meaningful to students, families, or communities. 

Overly Narrow Focus 

Compliance-based accountability entails narrow, time-based metrics of student achievement, cohorts of students with groups and sub-groups, and a one-size-fits-all approach. They measure whether students are “proficient” against arbitrary cut scores on a standardized test. These accountability systems have been designed to sort and identify the lowest performing 5% of schools in each state. Compliance-based accountability is about collecting data and reporting on that data disaggregated by subgroups to highlight the current inequities in the system. But now that those inequities are clear, accountability systems have been repeating that pattern over the last 21 years, rather than shifting to drive continuous improvement in teaching and learning or even create a better data model through personalized, competency-based education systems. These changes would collect better information on where students are in their learning progression and could be used to support more powerful student learning. 

Too Late

The standardized assessments that drive accountability provide only a snapshot of student ability at one moment in time as an autopsy at the end of the school year. They emphasize test taking skills rather than lifelong learning skills. They provide data far too late for educators to improve instruction or better support those students. As a result, the current system doesn’t focus on the continuous improvement and quality assurance that it purports to support. By the time educators have access to information on student progress, those students have moved on to new grade levels, schools, or systems. Systems must invest in building capacity for assessment literacy and continuous improvement for balanced systems of assessment that provide information on student mastery each and every day. Monitoring student progress effectively can be done with reliability in real-time. 

Top-down

Finally, the current accountability system is top-down. It misses the necessary input of community and local control by placing all power in the hands of the federal and state governments. Lawmakers need to create room for innovating and sharing power with local communities with reciprocity in accountability. To design these reciprocal accountability systems, we need to ask questions such as, “What does the federal government need to know?” “What do states need to know?” and “What do local communities want in an accountability system that is responsive to local needs?” States have the power to define standards, set cut scores, and define how they intervene with struggling schools. But local communities should have a role in creating performance metrics to hold schools directly accountable to their students, families, employers, and communities, not just the lawmakers at the state and federal level. 

Policy Recommendations

The current accountability systems fall short on providing timely, relevant information to families and educators, getting the resources to students who need them the most, and driving continuous improvement. To truly support student success, we must shift to accountability for continuous improvement, using evidence-based practices and performance frameworks to improve instruction, personalize learning and monitor progress in real time. In response to ongoing feedback and data, these systems drive change to improve and evolve practice. Continuously improving systems can evolve to ensure that students get the opportunities for learning, as well as the interventions and supports they need in real time.

Designing effective, future-focused accountability systems, however, will require making space to pilot and innovate new approaches that are rooted in more meaningful and modern definitions of student success. Local communities across the country and nineteen states are working on creating new graduate profiles. This starts with engaging communities – students, families, employers, and more – in developing Profiles of a Graduate for a more complete vision of what a student should know and be able to do upon graduation. These profiles provide a holistic picture of the knowledge, skills, competencies, and abilities beyond traditional academic outcomes that students need to know and do before graduating.  

From those graduate profiles, school systems can develop performance frameworks that encompass the most important aspects of success – this could include a frame using balanced systems of assessments of the following: tests, performance assessments, evidence of student work, and/or capstones. But it could also involve emphasis on opportunities for learning such as career explorations, access to experiences like paid internships, support services for mental health and social emotional learning, and a balance of inputs and outcomes measures through surveys of school climate and belonging. These new locally-derived frameworks would provide the basis for exploring community goals and rethinking an accountability system that moves beyond single snapshot test scores and towards a more meaningful capture of student learning and well-rounded youth. 

These local performance frameworks will help define the metrics that schools and systems collect. In competency-based learning systems, educators use a graduate profile to communicate and report on mastery as students build knowledge and skills in real-time. Every student, parent, and educator has access to data on where students are on their learning progressions and evidence of students demonstrating mastery daily. Students and families are more empowered. And through that real-time data collection, systems can focus on continuous improvement around assessments for learning to build true quality assurance systems that allow educators and administrators to reflect on outcomes, improve instructional practice, and be more responsive to their communities. 

Importantly, local communities should inform what those profiles, frameworks, and quality assurance systems prioritize. While many countries have been approaching quality assurance using performance frameworks informed by community and stakeholder engagement in accountability for years, it is an entirely new frame in the United States. As a result, we must open up avenues for piloting and testing innovative assessment and accountability approaches and allow communities to innovate and build capacity in real-time. 

The key to making this shift is “reciprocal accountability.” Reciprocal accountability is the practice of mutual responsibility within a system. It is based on the idea that everyone involved in a given relationship must take responsibility for their roles and actions. This means that all parties must be accountable to each other for outcomes and goals, as well as their individual actions. The goal is to create an environment with transparent information where each party works together to achieve success. 

In Bridging the Gap Between Standards and Achievement, the late Harvard Professor Richard Elmore explains: “Accountability must be a reciprocal process. For every increment of performance I demand from you, I have an equal responsibility to provide you with the capacity to meet that expectation. Likewise, for every investment you make in my skill and knowledge, I have a reciprocal responsibility to demonstrate some new increment in performance. This is the principle of reciprocity of accountability for capacity.”

To accomplish this goal and enable movement towards next generation accountability systems, states must:

• Create flexibility and enabling policy to allow for new accountability systems with room for prototyping, piloting and incubating next generation accountability models. 
• Rethink measurement systems to more effectively evaluate school quality through performance frameworks and new metrics.
• Engage states and communities to design new reciprocal accountability systems with permission to re-examine the purpose of schooling and rethink accountability.
• Allow for a learning agenda with research and development (R&D) to plan, develop, and pilot next generation accountability models.
• Be flexible to allow for new metrics, frameworks, and concepts for quality assurance in evaluation. 
• Launch investments to build capacity to modernize accountability and evaluate school quality using reciprocal accountability and results-based performance frameworks.
• Focus on accountability that provides coherence for education systems at all levels, emphasizing learning and continuous improvement, aligning to learning sciences research, advancing student-centered learning, and building trust.

Creating space for piloting and innovating next generation accountability systems that are informed by community and balanced will accomplish three major shifts:

1. Reciprocal accountability will ensure that students, families, employers, and the broader community are empowered to and engaged in driving the purpose of education systems. In turn, education systems will better meet the needs of their surrounding communities. 
2. Educators and administrators will have access to the real time data necessary to improve instruction in response to student needs and create true continuous improvement. This will also facilitate a shift from compliance to strengthening capacity and empowering leaders, educators, families, and learners with the data and support they need to succeed. By working with communities and school systems to articulate their visions for student success and align next generation accountability systems to hold stakeholders accountable to this shared vision, balanced systems of assessments can provide more mature data models to inform quality assurance and best practices.  
3. Systems will build the capacity to meaningfully measure student learning in ways that are tied to the science of learning and enable personalized, competency-based education. Building competency frameworks and engaging in best practices in making mastery – not time – the focus, can catalyze systemic change and transform student learning inside and outside of the classroom. Shifting the focus to learning and community engagement will result in more engaging learning experiences and open pathways for extended learning opportunities. 

We need to innovate in education toward new delivery models that are personalized and competency-based to create a more flexible student-centered system that ensures learners have pathways to meaningful credentials and graduate on the path to future prosperity and self-fulfillment. This requires addressing and re-architecting the most significant time-based policy barriers to serve a new purpose and addressing the twenty-year old policies around accountability that hold the components and conditions of today’s inequitable education systems in place. It is urgent that we advance policy to enable and support responsiveness to student, family, and local needs. 

Benefits to Students

Through the combined power of a more complete definition of success, as well as the collection of better data on that comprehensive picture of what a student knows and is able to do, education systems will be engaged in true continuous improvement. They will have the information they need to improve teaching and learning in real time, effectively supporting students towards success in both academic and personal pursuits.

At the same time, the accountability system will be better aligned with and supportive of  personalized and competency-based approaches, which reflect the science of learning. Systems will move away from time-based and age-based structures and towards meaningful assessments of learning that capture evidence of building knowledge and skills and reflect the development of youth and new abilities for learning how to learn. As a result, students will be able to take more ownership of their learning and move at more flexible pacing that adapts to their needs and interests. Systems will still hold high standards and learning goals for every student, but pathways and pacing will vary. It is important that systems build capacity around demonstrating mastery and a greater focus on hands-on learning and student work to provide evidence of progress beyond standardized testing. This will require creating pilots to seed and develop innovative assessments.

Ultimately, by freeing accountability from focusing solely on compliance and toward innovating teaching, learning, and assessment, systems will be able to build systems of continuous improvement and quality assurance. As a result, schools will be empowered to serve their students and communities with the goal of achieving human flourishing, and not just minimum levels of academic achievement. Literacy and numeracy are essential, but no longer sufficient. 

Taking an evidence-based approach that is engaging, responsive, and aligned to student-centered learning will result in more equitable outcomes, as students receive what they need when they need it. This means that students will receive support based on the goals they set and show evidence of their progress in building knowledge and skills as they explore and develop academic and career pathways. Students, in partnership with educators, could design learning experiences for either a path to college or a path to a vocational certification or other career outcomes that ensure prosperity. 

Next generation accountability involves the input, involvement, and wisdom of communities in co-designing and creating graduate profiles, articulating pathways to graduation, and co-designing new performance frameworks with reciprocity.

Opportunities

Opening up assessment and accountability systems for innovation and moving towards reciprocal accountability will shift power from state and federal governments and towards local communities, including employers and the broader workforce. This tradeoff will result in education systems that are designed to meet the needs of their communities and are held accountable to a more comprehensive set of inputs and outcomes based on those needs. In international examples, such as in New Zealand, reciprocal accountability systems can result in more meaningful accountability, while still upholding rigorous standards and benchmarks. The belief that democratizing accountability will result in lower standards and poor outcomes does not reflect realities around the world. Processes to ensure quality assurance are needed.

Another tradeoff is in how you define comparability. Comparability could be defined by benchmarks of learning outcomes. Comparability should not be defined by comparing to the old, antiquated system’s results. By allowing variation in performance frameworks at the system- and community-level based on community priorities, states may examine a broader range of metrics identified by communities. The state could select a subset of outcomes for measurement that is reliable and valid, prioritizing validity over comparability. New theories of change focusing on measures more appropriate to supporting youth development could help move systems toward broader improvements and broader aims. Thus, the goal of the resulting system will be more valid and meaningful to students, families, and employers, as they have been core constituents in developing it. By trading off comparability for validity, systems will be able to focus on accountability for building capacity and continuous improvement, rather than punitive measures.

There will be significant change processes involved in piloting and innovating new accountability models. There will need to be a research and development agenda within the pilot programs. Not every pilot will be successful. However, disruptive innovation theory tells us that to achieve true change, we must make space for innovation and not require systems to participate in the old and new system at the same time. It also tells us that we cannot compare the results in the same way. We should not use the old metrics to evaluate new pilots.

This approach to accountability is rooted in global best practices.

With a population of just over five million people, New Zealand represents the scale of a medium-sized state in the United States such as Alabama – 26 states and DC have populations below five million. Its focused approach to education system transformation and accountability offers a global best practice to study. 

New Zealand has a distinctly different accountability system than the United States – it provides reciprocal accountability through shared responsibility. Global education experts recognize New Zealand’s accountability system as having evolved to better support building a world-class education system for the future by sharing responsibility and engaging in reciprocity with communities. For accountability, schools use a results-based outcomes framework, setting metrics with communities, families, students, and stakeholders through local boards. In evaluating school quality, there is a balanced scorecard of performance metrics through outcomes-based models. The accountability and measurement quality are enforced by a different government entity, the Education Review Office, with an accreditation process similar to a charter authorizer, rather than through a national testing program. The National Ministry of Education sets the national curriculum frameworks and the learning goals as requirements. 

There is currently little use of large-scale, high-stakes summative testing, except in the final three years of secondary school. The New Zealand Qualifications Authority oversees the quality of a learner record (transcript) or high school credential, which also monitors the consistency and moderation of performance assessments through sampling. New Zealand’s system of education accountability is based around the frameworks set by the New Zealand Qualifications Authority (NZQA) and the Education Review Office (ERO). The implementation of accountability, however, is ultimately up to local control. At the local level, there is locally-driven reciprocal accountability between each school, their educators, leaders, members of the community, families, and students. 

The goal is a world-class K-12 education. Every student has a learner record or “Record of Achievement” that communicates mastery of competency development, including evidence of student work. It provides local communities with shared responsibility for educational excellence. Finally, this happens in the context of examining purposeful policies to drive human flourishing that shape the entire context for society in New Zealand.

The OECD recommends that accountability policy should both “Foster partnerships between communities, parents, providers, institutions, and education systems in order to create high-performing schools” and “develop a culture of continuous improvement in schools, with communities, parents, students, educators, and leaders taking ownership of school outcomes through reciprocity.” The American education system has been operating with a narrow approach to accountability outside of these recommendations for nearly 20 years. We have data clearly illustrating an inequitable system by race and income. However, we are not gearing an engine of change toward transformation of education. The time has come to empower systems and their communities to do something different.

This work is licensed under a Creative Commons CC BY Attribution 4.0 International License