Aurora Institute
Blogs Education Domain Blog Threshold Concept: Building Capacity for Assessment Literacy in Personalized, Competency-Based Education

Threshold Concept: Building Capacity for Assessment Literacy in Personalized, Competency-Based Education

Education Domain Blog

Author(s): Susan Patrick, Dale Frost, Maria Worthen, Natalie Truong

Issue(s): Federal Policy, Create Space to Pilot Systems of Assessments

 

There is a need to rethink the purpose and role of assessment in education systems. Assessment literacy, according to the New Zealand Ministry of Education, is “the possession of knowledge about the basic principles of sound assessment practice, including its terminology, the development and use of assessment methodologies and techniques, and familiarity with standards of quality in assessment.” This knowledge is essential to advancing personalized, competency-based learning. It is critical for educators to develop the professional judgment to reliably and accurately evaluate and determine student mastery in student-centered learning environments.

Assessment literacy is important for practitioners, and it is also important for policymakers and stakeholders throughout the system to understand the roles that different types of assessment play in student learning, how assessment and moderation are used to comparatively and fairly judge student mastery and how the information generated by assessments can be used toward a cycle of continuous improvement in teaching and learning. The lack of assessment literacy across the system is a major blind spot. Thus, building significant capacity for assessment literacy is needed to advance new competency-based approaches and address tough issues in our current system.

An important concept in assessment is comparability; that is the degree to which assessment results intended to measure the same learning targets produce the same or similar results. This involves documenting the reliability of judgments and not assuming that comparability is stable over time or invariant across multiple subgroups, such as English learners and special education students.

There are unique circumstances in the U.S. education system that have driven the need for much greater degrees of comparability than is true in most other nations. When the federal government became involved in K-12 education with the Elementary and Secondary Education Act of 1965, it was in direct response to deep inequities that have remained in the wake of desegregation. Because of the history of inequities in education offerings among student groups, concerns for equity are much greater than in many other countries. This circumstance drives, to a significant extent, the degree to which we need to take greater care that measures are fair and have common meaning among students, schools and districts. This drives the prevalence of standardized tests in our country, causing the concept of assessment to often be conflated with the end-of-year, statewide, summative accountability tests.

Practitioners working deeply in competency-based learning models quickly realized that current K-12 education systems lack systems for calibrating the quality of student work to ensure there is significant consistency in determining proficiency across schools and systems. Building professional educator capacity and policymakers’ understanding of assessment literacy is fundamental to shifting to personalized, competency-based systems at scale and with equity.

Personalized, competency-based learning requires us to reorganize systems around doing what it takes to ensure every student is attaining mastery. Redesigned systems will need to build capacity for clear evaluation criteria to make valid and reliable comparisons of students’ progress against outcomes (commonly understood outcomes) using evidence and common rubrics.

This process of developing clear expectations for common proficiency levels is a key part of a “calibration.” Calibration is a process that allows two or more things to be compared via a common standard (e.g., a weight in the physical sciences or commonly scored papers in an education system). The purpose of common performance tasks given to students by different schools and districts is to serve as a “calibration weight;” a way to compare the way one school or district scores students on the common task, with the way other schools and districts score those same students’ work.

To use the common performance tasks as calibration weights, districts need to re-score other districts’ common performance tasks. Calibrating expectations as well as grading and scoring processes for learning goals is very important in competency-based learning systems. Calibration may involve groups of educators who collaborate and develop consensus around rubrics for scoring student work. The calibration process makes scoring student work consistent and more aligned to the standards upon which rubrics and scoring criteria are based, as well as creating reflective processes focused on improving student learning.

In addition to calibration processes for consistently and accurately evaluating student work, assessment literacy also includes knowing which assessments are appropriate for what purpose (e.g., formative, progress monitoring or summative). This idea of common expectations—and evaluating evidence against common standards and rubrics to build and evaluate comparability across schools and systems—requires careful moderation of assessment practices across the system and perhaps across the state level. The capacity of educators to assess student evidence using calibration processes and developing rubrics with scales for evaluating performance tasks against criteria is central to a competency-based education system. A competency-based learning system that offers personalized pathways for students to meet learning goals and learning targets must rely on multiple forms of evidence against common standards and expectations.

Learn more about assessment literacy, balanced systems of assessments and developing next-generation accountability models in the paper Fit for Purpose: Taking the Long View on Systems Change and Policy to Support Competency Education.

Follow this blog series for more key insights:

Learn more:

Follow us on social media: @nacol, @SusanDPatrick, @mariaworthen, @DaleKFrost, @ntruongedu

Author(s)