{"id":5753,"date":"2017-06-22T00:00:00","date_gmt":"2017-06-22T04:00:00","guid":{"rendered":"http:\/\/aurora-institute.org\/blog\/cw_post\/threshold-concept-assessment-literacy\/"},"modified":"2022-12-12T13:51:19","modified_gmt":"2022-12-12T18:51:19","slug":"threshold-concept-assessment-literacy","status":"publish","type":"cw_post","link":"https:\/\/aurora-institute.org\/cw_post\/threshold-concept-assessment-literacy\/","title":{"rendered":"Threshold Concept: Assessment Literacy"},"content":{"rendered":"
\"\"
Courtesy of Allison Shelley\/The Verbatim Agency for American Education: Images of Teachers and Students in Action<\/strong><\/figcaption><\/figure>\n

This is the nineteenth article in a series leading up to the <\/span><\/i>National Summit on K-12 Competency-Based Education<\/span><\/i><\/a>. We are focusing on four key areas: equity, quality, meeting students where they are, and policy. (Learn more about the Summit <\/span><\/i>here<\/span><\/i><\/a>.) We released a series of draft papers in early June to begin addressing these issues. This article is adapted from <\/span><\/i>Fit for Purpose: Taking the Long View on Systems Change and Policy to Support Competency Education<\/span><\/a>. It is important to remember that all of these ideas can be further developed, revised, or combined \u2013 the papers are only a starting point for introducing these key issues and driving discussions at the Summit. We would love to hear your comments on which ideas are strong, which are wrong, and what might be missing.<\/span><\/i><\/p>\n

\u201cStudent assessment is essential to measure the progress and performance of individual students, plan further steps for the improvement of teaching and learning, and share information with relevant stakeholders.\u201d \u2013 OECD, 2013<\/i><\/strong><\/p>\n

Assessment literacy is important for practitioners but it is also important for policymakers and stakeholders throughout the system to understand the roles that different types of assessment play in student learning, how assessment and moderation are used to comparatively and fairly judge student mastery, and how the information generated by assessments can be used toward a cycle of continuous improvement in teaching and learning. The lack of assessment literacy across the system is a major blind spot. Thus, building significant capacity for assessment literacy is needed to advance new competency-based approaches and address tough issues in our current system.<\/span><\/p>\n

An important concept in assessment today is related to the concept of <\/span>comparability<\/span><\/i>. Comparability is defined as the degree to which the results of assessments intended to measure the same learning targets produce the same or similar results. This involves documenting the reliability of judgments and not assuming that comparability is stable over time or invariant across multiple subgroups such as English language learners and special education students.<\/span>1<\/sup><\/p>\n

There are unique circumstances in the U.S. education system that have driven the need for much greater degrees of comparability than is true in most other nations. When the federal government became involved in K-12 education with the Elementary and Secondary Education Act of 1965, it was in direct response to deep inequities that remained even after school segregation. Because of the history of inequities in education offerings among student groups, concerns for equity are much greater than in many other countries, which drives, to a significant extent, the degree to which we need to take greater care that measures are fair and have common meaning among students, schools, and districts.<\/span>2<\/sup> This drives the prevalence of standardized tests in our country, causing the concept of assessment to often be conflated with the end-of-year, statewide, summative accountability tests. <\/span><\/p>\n

Practitioners working deeply in competency-based learning models realize quickly how our K-12 education systems lack systems for calibrating the quality of student work, so we know that fundamentally there is significant consistency across schools and systems. As much of a systems challenge as this would appear across the states in the U.S. today, building professional educator capacity and policymakers\u2019 understanding of assessment literacy is fundamental to shifting to personalized, competency-based systems at scale and focusing on equity.<\/span><\/p>\n

A common misconception about assessment literacy is that it is only about how to interpret standardized test results. In contrast, assessment literacy is a much broader and more significant concept. T<\/span>he New Zealand Ministry of Education defines assessment literacy as: <\/span><\/p>\n

\u201cthe possession of knowledge about the basic principles of sound assessment practice, including its terminology, the development and use of assessment methodologies and techniques, and familiarity with standards of quality in assessment. The primary purpose of assessment is to improve students\u2019 learning, as both student and teacher respond to the information that it provides. Information is needed about what knowledge, understanding, or skills students need. By finding out what students currently know, understand, and can do, any gap between the two can be made apparent. Assessment is the process of gaining information about the gap, and learning is about attempts to reduce the gap.\u201d <\/span><\/em><\/p>\n

Personalized, competency based learning requires us to reorganize systems around doing what it takes to ensure every student is attaining mastery, rather than the ranking and sorting them into high achievers and low achievers that is created through variable A-F grading practices. Redesigned systems will need to build capacity for clear evaluation criteria to make valid and reliable comparisons of students\u2019 progress against outcomes (commonly understood outcomes) using evidence and common rubrics. <\/span><\/p>\n

Thus, progress isn\u2019t measured by ranking and sorting kids against each other, or through grading \u201ccurves,\u201d but instead for each student to measure their evidence against articulated, high-level, common expectations of success and with clear depictions for what success looks like. This process of developing clear expectations for common proficiency levels is a key part of a \u201ccalibration.\u201d Calibration is a process that allows two or more things to be compared via a common standard (e.g., a weight in the physical sciences or commonly scored papers in an education system). The purpose of common performance tasks given to students by different schools and districts is to serve as a \u201ccalibration weight;\u201d a way to compare the way one school or district scores students on the common task, with the way other schools and districts score those same students\u2019 work. In order to use the common performance tasks as calibration weights, districts need to re-score other districts\u2019 common performance tasks. Calibrating expectations as well as grading and scoring processes for learning goals, is very important in competency-based learning systems. Calibration may involve groups of educators who collaborate and develop consensus around rubrics for scoring student work. The calibration process makes scoring student work consistent and more aligned to the standards upon which rubrics and scoring criteria are based, as well as creating reflective processes focused on improving student learning. <\/span><\/p>\n

In addition to calibration processes for consistently and accurately evaluating student work, assessment literacy also includes knowing which assessments are appropriate for what purpose (e.g., formative, progress monitoring, or summative). This idea of common expectations, and evaluating evidence against common standards and rubrics to build and evaluate comparability across schools and systems, requires careful \u00a0<\/span>moderation<\/span><\/i> of assessment practices across the system and perhaps across the state level. Professional development of educators to assess student evidence using calibration processes and developing rubrics with scales for evaluating performance tasks against criteria, is central to building the capacity needed in a competency-based education system. A competency-based learning system that offers personalized pathways for students to meet learning goals and learning targets must rely on multiple forms of evidence against common standards and expectations. \u00a0<\/span><\/p>\n

Tackling Assessment Literacy in Policy: Balanced Systems of Assessments<\/span><\/em><\/p>\n

Assessment is integral to the process of teaching and learning. Teachers should be constantly checking for their students\u2019 understanding in formal and informal ways. They are checking for understanding with formative assessment, tracking progress with interim assessment, and checking mastery of standards with summative assessment. And yet, \u201cassessment\u201d today in the United States is often used as a shorthand term for, or conflated to mean, \u201cstatewide accountability test.\u201d <\/span><\/p>\n

To be clear, though intricately linked to each other in today\u2019s policy context, accountability and assessment are two separate concepts. We should examine our approach to policy regarding assessment. We can be very clear about the need to measure student learning and growth in valid, reliable, and comparable ways, while also opening up new approaches to assessment that support, rather than disrupt, the learning process. Reflecting on where we are in the United States, if assessment is conflated with accountability today, it is because our policies have been structured to do just that. Counter to some of the narratives that are dominating policy conversations today, assessment and learning need not be at odds with each other. Policy can and should help to drive coherence of K-12 education systems by ensuring that assessment, teaching, and learning are complementary and supportive of one another.<\/span><\/p>\n

Differentiating Between Assessment and Accountability<\/span><\/h3>\n

It is common today in U.S. education policy to see the terms <\/span>assessment<\/span><\/i> and <\/span>accountability <\/span><\/i>used interchangeably. This conflates a broad set of tools that generate information about student learning (assessments) with policy initiatives designed to incent desired behaviors, or disincent undesired behaviors in order to reach specific goals (accountability). Of course, accountability and assessment are linked concepts, to the extent that assessment provides data that can be used for accountability. However, problems arise when the goals in the accountability system are too narrowly defined and the incentives or disincentives are too limiting or too punitive. NCLB tied a single assessment (end of year summative state tests) to multiple high stakes (identifying schools for intervention, diverting their federal funds into proscribed uses, and, with the changes brought about under the ESEA waivers and Race to the Top, teacher evaluations sometimes used to make human resources decisions). So, it makes sense that accountability and assessment get confused with each other. A critical shift in thinking needs to happen around accountability and assessment, starting with accountability systems based in multiple measures that move the focus from performance on a single test, to multiple measures aligned with the profile of a graduate, and accountability that balances incentives\/disincentives with supports.]<\/span><\/p>\n

To start, policymakers could begin to think about assessment in terms of <\/span>systems of assessments<\/span><\/i> that serve multiple purposes for multiple stakeholders, rather than in terms of a single assessment that is designed to be used solely for accountability and has the end result of driving teaching and learning toward limited outcomes. <\/span><\/p>\n

Chattergoon and Marion (2016)<\/span><\/a>3<\/sup> argue that as states redesign their approaches to assessment, they should pursue <\/span>balanced systems of assessments<\/span><\/i> that meet the following three criteria:<\/span><\/p>\n