Our Blog

CCE staff and partner reflections on our collaborative work to create schools where learning is engaging and rewarding, and every student is set up for success.

Perseverance: The Craft in Calibrating the Scoring of 21st Century Skills

It is a cool spring afternoon at Uptown High School (a composite drawn from schools with whom the authors have worked). At the end of the day, groups of teachers are looking at student work - a school-wide task in which they are, for the first time, measuring student mastery of 21st century skills - and are using a calibration process to develop common understandings about these skills and what proficiency in demonstrating these skills looks like. 

All 11th grade students have completed their major term project. Student teams presented a business plan, social media campaign, and 3-D computer renderings of a new product/service to a paneled jury of academic and vocational teachers, business owners, bankers, and community members. Now the hard work begins: evaluating student performance based on a community-created, school-wide rubric addressing the relevant 21st century skills. 

According to the Glossary of Education Reform, “21st century skills refer to a broad set of knowledge, skills, work habits, and character traits that are believed … to be critically important to success in today’s world. Specifically, 21st century skills focus on learning skills (e.g., critical thinking, literacy skills (e.g., technology literacy), and life skills (e.g., perseverance). 

In most of the United States, 21st century skills are not new. However, implementation – bringing the plan to the classroom and the students – has been a challenge. A common issue is the lack of consistency in how teachers define and measure student attainment of these skills. While a particular teacher may have a clear idea of what “perseverance” or “critical thinking” or “initiative” look like as values or qualities of character, that teacher’s sense of these same qualities as learning targets is imperfect.  These conversations can become emotionally charged as teachers wrestle with diversity in their opinions. 

This is a challenge that can only be addressed through teacher discourse. Teachers explore the instructional implications of the task, student work, and rubric together through calibrating the scoring of student work.  Through calibration, teachers engage in a structured conversation, coming to agreement about what evidence of proficiency can be found in student work. Calibration safeguards that student work is assessed consistently and in alignment with the rubric. 

Calibration protocols help teachers establish guidelines for these conversations. Protocols take educators through a multi-step process of 1) examining student instructions and the accompanying scoring rubric, 2) reading and scoring the sample piece of student work, 3) sharing their individual scoring, and 4) having a discussion on their respective findings. Most often the focus of the conversation is on the areas where there are differences in scoring.  It is not uncommon to have one teacher in a calibration session score a particular piece as a 1 (Beginning) while another scores it as 3 (Proficient) on the scoring. Differences in scoring are the spaces where most learning occurs among educators. Educators check their understanding around the criteria they are assessing and engage in deep discussion with their colleagues about what student mastery of the desired skills and knowledge looks like. 

At Uptown High, the decision to create an 11th grade team project was made by the entire faculty after viewing the student vigor generated from the 12th grade culminating senior project.  Prior to the introduction of the 11th grade team project, morale was low among the senior class. Half the class did not have any postsecondary plans in place by May of their senior year. There was widespread belief among the faculty that something was not working in their curriculum. They spent a full year thinking through possible options, visiting other schools to see how they addressed comparable issues. Through their investigation, they saw schools integrating 21st century skills explicitly being assessed through performance assessments. 

As Uptown faculty observed these schools, they saw students apply their classroom learning to real-world issues. Teachers were impressed with students’ excitement and creativity in transferring what they learned in class to unfamiliar, real life challenges. Performance assessments provided an authentic opportunity for students to create or refine an original product. Additionally, faculty had an opportunity to attend calibration sessions held at the other schools and witnessed their counterparts’ enthusiasm in comparing notes on the student learning. From these experiences, the 11th grade project was created and a calibration process implemented.

Click the above image to see an example of how a calibration conversation can play out between teachers. 

Calibration has become a central part of Uptown High School’s culture. They, like many schools across the country, value the opportunity and support in having the difficult, professional conversations. It has enabled them to engage in a conversation that deepens their common understanding of expectations regarding student performance. At the same time, it also created opportunities to push teachers’ thinking about how their instruction, curriculum, and assessment helps students to cultivate 21st century skills that successfully enable them to handle challenges beyond the classroom. 

Featured

Blog Post

Placing Teachers at the Center of Assessment

March 13, 2017
The Massachusetts Consortium for Innovative Education Assessment seeks to redefine the measures of student learning. The consortium believes that “In this day and age, we need to re-conceptualize assessment rather than tinker to refine a testing model that has limited value in furthering public education,” and that the best way to assess student learning is to return the role of designing assessments to those closest to students – teachers.
Blog Post

A Reflection on the Big Picture: Bridging QPA Practice and Systems

February 1, 2018
Over these five years, we've deepened our understanding of what is necessary for the successful implementation of performance assessment and QPA systems.
Blog Post

Bringing Assessments to Life with Rapid Prototyping

May 2, 2018
QPA's Rapid Prototyping Process helps teachers produce performance assessments when time is precious, but it can also help students and the community get in on the performance assessment design process.
comments powered by Disqus