CCE staff and partner reflections on our collaborative work to create schools where learning is engaging and rewarding, and every student is set up for success.
Perseverance: The Craft in Calibrating the Scoring of 21st Century Skills
It is a cool spring afternoon at Uptown High School (a composite drawn from schools with whom the authors have worked). At the end of the day, groups of teachers are looking at student work - a school-wide task in which they are, for the first time, measuring student mastery of 21st century skills - and are using a calibration process to develop common understandings about these skills and what proficiency in demonstrating these skills looks like.
All 11th grade students have completed their major term project. Student teams presented a business plan, social media campaign, and 3-D computer renderings of a new product/service to a paneled jury of academic and vocational teachers, business owners, bankers, and community members. Now the hard work begins: evaluating student performance based on a community-created, school-wide rubric addressing the relevant 21st century skills.
According to the Glossary of Education Reform, “21st century skills refer to a broad set of knowledge, skills, work habits, and character traits that are believed … to be critically important to success in today’s world. Specifically, 21st century skills focus on learning skills (e.g., critical thinking, literacy skills (e.g., technology literacy), and life skills (e.g., perseverance).
In most of the United States, 21st century skills are not new. However, implementation – bringing the plan to the classroom and the students – has been a challenge. A common issue is the lack of consistency in how teachers define and measure student attainment of these skills. While a particular teacher may have a clear idea of what “perseverance” or “critical thinking” or “initiative” look like as values or qualities of character, that teacher’s sense of these same qualities as learning targets is imperfect. These conversations can become emotionally charged as teachers wrestle with diversity in their opinions.
This is a challenge that can only be addressed through teacher discourse. Teachers explore the instructional implications of the task, student work, and rubric together through calibrating the scoring of student work. Through calibration, teachers engage in a structured conversation, coming to agreement about what evidence of proficiency can be found in student work. Calibration safeguards that student work is assessed consistently and in alignment with the rubric.
Calibration protocols help teachers establish guidelines for these conversations. Protocols take educators through a multi-step process of 1) examining student instructions and the accompanying scoring rubric, 2) reading and scoring the sample piece of student work, 3) sharing their individual scoring, and 4) having a discussion on their respective findings. Most often the focus of the conversation is on the areas where there are differences in scoring. It is not uncommon to have one teacher in a calibration session score a particular piece as a 1 (Beginning) while another scores it as 3 (Proficient) on the scoring. Differences in scoring are the spaces where most learning occurs among educators. Educators check their understanding around the criteria they are assessing and engage in deep discussion with their colleagues about what student mastery of the desired skills and knowledge looks like.
At Uptown High, the decision to create an 11th grade team project was made by the entire faculty after viewing the student vigor generated from the 12th grade culminating senior project. Prior to the introduction of the 11th grade team project, morale was low among the senior class. Half the class did not have any postsecondary plans in place by May of their senior year. There was widespread belief among the faculty that something was not working in their curriculum. They spent a full year thinking through possible options, visiting other schools to see how they addressed comparable issues. Through their investigation, they saw schools integrating 21st century skills explicitly being assessed through performance assessments.
As Uptown faculty observed these schools, they saw students apply their classroom learning to real-world issues. Teachers were impressed with students’ excitement and creativity in transferring what they learned in class to unfamiliar, real life challenges. Performance assessments provided an authentic opportunity for students to create or refine an original product). Additionally, faculty had an opportunity to attend calibration sessions held at the other schools and witnessed their counterparts’ enthusiasm in comparing notes on the student learning. From these experiences, the 11th grade project was created and a calibration process implemented.
Click the above image to see an example of how a calibration conversation can play out between teachers.
Calibration has become a central part of Uptown High School’s culture. They, like many schools across the country, value the opportunity and support in having the difficult, professional conversations. It has enabled them to engage in a conversation that deepens their common understanding of expectations regarding student performance. At the same time, it also created opportunities to push teachers’ thinking about how their instruction, curriculum and assessment helps students to cultivate 21st century skills that successfully enable them to handle challenges beyond the classroom.