Don’t double down on one-size-fits-all measurement

A surefire way to get heads nodding at an education policy conference is to call for dismantling the Carnegie unit.

The question, and a legitimate fear, is whether efforts to replace this flawed measure will lead to something worse.

The Carnegie unit was conceived a century ago by some of the leading lights of American education. They needed a clear way to signal whether students had completed a full course. So they settled on a measure: 120 hours of classroom time.

The fruits of this effort have a major impact on public education. College “credit hours” and “instructional minutes” requirements for K-12 schools derive from the Carnegie unit. Schools receive funding based on the number of units they deliver. Students earn course credits and degrees based on the number of units they complete.

The Carnegie unit remains central to education policy and practice despite its flaws, including the fact that a student can complete a full course without actually mastering the material.

This has spawned a number of efforts to replace the Carnegie unit and related measures of “seat time” with new systems of focused on mastery.

The problem with the Carnegie Foundation’s new effort to replace the flawed measure that bears its name is that it doesn’t stop at the idea of replacing seat time with mastery. Instead, it involves efforts to devise new measures of student progress for a wider array of domains. As Max Eden of the American Enterprise Institute notes, there’s a real risk that these new measures will amplify the same one-size-fits-all thinking that shaped the original 20th century effort:

[T]hey want to evaluate the “whole child,” meaning broader human skills such as empathy, communication ability, leadership, and critical thinking. Things that, they reasonably note, parents want and employers demand. The only problem with this approach is that it’s practically and politically impossible and would likely prove counter-productive even if it weren’t.

If teachers want to evaluate math skills, the method is straightforward enough: give the student a math test and see if he gets the right answer. But how could a teacher evaluate 30 students accurately on domains like empathy and leadership? She couldn’t do it herself. So, Carnegie is throwing money at ETS to develop what its CEO Timothy Knowles calls “stealth assessments” for those domains. Color me skeptical that it’s possible to develop psychometrically reliable metrics for such soft skills at all. But if it were possible, it could only be done by AI deeply datamining student behavior.

Zooming out, this is a problem that surfaces in all kinds of debates about public education as we move beyond the industrial era. The Carnegie unit was designed to quantify teaching and learning with a standardized unit that schools and colleges all over the country could easily interpret. It proved so valuable to the batch processing of students that it became ubiquitous.

Now, education systems across the country are trying to create more individualized learning pathways for students that include online instruction, career and technical education, learning outside the classroom, and other approaches that confound assumptions that students should advance based on the time they’ve spent in class.

States like Wyoming, Kentucky and Vermont are working on approaches that fund schools or give students credit based on the knowledge and skills students demonstrate. A relatively new Arizona law gave public schools flexibility over how their students spend up to half of their learning time. This has helped unleash a wave of new learning environments that challenge conventions about what schooling can look like.

A few simple principles are at work. Learning can happen anytime, anywhere. And students can show what they know by completing a task or passing a test. It no longer makes sense for schooling to revolve around Carnegie units or other measures of “seat time.”

Freeing schools from industrial-era assumptions will help move a school system built to process students in batches to one that embraces their individuality. This transition is bound to be uneven and messy. Jal Mehta of Harvard University has described the work ahead like this:

If we are trying to create a world where students have more choice and flexibility, we are unlikely to do that by replacing the Carnegie Unit with some other single set of measures of performance competency. Instead, we should seek to diversify both how we count learning and what kinds of learning count and do so with the spirit of humility that a task of this magnitude requires.

This seems wise.

It would be a mistake to double down on a flawed industrial-era assumption—that learning can readily be distilled into tidy measures that are legible across diverse institutions—and extend that flawed assumptions into so-called 21st century skills like leadership or collaboration. Instead, we need to make peace with the idea that different families and educators will approach education with different goals and value systems. They will need diverse measurement systems to match.


Avatar photo

BY Travis Pillow

Travis Pillow is Director of Thought Leadership at Step Up For Students and editor of NextSteps. He lives in Sanford, Fla. with his wife and two children. A former Tallahassee statehouse reporter, he most recently worked at the Center on Reinventing Public Education, a research organization at Arizona State University, where he studied community-led learning innovation and school systems' responses to the Covid-19 pandemic. He can be reached at tpillow (at) sufs.org.