Trying to Find a Measure for How Well Colleges Do

What information exists has often been hidden from public view. But that may be changing.

In the wake of the No Child Left Behind federal education law, students in elementary, middle and high schools take standardized tests whose results are made public, inviting anyone to assess, however imperfectly, a school’s performance. There is no comparable trove of public data for judging and comparing colleges.

Pieces of such a system may be taking shape, however, with several kinds of national assessments — including, most controversially, standardized tests — gaining traction in recent years. More than 1,000 colleges may be using at least one of them.

“There’s a real shift in attitudes under way,” said David C. Paris, executive director of the New Leadership Alliance for Student Learning and Accountability, a coalition of higher education groups. “We used to hear a lot more of, ‘The value of college can’t be measured,’ and now we hear more of, ‘Let’s talk about how we can measure.’ ”

In January, the New Leadership Alliance released guidelines calling on colleges to systematically “gather evidence of student learning” — though not explicitly advocating standardized tests — and release the results. The report was endorsed by several major organizations of colleges and universities.

Advocates say the point is not to measure how each college’s students perform after four years, which depends heavily on the caliber of students it enrolls in the first place, but to see how much they improve along the way. The concern is less about measuring knowledge of chemistry or literature than about harder to define skills like critical thinking and problem-solving.

That vision still faces major obstacles. Colleges that use standardized tests vary widely in what they test, how and when. And many of them that use those tests or national surveys keep the results to themselves.

“I’d love for all the data to be public,” said Jennifer Carney, director of program evaluation at the Jack Kent Cooke Foundation, which conducts education research. But, she added, that would inevitably lead to some colleges manipulating the figures in pursuit of a higher standing, just as some have done with existing ranking systems.

In the best-known college rankings, by U.S. News & World Report, up to 40 percent of a college’s score is based on its reputation among educators and its selectivity in admitting students. Other factors include several indirect indicators of what happens in classrooms, like student retention, graduation rates and class sizes, but no direct measures of learning.

Critics of standardized tests say they are too narrow and simplistic.

“I’m not sure any standardized test can effectively measure what students gain in problem-solving, or the ability to work collaboratively,” said Alice P. Gast, president of Lehigh University.

In 2008, the Consortium on Financing Higher Education, a group of some of the nation’s most prestigious colleges and universities — including all of the Ivy League — issued a lengthy manifesto saying that what its students learn becomes evident over decades and warning against a “focus on what is easily measured.”

Many of those same colleges participate in the National Survey of Student Engagement, which measures many factors that educators say are good, though indirect, indicators of learning, like how many hours students spend studying and how much they interact with professors. The survey, which began with a handful of colleges in 2000, had more than 700 participate last year. Each college can see its own results and those of a group of comparable colleges, but the scores are not made public.

The view from state-supported colleges has been shaped in part by pressure from policy makers to show what taxpayer dollars are accomplishing, through standardized tests. Texas, a leader in the movement, has required its state colleges to administer tests since 2004, and it makes the outcomes public.