Capturing quality… whatever that is

Adding to the slew of academic rankings, the Times Higher Education released its own “state-of-the-art” ranking of world institutes of higher education. Harvard is #1 and seven out of the top 10 universities in the world are American, which shouldn’t be a surprise to those who have given more than a cursory glance to other ranking schemes.

This is more than just sour grapes for those who are unranked, or are ranked lower than they believe is realistic; there are very real political and social consequences to reducing the diversity of universities and colleges available in a given country to those who conform to a particular vision of “quality” (i.e. the Harvard model), especially when this ranking is marketed as a decision-making tool for students and parents who may not be well versed with the breadth of educational diversity. As Malcolm Gladwell notes in a recent New Yorker piece titled “The Order of Things”, rankings must deal with an inherent theoretical tension in their construction:

A ranking can be heterogenous, in other words, as long as it doesn’t try to be too comprehensive. And it can be comprehensive as long as it doesn’t try to measure things that are heterogenous.

One can also substitute “incommensurable” for heterogenous; favoring heterogeneity would mean favoring greater diversity of things represented precisely because they are incommensurable in a significant way. In detailing the US News and World Report rankings of universities as an example, he adds that

It aims to compare Penn State – a very large, public, land-grant university with a low tuition and an economically diverse student body, set in a rural valley in central Pennsylvania and famous for its football team – with Yeshiva University, a small, expensive, private Jewish university whose undergraduate program is set on two campuses in Manhattan  (one in midtown, for the women, the other in far uptown, for the men) and is definitely not famous for its football team.

What Gladwell neatly illustrates is that the question of rankings is not necessarily one about methodology. A ranking always sets out an “ideal” according to specific values and ideologies (a normative endeavor), against which all others are compared. Are we really willing to say that all colleges and universities ought to be exactly like Harvard? The THE rankings may be the latest thing in rigor, but there’s no novelty in exactly how the ranking itself is overreaching to be both comprehensive and heterogenous – an impossibility without sacrificing key elements of what we think to be educational or research “quality”.

This entry was posted in Metrics, Public Philosophizing. Bookmark the permalink.

One Response to Capturing quality… whatever that is

  1. Britt Holbrook says:

    Well said, Kelli! I would rank this as near the top of your posts.

    Seriously, though — very good points about methodology and rigor blinding us to the obvious: rankings are valuations!

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>