Each year numerous organizations publish quality rankings for institutions of higher education. Competition is fierce for the top spots, as they mean bragging rights, a better talent pool, and increased revenue. But can the various methodologies used actually result in unbiased and accurate comparisons between vastly different institutions?
Debate about the validity of college rankings has simmered for years in academic circles, and recently bubbled to the surface of the mainstream media. The US News and World Report rankings were for decades the standard metric by which every college and university measured itself. But institutions have figured out how to manipulate the results. Researchers from Stanford’s Graduate School of Education found that, “Many of the metrics used in these rankings are weighted arbitrarily and are not accurate indicators of a college’s quality or positive outcomes for students.”
The first problem with college rankings is that much of the data is self-reported. In 2012 Emory University admitted that it had knowingly provided false data for at least a decade. Universities can manipulate their numbers to appear more selective by, for instance, delaying the admission of students with low test scores until the spring. Even worse, the 20% of the US News score based on “academic reputation” is supposed to be determined by a nationwide survey of university presidents. But most admitted they delegate the chore to a staffer, who probably isn’t familiar with the hundreds of universities being ranked. Some even admitted to downgrading rankings of their peers to make their own institutions look better.
Michael Nietzel, former president of Missouri State, details other flaws with rankings in this article in Forbes. He concludes that, “It is too easy for schools to game the system and falsify data.” The vast difference in ranking methodology is illustrated by this year’s results for Quinnipiac University, which suffered an inexplicable 45-point drop in one ranking, but went up to a new higher level in the US News rankings.
For all these reasons we name college rankings our Measurement Menace of the Month. And for the same reasons we urge you to use caution before you add “college rank in US News and World Report” to your comms dashboard. If your leadership insists on using college rankings, then I suggest you use a FiveThirtyEight-style average of several rankings. The better to avoid the potential shortcomings of any single one. Happy Fall Semester! ∞
