What are the most credible rankings of US computer science departments?

Question

I am wondering what are the most well-regarded rankings of computer science departments in the US. Preferably, this would be broken down into rankings by research output and rankings by student labor market outcomes.

Surely, one can find a myriad of such rankings by google searching, but I am wondering whether there are some common rankings that are referred to as standards by academics and practitioners. Thank you kindly.

EDIT:

Let me clarify. Most answers below answered the question "are academic rankings useful/unbiased/etc." but this is not at all what I was asking. I know that such measures are noisy and subjective. This is well-known. But in every profession there are some rankings that are well-regarded among other researchers in that profession.

Please, I know it's hard, but try to stop yourself from answering why rankings suck, even if you feel very strongly about it.

Answer 1

Very few computer scientists (or academics in general) think highly of any ranking system. The main reason for this is that rankings are pretty arbitrary, highly biased by personal opinion, and not super informative. There's very minimal feedback pressure on rankers to get it "really right," and there's no particular reason to think that they do.

The methodologies vary widely, and small differences can result in wildly different rankings because the gradations between schools is usually not very big. In the US News and World Report Ranking, five schools all got a perfect score. Their methodology is to solicit rankings from researchers on a scale of 1-5 and then average them. So there were five schools that everyone gave top marks to. But there were also 17 schools that received higher than a 4.0 without receiving a 5.0. Due to their methodology, that means that there are at least 17 schools for which at least half of respondents said it was a 5/5. It seems unambiguous that all of those 17 schools are outstanding, because at least half of researchers gave them a 5/5. However, it likewise is very unclear what a 4.1 vs a 4.5 actually corresponds to.

This also shows how much the presentation of the results matter. The following three statements are all true of the USNWR ranking:

1. UIUC is ranked 4 slots behind CMU.
2. UIUC's ranking is five times that of CMU.
3. UIUC's raw score is 11% lower than that of CMU
4. UIUC is a top 3.3% school and CMU is a top 2.8% school.

Other sites use a weighted system that scores universities on a variety of factors and then take the weighted average. The issue with this methodology is that the results are highly variable with differing weights, and there doesn't seem to be any principled way to decide if "mean impact factor of faculty" should be weighted 0.3 or 0.1. Even choosing to measure "mean impact factor of faculty" could be disputed, and one could use median instead of mean or only look at the 5 most active professors. There are arguments for and against myriad tweaks like this, and again there seems to be no principled way to decide which is best. This wouldn't be very concerning if it mattered little, but it matters a lot. This is easily seen by looking at 5 random rankings and observing that there's a very high variance in the ranking of universities from site to site. This issue is exacerbated by the fact that universities try to game ranking systems (note there are two links there).

How your school is ranked can also vary massively with the discipline or sub-discipline. My alma mater had a fabulous CS theory program, a less-reputed systems program, and (at the time) a non-existent AI program. Someone who says "I studied theoretical computer science at Stella's alma mater" gets a very different response than someone who says "I studied programming language theory."

Not only is your discipline going to matter a lot, who your adviser is will as well. Sometimes the word experts on fields or subfields work at universities that are, in general, not thought as high of. But if you can go study the problem you're interested in with the world expert on it, you should jump at that chance, even if it means going to some podunk school you'd have never heard of if your adviser wasn't located there. And these kinds of concerns are entirely ignored by rankings. Some rankings have breakdowns by sub-discipline, but there's no real way to get them nuanced enough to be particularly meaningful.

University rankings are also comically America-centric, and actual perception of the quality of universities widely varies by region, both within the US and across the world.

Finally, university ranking doesn't matter that much compared to other factors.

Answer 2

There are many rankings available. For example, five minutes of Googling turns up the following.

1. http://csrankings.org/
2. https://www.usnews.com/best-graduate-schools/top-science-schools/computer-science-rankings and https://www.usnews.com/education/best-global-universities/computer-science
3. https://www.topuniversities.com/university-rankings-articles/university-subject-rankings/top-computer-science-schools-2017
4. https://www.timeshighereducation.com/world-university-rankings/2018/subject-ranking/computer-science#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/stats
5. https://lucatrevisan.wordpress.com/2014/10/23/an-alternative-to-the-seddighin-hajiaghayi-ranking-methodology/

You can see that they differ quite a bit. This is because such rankings are highly subjective. The first one listed even lets you adjust the ranking by counting or discounting different areas.

The rankings pretend to be objective. But the truth is that there is a lot of freedom exercised in how to weigh different factors. You can achieve drastically different rankings by tweaking your criteria slightly. (Weighting publications by venue. Comparing departments of differing size. etc.) I strongly suspect that the people who collate rankings will tweak their methodology until it "looks right" which means objectivity flys out the window.

This is why academics generally scoff at rankings (and mock them). Generally when I look through a proposed ranking my internal monologue goes as follows. "OK the first three seem reasonable. I don't think the fourth and sixth ones should be so high. Hey, wait, where is my alma mater? That's nonsense. This is garbage." Likewise, I think most academics quickly find something they strongly disagree with in any proposed ranking. All rankings are essentially nonsense, particularly outside the top 10 or so (which will have been tweaked to look reasonable).

The computing research association has a statement expressing how most academics view rankings: https://cra.org/cra-statement-us-news-world-report-rankings-computer-science-universities/

Unfortunately, some people do take rankings seriously (and thereby embarrass themselves). For example, http://www.chinadaily.com.cn/world/2017-10/30/content_33891752.htm and https://www.cs.utexas.edu/news/2017/ut-austin-ranked-top-computer-science-university-us-second-best-globally and https://www.umdrightnow.umd.edu/university-maryland-rankings

If you are interested in rankings, the first question you should ask is what you actually want to know. If you have a more specific question (e.g., where should I go to grad school?), then you can look at more specific -- and useful -- information. Unfortunately, this is probably going to be more work than just Googling for a list.

Answer 3

I understand why the other answers are doubling down on the "rankings are nonsense" stance, but to play devil's advocate, I'd be surprised if the US News Rankings differ very much (on average) from a hypothetical ranking based on which departments are most often recommended to students who are applying for grad school.

We shouldn't take rankings too literally, but they can be useful for finding out roughly what tier an unfamiliar department is in (which anyone who's cast a wide net in an academic job search has had to do). If department A is ranked in the top 30 and department B is in the 70s, there's a reason. One's research into the matter shouldn't end there, but the information is not meaningless.