If we assume (counterfactually) a completely meritocratic system, then the best (= most talented) high school students go to the most prestigious (= highest quality) universities for the first degree, the best go on to the doctorate at the highest quality universities, so that looking at the Ph.D. granting university would give a rather clear ranking. In this fictional scenario, there would even be some error correction, because of the relative ranking of students.
One can argue that the real world is not that_far from this ideal scenario.
A more down-to-earth argument looks at the decision process of the selection committee. There used to be a slogan "No-one ever got fired for buying from IBM". Replace IBM with Cal Berkeley or MIT, and take in addition the fact that the successful candidates at a famous university needs to have good publications and good letters, you can see that hiring candidates from the prestigious universities is much less of a risk.
Another argument is that it is easier to get a good education if you are surrounded by other good students. At one of the best universities in any large country, the Ph.D. students would be able to educate themselves if suddenly all faculty were to vanish.
This means that exceptional people from humdrum places will have it harder to get a really good job in academia. But they have second chances.