[HN Gopher] U.S. News pulls Columbia University from its 2022 ra...
___________________________________________________________________
U.S. News pulls Columbia University from its 2022 rankings
Author : selimthegrim
Score : 228 points
Date : 2022-07-09 12:12 UTC (2 days ago)
(HTM) web link (www.forbes.com)
(TXT) w3m dump (www.forbes.com)
| crikeyjoe wrote:
| sam-2727 wrote:
| The beginning of the conclusion of the original study [1] is
| worth repeating:
|
| No one should try to reform or rehabilitate the ranking. It is
| irredeemable. In Colin Diver's memorable formulation, "Trying to
| rank institutions of higher education is a little like trying to
| rank religions or philosophies. The entire enterprise is flawed,
| not only in detail but also in conception."
|
| Students are poorly served by rankings. To be sure, they need
| information when applying to colleges, but rankings provide the
| wrong information. As many critics have observed, every student
| has distinctive needs, and what universities offer is far too
| complex to be projected to a single parameter. These observations
| may partly reflect the view that the goal of education should be
| self-discovery and self-fashioning as much as vocational
| training. Even those who dismiss this view as airy and
| impractical, however, must acknowledge that any ranking is a
| composite of factors, not all of which pertain to everyone. A
| prospective engineering student who chooses the 46th-ranked
| school over the 47th, for example, would be making a mistake if
| the advantage of the 46th school is its smaller average class
| sizes. For small average class sizes are typically the result of
| offering more upper-level courses in the arts and humanities,
| which our engineering student likely will not take at all.
|
| [1]:
| http://www.math.columbia.edu/~thaddeus/ranking/investigation...
| (section 8)
| cuteboy19 wrote:
| This is just the wrong way to look at it. Clearly, there is
| real demand for rankings by students. No one is stupid enough
| to think that there is some real difference between #47 and
| #48. But obviously #47 is very different from #26.
|
| Just because you can't get an exact measurement does not mean
| that a metric does not exist or is not useful.
| lazyjeff wrote:
| I've been looking at the bias in rankings for a little while. I
| think one way to identify and raise awareness of the biases, is
| just put rankings together side-by-side. I did this for
| computer science programs, and there's some interesting
| differences that I noticed:
|
| https://jeffhuang.com/computer-science-open-data/#bias-in-co...
| function_seven wrote:
| In the same way that one restaurant flouted Yelp ratings[1],
| couldn't all the Ivy League schools just refuse to participate in
| US News' annual rankings? What would happen if Harvard, Yale,
| Princeton, Columbia, et. al. decided that this ranking is not
| helpful, so they won't supply any info?
|
| Readers of US News would more likely start to lose trust in the
| rankings rather than move their assessments of the Ivies
| downward, right?
|
| [1] https://arstechnica.com/information-
| technology/2014/09/why-t...
| ketzo wrote:
| But it's a prisoner's dilemma situation, right?
|
| If _none of_ the Ivy's are on US News, well, it's probably a
| shitty ranking.
|
| But if e.g. _just Penn_ falls off the list... most people are
| just gonna assume that Penn got worse.
| lbarrow wrote:
| These rankings consistently tell people that Harvard, Yale,
| Princeton, etc are the best schools in the country. Why would
| they boycott rankings that praise them?
| genericone wrote:
| Because it puts other 'less desirable' non-elite schools in
| the same lists for top twenty/thirty/forty, etc.
| CobrastanJorji wrote:
| The Ivies rank 1, (UNLISTED), 2, 5, 8, 13, 14, and 17. All
| of the "less desirable, non-elite schools" are listed BELOW
| these schools, which keeps them looking good.
| kupopuffs wrote:
| Is that even really a negative?
| bee_rider wrote:
| Yeah they should stick all the ivies, MIT, etc... in a top
| "non-rankable" rank. Like realistically a student will be happy
| to go to whichever one they get into, and if someone gets into
| multiple, they probably won't pick based on position in some
| list.
| ketzo wrote:
| > if someone gets into multiple, they probably won't pick
| based on position in some list
|
| Not trying to be contrarian, but I had two friends literally
| pick between Ivies based on this very list (in 2015).
|
| It's generally accepted as "The List" by a lot of people. If
| you're a parent with no other frame of reference, The List
| has a serious impact.
| buzzy_hacker wrote:
| I'll chime in with contrary anecdotal experience having
| gone to a high school with lots of selective college
| placements. I didn't know anyone who chose between two
| elite schools based on their relative US News ranking.
| colinmhayes wrote:
| I find this hard to believe. Obviously everyone had more
| important considerations, but I know a few people that
| got full rides to a few top schools and absolutely chose
| based off rankings.
| whimsicalism wrote:
| These rankings absolutely influence people's decisions.
| fleetwoodsnack wrote:
| Mine neither. Maybe it's because they're the prep school
| set, but what turned out to be the tie breaker for my
| multi-Ivy admit friends was the campus visit. I remember
| one who was adamant about getting in shape in college and
| chose the one where the freshman dorms were closest to
| the campus recreation building.
|
| If you're of a certain background, it's really your mom
| or dad's alma mater, and then one of a few "perfectly
| acceptable, fine schools."
|
| I think the difference between say #4 and #12 in a given
| year exists in the minds of middle class strivers. I went
| to a public university and a private one for
| undergraduate and graduate school, and at the top levels
| it really comes down to the professors on an individual
| level and perhaps the department, more than the
| institution itself.
| bee_rider wrote:
| Well, that's more direct experience than I have, at least.
|
| In any case, I bet Columbia will keep getting more really
| high quality applicants than they have seats.
| wil421 wrote:
| What happens if your pick drops in ranking by the time you
| graduate?
| perpetualpatzer wrote:
| Not much. Most peers and hiring managers stopped paying
| attention to these ratings when _they_ got into college,
| so the market 's perception of your degree is some
| blended average of the programs' ratings over the past
| ~40 years. By the time rankings after you matriculated
| represent a meaningful portion of the average, your alma
| mater is no longer a particularly relevant part of your
| resume.
| oneplane wrote:
| Wouldn't it make more sense to just make sure that all
| universities just pass the same requirements to be a university,
| and beyond that not add some ranking or other entertainment
| nonsense in the mix?
|
| This whole entertainifying business really isn't good for anyone,
| except perhaps for people making money off of numbers, which is
| still not good for most.
| twblalock wrote:
| The US News rankings were already a bit of a mess, but leaving
| out one of the Ivy Leagues is only going to be bad for US News.
|
| An Ivy League school does not need help from the rankings -- but
| rankings that leave out one of the most prestigious and well-
| known universities in the country are useless.
| prepend wrote:
| > An Ivy League school does not need help from the rankings
|
| I tend to agree, but why then did Columbia fudge the numbers?
|
| I think there's some envy from the "lesser ivys" that they wish
| they were Harvard or Princeton. A high USNews ranking proves
| little, but I guess Columbia wanted to be tied with Harvard.
| rr888 wrote:
| Ivy league is kinda obsolete now, esp with places like
| Stanford, Duke, John Hopkins better known than Dartmouth,
| Brown, Cornell.
| genericone wrote:
| Columbia incentivized administration MBAs to increase the
| rankings, perhaps attached to bonuses. They did it
| unscrupulously by fudging the numbers and so Columbia will
| hopefully will dis-incentivize the behavior in the future,
| and so perhaps will other schools... doubt it.
| bee_rider wrote:
| I wonder how much these measurements actually matter, for a place
| like Columbia. I mean, I can see why, like, people might want to
| look at the list and see "hey this school I've never heard of is
| actually OK."
|
| But like, people don't find out about Columbia from this list,
| right? It is, uh, older than the country "U.S. News" is named
| after and an Ivy League school. Not having them on the list just
| reflects poorly on the list.
| rel2thr wrote:
| Not really , Knowing about Columbia is kind of a new york,
| upper class thing
|
| They don't have good sports teams, no way a person in middle
| america would have heard of them if not for school rankings
| ladberg wrote:
| I absolutely know people who have had the choice of multiple
| top-tier schools and chose the one that ranked higher on U.S.
| News, so it definitely matters.
| zactato wrote:
| Why?
|
| MIT vs Harvard vs Stanford are all amazing programs but are
| better for different types of people/interests/goals
| willhinsa wrote:
| On the other hand, Coca-Cola spends 4 billion of dollars every
| year on advertising.
| toast0 wrote:
| In 2010, Pepsi switched at least their superb owl advertising
| budget to feel good community projects and lost sales at 6%
| vs a 4.3% overall decline for soda. Advertising drives sales,
| even if you can't tie a sale directly to an ad.
| bee_rider wrote:
| I dunno, Pepsi always seemed more faddish anyway, I'm not
| surprised their marketshare would be less durable.
| bee_rider wrote:
| Yeah.
|
| I dunno, nothing about advertising makes sense to me, so I
| guess I have trouble evaluating this kind of thing.
|
| Common sense would tell you that you don't give 4 billion
| dollars to the guy who says "I'm really good at convincing
| people to part with their money -- whether they should or
| not!" But then I don't run Coca Cola, so I guess they've
| figured out something I've missed.
| peanuty1 wrote:
| I imagine there is value in being in the same company of more
| prestigious schools like Harvard and Princeton.
| throwawayzXwEt wrote:
| These rankings are rampant with manipulation. For instance, I
| heard of classes at Hopkins being taught half in-class, half pre-
| recorded, so that they could say the "classroom" size was half
| its actual enrollment. It's especially frustrating when this kind
| of manipulation actually hurts students' education.
| endisneigh wrote:
| Unless a school is willing to accept a entirely random set of
| students all schools success or lack there of can be chalked up
| to selection and survivor bias.
|
| I'd have to find it but someone did an analysis of folks who got
| into elite schools and didn't go for whatever reason and saw that
| by mid career they had more or less the same outcomes.
|
| Thiel Fellows entire premise is around this effect, as well.
| labster wrote:
| One down, 99 to go. These university rankings are absurd, and
| just a status game that school administrators feel they are
| forced to play. Individual programs may be outstanding at
| otherwise middling colleges -- it's better to learn about
| specific departments than try to reduce the school as a whole to
| a number.
| kupopuffs wrote:
| Aren't there rankings for specific schools in schools? like
| Engineering, etc...
| labster wrote:
| Sure, but excellence in computer engineering does not imply
| excellence in chemical engineering does not imply excellence
| in civil engineering.
| sandstrom wrote:
| But someone looking for a good chemical engineering school
| won't look in the computer science ranking table, no?
| unosama wrote:
| Universities are a status game in general. Reputation and
| networking opportunities matter more than education quality.
| culi wrote:
| One of the heaviest part of their grading is just the opinions
| of academics. They just send a survey out and ask academics to
| rank universities. Of course they end up reproducing and
| reinforcing the existing rankings as always.
| aiisjustanif wrote:
| This has been brought up multiple time now over that last decade.
| At least this is one step in the right direction.
|
| Adam Ruins Everything, Season 2, Episode 7 [1] 1:
| https://youtu.be/EtQyO93DO-Q
| willcipriano wrote:
| If colleges manipulate rankings like this en mass I think we have
| a solution to the student debt issue.
|
| A FTC review of college advertising would be interesting, if they
| are selling you a degree with basically no career prospects while
| proudly displaying these gamed rankings, that sounds like fraud
| and misrepresentation. Perhaps the students can get a refund?
| foolfoolz wrote:
| a huge shift over the last 10 year and most visible in technical
| roles is the elimination of college rankings. i used to work at a
| company that had a ceo mandate "all resumes must come from top 20
| schools". saying something like that now as an engineering
| manager would raise questions and probably make you look dumb.
| there's many reasons for this but the biggest is it's impossible
| to do this with a global workforce. this is great for high school
| students, one less area of pressure
| dang wrote:
| Related:
|
| _An Investigation of the Facts Behind Columbia 's U.S. News
| Ranking_ - https://news.ycombinator.com/item?id=30603287 - March
| 2022 (46 comments)
| somethoughts wrote:
| Somewhat related - How Northeastern Gamed the College Rankings
|
| https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
|
| There are probably some significant parallels to any executive
| who manages to game the compensation structure at any large
| corporation to their financial benefit.
| lofatdairy wrote:
| If I'm not mistaken the news that Columbia "won't participate" in
| the rankings came a bit sooner[^1], but if anything that struck
| me as a "you can't fire me, I quit" sort of preempting of what
| was inevitable.
|
| Honestly though, college rankings are toxic, as is this whole
| prestige economy we've built around them. I don't even really
| care about colleges faking numbers, so much as other shady
| practices like sending students false promises to get their hopes
| up - pretty much for the express purpose of rejecting them down
| the line. As any unfortunate student who has either had to apply
| to college, or been the parent of someone applying, you can't
| avoid the "chance me" threads or the endless HYPSM dick measuring
| contest while trying to get legitimate information.
|
| [^1]: https://www.nytimes.com/2022/06/30/us/columbia-us-news-
| ranki...
| NelsonMinar wrote:
| I'm proud of my alma mater, Reed College, for refusing to
| participate in the US News rankings since 1995.
| kens wrote:
| There's a very interesting article "How to Game the College
| Rankings" about how Northeastern University's president focused
| on improving the university's ranking. In 1996, it was a "third-
| tier, blue-collar, commuter-based university" rated #162. The new
| university president had a singleminded goal: to improve the
| ranking. He got the university into the top 100 in 2006 and into
| the top 50 by 2013.
|
| https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
| pcurve wrote:
| #49 now. I guess it's harder to game the system past certain
| threshold
| saalweachter wrote:
| Now the interesting question: did
| Northwestern^H^H^H^H^H^H^Heastern get _meaningfully better_ for
| _students_ between 1996 and 2013?
|
| Did they graduate in higher numbers? Did they have better
| career outcomes? Did more get into better(?) grad schools? What
| is their median income? Are they happier, did they enjoy
| college more?
| myko wrote:
| Northeastern.
|
| Northwestern was already relatively prestigious.
| dannykwells wrote:
| Northwestern also - by the way. The president's goal when
| he joined was to break into the top 10.
| zerocrates wrote:
| Just a note that this is Northeastern (in Boston) not
| Northwestern (in Illinois). Northwestern was and is typically
| a very high ranker.
|
| As for what the improvements are like... hard to say. Some
| ways to improve the rankings like getting more students you
| don't want to come to apply (so you can appear more
| selective) don't actually help at all, while things like
| small classes even if done to "game" the rankings could
| actually help.
|
| There's also just the cyclical aspect of things: the quality
| of your experience is going to be significantly determined by
| the quality of the student body, so if you have a better
| applicant pool because of your better rankings you maybe have
| a bit of a virtuous cycle that's somewhat disconnected from
| whether the things you changed actually improved anything
| _real_ on their own other than the ranking.
| pyuser583 wrote:
| What about North Central?
| kaesar14 wrote:
| Was it that hard to read and get my alma mater right? ;)
|
| In my opinion as a student who entered Northeastern in the
| mid 2010's the answer to all your questions is an unequivocal
| yes. More competitive student body, better profile to attract
| employers, more student amenities to enhance on campus life
| (we could have more in this regard imho). Compared to the
| commuter school of yesteryear it's for sure a much better
| educational experience.
| throwaway2037 wrote:
| "better profile to attract employers" -- this is very
| important in the United States. If you are good student
| from a below average university, your job prospects are
| /much/ worse!
| ameliaquining wrote:
| Almost certainly yes, because the school could presumably
| attract a higher-quality applicant pool in 2013 than in 1996.
| This, of course, makes it impossible to tell how much of that
| value was produced by the university and how much was just
| selection effects.
| throwaway2037 wrote:
| From first hand experience, I can say that quality of
| selection makes a big difference in your university
| experience. My first university was /modestly/ selective.
| My second university was not-at-all selective. (I was a
| terrible university student.) The difference in class
| discussion and group projects cannot be understated.
| Students from the higher quality selection university were
| much more intellectually engaging.
| pyuser583 wrote:
| Most colleges have the ability to graduate close 100% of
| their students.
|
| If a school only admits students who have at least two years
| at another college with high grades, their graduation numbers
| will skyrocket.
|
| A highly selective college, like Harvard, could even refuse
| to grant credit for those two years.
|
| There are less nefarious ways of selecting a class of likely
| graduators: leadership in extracurriculars is highly
| correlated with post-secondary graduation.
|
| Foreign students with limited English fluency have very low
| graduation rates. They could (should?) be excluded.
| [deleted]
| dataphyte wrote:
| US colleges foresee a future where they are under attack for
| being absurdly selective, what some are now calling being "highly
| rejective". I think they want to avoid being targeted by the
| fairness wonks anymore than they already are.
|
| Also, US News and World Report rankings have major problems, for
| instance, the ranking algorithm considers campus aesthetics and
| food quality, but doesn't account for price, giving schools
| incentive to raise prices to fund campus improvements that boost
| rank, in turn boosting applications, in turn reducing %
| acceptance (the principal indicator of "quality" in US schools).
| This was how TCU went from a meh christian school to a "selective
| private college" in just a couple of years. World Report now
| publishes a "best dollar value" report to account for this, but
| few read it.
| jseliger wrote:
| The gap between the schools' rhetoric and reality is pretty
| funny; I was just writing about it the other day:
| https://seliger.com/2022/07/06/nonprofit-boards-of-
| directors...:
|
| _You can see a lot of hypocrisy that's uncritically accepted
| by a lot of organizations, including nonprofits. Exclusionary
| higher education is a particular notable example, given the
| soaring rhetoric of "inclusion" spouted by some people involved
| with higher ed, versus the reality of those same schools
| seeking to reject as many applicants as possible. Princeton
| University's president, Chris Eisgruber, has, for example,
| blathered extensively about the school's efforts to "combat
| systemic racism." Princeton has a $37 billion endowment. The
| school's undergrad acceptance rate is 5.6% and it charges a
| sticker price of $73,000 a year (yes, the school does accept a
| handful of token low-income students every year, but that the
| school's overall demographics reflect its target: the wealthy).
| Does that sound like a school devoted to combating systemic
| racism to you? How can people make these kinds of arguments
| with a straight face? Colleges and universities are run largely
| for the benefit of their administrators. The other exclusionary
| schools are doing the same things, as are their private-school
| feeders, despite their vigorous marketing to the contrary._
|
| _Regarding the above paragraph, let me be clear: describing
| how something is, is not the same thing as approving of it._
| rayiner wrote:
| It's truly mind-boggling. The Ivies were always bastions for
| maintaining the position of WASP elites. They still serve
| that function. Maybe there is a bit more wiggle room about
| skin color, but they still function to socialize the next
| generation of people to run WASP institutions like "JP Morgan
| Chase."
| selimthegrim wrote:
| I guess I must have hallucinated reading all those Stewart
| Alsop articles about the decline of the WASP Elite then.
| pyuser583 wrote:
| A big problem: admitting too many students actually decrease
| the number who apply.
| peanuty1 wrote:
| > the ranking algorithm considers campus aesthetics and food
| quality.
|
| wtf
| duk wrote:
| "the fairness wonks"
| deepzn wrote:
| A reminder that the growth and cost of College administration is
| one of the primary causes of the massive increase in College
| tuition across the U.S.
| onemoresoop wrote:
| https://www.theblueandwhite.org/post/michael-thaddeus
| burlesona wrote:
| Per the article this came in response to questions raised by a
| Columbia faculty member critiquing the university's data. I
| thought that report[1] was actually more interesting than the
| original article.
|
| As per the quote at the top of the report:
|
| > Rankings create powerful incentives to manipulate data and
| distort institutional behavior for the sole or primary purpose of
| inflating one's score...
|
| It would be shocking to me if Columbia was the only institution
| that fudged its numbers a bit in pursuit of a higher US News
| ranking. Given how important that ranking has become in US
| culture, I wonder if it's time for the methodology to change to
| require audited/auditable metrics rather than continuing to allow
| self-reporting.
|
| 1:
| http://www.math.columbia.edu/~thaddeus/ranking/investigation...
| aiisjustanif wrote:
| Adam Ruins Everything, Season 2, Episode 7 [1]
|
| 1: https://youtu.be/EtQyO93DO-Q
| twobitshifter wrote:
| It's like doping in cycling. Everyone was getting caught but
| we're supposed to believe the couple who snuck through weren't
| like the rest. Once the tests got good enough there was hardly
| any one left standing.
|
| https://en.m.wikipedia.org/wiki/Doping_at_the_Tour_de_France
| nus07 wrote:
| I have a friend who works at Duke University's marketing
| analytics and performance metrics team. It's filled with MBAs
| who's job is to massage the numbers and metrics that are asked
| by US News every year and provide them to US News.
| noipv4 wrote:
| username checks out ;) you guya have a collab prog with duke
| humanistbot wrote:
| See https://en.wikipedia.org/wiki/Goodhart%27s_law
| colinmhayes wrote:
| I took an Econ class with northwestern's president and he said
| that they straight up lie about their middle 50 gpa and
| standardized test scores. Can't say I was super surprised.
| Every top 25 school is saying 25% of their students got a 35 or
| better on the ACT, the numbers just didn't add up.
| whimsicalism wrote:
| Maybe it is because they "super score" (ie. take the best of
| each section if they take it multiple times).
| rayiner wrote:
| Morty? He was getting pretty chatty nearing retirement.
| colinmhayes wrote:
| Yea the class was basically him gossiping
| gofreddygo wrote:
| Funny how this reminded me of a PG essay[1]
| The most damaging thing you learned in school wasn't something
| you learned in any specific class. It was learning to get good
| grades.
|
| [1]:http://paulgraham.com/lesson.html
| [deleted]
| hardtke wrote:
| My kid just went through college selection process and I can
| assure you that the US News rankings are largely irrelevant in
| terms of decision making. The revealed preference score[1] or
| global university rankings[2] are much closer to how students,
| their counselors, and their parents rank the universities. For
| instance, any ranking that does not have Stanford first among
| non specialized US universities (i.e. excluding MIT, Caltech,
| Julliard, etc.) is suspect.
|
| [1] https://www.parchment.com/c/college/college-
| rankings.php?pag... [2]
| https://www.topuniversities.com/university-rankings/world-un...
| brianbreslin wrote:
| I work at a university, and honestly have never heard of this
| ranking that you referenced in [1]. Also seems super weighted
| towards military academies. What's their model? Are kids in
| the states now looking at more global options ?
| nemothekid wrote:
| > _What 's their model? Are kids in the states now looking
| at more global options ?_
|
| According to their methodology[1] page, it looks like for
| student who gets multiple offers they do an Elo style
| ranking. The first problem; and which probably explains the
| weight to military academies, is that students tend to self
| select into tiers of colleges, so a group of students who
| are interested in the Air Force might not even apply to
| Harvard (even though the USAFA is a competitive school) and
| upon admission opt to go there (after all if you even were
| even considering of joining the air force, why would you
| reject your one shot there to go any other school)?
|
| [1] https://www.parchment.com/c/college/college-
| rankings.php
| pyuser583 wrote:
| So this would favor specialized schools in general?
| hardtke wrote:
| The revealed preference rankings use co-admittance data
| (student got into Harvard and Stanford) along with that
| student's final choice (went to Stanford) to rank the
| schools. The military academies will have a huge edge
| because their decisions come out later and most students
| are highly committed once they continue their application
| to that point (they've gotten congressional
| recommendations, passed physical exams, etc.). So basically
| throw those out.
| marincounty wrote:
| rayiner wrote:
| > For instance, any ranking that does not have Stanford first
| among non specialized US universities (i.e. excluding MIT,
| Caltech, Julliard, etc.) is suspect.
|
| LOL wut? Stanford is modestly well known, but obviously
| Harvard is the one that is unambiguously first.
| gnicholas wrote:
| I'd give Stanford a bit more credit than this, but people I
| know who were admitted to both all went to Harvard.
| Stanford is surely the most prestigious general-purpose
| university west of, well, Harvard.
| corrral wrote:
| > Stanford is surely the most prestigious general-purpose
| university west of, well, Harvard.
|
| Is that no longer Yale? From the perspective of someone
| who had a rather poor exposure to universities during my
| high school years--so, precisely what someone gets from
| pop-culture and osmosis without anyone in particular
| acting as a guide--my impression was that Harvard and
| Yale were unambiguously the Big Two, and everything else
| fell somewhere after them in terms of recognition and
| prestige.
|
| Is this a CS perspective? I hear a lot about Stanford's
| CS program so I assume it's notably good.
| whimsicalism wrote:
| I think Stanford is viewed as having eclipsed Yale and is
| competitive with Harvard. Princeton probably also
| eclipses Yale.
| gnicholas wrote:
| Fair point. I would guess there would be some people who
| would choose Yale over Stanford, and vice-versa. It
| probably depends mostly on which coast you want to be on,
| and if you prefer a very old institution or one that is
| much younger.
| hardtke wrote:
| I've seen a few places that Stanford and Harvard are
| close to 50/50 on cross admits now, and that Stanford has
| even passed Harvard a few times (which no school has ever
| done). Likely a lot to do with press around Stanford
| students becoming rich Silicon Valley founders. My
| original point is that there has been a lot of changes in
| how students perceive schools that may not get reflected
| in the US News ratings.
| corrral wrote:
| Interesting! My impressions were formed some time ago, so
| I'm not surprised they're out of date. Thanks.
| t_mann wrote:
| I think it's easy to underestimate the impact of rankings.
| Just from your description, it seems that one of the most
| important overall decision factors for you seems to have been
| some sort of vague personal sense of how you'd rank them.
| That subjective ranking is probably influenced by a lot of
| factors, including many rankings that you consciously claim
| to disregard.
| hardtke wrote:
| My personal opinion is that students are highly influenced
| by where the best students in the grades above them chose
| to apply, where they got in, and where they eventually
| chose to go. The software the students use also ranks how
| likely they are to get into each college/university based
| on the past performance of students at their high school
| with similar grades and test scores.
| gnicholas wrote:
| I find revealed preference to be very useful in many
| situations, but a ranking that has Harvard just below
| Swarthmore, at #61, does not pass the smell test. I say this
| as a graduate of Swarthmore -- I cannot take seriously a list
| with Harvard so far down.
| pyuser583 wrote:
| It's really weird Cambridge is ahead of Oxford in world
| rankings.
|
| Cambridge has a reputation for strongly preferring UK
| students.
| mtlmtlmtlmtl wrote:
| This is a good example of Campbell's Law at work:
| https://en.wikipedia.org/wiki/Campbell%27s_law
| ethbr0 wrote:
| An obvious takeaway in this case: US News shouldn't be the
| primary ranker.
|
| If statistics were distributed in open format, transparently,
| and there were multiple rankers (each with their own weights
| and mixes), universities would be less incentivized to game
| "the US News metrics".
| t_mann wrote:
| That seems naive to assume. We already live in a world
| where there are multiple rankings, and many of them (even
| some of the most important ones, like QS or Shanghai) rely
| only on independently collected or public data.
|
| We already have what you suggested as a solution, and look
| at where we are. The problem is more fundamental.
| humanistbot wrote:
| It already exists and is a free US Dept of Education
| service. Schools have to report that data to the Dept of Ed
| already, so they have both an interactive search tool [1]
| and the raw data available [2].
|
| [1] https://collegescorecard.ed.gov/search/?page=0&sort=thr
| eshol...
|
| [2] https://collegescorecard.ed.gov/data
| t_mann wrote:
| Thanks for sharing. Sounds very similar to Goodhart's law:
|
| https://en.wikipedia.org/wiki/Goodhart%27s_law
| hackyhacky wrote:
| Your suspicion is correct.
|
| I've taught at several US universities, and it's an open secret
| that the administrative takes steps to manipulate numbers in
| order to secure better rankings.
|
| Typical example: a small, prestigious liberal arts college
| always has smaller class sizes in the fall semester than in the
| spring semester. Why? Because classes with less than 30
| students are ranked higher by US News, but they only take into
| consideration the fall semester. No one cares about class sizes
| in the spring.
| pishpash wrote:
| What's that adage? Every metric that becomes an optimization
| target ...
| CogitoCogito wrote:
| > Because classes with less than 30 students are ranked
| higher by US News, but they only take into consideration the
| fall semester. No one cares about class sizes in the spring.
|
| What a bizarre and totally unnecessary restriction.
| unosama wrote:
| You add a simple and reasonable (until it gets abused)
| assumption to your model which allows a lesser data
| collection burden on thousands of universities. I wouldn't
| call it a restriction.
| thaumasiotes wrote:
| > which allows a lesser data collection burden on
| thousands of universities.
|
| How's it supposed to do that? Universities all already
| collect the data on enrollment for every class they
| offer.
| genewitch wrote:
| Parents helping/guiding their near-adult children in the
| final spring of highschool to apply at or choose
| universities, where they would start university that same
| fall seem to be the target audience, here. Fall is when
| "most" incoming freshmen will start.
| [deleted]
| bogota wrote:
| Great to know many peoples graduations were delayed so
| someone could game some numbers
| abakker wrote:
| Gotta pay for another semester then, right?
| zerocrates wrote:
| I think they mean the size of individual classes, so
| they're paying for more sections/instructors/instruction
| time in the ranking-relevant periods.
|
| Though shrinking the admit size is another well worn
| tactic, as it's the easiest way to increase how "selective"
| you are which is another major ranking element, plus it
| helps you more cheaply achieve smaller average class sizes,
| though of course you're forgoing some revenue to get there.
| btilly wrote:
| Encouraging people to apply is also a common tactic. You
| can't reject them if they don't apply!
| javajosh wrote:
| Applications are also a revenue source.
| dhosek wrote:
| Apparently there's a common application thing now where
| you can apply to large numbers of schools for undergrad
| without spending large amounts of money.
| thaumasiotes wrote:
| The common application just means you don't have to fill
| out unique admission forms for every school. You still
| pay an application fee to every school.
| thaumasiotes wrote:
| > You can't reject them if they don't apply!
|
| You can if you admit on objective metrics. If you only
| take people who are 215 pounds or above, you can fairly
| claim to reject everyone lighter than that, regardless of
| whether they apply.
| tenpies wrote:
| What are they going to do? Transfer out and have only half
| their credits recognized therefore pushing them back 4
| semesters instead of the 1?
|
| It's a fantastically anti-consumer market they've managed
| to create.
| nostromo wrote:
| Gaming and manipulation should absolutely be addressed, but
| it seems Columbia was just straight up lying about their
| numbers, not gaming them.
| greymalik wrote:
| Goodhardt's law in action.
| eduction wrote:
| I attended a well known and regarded public university. I was
| admitted for spring, and for fall entered a program where I
| could live in a regular university dorm and take classes for
| full credit taught at the same university's extension, but
| special extension classes just for this program (so all
| spring admit students). Then you stay in the dorm in spring
| as you become a "regular" student.
|
| I always suspected it was a way to fudge numbers, but at the
| same time I knew I was genuinely on the bubble of being
| admitted so I took the opportunity gladly. (My memory is that
| the other spring admits were, like me, from racial groups
| considered "over represented" so they may have been trying to
| affect diversity numbers too.)
| lumost wrote:
| Why is small class size a good thing? Anecdotally, you get
| better class material and TA staff in larger lectures. Why
| aren't Universities able to innovate here?
| rglullis wrote:
| My uncle was a math teacher in Brazil - mostly test-prep
| course schools for university exams, later he opened some
| of his own private schools.
|
| He used to say two things: that he has estimated to have
| taught at least 100 thousand people in his life (seems just
| crazy boasting, but feasible when you are working in 5-8
| different schools per semester and each class has 100-250
| people) and _that a teacher is always teaching for the
| worst student in the class_ , and that the only reason that
| top universities are better is because of their selection
| process.
|
| Forget class material, forget infrastructure and fancy
| facilities. _What makes the best universities is the fact
| that the top students are there_.
|
| If you can not (dramatically) change your students and make
| them smarter, you can make your classes smaller and segment
| by performance. This at least stops your top 20 percentile
| from being dragged down by your bottom 20.
| acchow wrote:
| > that a teacher is always teaching for the worst student
| in the class
|
| It's been my experience in college that most teachers
| just let the bottom quartile of the class fail (and
| likely switch into another major).
| rglullis wrote:
| There is no "switching majors" in Brazil. You make the
| application for the major that you want, and you are
| competing against everyone that also applied to that
| course. If you decide to switch after starting a major,
| you almost certainly will be forced to drop out and re-
| apply.
|
| But in reality, even that doesn't matter. In the best
| universities, your worst students will be better than the
| average students at the second-rate institutions. If the
| teacher is simply giving the class to the top 75
| percentile, the class will be _even better still_ at the
| top uni.
| cm2012 wrote:
| This is 100% true. It's also why attempts to make things
| more fair by allowing lower testing kids into specialized
| schools fail. It just makes the specialized school a
| normal school.
| bbor wrote:
| Many courses rely on discussion!
| ghaff wrote:
| For liberal arts? It's probably a pretty interactive format
| and smaller interactive classes are better than large
| classes that are more broadcast. (Often. I've also had
| great lecturers in the humanities.) But mostly don't look
| at many liberal arts classes (or case study, etc.) through
| the lens of an engineering lecture.
| aquova wrote:
| Fewer students per teacher is widely seen as a positive
| thing. A greater percentage of the professor's time and
| attention can be given to any one student. I've definitely
| experienced this myself, it's easy to feel like just a
| number when you're in a large lecture hall of hundreds
| versus a small classroom of a few dozen.
| bachmeier wrote:
| I already said this elsewhere, but I do want to reply to
| this comment, because it demonstrates a common mistake
| that may not be clear from my other comment.
|
| If you have a given teacher, in a given classroom, with a
| given amount of TA help, blah blah, so that all else is
| equal, and then you reduce the class size, the students
| will do better.
|
| The problem is that all else is far from equal in the
| data used in these rankings. Instructor quality drops off
| rapidly as you have more sections. TA help falls. The
| classroom may no longer be appropriate for the class,
| etc.
|
| The rankings are not able to capture the effect you're
| talking about.
| poulsbohemian wrote:
| >Anecdotally, you get better class material and TA staff in
| larger lectures.
|
| Where is this anecdote coming from? A billion years ago
| when I studied at a small liberal arts college with small
| class sizes, professors were keen to share with us exactly
| why they were teaching from particular texts or why they
| had selected certain materials to make it clear that while
| the class wasn't being taught by that famous guy from MIT,
| it was still his book and materials we were using. I knew
| all my professors by first name, knew where many of them
| lived, had access to them anytime it was needed, and that's
| directly counter to my partner's experience at a large
| university.
| lumost wrote:
| My undergrad was at Umass Amherst, and I currently attend
| GA Tech's OMSCS program. Professors love sharing info on
| why they chose to talk about one topic vs. another, and
| by and large were happy to connect with any student
| motivated enough to attend an office hours or talk to
| them after class. I knew same personal information about
| the professors I worked with in the lab, but I don't
| generally see how that would improve my learning
| outcomes.
|
| Class size naturally dropped as you got into niche
| subjects, of which there were many!
| poulsbohemian wrote:
| What I've heard anecdotal bears out what you are saying,
| IE: once you get through various mega weed-out classes,
| in many universities you still get the luxury of smaller
| class sizes and more direct access to professors as you
| reach those upper level classes.
|
| Where I was asking for clarification was on the idea that
| somehow people were getting better materials and a better
| overall experience in thee 500+ people lecture hall type
| experiences, rather than those 20-person intimate
| learning style classes.
| jimhefferon wrote:
| One factor I don't see anyone mentioning here is the
| student. Some people can do great in a large lecture. I
| teach in a SLAC and often students tell me they don't
| think they could do well without the smaller courses.
| verst wrote:
| Exactly this.
|
| I went to a liberal arts college and had small class
| sizes (most around 9-15 in size, with the largest being
| 30) - I could see my professors 1:1 pretty much anytime I
| wanted, but there was also a preceptor (TA) for each
| class who didn't teach, but was available to answer
| questions related to assignments, projects etc. I went to
| the house of many of my professors for food or drinks
| too. Generally also was on a first name basis with
| professors.
|
| The downside is that liberal arts colleges aren't
| research institutions and they are generally
| undergraduate only. This means you don't have as many
| choices for advanced courses. However, the professors at
| liberal arts colleges really tend to be there for the
| teaching and tend to enjoy that. One of my math
| professors was the head of the mathematical association
| of America and also authored the books on Analysis (the
| mathematical subject - not a data science thing) we were
| using.
| poulsbohemian wrote:
| >The downside is that liberal arts colleges aren't
| research institutions and they are generally
| undergraduate only.
|
| What's ironic about this though is how many universities
| won't accept their own undergrads into their grad
| programs... IE: they know that their undergrad programs
| are sub-standard but they don't seem to take action to
| improve them because the money and the focus is on the
| grad programs.
| snowwrestler wrote:
| Many professors at small liberal arts colleges still
| conduct research. And because of small class sizes, there
| is a good chance for a motivated student to get to know
| their professor and participate in that research.
|
| There are also research consortiums of small colleges. I
| did original geological research 2 out of my 3 summers in
| a small college via the Keck Consortium:
|
| https://keckgeology.org/
| seanmcdirmid wrote:
| I did research for a professor as an undergraduate at a
| big university. Once you are out of lower division
| courses and into your upper division major classes, your
| classes are much smaller and the professors start to know
| you as you see them around a lot.
| Leary wrote:
| One thing is closer interaction with faculty. Crucial if
| you want to apply for grad school and need letters of
| recommendation.
| kmlevitt wrote:
| Small class sizes are better precisely because the class
| can then become more than a lecture. I am a professor, and
| if you give me a hall full of students I'll give a lecture,
| because that's all that can really happen under those
| circumstances.
|
| If it's a small group, I can actually learn everybody's
| names, get to know them, personalize the material more and
| spend big chunks of the class just answering your questions
| and starting discussions based on them. The class becomes
| less of a mere information dump and more of a mentorship.
| echelon wrote:
| In my three student organic chemistry III class, the
| professor had each of us reading ahead and taking turns
| teaching the material ourselves. We'd be corrected for
| mistakes, of course, but it was very much a fire drill.
| We moved a lot of electrons.
|
| That was quite the course.
| balls187 wrote:
| > If it's a small group, I can actually learn everybody's
| names, get to know them, personalize the material more
| and spend big chunks of the class just answering your
| questions and starting discussions based on them.
|
| I went to a relatively small private engineering school.
| It made a world of difference in my education knowing the
| professors, and having them know me, especially given
| that with small classes the professors offered office
| hours where I could come and discuss the material that
| was covered during class, as well as problems I
| encountered on midterms.
|
| I still remember nearly all of my professors, _fondly_ ,
| precisely because of the dynamic small class sizes
| allowed for.
| dan-robertson wrote:
| 1) they aren't rewarded for innovating
|
| 2) I guess they can actually by having massive classes in
| the spring and comparing to the fall.
| deltree7 wrote:
| It's a perfect example of reverse causation.
|
| Great Classes Implies Larger Classes (pure supply /
| demand); Larger Classes doesn't imply Great Classes
| [deleted]
| mattwest wrote:
| It's an interesting question, and that's probably why it
| ends up in so many econometrics and stats courses. You have
| the classic causality problem, along with ethical concerns
| of generating experimental data for such a study.
|
| Controlling for as many of the relevant variables as
| possible, there does seem to be a strong negative
| correlation between class size and test scores. Beyond
| that, who knows, but it makes sense that small classes
| would succeed more often (at least up to a certain point of
| course)
| bachmeier wrote:
| But...none of those studies justify a discontinuity at a
| particular value. And if they control for things like TA
| help, they cease to provide useful information, because
| in the real world TA help increases with the class size.
| bachmeier wrote:
| There are real issues with using class size as they do. If
| you have someone that's really good at teaching a
| particular class, it's better to have them teach one
| section of 60 students rather than have a second section
| taught by someone that does a mediocre job. You also have
| things like TA help for larger classes, and that doesn't
| factor in at all. There's also the discontinuity thing. 30
| is way better than 31, but equally better vs 300 - which
| obviously makes no sense. Overall, the measures they use
| for class size in these rankings are garbage.
|
| The same is true for selectivity. It doesn't account for
| education at all, only the difficulty of getting in. But
| since it doesn't account for the applicant pool, it doesn't
| account for the difficulty of getting in either. You could
| use things like SAT scores as a measure of the difficulty
| of getting in and ditch the acceptance % completely. I work
| at an R1 that gets hit hard because we admit most
| applicants because, you know, we exist to educate folks,
| not to serve as a signalling device.
| mbesto wrote:
| Which begs the question - if you want to get top in rankings
| wouldn't you simply design the school around the objective
| ranking parameters and nothing else?
|
| Ripe for abuse...
| colinmhayes wrote:
| This is sort of what Columbia did... they juiced their
| faculty numbers by counting the med school which has much
| lower class sizes and more educated teachers.
| nonameiguess wrote:
| Not sure if this is still true, but at one point, the biggest
| factor in the ranking was peer reputation rating. That kind
| of results in a self-reinforcing vicious cycle of top schools
| remaining top schools, but short of bribery, there isn't
| really any way to manipulate that at least.
| conqrr wrote:
| Explains how ASU has always been number 1 in innovation. It's a
| great school, but no.q for innovation is pushing it too much.
| dwater wrote:
| For context, U.S. News and World Report used to be a trusted
| periodical that reported national and international news. In the
| 80's they started ranking colleges and selling it as a separate
| publication, which turned out to be pretty profitable, so they
| repeated the trick by ranking hospitals (1990), cars (2007), and
| then states (2017). The news business went online only in 2011,
| then shuttered in 2015. Since then they've just been a rankings
| company, but they kept the name to trade on their reputation.
| UncleOxidant wrote:
| U.S. News and World Report was a highly respected & trusted
| news magazine in the 70s-90s, but it's online incarnation now
| is just another clickbait site (similar for Newsweek).
| uptownfunk wrote:
| Wow fascinating - so US News is now a rankings company.
| xanaxagoras wrote:
| If you think of clickbait as a crowdsourced ranking of
| emotionally manipulative hyperbole, that could be said of any
| corporate press rag.
| cuteboy19 wrote:
| I feel like it would not be possible to sell such a static and
| easily leaked thing as a ranking list
| lpolovets wrote:
| The influence of US News on colleges is wild. One absurd tidbit
| that I've heard: the rankings weigh freshman stats much more than
| transfer student stats, so a lot of competitive schools that
| won't let you in as a freshman if you would hurt their
| SAT/GPA/etc averages will happily let you in as a transfer
| because they'll make more revenue and your stats no longer hurt
| their rankings. *
|
| The ranking formula is also changed arbitrarily based on the
| reactions of colleges and readers. My school went from #9 to #1
| one year, there were a lot of "wth??" reactions, and the
| following year the formula was tweaked so that our ranking dipped
| down to #4. I'm pretty sure nothing changed materially in the
| school over those 3 years, but the ranking moved around a lot.
|
| The whole system is a farce.
|
| * I haven't researched freshman vs transfer data myself, but it's
| something I've heard from multiple startup founders in the
| college education space.
| rr888 wrote:
| Yeah this was one of the whistleblower topics. Columbia has
| highest number of transfers
| http://www.math.columbia.edu/~thaddeus/ranking/investigation...
| whimsicalism wrote:
| Depends on the school. At a place like Harvard or Yale it is
| incredibly difficult to get in as a transfer, much harder than
| to get in in the first place.
| ffguuficc wrote:
| For UW this is 100% true. A 4.0 is like a 50/50 to get in the
| CS program but for transfers it's easier.
| hprotagonist wrote:
| tit for tat:
| http://www.math.columbia.edu/~thaddeus/ranking/investigation...
| gumby wrote:
| I really wonder why Columbia even cares: it's a globally well
| known ivy league school obviously producing a good education and
| is in NYC.* Doesn't US News need Columbia more than the other way
| around?
|
| * No I have no connection to the school except I do have some
| friends who teach there and some others who attended decades ago.
| colinmhayes wrote:
| Having high ranking is like the entire job of administrators.
| Especially at Ivys they're selling prestige, being highly
| ranked gives you prestige.
| gumby wrote:
| If you're in the small top 'n' I think the prestige goes the
| other way. Will anyone care if MIT is +1 or -1 some year? I
| assume those top schools can just run themselves the way they
| want and still get highly ranked.
|
| And if US news were serious they could simply have kicked
| columbia out for a year, but it would have hurt them to do
| so. They'd probably do it for a small college ranked 100 or
| below though.
|
| I'd be embarrassed for Columbia that they gamed the numbers.
| Outside that small n though, I presume gaming the numbers is
| commonplace.
| lofatdairy wrote:
| I think professors and the actual scholars there don't really
| care. However, administration cares because ranking is tied to
| both how many undergraduates students apply and how much they
| can charge these students. If college rankings didn't matter
| financially then I hazard that Columbia would've had
| questionable numbers to begin with.
|
| That said I agree that being an Ivy in NYC certainly was a
| major contributor to its more recent rise in the rankings and
| applications.
| resoluteteeth wrote:
| It's interesting to think what most universities would do to a
| student who tried to falsify their data like this.
| data4lyfe wrote:
| The fact that U.S. News and World Report can pull a school from
| it's own rankings and it makes such a strong impact is really a
| strange phenomenon.
| epgui wrote:
| For all of the ways we can easily criticize the incentive
| structures and possible intentionality behind these bad numbers
| (which I agree merit scrutiny), I think it's worth noting and
| remembering that the issue was flagged by a mathematics professor
| of the same university: there is at least some well-aligned
| incentives, some measure of self-regulation, with respect tenured
| positions and academic freedom.
| geebee wrote:
| One interesting thing to come of these discussions - a lot of the
| problems with US News methodology arise from the attempt to push
| certain metrics from 90% to 100%. Class size, faculty with a
| "terminal" degree, class size, admissions rate. Many of these
| things are treated by the rankings as "the higher (or lower) the
| better." As an analogy, I'd use lean muscle mass percentage for
| elite athletes. Generally speaking, lower is better, and in many
| sports, all elite athletes are all under 10% body fat. But much
| below this starts to become harmful, and would eventually be
| fatal.
|
| I think this is kind of what's happening with Columbia here. For
| example, Columbia claimed that 100% of the faculty had a terminal
| degree. Prof Thaddeus (if you read his full blog post) questioned
| this from two angles. The first was to say it can't possibly be
| true. The second (which I found more interesting) was: why would
| you even want this to be 100%? Seems like Columbia has valuable
| faculty who don't have a PhD or MFA or whatever is the "terminal"
| degree int he field.
|
| The harm here isn't in wanting a terminal degree for a high
| percentage of faculty, it's trying to get this to 100%. I'd say
| that it probably is a good sign, overall, that most of the
| faculty has a doctoral degree, as this is the main training
| degree for research and teaching. But the way US news does it,
| you get more points for 95% than 90%, more points for 99% than
| 95%. Seems like a lot of harm comes from trying to wring out that
| last bit, since even if you do generally agree with the value of
| a PhD for research (or other terminal degree), it does seem like
| you'd build a stronger faculty overall with the ability to hire 1
| in 10 or 1 in 20 based on some other credential.
|
| So much else fits this pattern. Small class sizes.. kind of the
| same thing. Sure, it's a good sign, but they're only an
| indicator, not a goal in and of themselves. Low admissions rates,
| high test scores. All good metrics, but things that can become
| not just harmful but harmful and maybe even fatal to a university
| if pursued with a single minded intensity as if total purity in
| the metric is the goal.
|
| Honestly, overall we just have to reject the US news rankings. I
| appreciate what Prof Thaddeus did here, and it was a useful and
| very well reasoned and sourced takedown. But now and then, I
| realized that there isn't much to be added, just new and
| interesting ways to say what everyone knows at this point -
| yousnoozeandworlddistort rankings are pretty well idiotic. I
| actually think college rankings in general can be useful if
| managed cautiously and read critically, but this one is just a
| turkey.
| danielmorozoff wrote:
| Having pursued a PhD at Columbia and having taught classes there.
| I am surprised it took this long for someone to speak up. Also
| the fact that there are no checks in place to certify top ranking
| academic institutions is fascinating.
| xiaolingxiao wrote:
| could you expand on what you mean please?
| taveras wrote:
| If folks are curious about how schools are ranked, I highly
| encourage you to listen to the following podcast:
|
| https://www.pushkin.fm/podcasts/revisionist-history/lord-of-...
| saagarjha wrote:
| Kind of sad that US News decided to unrank just Columbia rather
| than suspending the whole thing. If one college games your
| ranking to the point that a professor from the university is like
| "hey you guys this seems pretty sus", it's probably true that all
| the other universities have the same problem and just don't have
| anyone willing to talk about it. As it stands it seems like just
| Columbia did something wrong, when they are the only university
| that did something wrong but also had someone who did something
| _right_.
| wombat-man wrote:
| Well, it's kinda their main business at this point.
___________________________________________________________________
(page generated 2022-07-11 23:00 UTC)