[HN Gopher] All models are wrong, but some are completely wrong
___________________________________________________________________
All models are wrong, but some are completely wrong
Author : magoghm
Score : 173 points
Date : 2020-04-05 15:55 UTC (7 hours ago)
(HTM) web link (rssdss.design.blog)
(TXT) w3m dump (rssdss.design.blog)
| grawprog wrote:
| What I've noticed about models, or at least when people are
| talking about them or trying to prove a point about them, is that
| people forget models are a simplified version of a specific part
| of reality, much the same as models of airplanes or something.
|
| No matter how many variables you include, you can never capture
| the utterly massive and unpredictable amount of variables that
| exist in reality. But they're not supposed to. They're tools that
| are supposed to be used to help get an idea about how something
| might happen, that's marginally better than just guessing, due
| again to models always being incomplete, despite any amount of
| best efforts.
|
| Like any tools, using models incorrectly, fucking around with
| them until you just get what you want to see or using the wrong
| one for the wrong task can be potentially dangerous, especially
| when those models are used to guide large decisions with serious
| consequences and impacts.
|
| Reasoning and common sense need to be used alongside models and
| if the models don't seem to reflect reality, the problem is with
| the model and not reality and the model needs to be adjusted or
| thrown out otherwise continuing to use it will just lead to
| lousier and lousier decisions.
| BiteCode_dev wrote:
| Humans have the tendency to think they understand something if
| they have a label on it. It helps them take decisions faster,
| whether it's good or bad.
|
| If you tell people you don't know what causes their respiratory
| illness, they may think you are incompetent, and feel helpless.
|
| If you tell them they have acute bronchitis, suddenly they feel
| empowered. It just means they have an inflammation of the
| bronchi. It says nothing of the direct cause, nor the indirect
| cause: a virus, tobacco, dust, air pollution or asthma. It says
| nothing about how bad it is. They don't even know really what
| it implies for their body or their treatment.
|
| But now they think they know.
|
| Call something a democracy, and nobody will check if it is.
|
| Call something a privilege, and people will start to want it,
| working hard for it.
|
| Models can be used to help your interact with the world. But
| they are also used to avoid interacting with the world: if you
| use them as labels, instead of comparing the model with
| reality, you get a shortcut to take decisions. It becomes a
| mere name to justify a decision process instead of a tool for
| the decision process.
| LudwigNagasena wrote:
| >If you tell them they have acute bronchitis, suddenly they
| feel empowered.
|
| In the The Imaginary Invalid (17th century) Molliere makes
| fun of it. Doctor explains how opium works through its
| "dormitive virtue."
| dchyrdvh wrote:
| "Mind is the great destroyer of reality" - a quote from one
| of those ancient dusty books. Our mind is a machine that
| creates models and labels for everything, to avoid dealing
| with the illigible reality.
| mattkrause wrote:
| I'm not sure I agree.
|
| While _technically_ bronchitis could be any inflammation of
| the brochial tubes, a _diagnosis_ of bronchitis usually means
| that that it 's the patient's major problem. It'd be odd to
| describe a cancer as "bronchitis" even if it involved brochi.
| So you do learn something, in a Griceian sense, from the
| relatively uninformative diagnosis. There's also the
| implication that there are some tools for managing it, the
| doctor's seen it before, etc all of which are reassuring.
| schnable wrote:
| Which leads to the saying that this headline is a play on: "All
| models are wrong, but some are useful." (George E.P. Box)
|
| We probably should be repeating that a lot to make sure people
| hear it.
| hinkley wrote:
| It's funny, but when you point out that a bad analogy is
| actually pretty accurate _if you actually know anything about
| the other concept_ , people don't want to talk about it any
| more.
|
| We all know that hill climbing algorithms are often naive and
| sometimes hilariously wrong. Nobody will disagree with you
| about this, until you start talking about prioritizing work,
| and then everyone vigorously defends their favorite hill
| climbing algorithm, from task management to performance tuning.
|
| My preferred strategy for optimization more closely resembles
| how a fruit grower would pick fruit, and the term, "low-hanging
| fruit" galls me because you would go bankrupt using that
| strategy. And probably lose most of your trees to disease. It
| could be a quite good analogy, if the lessons taken from it
| weren't just so childishly naive.
| tartoran wrote:
| The low hanging fruit metaphor meaning do the simplest or
| easiest work first is not really applicable to fruit growing
| but seems to have been coopted by the busieness world
| instead, then we started using it too in our contexts. Not
| crazy about this metaphor either, but it could be useful in
| some contexts and while also misunderstood in others. The
| same goes for most jargon.
| darkerside wrote:
| Low hanging fruit is a point in time tactic, not a
| generalized recommendation
| hinkley wrote:
| And when you do that for a very long time they call it
| short-term thinking. Myopia. Missing the forest for the
| trees.
|
| Plenty of that going around.
| edmundsauto wrote:
| Can you elaborate on your preferred strategy?
| discreteevent wrote:
| I doubt many people think of it from the perspective of a
| fruit grower. I think of it as picking wild fruit, in which
| case the analogy mostly works.
| puranjay wrote:
| I fail to understand how you can reasonably model an
| unprecedented event in modern history. We have no data on how
| people will act under a weeks, even months long lockdown. Will
| they stay indoors and follow guidelines? Maybe. Will they watch
| their livelihoods get affected, their mental health deteriorate,
| or get careless over time and break quarantine? Maybe.
|
| We just don't know because we just don't have the data or heck,
| even anecdotal evidence from similar events in the past
| ajuc wrote:
| There's very little unprecedented events in modern history, and
| pandemic certainly isn't one.
| puranjay wrote:
| Complete lockdowns across multiple countries and cessation of
| all economic activity certainly is unprecedented.
| not2b wrote:
| We aren't ceasing all economic activity, fortunately.
| martingoodson wrote:
| If you're interested in this specific area, here is a useful
| summary of what's known:
| https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
|
| tl;dr not much
| veddox wrote:
| On the contrary, epidemics are reasonably easy to model (at
| least at a basic level), and modelling them has a long academic
| tradition. As with all modelling, it all depends on what kind
| of detail you want to get out of your analysis/predictions.
|
| If you're looking at a simple population infection model,
| lockdown efficacy is just a factor affecting the contact rate
| among uninfected actors. You run multiple scenarios for
| multiple levels of this factor, and see where that gets you.
|
| Or did you mean something else?
|
| (Source: I'm not an epidemiologist, but as an ecological
| modeller I work with very similar tools.)
| puranjay wrote:
| So many of the models I've seen focus on the efficacy of
| lockdowns in controlling the spread of the disease. Most
| treat the length of the lockdown as a mathematical variable -
| X week lockdown leads to Y infection rate.
|
| My point is that we can't reasonably predict how people will
| behave in lockdown past a certain point. We've never had
| similar lockdowns in a world that was as globalized, as hyper
| connected as ours. You could go from 2 to 8 weeks of
| lockdowns if everyone was living in isolated villages a la
| 1918 Spanish Flu, but that's not our present world.
|
| How do you model a situation where after 4 weeks of
| lockdowns, a social media post about food shortages goes
| viral, causing mass panic and breaking of quarantine?
|
| We can't because we've never had a situation like this, or
| the tools for spreading (mis) information as we currently do.
| heurifk wrote:
| We have recent data from Wuhan.
|
| We have old data from 1918.
|
| This is a white swan, not a black one.
|
| More importantly, the mortality, while much higher than flu,
| it's still relatively low.
|
| Now imagine a virus as contagious as this one, but with 10%
| mortality over all age groups. That would be unprecedented and
| probably cause society meltdown.
| puranjay wrote:
| How do you reckon social media will impact your models?
| Rumors and information - real or fake - has never been easier
| to spread in the world (Wuhan's data becomes less relevant
| here). We've already seen calls for defiance of lockdowns in
| right wing circles. In my country, old videos are being
| circulated to spread misinformation about the treatment.
|
| Again, our data for all older epidemics is applicable to the
| epidemic in isolation. But there is no way to accurately
| model how the epidemic interacts with people simply because
| the way people live has changed drastically from past
| epidemics.
| a1369209993 wrote:
| > More importantly, the mortality, while much higher than
| flu, it's still relatively low.
|
| Anecdote: someone was trying to convice me to panic about
| Coronavirus because "three hundred and [something] people
| died just today!" $ units 8 billion /
| 80yr /day
|
| "Actually, three hundred _thousand_ people died today.
| Probably more, even. "
| koyote wrote:
| I think a lot of people would be very surprised if you told
| them how many people die in their country every year.
|
| It's nearly 3 million in the US.
| yk wrote:
| If you look at the models, they have something like an order of
| magnitude uncertainty. And the reason for that is precisely
| what you are stating, they rely on future behavior, which we
| just don't know yet.
|
| However, the utility of the models is to give us a sense of how
| the different parameters interact. There are parts of the model
| were we can have a lot of trust, for example that people with
| severe conditions will need hospitalization, or that people
| will react quite similar as they reacted yesterday. So for the
| short term, the models give us quite good guidance, and for the
| long term, they help to map out scenarios.
|
| So if you actually look at the report in question, you will see
| that they are actually trying to estimate the impact of various
| non medical interventions, like encouraging social distancing,
| by comparing different countries. It is just that newspapers as
| usual just run with the most immediately digestible number,
| independent wether that number is important or useful.
|
| The study in question:
|
| https://www.imperial.ac.uk/media/imperial-college/medicine/m...
|
| Some overview video from Dr. Campbell on youtube: (in general,
| I think his youtube channel is quite good)
|
| https://www.youtube.com/watch?v=c1aoULlMpn0
| firefoxd wrote:
| This is a good time for schools to go back to teaching about
| Ignorance:
|
| https://www.nytimes.com/2015/08/24/opinion/the-case-for-teac...
| IBCNU wrote:
| All models are wrong, some are deadly.
| DoofusOfDeath wrote:
| Regarding the "all models are wrong" maxim...
|
| Is that statement 100% true for the low-level models that
| physicists use and develop? In particular, I'm curious if
| quantum-physics models are 100% right, just not 100% precise.
| carapace wrote:
| The Standard Model has no known contradictions in the real
| world. https://en.wikipedia.org/wiki/Standard_Model
|
| In "QED" Feynman states that the predictions are as precise as
| being able to measure the distance between (points in) New York
| and Los Angeles accurately to within the width of a human hair.
|
| https://en.wikipedia.org/wiki/QED:_The_Strange_Theory_of_Lig...
| qchris wrote:
| At the level that you're talking about, you'll start running
| into the unresolved questions of modern physics.
|
| One of the best ways to look at this is through the Standard
| Model of particle physics, which essentially defines how the
| fundamental particles of the universe are related. Between the
| number of observations and large-scale experiments dealing with
| high-energy collision products, astrophysics, neutrino
| detectors, etc., some people consider the Standard Model to be
| the most thoroughly-tested and verified framework in all of
| science. That's a pretty grand claim, but hey.
|
| But it still falls short in some ways--for one, it starts
| breaking down past a certain scale. Classical field theory as
| defined by general relativity (another model that has had
| enormous success under test both theoretically and
| experimentally) and particle physics don't get along. Neither
| one fully explains reality, and the interface between those two
| models of reality hasn't been found. That's why people research
| things like string theory--they're attempting to find a
| mathematical framework that can resolve those two frameworks,
| among other things.
|
| So while each of them describes the universe extremely
| accurately in their own domain (check the sigma values and
| number of observations of experiments run on the LHC), they're
| not 100% right, since they can't be correctly extended to cover
| all scales and frames. The models remain just that--models
| which provide a useful framework to interpret reality, but
| don't fully describe the physical reality itself.
| tomrod wrote:
| I keep this distinction in mind:
|
| - Models are deliberate simplifications of reality, in order to
| guide thinking and otherwise pull in only important information
|
| - Formulations (formalizations) are encapsulation of principles
| into a mathematical framework
|
| While there is significant overlap, the two categories do not
| overlap 100%. I see formalizations of physics as the latter,
| and we use the former to help keep our understanding of the
| latter clear.
| [deleted]
| Jabbles wrote:
| Some predictions of Quantum Electrodynamics have been verified
| to within 1 part in 100,000,000.
|
| https://en.wikipedia.org/wiki/Precision_tests_of_QED
|
| However, we know it cannot be _completely_ accurate, as it has
| no way of explaining gravity.
| [deleted]
| paulsutter wrote:
| "the best material model of a cat is another, or preferably the
| same, cat" - Norbert Weiner
| kgwgk wrote:
| > The FT chose to run with an inflammatory headline, assuming an
| extreme value of r that most researchers consider highly
| implausible.
|
| The headline "Coronavirus may have infected half of UK population
| -- Oxford study" is not really out of line with the preprint the
| article talks about. What's questionable is that it was a good
| idea to write about the preprint at all.
|
| "Importantly, the results we present here suggest the ongoing
| epidemics in the UK and Italy started at least a month before the
| first reported death and have already led to the accumulation of
| significant levels of herd immunity in both countries."
|
| "Our overall approach rests on the assumption that only a very
| small proportion of the population is at risk of hospitalisable
| illness. [...] Three different scenarios under which the model
| closely reproduces the reported death counts in the UK up to
| 19/03/2020 are presented in Figure 1 . [...] [In two of those
| scenearios] By 19/03/2020, approximately 36% (R 0 =2.25) and 40%
| (R 0 =2.75) of the population would have already been exposed to
| SARS-CoV-2. [...] [The third scenario] suggests that 68% would
| have been infected by 19/03/2020."
|
| The secondary headline "New epidemiological model shows vast
| majority of people suffer little or no illness" was much worse as
| that's the assumption in the model and not the result. It was
| changed to "New epidemiological model shows urgent need for
| large-scale testing" in the amended article.
|
| > Since its publication, hundreds of scientists have attacked the
| work, forcing the original authors to state publicly that they
| were not trying to make a forecast at all.
|
| "Attacked" sounds as if the criticism was unwarranted.
| galaxyLogic wrote:
| Maybe the best thing about models is not the numbers they produce
| but the insight into how different variables affect each other,
| how steeply.
|
| But this information is already contained in the model itself.
| Therefore people should be reading the models, not their results.
|
| The models should of course be verified by comparing their
| results to empirical data. But that does not often exist with
| global things like pandemics and climtae change.
| cousin_it wrote:
| > _Journalists must get quotes from other experts before
| publishing_
|
| No, this isn't enough. This whole way of thinking isn't enough.
| It's a big part of the reason for the current situation.
|
| Journalists should report what's true, not what Tom, Dick or
| Harry said. If a journalist isn't qualified to make object-level
| claims on a given topic, don't write on that topic. For example,
| if Bob says there's a forest fire, then instead of publishing
| "Bob says there's a forest fire", you must do enough legwork to
| tell your readers "There's a forest fire" or "There isn't".
|
| I allow myself to ignore all journalism that don't follow that
| guideline, and it makes me happier.
| acqq wrote:
| > Journalists should report what is true, not what Tom, Dick or
| Harry said.
|
| Especially they should _not_ give 33% plausibility to Tom, 33%
| to Dick and 33% to Harry. Which is what they typically do, and
| call that "professional journalism."
|
| If Tom represents 95% of scientists and Dick and Harry the
| fringe 5% also financed by (let say) tobacco industry or oil
| corporations, or Boeing, or those paid by the CIA, they should
| _not even mention Dick and Harry_ in the same article (or TV a
| show), especially not in anything worth a major headline. They
| should appear with the smallest possible note in some smallest
| possible corner and with the title like "oil corporations paid
| these persons to support them again."
|
| Sadly, but that sounds like a dream. The world would be very
| different then.
| sgustard wrote:
| Um, so if you want to publish an article on how many people
| will die of the virus, how do you ensure it's true? Obviously
| you either don't write the article, or you quote the most
| reliable projections you can find. But those are still
| projections.
| gpm wrote:
| You can't honestly write "an article on how many people will
| die of the virus" which gives a number. No one knows.
|
| What you can do is write an article about projections of how
| many people will die. In it you talk about the data sources
| being used in the model(s), the assumptions being made in the
| model(s), and the result they give including likely major
| sources of error..
|
| You can put more or less detail depending on your audience.
| But it should always include enough detail that your audience
| understands that it is an estimate based on assumptions and
| likely flawed data, and you should always understand the
| model you are writing about even if you don't explain it.
| rossdavidh wrote:
| So, I basically agree with everything this article says, but it
| seems to miss a basic point. If journalists do what this paper
| suggests, they make less money.
|
| Journalists, and the newsmedia corporations and organizations
| that employ them, don't run with the most inflammatory headline
| possible as an accidental fluke of a mistake that they were too
| careless to catch. Even public sector newsmedia organizations use
| measures of how widely read their articles are, as a figure of
| merit to how well they are performing. Private sector newsmedia
| are rewarded financially in more or less direct proportion to how
| widely read (or at least clicked on) their articles are, not how
| well informed the reader is after they're done reading it (if
| they even do read past the headline).
|
| If there is one less to be learned from this whole Covid-19
| debacle (and I'm sure there are several), it is that our entire
| news ecosystem, public and private, is fundamentally structured
| wrong for doing what is supposed to be its purpose, which is to
| make people better informed. It's not bad at it by mistake, it's
| bad at it as an inevitable consequence of its design.
| sah2ed wrote:
| > _If there is one less to be learned from this whole Covid-19
| debacle (and I 'm sure there are several), it is that our
| entire news ecosystem, public and private, is fundamentally
| structured wrong for doing what is supposed to be its purpose,
| which is to make people better informed. It's not bad at it by
| mistake, it's bad at it as an inevitable consequence of its
| design._
|
| You speak of a failure in design as the root cause of the
| failure to achieve the purpose of "to inform".
|
| From a teleological perspective, viewing the media as a tool
| _designed_ "to inform" would be a mistake, it would amount to
| nothing more than a supposition.
|
| Having "better informed" people was never the stated purpose of
| mass media so it would be wrong to say it is not fulfilling its
| design objectives. What would be applicable here to help us
| better understand the _actual_ purpose of the media is POSIWID:
| _"the purpose of a system is what it does"_ [0]
|
| [0]
| https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...
| rytill wrote:
| > POSIWID
|
| Can we either make a reasonable-sounding word or phrase that
| means the thing or just say the thing? Who is going around
| really saying "POSIWID"?
| perl4ever wrote:
| "From a teleological perspective, viewing the media as a tool
| designed "to inform" would be a mistake, it would amount to
| nothing more than a supposition."
|
| I and perhaps many other people think the media _should_
| inform people; that is a statement of its purpose, and it
| appears that the collective actions of people in society have
| redesigned it to eliminate most of the informing.
|
| It doesn't have to have ever been designed to inform by
| specific and aware human intention; if it informed as a side
| effect of being unoptimized for disinformation, then in that
| sense it was designed to inform.
|
| By analogy, natural foods are not _designed_ , in one sense,
| to keep you healthy, as evolution can make some things good
| for you and others bad.
|
| ...but the purpose of eating them is healthy nutrition, and
| if we started systematically only eating the toxic ones, then
| we would say something about our society is structured wrong
| for the purpose that we have of getting nutrition.
| sah2ed wrote:
| > _I and perhaps many other people think the media should
| inform people; that is a statement of its purpose, and it
| appears that the collective actions of people in society
| have redesigned it to eliminate most of the informing._
|
| Your expectation that the media should inform people is
| reasonable but unrealistic.
|
| Today we have newspapers, radio, TV and of course the
| Internet as tools of (mass) media. The designers had fairly
| specific design goals in mind and it did not explicitly
| include "to inform", this can only be tacked on.
|
| If you were among the original designers of the
| newspaper[0], which was the first tool of mass media for
| centuries, but your preference to inform was out-weighed by
| competing interests (aka politics), then of course you are
| well within your rights to be outraged -- you are entitled
| to criticize the design for falling short of _your
| expectations_.
|
| In other words, unless you were personally part of the
| design team, you can't really speak about what the design
| _should be_ or _should have been_ , since you are in no
| position to influence the purpose one way or another. You
| can only deal with the consequences of the designers'
| creation.
|
| [0] _News was highly selective and often propagandistic.
| Readers were eager for sensationalism, such as accounts of
| magic, public executions and disasters; this material did
| not pose a threat to the state, because it did not pose
| criticism of the state._ culled from:
|
| https://en.wikipedia.org/wiki/History_of_newspaper_publishi
| n...
| kerkeslager wrote:
| If you're smart enough to come up with a point like this, why
| not use that intelligence to make a less inane point? You're
| not wrong, you're just correct in a way that responds to a
| very narrow reading of what your parent comment was saying,
| and doesn't make any attempt to figure out why they'd be
| saying it.
|
| Clearly the person you're responding to wants a system that
| makes people better informed, and I think there's a lot of
| other people (myself included) who want the same thing. So
| the question is obviously, how do we change the news
| ecosystem from what it is, to a system that makes people
| better informed?
| monocasa wrote:
| And particularly, as the concept applies to mass media, quite
| a few have noted that the purpose seems to be to create the
| consent of the populace for what the leadership wants to do.
|
| Sometime's that's informing from a real benevolent
| perspective of let's get everyone on the same page for what's
| best to the best of our knowledge. Other times it's an
| obfuscation scheme to allow some to profit off of the rest of
| us.
|
| https://en.wikipedia.org/wiki/Manufacturing_Consent
| mr_toad wrote:
| > the purpose seems to be to create the consent of the
| populace for what the leadership wants to do.
|
| Having spent a lot of time working for and with governments
| I can assure you that most of the time, behind closed
| doors, they truely hate the press, and wish it would go
| away. And this is true of even the most open and democratic
| governments.
|
| Autocratic governments, again and again have shown what
| they think of a free press.
|
| The idea that mainstream media is a mouthpiece of
| government just doesn't add up.
| monocasa wrote:
| That's not an issue unless you think of leadership as a
| homogeneous blob. Of course they don't like the portion
| of the media pushing the other guy's cognitive
| structures; and ultimately that's what they mean when
| they complain about the media.
|
| The way you can particularly see this is in the
| suppression of news that doesn't ultimately help out
| _any_ of the current leaders, only the populace.
| jbreckmckye wrote:
| A small (but important) correction: generally, journalists do
| not write headlines; sub-editors do. Most subs have been
| journalists but most journalists do not become subs.
| marcus_holmes wrote:
| I think the distinction is moot. They all work in the
| newsroom, and are that side of the "journalistic integrity"
| divide. Subs are not part of the management team, or
| commercially motivated.
| chris_f wrote:
| Exactly. I stated calling systems that work this way "Bad by
| Design."
|
| Sometimes the design is deliberately setup that way by an
| individual or group with something to gain, but other times it
| is just because the system contains different parties with
| competing interests.
|
| Anytime I see a clearly inefficient or ineffective system the
| first question I try to figure out is whether it works that way
| by design. In many cases once I learn more about it, the answer
| is yes.
| alharith wrote:
| I don't think the deficiencies of major news organizations have
| ever been as blindingly obvious to me as they have been over
| the past month. I knew it was bad. I didn't realize exactly how
| bad it was. For them, even a national crisis is just an
| occasion to twist against Trump. Yes Trump is far, far, far
| from perfect, but there's a huge difference between honest,
| factual reporting and these transparent efforts to use a
| tragedy to take him down. Nothing more obvious than the attempt
| to pit Dr. Fauci as somehow being "against" Trump or being
| "silenced" only for him to come on national radio and set the
| record straight.
| parasubvert wrote:
| The anti-Fauci stuff is mostly posted by GOP-partisan
| organizations though as a way of bolstering Trump.
|
| On the other hand, it is pretty standard mainstream news when
| in the same presser, the doctors are saying the opposite of
| what the President is saying.
| alharith wrote:
| This is simply not true. MSNBC, NPR and PBS ran stories,
| podcasts, and discussions for an entire week about the
| "growing disconnect" during this time.
| [deleted]
| MaxBarraclough wrote:
| > If journalists do what this paper suggests, they make less
| money.
|
| This is clearly true, but I wonder if must always be so.
|
| In academia, lying in a publication can be career-ending, which
| acts to broadly 'scare them straight'. For academics, it's
| desirable to be highly read and cited, but to be seen to be
| dishonest is the end of the road. Could it be possible to
| create similar incentives for journalists?
| sbierwagen wrote:
| >In academia, lying in a publication can be career-ending,
|
| _Can_ be. Among others, Matthew Walker still has a job.
| https://yngve.hoiseth.net/why-we-sleep-institutional-
| failure...
| mola wrote:
| It's easy to lie without lying. As a society we replaced any
| sort of value for our value proxy, money.
|
| So truth ethics etc. are all secondary values.
| chrischattin wrote:
| This sounds good in an edgy r/im14andthisisdeep sort of
| way. But in reality, having integrity and ethics in
| business is actually more profitable over the long term.
| kick wrote:
| The two richest people on the planet right now are both
| known for being almost entirely void of morals when it
| comes to their business dealings.
| SpicyLemonZest wrote:
| They're known for this because both activists and
| entrenched business interests benefit from promoting that
| narrative. Gates and Bezos both regularly talk about the
| impact they have on the world and why that matters more
| than just money - there are reasonable arguments that
| they've done bad things, but the idea that they're
| completely amoral voids is just silly.
| JadeNB wrote:
| > > It's easy to lie without lying. As a society we
| replaced any sort of value for our value proxy, money.
|
| > But in reality, having integrity and ethics in business
| is actually more profitable over the long term.
|
| Both your parent's view and your view could spring from a
| belief about how the world is, rather than from non-
| anecdotal hard data. I'd like to believe the more
| optimistic version, but I find it hard to do so. Do you
| have some data to support it?
| wayoutthere wrote:
| That really depends on the size of the fish and the size
| of the pond. If you're a whale in an ocean of minnows
| (hello Amazon) you can effectively dictate the terms of
| business, ethics be damned.
| JadeNB wrote:
| > In academia, lying in a publication can be career-ending,
| which acts to broadly 'scare them straight'. For academics,
| it's desirable to be highly read and cited, but to be seen to
| be dishonest is the end of the road. Could it be possible to
| create similar incentives for journalists?
|
| The problem is that these incentives for academics aren't
| created out of purity of heart, but because of the system to
| which they belong; you will no longer be highly read and
| cited once it is clear that your results can't be trusted.
| Unfortunately, this seems not to be true in journalism--you
| can purvey intentionally, and explicitly, wrong information,
| and it will still be consumed actively by those whose biases
| are confirmed by it. We can discuss how to change _that_ ,
| but changing what kind of news people want to read is surely
| even harder than changing what kind of news journalists write
| and publishers publish.
| hutzlibu wrote:
| Well, also as a journalist you can not get away with making
| up stories. There was a recent case of a high class
| journalist, who did that and was caught.
|
| https://en.wikipedia.org/wiki/Claas_Relotius
|
| But he did so very brutal so to say. The problem with
| journalism is mostly not straight lying, but missleading
| and bending the truth until it fits the agenda. So a
| classic journalist should not have another agenda than the
| truth. But this type seems to be very rare today.
| em500 wrote:
| Best we can tell, Bloomberg's Chinese Big Hack story was
| completly made up, but it doesn't appear anyone involved
| suffered any consequences.
|
| https://appleinsider.com/articles/19/10/04/editorial-a-
| year-...
| DangitBobby wrote:
| What you're saying is probably broadly speaking true for
| journalists in general. But what many people actually
| consume news-wise can only be compared on the surface to
| actual journalism. Consumers of this material have no
| interest in determining if what they're reading is true,
| and there are no consequences for those spreading
| misinformation while pretending to be journalists.
| https://news.ycombinator.com/item?id=22784665
| wayoutthere wrote:
| This doesn't really solve the Fox News problem: if an
| audience agrees with what you're saying, they will begin to
| trust you regardless of the facts. Many people have no way of
| knowing what the truth really is when they get their news
| exclusively from the a small group of propagandists
| coordinating a message.
|
| Who gets to decide when someone is lying? What happens when
| that power to determine truth from lies is obtained by a bad
| actor to create a new truth?
| sgustard wrote:
| I get why corrupt politicans hate the media, which is because
| it holds them accountable. Why do you choose to help them tear
| it down? Are we better off with no media and just the raw
| tweets of our leaders?
| renewiltord wrote:
| The problem is on the demand side, not solely on the supply
| side. People desire reinforcement not challenge. That is
| natural. I suspect it is true of us all. Those who are capable
| direct their potential to be challenged in specific ways,
| allowing themselves to have biases reinforced in the
| unimportant places. And they have access to the knowledge in
| their specific areas.
|
| This is the reason for the enduring power of the filter bubble:
| it is a stable equilibrium because it serves both the purposes
| of the demand and supply side.
|
| You can test this by making high-reliability websites that
| state honest priors. You'll get an audience but the audience
| will be pretty specific to your subject matter and not be
| popular. No information source has had all of the following
| characteristics:
|
| * Broad-based popular support
|
| * High information content
|
| * Novel information, i.e. information you can't get elsewhere
|
| * Sustained presence
|
| This may actually be desirable. Novel reliable information is
| an advantage, but it may not be a present sufficient advantage,
| and species survival may depend on presently boosting those
| capable of acquiring and utilizing information advantage. i.e.
| a time may come when we need to be good at it - if we have more
| people with this characteristic then, it'll lead to better
| outcomes.
| Viliam1234 wrote:
| The problem is also in the change of how the demand can
| manifest. For example, I would never buy a clickbait
| newspaper. But sometimes I click on clickbait articles
| online. In both cases, there is a part of me that is
| interested, and a part of me that knows better. If it only
| requires a mouse click, the interested part sometimes wins.
| If it requires taking out my purse, the interested part loses
| this fight.
|
| The horrible thing about online advertising is that it allows
| people to make profit from making you look at something, even
| if it immediately makes you disappointed. From that moment,
| the need to write non-disappointing articles has decreased
| significantly.
| amitdeshwar wrote:
| What about 538?
| jlawson wrote:
| Definitely true. We could compare it to the food industry
| (from production to retail including restaurants).
|
| Its ideal purpose is to provide people with nourishing,
| healthy, enjoyable food.
|
| But its actual incentive is to give people the food they
| choose to buy, which often isn't nourishing or healthy.
| marcus_holmes wrote:
| This is true. You can do good journalism and lose money (or
| have a wealthy backer who funds your loss-making newspaper)
| or you can make a profit by getting your "journalists" to
| publish lots of sensationalist stories. Y'know, Clickbait.
| (source: I ran a newspaper).
| watwut wrote:
| The majority of publications and articles did not got it
| fundamentally wrong. There is subset on right wing that is
| popular and got it wrong intentionally. But mainstream had bad
| article here and there while majority being not fundamentally
| wrong.
| lonelappde wrote:
| Thia isn't specific to news. It's everything in the
| marketplace. Anything for sale is optimized for short term
| profit -- lowest cost of production, highest impulse purchase
| pressure -- junk food, fake healthy food, furniture made of
| shoddy materials, shiny appliances that will break, etc.
| smitty1e wrote:
| One feels that, irrespective of the models, the data in the
| covid-19 case may be unusually bad.
|
| It may be time to add a third error category[1]
|
| I. False positive
|
| II. False negative
|
| III. Deliberately skewed off the map for propaganda reasons.
|
| [1]
| https://en.m.wikipedia.org/wiki/Type_I_and_type_II_errors#Ty...
| YZF wrote:
| This is driving me crazy ;) the poor quality of data.
|
| You'd want health organizations around the world to be
| publishing every possible detail (anonymized) so that the
| disease can be better understood. Yet three months in, with
| over a million cases worldwide, we still have experts
| disagreeing about things like asymptomatic transmission, use of
| masks, droplets vs. aerosol, how much distance one should stand
| from another, viability on surfaces, etc. etc. Even for
| treatment options rather than insisting on randomized double
| blind trials start by using the natural experiments that are
| already happening.
|
| We should have the data to answer a lot of these questions (or
| at least draw out some probability distributions), or at least
| someone has it. This stuff is going to be critical in informing
| exit strategies.
| magoghm wrote:
| My experience with data is: you never get good data. Always,
| the closer you look at it the more problems you find in how
| it was collected.
| thaumasiotes wrote:
| > You'd want health organizations around the world to be
| publishing every possible detail (anonymized)
|
| If you're publishing every possible detail, then your data
| isn't anonymized. Anonymization consists of the removal of
| almost all of the details.
| smitty1e wrote:
| Or even if the data are reasonably tidy, they might expose
| more when juxtaposed with another dataset.
| nitrogen wrote:
| _Would we encourage an epidemiologist to apply 'fresh thinking'
| to the design of an electrical substation?_
|
| Yes, absolutely. If an epidemiologist identifies and models a
| trend in human disease around substations, or a trend in failures
| of substations, or a new way of modeling the ways electrical
| demand can change over time and influence demand in other times
| and places, etc., then their input should absolutely be
| considered.
|
| Just as it's annoying when a total outsider claims to know
| everything about a field, it is equally problematic when insiders
| refuse to acknowledge anyone on the outside.
| m0zg wrote:
| I'd like to point out that we _still_ don't know if that "high
| R0" model was right or not. And we won't know that until we
| randomly test a sufficiently large random sample of the UK
| population for antibodies. They imposed containment _very_ late,
| and they go to pubs _all the time_. It is not implausible that
| the majority of their population already had COVID19 without even
| knowing what it was.
|
| Also, the models that just last week were predicting 50K beds
| needed in NY are off by a factor of 3-4, and hospitalization are
| starting to flatten out already. You can guess the direction they
| were wrong in. In the meanwhile NY hoarded the ventilators and
| medical supplies because it anticipated this prediction to be
| true. To be clear, I don't blame NY - they used the best
| information they had, which turned out to be bullshit.
|
| These are not harmless errors. When this is over, someone should
| study these fiascos and estimate the death toll just from bad
| models alone.
|
| https://twitter.com/AlexBerenson/status/1246465515704463360
| martingoodson wrote:
| Author here: happy to take comments or criticism
| tomcatfish wrote:
| I very much enjoyed this article you linked with the Times
| reporter and the law professor
|
| https://www.newyorker.com/news/q-and-a/the-contrarian-corona...
| svat wrote:
| This is a great start on how scientists (and journalists)
| should communicate to the public! Thanks for writing this, and
| the world would be a better place if everyone remembered to
| follow these principles.
|
| I think in fact it would be better to go even further: not
| speak from a position of superiority (even if one knows more),
| but try to acknowledge the audience and persuade effectively.
| Here's a recent article on the topic:
| https://undark.org/2020/03/19/coronavirus-myths/ and here's one
| of my favourites (in a different field of science) from three
| years ago: https://deansforimpact.org/why-mythbusting-fails-a-
| guide-to-...
| YZF wrote:
| I'm finding myself in disagreement with rule #6. Using a model
| effectively is about a lot more than just the domain knowledge.
| I'd value analysis from a mathematician/statistician more
| highly than from an infectious disease physician. There's the
| stuff that informs models, i.e. the observations, the
| experimentation etc. and then there's the science of modelling
| itself which isn't really in the same domain.
| martingoodson wrote:
| I agree with this - but I do think it should be clear that
| the model is from outside the mainstream. Not to dismiss it
| but to clarify its status. Check out the New Yorker piece I
| link to in the article - it's quite shocking the
| misinformation that's out there.
| garmaine wrote:
| Well, we could have benefited greatly from the mainstream
| media and politicians taking the outside predictions
| seriously at the beginning of this crisis. Instead we had
| to wait a month for the Imperial College London to say the
| same exact thing before certain leaders got their heads out
| of the sand.
|
| Likewise now with hydroxychloroquine--if you listen to the
| epidemiologists all you'd hear is how it's an UNPROVEN
| drug. What we need instead is coverage of sample sizes, p
| values, bayesian predictions of effectiveness (in the
| absence of controlled studies) and serious modeling of the
| number of ICU beds and ventilators required with and
| without various levels of treatment, from emergency care to
| prophylactic use.
|
| The epidemiologists have their head in the sand and think
| we can just wait 6 months for a proper set of randomized
| trials. It's the less attached data modelers you need to
| turn to get predictions that are useful for effective
| policy choices.
| martingoodson wrote:
| That's not really a fair representation. (Harvard
| epidemiologist) Marc Lipsitch raised the alarm back in
| February: "it's likely we'll see a global pandemic" of
| coronavirus, with 40 to 70 percent of the world's
| population likely to be infected this year."
|
| https://thehill.com/changing-america/well-
| being/prevention-c...
| oivey wrote:
| Don't discount domain experts. Models are meant to predict
| the real world. In order for them to be accurate, the model
| itself needs to capture how the real world works, and the
| math underlying the model has to be correct. Domain experts
| are the most likely to have experience in both areas. Drawing
| an example, a physicist models the universe and knows the
| math and model behind electromagnetism. A mathematician
| probably knows the math but maybe not the model.
| skat20phys wrote:
| I'm at the point where it shouldn't matter who did it, what
| should matter is its integrity. I'd prefer everything was
| anonymously posted at some level so author background
| didn't go into consideration regarding how it was received.
|
| I agree with you completely, but the flip side of the coin
| is that experts can be blinded by assumptions that the
| field has. Sometimes outsiders aren't aware of these basic
| assumptions and so are less biased.
|
| There's also the simple issue that sometimes expertise
| comes from places you least expect for reasons you might
| not anticipate.
|
| For me there's as many problems in this pandemic related to
| appeals to authority (at least in the US) -- problems with
| testing related to FDA regulation and the CDC, problems
| with lack of healthcare providers due to long-term rent-
| seeking monopolies in licensing and practice scope,
| problems related to academic fraudulence and incentives
| (see: Didier Raoult) -- that I think it's dangerous to
| raise appeal to authority as anything but a bias.
|
| For me there's multiple levels of problems to this, the
| first of which is the conspiracy and anti-science culture
| surrounding the pandemic. Above that is an appeal to
| medical and scientific expertise and authority that has
| sometimes been helpful but sometimes harmful. Above that
| still is an appeal to rigorous thinking and risk
| management, which transcends expertise boundaries.
| guscost wrote:
| If any of the scenarios from the famous Imperial College model
| turn out to have been based on just-as-bad assumptions, would
| you be willing to write a follow up about that?
| martingoodson wrote:
| I'm not taking a position on the Imperial College model. I'm
| explicitly advocating that _all_ models should have their
| assumptions examined. And that policy makers should use a
| range of model and not depend on just one.
| gadders wrote:
| Why do you think the Oxford model compares poorly to the
| Imperial one? Both are created by eminent people in their
| field. Neither has been peer-reviewed.
| guscost wrote:
| You will have nothing to add if the "2.2 million deaths in
| the US" scenario, which was blasted across every newspaper
| front page a few weeks ago, turns out to have been
| impossible all along?
|
| If that scenario was "completely wrong" too, it seems like
| it would serve as a perfect example of the _consequences_
| of this kind of (still hypothetical) misinformation.
| watwut wrote:
| Wasn't ot explicitely worst case? No measures taken and
| so on? The imperial articles are free to download and all
| I have read had assumptions stated very clearly.
| not2b wrote:
| We can't rerun the experiment with a control version of
| the US in which nothing was shut down and we continued to
| have crowded sports events and night life. So it won't be
| possible to determine that the 2.2 million deaths
| scenario is impossible, especially if it's interpreted as
| 2.2 million extra deaths from either COVID-19 or other
| causes that could have been treatable by a medical system
| that wasn't completely overwhelmed.
| garmaine wrote:
| The final point seems like a bunk attack. A lot of
| epidemiologists were upset at Silicon Valley data scientists
| and hedge fund quants who were putting together prediction
| models that disputed what the government was saying. Well, so
| far these models have been more accurate than the official
| ones.
|
| As we should expect, because data scientists and quants are
| EXACTLY the people with the set of skills necessary to make
| such predictive models. At best, it's an ancillary skill for
| epidemiologist, and we've seen many cases in this pandemic
| where they have wielded these tools incorrectly.
| etangent wrote:
| Your entire post seems to be about #6. And therefore it's
| entirely wrong.
|
| Given that "people with background in infectious diseases" have
| largely failed, as a group, to warn us in January-February
| about this pandemic (see this Twitter thread for a slew of
| concrete examples
| https://twitter.com/RokoMijicUK/status/1246509433145917443), my
| conclusion is that unless someone has a background in hard
| quantitative field ( _regardless_ of what that field is), that
| person should not be let anywhere near quantitative models.
| dwohnitmok wrote:
| This is a hard problem, mainly because of the problem of
| motivated reasoning as mentioned by another comment.
|
| You're requiring cooperation from multiple different parties
| here (scientists, journalists, policy makers, readers, etc.)
| and any of these parties can warp the results in any number of
| ways regardless of the cooperation of other parties.
|
| Climate science still hasn't solved this problem despite trying
| to implement what you're talking about.
|
| It's an age-old problem, if a priori you're looking for
| something hard enough you're bound to find it.
| martingoodson wrote:
| I think, at the least, if journalists get some quotes from
| other scientists before publishing a piece on a new model it
| would be a major win.
| TwoBit wrote:
| Journalists and their publications don't give a crap about
| scientific truth. They care about maximizing clicks and
| traffic. If a group of scientists give them the truth and
| another group gives them a more sensational story, they'll
| print the latter. Scientists will never win this battle.
| dwohnitmok wrote:
| That's fair. I mean asking folks to be more circumspect in
| general is probably a good thing.
|
| I prefer though to look at problems through the lens of
| incentive structures (keeping in mind humans generally
| _heavily_ time discount incentives and what incentivizes
| people is not always obvious! Death isn 't always much of a
| disincentive beyond a rather short time horizon). And here
| I'm having a hard time seeing easy ways to tweak the
| incentive structure.
| thaumasiotes wrote:
| Often they do, but that doesn't affect the piece they
| write. It's not at all difficult to find people complaining
| "I was interviewed for this article, most of what I said
| was left out, and to the extent I am quoted, it's to give
| the impression that my beliefs are the exact opposite of
| what I explained to the journalist at length".
| excalibur wrote:
| Asking more of journalists at this juncture may be a tall
| order. The bar for them has been trending downward, due
| largely to a lack of funding and an abundance of
| competition, many of whom have little regard for
| journalistic standards.
| guscost wrote:
| I don't think there's value in doing this, _unless_ the
| second opinion disagrees with the original model. It's not
| hard to find a second "expert" to agree with just about
| anything, if you look hard enough.
|
| The number of people who agree with something tells you, as
| a rational person, practically nothing - it reminds me of
| the absurd compilation argument "One Hundred Authors
| Against Einstein": https://archive.org/details/HundertAutor
| enGegenEinstein/mode...
|
| Einstein's famous reply: "If I were wrong, then one would
| have been enough!"
| scribu wrote:
| Reading the examples, where people jumped on a single mistake to
| discredit an entire report points to some sad conclusions:
|
| 1. Scientific literacy is super low in the general population.
|
| 2. Motivated reasoning is rampant. People will believe anything
| that enables them to do what they wanted to do anyway.
| xg15 wrote:
| > _2. Motivated reasoning is rampant._
|
| This is an important point. Scientific scrutiny is extremely
| important, but there is still a difference between a judge that
| is stern but fair - and one that actively wants you to fail.
|
| Motivated reasoners have no problems holding opposing parties
| to impossibly high standards while accepting claims without
| _any_ evidence as valid arguments for their side.
|
| Today, climate scientists have learned the lessons and improved
| communication and modeling considerably, even to the point
| where we now how "attribution science" we we can discuss
| climate change in the context of _particular weather events_.
| We also start seeing changes in weather patterns that are hard
| to ignore even for laymen.
|
| Nevertheless we are still having the same discussions as
| before.
| lopmotr wrote:
| > We also start seeing changes in weather patterns that are
| hard to ignore even for laymen.
|
| Hard to ignore by motivated reasoners! Changes in weather
| patterns observed by individuals using their own experience
| are not evidence of global climate change. If there's science
| showing that, then yes, but personal experience doesn't add
| any value to the conclusion, it just reinforces whatever the
| person already wants to believe.
| chrisco255 wrote:
| I think the author oversimplifies the problems with climate
| models. They've had numerous mistakes and extremely critical
| ones. Not least of which is that they haven't factored for
| Multi-decadal changes in cloud cover albedo. The climate system
| is incredibly complex and our models do not have good models
| for all the subsystems that make it up, including the oceanic
| oscillations like Atlantic Multi-decadal Oscillation, PDO, etc.
| We can't even predict many of these subsystems with any degree
| of accuracy so we are hopelessly inaccurate at the higher
| level.
| djsumdog wrote:
| I agree, and there are the same politician leanings in
| climate change non-profits and think tanks as with anything
| else. Sometimes they may justify sightly more alarming views
| of data to gain more funding; justifying it with, "Well, it's
| going to be bad anyway"
| [deleted]
| cassiet wrote:
| The headline really cuts off the points nose.
| martingoodson wrote:
| Yeah - I was trying to play off the George Box quote but maybe
| it's too obscure
| https://en.wikipedia.org/wiki/All_models_are_wrong
| KerryJones wrote:
| Interesting, I did _not_ know the reference (I'm a SWE, not
| in scientific community), and I thought it was a clickbait
| headline. Obviously this differs from the person above me ^
| who clicked the link _because_ of the reference.
|
| Just perspective, and I'm very happy to know the reference
| now.
| magoghm wrote:
| The reference to the George Box quote in the title is what
| originally got my attention.
___________________________________________________________________
(page generated 2020-04-05 23:00 UTC)