[HN Gopher] Time to assume that health research is fraudulent un...
___________________________________________________________________
Time to assume that health research is fraudulent until proven
otherwise?
Author : lnyan
Score : 447 points
Date : 2021-07-19 16:12 UTC (6 hours ago)
(HTM) web link (blogs.bmj.com)
(TXT) w3m dump (blogs.bmj.com)
| MauranKilom wrote:
| This article talks about "zombie trials" as if it were known to
| the reader what exactly that means. Anyone who could clue me in?
| danuker wrote:
| Following the reference trail leads to this:
|
| > I categorised trials with false data as 'zombie' if I thought
| that the trial was fatally flawed.
|
| https://associationofanaesthetists-publications.onlinelibrar...
| nabla9 wrote:
| >I have chosen the word 'zombie' to indicate trials where false
| data were sufficient that I think the trial would have been
| retracted had the flaws been discovered after publication. The
| varied reasons for declaring data as false precluded a single
| threshold for declaring the falsification sufficient to deserve
| the name 'zombie',
|
| 1. Carlisle JB. False individual patient data and zombie
| randomised controlled trials submitted to Anaesthesia.
| Anaesthesia 2020; https://doi.org/10.1111/anae.15263.
| argvargc wrote:
| Yes, though it's not much of a problem for those unwilling to
| take the leap based on only one or two emergent studies, who
| might prefer relying on decades of safety data.
|
| Oh, but then that entirely sensible and understandable action
| might get one censored, deleted, fired, fined and/or
| discriminated against as a third class citizen. Cui bono?
| tasogare wrote:
| Exactly. I've argued on social media with "trust the science"
| people: none of them were scientists while I am one. People in
| general have no idea how the sausage is made and how broken the
| system is. A year or more ago I was joking with a coworker "I
| hope medical studies are done more seriously than what we do".
| Today I wouldn't make the same joke given how the situation
| went out of control with states enforcing authoritarian
| policies based on poor understanding of how science work.
| supperburg wrote:
| People cite paper titles like they are facts. Nobody even knows
| whether or not it was an epidemiology study or an interventional
| study, they just say "they did a study showing that underwater
| basket weaving lowers your risk for colon cancer!" Nobody
| actually reads the studies.
|
| During a debate or conversation, many people cite studies in
| support of their point of view. This is a problem because now the
| other person is swamped with a dozen studies to analyze and
| debunk before proving he's right. And academia is producing huge
| volumes of these bullshit studies so no matter how you slice it,
| a huge unnecessary burden has been created of digging through all
| of them and circling the flaws.
|
| Thankfully, there is an emerging cultural mechanism to deal with
| this in the growing "epidemiology is bullshit" sentiment. This is
| good because it reduces the bulk of bullshit that will ultimately
| need to be processed and debunked. If the study is epidemiology
| just cross it out by default. Those studies need to burn in hell.
| Shine the light of day on them and brandish the holy crucifix
| whenever you see one.
|
| The only thing worse than a science denier is a person who
| blindly parrots study titles without ever reading the body of the
| paper let alone understand it. People complain endlessly about
| armchair scientists who are spreading misinformation based on
| their uneducated assessment of scientific data. And the people
| who complain about this are always the same people who cite
| studies that they don't understand like complete idiots,
| spreading misinformation just as widely.
| wolverine876 wrote:
| > there is an emerging cultural mechanism to deal with this in
| the growing "epidemiology is bullshit" sentiment
|
| There is an emerging cultural mechanism - more a rampaging mob
| - that says 'X is bullshit' as a simple way of denying facts or
| issues that are inconvenient or difficult. It's used for the
| news media, academia, non-partisan government agencies (e.g.,
| the CDC), etc., etc. and for everyone who disagrees.
|
| I say this social mechanism is bullshit - the sources they
| disregard come with plenty of evidence, saying they are
| bullshit comes with none - it's just easy to say.
|
| It's also very destructive. Where do we get our epidemiology or
| news or whatever else if anyone can claim anything is bullshit
| at any time, halting everything until they are proven wrong?
| It's up to them to prove their claim right; we can't all halt
| and freeze in place every time someone makes the minimal effort
| to vocalize, 'X is bullshit'. Without evidence, their claim is
| meaningless and should be ignored.
|
| Epidemiology, the imperfect human institution it is, provides
| many successes.
| adolph wrote:
| Is this some vaccine hesitancy in disguise from BMJ?
|
| _Stephen Lock, my predecessor as editor of The BMJ, became
| worried about research fraud in the 1980s, but people thought his
| concerns eccentric. Research authorities insisted that fraud was
| rare, didn't matter because science was self-correcting, and that
| no patients had suffered because of scientific fraud. All those
| reasons for not taking research fraud seriously have proved to be
| false, and, 40 years on from Lock's concerns, we are realising
| that the problem is huge, the system encourages fraud, and we
| have no adequate way to respond._
| artifact_44 wrote:
| That was my first thought as well. Casting doubt, and then in a
| few days we'll see this opinion piece references by some right
| wing politician who is vaccinated but "can understand their
| constituents hesitency to trust the medical establishment".
|
| Also I haven't seen anyone bring up the perverse incentive of
| capitalism in this thread. Like.. let's just pretend this is
| moral failing rather than the literal race to the bottom of
| capitalism.
| adolph wrote:
| Perverse incentives exist in many socio-political systems,
| capitalism isn't unique in that regard. For example, the
| Soviet whaling industry [0] sounds almost like the paperclip
| maximization AI [1].
|
| 0. https://en.wikipedia.org/wiki/Whaling_in_the_Soviet_Union_
| an...
|
| 1. https://en.wikipedia.org/wiki/Instrumental_convergence#Pap
| er...
| zucker42 wrote:
| What? How do you figure that? Some person points out actual
| instances of poor quality research and you imply he is anti-
| vaccine? I can find no record of this person discouraging
| vaccination.
| adolph wrote:
| Ok, maybe its just a coincidence that it was published right
| when the current dangers of misinformation are so high and so
| many are failing to believe science.
| wizzwizz4 wrote:
| The solution to "people aren't believing academia" is to
| make academia more trustworthy (probably by filtering for
| trustworthiness rather than making individuals more
| trustworthy), not to encourage scientism. Seeking the truth
| is still important, after all.
| zucker42 wrote:
| Well of course I'm not going to believe blatantly
| fraudulent "science". That has nothing to do whether or not
| I would take a COVID vaccine (I have). Trying to suppress
| the truth that there is shoddy or false research to keep up
| appearances is ridiculous. There's something between
| ignoring scientific evidence you dislike and credulously
| taking everything a scientist says as fact.
|
| And the "dangers of misinformation" have been high for a
| long time now (perhaps since antiquity). Modern climate
| change denial has its roots in the 90s, and modern vaccine
| skepticism is similarly old.
| adolph wrote:
| What is your plan for identifying the "blatantly
| fraudulent?"
|
| How do you know your medical provider agrees with you
| about what is "blatantly fraudulent?"
| joshuaissac wrote:
| > What is your plan for identifying the "blatantly
| fraudulent?"
|
| One way is to assume that all health research is
| fraudulent, unless proven otherwise through replication
| by multiple unrelated parties.
| faeriechangling wrote:
| >failing to believe science
|
| Believing in science means believing in falsification and
| sceptism. This was published during the ongoing replication
| crisis in medicine where we're finding that more than half
| of cancer studies don't replicate [1]. This replication
| crisis hasn't been put on hold and all science is now
| deemed irrefutable or you are VACCINE HESISTANT and
| DANGEROUS.
|
| I'm mostly confident in the COVID vaccines because we're at
| billions of doses and there is so much uncorrelated data
| about the vaccines, and they're so controversial and their
| safety/efficacy is being looked at by so many people that
| one can be relatively confident about their efficacy and
| safety (so long as you look at uncensored information to
| avoid systemic bias). We don't need to be science
| denialists and start denouncing scepticism on safety
| grounds, we can point out there are special reasons to be
| confident in the vaccines.
|
| [1] https://www.npr.org/sections/health-
| shots/2017/01/18/5103048...
| adolph wrote:
| don't replicate != fraudulent
|
| _I 'm mostly confident in the COVID vaccines_ <- Is this
| "mostly" qualifier not the dangerous front end of
| misinformation and vaccine hesitancy?
| faeriechangling wrote:
| I've got both vaccines especially early for my age group
| (clinical vulnerability) and I've been lobbying my
| vaccinated peers to at least get one dose, so apparently
| not.
|
| I say "mostly confident" because we have literally zero
| data of effects after 3 years and can only make
| inferences, and it's foolish to feign knowledge you do
| not have to avoid being called "hesitant", but my lack of
| confidence does not make me hesitate to recommend the
| vaccine. I also lack knowledge of long term effects of
| COVID itself which may in fact be worse than the long
| term effects of any vaccine given COVID does most of what
| the vaccines do plus extra nasty stuff.
| adolph wrote:
| You are recommending something over nothing despite not
| being confident in the data about your recommendation?
|
| Which vaccines were in "both vaccines?" An mRNA and J&J,
| or just both administrations of a two-shot mRNA vaccine?
| wizzwizz4 wrote:
| > _You are recommending something over nothing despite
| not being confident in the data about your
| recommendation?_
|
| (Not OP but) Yes. Reasoning despite uncertainty is
| entirely possible; if you want a mathematical formalism,
| look into Bayesian statistics.
| long_time_gone wrote:
| Is there a name for the phenomenon where an internet comment is
| guilty of doing the thing it accuses others of doing?
| adolph wrote:
| also need the name for the one where an accusation is made
| through a question
| hammock wrote:
| It's called transference in the spook world, and you'll see
| it in the media/politics all the time.
| bena wrote:
| This is almost an apples/oranges comparison.
|
| There's research and then there's research.
|
| Yes, all approved medicines are the result of medical research.
| But approving medicines isn't the entirety of medical research.
|
| I'm sure you've heard of the adage: egos are large when the
| stakes are low. This is related to that. The push to "get
| published" is so great that if you know that your research has
| no practical effect or that it won't affect anyone, you can
| kind of make up whatever results you want. As long as it
| publishes. The goal isn't to find the truth or to answer a
| question, the goal is to get a byline in a paper. And you get
| more and better bylines by discovering something radical and
| novel rather than by saying "Nope, doesn't work, just like
| expected".
|
| On the flip side, when the results really matter, you'll find
| people do proper due diligence. Especially when your results
| will be essentially confirmed practically by billions of people
| on the planet. When the stakes are high, we wind up being way
| more cautious.
|
| Of course, I hear your meta-concern. Because, yes, people will
| use this paper to pull the "Science is a lying bitch"* card.
| But it is also an issue that must be dealt with or at the very
| least acknowledged. As the article itself notes, someone did
| notice it in the 80s, but due to very concern of casting doubt
| on medical research, they kind of just hoped it wouldn't be an
| issue. And now the issue is too great to deal with simply.
|
| In the end, medical, and really all scientific, research cannot
| be "hit driven". Failure _must_ be an option. And if this month
| 's issue of The BMJ is a bit "boring" or "thin", then so be it.
| The focus must be more on finding the correct answer rather
| than the flashier answer. Even when the stakes are small.
|
| *Science is a lying bitch: From the It's Always Sunny in
| Philadelphia episode "Reynolds vs. Reynolds: The Cereal
| Defense". One of the characters uses the fact that certain
| noted scientists had incomplete ideas about certain scientific
| phenomenon or weren't completely right on every subject as
| proof that science itself was flawed and couldn't be trusted on
| the subject of evolution and therefore one should believe the
| biblical account of creation because the bible hasn't been
| changed since it was written.
| AnimalMuppet wrote:
| No, this is about _all of medecine_ , not about vaccines for a
| specific illness.
| adolph wrote:
| Are "vaccines for a specific illness" not in the set "all of
| medicine?"
| AnimalMuppet wrote:
| Yes. But "all of medecine" is not in the set "vaccines for
| a specific illness".
| adolph wrote:
| _Affirming the consequent, sometimes called converse
| error, fallacy of the converse, or confusion of necessity
| and sufficiency, is a formal fallacy of taking a true
| conditional statement (e.g., "If the lamp were broken,
| then the room would be dark,") and invalidly inferring
| its converse ("The room is dark, so the lamp is broken,")
| even though the converse may not be true._
|
| https://en.wikipedia.org/wiki/Affirming_the_consequent
| AnimalMuppet wrote:
| You are trying to take a very general statement and make
| it a statement only about one very specific subset. That
| does not mean that someone pointing out your logical
| flaws is falling into the converse fallacy.
|
| It's like someone writes an article about drought in the
| west, and you're saying that they must really be talking
| about El Centro, California. Yes, it probably includes El
| Centro, but it's also talking about Phoenix, and LA, and
| Salt Lake City, because it's really talking about the
| west as a whole. Trying to make it be "about" El Centro
| is missing the point.
| adolph wrote:
| The article is about all of medical research. Vaccine
| development is part of medical research and is therefore
| included. I did not attempt to "make it a statement about
| one very specific subset." I pointed out that the article
| is similar to the kind of article seized upon by people
| who have vaccine hesitancy ideations.
|
| Take your straw elsewhere.
| wxnx wrote:
| I work on large multi-center clinical trials as a machine
| learning engineer. One of my projects involves the semi-
| automation of the detection of fraudulent data.
|
| There's one link in the chain here missing that some people here
| seem to be ignoring. The authors of this post (while entirely
| correct) draw no link between "bad data" (which is doubtlessly
| responsible for a large number of "bad papers"/"bad trials") and
| "bad clinical practice."
|
| I don't know a single clinician who would base their care on the
| findings of a single-center RCT of the kind described in this
| article. Or the findings of a meta-analysis of single-center
| RCTs, for that matter.
|
| Bad data happens in multi-center RCTs too, and in fact that's
| what I'm focused on, but a lot of work already (and therefore $,
| for the cynical) goes into the validation of data (see [1] for a
| brief description). Phase III clinical trials in the west
| practically require a robust multi-center RCT, where systemic
| fraud is very difficult to perform (but not impossible [2]). By
| the time a Phase III trial is conducted, the efficacy of the drug
| can already be estimated, and the focus of the drug company (who
| yes, often fund these trials) is to conduct a trial which is
| unimpeachable in the face of a regulatory board (who are
| generally good at their jobs, although the revolving-door tends
| to reduce public trust and should be legislated away).
|
| In short, I support most of the proposed changes to incentives
| around publish-or-perish. I reject the notion that these
| incentives are (currently) significant drivers of decreased
| quality of standard of care in the West. I think global
| governance structures, as suggested in this article, could
| improve understanding among both clinicians who are not
| necessarily scientists and the general public about just how
| validated a given standard of care is.
|
| tl;dr Most good evidence-based practitioners already think this
| way -- not because they inherently believe fraud is rampant,
| necessarily, but because evidence says the kinds of studies where
| fraud is most prevalent are untrustworthy for other reasons.
|
| [1] doi:10.1177/1740774512447898
|
| [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4340084/
| jonnycomputer wrote:
| I think institutional incentives matter a lot, and the
| reasonably lucrative prospect of careers outside of academia if
| things don't work out. That is perhaps why we see such stark
| regional differences.
|
| No one I know has committed fraud in their research. I've seen
| mistakes in their code however, but that is another story.
| SubiculumCode wrote:
| Some segments on HN have strong anti-scientist sentiments (even
| while they proclaim to be pro-science), assumine we all are
| crooked, stupid or both, hence your insightful and reasonable
| comment being downvoted.
|
| Is there fraud? Sure. Is there a lot of fraud happening in
| American science? I don't think so. To quote the article:
|
| "Many of the trials came from the same countries (Egypt, China,
| India, Iran, Japan, South Korea, and Turkey), and when John
| Ioannidis, a professor at Stanford University, examined
| individual patient data from trials submitted from those
| countries to Anaesthesia during a year he found that many were
| false: 100% (7/7) in Egypt; 75% (3/ 4) in Iran; 54% (7/13) in
| India; 46% (22/48) in China; 40% (2/5) in Turkey; 25% (5/20) in
| South Korea; and 18% (2/11) in Japan. Most of the trials were
| zombies. Ioannidis concluded that there are hundreds of
| thousands of zombie trials published from those countries
| alone. "
| swayvil wrote:
| If lying is the more profitable course then you can trust that
| it's a lie.
|
| Is that capitalism? Is that what we're looking at here?
| UnFleshedOne wrote:
| It's humans. You should look at how research was done in
| communist countries or anywhere with strong ideology. People
| are built to game systems.
| sharikone wrote:
| I was terrified approximately 14 months ago when the big pharma
| companies started to get billions to develop the vaccine. My fear
| was that their research was fraudulent and they would be exposed
| and discredit science as a whole. I am very happy that this did
| not happen.
|
| I guess we are not in a total catastrophe situation but there is
| significant rot.
| throwawayboise wrote:
| That would never happen. Too much political capital is invested
| in the success of the vaccines.
| FL33TW00D wrote:
| This is why I left academia.
| Ajay-p wrote:
| Is the incentive for publishing research that is fraudulent
| primarily money, prestige, or are there just that many professors
| that graduate students required to publish something?
| ghoward wrote:
| My thoughts: https://gavinhoward.com/2019/12/replication-and-
| retraction-c... .
| dleslie wrote:
| Shouldn't all research be considered _tenuous_ until it is
| corroborated by a third-party without financial or social
| connections to the original researchers?
| vadansky wrote:
| Not even that is enough because you might end up with the
| filling drawer problem. For example you might end up with 50
| people trying to replicate it, 49 failing and 1 "succeeding" by
| chance. The 49 won't publish because they probably got the
| "wrong" answer by chance, and they don't want to publish
| negative results, so they will leave it in the filing drawers.
|
| The one that did "replicate" it will publish and then in round
| 2 there is even more pressure not to publish negative results
| because "hey, this was replicated before already"
| NoblePublius wrote:
| "He is now sceptical about all systematic reviews, particularly
| those that are mostly reviews of multiple small trials."
|
| This describes basically all "meta analyses" of Covid claims,
| especially the effectiveness of masking and lockdown. This is at
| the heart of "epidemiology" which is, as far as I can tell, the
| science of organizing data to meet predetermined political
| demands.
| aaron695 wrote:
| Feels like the gun lobby pushing blame to computer games.
|
| Everyone knows explicit fraud has nothing to do with it.
|
| It exists because academia is broken and can't self correct.
| sjwalter wrote:
| What I want to know is how does this issue impact the notion that
| we all seem to buy into that we should "follow the science".
|
| Scientists themselves have a hard time "following the science".
| Add to it the observation that when an issue is getting lots of
| attention outside of academia, then there are usually some really
| strong incentives (profit, prestige) associated with doing the
| science and applying it (e.g., epidemiological science during a
| global pandemic).
|
| The question seems not to be about how can normal people "follow
| the science" but rather, why should normal people trust at all
| that any touted science is anything more than bullshit spouted by
| highly-motivated sophists?
| version_five wrote:
| Follow the science is only used as a rhetorical device outside
| of science to try and convince people of something political.
| You would never hear an actual researcher say that.
|
| There is a realistic, weaker statement about the best available
| information we have, that a specialist could use to explain to
| a non specialist why they are making a recommendation about
| something emerging or theoretical. But what we are hearing with
| "follow the science" really means follow the carefully crafted
| political message that politicians with scientific credentials
| have put out.
|
| It's easy to see a distinction. Nobody needs to be told to
| follow the science on antibiotics or birth control or
| something. I think the blatant anti-intellectualism in the
| follow the science type statements is why we have so much worry
| about vaccines e.g. People aren't stupid and they can tell the
| difference between being manipulated and being presented with
| something objective. Even if you're right, it's a bad strategy
| to try and trick people or use religion to get your point
| across. See "the science is settled". Nothing makes people stop
| listening faster.
|
| Edit: and ironically, people call those who don't "follow the
| science" anti-intellectuals, as if intellectuals take things on
| blind faith. Every time I hear mention of anti-intellectualism,
| I have to remember that people are referring to those that
| question official doctrine, as opposed to those who have framed
| religion as science to try and short circuit debate.
| tshaddox wrote:
| What is the alternative to following the science? Following
| people who are not scientists and who are _explicitly_ making
| things up? This sounds a lot like "most plane crashes are due
| to pilot error, so maybe we should give non-pilots control of
| the planes."
| read_if_gay_ wrote:
| > why should normal people trust at all that any touted science
| is anything more than bullshit spouted by highly-motivated
| sophists?
|
| In the current climate, frankly I think it's absurd that we're
| putting so much trust in science, or rather what it has become.
|
| The fundamental problem is that science as in the method is
| absolutely worth putting your trust in, but a lot of what's
| sold as Science^TM has diverged from it far enough to be
| worthless. However, it still bears the same name and borrows
| its credibility. There are countless examples even from the
| places one would think to be the most trustworthy.
|
| What science as in the method hinges on as opposed to
| Science^TM is _verifiability_. Disciplines that aren 't easily
| verified suffer from the replication crisis to the point where
| it's basically synonymous. I would go as far as arguing that
| unless something has been verified several times it should be
| nothing more than a hypothesis. Note how popular science media
| are basically living off doing the opposite (though I don't
| think much better can be expected from the media honestly.)
|
| Math and social sciences form the two ends of the verifiability
| (and reproducibility) scale. CS is close enough to math that
| it's not a dumpster fire like psychology but I would say we're
| still suffering a lot of BS research. To fix this we need
| actual rigor, more openness about the methods, and frankly,
| motivation to reproduce results.
| version_five wrote:
| I would just add that science and the scientific method are
| designed to be used in good faith. Science doesn't really
| withstand political manipulation. If you're a researcher
| interested in learning more about the universe, science
| provides a framework for questioning and testing ideas, and
| for using established ideas as a jumping off point for
| further advances. As soon as there are other motivations than
| learning, the answers that "science" provides basically
| become unknowable because the whole process, from what to
| study to how to interpret and report findings, becomes
| corrupted.
|
| We need good politicians to negotiate a consensus on how we
| move forward in light of human desires and modern thinking
| about cause and effect. Pretending that "science" provides us
| with a way forward is abusing science for something it is not
| designed to do nor capable of doing.
| aabaker99 wrote:
| I think this is a very important question. This is something
| that I struggle with.
|
| I have read a lot of papers. I generally think science can be a
| force for good. I understand analytic methods developed by or
| used in papers from my field of interest. I generally believe
| that those methods are capable of answering important and
| interesting questions.
|
| In my view, the problem is that you can't know if an article is
| good or bullshit until you sit with it for, say, at least 2 or
| 3 hours (some papers even more). And that is for someone with
| my background. I tried to do this same thing when I had an
| undergraduate level of education and it (a) took me a lot
| longer (at least 10x), and (b) I missed a lot of the
| mistakes/scams/lies that I would not miss now. (I'm sure I am
| not able to detect some bullshit even still.)
|
| We should follow the good science. We should not follow the
| bullshit science. This sounds hard because science, being more
| technical, is harder to vet. But upon further reflection, it
| seems that society hasn't figured out how to deal with much
| simpler lies, either.
| bluGill wrote:
| > But upon further reflection, it seems that society hasn't
| figured out how to deal with much simpler lies, either.
|
| outside of your field how much of the BS papers can you
| catch? I know enough about computers that I could probably
| figure out at least some of it in that field (after spending
| 10x longer than someone who actually reads papers regularly),
| but give me a paper in something else and I'm not so sure.
| Consultant32452 wrote:
| We're indoctrinated from Kindergarten to trust the folks
| wearing the white lab coats. This is why the young push the
| "follow the science" stuff and the older generations are much
| more skeptical. The older people have been through several
| cycles of bullshit.
|
| The first "big lie" I experienced is the food pyramid. This was
| a big government push in the schools that told us all to eat
| carbs like crazy. Turns out it was just pure corruption, paid
| for by the grains industry. They have killed literally millions
| of us with this lie alone. And there were no consequences for
| this. No one went to prison. At some point you have to ask
| yourself: "How many millions of people does the
| government/industry have to kill before we stop believing
| them?" For me, it was the first million who died of diabetes
| and other obesity related diseases.
| kingkawn wrote:
| "follow science" is just another way of saying "do as we say
| not as we do."
| nitwit005 wrote:
| I would assume the people saying "follow the science" generally
| don't mean "believe recent research publications".
|
| I still occasionally see things like "hanging a potato to your
| wall will cure your child's flu" being debated by friends of
| friends on Facebook. You'd need to take a time machine several
| hundred years back for it to be within the realm of realm of
| genuine scientific debate.
| hallway_monitor wrote:
| You seem to be indicating the preferred time window for which
| research to trust. Not too new, not too old. Not the worst
| algorithm you could choose, and I agree. This is why I stay
| away from drugs, procedures, and any kind of guidance from
| the medical profession that is less than 20 years old.
| injidup wrote:
| I recently had the privilege of trying to do the right thing when
| I identified fraudulent research carried out by an institution in
| Austria. The initial response of the institution was positive but
| when I pressed for further details on how such things could
| happen suddenly nobody anymore would take my call. The research
| was paid for by a private company to pimp a nonsense product. The
| research was never published in a research journal but it didn't
| stop the company using the name of the university alonside
| exerpts from the paper in marketing material alongside gushing
| claims of "proved by science".
|
| The company threatened to sue me and the university threatened me
| as well. Neither has followed through on threats. The company
| wants to keep selling their rubbish magnetic health ding ding and
| I assume the university wants nobody to look into how positive
| results for the product came out of their institution. Allround
| an education on how the real world works.
| OriginalNebula wrote:
| Did you contact Arbeiterkammer, VKI or Peter Kolba?
| injidup wrote:
| I contacted and spoke at length with the Austrian
| Kosumantenschutz. https://www.arbeiterkammer.at/beratung/kons
| umentenschutz/ind... The product is clearly and at minimum a
| case of false advertising as there is nothing approaching a
| computer chip inside. However and though sympathetic the
| KS/AK said they couldn't do anything and couldn't refer me to
| anybody who could. I found this rather surprising.
| OriginalNebula wrote:
| Try to contact the other two. You can reach Peter Kolba
| here https://www.verbraucherschutzverein.at/Kontakt/ or on
| Twitter even https://twitter.com/KolbaPeter. He won the
| case against Volkswagen so if anyone can help you then him.
|
| The University of Salzburg also has a "Ethikkommission"
| https://www.plus.ac.at/service/uni-
| administration/gesamtuniv... and "Kommission zur Sicherung
| guter wissenschaftlicher Praxis"
| https://www.plus.ac.at/service/uni-
| administration/gesamtuniv...
| injidup wrote:
| Thankyou for the tip. I'Ll look into it tomorrow.
| OriginalNebula wrote:
| No, thank you!
|
| Austria has a weird thing with pseudoscience scams
| [0][1][2] and it needs to be dealt with.
|
| [0] https://de.wikipedia.org/wiki/Belebtes_Wasser
|
| [1] The current Minister of Economy was working as an
| "energizer"
| https://www.diepresse.com/5395317/wirtschaftsministerin-
| schr...
|
| [2] During the construction of a hospital in Vienna, an
| "energetic" ring was built around the construction site
| for 95000 Euro. https://www.derstandard.at/story/20000761
| 99184/krankenhaus-w...
|
| among others
| read_if_gay_ wrote:
| Meanwhile I'm trying to launch a simple health related
| mobile app and it's an absolutely insurmountable amount
| of paperwork for a solo dev.
| jahnu wrote:
| One "hack" to get around that is to change your health
| app into a "beauty" or "cosmetic" or "wellness" app. It's
| a very different set of regulations.
| read_if_gay_ wrote:
| Yep, I'd love to do that, but I'm squarely in the health
| sector, no getting around that sadly. I might end up
| cutting some of the main features to qualify as a
| lifestyle app though. Sad but better than not launching I
| guess.
| jahnu wrote:
| Not to mention almost all pharmacies selling homeopathic
| rubbish. I find this particularly irritating given the
| ridiculously regulations and control around things like
| buying paracetamol or ibuprofen.
| injidup wrote:
| @OriginalNebula There is also this
|
| https://www.air-innovation.fr/en/produit/vague-de-bien-
| etre-...
|
| and were installed into hospitals in austria at great
| cost
|
| https://www.salzburg24.at/news/salzburg/wo-in-salzburg-
| noch-...
|
| Unsurprisingly the owner of powerinsole a Mr Martin
| Masching is also involved in this enterprise via
|
| https://geowave-
| shop.at/epages/c0f45b90-03b3-4b2d-8e1d-55912...
| OriginalNebula wrote:
| I think its time you contact some investigative
| journalists with this Powerinsole/Geowave connection.
|
| https://twitter.com/florianklenk klenk@falter.at
|
| https://twitter.com/ThomasWalach redaktion@zackzack.at
|
| https://twitter.com/mnikbakhsh
| nikbakhsh.michael@profil.at
|
| https://twitter.com/ukschmid
| biztos wrote:
| Wow, that water looks amazing! Is anybody using it to
| brew bulletproof coffee[0]?
|
| [0]: https://www.nytimes.com/2014/12/14/style/the-cult-
| of-the-bul...
| [deleted]
| injidup wrote:
| tweet sent
| https://twitter.com/sltyhkr/status/1417213194540625925
| exact_string wrote:
| I don't understand what the problem is: The data in the
| double-blind study clearly shows no statistically
| significant effects (just look at N and the standard
| deviation bars) and the company doesn't claim that there
| would be such effects.
|
| The company only says there is a "positive trend" which
| there indeed is.
| dingidong wrote:
| Doesn't matter if the SD bars make it obvious. No layman
| can read such a chart. The text 110% makes it sound like
| the study PROVES without any doubt that the product had
| an effect.
|
| "the lactate measurement shows a difference" "The lower
| lactate value with Powerinsole means longer performance
| and a later onset of muscle fatigue." and finally "Even
| with the first application of the power insole, a lower
| skin conductance is evident compared to the placebo. This
| means that the power insole can help reduce stress
| levels."
|
| Nowhere are they talking about statistical significance.
| menzoic wrote:
| It seems well intentioned, but I'm not surprised at all about
| the outcome. I don't think you can expect the institution to
| implicate themselves like that. You have to realize that
| preventing legal and financial liabilities is the #1 priority
| for institutions. Unfortunately this includes when they are
| wrong. Even if the employees are "good" their lawyers will
| advise them against providing any details like that and most
| are afraid of retaliation. Anyone asking questions like that is
| the enemy to them. This type of behavior is bad for society but
| fully expected giving the incentives and consequnces of telling
| the truth.
| nixpulvis wrote:
| So what should be done? Hire your own lawyers to go after
| them pro bono?
|
| Write an article and send it to the papers?
|
| This kind of thing needs to be shamed and punished. While I
| agree that the institutions can be expected to cover there
| ass, there _must_ be a course of action.
| MeinBlutIstBlau wrote:
| I have a personal philosophy on most health oriented fads: if
| someone is actively trying to sell something, it's snake oil.
| Too many people are looking for everlasting life that they're
| okay with being swindled in the pursuit of it. Naive enough to
| believe that people are honest when promoting products they're
| were paid to promote.
| bserge wrote:
| You can pair it with: if it works, it's controlled. Meaning
| you can't get it without prescription and have to go through
| the tortures of a health "care" system.
| hellbannedguy wrote:
| This needs to end.
|
| I would take a pharmaceutical/drug related test
| (administered through the state), and it could be
| comprehensive; If I could renew my long term prescriptions.
|
| The pharmacist wants to see a blood panel before refilling,
| fine, I should be able to order one.
|
| (Only on long term medications (Blood pressure, diabetes,
| and most psychiatric drugs within reason) , and never
| antibiotics)
| bluGill wrote:
| While I agree with the sentiment, anyone on long term
| medication needs regular monitors by a doctor anyway. If
| you are only 25 this won't make sense, but by the time
| you get to 50 you need regular checkups for lots of
| things that are best caught early. I've lost enough
| family to colon cancer (spouse of a second cousin - can I
| even claim him as family?), and several others are only
| alive now because their cancer was caught in time. Then
| there is heart problems which again are best treated
| before the heart attack (if possible) - I just named the
| two most common killers of old people that you can be
| sure is in your family and coming to get you in the
| future (I know of exceptions - genetic disorders that
| will kill at 50 or so), but regular doctors visit can
| hold off.
| heavyset_go wrote:
| I mean, L-Dopa works and it's also available as a
| supplement, and there are plenty of OTC medicines with
| active ingredients.
| TeMPOraL wrote:
| That's too harsh, IMO: not everything that works is gated
| behind a prescription.
|
| What I'd look into instead is how it's legally classified.
| My heuristic is: if it works in any meaningful way, it's
| classified as drug or medical equipment. Might be OTC, but
| it's clear about its status. Stuff that's _not_ classified
| like that can at best correct some nutritional
| deficiencies, but typically does nothing at recommended
| doses (but sometimes can still hurt you if you severely
| overdose).
|
| There are substances gated behind a prescription that I
| wish people would have easier access to - but I understand
| the need for regulatory control. If people selling all
| these fraudulent cures can dupe so many regular folks,
| imagine what would be happening if they were allowed to put
| medically relevant quantities of an active compound in
| them.
| robotresearcher wrote:
| What fraction of your drug store purchases involve
| prescriptions? For me it's <10%.
|
| Aspirin and fluoride toothpaste are each pretty darn
| effective.
| birdyrooster wrote:
| I guess if you don't even feel comfortable sharing the identity
| of any of the parties, we really have no chance to identify bad
| actors from good as a community.
| injidup wrote:
| https://shop.powerinsole.com/en/blogs/news/doppelblindstudie.
| ..
|
| I actually bought one of the devices and took it apart on
| video. There is nothing inside except 4 magnets and a plastic
| printed card. No active circuits and no power source and no
| components such as resistors or capacitors and certainly
| there is no microchip inside.
|
| _edit below are links to pictures of the device and
| accompanying text as originally on facebook posted_
|
| =======================================
|
| Hi Powerinsole I ordered one of your power chip devices and
| took a detailed look at it. My analyses is as follows. 1.
| There is no battery and no system for harvesting energy. All
| electronic circuits require an energy source and a lack of an
| obvious system for powering the device is a problem. 2. There
| are no components such as integrated circuits, transistors,
| capacitors, diodes or inductive devices that would be
| required to create a "circuit" or "chip". A "chip" is not
| just a random configuration of tracks. The tracks are there
| to transfer electricity between components that shape and
| switch the electric current according to purpose but given
| that there are no components what are the tracks for? 3.
| There are 4 magnets. Probably neodymium. They produce a
| constant magnetic field. They do not generate "frequencies".
| The device sticks to a metal wall like a fridge magnet and
| doesn't vibrate. 5. The tracks are configured in such a way
| that even if components were attached at the "solder points"
| nothing would happen because the tracks are all shorted
| together. Electricity always takes the easiest path. If all
| the tracks are are shorted then the components will receive
| no energy input. 6. After testing with a multimeter I found
| that the tracks on the "circuit board" do not conduct
| electricity. If the tracks do not conduct electricity there
| is no possibility of transferring energy to components. (
| there are no components ) 7. The magnets are isolated from
| the "tracks" and each other by a plastic layer and glue. It
| is not clear what the relationship the position of the
| magnets to the tracks might be. 8. There is no NVRAM,
| magnetic storage, optical storage, ROMs or other known
| systems for storing information. So claims from PowerInsole
| that they load information onto the device is difficult to
| comprehend. 9. There is no crystal or LC circuit to drive an
| oscillator. Even if there was there is no battery to drive it
| and the tracks are all shorted and the tracks do not conduct
| electricity. Given the above observations I find it difficult
| to believe that the device can function as advertised. What
| you essentially have is 4 small magnets on a printed card in
| a gel cushion for 69 euros. If I am wrong about any of the
| above I would be happy to have a respectful and open
| discussion about your technology.
|
| https://scontent-
| vie1-1.xx.fbcdn.net/v/t1.6435-9/163585013_1...
|
| https://scontent-
| vie1-1.xx.fbcdn.net/v/t1.6435-9/163740448_1...
|
| https://scontent-
| vie1-1.xx.fbcdn.net/v/t1.6435-9/164072709_1...
|
| https://scontent-
| vie1-1.xx.fbcdn.net/v/t1.6435-9/164005668_1...
|
| https://scontent-
| vie1-1.xx.fbcdn.net/v/t1.6435-9/164045289_1...
| uhhyeahdude wrote:
| Thank you. What a load of infomercial buzzword nonsense.
| The last part about some "further research and experience"
| showing even _better_ results if the product is worn all
| the time is especially galling.
| injidup wrote:
| But one should look into Darsch Scientific if you want to
| get deeper into how this is all organised.
| https://www.dartsch-scientific.com/en/ they have produced
| papers for powerinsole and are used by other companies in
| the magnetic magic industry to provide a veneer of
| scientific credibility. For example
| https://waveguard.com/en/studies/
| janekm wrote:
| Kind of hilarious how they submitted what looks like a
| passive coil for FCC certification ;)
| inasio wrote:
| I love how the error bars on the plots go only on one
| direction, and conveniently the opposing ones between the
| two curves, definitely not scammy...
| ineedasername wrote:
| They had me at the work "power insole". Though I guess not
| in a good way. I expect a fairly meticulous definition of
| terms. If a piece of research sets off my marketing-BS
| sensors this early, it's usually all downhill from there.
| yarcob wrote:
| Somehow I'm not surprised the University of Salzburg is
| involved. I remember a "less than stellar" experience with
| them that I was tangentially involved with.
|
| It involved a research project that they stopped funding,
| but didn't want to let go. The only researcher who
| continued to work on the project wanted to take the project
| to a different institution where the project could get
| funding, but Uni Salzburg refused and said it's their
| project. They would rather have the project be abandoned
| than let it thrive somewhere else. If their name wasn't on
| the project anymore, they would rather have it die.
|
| And not to forget, Uni Salzburg was also home to our most
| famous case of scientific misconduct where Robert
| Schwarzenbacher fabricated measurements using simulation
| software. The handling of that case was also interesting
| (they terminated his employment after verifying the
| accusations of fraud, but one guy from the union tried to
| convince someone from HR to delay some paperwork so they
| could later challenge the termination... crazy stuff)
| bserge wrote:
| You probably just created a winning class action suit.
| Hell, the founders could go to jail for this level of
| fraud. Or at least, I hope so.
| javajosh wrote:
| "Naming and shaming" comes with risk - probably a lot more
| risk than upside. The risk is that the parties get something
| actionable, kind of like "probable cause" in criminal cases.
| The upside is _maybe_ a small effect on a few HN minds that
| _might_ remember this when considering this Uni 's
| reputation.
|
| The other risk is that it's an act that can easily be abused.
| It is very easy to level charges against someone without
| proof; somehow we tend to believe the first salvo (myself
| included). In this case it sounds relatively straight-
| forward, but it really is irresponsible to take a stranger's
| word for it.
|
| So, if you really care, you might reach out to the OP and get
| the details. That eliminates the downside risk to the OP and
| acts as a shibboleth that ensure only people that actually
| care enough to look into it know the details.
| birdyrooster wrote:
| Yes, that is why you would expect their claim to be
| corroborated with (even circumstantial) evidence when
| dropping names.
| [deleted]
| robomartin wrote:
| This is one of the problems I have with the absolute freak show
| that climate science/global-warming/saving-the-planet has
| become. It is, at a minimum, a triangle of bad actors with one
| corner being politicians --fear-based vote harvesting--, the
| second being business --jump on the bandwagon and print
| money...facilitated by politicians who want votes in exchange
| for fear mongering-- and, finally, religious-based detractors
| --using the best ignorance can offer in order to advanced
| humanity.
|
| You combine these three factors (and likely a few more) and the
| entire thing is a rotten stinking mess that exists on a binary
| state between religious deniers and religious zealots.
|
| What's a researcher to do? Tell the truth? Ha! Only if you want
| your career completely destroyed as well as never seeing even a
| hint of a grant. Going against these forces is a sure path to
| having more PhD's driving taxis.
|
| I have to say, I have become very cynical about what we call
| "science" these days. It seems you have to be very guarded
| about accepting anything you are told, because the forces at
| play could be beyond anyone's imagination in scale, breath and
| reach. The problem is that the general voting public is ill-
| equipped to take an intellectual stab at what they are being
| told, which means they are easily duped and herded like cattle
| in and direction that might be of benefit to the puppet masters
| in politics.
| SubiculumCode wrote:
| Are you a scientist?
| robomartin wrote:
| > Are you a scientist?
|
| How is that relevant to my comment?
| SubiculumCode wrote:
| You suggest how scientists can act, their motivations,
| their freedom to disagree with the status quo, or those
| higher up the ladder in their field etc. You suggest how
| scientists will lose grants if they produce inconvenient
| science, etc etc. So I ask again. Are you a scientist?
| Have you written a grant application? Have you been
| punished? I AM a scientist, and I have never seen this
| hypothetical world.
| naasking wrote:
| Not sure I'm following your point. If you weigh the bad
| actors and financial incentives of climate change proponents
| against the bad actors and financial incentives on the fossil
| fuel side, do you think the scale tips in favour of more
| honesty for the fossil fuel side or climate change side?
|
| There's no doubt groupthink happens in academia on many
| issues, but the need to displace fossil fuels really is very
| important. Not just for climate change reasons, but overall
| human health. For instance, air pollution from fossil fuels
| kills tens of thousands of people every year.
| robomartin wrote:
| No. There is no honesty anywhere, that is my point.
|
| > the need to displace fossil fuels really is very
| important.
|
| Why? I don't necessarily disagree. But reality isn't a
| problem managed through a single variable. The things you
| list are not singularly caused by fossil fuels.
|
| In fact, a very solid argument could be put forth about
| just how much uglier things might be without fossil fuels.
|
| Here's the basic math someone would have to do before
| making the assertion that the elimination of fossil fuels
| --as a single causally-connected variable-- would make
| things better:
|
| The simplest (well, not so simple) calculation is that,
| while we might eliminate fossil fuels we do not eliminate
| the need for the energy they provide. In other words, in
| rough terms, you still have to explain how we would
| generate, harness, create, transport and distribute a
| certain amount of energy per unit time (hour, day, week,
| month, year, whatever).
|
| In fact, I think we can, in historical terms, state that
| energy requirements increase over time, they do not
| decrease.
|
| The next element of the story is how we are going to
| replace the massive number of byproducts of fossil fuels
| that modern life pretty much depends on. We know that
| making complex hydrocarbons any other way is in a range
| between highly inefficient (which would increase the
| aforementioned energy requirement) and impossible.
|
| My point --in stressing that reality is a rather complex
| multivariate problem-- is that, while it would be nice to
| think of a desirable reality without fossil fuels, in the
| real reality (just go with it) this is much more of an
| aspirational thing than an attainable objective.
|
| The same is the case with electric vehicles. I have yet to
| see someone do the math on the total daily energy
| requirement of the installed fossil-fuel based vehicle
| fleet and explain how on earth (literally) we are going to
| generate that much energy without causing even more
| problems. Our current electrical grid is designed for
| current energy requirements (and power requirement, which
| is equally important). The current system, in any country I
| know of, doesn't magically have an extra 100% in
| power/energy generation capacity to support every vehicle
| going electric.
|
| Reality: A multivariate problem. You push here and it pulls
| there. Not so simple.
|
| > For instance, air pollution from fossil fuels kills tens
| of thousands of people every year.
|
| Fair enough. Containerships, as a simple example, burn
| bunker fuel, one of the nastiest things you can burn. They
| are singularly responsible for more pollution along certain
| vectors than the entirety of the ground transportation
| industry. And yet we do nothing about it.
|
| Why?
|
| I can only guess. Part of it has got to be a case of "well,
| what we have works". The other issue --which I think is
| very real-- is that bunker fuel is, quite literally, the
| bottom of the barrel. It is what is left after you extract
| everything else from petroleum.
|
| So, next Monday we stop using bunker fuel everywhere in the
| world. No problem. Right?
|
| Wrong.
|
| You see, all the other oil byproducts are still needed.
| Which means that the bottom of the barrel...the bunker
| fuel...would still be produced in absolutely massive
| quantities. Except now we are not using it, because we want
| to clean-up the planet.
|
| Wait a minute. What do we do with it?
|
| Well, we likely have to bury the stuff, dump it somewhere,
| make huge mountain-sized piles out of it. We would now use
| massive amounts of fuel (yes, everything is "massive") to
| run the machines that have to haul and manipulate this
| stuff. We also have to devote massive (sorry) resources,
| land and ecosystems to burying what we are not using. Where
| it goes from there I cannot even guess.
|
| Once again, reality isn't a single variable problem. Bunker
| fuel == bad? Yes, no, maybe, hard to say. Because the
| alternative could be worse, far worse.
|
| This is precisely what I don't see treated fairly these
| days. Imagine a politician taking the time and making the
| effort to fully analyze and understand the bunker fuel
| ecosystem and also taking the time to present this analysis
| to the voting public. Good luck. It is far easier to say
| "bunker fuel == bad", get votes, stay in office and move
| on. It's easy to show how horrible the stuff is (and it
| is!). It is impossible to show how much worse things could
| be if we don't fully understand what reality looks without
| it.
|
| I'll overstay my welcome and give another example from real
| life.
|
| A number of years ago a well-intentioned yet
| mathematically-challenged "science" teacher at my kid's
| school showed the kids this gut wrenching video animation
| that pretty much says humans are a pile of shit destroying
| the planet. The thing is a close as you can get to an
| ignorant politically-motivated pile of lies.
|
| She was receptive to having a conversation. I asked if we
| could go through a simple exercise where we would try to
| understand what our small town would look like if we did
| not use the products of evil industrialized society.
| Petroleum is a favorite, of course.
|
| I won't bore you with the details. Before we got done we
| had destroyed every forest in sight, had piles of human
| excrement the size of mountains, all possible fields where
| you could grow something in the region were dead, sources
| of water were polluted (human waste and other by products
| of inefficient source for everything) and more. At the
| extreme we were using horses to get around, etc. A town of
| a few tens of thousands of people relying on horses has a
| serious manure problem. We would burn trees for heat and
| cooking, etc.
|
| As we extrapolated this from a town of tens of thousands to
| cities with millions and regions with tens to hundreds of
| millions of people, it became very obvious that modern life
| (or more accurately, modern population levels) would
| quickly become unsustainable if we demanded that humanity
| abandon how we got here and embrace everything "natural" an
| "sustainable". She was certainly surprised to understand
| the scale of the problem.
|
| Once you start thinking at scale --planetary scale--
| "natural" and "sustainable" quickly end-up with razed
| forests, depleted marine life, polluted water sources and a
| sky blackened with thick pollution.
|
| Not to end on a depressing note. Yes, we are doing better,
| have been so for decades. We just have to be careful that
| we don't reduce reality to single variable problems,
| because that isn't reality, it's a fantasy, and a dangerous
| one at that.
|
| Climate change is one of those. It is hard to find truth
| that is being discussed with honesty in the mainstream.
| naasking wrote:
| > In fact, I think we can, in historical terms, state
| that energy requirements increase over time, they do not
| decrease.
|
| They do, and all energy needs can be met with solar, wind
| and grid energy storage. Or nuclear if you don't want to
| invest in energy storage for whatever reason.
|
| > The next element of the story is how we are going to
| replace the massive number of byproducts of fossil fuels
| that modern life pretty much depends on. We know that
| making complex hydrocarbons any other way is in a range
| between highly inefficient (which would increase the
| aforementioned energy requirement) and impossible.
|
| Burning fossil fuels are the biggest immediate problem.
| Other fossil fuel products may or may not be a problem.
| But you don't ignore the heart attack because you just
| noticed a rash that may be flesh eating bacteria. Triage
| is key.
|
| > You see, all the other oil byproducts are still needed.
|
| "All" is overselling. Some are arguably useful, but for
| example, most product packaging is likely superfluous and
| a product of our current economic incentives. For
| instance, why do we have disposable containers for each
| unit of cleaning product we buy rather than reusing
| containers that you get refilled at the store? These
| choices are driven by market incentives that prioritize
| convenience over sustainability.
|
| Some products may never get rid of their plastic
| packaging, perhaps something like sterilized vacuum
| packed needles that hospitals use. Those would be the
| exceptions but not the rule.
|
| > We just have to be careful that we don't reduce reality
| to single variable problems, because that isn't reality,
| it's a fantasy, and a dangerous one at that. Climate
| change is one of those.
|
| Climate change isn't a single variable problem, and I
| don't think anyone serious is pushing it as such. If you
| look into the IPCC report on climate change, you'll see
| all sorts of factors being accounted for including cloud
| cover, contrails, methane, water vapour, CO2 and more.
|
| We only have so much influence over some of these
| factors, but the biggest and most obvious factor _for
| which we have alternatives_ , is CO2 emissions. Do you
| deny that?
|
| > Once you start thinking at scale --planetary scale--
| "natural" and "sustainable" quickly end-up with razed
| forests, depleted marine life, polluted water sources and
| a sky blackened with thick pollution.
|
| You and I clearly have different understanding of what
| "sustainable" means.
| goatlover wrote:
| Too bad everyone has been convinced nuclear is way worse
| because of a couple of accidents. That was a legitimate
| alternative that didn't need to wait for the 2010s to
| become an economically viable 15-20% of energy production.
| lumost wrote:
| You know, I've seen an increasing trend towards mediocrity
| and outright fraud across multiple public and private
| institutions that reward individuals based on some power law.
|
| If the author of the most cited paper in a field is going to
| get all the grants, and a standard "useful" paper is going to
| get no continued funding - then researchers will push to make
| their work sensational. Eventually the professors and
| everyone left in the field is fighting sensational with
| either outright fraud or alternate funding sources.
|
| Same goes for VC funding of startups, employees at companies,
| and government programs. The baseline "useful" work is
| rotting away in favor of aiming to be the top ~5-10%.
| btilly wrote:
| https://www.sciencealert.com/non-replicable-studies-make-
| the... may interest you. Papers that can't replicate get
| cited more than papers that can.
|
| In other words if your goal is to craft a paper that draws
| attention, that's harder if you're being meticulous about
| limiting yourself to saying to what evidence shows is
| clearly true.
| lumost wrote:
| Unfortunately this will be a baseline that keeps moving.
| scientists compete on publications so they will make more
| sensationalist headlines, There will then be more
| sensationalist headlines to compete with forcing fraud/or
| irrational exuberance.
| dqpb wrote:
| Can you be sued for asking questions?
| Ajay-p wrote:
| IANAL but my gut tells me no, unless you're in a place like
| Russia, North Korea, China, most of the Middle East, parts of
| Africa and Latin America, and some parts of the former Soviet
| Bloc. In western nations, you might get arrested for asking
| questions if it upsets the powers that be and they deem you a
| nuisance.
| White_Wolf wrote:
| In UK you have a chance to end up with a "hate crime" if
| you don't follow the official line of (no)thinking(with a
| low change of being arrested). Pretty crazy stuff going on
| lately(past few years).
| peteretep wrote:
| Are you confused that a "question" can be a hate crime?
| Seems pretty straight-forward to me?
| ghoward wrote:
| Asking "What's your birth sex?" might end up being a
| "hate crime" with the way things are going.
| AlexMoffat wrote:
| For what possible reason are you asking that question of
| anyone?
| kxkdkkf wrote:
| Well, maybe you own a waxing salon that does Brazilian
| waxes and you're Muslim and don't believe you should see
| or be in contact with male genitalia besides that of your
| husband.
|
| https://en.m.wikipedia.org/wiki/Jessica_Yaniv
|
| > In 2018, Yaniv filed discrimination complaints with the
| British Columbia Human Rights Tribunal against multiple
| waxing salons alleging that they refused to provide
| genital waxing to her because she is transgender.[15][16]
| Yaniv's case was the first major case of alleged
| transgender discrimination in retail in Canada.[17] Yaniv
| was seeking as much as $15,000 in damages from each
| beautician.[18] In their defence, estheticians said they
| lacked training on waxing male genitalia and they were
| not comfortable doing so for personal or religious
| reasons.[19] They further argued that being transgender
| was not the issue for them, rather having male genitalia
| was.[20] Yaniv rejected the claim that special training
| in waxing male genitalia was necessary,[21] and during
| the hearings equated the denial of the service to neo-
| Nazism.[22][16] Respondents were typically working from
| home, were non-white,[23] and were immigrants[24] who did
| not speak English. Two of the businesses were forced to
| shut down due to the complaints.[25]
|
| Just one example..
| dragonwriter wrote:
| > Well, maybe you own a waxing salon that does Brazilian
| waxes and you're Muslim and don't believe you should see
| or be in contact with male genitalia besides that of your
| husband.
|
| Then you'd probably want to ask "What kind of genitalia
| do you currently possess". Sex assigned at birth is not a
| reliable indicator of that, for reasons very similar to
| why current gender identity isn't.
| chc wrote:
| More like "just the one example." This same case is
| trotted out every time somebody wants to make this point
| because it does not actually seem to be a trend.
|
| But besides that, it doesn't answer the question you were
| responding to. Asking someone's birth sex does not tell
| you what genitalia they have. It doesn't even really tell
| you what genitalia they had at birth. Since those salon
| owners say they specifically objected to the woman's
| current genitalia rather than the woman's status as
| transgender, birth sex is the wrong question to ask.
| MikeTheGreat wrote:
| I would kinda hope that phrasing stuff as a question wouldn't
| affect the viability of a lawsuit one way or the other.
|
| Otherwise I'd expect people to leak trade secrets as a series
| of questions, leak top secret info as a series of questions,
| etc.
|
| I kinda hope it's the underlying substance, not how you
| phrase it, that determines this sort of thing :)
| dpifke wrote:
| You can be sued by anyone, for anything, at any time.
|
| Whether or not that lawsuit has any merit is irrelevant.
|
| With certain exceptions based on subject matter (e.g. SLAPP)
| or jurisdiction, it still costs money to hire a lawyer to
| defend against the suit.
| joshspankit wrote:
| American law does not apply in Canada either
| bmn__ wrote:
| American law does not apply in Austria
| pessimizer wrote:
| Find a reason to sue them in the US. That's how people
| use the UK's horrible libel laws.
| 0x6A75616E wrote:
| Why not? What state is that in?
|
| /s
| bluGill wrote:
| True, but while I don't know Austria's laws, I know that
| in any country that isn't fully corrupt (and even in many
| of them) anyone can take anyone to court for anything.
| There are different rules around loser pays (there is no
| good answer here - only bad compromises) and how fast the
| judge will dismiss fraudulent claims, but if there is
| anyone who can't get a day in court over a legitimize
| issues the country is corrupt. Austria isn't perfect (no
| country is), but there international reputation isn't
| nearly bad enough that I would expect someone to be
| unable to get your in court for anything they want.
| beebeepka wrote:
| I don't think that's how things work in most European
| countries. Yes, there are costs but they will be covered by
| the loser. No way this person loses against these
| particular fraudsters
| ekianjo wrote:
| still costs money upfront
| askonomm wrote:
| Also, a lot of the cases these lawsuits in EU countries
| are thrown out because of how absurd they are.
| menzoic wrote:
| It still cost time and money to get to that point
| Archelaos wrote:
| The situation in the EU in general is all but
| satisfactory, especially for investigative journalism.
| There exists a term for it: SLAPP - Strategic Lawsuit
| Against Public Participation. There is currently an
| initiative of a group of European MEPs underway for
| better anti-SLAPP legislation in the EU member states.
| You may read more about the SLAPP problem in this
| article, published by the European Centre for Press and
| Media Freedom: https://www.ecpmf.eu/slapp-the-background-
| of-strategic-lawsu...
| remram wrote:
| Some countries are also sensible to abuse of the legal
| system to this end, e.g. Lenovo had to pay 20,000 EUR in
| damages after they decided to drag the case of a consumer
| who wanted his 42 EUR Windows license refunded:
| https://fsfe.org/news/2021/news-20210302-01.en.html
| injidup wrote:
| I have an amusing letter from the lawyers working for
| powerinsole threatening legal action for asking questions and
| that I should remove all my comments from social media. It
| was also demanded that I pay approx 350 euro that the letter
| cost to write. My response was to invest in the cost of the
| device, tear it down on video and post the analysis to
| facebook. That was more than 6 months ago. I have not heard
| from the lawyer since. I'll happily go to court with a
| printed t-shirt with the text "where is the battery?" but it
| won't come to that I guess.
| 1MachineElf wrote:
| Is that a public post? Would love to read it.
| mnw21cam wrote:
| Sounds like something BigClive would be interested in.
| [deleted]
| fasteddie31003 wrote:
| I've had a thought on how to solve this issue by using basically
| a research paper futures market. You could implement this with
| Ethereum Smart Contracts. You have a market around the validity
| of research papers. You would need some authority that would act
| as the oracle of the paper's validity. If the paper is found
| invalid before some time period the people who bet on the paper's
| truth would lose their money to the people skeptical of a paper.
| This would also act as a mechanism to signal which papers people
| don't trust by the amount of people betting against a paper's
| validity.
| mrits wrote:
| This could actually make it worse. Companies could pump their
| own research anonymously
| ta988 wrote:
| When you want to do a proper work, your grants and papers get
| rejected because they are not innovatove enough or don't go far
| enough. So it is not a surprise that people that lied in their
| applications about what they can realistically do also lie when
| it comes to reporting results. Unfortunately there is no way out.
| I stopped counting how many reviewers of my grants disagreed on
| what was proposed, one saying that it was not innovative, the
| other saying that is was too risky to use this approach. We have
| a big problem in science, peer-review is broken and everything
| relies on it. And many reviewers are way out of touch about what
| happens in their field, I see reviews that clearly show the
| reviewer was sleeping for the last 10 years.
| nextos wrote:
| You are absolutely right.
|
| Furthermore, universities tend to require tons of publications
| to promote you. Things are spinning out of control. I know a
| few EU countries where the written norm is to need > 100
| publications to qualify for a full professorship, with equally
| ridiculous requirements for associate and assistant positions.
|
| Obviously, this encourages and rewards completely broken
| practices. Many associate and full professors in my area only
| care about stamping their names into as many journal articles
| as humanly possible. Some of them are already beyond 500, with
| many of these in top tier journals (Nature, Science, Cell,
| NEJM). Obviously, they hardly ever contribute anything. Their
| serfs do all the work. Their job is basically to plot in order
| to stay on top of their neofeudal shire.
|
| In addition to this, funding bodies do nothing after fraud has
| been proven. ERC only terminates grants on rare occasions.
| https://forbetterscience.com/ discusses many cases of serial
| fraudsters who keep getting funded despite having retracted 10
| or 15 articles in major journals.
| tasogare wrote:
| > Many associate and full professors in my area only care
| about stamping their names into as many journal articles as
| humanly possible. [...] Obviously, they hardly ever
| contribute anything. Their serfs do all the work.
|
| This describe my lab's head perfectly. At first I found
| strange he was so angry about a side project paper I wrote
| alone quickly on my free time and asked to publish at a
| conference. Then I understood why: in his view, every minute
| I spend on my projects is one I don't spend on _his_
| projects. The guy approved my first journal paper submission,
| which had his name on it without even reading it. It was
| obvious by the lack of comments and when he asked a few days
| later during a lab meeting to change half the content of the
| paper...
|
| I'm not against putting name of people contributing to the
| research, even slightly and informally, but at this point
| this is pure leeching and exploitation. Then he wonders why
| my thesis isn't progressing (hint: because when I chose
| nothing about the topic, method and experiment setting I'm
| not really motivated to work on that).
| derbOac wrote:
| One of the clearest examples of the publishing problem to me
| was the shift in meaning of last authors on papers over the
| course of my career. When I first started, last author meant
| the person who had contributed the least to the paper (in
| cases where ordering of effort can actually be determined --
| often there's genuinely equal contributions). Often this was
| the senior faculty member, as they did little but sort of
| read over a paper or maybe supervise someone independently
| functioning.
|
| Over time though the last author came to mean "the more
| senior person" and then "the person whose idea it really is".
| So being last went from this thing that no one wanted, to
| this thing that people would kind of argue over. In the
| process the more manipulative cases, people would kind of
| casually say "oh I can be last author" realizing the gains
| from that position.
|
| It seems when a more junior person is doing all the work and
| is first author, an unscrupulous senior researcher will claim
| that "it's the idea that counts"; when that senior researcher
| is first author, it's "ideas are a dime a dozen, it's getting
| it done that matters."
| BurningFrog wrote:
| I keep getting more convinced that Science needs to be
| rebooted.
|
| How do we start over in some sane way?
| heisenbit wrote:
| The problem is not that people lie on their application but
| that these people are now being judged by people who lied on
| their applications some time back. The lying has been
| institutionalized and leaves little resources for small but
| meaningful progress.
| WalterBright wrote:
| I'm sadly amused by all this. The complaint I hear about
| privately done research is it's all tainted by the profit
| motive, and so research should be funded by the government, as
| then it'll be pure and untainted by selfish motives.
|
| Of course, government funded motives are just as tainted by
| selfish motives, if not more. Even worse, the people who make
| the funding decisions aren't spending their own money, so they
| have little reason to care.
|
| At least with privately funded research, the people providing
| the money aren't going to fund bullshit fake research. This is
| why market systems work better than government systems.
| LadyCailin wrote:
| > At least with privately funded research, the people
| providing the money aren't going to fund bullshit fake
| research.
|
| Citation needed, first of all, but governments are at minimum
| accountable to voters. Private money is in no way
| accountable.
| WalterBright wrote:
| Another way to look at it is voting is not the same as
| choice.
|
| If you wanted a mcburger but 51% voted for mcnuggets, you
| got mcnuggets.
|
| If you knew in advance that 75% were going to vote for
| mcnuggets, you likely wouldn't even bother to vote. You
| knew you had _no choice_ at all.
| dragonwriter wrote:
| > but governments are at minimum accountable to voters.
|
| Governments are at a _minimum_ accountable to the people
| willing to use force against the government if they are
| sufficiently displeased. They may also be accountable to
| voters _qua_ voters, depending on whether they have voting
| at all, and, if they do, what options are presented to
| voters and how fairly voted are counted, all of which are
| axes on which governments vary considerably, with many
| falling into ranges resulting in little or now
| accountability to voters.
| WalterBright wrote:
| > governments are at minimum accountable to voters
|
| Voting on how government spends money is in no conceivable
| way like you deciding how to spend _your_ money.
|
| > Private money is in no way accountable
|
| It's accountable to the people who are providing the funds
| out of their _own_ pockets. People do not like wasting
| their _own_ money.
|
| I bet you look at your own budget. You have to, otherwise
| you'll be in jail for bouncing checks and tax evasion. I
| also bet you've never looked at your city, county, state or
| federal budget. It's other peoples' money, so who cares!
| bluGill wrote:
| > I also bet you've never looked at your city, county,
| state or federal budget.
|
| I have. Not in great detail though. The problem is I
| can't really do anything about it. Even if I find
| something bad and by lucky chance get people to care
| (there are plenty of slow news nights) - there is far
| more bad things in the budget than I can expose before
| people get tired of the corruption and give up listening.
| I try to elect politicians who will do something about it
| - I have low success: people who benefit from any
| specific spending are more powerful than people who are
| just against waste in general. That is assuming I can get
| my person on the ballot in the first place (low odds),
| and they don't realize once elected that reelection (read
| power) comes from handing out pork to those who want some
| specific waste. There are more things that make it hard -
| I just scratched the surface.
|
| Pork is hard to figure out. Is spending money not to
| repair something that isn't broken good money or bad?
| I've seen perfectly good buildings get needless remodels
| and I've seen perfectly good buildings suffer because
| they never got maintained. I've seen towns put in sewer
| systems they don't need, and other towns fail to put in a
| sewer system until it was an expensive emergency. Flint
| had 40 years to replace the lead pipes in their water
| system - or they could have investing in water treatment
| chemicals that makes lead not leach from the pipes for
| much less money even over 40 years (you can pick anything
| from 60 to 30 years ago as the date when lead is bad
| became known - 40 was my somewhat arbitrary pick).
| buitreVirtual wrote:
| Actually, companies pay universities to conduct fake research
| that "shows" that their products work.
| native_samples wrote:
| That's the same problem in disguise. The reason they don't
| do the "our product is great" research themselves is
| because if they did people would switch their brains on and
| properly evaluate it. They pay universities (i.e.
| government funded organizations) because of the false
| belief in our society that government funding means
| universities are neutral, trustworthy, competent research
| institutions, when in fact they are really quite corrupt
| and filled with easily bribed researchers who will publish
| basically anything if it means they get another paper or
| grant out of it.
|
| If/when the perception of government funded researchers
| finally aligns with the reality, businesses would stop
| doing that because there'd be no reputational misalignment
| to exploit.
| WalterBright wrote:
| That's a good point. But if the companies are paying for
| research in how to build better products, they aren't going
| to pay for bullshit research.
| mhh__ wrote:
| It has happened before i.e. Hendrik Schon, however these
| incidents are more to do with humanity as a species than
| their employers.
| WalterBright wrote:
| Nothing is 100%.
|
| I'm talking about _incentives_ here, and people do things
| almost entirely on selfish impulses. Money is a powerful
| motivator, and people are strongly motivated to not spend
| their own money on bullshit. That motivation is absent
| when government funds things - but other motivations
| remain.
| mhh__ wrote:
| Sure, I just think its an interesting story considering
| he might have been nominated for the Nobel Prize in
| physics for totally fraudulent work
| WalterBright wrote:
| Pulitzer Prizes have been awarded for work later shown to
| be complete frauds. Those severely damaged the value of
| getting one. I know I don't attach any respect for
| Pulitzer Prizes.
| _jal wrote:
| They pay for bullshit research all the time. It comes
| from the advertising budget instead of the development
| budget.
| version_five wrote:
| I think there are two separate versions of "private research"
| that people below are responding to. In one, a company has a
| problem and they pay researchers to work on it. The key
| metric is solving the problem or making progress on in
| depending on the time scale- good orgs have different scales
| (usually from 3 months to 5 years at most) that they are
| investing in. In this case, there is little room for fraud or
| deception, but it goes up with time scale because of how you
| frame early results. (I work doing applied research for
| companies, and they want and will only pay for something they
| can use to improve their business. Actually a lot of my time
| is spent helping make a clear connection between how research
| findings will move the needle on business objectives). I
| think it is this kind of research you, the parent, are
| referring to.
|
| There is also "sponsored" research as others have pointed
| out, that is more of a bought study that a business hopes
| they can use for marketing. These have a big conflict.
|
| I agree that government is probably the worst system in most
| cases. It's the same kind of "picking winners" that doesnt
| work in corporate funding. I'm from Canada where our tech
| industry basically runs on subsidies, and very little escapes
| the bubble of trying to get more government funding and
| actually becomes self sustaining.
|
| Personally, I have seen there is a legit appetite for
| corporate funded research that advances the company's goals.
| As an academic, I would rather seek out companies for
| funding, knowing that I'm working in something that someone
| wants, and not trying to optimize for government priorities.
| I'm coming at this from a hard science perspective. I imagine
| the dynamics are very different for drug trials or other
| efficacy type studies, which are maybe more relevant to this
| discussion.
| WalterBright wrote:
| Good points, but there's another wrinkle. If a company pays
| a research institution to do a fraudulent study, the
| research institution risks losing their status as a
| reputable research outfit, and thereby loses a multiple of
| that as other companies avoid funding them.
|
| A prestigious reputation is like glass - easy to break,
| very hard to put back together.
|
| You'd think this would work with government funding, too.
| But it appears it does not. It could be because one's
| "reputation" is based on how many papers are published and
| how many cites. This is like rating a programmer on how
| many lines of code written.
|
| It is not a measure of quality at all.
| pjc50 wrote:
| > At least with privately funded research, the people
| providing the money aren't going to fund bullshit fake
| research
|
| They absolutely are if it helps them promote something.
| Cigarettes and asbestos industries helped produce plenty of
| fake safety studies.
|
| The problem is that research has been marketized; you have to
| "sell" your proposal to get funding, so naturally you big it
| up as much as possible. And thus the incentive to fake
| results.
| WalterBright wrote:
| If you are _personally_ funding Professor X to do some
| research say, on making a better LCD display, and Professor
| X comes up with nothing but personal aggrandizement passed
| off as "research", are you _personally_ likely to fund him
| some more?
|
| I seriously doubt it. Any more than you'd continue taking
| your car to an auto shop that took your money but didn't
| fix it.
| [deleted]
| jerf wrote:
| Worse yet, it compounds. The people approving grants, seeing
| all these amazing results promised, will then raise the bar for
| what kind of results you're promising. Which means the next
| batch of promises will need to be that much more extreme to get
| approved. It's a race to the top... or the bottom, depending on
| your point of view.
| ethanbond wrote:
| Yep. The incentives in science are all wrong. To maximize your
| chances of publication (i.e. keeping your job), you have to
| make the most outlandish claims you can possibly _maybe_
| defend. Additionally, the complexity of data /analysis is
| increasing every day while also the esoteric domain knowledge
| required to make any progress is deeper and more specialized.
| BitwiseFool wrote:
| Not enough people realize that science and academia are just
| as prone to organizational politics and corruption as
| everything else. Peer reviewed studies are great, _but_ ,
| just because it was published doesn't mean it represents "The
| Truth". And sadly, being skeptical of studies makes you
| appear less credible in arguments.
| diognesofsinope wrote:
| Not only this, academia is a really cushy job.
|
| When I was an economics RA literally half of the econ
| professors didn't work Fridays and barely worked summers. It
| was incredible you get paid 150k with that kind of schedule.
| IX-103 wrote:
| Interesting. In my experience professors might not be _on
| campus teaching_ one or two days a week or in summer, but
| that 's only because they are working their ass off from
| home, writing grant proposals, reviewing papers, doing
| basically anything to get funding, and trying to find time
| to manage their own research.
|
| I used to think it was a great gig too, since most
| professors had one or more small businesses on the side.
| Then I realized they have those businesses and consulting
| companies because that means they can also apply for small
| business grants (which they use to subcontract the research
| out to the university) in addition to the normal academic
| research grants. If you also count teaching, then that
| means those professors are working three jobs for one
| salary.
|
| I made the decision that I'd rather make 50% more working
| in industry doing easier (if boring) work.
| caddemon wrote:
| I'd guess that's pretty field dependent. What you're
| saying matches my experience with biology profs -
| technically once they get tenure they could chill out,
| but then they wouldn't have any funding for research
| anymore, so they wouldn't be able to do much of anything
| in their field.
|
| In CS I saw more of a mix though. It's feasible to fund a
| small research group without busting your ass, and it
| also seemed to me that putting time into coursework,
| writing books, etc. was culturally a more acceptable use
| of time in that department than it was in biology.
|
| I knew a few CS PIs that actually purposely scaled back
| their research once they got tenure because they were
| more excited about teaching and some of the educational
| initiatives the school was working on. That's not the
| norm of course, but I literally can't imagine that ever
| happening in a bio department lol.
| giantrobot wrote:
| > To maximize your chances of publication (i.e. keeping your
| job), you have to make the most outlandish claims you can
| possibly maybe defend.
|
| This is compounded by publishing negative/null results being
| disincentivized. Knowing what doesn't work can be as
| important as knowing what works.
| eloff wrote:
| Peer review is shit. It's elitist, it's actively anti
| innovation and enforces the status quo.
|
| Peer review should happen out in the open and not just be
| limited to academics.
| pm_me_your_quan wrote:
| academics are largely the only people who will be able to
| understand the work, but sure.
|
| The fact that it's not out in the open is somewhat
| complicated. You're perhaps right that it would lead to
| better outcomes, but it's also important that researchers
| feel free to speak openly.
| eloff wrote:
| I understand the concern about being free to speak
| candidly, but I think it's trumped by the need for
| transparency to ensure that if improper gatekeeping or
| other unethical behavior is happening, their reputation is
| also on the line. Basically if you can't say it to your
| peers in public, don't say it at all.
|
| This also fixes the problem of incompetent peer review,
| because it will be called out as such and the reviewer's
| reputation will suffer.
| ModernMech wrote:
| > When you want to do a proper work, your grants and papers get
| rejected because they are not innovatove enough or don't go far
| enough.
|
| Not being innovative enough isn't the _root_ cause though. The
| real issue is there isn 't enough funding to go around, and so
| the bar is higher than it needs to be. Available research
| funding in the US is a paltry sum considering the aggregate ROI
| of discoveries and technologies that originate in universities.
| Funding rates can be as low as 10-20%, with thousands of
| researchers competing for the same grants. They need to all
| paint a tortured story of how their idea will be the next big
| invention.
|
| The problem with our system is that we put public money into
| research, which is then commercialized by corporations and sold
| to consumers, and corporations/universities end up capturing
| the profits. Those profits are then invested in ways that yield
| short-term returns instead of being reinvested in research.
|
| Some of those profits are supposed to come back to the
| government and reinvested in research, but more and more
| corporations (and I consider universities to be a kind of
| corporation with the way they act like hedge funds that do
| education as a side hustle) are figuring out how to keep as
| much of those profits as possible, despite those profits only
| being made possible in the first place due to publicly funded
| research.
|
| What if we increase funding into research? VCs are willing to
| pour millions into ridiculous or tenuous ideas because they
| know a single success will more than make up for the duds.
| Lower the stakes, make funding more available to researchers,
| and then maybe we won't need to squeeze every bit of
| "innovation" out of every research dollar. Make room for
| research that fails or yields a negative result. This is
| important work that is valuable and needs to be done, but
| there's no funding for it. We could double the amount of
| funding for e.g. the NSF and it would still be a drop in the
| federal government's proverbial bucket.
| derbOac wrote:
| I get the sense from colleagues and visiting different
| universities that this varies across the US, Canada, the UK,
| and the EU, but grants are now the bread and butter at most
| US universities. It's not really enough to publish 100s of
| articles, or have a high h index, it's to bring in money even
| if it's not strictly necessary for your research.
|
| Part of the reason we have the problem you're mentioning is
| not that there isn't enough money to go around, it's that
| universities (at least in the US) now depend on inflated
| costs to function. The costs of research are kicked down the
| road to the federal government, and the research itself is
| seen in terms of profits rather than discovery. So if you
| have all these universities essentially telling researchers
| their jobs are on the line if they don't bring in profits,
| you're going to have everyone scrambling to bring in as much
| money as they can. It's not just postdocs or untenured
| research professor lines, it's tenured professors as well,
| whose income can be brought down below some necessary
| standard of living, or who can have salaries frozen or
| resources cut.
|
| I was thinking about this the other morning. I had a grant
| proposal that the program officer was really excited about.
| This program of research could probably be conducted for
| almost nothing because it involved archival data analysis.
| _However_ if you put a dollar amount on the time, it _might_
| realistically actually cost around 250k USD, maybe 500k max,
| pretty generously in terms of staff effort. However, the
| university managed to inflate the budget ask to around 2
| million for the sole purpose of indirect funds.
|
| When you have that kind of monetary incentive (carrot or
| stick), of course you're going to have thousands of persons
| applying for each opportunity. It's what led to the graduate
| student ponzi scheme, inflated numbers of surplus graduates,
| etc and so forth.
|
| It all trickles down too, in terms of research claims,
| p-hacking, etc and so forth.
|
| There's a place for profit, but there's also some realms
| where it does nothing but corrupt.
| native_samples wrote:
| The problem here is not profit but the reverse, the
| corruption comes from the absence of profit.
|
| Universities and grants are this firehose of tax money
| being sprayed everywhere without even the slightest bit of
| accountability in how it's used. The government effectively
| "loses" all of it in accounting terms, but because it's tax
| it doesn't matter. The buyer is blind and doesn't even
| bother looking at the papers they've paid for, let alone
| caring about the quality.
|
| Now go look at the results coming out of corporate labs
| when the corporates actually want to use the tech. You get
| amazing labs that are consistently re-defining the state of
| the art: Bell Labs, DeepMind, Google Research, FAIR, Intel,
| ARM, TSMC etc. The first thing that happens when the
| corporate labs get interested in an area is that
| universities are immediately emptied out because they
| refuse to pay competitive wages - partly because being non-
| profit driven entities they have no way to judge what any
| piece of research is actually worth.
| ModernMech wrote:
| > Universities and grants are this firehose of tax money
| being sprayed everywhere without even the slightest bit
| of accountability in how it's used.
|
| This is definitely not true, recipients of grants are
| heavily restricted on what kind of things they can spend
| that money on. I can't even fly a non-domestic carrier
| using grant money without proving no other alternatives
| exist.
|
| Do research projects sometimes fail to deliver? Yeah. But
| that's just the reality of doing research. The problem I
| see is people expect research to be closer to
| development, with specific ROIs and known deliverables
| years ahead of time. Sometimes in the course of research
| you realize what you said you were going to do is
| impossible, and that's a good result we need to embrace,
| instead of attaching an expected profit to everything.
|
| > Bell Labs, DeepMind, Google Research
|
| I don't know so much about all the labs you listed, but
| just taking these three, they certainly don't have a good
| feeling for what their research is worth either. Do you
| think Bell Labs fully comprehended the worth of the
| transistor? For all the research Google does, ad money
| still accounts for 80% of their revenue. DeepMind is a
| pretty ironic choice because Google has dropped billions
| into them and it's still not clear where the profit is
| going to come from. So it's not clear anyone, even those
| with a profit motive, have any way to judge what any
| piece of research is actually worth.
|
| But that's not to say there's anything wrong with that...
| that's just how research works. You don't know how things
| are going to turn out, and sometimes it takes a very long
| time to figure that out, and it. This is why massive
| corporations like AT&T, Intel, Google, Xerox, MS etc. are
| able to run such labs.
|
| > The first thing that happens when the corporate labs
| get interested in an area is that universities are
| immediately emptied out because they refuse to pay
| competitive wages
|
| I've seen this happen first hand. In my experience these
| researchers usually go on to spend their time figuring
| our how to get us to click on more ads or to engage with
| a platform more. In one instance, I remember one of my
| lab mates being hired out of his PhD to use his research
| to figure out which relative ordering and styling of ads
| on a front page optimized ad revenue for Google. They
| paid him quite a lot of money to do that, and I guess it
| made Google some profit. But is the world better off?
| bluGill wrote:
| > This is definitely not true, recipients of grants are
| heavily restricted on what kind of things they can spend
| that money on. I can't even fly a non-domestic carrier
| using grant money without proving no other alternatives
| exist.
|
| That is pure corruption: the grant is funneling money
| from you to a domestic ariline. If it was about
| accountability you would have to prove the flight was
| really needed in the first place, and then that you found
| the best price. (though the grant should allow you to
| ignore the "skip maintenance and pilot training to give
| you a lower price" airline, but if that best happens to
| be foreign it shouldn't matter to the grant unless there
| is corruption involved)
| pm_me_your_quan wrote:
| > If it was about accountability you would have to prove
| the flight was really needed in the first place,
|
| Friend, at a certain point the overhead to administrate
| these kinds of checks is more costly than just letting
| people buy tickets to go to conferences. And at this
| point it isn't corruption in the university, it's in the
| form of handouts to large corporations.
| native_samples wrote:
| _recipients of grants are heavily restricted_
|
| They are restricted in trivial ways that are easy for a
| bureaucracy to mechanically enforce, as is true of
| employees at every institution.
|
| What I meant by accountability is deeper: people are not
| accountable for the quality or utility of their work,
| hence the endless tidal wave of corrupt and deceptive
| research that pours out of government funded 'science'
| every day. These researchers probably filled out their
| expenses paperwork correctly but the final resulting
| paper was an exercise in circular reasoning, or the data
| tables were made up, or it was P-hacked or whatever. And
| nobody in government cares or even notices, because
| nobody is held accountable for the quality of the
| outputs.
|
| Whilst DeepMind is not especially interested in profit
| it's true, and is just doing basic research, Google
| itself is an excellent example of how to seamlessly
| integrate fundamental research with actual application of
| that research. That's what profit motivated research
| looks like: just this endless stream of innovative tech
| being deployed into real products that are used by lots
| of people, without much drama.
|
| We have come to take this feat so much for granted that
| you're actually asking if someone working on ads is
| leaving the world better off. Yes, it does. Google ads
| are clicked on all the time because they are useful to
| people who are in the market to buy something. Those ads
| are at the center of an enormous and very high tech
| economic engine that powers staggering levels of wealth
| creation. If I understand correctly, a lot of academic
| papers are actually never cited by anyone - a researcher
| who optimises search ads by just 1% will have a positive
| impact on the world orders of magnitude greater than
| that.
| ChrisMarshallNY wrote:
| I'm actually kind of flabbergasted that people -no matter who
| they are- are automatically given the benefit of the doubt,
| without question.
|
| I'll bet that a lot of folks just assume that anything they do
| will be taken at face value, without question or inspection. I
| also suspect that many "brought and paid for studies" are done
| this way.
|
| For my own work, I generally assume that most of these studies
| are pretty much worthless, and tend to do some of my own homework
| before accepting them. Since most don't concern me at all; it's
| not a big deal.
|
| Health is just one place this kind of thing happens. Software
| Development is absolutely _rife_ with bad implementations. I am
| not in AI, but I have heard from a number of people that AI has a
| big problem with irreproducible results.
|
| https://web.archive.org/web/20190926055757/http://www.jir.co...
| version_five wrote:
| I work in ML research and I used to do experimental physics.
| I'd agree that specific results in papers can be hard or
| impossible to reproduce, but that never really bothers me
| because at least in my work, the specific experimental result
| is rarely material to why I'm interested in the paper. It's
| more of a demo, and like a demo, you know its orchestrated to
| look good. What I'm interested in is what is the mechanism
| behind the advance and do I think its applicable or relevant to
| what I'm doing. If the paper is really just a random
| observation of something that worked better, without a causal
| explanation, it's not very interesting, but I don't see those
| often.
|
| Maybe health research is very different, and people are
| latchjng on to surprising results they find in papers, but I
| doubt it's a big problem in academia, much more likely in the
| media. If I was a doctor and saw an out of the blue study
| claiming a surprising result, I'd discount it accordingly. If I
| saw a causal explanation with evidence, I'd give it closer
| scrutiny and follow up if it seemed relevant to me. That is how
| research works in my experience.
| AlexCoventry wrote:
| The video is worth watching. It goes into much more detail about
| just how extensive, brazen and destructive the fraud has been.
| https://www.youtube.com/watch?v=3BXEOey62O4
| xyzzy21 wrote:
| Sadly true.
|
| And especially government agencies and non-profits related to
| health.
| sjwalter wrote:
| "The case against science is straightforward: much of the
| scientific literature, perhaps half, may simply be untrue.
| Afflicted by studies with small sample sizes, tiny effects,
| invalid exploratory analyses, and flagrant conflicts of interest,
| together with an obsession for pursuing fashionable trends of
| dubious importance, science has taken a turn towards darkness."
|
| Richard Horton, current editor of The Lancet:
| https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
| sjwalter wrote:
| This is also especially ironic considering The Lancet published
| a totally fabricated study that supposedly demonstrated how
| dangerous a Trump-touted Covid treatment was.
|
| https://www.the-scientist.com/features/the-surgisphere-scand...
| [deleted]
| cheaprentalyeti wrote:
| Well, while you're at it, can you tell me which side is telling
| the truth about Remdesivir, the US health care bureaucracy or the
| WHO?
| wolverine876 wrote:
| The headline is more than a bit sensationalist. I never know what
| to make of BMJ, which sometimes seems sensationalist: Can anyone
| in the industry or profession characterize who they are, what
| they do, and what their reputation is?
| dpatru wrote:
| As shown both in the article and in the comments, scientific
| establishment seems content to tolerate fraud. But when research
| goes against big money interests, suddenly standards become very
| strict. See how Andrew Wakefield was treated.
| https://www.bmj.com/content/342/bmj.c7452
| arcticfox wrote:
| > See how Andrew Wakefield was treated.
|
| Are you implying that he was let off easy? Should have gone to
| jail, IMO: "The panel found he had subjected 11 children to
| invasive tests such as lumbar punctures and colonoscopies that
| they did not need, without ethical approval."
|
| Performing unfounded experiments on children while committing
| fraud should be treated as criminal, not simply reputation-
| destroying, IMO.
|
| https://www.theguardian.com/society/2010/jan/28/andrew-wakef...
|
| (I understand we need to protect researchers from some
| liabilities in the ethical pursuit of progress, but he was so
| far over the line that the "slippery slope" argument is kind of
| silly. Should medical researchers be immune to _all_
| prosecution no matter what they do or what lies they tell?)
| Animats wrote:
| There's the famous story of the doctor who figured out, in the
| 1980s, how to cure stomach ulcers.[1] Stomach ulcers are
| usually a bacterial disease, and antibiotics work.
|
| _The microbiologists in Brussels loved it, and by March of
| 1983 I was incredibly confident. During that year Robin and I
| wrote the full paper. But everything was rejected. Whenever we
| presented our stuff to gastroenterologists, we got the same
| campaign of negativism. I had this discovery that could
| undermine a $3 billion industry, not just the drugs but the
| entire field of endoscopy. Every gastroenterologist was doing
| 20 or 30 patients a week who might have ulcers, and 25 percent
| of them would. Because it was a recurring disease that you
| could never cure, the patients kept coming back. And here I was
| handing it on a platter to the infectious-disease guys._
|
| In 2005, he got a Nobel Prize.
|
| [1] https://www.discovermagazine.com/health/the-doctor-who-
| drank...
| rubatuga wrote:
| Sounds like ivermectin is going through something similar.
| This off-patent, cheap, and safe drug is able to
| significantly reduce COVID-19 symptoms [0], and is
| _extremely_ likely to be a potent prophylactic and treatment
| for COVID-19 [1], to the extent that the third wave in North
| America wouldn 't have happened if it was used. The inventor,
| Satoshi Omura, won a Nobel Prize for inventing the drug in
| 2015, and tried to convince Merck (the original manufacturer)
| for many months to conduct an ivermectin trial for COVID-19,
| to no avail. On July 1st, he finally found a Japanese company
| called Kowa to charitably conduct a clinical trial for
| ivermectin, without Merck's help. Amazing right? But what
| response does Omura get from Western media? Quickly, his
| announcement video was deleted from YouTube [2]. You cannot
| find any English news about the Kowa trial being conducted. A
| few days ago, Omura was interviewed about ivermectin for the
| first time, on Yahoo Japan News [3]. Quoting him (using non-
| ideal Google Translate)
|
| > "My impression of WHO is that I feel sorry for being caught
| in a dilemma. Until now, I have only seen bright light in my
| life as a researcher. But this time, I learned for the first
| time after reading this article that shadows also exist in
| the world... Ivermectin is no longer a scientific issue, but
| a political issue." --Satoshi Omura
|
| It seems like big tech's misinformation crusade is biting us
| and science in the ass.
|
| [0]: https://pubmed.ncbi.nlm.nih.gov/33495752/
|
| [1]: https://academic.oup.com/ofid/advance-
| article/doi/10.1093/of...
|
| [2]: https://twitter.com/michaelcapuzzo/status/14106267691667
| 3331...
|
| [3]: https://news.yahoo.co.jp/articles/92c85ad9476f56a6fc51da
| ec56...
| Animats wrote:
| Here's a list of ivermectin trials on COVID-19.[1] It does
| seem to have some effect, cutting recovery time in mild
| cases by 20% or so. But that's not anything close to "would
| eliminate the third wave".
|
| This study [2] from the early days of the epidemic
| indicated that the patients receiving ivermectin needed
| invasive ventilation much earlier. Which is not a good
| result.
|
| [1] https://www.covid19treatmentguidelines.nih.gov/therapie
| s/ant...
|
| [2] https://pubmed.ncbi.nlm.nih.gov/34215210/
| kobieyc wrote:
| Honestly I was surprised not to see any mention of IVM in
| the original post. Many of the points in the original
| article apply to what's going on with systemic reviews of
| IVM - see for example allegations of misconduct/fraud by
| theguardian to Elgazzar's big IVM study
| https://www.theguardian.com/science/2021/jul/16/huge-
| study-s...
|
| It does also bring into question the validity of Tess
| Lowrie's systematic review of ivermectin efficacy.
|
| Also, your systematic review [1] includes the Elgazzar as
| "low risk" of bias... when in fact Elgazzar had GLARING
| errors.
|
| It also makes me question the competence of everyone
| involved in this systematic review that they can't find
| these glaring errors but some random med student can.
| rubatuga wrote:
| I agree, some of the trials have issues. But if you would
| like to cherry pick one trial and use that as evidence to
| the contrary, I will direct the reader to a firehose of
| ivermectin studies, which the reader can evaluate on
| their own:
|
| https://ivmmeta.com
|
| You really _cannot_ trust the raw risk estimates that
| they give on this site. But it 's the most comprehensive
| list of trials for ivermectin there is, and a place for
| you to form your own opinion. Elgazzar has already been
| removed as a data point.
| closewith wrote:
| > I agree, some of the trials have issues. But if you
| would like to cherry pick one trial and use that as
| evidence to the contrary, I will direct the reader to a
| firehose of ivermectin studies, which the reader can
| evaluate on their own:
|
| Surely the point of the OP's post is that the reader _can
| 't_ evaluate these studies on their own. At least not
| without undertaking the kind of review and background
| research that is not reasonable for even the expert
| reader.
|
| Not to mention the irony of accusing the GP of cherry-
| picking when the site you linked is a cherry-picked list
| of trials curated by anonymous alleged HCWs.
| pjc50 wrote:
| Wakefield was treated far too leniently? He's lucky that he
| only got struck off, and far too late. He managed to get a lot
| of media complicity, too.
| Woberto wrote:
| It took 12 years for the paper to be retracted, so I wouldn't
| necessarily say "suddenly".
|
| Also, I think fraud is easier to miss (which may seem as it
| being tolerated) if it's something people expect or is not such
| a big change from the norm. For example, I don't remember the
| specifics but there's a story of some constant that was
| estimated a long time ago, and as people tried to measure it
| more accurately, those who calculated a value too different
| from the previous estimate were rejected.
|
| I bring that up because in this case, Wakefield's claim may
| have been so outlandish as to provoke intense scrutiny, which,
| as the paper you linked to says, led to findings of fraud.
| giantg2 wrote:
| I think just as importantly is reining in the misapplication of
| studies. Too often I see some news/blog/politician/other say some
| _policy or fact_ is _proven_ by a study only to find that the
| study doesn 't mean what they are claiming.
|
| This can be stuff like using animal studies not supported by
| further human studies and claiming the effect is true for humans.
| Or confusing correlation for causation. Or viewing the
| _speculated_ application of the study found in the conclusion to
| be absolute truth when many times the authors themselves claim
| additional studies would be needed to evaluate other aspects or
| confirm their findings.
|
| A classic example was gender wage gap misrepresentation about 6-8
| years ago. Many news groups and even the president were
| misinterpreting (and spreading that misinformation about) the BLS
| study to mean that a man and a woman in the same job with all
| else equal, the woman would only make $.80 on the dollar, when in
| fact the issue is an aggregate level issue mostly due to
| structural issues (and require different remedies than proposed).
| At least it seems many places have since realized their mistake,
| yet the misinformation persists in the general public.
| ineedasername wrote:
| _he set about investigating the trials and confirmed that they
| hadn't ever happened. They all had a lead author who purported to
| come from an institution that didn't exist_
|
| To me, this doesn't mean that simple distrust is the answer.
| These are basic issues that should be revealed with even minimal
| due diligence during the editorial & peer review process.
|
| _Peer reviewers_ and _Journal Editors_ should brink a skeptical
| mindset to article submissions from the outset before they 're
| ever accepted for publication.
|
| After that? Well, whenever research is on an emerging topic there
| is a certain amount of _scientific_ skepticism you should use.
| Same if results go against an established consensus on a topic.
| However this is where the "replication problem" enters the
| picture because replicating research has a lower status.
|
| When it comes to media reporting, things get even more
| complicated. New science is messy. You only have to look at COVID
| research for the past 1.5 years, and when it's an issue of such
| public urgency, _EVERY_ development hits the public eye, pulling
| back the curtain on the sausage factor. Because new science is
| rarely "Hey look what I discovered!" followed by "Yay we all
| agree!" It's more of a conversation or dialectic, with ever more
| research revealing the picture a bit more until there's enough to
| be confidence in a given interpretation. And even there, work
| proceeds on alternatives.
|
| The above is very much _NOT_ how science is taught to the public
| in schools. You learn "Darwin Discovered Evolution!", not the
| significant years-long process of researchers arguing it out,
| sometimes even with heated vitriol. You learn "Newton Discovered
| Gravity!", not all of the complexities and disagreements that
| continue even to today.
|
| Out education systems have failed society when it comes to truly
| understanding the scientific process. This is why distrust of
| science increased. Because in past decades awareness of
| scientific advances often only reached the public after at least
| part of the sausage was made, meaning now it looks like it's
| descended into total disarray.
| enriquto wrote:
| why just health research? all science is already based on that
| very principle. In fact, a stronger one: you cannot ever prove
| that some research is "true", only that no inconsistency has yet
| been observed.
| throwawayboise wrote:
| Replicating the results would be evidence that the original
| research was "true."
|
| Unfortunately nobody wants to fund research that only attempts
| to replicate an already published result, when they could spend
| the same money on novel research.
| hirundo wrote:
| I doubt most people understand the intensity of the incentive to
| mess with the data. In college I was a lab assistant for a
| professor who taught courses on research integrity and how to
| evaluate the quality of scientific papers. After thousands of
| hours of work on a study with routine 20 hour days collecting
| data, he wasn't getting what he needed to publish. At the tail
| end of one of those days I caught him with the equivalent of his
| thumb on the scale. He gave an excuse that he would have failed
| as an answer on one of his own tests. I argued a bit but then
| shut up. I kept shut up while that data was not excluded from the
| analysis that was eventually published. It wasn't enough to
| change the result, but still bothered me.
|
| So yeah, trust maybe but verify definitely. The rewards for
| faking it are just too great for an honor system to be reliable.
| diob wrote:
| I think it's more that the punishment for not faking it is too
| great. We need to be okay with following the scientific method
| and rewarding folks regardless of the outcome. Otherwise we're
| bound to see everything "succeed".
| marsven_422 wrote:
| This just in : Scientists are just as human as everyone else.
| miga wrote:
| No, just a statistical reality of multiple hypothesis testing.
|
| Just like you wait a few blocks for a confirmation on a
| blockchain, you have more and more confidence with confirmation
| of health research by independent papers.
|
| See statistical analysis in Ioannidis' "Why most research
| findings are false":
| https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
| 0xcde4c3db wrote:
| I knew there were issues with various kinds of research. Things
| like p-hacking, "touching up" data, and so on. But the lead
| example is pretty wild:
|
| > As he described in a webinar last week, Ian Roberts, professor
| of epidemiology at the London School of Hygiene & Tropical
| Medicine, began to have doubts about the honest reporting of
| trials after a colleague asked if he knew that his systematic
| review showing the mannitol halved death from head injury was
| based on trials that had never happened. He didn't, but he set
| about investigating the trials and confirmed that they hadn't
| ever happened. They all had a lead author who purported to come
| from an institution that didn't exist and who killed himself a
| few years later. The trials were all published in prestigious
| neurosurgery journals and had multiple co-authors. None of the
| co-authors had contributed patients to the trials, and some
| didn't know that they were co-authors until after the trials were
| published.
| wolverine876 wrote:
| It's one example, chosen and presented by someone with
| something to prove, and which fails to provide any evidence
| (such as the names of the studies or lead author).
| sjwalter wrote:
| There are plenty of other good examples.
|
| Try this one: https://www.the-scientist.com/features/the-
| surgisphere-scand...
|
| "It sounds absurd that an obscure US company with a hastily
| constructed website could have driven international health
| policy and brought major clinical trials to a halt within the
| span of a few weeks. Yet that's what happened earlier this
| year, when Illinois-based Surgisphere Corporation began a
| publishing spree that would trigger one of the largest
| scientific scandals of the COVID-19 pandemic to date.
|
| "At the heart of the deception was a paper published in The
| Lancet on May 22 that suggested hydroxychloroquine, an
| antimalarial drug promoted by US President Donald Trump and
| others as a therapy for COVID-19, was associated with an
| increased risk of death in patients hospitalized with the
| disease."
|
| They completely fabricated the data.
|
| To hurt Trump.
| mcguire wrote:
| " _...Ian Roberts, professor of epidemiology at the London School
| of Hygiene & Tropical Medicine, began to have doubts about the
| honest reporting of trials after a colleague asked if he knew
| that his systematic review showing the mannitol halved death from
| head injury was based on trials that had never happened. He
| didn't, but he set about investigating the trials and confirmed
| that they hadn't ever happened. They all had a lead author who
| purported to come from an institution that didn't exist and who
| killed himself a few years later. The trials were all published
| in prestigious neurosurgery journals and had multiple co-authors.
| None of the co-authors had contributed patients to the trials,
| and some didn't know that they were co-authors until after the
| trials were published. When Roberts contacted one of the journals
| the editor responded that "I wouldn't trust the data." Why,
| Roberts wondered, did he publish the trial? None of the trials
| have been retracted._"
|
| I realize that meta-analysis is regarded as a valid research
| method, if not one of the best, but honestly, I don't know why.
| If the original studies are garbage, no amount of statistical
| manipulation is going to make them not-garbage.
| metalliqaz wrote:
| well if the original studies are inadequate simply because N is
| too low, then grouping many of them can resolve that problem.
| mahogany wrote:
| Is that really true from a statistical standpoint? It seems
| to me that running ten 20-person studies is different and
| less valuable than running a single 200 person study, because
| each of the 20-person studies has a much larger error range
| that you have to account for. But I'm also not good with
| stats.
| mcguire wrote:
| That's ideally what the meta-analysis is supposed to
| correct for.
|
| But if two of the studies are just completely made up and
| another couple have errors in their design that aren't
| mentioned in the write-up....
| FeteCommuniste wrote:
| Meta-analysis normally tries to exclude "low-quality" studies
| but if the standard of honesty in a field or sub-field is truly
| abysmal, I guess it's GIGO.
| rohanphadte wrote:
| Some highlights to show how health research is published:
|
| > Mol, like Roberts, has conducted systematic reviews only to
| realise that most of the trials included either were zombie
| trials that were fatally flawed or were untrustworthy.
|
| > But the anaesthetist John Carlisle analysed 526 trials
| submitted to Anaesthesia and found that 73 (14%) had false data,
| and 43 (8%) he categorised as zombie. When he was able to examine
| individual patient data in 153 studies, 67 (44%) had
| untrustworthy data and 40 (26%) were zombie trials.
|
| > Others have found similar results, and Mol's best guess is that
| about 20% of trials are false. Very few of these papers are
| retracted.
| mcguire wrote:
| You missed a sentence.
|
| " _Many of the trials came from the same countries (Egypt,
| China, India, Iran, Japan, South Korea, and Turkey), and when
| John Ioannidis, a professor at Stanford University, examined
| individual patient data from trials submitted from those
| countries to Anaesthesia during a year he found that many were
| false: 100% (7 /7) in Egypt; 75% (3/ 4) in Iran; 54% (7/13) in
| India; 46% (22/48) in China; 40% (2/5) in Turkey; 25% (5/20) in
| South Korea; and 18% (2/11) in Japan._"
|
| I find it particularly sad, since actively promoting academic
| integrity would do more for those countries than anything else,
| bang-for-your-buck-wise. Instead, many seem to be seeking the
| appearance of academic success.
|
| (OTOH, I suppose Japan and South Korea may be on that list due
| to some kind intense pressure to succeed.)
| peytn wrote:
| No, but you need to be able to ask people who're "in the know,"
| e.g.
|
| > When Roberts contacted one of the journals the editor responded
| that "I wouldn't trust the data."
|
| People already know what the bullshit results are.
| diognesofsinope wrote:
| Similar to how in psychology replication studies when they
| asked professors to predict which studies would and wouldn't
| replicate I think they were ~75% correct.
|
| Replication was ~50%.
|
| They have an idea of what is bullshit, but there's a very
| strong culture of 'don't call others out on bad research'
| caseysoftware wrote:
| If the editor didn't trust the data, why did they publish?
|
| The people who keep informed on their field (aka "in the know")
| would then be tainted because they would likely believe a
| journal would vet the data, process, and researchers before
| publication.
|
| Unless you mean "in the know" in that they know the entire
| publishing system is a scam... and the ripples from that are
| huge.
| topspin wrote:
| "If the editor didn't trust the data, why did they publish?"
|
| They're in on it. Without material the journals have nothing
| to publish. So their inclination is to accept and publish
| with as little friction as possible.
|
| Publish or Die. Remember? That applies to the whole supply
| chain, not just poor put upon individual researchers.
| KittenInABox wrote:
| How do you know if any individual person is "in the know"?
| earleybird wrote:
| With an appeal to authority :-)
| wrycoder wrote:
| You evaluate their background, publications, and associates,
| and then you use your best judgement.
| codingwageslave wrote:
| Non elite stem academia is essentially a back door immigration
| program. The research is basically useless, and incremental at
| best
| alanbernstein wrote:
| The "time to assume" this was at least 15 years ago:
| https://en.wikipedia.org/wiki/Why_Most_Published_Research_Fi...
| gotoeleven wrote:
| The war on standards is creating a low trust society.
| onlyrealcuzzo wrote:
| I don't know. The amount of people that take one poorly
| measured data point and use that as a north star seems more
| troubling to me - than a society with low trust.
|
| I'm not convinced a low-trust society is inherently bad. I am
| convinced that a society that has high trust in garbage data is
| bad.
| patrec wrote:
| I know this is a forum for nerds, but I struggle to
| understand how anyone could seriously believe that being
| surrounded by people who are either psychopathic or paranoid
| is probably A-OK, as long as they are all competent
| statisticians.
| onlyrealcuzzo wrote:
| You're a psychopath or paranoid if you don't blindly trust
| data - without putting some effort into seeing what the
| quality of the data is and how much evidence supports it???
| patrec wrote:
| > I'm not convinced a low-trust society is inherently
| bad.
|
| The term "low-trust society" is generally not used to
| distinguish societies where people put enough emphasis
| into seeing what the quality of the data is and how much
| evidence supports it. Maybe because no one ever heard of
| such a thing. I do share your lack of conviction that
| such a society would inherently be a bad thing.
|
| Maybe we in fact agree that a low trust society in the
| conventional sense -- one characterized by low
| interpersonal trust -- is less appealing?
| long_time_gone wrote:
| ==I'm not convinced a low-trust society is inherently bad. I
| am convinced that a society that has high trust in garbage
| data is bad. ==
|
| It feels like we currently have both at the same time. There
| is "low trust" in any data that doesn't confirm an existing
| bias, but "high trust" in the data we want to be true.
|
| I see it all the time in HN comments. Any study that
| contradicts the consensus is met with comments about
| "correlation =/= causation", while anything that supports the
| census is supported and correlation/causation isn't
| mentioned.
| UnFleshedOne wrote:
| I think a better term for that is "brain damage". So what
| if all our brains are built damaged. Saying people are not
| reasoning machines, but social maneuvering machines doesn't
| change the fact they are kinda broken...
| gotoeleven wrote:
| I meant more that society's attitude toward standards of any
| kind--honesty, proper behavior, civility--has become
| increasingly suspicious. These standards are, some argue,
| simply power structures to promulgate the various -isms that
| plague society.
|
| If you've got scientists that believe that the standard of
| honesty is an artificially constructed power structure then
| obviously you can't trust them.
| vixen99 wrote:
| I wonder if this will apply to research published by
| pharmaceutical companies with a proven history of illegal
| activity in support of the products they release.
| RandomLensman wrote:
| I think the first important question to ask is: is the research I
| am looking at directly applicable and relevant to other people?
|
| If stuff is highly applicable and relevant, then the chance to
| get away with made up rubbish starts to drop quite a bit, because
| people will want to try for themselves - and fail. It gets more
| problematic when the application takes a long time to show
| effects, i.e. long observation periods - there more can be faked.
|
| Most published research is largely inconsequential and not
| interesting to but a tiny few, so "fraud" can easily hide in
| there. Yes, it increases the sum total of knowledge, but that
| might be about it. I am not saying most research isn't done well,
| just that there isn't a way to assess correctness at scale and
| only high-profile results might get a fast check (and even
| there...)
| kazinator wrote:
| Betteridge's Law of headlines says: no!
|
| We can be skeptical of researchers' results without the hostility
| of assuming that they are guilty of fraud until proven innocent.
| i_left_work wrote:
| From my experience, most of it is. I just left a high paying
| position working in the healthcare space as a data scientist,
| because it became clear this was known and there was no intention
| to improve the situation. Instead, the focus was on selling and
| making a quick exit.
| tarere wrote:
| I will address the elephant in the room on this one.
|
| 168 comments so far, I did a research for word "vaccine" in the
| page, not a single result. So I assume this has not been
| discussed despite the current situation we are living right now.
|
| I've just read 2 articles about Moderna in general and Bancel in
| particular :
| https://www.bloomberg.com/news/features/2021-07-14/moderna-m...
| And Vanity Fair, unfortunatly in french
| https://www.vanityfair.fr/pouvoir/business/story/stephane-ba...
|
| This is a frightenning read as we people in France are being
| literraly forced by our gouvernement to be vaccinated (Moderna,
| Pfizer) or be socially terminated.
|
| I did not know, I'm pretty sure 99,99% of people also don't, that
| Moderna did not release a single product before 2020 and it "all
| in" bet in the Covid Vaccine with up to 1 billion funding from
| operation warp speed. Moderna was sometimes refered as the next
| Theranos. His leader fits clearly in the sociopath territory of
| Silicon Valley tranhsumanist billionnaires. "risk very big, win
| very big" is his mantra, this man wants to vaccinate billions of
| people annualy (read article please).
|
| More, if you read those two articles, you could change Vaccine by
| any software product and you would have a typical business
| article about a Silicon Valley startup. This is frightening to
| death to think that this technology "software vaccine" is to be
| used on the whole population with only a few month of study and,
| worse, with "forced consentment" on populations.
|
| Guess what happens when money conflict with health.
| nabla9 wrote:
| No results because vaccines are not accepted based on
| publications and peer review on scientific papers.
|
| They go trough completely different and extremely rigorous
| testing process where everything is documented carefully,
| documents are examined and double checked.
|
| It's great to see 168 comments before first anti-vaxxer
| comment.
| tarere wrote:
| I was speaking of the HN page discussion, but it seems my
| search on Firefox is completely broken, so I withdraw my
| mention of no vaccine reference, but I stand to be commented
| on the subject.
| kobieyc wrote:
| Fraud vs incompetence is an important dimension here. I think a
| lot of the time people are just incompetent. In fact I assume
| that > 80% of scientific papers in fields with high levels of
| environmental noise (social science, environment science,
| medical, etc) are bad science.
|
| So TL;DR I just assume incompetence instead of fraud.
| 0xbadc0de5 wrote:
| Wouldn't this be the default scientific position? Or to take some
| of the hyperbole out of the statement, rephrase as: "is it time
| to assume that health research hypotheses are incorrect until a
| preponderance of data and reproducible studies prove otherwise?"
| A4ET8a8uTh0 wrote:
| Scientific.. yes. But the studies are intended and/or used for
| a specific business purpose. The moment you recognize this
| simple reality, it becomes extremely difficult to take anything
| at face value. My wife is on the other side of the spectrum.
| She explicitly believes that companies/researchers/people
| generally want to do the right thing. It is infuriating,
| because my personal approach is my approach to games: 'shit,
| until proven otherwise'.
| ChuckMcM wrote:
| Exactly, that's the thing about science, nobody believes it
| until everything that has been tried to show it is wrong fails.
| I wish this were taught more in schools.
| hanniabu wrote:
| Yeah but you also can't go test 100 previous studies that the
| work you want to do is based on before you can even start
| yours. That's extremely inefficient, wasteful, and will
| tremendously slow progress.
| guscost wrote:
| > That's extremely inefficient, wasteful, and will
| tremendously slow progress.
|
| Democracy is also "extremely inefficient, wasteful, and
| slow to progress". A process with those deficiencies can
| still be the best available approach (although your example
| is perhaps too far in that direction).
| Dylan16807 wrote:
| "preponderance" is doing a lot of work in that statement. I
| would argue that this is mostly about reevaluating our
| standards for preponderance.
|
| And it's really hard to know if a study is reproducible. We
| could assume everything is incorrect until it has already been
| reproduced by an unrelated party, and I think that _would_ be a
| major change in thought.
| aqme28 wrote:
| There's a pretty big difference between an "incorrect"
| hypothesis and "fraud."
| FeteCommuniste wrote:
| Exactly. It's one thing to, say, do a study that arrives at
| wrong conclusions because of insufficient controls or subtle
| mistakes in statistics. Quite another to simply invent
| patients or make up numbers.
| satchlj wrote:
| This should be the default scientific position, however because
| people (including scientists) care greatly about their health and
| the health of their loved ones, they are very likely to latch on
| to things that they would like to be true
|
| science is almost never practiced in its ideal form; maybe it's
| _Time to assume that results from our scientific institutions are
| flawed_ which is my assumption
| jgeada wrote:
| It is about time to focus on the right problem: management
| standards that cause this crap to be pushed, and the effective
| immunity from consequences companies have when they lie.
|
| Why blame scientists when power is actually with management?
|
| And why let management and investors get away with this? It is
| about time "limited" liability had a pass through liability for
| this type of stuff: if you lie, knowingly or not, there are
| consequences and the consequences bypass limited liability. I bet
| if that happens this type of crap would immediately cease!
| nathanaldensr wrote:
| Unfortunately, you yourself are assuming that the actors in
| charge of holding people are accountable are _themselves_
| trustworthy.
|
| I feel we are approaching a singularity of low trust between
| people. It's only getting worse. You can't trust the watchers
| (journals) and you can trust the watchers' watchers
| (governments and the law). You can only trust yourself at the
| end of the day.
| east2west wrote:
| Already reached the conclusion when I saw a genetic researcher
| presenting his p-value < 10^-40 as better than < 10^-10. I kept
| my mouth shut because I didn't want to ruin the poor guy's moment
| in the sun, but I knew it was time to get out.
| function_seven wrote:
| My naive understanding is that "smaller p-value" == "more
| likely result is true".
|
| I know there's always more nuance in statistical reasoning, but
| the first number _is_ vastly smaller than the second one,
| right? Is it just that both are hilariously tiny and not
| credible? Or is there no additional value after you get into
| the one-in-billions territory?
| timy2shoes wrote:
| > My naive understanding is that "smaller p-value" == "more
| likely result is true".
|
| I think you're making the classic Prosecutor's fallacy:
| https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy. In my
| experience, smaller p-value tends to be more of a measure of
| sample size than anything else, or an overly restrictive null
| distribution that is almost certain to be rejected.
| east2west wrote:
| Exactly, numerical errors could easily have accounted for the
| difference between already tiny p-values. The point isn't
| that the smaller p-value isn't better than the bigger one, it
| is, but that small significance should have been attached to
| the difference.
|
| This example is a gnome-wide genetic association study. Every
| genetic variations are tested, so at least 500K or more
| linear regressions were performed. This many statistical
| tests could lead to many false positives just by chance, so
| one must do multiple-testing corrections. The end result of
| multiple-testing correction is much bigger and therefore
| worse p-values. Hence the drive toward ridiculously tiny
| p-values.
| native_samples wrote:
| Yeah I'm also mystified by that comment. You are correct that
| smaller P is better. Those near physics level P-values are
| not totally unheard of for genetics either, because they have
| very large databanks with hundreds of thousands of data
| points in them and the ability to do large analyses over
| them, so they can obtain a lot of statistical power.
| CrazyStat wrote:
| Precision in p-values that small is more or less
| meaningless in almost all cases, because any violation of
| model assumptions will result in p-value imprecision far
| greater than 10^-10. p-values are (almost always)
| approximations based on an approximate model, and the
| variation between the model and reality is probably more
| than 10^-10.
|
| Some tiny aspect of the real process that your model falls
| to capture might mean that that 10^-10 is actually 0.001,
| and 10^-40 is also 0.001. In complex biological fields it's
| fair to assume that there are always such tiny aspects.
| brilee wrote:
| I estimate the risk of human error (chose the wrong modeling
| assumptions, bug in data processing code, etc.) at least ~1%,
| so there really isn't any point in claiming any statistic
| that is smaller than that.
| native_samples wrote:
| P values in particle physics are much, much lower than the
| base human error rate though. Unless you think those are
| wrong?
| rich_sasha wrote:
| It would be, but such an imbalance of p-values is
| unrealistic. 10^-10 probability? If your probabilistic model
| includes even a one in a billion chance of messing up
| (10^-9), a p-value of 10^-10 is already too small. That's
| before you look at 10^-40... so they are probably both wrong.
|
| A nice demo of this effect is DNA matching in criminology.
| Although DNA matching of suspects to DNA samples can be
| insanely accurate, in practice it is limited by the incidence
| of monozygotic (identical) twins, which is about 3 in 1,000.
| You cannot be more certain than this that you got a match,
| essentially.
| ajsnigrutin wrote:
| Another problem with studies is, that negative results are rarely
| published unless it's something really really "interesting".
|
| "we tried treating X with Y, and it didn't help (even though in
| theory it should have some effect)" is harder to get published
| than "we treated X with Z in vitro and it killed all the cancer
| cells (and noncancer ones too, whoops)".
| briantakita wrote:
| A major investor & leader in healthcare that shall remain unnamed
| had a book "How to Lie with Statistics" in a public reading list.
| It's a good book & a quick read. Highly recommended.
|
| It's interesting that it takes an editorial to make people
| suspicious of statistics, how statistics can be abused, & the
| conflicts of interests that many people who utilize statistics
| have. Sample bias needs to be treated as deliberate dishonesty
| rather than a simple mistake. These people who make these
| mistakes are professionals and should know better. Their code of
| conduct should penalize them harshly for making these sort of
| mistakes.
|
| A strict code of conduct with harsh professional penalties are
| necessary to remove bad actors who hide behind subtle lies that
| have a major impact on public policy & public opinion. A slap on
| the wrist means it's always worthwhile to lie with statistics. A
| removal of license & banishment from the profession on the 1st or
| 2nd offense would quickly remove the bad actors. This code of
| conduct should also extend to the peer review process. If the
| peers pass bad statistics, the peers need to be held accountable
| as well.
___________________________________________________________________
(page generated 2021-07-19 23:00 UTC)