[HN Gopher] Medicine is plagued by untrustworthy clinical trials
___________________________________________________________________
Medicine is plagued by untrustworthy clinical trials
Author : headalgorithm
Score : 278 points
Date : 2023-07-18 11:27 UTC (11 hours ago)
(HTM) web link (www.nature.com)
(TXT) w3m dump (www.nature.com)
| Eumenes wrote:
| The over prescription of statins is a great example of what
| pharma sponsored clinical trials result in
| epistasis wrote:
| For your comment to make sense to me, you are saying that the
| pharma sponsored clinical trials are untrustworthy data?
|
| I've not heard that claim but am interested. Overprescripton
| certainly doesn't require that the trials were bad in any way.
|
| And I'm also curious about this idea of overprescription,
| because I hear it sometimes from extremely political people but
| have never heard it from scientists (and scientists are always
| trying to find some way to critique current practice, so that
| statins don't rise to that level is a surprise to me).
| remote_phone wrote:
| This. Statins claimed to be free from adverse reactions, but it
| turns out that about 30% of the participants were taken out of
| their clinical study because of "non-compliance". However, if
| you dig in further, the non-compliance was because of adverse
| reactions.
|
| You can't trust pharma companies if their data is secret.
| nradov wrote:
| That is medical misinformation. No drug company ever claimed
| that their statins were free from adverse effects. There are
| many statins on the market now, and patients who experience
| bad side effects from one will often do well on another.
| Getting the treatment right is a trial and error process.
|
| https://peterattiamd.com/why-a-recent-study-hasnt-shaken-
| my-...
| anonuser123456 wrote:
| If you dig further into those adverse reactions, you'll find
| they are approximately equal to adverse reactions of placebo.
| Madmallard wrote:
| Except for the part where there is a clear mechanism and
| cause related to muscle damage and cellular dysfunction.
| TheBigSalad wrote:
| There must be more to it. Everyone who drops out is
| scrutinized.
| obblekk wrote:
| There should be Nutrition Facts but for scientific trials.
| Independent agency just publishing quality assessments of the
| trial.
|
| This should be an async non blocking evaluation. The
| statisticians who do it should be anonymous by default. There
| should be an appeals process for a scientist to explain why an
| unconventional new method is actually robust.
|
| There should not be a single number published by this process,
| but rather a list of stats that speak to the overall quality of
| the trial on many dimensions (power, sources of bias, etc).
|
| Only information that would not be the same on 99% of trials
| should be written on this label (no sec style everything is a
| risk word vomit disclosures).
|
| There should not be a pre-emptive application for a label - it
| can only be gotten after paper submission to reduce gaming.
|
| There should be an independent advisory org that scientists can
| literally call to ask for advice on structuring the trials. These
| calls must not be disclosed. Much like farmers can call the
| government to ask for help on xyz crop problem.
|
| And these labels should never be used as the primary source of
| punishment. Any and all sanctions/penalties/dismissals must go
| through a new review process done by a different group.
|
| Any scientist who gets a label in a particular year should be
| given a vote to review the review agency on several dimensions.
| These aggregate reviews should be published broadly but not
| trigger any automatic consequences.
|
| Clear, accessible information is the basis for any self
| regulating human system. We need more of it in this field.
| mike_hearn wrote:
| There's the Cochrane Collaboration. They don't tick off every
| item on your list but it's fairly close to what you're asking
| for. It's mentioned in the article as they do a lot of meta-
| studies. Unfortunately they only started trying to spot
| fraudulent RCTs in 2021. Also in recent times some people don't
| like them, because they did a big review of mask studies and
| found there was no reliable evidence that masks worked against
| COVID.
|
| _Cochrane (formerly known as the Cochrane Collaboration) is a
| British international charitable organisation formed to
| organise medical research findings to facilitate evidence-based
| choices about health interventions involving health
| professionals, patients and policy makers.[4][5] It includes 53
| review groups that are based at research institutions
| worldwide. Cochrane has approximately 30,000 volunteer experts
| from around the world.[6]
|
| The group conducts systematic reviews of health-care
| interventions and diagnostic tests and publishes them in the
| Cochrane Library.[7][4]_
|
| https://en.wikipedia.org/wiki/Cochrane_(organisation)
| camelite wrote:
| "Many commentators have claimed that a recently-updated
| Cochrane Review shows that 'masks don't work', which is an
| inaccurate and misleading interpretation."
|
| https://www.cochrane.org/news/statement-physical-
| interventio...
| mike_hearn wrote:
| Next sentence: _" It would be accurate to say that the
| review examined whether interventions to promote mask
| wearing help to slow the spread of respiratory viruses, and
| that the results were inconclusive."_
|
| ... which is what I just said: some people got mad at them
| because their review found no reliable evidence that
| masking worked (or rather, that mask mandates worked, but
| these are virtually the same thing).
|
| The null hypothesis for any medical intervention is that it
| has no effect. You start from that and then try to prove
| your hypothesis that it does have an effect, which is what
| medical studies are for. If you can't prove something works
| then we fall back to the null and assume it doesn't. So
| that isn't a misleading or inaccurate interpretation of the
| results, though it would certainly have been politically
| convenient for the Cochrane organization if their reviewers
| could have supported the claims of public health
| authorities.
| chowells wrote:
| That sentence doesn't say what you think it says. It says
| "interventions to promote mask wearing". That's not mask
| wearing, it's telling people to wear masks. It is both
| true that wearing masks helps and that it's hard to tell
| if promoting mask-wearing changed enough behavior to
| matter. Mostly, those interventions do nothing.
| krona wrote:
| That's an ambiguous sentence. The main results of the
| study conclude:
|
| _Wearing masks in the community probably makes little or
| no difference to the outcome of influenza-like illness
| (ILI) /COVID-19 like illness compared to not wearing
| masks (risk ratio (RR) 0.95, 95% confidence interval (CI)
| 0.84 to 1.09; 9 trials, 276,917 participants; moderate-
| certainty evidence._
|
| Which I think is definitive.
| renaudg wrote:
| _The original Plain Language Summary for this review
| stated that 'We are uncertain whether wearing masks or
| N95/P2 respirators helps to slow the spread of
| respiratory viruses based on the studies we assessed.'
| This wording was open to misinterpretation, for which we
| apologize._
| fzeroracer wrote:
| > ... which is what I just said: some people got mad at
| them because their review found no reliable evidence that
| masking worked (or rather, that mask mandates worked, but
| these are virtually the same thing).
|
| This is not virtually the same thing. Comparing those two
| is wildly disingenuous and you know it.
| renaudg wrote:
| > no reliable evidence that masking worked(or rather,
| that mask mandates worked, but these are virtually the
| same thing).
|
| No it's not the same thing, and that's the key point. If
| you tell people that masking doesn't work (which is
| false) then of course mask mandates won't work because
| adherence will be low. A self-fulfilling prophecy really.
| mike_hearn wrote:
| Compliance for COVID mask mandates was measured and found
| to be extremely high, especially at the start (>95%).
| These mandates were enforced by harsh penalties so high
| compliance levels is no surprise. Thus you can't argue
| mask mandates didn't work because of low compliance.
|
| Also health authorities told people masks were highly
| effective. That's what justified the mandates. So you
| can't argue mask mandates didn't work because people were
| told it wouldn't work.
|
| Therefore there's no self fulfilling prophecy here. It
| didn't even matter what individuals thought anyway, we
| all had to wear masks.
|
| Although Cochrane much prefers to use RCTs, people have
| run regressions over the data and there was no link
| between levels of mask wearing and infection rates. It
| sucks but it appears that masks just can't stop
| aerosolized virus, which spreads like a gas. They aren't
| designed to do that so it's no knock against the
| manufacturers, who in some cases explicitly warned people
| that their products would be useless for that purpose (ht
| tps://pbs.twimg.com/media/EfNmzptXkAEg9Od?format=jpg&name
| =...).
| [deleted]
| krona wrote:
| Yes because a null hypothesis cannot be proven. Basic
| science.
| dekhn wrote:
| Nothing can be proven in science- only in math.
| renaudg wrote:
| > they did a big review of mask studies and found there was
| no reliable evidence that masks worked against COVID.
|
| No, that was a misinterpretation of the review in the Covid-
| skeptic sphere. Cochrane have had to issue a statement to
| clarify : https://www.cochrane.org/news/statement-physical-
| interventio...
|
| tl;dr : half of the people given masks in these studies
| didn't wear them consistently or at all, dragging efficacy
| results down.
| Izkata wrote:
| That link agrees with what GP actually said: "no reliable
| evidence".
| BoringSalad637 wrote:
| Also, while _this study_ was inconclusive as to whether
| masks help prevent covid, it doesn 't mean that _all
| studies_ are inconclusive.
|
| For example,
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8768005/ is "a
| detailed performance evaluation of the mask is studied from
| an engineering point of view," which aim to look at how the
| _physics_ of N95 masks hold up against covid. What the
| _physics_ shows is that N95 filtration helps block covid
| particles.
| readthenotes1 wrote:
| I'm mass transit where masks will required, most of the
| people wore the masks covering their chin. I believe the
| research is accurate and that saying that masking up more
| people doesn't work because they won't wear them properly
| db48x wrote:
| Which is actually good proof that _requiring_ people to
| wear masks doesn't help. Mask mandates are pointless even
| when masks are useful.
| callalex wrote:
| By that logic why make any laws? Why make murder illegal
| if some people are going to kill anyways?
| readthenotes1 wrote:
| That is why we have police officers to try to stop the
| people who have proven they are willing to murder.
|
| Do you want to be the person going around policing mask
| wearing?
| lesuorac wrote:
| Well depends on who you ask.
|
| I generally think laws should be codifications of
| societal norms. Which also implies that as societal norms
| change so should laws.
|
| So even things such as murder which people do and we
| don't want should be codified as illegal. But even if
| nobody committed murder anymore it should still be
| illegal as its against societal norms.
| s1artibartfast wrote:
| Three points.
|
| First, some laws probably don't have any positive impact.
|
| Second, there's a difference between accurately
| summarizing trial results and extrapolating that to the
| impact of a new law. If there was a death penalty for not
| wearing masks, perhaps compliance would be better than in
| the trials and an effect would be shown. This doesn't
| mean that the trial analysis is wrong, you just can't
| draw a conclusion about the law from the trial data.
|
| Third, laws have multiple purposes including Justice and
| Punishment. Some murderers might have zero chance of re-
| offending but we still want to punish them as a matter of
| Justice, not because it makes Society safer.
| concordDance wrote:
| Presumably less people kill in that case.
| t0bia_s wrote:
| Would you kill of that would be legal?
| penultimatename wrote:
| [flagged]
| timr wrote:
| > that was a misinterpretation of the review in the Covid-
| skeptic sphere.
|
| No, it wasn't. You should read the paper itself, instead of
| relying on (sadly) biased editorials about the paper. It
| _literally says_ what the OP wrote:
|
| > Wearing masks in the community probably makes little or
| no difference to the outcome of influenza-like illness
| (ILI)/COVID-19 like illness compared to not wearing masks
| (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to
| 1.09; 9 trials, 276,917 participants; moderate-certainty
| evidence. Wearing masks in the community probably makes
| little or no difference to the outcome of laboratory-
| confirmed influenza/SARS-CoV-2 compared to not wearing
| masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919
| participants; moderate-certainty evidence)
|
| The only place they found any plausible signal was
| comparing N95 respirators against surgical masks, but the
| evidence was extremely weak:
|
| > We pooled trials comparing N95/P2 respirators with
| medical/surgical masks (four in healthcare settings and one
| in a household setting). We are very uncertain on the
| effects of N95/P2 respirators compared with
| medical/surgical masks on the outcome of clinical
| respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3
| trials, 7779 participants; very low-certainty evidence).
| N95/P2 respirators compared with medical/surgical masks may
| be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5
| trials, 8407 participants; low-certainty evidence).
| Evidence is limited by imprecision and heterogeneity for
| these subjective outcomes. The use of a N95/P2 respirators
| compared to medical/surgical masks probably makes little or
| no difference for the objective and more precise outcome of
| laboratory-confirmed influenza infection (RR 1.10, 95% CI
| 0.90 to 1.34; 5 trials, 8407 participants; moderate-
| certainty evidence).
|
| The editorial you cited was a low point in the history of
| Cochrane, where they gave in to public outrage and
| attempted to cast doubt on their own data.
|
| https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.C
| D...
| classichasclass wrote:
| From the editorial: "It would be accurate to say that the
| review examined whether interventions to promote mask
| wearing help to slow the spread of respiratory viruses,
| and that the results were inconclusive. Given the
| limitations in the primary evidence, the review is not
| able to address the question of whether mask-wearing
| itself reduces people's risk of contracting or spreading
| respiratory viruses."
|
| Whether you think the editorial was them caving or not,
| they also issued it under their own name with the same
| weight as their other reviews, so they must have thought
| enough of it to do so.
|
| Given that there's ample laboratory evidence of the
| filtering capacity of a good N95 or even a KN95 mask, and
| having worked with an N95 respirator in tuberculosis
| control settings for 17 years and never converted my TB
| test, I think I'll stick with the mask in future and I
| have no hesitation recommending winter masking to others
| who believe they are at risk of complications.
|
| I've liked not being sick for the last three years.
| mike_hearn wrote:
| The review found very few studies into the effectiveness
| of N95/respirators against ILIs, and from those studies
| they concluded "wearing N95/P2 respirators probably makes
| little to no difference".
|
| Bear in mind a possible source of confusion here: TB
| bacterium are ~3 microns in size, but viruses are about
| 0.2 microns. The Cochrane review I mentioned is only
| about respiratory viruses. So it's possible that they may
| work against TB but not against flu or COVID.
| classichasclass wrote:
| I'm pretty aware of how large a TB bacillus is, thanks.
|
| The NIOSH definition for an N95 is a device able to
| filter at least 95% of airborne particles that have a
| mass median aerodynamic diameter of 0.3 micrometers.
| While SARS-CoV-2 is around 0.1 microns in size, naked
| COVID-19 viruses in air are rare as they would be torn up
| nearly immediately, so they are almost always within
| aerosols. Typical respiratory aerosol range is around
| half a micron or so [0], and as the aerosol particle size
| gets smaller, so necessarily must be the amount of virus
| that is present.
|
| Is this perfect filtration? No, but no one gets sick from
| a single virus they inhaled either, even with as
| communicable as the current Omicron variants are. There's
| a minimum infective dose and they help keep exposure
| under it.
|
| [0] https://www.nature.com/articles/s43856-022-00103-w
| vibrio wrote:
| The size of the single virus is a false metric here.
| There is a wide range of respiratory droplets containing
| virions. Those droplets can range from visible (way
| bigger than a mycobacterium ) to only large enough to
| hold one virion. The size distribution of those particles
| is the metric.
| timr wrote:
| > they also issued it under their own name with the same
| weight as their other reviews, so they must have thought
| enough of it to do so.
|
| Data is data. Editorials are editorials. The fact that
| they're published on the same website doesn't change the
| data. If the Higgs boson was published in the same issue
| of Physics Letters B as another letter that claimed
| uncertainty of the result, would you treat them with
| equal weight?
|
| > and having worked with an N95 respirator in
| tuberculosis control settings for 17 years and never
| converted my TB test
|
| I mean...that's fine? Nobody is telling you what to
| believe or do. Most of what we do comes without evidence.
| But let's be _slightly_ rigorous thinkers for a moment:
| there 's a fairly obvious difference between a fit-tested
| n95 mask in a laboratory setting, where there are _lots
| of other interventions happening at the same time_
| (negative pressure labs, hoods, etc.), and putting on a
| loose surgical mask on a bus. We should be able to talk
| about that rationally, and not resort to superstition.
|
| > I've liked not being sick for the last three years.
|
| I haven't worn masks and I haven't gotten sick either.
| Other than Covid -- which I got when we were all wearing
| masks.
|
| "post hoc, ergo propter hoc."
| classichasclass wrote:
| That's drawing an unnecessarily sharp description. To a
| first approximation all Cochrane pieces are editorials.
| They're interpreting what's actually out there.
|
| > But let's be slightly rigorous thinkers for a moment:
| there's a fairly obvious difference between a fit-tested
| n95 mask in a laboratory setting, where there are lots of
| other interventions happening at the same time (negative
| pressure labs, hoods, etc.), and putting on a loose
| surgical mask on a bus. We should be able to talk about
| that rationally, and not resort to superstition.
|
| No one's resorting to superstition. You're the one saying
| there's no value in an intervention that has empiric
| laboratory evidence to support it. The argument here is
| what matters at the population level. If the problem is
| performance, then we train people to select and use masks
| better, not simply say that there's no point to it at
| all.
| PathOfEclipse wrote:
| Seriously, what's the difference between what OP wrote:
|
| " [Cochrane] found there was no reliable evidence that
| masks worked against COVID/"
|
| And the editorial: "the review examined whether
| interventions to promote mask wearing help to slow the
| spread of respiratory viruses, and that the results were
| inconclusive"
|
| How is "inconclusive" functionally different from "there
| was no reliable evidence?" Seriously, how do you justify
| this pedantry while ignoring and obfuscating the truth?
|
| People do much evil by focusing on the wrong facts, the
| wrong stories, and the wrong lessons learned, while
| ignoring the right ones. That you are willing to focus on
| apparently frivolous pedantry while ignoring the fact
| that so many were forced to use masks without any high-
| quality scientific evidence that they actually did
| anything, including children, and all the lessons that
| should derive from this, is in my opinion, very
| representative of this type of evil.
| CorrectHorseBat wrote:
| It's not "inconclusive" and "there was no reliable
| evidence" that are different, it's the promoting part
| that makes them completely different.
|
| "We found no reliable evidence that abstinence prevents
| teen pregnancy"
|
| "We examined whether promoting abstinence prevents teen
| pregnancy and the results were inconclusive"
|
| The first is obviously wrong, and if the the second is
| true it would mean the government should look for other
| ways to prevent teen pregnancy, but it wouldn't mean that
| practicing abstinence as an individual doesn't work to
| prevent pregnancy.
| NoPie wrote:
| Cochrane review doesn't make this distinction.
|
| In medicine you cannot distinguish. It is all about the
| intervention and not about some theoretical best-case
| scenario.
|
| The intervention is to ask people to wear masks. People
| comply as they do in real real life and then we measure
| the results. There was no reliable evidence that this
| made any noticeable difference.
|
| Now you can change the intervention - instead of asking
| and mandating masks as we did, we could educate masks
| wearers more. Unfortunately we have no evidence that it
| helps.
|
| Perhaps masking could help to an individual wearer? Alas,
| we didn't collect such evidence either.
|
| Some studies are lab based. In those masks had some
| effect. But that's not how people use masks in real life,
| so these results don't mean much.
| autoexec wrote:
| > But that's not how people use masks in real life, so
| they don't mean much.
|
| I think saying "Using X is effective, but only if you
| actually use X" is obvious. The thing people want to know
| is "do masks stop the virus" which is an entirely
| different question from "How many people will wear
| masks", which is a different question from "What is the
| effectiveness of interventions to promote mask wearing"
| NoPie wrote:
| The first question is pointless for someone responsible
| for public health. People want the answer to it because
| they don't want to think about all these related issues
| and have simplistic idea that they can protect
| themselves. But chances are their compliance is exactly
| the same as among people in those studies.
|
| Therefore the real question is how effective is the
| intervention. It will be (or should be) asked by people
| responsible with public health policies.
|
| P.S. Cochrane group is not for giving scientific answers
| to individual people. Its main aim is to evaluate the
| evidence of different treatments and provide guidance to
| policy makers and healthcare authorities.
| autoexec wrote:
| If you are responsible for public health and the answer
| to the first question is "no" then you have no need to
| ask the other two. Figuring out what we can do to get
| people to do what works is important too, but it's not
| the only thing that matters. People can be educated and
| their habits changed.
|
| We have similar problems getting schizophrenics to take
| their meds and getting communities with high rates of
| open defecation to use toilets, but nobody suggests that
| we give up on antipsychotics or sanitation facilities.
| NoPie wrote:
| The first answer is too vague to have a meaningful answer
| in case.
|
| Every other treatment in medicine including schizophrenia
| is tested how it works in practice. It is incurable
| disease and the treatments have many side-effects. Thus
| the question becomes not "does this medicine cure
| schizophrenia" but "does this treatment works better than
| placebo or another treatment?". When studies are
| completed, we gather evidence by monitoring real life
| experience with this treatment.
| autoexec wrote:
| > Every other treatment in medicine including
| schizophrenia is tested how it works in practice.
|
| Medicine is tested according to how it works when people
| actually take it. People participating in research
| studies who fail to take their medications (or their
| placebo for that matter) are kicked from the program and
| their data is typically discarded entirely.
| NoPie wrote:
| That is generally not true.
|
| In fact, often clinical trials are statistically analysed
| by intention-to-treat, including all people who have been
| randomised even if they later don't receive the
| treatment.
|
| Per-protocol-analysis (including only people who follow
| the study protocol) can also be used but it is more prone
| to bias.
|
| Besides, with masks it is not simply wearing or not
| wearing a mask. Even a very diligent mask wearers may
| wear it in a way that makes it less effective without
| being aware of that.
|
| In short, when the doctor prescribes a medicine it is
| important to understand the factors why the patient may
| not take the medicine as prescribed. If the real life
| situation is that most people take medicine in a way that
| makes it ineffective and so much that the clinical trial
| cannot find significant effect, then he shouldn't
| prescribe it. It is just a waste of resources and giving
| people false hopes.
| timr wrote:
| > > "We found no reliable evidence that abstinence
| prevents teen pregnancy"
|
| > > "We examined whether promoting abstinence prevents
| teen pregnancy and the results were inconclusive"
|
| > The first is obviously wrong,
|
| No. They're equivalent. They both mean "we looked, and we
| didn't find any confirming evidence." You're confusing
| "we found no reliable evidence of X" with "we found
| evidence of NOT X", which is different, and essentially
| _never_ achievable in empirical studies (note: this is
| not an invitation to get side-tracked in pedantic debates
| about proving the null; I 'm telling you how actual
| randomized controlled trials work, in real life.)
|
| Proving a negative via statistics is ~impossible, so what
| you do instead is to look for significant differences in
| X, attributable solely or partially to the intervention.
| If you _don 't_ find such a difference (as was the case
| in the mask review), you say "we found no reliable
| evidence of X".
|
| But when the Cochrane authors wrote _" Wearing masks in
| the community probably makes little or no difference to
| the outcome of influenza-like illness"_, they really did
| mean exactly what it sounds like -- the effect size in an
| aggregated pool of randomized controlled trials was
| _statistically indistinguishable from zero._ You can
| debate whether or not they looked for the right thing
| (X), you can debate whether or not adding another big
| randomized trial would help find X, and so on. But the
| plain-text interpretation is correct.
| sfn42 wrote:
| We may not be sure that masks help, but we're completely
| sure that they don't hurt so I don't see the problem
| personally.
| Mountain_Skies wrote:
| Absolutely false. There are lots of negatives to mask
| wearing, starting with inducing developmental problems in
| children and continuing on with massive increases in long
| lasting trash and then into more speculative issues with
| breathing. It's not a harmless activity.
| dllthomas wrote:
| I'm completely sure masks hurt my pocketbook and my
| ability to keep my car tidy, and that _forcing_ people to
| mask has additional costs. There are cost /benefit
| questions that aren't as trivial as you imply, and they
| should be made based on reliable data.
| hdior wrote:
| [dead]
| classichasclass wrote:
| "Many were forced" != "there's no value"
| ecuaflo wrote:
| I'm reading these as completely different.
|
| The latter sounds like advertising and education about
| masks rather than wearing the masks themselves. ie
| telling people to wear masks made no difference in spread
| probably because people's minds were already made up
| about masking.
|
| I din't see it making any conclusion about masking itself
| LorenPechtel wrote:
| Yup, we lead lives where it's simply not that big an
| issue to protect ourselves. While I think my chance of
| dying from getting it would be very low the issue of long
| term damage is another matter--it certainly looks to me
| like it damages everybody, just not always to the point
| they notice. The damage is probably cumulative.
| autoexec wrote:
| It also literally says "The high risk of bias in the
| trials, variation in outcome measurement, and relatively
| low adherence with the interventions during the studies
| hampers drawing firm conclusions."
| timr wrote:
| It does, and that's true, but that doesn't contradict
| what OP wrote.
|
| They found only mid-to-low quality evidence supporting
| the use of masks to prevent ILI. That evidence, for
| everything but the question of "n95 vs. other", showed an
| effect size _statistically indistinguishable from zero._
|
| You're essentially saying that the error bars on that
| effect size are big. They are. But they're still centered
| on zero.
| autoexec wrote:
| The evidence they had was of such low quality that no
| solid conclusions could be made from it. What they found
| in the research may not reflect reality. They are
| explicit about this and stress the need for better
| research.
|
| > "There is uncertainty about the effects of face masks.
| The low to moderate certainty of evidence means our
| confidence in the effect estimate is limited, and that
| _the true effect may be different from the observed
| estimate of the effect_...There is a need for large,
| well-designed RCTs addressing the effectiveness of many
| of these interventions in multiple settings and
| populations, as well as the impact of adherence on
| effectiveness, especially in those most at risk of ARIs.
| "
|
| They admit that they were unclear about it and later were
| even more explicit.
|
| "Given the limitations in the primary evidence, the
| review is not able to address the question of whether
| mask-wearing itself reduces people's risk of contracting
| or spreading respiratory viruses."
|
| The review is not able to "address the question" let
| alone conclude _anything_ about the impact of mask
| wearing. The review is inconclusive.
| LorenPechtel wrote:
| The basic problem is whether the data says masks don't
| work, or says that people aren't consistent enough in
| wearing masks.
|
| I've seen it directly--one woman putting on a mask when I
| approached. The thing is she had been hiking near the
| back of the pack in a group that got together for the
| hike. She was at a far higher risk from being downwind of
| her group (this was not a family bubble) than of me being
| off to the side.
|
| I can basically guarantee nobody there was experiencing
| any appreciable symptoms (10,000' up, miles from the cars
| --not something you're doing with any sort of respiratory
| infection) but most Covid spread is presymptomatic.
|
| A solo hiker masking when someone approaches makes sense
| (and is what I did pre-vaccine), but not masking with
| your group but masking for a stranger? That's merely an
| illusion of safety and why masks "don't work".
|
| There's also the problem that the Cochrane data included
| mostly studies of things other than Covid--when you go
| over their own data only looking at Covid you do see some
| benefit. Note, also, the pooling of masks and respirators
| --we already know masks do little against the Omicron
| variants. Respirators or don't bother.
|
| Cochrane messed up badly in this case by looking at the
| wrong thing. I'm reminded of the BMJ study showing zero
| safety benefit from parachutes when jumping from an
| airplane.
| peyton wrote:
| I would like to point out what "makes sense" to people
| rarely reflects the underlying fluid dynamics at the
| relevant scales. Couple that with a poor understanding of
| just how many particles one infected person emits and
| it's clear masks as worn are very ineffective for the
| vast majority of people.
| anonymous344 wrote:
| Yes, the masks didn't work. Now everybody should know it.
| First of all, they were using paper mouth shields or
| adidas branded useless cloths, not masks. But even the
| dumb fcks using real n95 mask, i see people everywhere
| touching the mask from outside (where the viruses should
| be stopped if the mask works) and then touching
| everything else. And when coughing opening the mask and
| coughing inside the palm...
| chaxor wrote:
| This is important to point out.
|
| I was actually surprised by the mouthwash outcomes as well.
| Almost no one really talked about mouthwash, but it looked
| to be useful in the study.
| bena wrote:
| Mouthwash is typically alcohol based. Alcohol is a pretty
| good disinfectant in general.
|
| But it's efficacy will really only be decent while it's
| in your mouth. Once it gets diluted past a certain point,
| it's not going to be doing anything. You'd probably have
| similar results with vodka.
| autoexec wrote:
| I know doctors who recommended drinking whiskey early in
| the pandemic for that reason (and also the usual reasons
| people dealing with trauma reach for whiskey)
| specialist wrote:
| > _Also in recent times some people don 't like them, because
| they did a big review of mask studies and found there was no
| reliable evidence that masks worked against COVID._
|
| Oh.
|
| I quickly found this:
|
| "The new scientific review on masks and Covid isn't what you
| think" Kelsey Piper
|
| https://www.vox.com/future-
| perfect/2023/2/22/23609499/masks-...
|
| Based on the criticisms, I expect Cochrane will revisit this
| topic.
|
| Progress isn't a straight line.
| epistasis wrote:
| If science is going to be "self-correcting" then it has to
| make mistakes in the first place.
|
| These mistakes will happen from the original scientists,
| they will happen at the stage of editorial boards, they
| will happen at peer review, they will happen if external
| third parties start systematically reviewing every RCT.
|
| So Cochran must similarly be scrutinized for their errors,
| because they will be making them as well.
|
| And that's even before we get to the political factors
| outside of science misinterpreting complex data for their
| own purposes...
| specialist wrote:
| Yes and:
|
| It's wicked hard just to get reproducible results (one
| facet of the replication crisis). Much less the
| challenges you list.
|
| Confusion and miscommunication is the norm. Rising above
| that takes Real Effort(tm).
|
| One of my formative experiences was on a team trying to
| adopt the processes from the book Applying Use Cases. So
| simple. Like a recipe. Really, what could be more simple?
|
| We had shared purpose. We all read the book (among
| others). We discussed. We all thought we were good to go.
|
| And then the wheels fell off once real work started.
| Turns out we didn't agree. On anything. What is "the
| system"? What level of abstraction are we working at?
| What does this line (points at diagram) here mean?
|
| Writing this now, experiencing PTSD flashbacks, I can
| confidently say I would have never succeeded as a
| scientist.
| b59831 wrote:
| Vox is an awful source.
| tapland wrote:
| In this case it's the source of nothing more than an
| explanation of the study which we are already discussing.
| sfn42 wrote:
| "An explanation" can be wildly misleading.
|
| For example i might "explain" to you that clean code is
| about writing the least amount of code possible and you
| might start code golfing your production systems.
|
| If you want to know what the paper says, read the paper.
| Journalists are not scientists, most of them do not have
| the necessary knowledge to understand academic papers,
| nor do they have an incentive for doing it well. They do
| have an incentive for generating clicks though, generally
| by twisting the truth to make things sound more
| interesting or provoking than they are.
| mike_hearn wrote:
| Cochrane revisit topics from time to time to update their
| reviews as new studies appear. The question of mask
| effectiveness was reviewed in the past also. There's an
| interview with one of the authors of this round's review
| here:
|
| https://dailysceptic.org/2023/02/06/dr-carl-heneghan-
| intervi...
|
| _So, a Cochrane review is a study which synthesises all
| available studies - all that we can find or identity - on a
| particular topic. It follows a highly structured format and
| is always preceded by publication of a protocol. All this
| is to minimise the bias. Also, it is extensively
| transparent. In this case we are looking at about 300 pages
| of review. Now, the review called "Physical interventions
| to interrupt or reduce the spread of respiratory viruses"
| is called in code A122 for short and I will be using that
| acronym simply because it is just too long a title. So the
| protocol was first published in 2006 and then the first
| version was published in 2007, updated in 2009, 2010, 2011,
| and then 2020, so this 2023 is the fifth update of this
| review. And the reason why we update the reviews is they
| are soon out of date if we don't do that, especially in
| some fast moving topics._
|
| This update didn't change the conclusions from any of the
| prior reviews.
|
| Because masks are so politicized there were numerous
| attacks on Cochrane this time around, though nobody cared
| in any of the previous rounds. The Cochrane authors are
| aware of all the criticisms, but there were no
| justifications found in any of them to alter the
| conclusions of the review or their procedures for doing
| them.
| specialist wrote:
| > _This update didn 't change the conclusions from any of
| the prior reviews. ... The Cochrane authors are aware of
| all the criticisms, but there were no justifications
| found in any of them to alter the conclusions of the
| review or their procedures for doing them._
|
| True. But as noted elsethread, Cochrane is not
| responsible for others misinterpreting the conclusions.
|
| "Statement on 'Physical interventions to interrupt or
| reduce the spread of respiratory viruses' review"
| https://www.cochrane.org/news/statement-physical-
| interventio...
|
| _" The original Plain Language Summary for this review
| stated that 'We are uncertain whether wearing masks or
| N95/P2 respirators helps to slow the spread of
| respiratory viruses based on the studies we assessed.'
| This wording was open to misinterpretation, for which we
| apologize. While scientific evidence is never immune to
| misinterpretation, we take responsibility for not making
| the wording clearer from the outset. We are engaging with
| the review authors with the aim of updating the Plain
| Language Summary and abstract to make clear that the
| review looked at whether interventions to promote mask
| wearing help to slow the spread of respiratory viruses."_
|
| > _masks are so politicized_
|
| Indeed.
| claytongulick wrote:
| Vox? I would recommend seeking elsewhere for truth.
|
| For example, after parsing through the ad hominem attacks
| and nonsense in that article, their main point is that the
| Bangladesh study found masks to be effective.
|
| Except that study is junk and all the reported effects were
| found to be a result of researcher bias [1].
|
| Vox also misrepresents the Danish study, which is probably
| the best study to date we have on masking effectiveness.
|
| > Progress isn't a straight line.
|
| Yes, but truth is most likely to be found in whatever facts
| are orthogonal to vox' narrative.
|
| [1] https://trialsjournal.biomedcentral.com/articles/10.118
| 6/s13...
| specialist wrote:
| > _parsing through the ad hominem attacks_
|
| https://en.wikipedia.org/wiki/Ad_hominem
|
| > "Re-analysis on the statistical sampling biases of a
| mask promotion trial in Bangladesh: a statistical
| replication"
|
| I'm not remotely qualified to have an opinion.
|
| That said...
|
| The open (public) process as well as the critics sharing
| their source code is just awesome.
|
| https://trialsjournal.biomedcentral.com/articles/10.1186/
| s13...
|
| https://github.com/mchikina/maskRCTnote
|
| I share the reviewer's hope that authors of the original
| study will respond.
| [deleted]
| jonlucc wrote:
| Why would the statisticians be anonymous? I'm aware of at least
| a couple cases in which an independent set of statisticians
| were provided the data from a clinical trial specifically for a
| re-analysis. In one case, they showed some pretty concerning
| inconsistencies and the other confirmed no effect on the
| primary analysis, but suggested some sub-populations that might
| have shown an effect if a future study was properly powered.
| That follow-up clinical trial was just published showing pretty
| remarkable effect in the sub-population. I don't think there's
| reason to believe either independent analysis was anything
| other than independent.
|
| There is already a mechanism for companies to submit questions
| to the FDA prior to clinical trial initiation. I'm not in these
| conversations, but I know the type of questions can be things
| like: would you accept this endpoint as a proxy for this
| indication, would you be satisfied with the effect size we
| expect, and are there other safety concerns you would expect us
| to evaluate other than those in our current plan. I assume EMA
| and other regulatory bodies have a similar process, but I'm not
| positive.
|
| Disclaimer: I work in pharma, but pre-clinically. I am not
| involved in these clinical or regulatory issues.
| obblekk wrote:
| So junior scientists can be hired without them being
| concerned for future career prospects.
| alphazard wrote:
| The success of the Nutrition Facts labeling does not get enough
| publicity.
|
| Rather than outlawing certain ingredients, or creating some
| kind of health score which a product must be above, Nutrition
| Facts is a way for suppliers to attest to information about a
| product in way that is legally binding. If it isn't accurate
| the penalties are steep.
|
| Consumers then have the information they need to vote with
| their wallets. Markets cannot function properly without
| symmetric information, and Nutrition Facts essentially creates
| a functioning market where one did not exist previously.
|
| Any effort to regulate what's in food would probably be better
| spent expanding what must be in the Nutrition Facts label. I
| guess it's nice that we are finally getting around to banning
| artificial trans fats, but anyone who can read has been able to
| keep those out of their diet for years. The same can be said
| about the next bad ingredient, and the one after that.
| taeric wrote:
| This isn't without downsides, of course. The case of
| manufacturers adding allergens to food deliberately is
| alarming in its own way.
| hermitdev wrote:
| Surely, you aren't complaining of peanuts in peanut butter,
| so can you share an example, and why the allergen
| presumably shouldn't be there?
| hguant wrote:
| The penalty for having an allergen present is steep, and
| the process of certifying that yes, you are in fact
| allergen free, is expensive and difficult, while the cost
| of adding an allergen into your process, for pretty much
| any foodstuff, is cheap, and the cost of slapping a "may
| contain peanuts" label on is cheaper.
|
| I believe the original comment was complaining about the
| perverse incentive there.
| Ekaros wrote:
| And if may contain is not enough or allowed, may as well
| throw some amount of peanuts in and list it. And cover
| all the bases.
| LorenPechtel wrote:
| No, we are complaining about the way the government
| handles allergen labeling.
|
| It used to be that companies could slap a "may contain
| [allergen]" label on things that didn't contain it but
| were produced in a factory where cross contamination was
| a possibility. Such labels are *widespread*.
|
| I don't understand the government's incentive in trying
| to stop this--the actual result was when cross
| contamination was a possibility the companies reacted by
| deliberately adding the offending material.
|
| The problem is that it's being looked at in a binary
| sense. Either it contains the offending material and
| poses a danger to those affected, or it doesn't and is
| safe. However, in the real world there's a third
| population--those who are sensitive to the offending
| ingredient but not dangerously so. Possibility of cross
| contamination? That is not going to be a deterrent to me
| as the worst case outcome is merely unpleasant. Does
| contain? I'm going to treat it with great skepticism.
| taeric wrote:
| Apologies, I should have included a link. I am referring
| to https://apnews.com/article/sesame-allergies-
| label-b28f8eb3dc....
|
| Mayhap that is overblown? I confess I have not followed
| it too heavily. Very thankful that I am not allergic to
| anything in my adult life.
| hermitdev wrote:
| I don't have any food allergies, either, so I admittedly
| don't pay much attention to allergens (or even listed
| ingredients most of the time). I'm genuinely curious,
| too.
| swsieber wrote:
| I had the same questions and found this article:
| https://snacksafely.com/2016/06/kelloggs-unintended-
| conseque...
|
| Basically, there were stricter measures put in place
| called HARPC. From my linked article:
|
| > The new directives mandate that the "Top 8 allergens"
| identified by FALCPA (peanuts, tree nuts, milk, eggs,
| wheat, soy, fish, and crustacean shellfish) must either
| be ingredients of the product and identified as such, or
| the manufacturer must take extra care (and cost) to
| ensure that there is no cross-contact with them. There is
| no middle ground or "out" for the manufacturer, which is
| why we believe "May contain" type label advisories are
| heading for extinction. And that poses a problem, at
| least in the short term.
|
| > companies, when faced with the added burden of
| instituting and documenting cross-contact prevention
| measures as dictated by HARPC, may instead choose to add
| trace amounts of the allergen to the product, as doing so
| makes the allergen an ingredient of the product and
| obviates the need for preventative cross-contact measures
| for that allergen.
|
| > it means that manufacturers will either take stricter
| measures to prevent cross-contact or add a trace amount
| of the allergen and list it in the ingredient list, thus
| eliminating the ambiguity that currently plagues us all.
|
| > A compromise that might have avoided the unintended
| consequences of companies like Kellogg's adding traces of
| allergens to their products is to have offered them a
| third option: A mandatory "May contain" label advisory
| for any product made on shared equipment or in shared
| facilities that did not meet the FSMA threshold for
| cross-contact prevention. Such label advisories are
| voluntary today, rendering them ambiguous at best, but a
| definitively worded and located advisory statement
| included on all such products would have provided a way
| for manufacturers to meet the requirements of HARPC
| without resorting to the addition of allergens.
|
| So I'd be surprised if it was happening on an ongoing
| basis, but I can definitely see why people would be
| irked.
| kneebonian wrote:
| I'll add, a family member has celiac which makes it so they
| can't eat gluten. Becoming certified "Gluten Free" requires
| a certification process that can be expensive and
| difficult. However many companies have realized they can
| label their product "Gluten Friendly" and get around the
| requirements. It is annoying.
| vxNsr wrote:
| Interestingly, there is some gaming of the main number
| everyone looks at on the nutrition facts chart: calories per
| serving.
|
| All snacks aim to fall at or below a certain number the FDA
| (or some other agency) put out as being considered a snack.
| Planet Money did an episode on different M&M varieties having
| different total weights to account for their different
| calorie counts. So you get fewer by weight peanut butter M&Ms
| because they're more calorie rich
| taeric wrote:
| This doesn't sound bad to me? At large, people eat a
| package of whatever snack they choose to buy. Also at
| large, people assume different packages of snacks should be
| roughly comparable for important metrics. And calorie count
| is probably up there for important metrics.
| [deleted]
| csours wrote:
| Yes, I have said this as "The UX of [medical] study papers is
| terrible". Some people do not agree, they think that it should
| not be made easier to understand, that non-experts cannot
| really understand medical studies, so they should not be more
| approachable. I think that's dead wrong.
| p-e-w wrote:
| > There should be Nutrition Facts but for scientific trials.
|
| No, there should be prison time for scientists who conduct
| unethical trials or publish fake results.
|
| The public (and policy makers) place such immense trust in
| those people and what they publish that nothing less is even
| remotely adequate.
|
| When someone puts arsenic in food, they go to prison - labeling
| the food with "contains arsenic" doesn't cut it.
|
| Do this and watch science magically fix itself.
| ekianjo wrote:
| Revoke their license and titles to start with so they cannot
| operate anymore in the field
| Spinnaker_ wrote:
| I would like to see Universities take the lead here. A
| bunch of high profile degree revocations would generate
| some waves at least.
| tcmart14 wrote:
| As reasonable as that would be. I feel like it would just
| turn into people getting their degrees revoke claiming
| 'cancel culture' and becoming gurus with many mindless
| followers pushing loose weight quick schemes kind of
| thing. Because today if you face consequences, its no
| longer your fault, its everyone else trying to cancel you
| from doing the bad thing you are doing.
| twic wrote:
| Which prison? As the article says:
|
| > Ultimately, a lingering question is -- as with paper mills
| -- why so many suspect RCTs are being produced in the first
| place. Mol, from his experiences investigating the Egyptian
| studies, blames lack of oversight and superficial assessments
| that promote academics on the basis of their number of
| publications, as well as the lack of stringent checks from
| institutions and journals on bad practices.
|
| A substantial part of what's happening here is that first-
| world countries with generally good cultures of research
| integrity are basing medical policy on studies done in
| countries where the system encourages researchers to cheat.
| British and US authorities can't put Egyptian or Chinese
| researchers in prison, can they?
| p-e-w wrote:
| There's no shortage of scientific fraud happening in the
| "first world" also. Dealing with those people would be a
| good start.
| chaxor wrote:
| This is the correct answer. Typically if something is
| done at an ivy league, other US and UK universities
| follow. Perhaps the other countries would follow as well
| shortly after that, or there would just be a divide
| between 'real' research and 'not', similar to when many
| of my friends stop reading after the word 'Hindawi'.
| t0bia_s wrote:
| - Do this and watch science magically fix itself.
|
| Imagine same approach in politics. If some politics put lies
| in their speeches to manipulate with people, we should call
| them liers and put them in prison. But somehow it doesn't
| happen. Looks like society prefer conformity over
| responsibility.
| neaden wrote:
| Just to be clear, if you actually did this what you would see
| is stuff like Florida locking up every climatologist for
| doing "false science".
| psychlops wrote:
| It wouldn't be limited to Florida. Elsewhere, having any
| reasonable questions about the severity regarding the
| religion of Climate Change would get one jailed for
| blasphemy.
| LorenPechtel wrote:
| They would have to prove it to a jury.
|
| I do agree it would be a major deterrent to doing such
| research, but as it stands you'll get fired for it anyway
| which is a pretty major deterrent.
| s1artibartfast wrote:
| Im struggling to see the difference between this and the
| current FDA process, and think it is 90% the same.
|
| Drugs have "prescribing information", referred to in industry
| as "labeling", which follows a consistent format containing
| safety, trial results, side effects, and mechanism of
| action.[1] I recommend people read them for drugs they take.
|
| _This should be an async non blocking evaluation. The
| statisticians who do it should be anonymous by default. There
| should be an appeals process for a scientist to explain why an
| unconventional new method is actually robust._
|
| Third party analysis is the main difference here. In the
| current state, firms run the analysis for FDA review using
| standard practices, and must explain and get approval for any
| unconventional methods
|
| > _There should not be a single number published by this
| process, but rather a list of stats that speak to the overall
| quality of the trial on many dimensions (power, sources of
| bias, etc)._
|
| Labeling contains many relevant numbers. Trial sizes, how many
| per arm, what was measured, and and final results. Maybe there
| could be some squishy qualitative summary, but that seems more
| risky. I would rather know that 1 out of 20 patients died than
| it got a "2" on the safety scale.
|
| >Only information that would not be the same on 99% of trials
| should be written on this label (no sec style everything is a
| risk word vomit disclosures).
|
| Labeling contains drug specific information.
|
| _There should not be a pre-emptive application for a label -
| it can only be gotten after paper submission to reduce gaming._
|
| Drug labeling requires pre-application and and a standard 12
| month review period by the FDA prior approval
|
| _There should be an independent advisory org that scientists
| can literally call to ask for advice on structuring the trials.
| These calls must not be disclosed. Much like farmers can call
| the government to ask for help on xyz crop problem._
|
| The FDA provides advice on structuring trials and acceptable
| design, size, power, endpoints. Firms do this by scheduling
| calls with FDA staticians and experts. [2]
|
| _And these labels should never be used as the primary source
| of punishment. Any and all sanctions /penalties/dismissals must
| go through a new review process done by a different group. _
|
| Maybe there is a difference here. I'm not sure what you mean by
| punishment? In the current system, The FDA can use the label as
| "punishment". The FDA may require addition "black box warnings"
| for drugs that are found to have serious side effects (e.g.
| high chance of death). They can also pull the label entirely,
| meaning the drug can not be sold.
|
| _Any scientist who gets a label in a particular year should be
| given a vote to review the review agency on several dimensions.
| These aggregate reviews should be published broadly but not
| trigger any automatic consequences._
|
| This is basically how it works for medical device labeling in
| the EU. There are several "notified bodies" [3] which are
| private agencies to review the safety and efficacy. The firm
| then takes their mark of approval to the government agency.
|
| https://www.accessdata.fda.gov/drugsatfda_docs/label/2014/20...
| https://www.fda.gov/media/72253/download
| https://climedo.de/en/blog/list-of-mdr-certified-notified-bo...
| obblekk wrote:
| The async thing means papers can be published without waiting
| for a gov agency to do its review. FDA takes the opposite
| approach.
|
| Generally, reading a paper is much less risky to a person's
| health than taking a drug so the differences in review
| process add up meaningfully.
| dablweb wrote:
| Ironic that nature publishes this despite being guilty of
| constantly promoting it.
| jet_32951 wrote:
| Have a look at Derek Lowe's excellent blog [0] to view the depths
| of malfeasance, if not outright fraud, in which "studies" are
| created.
|
| [0] https://www.science.org/content/blog-post/fakin-it-modern-
| wa... is just his latest in a long string of well-documented
| posts.
| light_hue_1 wrote:
| This is a problem everywhere where the raw data isn't released
| (suitably anonymized).
|
| In cognitive science, psychology, even computer science / ai /
| ml, business.
|
| And the problem with rejected papers getting in somewhere else
| while being total garbage is pervasive. I've rejected a lot of
| papers because they were mathematically or statistically bogus
| only to see them get published elsewhere where reviewers were not
| so careful (a few times in Nature and Science).
|
| We need an open science movement where you must release
| everything with your paper. The full pipeline to reconstruct
| every single result from the raw data. No hiding data. No hiding
| fmri scans. No "our code only runs on our machine". Etc.
| v4dok wrote:
| I think value-based care is the only real incentive on this.
| Otherwise, there is simply no reason for anyone to care enough.
| Even the insurers, they found a way to make money by making
| sure their premiums factor in these things. In the expense of
| the patient. As long as the drug doesn't kill people, who cares
| if it works if I make money off it as a pharma? Unfortunately,
| value-based care can only be pushed top-down. Patients are not
| in a position of power against pharma companies on this matter.
| derbOac wrote:
| I'm empathetic to what you're suggesting -- I've published on
| open science and meta-science specifically, and think open data
| should be the default norm. The problem with clinical research,
| though, is that it starts running into conflicting
| considerations about patient and participant privacy. Even when
| people aren't patients per se, the focus often involves
| sensitive information.
|
| You can just say "anonymize it" but that turns out to be more
| difficult than it seems initially, especially with many
| questions of interest.
|
| Also, there's often too many opportunities to do science that
| is of real public benefit that comes with privacy expectations
| attached for all kinds of reasons. Cases where there is
| legitimate consented access but an expectation of privacy
| without data sharing.
|
| People have tried to solve this problem in different ways (for
| example, methods where someone can analyze data without having
| access to it directly) and maybe those solutions will lead to a
| good resolution. But they often have problems of their own
| (overhead costs associated with providing anonymized remote
| data analysis), and don't solve all problems (guarantees of
| absolutely restricted access to personal data).
| Spinnaker_ wrote:
| We wasted decades, billions of dollars, and countless
| promising careers due to bad and fraudulent research in areas
| such as Alzheimers.
|
| Whatever the costs and challenges are, they are not nearly as
| high as maintaining the status quo.
| mydriasis wrote:
| More and more it seems that science is being clouded by moneyed
| interests and greed. If we can't trust science, what can we
| trust?
| thechao wrote:
| This is just reporting bias. I've worked (on-and-off) in
| various scientific fields for ~30; it has always had its bad
| actors. (I even helped do the statistics for some!)
|
| I'd urge you to consider following the situations: 1.
| Prescientific inquiry; and, 2. PreFDA food, drug, and medicine.
|
| Both of those were orders-of-magnitude worse than what we have,
| now. Could we do better? Sure! Is it broken? No.
| mydriasis wrote:
| That brings a bit of hope!
| LorenPechtel wrote:
| Disagree. Broken isn't a binary--the current system is far
| better than what came before, but that doesn't mean there
| aren't serious flaws in the current system.
| jasmer wrote:
| [dead]
| [deleted]
| isaacremuant wrote:
| No. no. This is not permitted. Trust the science or you're an
| antivaxer, Trumper, denier, racist, white supremacist.
|
| Media and gov, or media at the behest of gov (as the twitter
| files prove, but it was obvious without them), censored as much
| as thet could, everything that went against what they wanted to
| push. It wasn't science but security/hygiene theater and it
| worked, because people did go along and did turn on their
| neighbours who opposed the measures.
|
| But now we slowly get tidbits of things we can debate again ...
| Funny that, but no recognition that the entire lockdown, masks
| and vaccine mandate effort + economic destruction and theft (tax
| money to corps as "aid") was never a reasonable, logical or
| scientific response. It was an authoritarian and corrupt response
| of extreme Effectiveness and cynicism.
|
| But here comes some guy to say "people have always quarantined in
| pandemics" or some other disingenuous claim that ignores the
| reality of what happened: healthy people denied basic human and
| constitutional rights.
| LorenPechtel wrote:
| The problem here is "healthy people". Covid's real key to
| success is the fact that most spread is presymptomatic. It's
| the apparently healthy people spreading it!
|
| And note the lessons of history:
|
| 1) There will always be those who choose short term economic
| interests over safety when the threat isn't absolutely proven.
| They'll always close the barn door too late.
|
| 2) Places that take epidemic/pandemic threats seriously tend to
| fare better economically in the long run.
| isaacremuant wrote:
| No. The problem is that fundamentally, you and people like
| you decided that authortiarianism is ok if the gov says
| "things are scary" and everyone should suffer the
| consequences. No debate allowed.
|
| The risk profile for people was always ridiculous and you
| were never going to contain it once it was widespread, which
| it was, but somehow we believed in the rolling "it's just 2
| weeks".
|
| 1) it wasn't short term. It wasn't safety. Short and long
| term you hurt the vulnerable and you helped the rich and
| powerful.
|
| 2) Rich places will keep being richer and poor places poorer,
| and when the powers that be decided that enough was enough,
| all the concerns of the hypochondriacs suddenly didn't
| matter.
|
| You could tell, if you paid attention, that politicians
| weren't afraid after the initial surge, but wanted "the
| masses" to be. You even have definite proof in many places,
| one of them being UK and number 10. Can't link it now, sorry.
| But Google downing Street covid rules or something on that
| note and you can probably find a lot. It wasn't limited to
| the UK. It was everywhere you looked properly.
|
| You were scammed and you either were well off and didn't mind
| that much or you want to pretend you weren't for your own
| mental health. Because the truth is a hard pill to swallow.
| OnlyMortal wrote:
| To be fair, science is now a "publish or be damned" business. So
| many papers are not worthy of publication and, frankly, are of
| common knowledge anyway.
|
| Of course, this is driven by the money backing the research
| which, I'm sure, this is why clinical trials in the medical areas
| can be of a poor quality.
|
| People want to keep their jobs.
| j-pb wrote:
| That and lawyer driven development.
|
| I have multiple sclerosis, and I never know if a new medication
| came out because it's actually better or simply because they
| could get a new patent for a slightly modified molecule.
| LorenPechtel wrote:
| Disagree--you're talking about patent-driven development, not
| lawyer-driven development.
| egberts1 wrote:
| I now know of a few more people who have completely lost both of
| their hearing after taking just one Wellburtin-class pill.
|
| At the time, no mention was made in the pill's warning pamphlet.
|
| It is still difficult to secure a class-action suit in America.
|
| Meanwhile, such quality of life would plummet into a silent
| world, even if one knew American Sign Language fluently
| beforehand, that tidbit can go against the victim in court.
| appleflaxen wrote:
| If you read the article, the headline is _wildly_ editorialized.
|
| Whatever. Par for the course in 2023, right?
|
| But this is _Nature_ , a paragon of scientific literature,
| fueling the distrust of medicine.
|
| There are great reasons to be skeptical of all trials and
| strengthen peer review and transparency, but this kind of
| headline is editorial malpractice, in my opinion.
| twic wrote:
| No, not really. The headline says "plagued with", the subhead
| says "in some fields, at least one-quarter of clinical trials
| might be problematic or even entirely made up", and the article
| substantiates that. One quarter of all trials is more than
| adequate to plague the whole enterprise, and there isn't a
| problem with this headline.
| ethanbond wrote:
| Ah yes, the greatest of all weasel words: "problematic."
| peteradio wrote:
| Want to find out what they mean by "problematic"? Read
| beyond the subhead..
| notjoemama wrote:
| That word has gotten a bad wrap by people on social media
| using it without substantiating it, or hiding behind an
| excuse of "Google it"; meaning 'I know but you don't so
| go educate yourself because I'm subtextually declaring
| your opinion invalid by way of your ignorance'.
|
| We are seemingly in a fifth generation war amongst
| ourselves for the prizes of attention and public
| acceptance. Or more succinctly, "being right on the
| internet".
|
| What may in fact be the next great filter. :)
| bowsamic wrote:
| We _should_ be distrustful of medicine, of all science in
| general. Ignoring flawed methodologies or inconclusive results
| just means it 's no longer science, it's ideology. I'm a
| physicist though so perhaps I could be overly jaded about
| science and peer review compared to most scientists
| dekhn wrote:
| You simply can't apply the rules of publication in physics to
| medical biology research. Even highly quantitative biology is
| noticeably different in terms of standards of proof and
| quality of models.
| bowsamic wrote:
| Are you saying that we should be less distrustful of
| medical biology than of physics? I don't see why that
| should be so
| dekhn wrote:
| no, the other way around (obviously?)
| bowsamic wrote:
| But, that's my point. I'm telling you that you should be
| distrustful of physics, so you should be _really_
| distrustful of medical science
| dekhn wrote:
| I haven't seen any real serious replication problems in
| physics that didn't get cleared up, or anything else that
| would make me doubt the results.
|
| I would generalize the statement: assuming a reductive
| order of sciences
| (medicine->biology->chemistry->physics), if one cannot
| trust a layer, it seems even more likely that layers
| above it should be trusted even less.
| taeric wrote:
| Agreed that the headline makes it sound like a strong majority,
| when the article isn't nearly as strong on that.
|
| Still, "Carlisle rejected every zombie trial, but by now,
| almost three years later, most have been published in other
| journals -- sometimes with different data to those submitted
| with the manuscript he had seen. He is writing to journal
| editors to alert them, but expects that little will be done."
| is concerning. I'm almost afraid to know what the list of
| rejected papers covers.
| peteradio wrote:
| If a quarter of your body was covered in leaches would you
| consider yourself plagued by leaches?
| taeric wrote:
| I mean, fair that "plagued by" is a very vague term that
| has no quantifiable meaning. But appeals to emotion to cast
| doubt on all studies is frustrating. "Healthy skepticism in
| the face of bad studies" would be a great headline and is
| accurate. But too much of the skepticism we are exposed to
| on a regular basis is not healthy.
| lesuorac wrote:
| I don't think plagued has ever meant X%.
|
| You can be the sole person in the world with the Bubonic
| Plague and you'll still be plagued by it. Whether or not
| untrustworthful clinical trials "cause distress to" [1]
| Medicine I don't think is debatable; medicine should be
| based on treatments that have reproducible effects or
| else people don't get better.
|
| [1]: https://www.google.com/search?q=define+plagued
| taeric wrote:
| At an individual level, absolutely. In group dynamics,
| though, you avoid things that have the plague. Same as
| you avoid things that have rabies.
|
| Which is part of my point. If you say that "medicine is
| plagued" than a natural response is "avoid medicine."
| But, that is clearly a nonsensical outcome, all told.
| peteradio wrote:
| If 1/4 of all studies have fundamental data issues then
| that means in some corners we are making medical
| decisions based on bunk, that absolutely is a plague. Why
| moderate the language to extenuate?
| taeric wrote:
| Because if 100% of that 1/4 of studies is all in, say,
| homeopathy, that gives a very different action plan than
| if it is a random sampling of all studies.
|
| Still bad, mind you, but unfocused skepticism is its own
| plague that will cause more trouble.
| peteradio wrote:
| Exactly. Its important to see this stratified across
| subfields. If the vast majority are in homeopathy then
| maybe who cares? If a substantial portion are in, oh lets
| say Alzheimers treatment then maybe that's more of a
| problem?
| taeric wrote:
| Basically this. So, fair that I shouldn't just be calling
| for moderating the language. I'm more wanting specific
| language with a distaste for unfocused skepticism. I say
| this as a skeptic. :D
| bmh wrote:
| After reading "The Real Anthony Fauci" I'm not so sure the
| headline is hyperbole. The article says that 1/4 of the trials
| studied were badly flawed.
| ekianjo wrote:
| Closer to 50% of trials are probably junk since they cant be
| reproduced independently
| throw9away6 wrote:
| That's being generous as only 1/10 can actually be fully
| reproduced from my understanding. It's so bad that if you
| want to create a product based on research your first need
| to reproduce the result to make sure it's not bs.
| MichaelZuo wrote:
| I haven't heard of the 10% figure before, can you link to
| the source?
| defrost wrote:
| [flagged]
| dablweb wrote:
| [flagged]
| bmh wrote:
| [flagged]
| swader999 wrote:
| [flagged]
| smrtinsert wrote:
| I can't think of anyone less qualified to write on the matter
| than RFK jr, maybe MTG.
| logicchains wrote:
| Well his book is well-sourced so fortunately you don't have
| to take him at his word, you can check the references if
| you want.
| LorenPechtel wrote:
| Doesn't preclude being very deceptive with the facts.
| ramraj07 wrote:
| There's a difference between a scientist calling bs on some
| scientific practices than a grifting, crazed group of people
| calling bs on it. Can you cite the exact trials from your
| source that are potentially problematic so we can discuss the
| actual legitimacy of the methodology?
| LatteLazy wrote:
| [flagged]
| ethanbond wrote:
| RFK Jr is just a blatant, proven liar.
|
| Here's one such case that I'll point out only because the
| bullshitting is so clear:
| https://www.cnn.com/2023/06/22/politics/robert-f-kennedy-
| jr-...
|
| He claims to have worked with Tapper for "three weeks on a
| documentary" (they worked for 2 days on a 2-minute spot) that
| was "killed by corporate" (it went live one day later than
| planned).
|
| I'd recommend not using pathological liars' words as evidence
| for other claims.
| burkaman wrote:
| Can you share what you read in that book that is relevant to
| this article?
| thenerdhead wrote:
| Another good reason why you should learn how to read studies:
|
| https://peterattiamd.com/ns001/
|
| https://biolayne.com/reps/how-to-read-research-a-biolayne-gu...
|
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7392212/
|
| And as a bonus, read books by controversial figures who talk
| about these challenges through their own published studies and
| decided it isn't worth fighting.
|
| What would be cool is to see "trust indicators" as part of a
| study's metadata as it is reviewed through time and continually
| shared by others. Could be a "study health score" or a checklist
| showing the study isn't biased heavily by sponsors, methods, or
| misleading language.
|
| Especially given that medicine is becoming more personal with the
| advancements of AI and accessibility of tests, someone should be
| able to understand the health of a study at-a-glance reviewed by
| peer reviews rather than disclosed only by the authors.
| randcraw wrote:
| A primary trust indicator of any research result is the
| reputation of the journal publishing the paper. Better journals
| demand more, not just in terms of the impact/insight of the
| results, but also the rigor of the experimental methodology
| used and how well the data was curated and the confounding
| variables identified and isolated.
|
| And of course, the criteria for observational-based research
| differ a lot from mechanism-based, especially since the former
| can't control for nearly as many variables. The same goes for
| simulations or interpretive modeling, where experimentation is
| minimal.
|
| Research studies differ quite a lot in how precisely the
| mechanism of action is identified, isolated, tested, and
| results interpreted. IMHO, blackening the trustworthiness of
| all of science is unhelpful, especially when some models and
| methods are surely more trustworthy and replicable than others.
| Better to identify and catalog specific sources of error (or
| imprecision) in order to remedy them than to just toss the
| baby.
| tails4e wrote:
| Nutrition for me it a big issue. It seems like and ideal area for
| scientific and medical study to give us light, but seems so hard
| to get truly objective info. Fat is the devial/good/bad/OK, sugar
| is the devil/bad/ok, etc. Should I avoid white bread like a hole
| in the head, or is it fine? Too many agendas and not enough
| truth.
| darkclouds wrote:
| I'd trust the NHS if the National Institute of Clinical
| Excellence (NICE) actually published their minutes online. Most
| employees only know what they have been taught, so whilst I agree
| with the headline that some trial data is dodgy, it doesnt just
| end there.
|
| There are multiple pathways to factor in, there is redundancy
| built in, ie secondary pathways, there is the fact that not all
| chemicals go where intended (best highlighted by radioactive
| isotopes). When looking at the history of patented medicine, this
| really started between WW1 and WW2, before WW1, most GP's
| prescribed what was found in the body on a like for like basis,
| and in some cases prescribed organs in various forms, like
| desiccated thyroid gland for thyroid related problems, for
| pernicious anaemia, raw liver used to be prescribed to women and
| so on, but that has its problems like contamination and diseases.
|
| So from WW1/WW2 onwards the rise of patented medicine took hold,
| but the main problem with patented medicine is the human body
| hasnt evolved to use these new chemical compounds in the same way
| as unpatented chemicals which have been around for thousands of
| years. And todays GP's dont really highlight the side effects of
| the patented medicine, and because they simply dont ask what you
| have been eating and drinking etc, they have this hubris which
| sucker punches your trust and sucks you in like a black hole,
| until before you know it you are on half a dozen different drugs,
| your quality of life is going down the pan and you've been left a
| zombie wondering where did Hitler go wrong modelling the German
| state of the time on the British Empire? You Americans
| complaining about the cost of healthcare, should count yourself
| lucky those insurance companies are looking out for your long
| term interests and theirs!
|
| Fortunately, hospitals in other countries publishes studies and
| as english seems to be the main language used for science, I have
| to tip my hat to the Chinese who are roaring up the charts in
| terms of investigating and publishing relevant studies that will
| complement a quality of life one hopes to achieve, and we cant
| forget Wikipedia, Pubmed and Google for connecting users with
| pertinent studies.
|
| Saying that I do sometimes wonder if something like ChatGPT has
| written some studies due to the poor quality of english used, but
| generally they stand out like a sore thumb.
|
| Anyway, does any know why there is a connection between MDMA,
| blood clotting and Manganese?
| gordian-not wrote:
| Masters and Phd requirements should include repeating another
| research and checking whether they can recreate the results
| sealeck wrote:
| Unfortunately some of the experiments require a whole team to
| carry out and take a huge amount of time to set up as well as
| having to be carried out under very specific conditions.
| Slaminerag wrote:
| My institution's been included in studies primarily for patient
| access. If we're not included, then good luck getting enough
| patients, and even then it can take several years to enroll
| enough patients. Replicating such a trial would be near
| impossible.
| gigatexal wrote:
| In the age of QAnon and Alex Jones and science and vaccine
| deniers we can't have things like clinical studies be corrupted.
| Ugh.
| linuxftw wrote:
| The 'vaccine deniers' have been calling the 'studies' and
| 'trials' trash for a long time. The evidence comes out that
| they're trash, and somehow the 'vaccine deniers' are still
| wrong?
|
| They removed liability from manufacturers, and suddenly the CDC
| schedule exploded with new products. I mean, chicken pox has a
| vaccine now?
| Dig1t wrote:
| I agree that the economic incentives changed, and it does
| seem somewhat suspicious.
|
| I have a lot of work to do ahead of me, researching all of
| these vaccines for my kids. Makes my head hurt just thinking
| about it.
| linuxftw wrote:
| If you look at the individualized risk for childhood
| vaccines and flu vaccines, there's effectively zero benefit
| if you live in a 1st world country, and possibly a great
| risk of neurological or immunological side effects.
| nocoolnametom wrote:
| That which was asserted without evidence could correctly be
| dismissed without evidence. AT THE TIME the anti-vax crowd
| was basing their positions entirely upon anecdote, rumor, and
| often badly misread prepublication research and stats. Their
| methodology was inherently flawed. Even if the conclusions
| they came to have been "validated" their position was still
| built upon this same flimsy scaffolding. It's not like the
| "do your own research" blogs and videos somehow gathered the
| same evidence used by this paper. This also does not indicate
| that other positions held by the same crowd, which are
| similarly based upon "anecdata" and rumor, are somehow made
| more evident by this paper in Nature.
| logicchains wrote:
| >That which was asserted without evidence could correctly
| be dismissed without evidence. AT THE TIME the anti-vax
| crowd was basing their positions entirely upon anecdote,
| rumor, and often badly misread prepublication research and
| stats.
|
| The was an abundance of evidence that the covid vaccines
| had a reasonable likelihood of being unsafe. Every single
| previous attempt at a coronavirus vaccine had failed,
| sometimes catastrophically (killing all the test animals),
| that's why there wasn't an existing coronavirus vaccine on
| the market. Every single previous attempt to bring a mRNA
| treatment to the mass-market had failed due to safety
| issues. Even in the Pfizer vaccine trial there were overall
| more deaths in the vaccinated group than the placebo group,
| due to cardiac deaths (although it wasn't a statistically
| significant enough amount to draw a conclusion, it does
| demonstrate that the trial had no power to identify if the
| vaccine was net-harmful, as it didn't have enough
| participants to make a meaningful conclusion about the
| effect of the vaccine on excess deaths).
| linuxftw wrote:
| Don't forget, the Pfizer phase 3 trial was ended early
| because they claimed that it was 90% effective. So, any
| mid/longer term issues were missed.
|
| The pregnancy trials were outright abandoned.
|
| They didn't even conduct clinical trials for the bivalent
| boosters.
|
| Zero efficacy in children, yet still strongly recommended
| by the media and the state.
| SV_BubbleTime wrote:
| >Every single previous attempt to bring a mRNA treatment
| to the mass-market had failed due to safety issues.
|
| Not a single prototype mRNA-based drug passed phase3
| trials at any point - right up until the multiple ones
| within a month of each other were deployed globally.
|
| The massive and remarkable coincidence of that, is truly
| a special moment in history.
| linuxftw wrote:
| I agree, the assertion that 'vaccines' are safe can be
| dismissed without evidence. There's no evidence concluding
| they're actually safe. In fact, we have given the
| manufacturers immunity because they're 'unavoidably
| unsafe.'
|
| Just look at how the COVID trials were conducted. They
| didn't even test each patient. Only some patients that
| presented symptoms, and then not even all of those
| patients.
|
| How long did they follow the health outcomes for approved
| vaccines in the test groups? 3 months at most, and many
| trials, not even that long. So if someone suffers a
| neurological condition, well, we just won't know about it.
| sonicshadow wrote:
| Yeah not a great look, maybe science isn't the truth after all
| mjfl wrote:
| It's hard enough to run a clinical trial guys. It literally costs
| $100 million at a minimum, yet the requirements that make it cost
| this much are not enough. We are basically going to regulate new
| medicine out of existence.
| droopyEyelids wrote:
| Hamilton Morris recently did a podcast that touched on some of
| this, I think it was
| https://www.patreon.com/posts/pod-78-legal-84786504
|
| An interesting point he made was that in the wake of the
| Thalidomide (https://en.wikipedia.org/wiki/Thalidomide)
| scandal, the FDA started requiring drugs to be both safe _and
| effective_
|
| https://www.fda.gov/about-fda/histories-product-regulation/p...
|
| The 'effective' part has proven to be a big source of
| complexity in the following years, because while it's
| relatively easy to prove a drug is relatively safe, it's much
| more difficult and subjective to prove a drug is effective.
| That closes off a lot of areas of research and development.
|
| The kicker is that Thalidomide was never sold in the USA to
| begin with.
|
| Anyway, as any reader can imagine, there would be a lot of
| negative social outcomes to allowing the sale of ineffective
| drugs. There's a lot of trouble now with medical devices and
| drugs not being effective, even though we have the rule. I'm
| not against regulation, I think medical sales are a really
| complex issue and I don't know how to even judge where the
| right balance of safety/effectiveness and innovation/freedom
| could be.
| dekhn wrote:
| Thalidomide is sold in the USA as a treatment for several
| conditions- it's a highly effective drug and is mostly safe
| within the target population.
|
| (I point this out because most people only tell the very
| first part of the thalidomide story).
| throw9away6 wrote:
| They kind of broke that when they allowed the approval of
| Aduhelm which is basically shown to be expensive and
| ineffective
| WastingMyTime89 wrote:
| Please read the article before commenting. The problem is not
| how hard it is to run clinical trial. It's that made up data is
| an endemic problem. It doesn't matter if clinical trials are
| hard or easy to organise when up to a quarter don't actually
| bother and just forge their results.
| chaxor wrote:
| But they _have_ to make up that data! Because the work they
| are doing now was based on a other trial where they made up
| data, so you have to fix this data to match what was expected
| from the previous studies. Of course we need to protect their
| right to make up data. /s
| sonicshadow wrote:
| Yeah who needs truth and accuracy anyway? Think of the small
| businesses who will never be because they couldn't get a simple
| drug on market with minimal testing
| mjfl wrote:
| You have to put $100 million into effective testing. Your
| comment is disingenuous.
| xhkkffbf wrote:
| Even if they aren't faked, the ability to shut down a trial that
| isn't delivering the right preliminary data is a big problem for
| society. Why should we trust these drugs?
| throw9away6 wrote:
| A lot of these studies are prerun in the 3rd world before the
| real one is done for credit. If sideffects show up the drug can
| sometimes be mixed with one that has the same known side
| effects to fool the studies.
| jmpeax wrote:
| Raw data examined: Ok 56%, Flawed 18%, Zombie 26%
|
| Raw data not available: Ok 97%, Flawed 2%, Zombie 1%
|
| Perhaps it's not good to call an unknown as "Ok"? Maybe Carlisle
| should add his own paper to the mix?
| twic wrote:
| That's literally the point he's making! The terms "flawed" and
| "zombie" reflect positive identification of dodgy data, so of
| course when the data isn't available, they are less likely to
| apply, hence why:
|
| > This finding alarmed him, too: it suggested that, without
| access to the IPD -- which journal editors usually don't
| request and reviewers don't see -- even an experienced sleuth
| cannot spot hidden flaws.
| freedude wrote:
| "I think journals should assume that all submitted papers are
| potentially flawed and editors should review individual patient
| data before publishing randomised controlled trials," Carlisle
| wrote in his report.
|
| This should be considered part of every journal's idea of due
| diligence and this shouldn't be a new idea. Shysters, con artists
| and snake oil salesmen have been around for a long time. The
| purpose of a Journal is to publish reliable information and weed
| out the garbage. How can you do that if you are not looking at
| the entire picture?
| Roark66 wrote:
| They say "medicine". I would say science in general, perhaps we
| could generalise even further to "any human activity is full of
| unethical people trying to exploit it". But 25%!? That suggest
| there is a big problem with how we "do science". Unfortunately I
| have no solution to the problem. Publishing everything (including
| raw data) for every research would probably help somewhat, but
| only teams repeating experiments/trials would ensure it.
|
| I wonder if we suddenly took 10% of all money spent on science
| (let's say in medicine) and instead of novel research we used it
| to redo randomly chosen previous research. Would we loose or gain
| in terms of new cures? And if we gained, what if we spent 15%, or
| 25%? That's a great idea for a scientific study to find a point
| of diminishing returns on "research verification".
|
| Would someone please write a research grant request for this?
| bobbylarrybobby wrote:
| That 10% investment would have a huge payoff too because it
| would shut down avenues that were only opened due to p-hacking
| long before they'd had a chance to seek further investment.
| davidktr wrote:
| Of course there are big problems with how we do science. Much
| of it is garbage. Imagine most software was written by junior
| engineers, without any code review or input from seniors. That
| is today's science.
|
| Most scientific legwork is done by absolute beginners, i.e.,
| graduate students. They often lack a support structure to focus
| on what they have learned so far. Most of the world is not
| Oxbridge, MIT, Stanford.
|
| Where are the beginners' supervisors, you might ask? Chasing
| the latest trend to secure funding. Pondering how their line of
| research can be formulated as buzzword-du-jour markov chain.
| Ass kissing the dean to get department funding.
|
| Having worked in research for 15 years, I am certain about two
| things: (1) The scientific method yields better results than
| doing things freehand. (2) Randomly axing 50% of academia would
| improve the situation.
| alexb_ wrote:
| I'm not too knowledgeable when it comes to how scientific
| experiments/trials are done - are the people who collect the
| data, the people who interpret the data, and the people who
| fund/benefit from the data different parties? Or are they the
| same people?
| Balgair wrote:
| Great question, and I'm unfortunately going to have to give the
| answer of 'it depends'.
|
| Each study is different and therefore run differently. Many,
| _many_ , factors determine how a study is run, analyzed, and
| published.
|
| Most studies are very small, using only one site and a few
| volunteers. Most of these never see the light of day, as the
| results aren't publishable or are uninteresting. Think power
| law distributions with studies, not normal distributions. These
| studies are often so small that the collector and interpreter
| are the same person, typically they are also the grant writer
| and admin. If lucky, they may get some nurses or undergrads to
| help out. Again, I'd say this is ~80% of studies.
|
| The really large studies that places like Pfizer run will
| separate out nearly all parts of a study. So consenters,
| nurses, intake, data admin, funding admins, stats guys, etc are
| all different people. These are very expensive studies to run
| so it's really only for FDA approval, not scientific inquiry
| and case studies.
|
| Generally, most studies are very small and not publishable.
| They don't need to separate out everyone. Everyone kinda trusts
| that everyone else is doing their best. If something snazzy is
| found, then follow up studies will build on it's findings. Most
| of the time though, nothing is really found.
| WaitWaitWha wrote:
| In my experience, they are different (speaking as someone
| witnessing it from layman's perspective). Here is what I have
| seen in Phase II & III trials:
|
| a) Pharma identities the type of patients they need (e.g.,
| 25-50 female, not pregnant, with specific ailment if Phase
| III), specific tests, and measurements required throughout the
| study.
|
| b) pharma contacts third party (3P) to manage study patients.
|
| c) 3P has relationship with dozens or even hundreds of doctors'
| offices, knowing what office can fulfill the test & measure
| requirement, and has the potential trial patient pool.
|
| d) 3P has existing contract with these doctors' & hospitals.
| They get patients onto the study. ( <--- #1 reason this is
| farmed out in my opinion)
|
| e) Doctors & hospitals perform the study and collect the data.
|
| f) doctors & hospitals pass the data to the 3P
|
| g) 3P passes it to the pharma
|
| h) repeat e) through g) as many times as the study requires it.
| This can be once, or many times over years.
|
| i) pharma pays 3P, 3P pays doctors & hospitals, and they pay
| the trial patients - each taking their cut along the way.
|
| There are variations on how this is done, sometimes no 3P,
| sometimes pharma will have their own pool of patients and 3P.
| Also this is a very rough flow as there are often checks,
| audits, and validations (should be) done during the study.
| linuxftw wrote:
| For vaccine manufacturers, it's all the same people. Even if
| you have whistleblowers come forward, they're ignored.
|
| Here's one quick way to rig any clinical trial: Anyone and
| everyone that has any kind of negative reaction or health
| condition gets disenrolled. Since it's 'double blind' it
| appears on the surface level that there's no way to know who's
| in what group. Naturally, the end result is always the same:
| The test group had the same number of reactions as the control
| by the end of the study.
| nocoolnametom wrote:
| > "Even if you have whistleblowers come forward, they're
| ignored."
|
| Do you have any sources for this? I'm rather disbelieving of
| it, but would love to be proven wrong. I can't imagine that
| _some_ major news outlet wouldn't love to stick it to the
| status quo with a whistleblower, unless the "whisteblower"
| made false claims about their proximity in the company to the
| dangerous/illegal actions they are trying to bring attention
| to.
| linuxftw wrote:
| Here's a whistle blower from the Pfizer covid trials:
| https://www.bmj.com/content/375/bmj.n2635
|
| We can also see that the FDA does nothing to investigate
| the integrity of the trials. They just accept whatever the
| manufacturers tell them.
| epicEHRsucks wrote:
| There unfortunately are too many perverse incentives that
| encourage fraudulent studies. Everyone should take findings
| "established in the literature" with a grain of salt. We should
| also incorporate more intuition and first-principles reasoning,
| i.e. obesity is a harmful state for humans even if 100 RCTs
| proved otherwise.
| constantcrying wrote:
| How can you do science if around a quarter of the data is just
| straight up noise? Even when analysing large amounts of studies
| the results becone contaminated very easily.
|
| And _why_ is it not standard practice to provide anonymous data
| _or even publish the data_? What reason exists for that? So that
| only the researchers them selves can analyze it?
| thechao wrote:
| The working stuff becomes part of a body of lore of "real
| science" that you soak up in the lab. Not a great method, for
| sure.
| constantcrying wrote:
| But the point of RCTs is that you can get unbiased, high
| quality results _without_ having to rely on "lore" spreading
| among medical professionals about which treatments are
| effective and under which circumstances.
| twic wrote:
| That's actually mentioned in the article, which is quite good
| and worth reading:
|
| > In 2016, the International Committee of Medical Journal
| Editors (ICMJE), an influential body that sets policy for many
| major medical titles, had proposed requiring mandatory data-
| sharing from RCTs. But it got pushback -- including over
| perceived risks to the privacy of trial participants who might
| not have consented to their data being shared, and the
| availability of resources for archiving the data. As a result,
| in the latest update to its guidance, in 2017, it settled for
| merely encouraging data sharing and requiring statements about
| whether and where data would be shared.
| mike_hearn wrote:
| Yes the problem is that the research system rewards publishing
| papers, but most of the work is collecting the data. So if you
| release your data then other groups can write papers based on
| your effort for far less cost. It's sort of analogous to the
| problem of open source business models in the software world:
| if company A writes the code and releases it for free and earns
| money from running a cloud service, and company B just offers a
| cloud service, then the second company can get much higher
| margins because they don't have to develop it.
|
| Unfortunately it's not easy to see what the alternatives are,
| beyond simply not funding research through
| government/foundation grants. When science is paid for by
| companies you don't have this incentive issue because the
| research is judged based on (ultimately) whether it leads to
| successful products, not whether it leads to lots of papers
| getting published. You have other incentive issues of course.
| jonlucc wrote:
| I work in preclinical pharma research, so I have spent a good
| amount of time trying to recapitulate published data in animal
| models, so not quite clinical trial data. People who do this
| work learn how to evaluate trustworthiness. It can be as
| granular as "this lab is the only one publishing this kind of
| information, so we'll be skeptical" to a bit more broad "this
| class of drug isn't expected to have that biology" to "I trust
| this company over that institution". We route around the fact
| that some information isn't reliable, and that's not always
| because of dishonesty or fraud.
|
| I'm not a clinician and don't deal with them regularly, but the
| impression I get is that new studies are published by
| researchers who have a lot of connections (dubbed thought
| leaders). They present at conferences. Other clinicians pick up
| the use case that matches their need (this patient has failed
| other therapies for this indication, let's try this new thing
| I'm now aware of). Then as experience grows, clinicians have
| more nuanced understanding of the use cases for that new
| information and its reliability. Frustratingly, this can take
| years, but that's bug that's also a feature.
| evandijk70 wrote:
| Privacy concerns are the most important reason. A more cynical
| reason might that the odds that a papers is deemed 'ok' is far
| larger if the raw/anonymous data is not provided. Remember, the
| authors of the suspect studies provided their data voluntarily,
| and in the end it only hurt their reputation/impact.
| amai wrote:
| There should be a paperswithdata for medicine like there is a
| https://paperswithcode.com/ for data science.
| gumby wrote:
| Note that the article discusses research studies and not clinical
| trials for drug or device approval.
|
| These research studies are important (look at how many were
| conducted on COVID-19 over the last few years) but are typically
| not held to a particularly high standard, as with most science.
| Which doesn't excuse bad data or poor statistics (the latter
| supposedly supposed to be picked up in peer review).
| piqufoh wrote:
| Hmm, I read the article as explicitly calling out "clinical
| trials" (as referenced in the title and abstract) and it makes
| no reference research studies. I don't understand the
| distinction between "research studies" and "clinical trials",
| surely all research studies where an RCT is performed with real
| patients and real drugs is a clinical trial?
| gumby wrote:
| I meant "trials for research studies" as opposed to "trials
| for drug or device approval."
|
| The amount of record keeping and oversight of a drug approval
| trial is enormous (and as a consequence insanely expensive)
| -- data handling, having disjoint groups at each stage
| handling and analyzing data, etc and detailed records of
| every manufacturing step -- think ISO9000 on steroids.
|
| Nobody would bother to go to that effort for a scientific
| exploration, nor should they. So the bar is much lower.
|
| I am making no excuse for shoddy science! But it is quite
| unlikely for a licensed drug.
| [deleted]
| __666__ wrote:
| [flagged]
| wahnfrieden wrote:
| Is it structural greed?
| oldgradstudent wrote:
| With the risk of starting a flame war, we recently had several
| well-publicised clinical trials that reported 95% efficacy of
| some certain modality. Yet, in reality, efficacy as defined in
| the trial turned out to be closer to 0%.
|
| Instead of investigating what in the design and execution of the
| trial led to such a discrepancy, the problem was handled by
| denying there was a problem, changing the goalposts, reporting
| ad-hoc hypotheses as facts, silencing all critics, and forcing
| the public to take the modality anyway or lose jobs, school, and
| freedom of movement.
| taeric wrote:
| I can only guess you mean something regarding the pandemic? Do
| you have links to show things were "closer to 0%?" Sounds more
| than a touch outlandish. :(
|
| I'm confident we will know even more about things as time goes
| on. I'm less confident on any nefarious motivations in most of
| it. Reality is that a lot of people died, and everyone was
| trying to gain control and an advantage over the situation.
| Mistakes were certainly made, but I am back to low confidence
| in thinking that everything was a mistake.
| oldgradstudent wrote:
| > I'm confident we will know even more about things as time
| goes on. I'm less confident on any nefarious motivations in
| most of it. Reality is that a lot of people died, and
| everyone was trying to gain control and an advantage over the
| situation.
|
| It's not a matter of being nefarious. They (CDC, FDA, health
| authorities all over the world) really though it was
| important, but they've used unacceptable means to enforce
| their beliefs.
|
| Science dies when PR takes over reality.
|
| If reality disagrees with the trial, you have to debug the
| rial, and find what in the design or exection went wrong.
|
| > Mistakes were certainly made, but I am back to low
| confidence in thinking that everything was a mistake.
|
| Silencing critics by health authorities is not a mistake. It
| is an intentional act to enforce your views.
| oldgradstudent wrote:
| >Do you have links to show things were "closer to 0%?" Sounds
| more than a touch outlandish. :(
|
| Take the Pfizer vaccine. The clinical trial's main endpoint
| was ~95% efficacy in one thing and one thing only, prevention
| of symptomatic Covid.
|
| Not reduction in mortality, not reduction in serious disease,
| not infection, and not spread. The only thing the trial
| tested and reported was prevention of symptomatic Covid. This
| is also the sole indication in the package insert as approved
| by the FDA.
|
| In reality, everyone I know got vaccinated and got
| symptomatic Covd. I mean everyone, no exceptions. The
| situation is the similar in my entire country and around the
| world.
| sonicshadow wrote:
| Downvoted for not parroting Democratic Bay Area values. We
| will be contacting all FAANG companies and everyone listed
| in Crunchbase to let them know your an anti-vaxxer.
| Madmallard wrote:
| hahahahaha
| sonicshadow wrote:
| Repeat after me:
|
| GET ALL COVID BOOSTERS
|
| WEAR A MASK AT ALL TIMES
|
| VOTE FOR BIDEN FOR 2024, 2028, 2032, 2036
| taeric wrote:
| Do you know of any studies on the discrepancy? My
| understanding was that Omicron came out and basically gave
| the middle finger to everyone's precautions. With what
| seemed like literally nothing working against it.
| oldgradstudent wrote:
| > Do you know of any studies on the discrepancy?
|
| The discrepancy is so massive you don't need large
| studies. You can easily observe yourself.
|
| (a) Make a survey of the people you know and compare
| their vaccination status to getting symptomatic Covid.
| Apply a simple statistical test to test whether it is
| consistent with the trial results.
|
| [Spolier: it is not]
|
| (b) [advanced] what is your best estimate of the vaccine
| efficacy given your results of the survey in (a).
|
| The measles vaccine has 95% efficacy. You vaccinate and
| the disease effectively disappears.
|
| > My understanding was that Omicron came out and
| basically gave the middle finger to everyone's
| precautions. With what seemed like literally nothing
| working against it.
|
| That's an ad-hoc hypothesis.
|
| https://en.wikipedia.org/wiki/Ad_hoc_hypothesis
|
| It was quite clear that the numbers are inconsistent with
| 95% efficacy way before Omicron.
| taeric wrote:
| But this is exposing ignorance of a different kind? The
| hopes for a sterilizing vaccine were remote, at best.
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9595357/ is
| a good overview of that line.
|
| Folks don't like comparing to the flu, but in this there
| are obvious similarities. With obviously similar outcomes
| on the ability of a vaccine to give sterilizing immunity.
|
| Much to your chagrin, though, I actually can say that
| among my contacts, getting the vaccine basically led to
| people not getting symptomatic covid. Folks got what they
| thought of as a bad cold. Almost flu like, but I know
| very few, if any, folks that were so bad off that they
| were symptomatic covid. Most wouldn't have even qualified
| as having a bad flu. (It is frustrating how many folks
| underestimate how hard the flu hits.)
|
| Contrast with family members that did not get the vaccine
| in time, and were hospitalized. It was truly different.
| oldgradstudent wrote:
| > Folks don't like comparing to the flu, but in this
| there are obvious similarities. With obviously similar
| outcomes on the ability of a vaccine to give sterilizing
| immunity.
|
| And there's quite a controversy whether the flu vaccine
| is worthwhile becuase of that. The Cochrane systematic
| reviews are quite scathing.
|
| > Much to your chagrin, though, I actually can say that
| among my contacts, getting the vaccine basically led to
| people not getting symptomatic covid. Folks got what they
| thought of as a bad cold. Almost flu like, but I know
| very few, if any, folks that were so bad off that they
| were symptomatic covid.
|
| That's the definition of symptomatic Covid - a positive
| Covid test + flu-like symptoms (regardless of severity).
| That is what the trial measured and reported.
|
| (This is in contrast to Asymptomatic Covid which is a
| positive Covid test but without any symptoms at all)
|
| > Most wouldn't have even qualified as having a bad flu.
| (It is frustrating how many folks underestimate how hard
| the flu hits.)
|
| No one I know experience anything close to a bad flu.
|
| > Contrast with family members that did not get the
| vaccine in time, and were hospitalized. It was truly
| different.
|
| Around me it was a mild cold to medium flu regardless of
| vaccination, including people in their 80s and 90s, with
| all the pre-existing conditions you can imagine. The only
| exception was a vaccinated friend (late 40s) who got
| scary chest pains for several days when he contacted
| Covid. No treatment beyond Paracetamol and Ibuprofen.
| ifyoubuildit wrote:
| > getting the vaccine basically led to people not getting
| symptomatic covid. Folks got what they thought of as a
| bad cold.
|
| Am I mistaken in thinking that "bad cold" == symptomatic?
| Doesn't symptomatic just mean had symptoms? It sounds
| like you're talking about severe covid.
| taeric wrote:
| Not mistaken, but also not useful. In particular, it is
| hard to tease out folks that did have a common cold from
| those that had reduced covid. The vast majority of the
| covid positive folks I knew post vaccine were
| asymptomatic. Almost apologetic that they tested positive
| for it, but not at all sick or scared. Even my kids, when
| they tested positive, were more upset about implications
| than they were physically ill. (Indeed, for our kids,
| when they finally tested positive, we didn't see any
| symptoms from them at all...)
| ifyoubuildit wrote:
| What is reduced covid? The ifr for a 30 something was
| .06% before vaccines according to the study below.
|
| If my math is correct, thats one 30-something dying for
| every 1667 infected _before vaccines_. I don 't have
| hospitalization data handy, but I think "reduced covid"
| is just what most people had, vaccinated or not. That's
| not to discount the ones that did get it bad of course,
| and my condolences for any losses you suffered.
|
| Of course it can still be true that the deaths happened
| more often in unvaccinated people (did that continue to
| be true the whole time?), while your individual risk of
| death was low (the .06 above in my case, and I had a
| pretty standard cold both times thankfully).
|
| https://www.thelancet.com/journals/lancet/article/PIIS014
| 0-6...
| taeric wrote:
| Just look up the hospitalization and death rates for
| folks vaccinated and not. It is stark in difference.
|
| I had what was probably covid early on. Was like the time
| I got pneumonia. Asthma attacks in my youth were
| comparable, if much shorter lived. Getting a positive
| test case later was something that gave me a fever for a
| few hours. Scary, due to circumstances. But I was back up
| and moving in basically no time.
| ifyoubuildit wrote:
| > Just look up the hospitalization and death rates for
| folks vaccinated and not. It is stark in difference.
|
| Are those rates an argument against the claim that most
| people didn't have a bad case, vaccinated or not?
|
| > I had what was probably covid early on. Was like the
| time I got pneumonia. Asthma attacks in my youth were
| comparable, if much shorter lived. Getting a positive
| test case later was something that gave me a fever for a
| few hours. Scary, due to circumstances. But I was back up
| and moving in basically no time.
|
| How do you know that your possible second case's low
| severity is due to the vaccine and not the immunity you
| would have developed in the first case, or weakening of
| variants (or some mix of all 3), or even just random
| chance?
|
| It's hard to ignore personal experience, but it only
| tells us so much. Like me with my 2 unvaccinated cases
| having an easy time, I'd be remiss if I generalized that
| to everyone.
| taeric wrote:
| What are you driving at? The rates for vaccinated versus
| not are a clear indicator that the vaccines helped. Hard
| to see any other way of interpreting that data.
|
| You are correct that, if I did, in fact, have an early
| case of covid, I cannot be sure that the vaccine helped
| me with the later case. So, as far as that goes, my
| "evidence" is anecdotal at best and can't be taken fully
| as proof of anything.
|
| You will have a hard time arguing against vaccines with
| the aggregate evidence above, though.
| ifyoubuildit wrote:
| Sorry, let me clarify. I'm not trying to argue against
| vaccines.
|
| I entered the thread at
|
| > getting the vaccine basically led to people not getting
| symptomatic covid. Folks got what they thought of as a
| bad cold.
|
| I asked for clarity there because it didn't line up with
| what I understood to be symptomatic covid (have covid and
| have any symptoms). It sounded like you were really
| saying the vaccine led to people in your circle not
| having severe covid.
|
| I believe it is true that the vaccine reduced instances
| of severe covid. But my point in this thread is that most
| people already weren't going to have severe covid (based
| on ifr rates pre vaccine, though hospitalization data
| would be more useful here).
|
| In other words, "The rates for vaccinated versus not are
| a clear indicator that the vaccines helped" is true as I
| understand it, and not something I'm arguing against. It
| does not contradict "most cases of covid were not severe,
| vaccinated or not" though.
|
| Does that make sense?
| taeric wrote:
| Ah, fair. I am definitely playing loose in that area.
|
| For specifics in my circle, I really only have my
| immediate family and some coworkers as direct evidence.
| Among those, I don't know anyone that got symptomatic
| anything if they were vaccinated. We had plenty of colds,
| but only tested positive during a time when that wasn't
| going through the family. (We only tested due to kid's
| having contacts that got covid.)
|
| So, to that end, only vaccinated person in the family
| that ever had symptoms was me. And, as I said, it was
| super quick. Such that I can't say for sure the kids
| didn't have symptoms overnight that we just didn't see.
|
| Pulling it back to "most cases overall were not severe,"
| is tough, though. If that is somehow indicative that the
| vaccines didn't help me, that would also imply that they
| didn't help the population at large. And the data just
| doesn't agree with that.
|
| Is that where you are asking? Or did I avoid the
| question?
| ifyoubuildit wrote:
| I'm just trying to make the point that the vaccines
| helped at a population level (going from .06% to .0006%
| or whatever IFR is real numbers when you're talking about
| the whole world), but I think people overestimate the
| impact it had on them individually.
|
| And it's easy to see why they would! Given the
| environment at the time (daily press conferences, scary
| news articles, demonization of the unvaccinated,
| mandates) I think it's easy to believe that the vaccine
| saved you from a death sentence if you get vaccinated and
| then have an easy case.
|
| It's easy to not notice that in a room of 1667 infected
| unvaccinated 30 year olds (I don't know how old you are,
| just using that as an example), maybe over a thousand of
| them would have had a similar case that you did, and only
| one of them would have died.
| taeric wrote:
| On that, I think I'm in violent agreement with you. In
| particular, I actually was annoyed with how much stress
| folks put pre-teens through regarding vaccination. I had
| friends that were terrified of doing anything with their
| toddlers before they got vaccinated, despite the odds
| still being higher for the parents with a vaccine than
| the kids without. It was truly baffling.
|
| For my part, I suspect it helped me. Childhood asthma and
| general obesity being what they are. I was almost
| certainly in elevated risks for my age group. To your
| point, my age group was still moderate risks, all told.
| oldgradstudent wrote:
| > Folks got what they thought of as a bad cold.
|
| Symptomatic Covid is simply a positive Covid test + any
| flu-like symptoms. What you're describing is symptomatic
| Covid. This is what was measured and reported in the
| trial.
|
| You might say that's not very interesting because it
| doesn't measure anything of importance. You would be
| right. That is exactly what critics say before the
| trials.
|
| https://www.bmj.com/content/371/bmj.m4037
|
| The trials were never meant to test whether there would
| be any mortality benefit, any reduction in serious
| disease, any reduction in hospitalization, or any effect
| on infection or transmission.
|
| What they did meausre, turned out to be inconsistent with
| reality, though.
| taeric wrote:
| Symptomatic covid for the first round was far worse than
| that. Hell, even for later rounds, symptomatic covid was
| pretty intense. Again, I had family that neglected
| getting the vaccine and almost died with that decision.
| We know of many people that neglected the vaccine and did
| die.
|
| So, if the concern is you are upset a miracle vaccine
| didn't get developed, you're losing my interest quick.
| Anyone that got upset that you had a few symptoms is
| overblowing concerns to a non-useful degree.
| oldgradstudent wrote:
| > So, if the concern is you are upset a miracle vaccine
| didn't get developed, you're losing my interest quick.
|
| No, the concern is not that a miracle vaccine didn't get
| developed. The trial measured and reported whether people
| who got vaccinated got those "few symptoms" vs people who
| got the placebo. It claimed 95% efficacy in preventing
| those "few sysmptoms", but it did not do so in reality.
|
| The concern is that the trial results do not agree with
| reality. That means that something is wrong in either the
| design or execution of the trial. It's a bug in the
| trial, and a bug should be debugged.
| LorenPechtel wrote:
| I don't think it's a bug in the trial, but rather
| evolution at work.
|
| The vaccine worked pretty well against the Wuhan strain,
| but Covid breeds variants like it was a rabbit. The
| farther from the strain coded into the vaccine the less
| effective the vaccine is. It still seems to be pretty
| good at reducing the severity, though--the unvaccinated
| are dying at a far higher rate than the vaccinated.
| oldgradstudent wrote:
| > I don't think it's a bug in the trial, but rather
| evolution at work.
|
| That's called an ad-hoc hypothesis.
|
| _In science and philosophy, an ad hoc hypothesis is a
| hypothesis added to a theory in order to save it from
| being falsified._
|
| https://en.wikipedia.org/wiki/Ad_hoc_hypothesis
|
| It could be true, but it is not enough to assert it, it
| has to be proven.
| linuxftw wrote:
| Even in the initial data released by the FDA, Pfizer
| didn't test all patients for COVID during the trial. In
| fact, they didn't even test all 'suspected' cases during
| the trial. In fact, there were more 'suspected but not
| verified' cases among the test group than the control.
|
| It was junk science from top to bottom, and this assumes
| any science was conducted at all. According to a whistle
| blower, the science was fraudulent.
| taeric wrote:
| But it is easy to see that the "few symptoms" in the
| trial patients easily proxied to "safer outcomes" in the
| wild? I seriously cannot underline hard enough that folks
| that didn't get the vaccine put their lives in extreme
| risk for basically no reason.
|
| Seriously, the numbers were drastic for vaccinated versus
| not in hospitalizations alone. To push the narrative that
| they were wrong to get vaccines out just feels misguided.
|
| If you are pushing that we should continue to get better
| at trials and reporting? I agree with that. Any harder
| push there, though, feels nitpicking at best, and I don't
| see the direction you are hoping to go.
| YPCrumble wrote:
| At first I didn't believe this could be true but the link
| is here:
| https://www.nejm.org/doi/full/10.1056/nejmoa2034577
|
| It seems that Pfizer basically rammed the vaccine through
| because it prevented covid with 95% efficacy for a couple
| months and made the case that it was too effective to
| continue the study.
|
| We now know that antibodies from Pfizer decrease
| significantly and quickly after a couple months, so it
| seems very likely that Pfizer knew this as well and decided
| that after two months was the perfect time to conclude
| their study and start selling vaccines.
| SV_BubbleTime wrote:
| >selling vaccines
|
| To the governments, who have no money but from tax
| payers.
|
| This I think was the most egregious marketing lie in
| recent history. That everyone who was jumping up and down
| for their vaccine was under an impression it was free.
|
| The same people rabbling all day about "transfer of
| wealth" saw no issue there.
|
| I don't have a stance on covid or vaccines that is
| terribly unique. But that most people overlooked the
| massive economic reasons to move in the direction that it
| did, annoys me.
| LorenPechtel wrote:
| No. The trial was intended to conclude when they had
| sufficient data to get an acceptable confidence interval.
| It was to be periodically reviewed to see how it was
| faring against that yardstick.
|
| They ended up tossing one of the intermediate reviews
| because it was overtaken by events--the objective was
| met, spend the time on analyzing that data rather than
| the now-irrelevant intermediate review.
|
| The test did nothing towards establishing how long the
| protection lasted--they can't have rushed it through
| based on that being short because they had no measurement
| of it then.
|
| You simply can't measure time effects in medicine other
| than by observing them. If you want to know what
| protection is like after a year you have to wait a year
| and then measure it. (This is also why we saw repeated
| changes to the shelf life of the vaccine--the vaccine
| makers simply didn't have the time to establish what the
| true shelf life was and thus could only claim what they
| had measured. Note that this is pervasive in medicine--
| stored properly most drugs are effective far beyond the
| stated shelf life. It's just the manufacturers have no
| reason to spend the money to certify this.)
|
| And in blaming Pfizer you show your bias--why did every
| vaccine maker do the same thing at the same time??
|
| If anything I'll blame Pfizer for making a weak vaccine.
| Moderna chose to go with a higher dose that appears to
| provide slightly more protection at the cost of more side
| effects at the time.
| sonicshadow wrote:
| You have just made enemies with the entire sheep-mind of the
| Democratic Party. Prepare to be cancelled - please share your
| driver's license #, LinkedIn profile, and most convenient
| checking account number so we may donate your savings to the
| Joe Biden Life Support/re-election campaign.
| [deleted]
| Thoeu388 wrote:
| [flagged]
| CrampusDestrus wrote:
| it's 2023, we have the means to cheaply record and store audio
| and video evidence for basically any medical experiment. we can
| record every patient reaction and opinion without relying on the
| reasearchers' hearsay. we also have the means to store and
| distribute all the binary/textual raw data gathered throughout
| the experiments.
|
| maybe as an intermediate step we could make available all the
| recordings to the peer reviewers and only offer the raw
| experimental data bundled in the paper publicly? maybe in the
| future we can have 1TB studies without breaking a sweat? maybe
| all the money we give to publishers can be spent on servers to
| archive all the primary data so at least we aren't simply filling
| the pockets of MBAs?
| jamesdwilson wrote:
| how do you prevent cherry picking?
| CrampusDestrus wrote:
| that's such a vague question
|
| for example, if you have 50 partecipants but only provide the
| multimedia evidence for 20 of them your study should be
| thrown out the window
| dolni wrote:
| What is preventing someone from having 200 participants,
| but saying they only had 100 participants, and then only
| providing evidence for 100 participants?
| throw9away6 wrote:
| You have to declare the study population before you
| request the next round of funding. Thereby fixing the
| problem.
| CrampusDestrus wrote:
| The researchers will have to have gone through some kind
| of third party agency to get the partecipants. This
| agency should be queried to see the number they report
| dolni wrote:
| How does this agency determine who can meaningfully
| participate in the study? Are they going to have the
| expertise to make that determination for _every_ study
| that could possibly be conducted?
|
| What is the difference (to a layperson) between cherry-
| picking participants and rejecting participants because
| they do not meet your study's criteria?
|
| Who funds this agency?
|
| Do the members of the reviewing agency have their own
| biases, and might those biases tarnish the reputation of
| a study that is actually well-conducted? (hint: this
| already happens in journals)
| linuxftw wrote:
| Nothing. Pfizer did this, albeit wasn't 50% of the
| enrollment.
|
| You can see that here:
| https://kirschsubstack.com/p/pfizer-phase-3-clinical-
| trial-f...
| lesuorac wrote:
| No Pfizer did not do that.
|
| Pfizer had a trial with 21823 people in the Expirement
| group and 21823 people in the Placebo group. In the
| results they excluded data for 1790 from the Expirement
| group and 1585 Placebo group. However, _crucially_ Pfizer
| never claimed there were only 100 people in the study
| after starting with 200; you know pfizer excluded 3375
| people because _Pfizer told you_.
| raverbashing wrote:
| "But we have 20, the other 30 volunteers were removed by
| unrelated reasons" (and it is common to exclude volunteers
| from experiments)
| jonlucc wrote:
| Every clinical trial paper I've read has a discussion of
| inclusion and exclusion criteria. I think for the trial
| to be registered, it has to include this information.
| CrampusDestrus wrote:
| "unrelated reasons" should not be an acceptable excuse
| though. either state the reason or it goes into the
| trash. and if they were private reasons you can still
| contact them to confirm they left on their own volition
| and/or they didn't finish the trial without getting into
| specifics. you only need one lie to suspect the whole
| thing
| ttpphd wrote:
| LOL looks like someone has not had to get data collection
| protocols through IRB approval...
| CrampusDestrus wrote:
| if researchers are so untrustworthy then what's your
| solution?
| Balgair wrote:
| For a 'quick' overview into the _mess_ that is IRBs, this
| book review is a good starting place:
| https://astralcodexten.substack.com/p/book-review-from-
| overs...
|
| TL;DR: IRBs are a mess, hyper-individualized, and the problem
| ain't getting any better any time soon.
| WastingMyTime89 wrote:
| > maybe as an intermediate step we could make available all the
| recordings to the peer reviewers
|
| The issue is clearly not the amount of data available to peer
| reviewers considering it's already easy to detect major flaws
| in a quarter of published peer reviewed research. The issue is
| that peer reviewers do a shoddy job which should surprise no
| one having ever published peer reviewed research.
|
| And to be fair why should they do better? It's generally
| unpaid, it's poorly paid when it is paid and it's not
| particularly well considered.
| sonicshadow wrote:
| Sounds like a YC idea?
| pbmonster wrote:
| These are medical trials. How do you preserve the patients
| privacy in all of this?
|
| Or do subjects need to wave all their doctor-patient privacy
| rights before joining any trial?
| CrampusDestrus wrote:
| If we discover that we can't trust researchers then what else
| are we left with? Doctor-patient privacy works if the doctor
| is truthful in their reporting
| Slaminerag wrote:
| It's generally permitted to share de-identified patient data.
| As long as you're not sharing patients names, medical record
| numbers, birthdays, and a couple of other fields, you should
| be fine.
| jononomo wrote:
| I don't think it makes sense to publish the results of clinical
| trials and other scientific experiments until after they have
| been independently replicated.
___________________________________________________________________
(page generated 2023-07-18 23:01 UTC)