[HN Gopher] "Amateur" programmer fought cancer with 50 Nvidia Ge...
___________________________________________________________________
"Amateur" programmer fought cancer with 50 Nvidia Geforce 1080Ti
Author : coolwulf
Score : 667 points
Date : 2022-05-20 15:54 UTC (7 hours ago)
(HTM) web link (howardchen.substack.com)
(TXT) w3m dump (howardchen.substack.com)
| ChicagoBoy11 wrote:
| What a service to society. Hats off to you to using your skills
| like that!
| themantalope wrote:
| This is very cool work. I'm a radiologist, I also work on
| developing ML/AI based systems for cancer detection and
| characterization. Literally just took a break for a few minutes
| from creating some labels and saw this as the top HN post!
|
| I think in some ways making the model available online can be
| good, but in other ways could be harmful too. Very complicated
| topic.
|
| Gong Xi coolwulf, Zhu Ni Ji Xu Cheng Gong .
| DantesKite wrote:
| I've always felt the "could be harmful" was a rationalization
| by radiologists worried about their job security since it's
| easily mitigated with a warning and multiple tests.
|
| And especially because in the future, most radiology work will
| be done by software. It's just a matter of whether it's 10
| years or 100 years from now.
| kashunstva wrote:
| > I've always felt the "could be harmful" was a
| rationalization by radiologists worried about their job
| security
|
| Surely concern for the well-being of the patient figures in
| there somewhere...
|
| Or imagine this: A liver lesion is incidentally discovered on
| your abdominal CT performed for unrelated reasons. Its
| radiographic characteristics are equivocal. Additional
| imaging studies fail to completely exclude the possibility of
| a liver malignancy. You undergo a biopsy. But the biopsy is
| complicated by hemorrhage. Surgery is required. You develop a
| post-operative nosocomial infection. etc. etc.
|
| To the extent that risks along this chain of unfortunate
| events is known, yes, warnings could put some of the
| quantified decision making in the patient's hands. Well,
| except for the rampant innumeracy in the general
| population...
| anamax wrote:
| Perhaps a better example is prostate biopsies. They have a
| significant risk of producing incontinence or impotence.
| themantalope wrote:
| This is a real scenario that happens regularly.
| [deleted]
| ska wrote:
| > "could be harmful" was a rationalization
|
| This topic (breast screening) is a good example due to the
| sheer scale. If you increase the work-up rate by even a
| smallish amount, you are statistically pretty much guaranteed
| to kill people who did not have the disease. How this
| balances about gain (i.e. save other lives) is not obvious.
| Figuring out the "right" way to do this is real work.
| lhl wrote:
| Also a good example because there's actually been recent
| push back on mammography because the risks may outweigh the
| rewards (especially in younger age brackets). More testing
| does not automatically equal better outcomes. Here's one
| summary from the Nordic Cochrane Center (for those that
| don't know Cochrane is one of the gold standard
| organizations for evaluating the quality of biomedical
| research via systematic reviews
| https://en.wikipedia.org/wiki/Cochrane_(organisation) ):
|
| Gotzsche PC. Mammography screening is harmful and should be
| abandoned. J R Soc Med. 2015;108(9):341-345.
| doi:10.1177/0141076815602452
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4582264/
| themantalope wrote:
| Extremely important, and is the most important question to
| ask in any screening scenario.
|
| It's not the case the more screening is always better.
| There are plenty of screening regimes that you've never
| heard about because the trial data didn't support it.
| quasarj wrote:
| Yeah, I've read all the arguments about harm, and I just
| don't buy it. I'd get a CT scan every year if I could
| convince someone to sign off on it. There is absolutely no
| substitute for knowing.
| selectodude wrote:
| You'd dramatically raise your risk of cancer from x-ray
| exposure for almost zero clinical benefit.
| quasarj wrote:
| Yeah, that is the only concern. I guess whole-body MR
| would be fine too. I'd be willing to lay in the tube for
| 20 hours a year to know that I'm at least structurally
| normal still.
| ska wrote:
| > Yeah, that is the only concern.
|
| It really isn't. Your chances of negative consequence for
| unnecessary follow up procedures would rapidly become
| significant.
| KennyBlanken wrote:
| > Yeah, I've read all the arguments about harm, and I just
| don't buy it.
|
| "I have no medical background whatsoever but I do not buy
| undisputed medical science."
|
| > I'd get a CT scan every year if I could convince someone
| to sign off on it
|
| 20mSv of unnecessary ionizing radiation exposure _a year_ ,
| what could possibly go wrong? That whole-body CT scan is
| equivalent to getting _at least_ EIGHTY chest x-rays, and
| ten times what a uranium miner receives in a year, and well
| within the range where an increased risk of cancer is
| noticeable in epidemiological data.
|
| Case and point why this tool should not be generally
| available.
| kashunstva wrote:
| > I'd get a CT scan every year
|
| You would choose annual body CT even if outcomes-based
| evidence showed no benefit?
|
| Even among those at highest risk for lung carcinoma,
| studies of annual screening chest CT do not uniformly show
| improved disease-specific survival as compared to an
| unscreened but otherwise matched cohort.
| themantalope wrote:
| https://www.nejm.org/doi/full/10.1056/nejmoa1102873
|
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3762603/
|
| What I found mind blowing when I first read these studies
| (particularly the second link) is that the positive-
| predictive value of the CT screen was only 3.8% for lung
| nodules that were 4mm or larger in size. Basically just
| means we find all kinds of lung abnormalities all the
| time that "could be cancer" but aren't.
|
| Keep in mind, that the screening criteria was people over
| age 65 who had a 30+ pack-year smoking history (1 pack-
| year = smoking 1 pack of cigarettes per day in one year),
| and had to have smoked within the last 15 years.
|
| I can also tell you from doing CT lung biopsies that it's
| not a procedure to take lightly. Lot of important
| structures in the chest.
| quasarj wrote:
| Absolutely. Those studies show no improved survival
| across cohorts. Not for individuals. My only chance of
| living is to know. That's worth it for me.
|
| I mean, plus I have extreme health anxiety, and I've had
| doctors say "nah you're too young, we don't need to test
| for that" and later it turned out I _did_ have kidney
| failure at 35.
| themantalope wrote:
| Until you've had an unnecessary biopsy that kills you :)
| martincmartin wrote:
| How could it be harmful? Is it just because of errors, i.e.
| false negative means person won't talk to doctor and won't be
| caught early; false positive means needless worry, maybe
| tarnish more general medical industry? Or something else?
| DantesKite wrote:
| Well this radiologist could lose their job to an algorithm.
| That's pretty harmful to the industry.
| [deleted]
| themantalope wrote:
| I'm not worried. If things are that different, the rest of
| the economy will be too. The job will also evolve over
| time, like any other profession that changes with new
| technology.
|
| It's our job (as radiologists and engineers) to shape that
| in a way that benefits patients.
| tryptophan wrote:
| If AI automates radiology, it will automate the rest of
| the world first.
| themantalope wrote:
| Yeah, those are part of the issue. For example on the
| webpage, there is no discussion about the false
| negative/positive rate of the system, and no comparison or
| link to an academic paper that compares the performance of
| the system to a trained breast radiologist.
|
| Does a patient take a negative exam as "I'm good!" and forego
| actually seeing a doctor, or having the exam read by a
| radiologist?
|
| A positive result is also a challenging situation. I've
| talked to patients before after diagnosing them, and as you
| could expect most people are shocked, scared or they want to
| know what they should do next.
|
| In breast radiology specifically (at least in the US) there
| is a well defined reporting lexicon and classification
| system. From what I can tell, the system does not use the
| lexicon or BI-RADS classification.
| https://www.acr.org/-/media/ACR/Files/RADS/BI-RADS/BIRADS-
| Re...
|
| Breast cancer can be very subtle, and can be hard to figure
| out, especially if a patient has had surgery or other benign
| abnormalities. https://radiopaedia.org/cases/development-of-
| dcis?case_id=de...
|
| Diagnostic and screening images also undergo rigorous QA for
| the entire system. Does the model have some kind of QA built
| in?
|
| There is a just a small disclaimer at the top that says "we
| will not store your data on the server. please do not worry
| about privacy issues". Not exactly a formal or legal binding
| agreement.
|
| In fact, you can't even connect over https.
| http://mammo.neuralrad.com:5300/upload
|
| These are just a few things that come to mind off the top of
| my head. Like I said before, I do think that what coolwulf
| works on and is trying to do is good, and in the long run can
| help doctors better characterize findings and help patients.
| But like anything in health care, there are a lot of edge
| cases and side effects that you have to think about. The
| stakes are also very high.
| evanmoran wrote:
| I think care with messaging positive/negative rate is the
| right reasoning for a productized version of this
| positioned as an alternative to human radiologists. The
| difference here is that we know the user has a scan of
| their body and we can be pretty sure they couldn't have
| gotten the scan without doctors already being involved.
|
| So we don't have to worry as much about the "Does a patient
| take a negative exam as 'I'm good!'", because in the normal
| flow of medicine that gets this scan the doctor will check
| the scan for them either way. The website probably could
| have better explanations of this, but most likely negatives
| will already be double checked independently by doctors,
| and positives will most likely be handled correctly by
| bringing it to their current current attention to double
| check the scan or run more tests.
| generalizations wrote:
| > Does a patient take a negative exam as "I'm good!" and
| forego actually seeing a doctor, or having the exam read by
| a radiologist?
|
| I wonder how the false positive/false negative rates for
| this tool compare to that of a trained human. At some point
| we may reach parity, in which case what's the harm of
| trusting the automated result?
| themantalope wrote:
| Secondary follow up.
|
| I don't think that a human reader in the loop will be
| going away for a long time. Sure, they can be at parity,
| but does a combined read have better results? This is an
| active research question.
|
| I think what will happen is that over time, human
| interpreters and AI systems will "co-evolve" in a sense,
| where people will pick up on where models are wrong and
| also learn how to use models to understand their own
| blind spots. These are also active research topics that
| are in their infancy.
| themantalope wrote:
| https://www.rsna.org/news/2022/march/AI-Potential-in-
| Breast-...
|
| EDIT: To be clear, I believe that this is a different
| group of authors from coolwulf. Pubmed link:
|
| https://pubmed.ncbi.nlm.nih.gov/35348377/
| quasarj wrote:
| To be fair, how is a patient going to get a mammogram or
| xray without having seen a doctor?
| dekhn wrote:
| The consequences are false positives and negatives in cancer
| detection are well known (easily googled), and are actually
| extremely important when looking at cancer treatment in large
| populations. You nailed the most important ones.
| ska wrote:
| >Very complicated topic.
|
| This is very true. Data availability (and moreso, label
| availability) is the biggest barrier to improvement here I
| suspect [ thanks for labeling!]. Access being another. Using a
| public site to bootstrap that could do very interesting things.
| On the other hand, public access to a poorly RA/QA'd algorithm
| could also cause more trouble than help, easily.
| themantalope wrote:
| Label availability is a big problem. There are some academic
| collaborations for some diseases which is making this better,
| but datasets in general are minuscule compared to what is
| available for more general computer vision applications.
|
| I am hopeful though. Few-shot learning and self supervision
| are very active questions right now and there are a lot of
| papers in the medical AI field that are getting published on
| these topics.
|
| I'm personally interested in liver cancer, which does not
| have large, well-curated and shared database of cases.
|
| Sharing data gets tricky, especially in the US. Labels I'm
| working with and creating (at least at this point) are for my
| own research, which won't be shared publicly any time soon.
| ska wrote:
| > n the medical AI field that are getting published on
| these topics.
|
| Most of what I've seen isn't very promising. The energy in
| these research areas are because it would be so much
| cheaper than the "right" way, far more than because of the
| likelihood of success. And also perversely, because it's
| hard for the academic researchers to get enough data to do
| other studies :) NB: I'm not saying there is nothing useful
| coming out of the learning literature in last few years,
| just that it a) isn't a silver bullet and b) is often being
| misapplied in these areas anyway.
|
| Label quality and availability isn't the only big problem
| though. Many data sets exhibit problematic sampling bias,
| as well as being order(s) of magnitude too small, because
| of the way they are gathered and how access is granted.
| themantalope wrote:
| Problem is though for most of these diseases there just
| aren't the number of samples available, period, to do it
| the "right way". HCC for example, has around 50K new
| cases/year in the US. Even if every single case went into
| a repository with perfect labels, would still take a long
| time to collect that info. Not to mention you need either
| a radiologist (4 year of medical school + 6 years of
| post-school training) or a very skilled and experienced
| technician to label the data.
|
| Not to mention imaging protocols are not standardized,
| and the imaging technology is also evolving so scans we
| do today may not be "correct" or standard in 5-10 years.
| ska wrote:
| Definitely diseases have different challenges. Breast
| cancer screening being a notable outlier as far as data
| availabilty. For some diseases ML is probably always
| going to be problematic although may help in diagnostics
| mostly by helping get rid of other possibilities.
|
| I suspect we have similar overall views of the problem,
| but I'm pretty strongly in camp that recent advances in
| ML/AI are mostly really driven by data & label
| availability, not algorithmic advances - this colors
| where I think the wins to be had in medical ML can happen
| most easily. Either way though the non-technical barriers
| seem clearly higher than the technical ones still.
| themantalope wrote:
| Second follow up.
|
| Results are not impressive until they are :)
|
| It's certainly not a solved problem, and it's easy to
| have a pessimistic view now but I'm generally bullish on
| where things will be 10 years from now.
| ska wrote:
| > Results are not impressive until they are :)
|
| True! I certainly wouldn't discourage anyone from trying.
|
| On the other hand, I think it would be a huge mistake to
| trust that fancy learning approaches will solve
| everything so we shouldn't try and improve access and
| labling. Getting better there is still by far the most
| high probability of successful impact, imo.
| tfgg wrote:
| What peer review or regulatory approval process has this been
| through? Seems pretty irresponsible -- there are many notorious
| pitfalls encountered with ML for medical imaging. You shouldn't
| play with people's lives.
| bsder wrote:
| In addition, these kinds of things will still miss _lobular_ (
| "normal" cancers are ductal) breast cancers as they don't form
| lumps.
|
| 15% of the women with breast cancer are waiting for a non-
| invasize diagnostic imaging system that can see their cancer.
| The only thing that can see these is an MRI with gadolinium.
| And that gadolinium contrast causes issues in about 1 in 1000
| women, so it can't be used as a general screen.
| mromanuk wrote:
| This is like taking your temperature at home, are you making a
| diagnostic yourself? Not quite. But you can know some symptoms
| and take action (going to the doctor) maybe with less anxiety
|
| edit: grammar
| KennyBlanken wrote:
| A thermometer mostly tells you if you have an infection, and
| how close you are to your body temperature becoming a
| medically urgent or life-threatening situation.
|
| Not even remotely the same thing.
| Thaxll wrote:
| What's the chance of taking the wrong temperature though?
| mordae wrote:
| Pretty high, actually.
| tfgg wrote:
| Thermometers are well understood, simple devices, and there
| are other complementary checks (e.g. does my forehead feel
| hot) if they fail.
|
| This project might lead to people thinking they're in the
| clear and not seek appropriate medical treatment, or be
| overtreated due to an error. You should always talk to a
| qualified doctor if you're concerned about your health, and
| not use projects like these for decision making.
| arbitrandomuser wrote:
| I don't see how a passive scan like this can be harmful .
| Ofcourse if it does show a positive one should confirm
| further with a biopsy or other standard confirmation
| diagnostic. As for false negatives... If you feel something
| is wrong you should get it checked thoroughly anyway .
| dekhn wrote:
| you literally just said the scan has no value. THe point
| of a scan like this is to have absurdly low false
| negative and positive rates so that it's actionable.
| Unactionable medical diagnostics are worthless and just
| cost (money, fuel, time).
| tfgg wrote:
| So you should ignore it and see a doctor regardless.
| Arisaka1 wrote:
| In an ideal world, that would be the case. However,
| people aren't 100% rational agents motivated by logic.
|
| My aunt was diagnosed with Multiple Sclerosis which led
| her to lose her eyesight from her left eye, because she
| refused to get a thorough checkup by a professional, and
| even today whenever I tell her that I visited the doctor
| for an issue she has... not very good words to say
| (something something "you are a chicken, you're
| hypochodriac etc). And I'm saying this without
| entertaining the probability of her visiting a
| professional who just happened to be in a bad day, which
| could potentialy lead to a wrong diagnosis.
|
| I've been hunting down my own diagnosis for symptoms
| everyone seems to tell me that aren't serious (nail
| discoloration and a 24/7 headache that feels like my
| arteries are pulsing, which lasts for YEARS, cold
| fingertips during the winter, and more).
|
| I get what you're trying to say and I agree with the
| general message. However, more checkpoints to catch a
| potential failure are good. For example, if someone were
| to make a take-at-home device which scans nailfold
| capillaries (no reason for something like that to exist)
| I'd get that in a heartbeat. I'm being actively ignored
| by every medical professional that I have visited, and if
| I'm not ignored they give the minimum amount of
| attention, kind of like "well, it's not like you're dying
| so who cares?"
|
| Fair note: I'm from Europe.
| arbitrandomuser wrote:
| If you don't feel good yes you should see a doctor
| regardless. Let's say you get a scan every year . If the
| scan is able to detect something earlier than a
| radiologist is able to identify i think it's worth paying
| attention to.
| kelnos wrote:
| I agree with you, but what's most important is the impression
| that the average person who uses it will have. And I don't
| think most people would think of this as like "taking your
| temperature at home". I think most people who might upload
| their x-ray scans would take this a lot more seriously.
|
| A false positive could create a lot of anxiety and emotional
| distress, and the patient might need to go to 2, 3, or 4
| other doctors to get second opinions before they feel
| comfortable that they really don't have cancer.
|
| A false negative could be even worse. A patient might think
| "oh, the official-looking online thing said I don't have
| cancer, so I don't need to wait for or consider a human
| radiologist's results", and not believe they need treatment.
|
| I think it's very important that people understand that --
| until more research is done -- this is still not a substitute
| for having a human look at your x-rays. If we could be
| reasonably sure that everyone (or at least a very large
| majority) understood this when using this tool, then I think
| people would have far fewer objections. But I don't think
| that's the case.
|
| Having said that, I think it's safe to assume that this tool
| has saved lives, so it's almost certainly been a net positive
| for people.
| Mikhail_K wrote:
| I don't understand why this comment is downvoted. Automated
| screening of radiological images by means of neural net is an
| extensively researched topic. Ten years ago there had been
| predictions that such automated screening will displace the
| radiologists, but that clearly did not happen.
|
| For instance, this article is silent on false positive/false
| negative rates of the software. There is no comparison with
| other research. It reads like a corporate press release
| promoting a product.
| hackernewds wrote:
| The quotations around "amateur" should be moved to "fought".
| Nonetheless, it's encouraging that this level of research can
| be executed at home, however the strict burden of proof
| required should still be maintained.
| zmmmmm wrote:
| Well, he was director of R&D for a medical imaging company
| and worked directly with academia. So I think its
| appropriate to contextualise the "amateur" because his work
| looked much more amateur than it really was.
| zmmmmm wrote:
| In this case I feel better about it because there is a natural
| limitation in that most people doing this will only have the
| scan because they are getting tested through a real clinical
| process. So effectively they are getting "standard of care"
| treatment implicitly, and all this does is accelerate their
| response to true positives. The worst case scenario is a false
| positive gives them a lot of anxiety / costs them money through
| trying to accelerate their real diagnosis only to find it isn't
| real.
| dekhn wrote:
| This is an incredibly important point. Medical research must be
| taken seriously and I see many problems with the processes
| being applied here.
|
| (for those who care- I'm a published ml biologist who works for
| a pharma that develops human health products. Having worked in
| this area for some time, I often see people who have no real
| idea of how the medical establishment works, or how diagnostics
| are marketed/sold/regulated. Overconfidence by naive
| individuals can have massive negative outcomes.
| plandis wrote:
| If your decision making process is a negative result tells you
| nothing and a positive result warrants immediate follow up,
| what's the risk here? I'm assuming doctors recommending that
| women get checked for breast cancer is the primary breast
| cancer is tested and diagnosed which presumably wouldn't change
| because someone make a website.
| quasarj wrote:
| Ahh yes, why would we want to give poor people a potential
| route to improve their health? it would definitely be more
| ethical to let them die.
| 725686 wrote:
| For an evolutionary perspective of cancer and possible new ways
| to treat them, I recommend the new book "The cheating cell" by
| Athena Aktipis.
| light_hue_1 wrote:
| As an AI/ML researcher who publishes in this area regularly, I
| will be using this as a case study for AI ethics classes. That
| this is allowed to go on is shocking.
|
| > In 2018, a programmer named "coolwulf" started a thread about a
| website he had made. Users just need to upload their X-ray
| images, then they can let AI to carry out their own fast
| diagnosis of breast cancer disease.
|
| Literally the worst fears that we have as a community is that
| people will recklessly apply ML to things like cancer screening
| on open websites and cause countless deaths, bankruptcies,
| needless procedures, etc. How many people went to this website,
| uploaded images, were told were ok and didn't follow up? How many
| were told they have cancer and insisted on procedures they didn't
| need?
|
| The website is totally unaccountable. Totally unregulated.
| Totally without any of the most basic ethical standards in
| medicine. Without even the most basic human rights for patients.
| This is frankly disgusting.
|
| In the US this would have been shut down by the FDA immediately.
|
| We should not be celebrating this unethical "science" that
| doesn't meet even the most basic of scientific standards or
| ethical standards.
|
| I can't believe this is getting upvoted here.
| karolist wrote:
| I share your sentiment, people are focusing on successes too
| much but not scrutinise what potential outcomes false negatives
| in software like this can have.
| light_hue_1 wrote:
| This is unimaginably worse than what Theranos did!
| acidoverride wrote:
| > a case study for AI ethics classes
|
| What is unethical about this citizen science project? What is
| ethical about keeping it only for yourself, and not sharing it
| with the world?
|
| You are saying you have the expertise to build a similar
| product, but releasing it would mean the worst fear of your
| community?
|
| > people will recklessly apply ML
|
| What are the indications that this is a reckless application of
| ML?
|
| > How many people went to this website, uploaded images, were
| told were ok and didn't follow up?
|
| Common sense dictates exactly zero. Their follow up was taking
| _their_ images and getting an automated second opinion. Either
| a doctor already deemed them OK, or a doctor deemed them not
| OK, in which case, they would not rely on a second opinion, to
| think they are suddenly OK.
|
| > How many were told they have cancer and insisted on
| procedures they didn't need?
|
| Again, exactly zero. The app returns probabilities not binary
| diagnostics. No hospital would do anti-cancer procedures on a
| patient without cancer, even when they insist, because some
| website, friend, or religious leader told them so.
|
| > The website is totally unaccountable.
|
| Good. Or make the good-faith open-source project accountable
| and liable? That would simply mean shutting it down. No more
| diagnostics help for low-expertise hospitals: not good at all.
|
| > Totally without any of the most basic ethical standards in
| medicine.
|
| List a basic ethical standard in medicine which this project
| runs afoul of.
|
| > basic human rights for patients
|
| What right is that? The right not to upload _your_ images to a
| site of _your_ choosing? I thought human rights include self-
| determination, and keeping possession of _your_ imaging to do
| however _you_ see fit.
|
| > In the US this would have been shut down by the FDA
| immediately.
|
| But is that a good, ethical thing? Or simply that red tape and
| authority in US does not allow for such projects?
|
| > We should not be celebrating this unethical "science" that
| doesn't meet even the most basic of scientific standards or
| ethical standards.
|
| You should not talk about ethics or science, when you did not
| do even a proper evaluation of the work of a fellow scientist.
|
| > I can't believe this is getting upvoted here.
|
| Awaiting your work on cancer research and ML. Post it here. If
| devoid of ethical issues, and strongly scientific, it will also
| be upvoted and celebrated. Or is your major contribution going
| to be a snipe at someone who actually contributed?
| [deleted]
| endisneigh wrote:
| Should the internet also be shut down because people get false
| conclusions from WebMD, Reddit, Twitter, Google search results,
| etc?
| coolwulf wrote:
| Thank you for your reply. On the site, it's cleared marked and
| noted this is not for diagnosis.
|
| "This tool is only to provide you with the awareness of breast
| mammogram, not for diagnosis."
| analyte123 wrote:
| The words "found malicious mass", disclaimer or not, could be
| considered a diagnosis. You can probably say "anomaly", or
| _maybe_ even "classified as malicious mass". With you having
| connections in the US, it is probably worth talking to an
| expert if you want to keep this online.
|
| Also, 50 GPUs seems like more than necessary!
| markus92 wrote:
| It is, you can easily do this one one GPU, but it will a
| bit longer. Doubt it'll be prohibitively so.
| 6gvONxR4sf7o wrote:
| A disclaimer that it isn't for diagnosis isn't generally
| useful when everything else says the tool does diagnosis.
| markus92 wrote:
| Unfortunately, it's just a matter of time before someone gets
| severe complications for whatever outcome your program gets
| wrong. You have a false negative? Big problem, you just gave
| someone worse treatment options (if lucky) or led to someone
| being diagnosed too late and potentially getting metastasized
| breast cancer. Those bone metastases are quite painful, you
| know.
|
| False positives are even worse, because they are far far far
| more likely to happen in practice. Imagine your program
| telling someone it has a malignant mass (VERY bad wording,
| only the pathologist can say something is malignant). I speak
| from experience that this WILL lead to the patient going to
| the doctor, and the doctor, seeing these very strong words,
| might want to take a biopsy to confirm the malignancy. These
| are painful procedures that can and will go wrong, eventually
| (complication rate is ~1%, which is not a whole lot on its
| own, but is unacceptable on a known healthy population). If
| lucky, the biopsy can be done stereotactical, but if unlucky
| it'll have to be done MRI-guided. You just cost society a few
| thousand dollars/euros/yuan. And that's if everything goes
| right, worst case it'll be hospital admission due to
| complications, like an infection.
|
| Your blog post says you are trying to fight cancer, which is
| a noble cause. If the tool is not for diagnostical purposes,
| it's not doing a whole lot in fighting cancer as it is just a
| play thing then. At the moment, it's more like hindering
| cancer by taking resources from people who need them and
| giving them to people who don't need them.
|
| Source: researcher in AI for breast cancer
| screening/diagnosis.
| skybrian wrote:
| Is there any possible way that a test that is not entirely
| reliable can be used for screening? Like, couldn't they
| have a radiologist look at the photo to confirm the result?
| markus92 wrote:
| Yeah you could do that and it's happening in practice.
| Many countries implement double reading where two
| radiologists look at it. In the USA, single reading +
| computer-aided diagnosis is quite common using
| commercial, FDA-approved software.
| tasuki wrote:
| Of course, this is going to harm some people. Is it
| possible the number of people it helps is significantly
| larger than the number of people it harms? If it were so,
| would you not consider that a reasonable tradeoff?
|
| A slight digression: I find in certain countries (cough
| cough US), everyone is free to give advice/opinion on
| anything, except for medical/legal matters, which are
| considered sacred and so no one is free to give any
| advice/opinion on those. I saw a person scolded by HR for
| literally saying "make sure you stay well hydrated". This
| is madness! Medical professionals should _not_ have a
| monopoly on advice concerning hydration.
| dekhn wrote:
| The proper response to being scolded by HR for saying
| that is to ask what the person's username is, so you can
| speak to their manager.
|
| Then, when you speak to the manager, ask them not to have
| their report telling people not to do things that HR
| doesn't have authority over (coworkers discussing health
| topics like that are not within the scope of HR).
| dekhn wrote:
| You know, if people who teach ML classes at universities
| point out to you that what you're doing is going to be taught
| in classes as an example of "what not to do and how not to do
| it", you should probably immediately stop what you are doing
| and get opinions from experts in the field.
|
| I will repeat what others are say: this is irresponible due
| to naivete and could be harmful to people. Please consult
| with experts on how to proceed.
| IG_Semmelweiss wrote:
| and this is why healthcare is the #1 source of bankruptcy in
| the US.
|
| Some people believe that every single person must have
| Mercedes-Benz type of care in the US.
|
| They cannot fathom that some of the plebs (do they even exist
| for them?) may want to make their own independent healthcare
| choices, and are willing to accept the risk ... (or can only
| afford!) a Suzuki.
| jseliger wrote:
| _and this is why healthcare is the #1 source of bankruptcy in
| the US._
|
| May not be true: https://www.washingtonpost.com/blogs/post-
| partisan/wp/2018/0...
|
| The regulatory burden on medical treatments is far too high,
| however, and we should have the right to try:
| https://fee.org/archive/topics/Right%20to%20Try
| ska wrote:
| It's an interesting subject, with a long history; I think many of
| the biggest challenges are not technical.
|
| The first commercially available AI/ML approach to breast cancer
| screening was available (US) in the late 90s. There have been
| many iterations and some improvements since, none of which really
| knock it out of the park but most clinical radiologists see the
| value. Perhaps the more interesting question then is why are
| people getting value out of uploading their own scans, i.e. why
| does their standard care path not already include this?
| coolwulf wrote:
| The reason I made this project 100% free and available to the
| general public is to help patients, especially in the remote
| area who has limited access to experienced radiologists for
| diagnosis, to at least get a second opinion on their mammogram.
| And I think this has certain value and this is why I'm doing
| this project.
|
| Hao
| hn_throwaway_99 wrote:
| Great project and really cool to see this.
|
| Question I have for you is that one of the biggest problems
| with cancer diagnoses is false positives: "Yes, there is
| something on your scan, we're not sure what it is, so we'll
| biopsy it." Biopsy is not a 0-risk procedure, and it can
| cause a lot of worry and pain, so it's not something to be
| taken lightly. Also, there are many cases of "OK, it's
| probably cancer, but the cure may be worse than the disease."
| This is the classic problem with detecting prostate cancer at
| an advanced age - it's very likely/probable something else
| will kill you before the cancer does.
|
| How does your software deal with this issue? I'd be worried
| if, as you put it, people in a remote area with limited
| access to experienced radiologists, were given access to this
| and it came back with "Fairly decent chance of cancer" - what
| do they do then?
| coolwulf wrote:
| Great question. Definitely sensitivity / specificity
| balance is a crucial topic in AI-assisted diagnosis. I have
| to admit this model and website for mammography was done in
| 2018 and might not be the leading solution out there. At
| the moment, if I want to improve the results of my earlier
| work, I will add additional stage of radiomics model to do
| false positive reduction, and in the mean time lower the
| threshold in the first model to increase sensitivity. By
| doing this, combining the feature extractions of deep
| learning and the past 20 years of knowledge in medical
| imaging using Radiomics might give better performance in
| terms of sensitivity / specificity.
|
| Hao
| areoform wrote:
| How can we help? Is there some way for us to contact you
| regarding this?
| dekhn wrote:
| You're operating in a really serious area of medicine and
| I encourage you to takes the comments about false
| positives (and false negatives) more seriously. It's not
| really just a matter of making adjustments to a model; it
| has to be pervasive in the entire process of making
| reproducible modeels that are used for making decisiojns
| about humans.
| ska wrote:
| I think this comment is perhaps a bit uncharitable.
|
| Even if you have done everything you are supposed to do
| in the process, at the end of the day you are looking at
| ROC curves or equivalent and trying to understand
| sensitivity vs. specificity trade offs, and often you
| have some sort of (indirect) parameters than can move to
| different points of that tradeoff.
|
| This is quite critical in deployment, if you are
| screening you usually want something different than in
| diagnosis; as alluded to elsewhere if you raise the work-
| up rate too much you definitely risk killing more people
| from biopsy complications than you help with higher
| sensitivity (it's more complicated than that in practice)
| dekhn wrote:
| deploying ML medical diagnostics is like everything else
| in ML: the ML part is 1% of a much larger "thing", which
| involves business, legal, and many other concerns far
| beyond the data analytics.
|
| Nothing abotu what I'm saying is uncharitable- in a
| sense, it's charitable because I'm helping warn a person
| who is going down a dangerous path to consult more with
| experts in this field.
| ska wrote:
| The charitable response would have been to assume they
| _do_ have some understanding the broader context, and
| perhaps raise specific concerns or points of interest.
|
| What you did was assume that they were ignorant in
| potentially dangerous ways, and assert they should do
| something different.
|
| I don't think a reading of the comments/responses (at
| least at your time of posting) really supported that
| assumption, especially considering the limitations of the
| medium. Hence my reply to you, while also detailing the
| trade offs a tiny bit.
| dekhn wrote:
| I read the entire article and I didn't see anything in
| there that would convince me this author is anything
| other than an amateur programmer (I can't parse the
| section about bruker- is the "amateur programmer" also a
| director at Bruker who ddevelops medical devices full-
| time"?).
|
| Please be assured that I put a fair amount of thought
| into this - for example, I used to do due diligence for a
| VC firm evaluating proposals like this all the time and
| we had to reject most of them because the founders didn't
| understand the basic rules of deploying medical
| technology in highly regulated environments like the US.
|
| Based on my interactions with the author in the various
| parts this post, I continue to conclude this individual
| is lacking core knowledge and wisdom required to execute
| a project like this successfully at scale.
| ska wrote:
| The article was pretty fluffy, but it was about them not
| by them. If article was accurate about the role at
| Siemens they have for certain been exposed to RA/QA work
| and know what a DHF is, etc.
|
| Anyway and least at they time you posted (since then
| there were more interaction) I didn't find the same
| information nearly enough to dismiss their competence out
| of hand.
| dekhn wrote:
| I went back and dwetermined that the article was wrong.
| He wasn't a "director" at Bruker, he was a "detector
| imaging scientist". There's nothing about Siemens.
|
| What I didn't totally grasp from the article is this is a
| company https://www.reddit.com/r/MedicalPhysics/comments/
| t5u2c9/intr...
|
| So this isn't an amateur programmer, it's a person who
| got a phd in nuclear engineering and radiological
| sciences, was a scientist at bruker, has some experience
| with health systems, and then became a serial entrepeneur
| with a small company that has some funding. BTW, people
| who have the job title "Director" are normally fairly
| senior (old), as well.
| coolwulf wrote:
| The website is cleared marked as a breast health
| awareness tool and not for diagnosis. We are not doing
| any decision making for humans. However I would like to
| point out there are quite some FDA approved mammo AI
| products in the market at the moment.
| dekhn wrote:
| So, you think if you put up a website, people won't use
| it for diagnosis anyway?
|
| yes, I know mammo AI products are on the market- those
| that were approved followed a collection of regulations
| that I think you are not.
| endisneigh wrote:
| Even if people use it for diagnosis, so what? Take the
| info and if it's positive to confirm with doctor, if it's
| negative but you have reservations go see doctor.
|
| It doesn't really change anything.
| dekhn wrote:
| so you're saying the test is worthless in terms of
| actionability? Why woudl you want to take it then?
| endisneigh wrote:
| It's not that it's worthless, it's simply another data
| point to give to a professional.
| billiam wrote:
| You will save even more lives if you over-communicate
| about the need to use your awareness tools (with better
| false positive reduction) to drive more effective
| diagnosis.
| [deleted]
| ska wrote:
| Inconsistent care is a really good point. I wasn't trying to
| be negative - hope it didn't come across that way. I was
| hoping to point out that systemic issues in health care
| management, at least in a lot of countries, seems to be more
| of a problem that tech for things like this.
|
| Out of curiosity, how are you handling the data access and
| labelling issues here? I suspect that's the key issue that
| has limited the performance of the commercial offerings
| (hardly limited to this problem or this space).
|
| OTOH in terms of real impact, properly leveraging a more
| modestly successful algorithm will probably help more people
| than getting a few more %. With the (strong) caveat that in a
| space like this you really have to look at work-up rate and
| balance risks.
| monkeydust wrote:
| There is history of breast cancer in my family and anything
| that can be done to improve outcomes has to be highly
| commended. I did however have the same question and this
|
| > to at least get a second opinion on their mammogram.
|
| for me makes a lot of sense, even in developed countries
| where you get a result but want extra assurances.
|
| It would be interesting to know (assuming you have the data,
| even anecdotally) if the second opinions using this
| overturned professional ones and from those how many were
| corrected an original false negative mistake.
| ska wrote:
| Not speaking for coolwulf obviously but I can perhaps shed
| some light.
|
| Screening breast mammo has an occurrence rate problme.
| Something like less than 10 in 1000 studies will require
| further review; this means in practice as a radiologist you
| look at a lot of negative films before seeing a TP. It also
| means a typical read is done _fast_. Seconds-to-small
| minutes.
|
| This results in a couple of things. Reader variability
| based on experience/throughput, and false negatives. There
| were some double reader studies that caught something like
| 15% (going from memory here) of FN - but nobody can affort
| to have two radiologists read everything.
|
| So the profession is already conceptually used to the idea
| of using an algorithm as a "second read" and reconsidering.
| Typically this won't "overturn" anything here but rather
| say 'hey have another look', but the decision to proceed or
| not is still the clinicians. Having a positive from the
| algorithm makes them review carefully, but you have to
| watch the FP rate here or nobody would get anything else
| done.
|
| I have heard of health systems using algorithms as a first
| pass too (i.e. radiologist only see films that have had a
| postive in a tuned-to-be-senstive version), but that has
| it's own set of issues.
| Hitton wrote:
| Is this machine translated? Some parts don't make much sense.
| coolwulf wrote:
| Recently a Chinese media interviewed me and I talked about a few
| side projects I have done in the past. I talked about the
| Neuralrad Mammo Screening project and Neuralrad multiple brain
| mets SRS platform. More awareness on radiation therapy to the
| general public will greatly help the community and we believe
| Stereotactic Radiosurgery (SRS) will eventually replace majority
| of the whole brain radiation therapy (WBRT) in the next five
| years.
|
| Here is the link to the original article:
| https://www.toutiao.com/article/7094940100450107935/
| abfan1127 wrote:
| Google Translate link - https://www-toutiao-
| com.translate.goog/article/7094940100450...
| gregsadetsky wrote:
| Thanks -- and here's the site as well --
| http://mammo.neuralrad.com:5300/upload
| Simon_O_Rourke wrote:
| Thank you for all you've done for people, it's amazing and
| inspiring!
| rob_c wrote:
| Fantastic work dude. On behalf of anyone who might one day
| benefit thanks and congrats.
| jabrams2003 wrote:
| What's the best way to contact you? I've been fighting brain
| cancer for 7 years and work closely with a group of neuro-
| oncologists, researchers, non-profits, and investors in the
| space.
|
| I'd love to chat.
| throwaway122385 wrote:
| coolwulf wrote:
| Feel free to send me emails: coolwulf@gmail.com
| koprulusector wrote:
| > Recently a Chinese media interviewed me and I talked about a
| few side projects I have done in the past.
|
| I apologize if this has been asked and answered before, but do
| you speak Mandarin, or was the interview in English?
|
| Asking out of curiosity if it's the former, and if so, how
| difficult was it to learn whilst also working on this and other
| things? And are there any resources or tips you might share
| that you found helpful?
| qzw wrote:
| According to the article, Coolwulf went to Nanjing University
| for undergrad, so pretty safe to assume he would do an
| interview with Chinese media in Mandarin. And since he likely
| grew up in China, it was probably very easy for him to learn
| Mandarin indeed!
| jacquesm wrote:
| Super effort. I understand your reluctance to accept funding
| but if you ever change your mind on that be sure to publish it
| here on HN. If giving you more tools means more progress in
| this domain without the usual red tape then I'm all for giving
| you as much of a push as possible.
| iaw wrote:
| You're clearly well accomplished in multiple areas. How do
| approach learning something new?
| hehepran wrote:
| Sir, you are super cool.
| Billsen wrote:
| Nice job!
| onetimeusename wrote:
| Where did you learn to program on distributed Nvidia GPUs? The
| article implied you were self taught and learning to do this is
| quite challenging for various reasons.
|
| Not least, Nvidia's documentation is not the best resource to
| learn from. This seems like quite a lot of work to understand
| ML and write custom CUDA code to get this to work. Do you have
| any insight about how you taught yourself these things and what
| tools you use?
| MaximumYComb wrote:
| I'm not your OP but I learnt all these things at univeristy
| during my BCompSci. Understanding ML algorithms came down to
| a lot of math / statistics units. I learnt about parallel
| computing during a dedicated unit called "Distributed and
| Parallel Computing"
| jacquesm wrote:
| Not the OP but I taught myself in a couple of weeks picking
| apart some of the sample CUDA code and reading some of the
| (excellent) pdfs on the architecture of the Nvidia range. At
| the time the GTX285 was hot stuff, the same code runs
| unchanged on a 1080ti and I would expect it to continue to
| work on even more modern incarnations. CUDA is pretty good as
| a platform to build on if you understand the basic idea
| behind the engine. And ML on CUDA vs ML in C or some other
| language is typically a matter of shuttling the data and the
| results back and forth between main memory and the card as
| well as implementing the most time consuming portions of the
| algorithm you are using in a custom kernel, you can usually
| get 50% or so of the theoretical maximum speed with
| relatively little effort. Getting to full speed is going to
| be a lot harder, but then you could of course also add
| another card (or another three) get get an instant boost.
|
| Usually you would - nowadays at least - use someone else's
| optimized kernel + ML library but if you wanted to roll your
| own that's doable.
| sylware wrote:
| javascript only link. Any compatible link with noscript/basic
| (x)html browsers?
| jjeaff wrote:
| And I would like the content sent to me in Morse code via
| telegraph.
| daniel-cussen wrote:
| Unless it's more expensive than existing treatments the medical
| industry will close the circles around you excluding you.
|
| That's why not one startup has hacked healthcare in America,
| not one. No breakaway successes making pharma cheaper. Like
| those incubators in Bangladesh, for premature babies not
| startups that is, those did OK. Some pill startups yes, but
| again that's an expensivification of medicine. If you can make
| medicine more expensive, they welcome you in!
|
| Jim Clark tried this, he was on a roll after Silicon Graphics
| and Netscape. Huge roll about as strong as Elon Musk as a
| serial entrepreneur. Then he targeted healthcare and couldn't
| do shit, just couldn't get anything to happen. He literally
| talked about getting "rid of all the assholes" by which he
| meant insurance and doctors and hospitals and middlemen and
| pharma and all the other "assholes" of that nature in his own
| words, but leave "only one asshole in the middle--us
| [paraphrased]." It's in a book. That book also talks about guys
| going on airplanes and chasing goats off cliffs, saying "Some
| people do this."
|
| Well the real structure of medicine isn't designed around the
| human body, it's designed around cornering the market. Market
| dominance. So of course it has this immune system against cost
| reduction and efficiencies--efficiencies especially--and you do
| know it lobbies, don't you? And can bribe the FDA like the
| Sacklers did? Or lobby the FDA, and then bribe underneath so
| when people see favoritism they think it's the over-the-counter
| placebo causing a placebo effect without suspecting an
| additional more potent under-the-table dosage of money. In case
| the administration has built up a tolerance to the over-the-
| counter stuff.
| quickthrower2 wrote:
| What about the NHS in the UK? They should be more aligned to
| wanting cheaper cancer diagnosis and also anything to help
| people.
|
| If America has dysfunctional healthcare there is still the
| rest of the world. Which might be good for Americans
| eventually as the tech will come across one way or another.
| londons_explore wrote:
| I believe it's too small a market.
|
| Modern treatments are hugely expensive to develop, and they
| also tend to be very specific (ie. only 1 in 5,000 people
| might get the exact right kind of brain tumour for your
| treatment to be an option).
|
| With only about 1 million all-cause deaths per year in the
| UK, that means your treatment for a specific terminal brain
| tumour might only have 200 patients per year.
|
| The 'we just saved 4 hours of clinician time by using fancy
| AI' just isn't worth it if it only saves 800 hours of
| clinician time per year, yet costs millions to develop.
|
| The fix is to roll this out somewhere with more patients
| (eg. China) and where trials are cheaper (ie. China).
| anamax wrote:
| > If America has dysfunctional healthcare there is still
| the rest of the world. Which might be good for Americans
| eventually as the tech will come across one way or another.
|
| Which raises the question - why does the vast majority of
| healthcare tech development come from the US? (I included
| "development" to get around the out-sourcing of testing to
| China.)
| danielheath wrote:
| It's hard to understand how much more available capital
| is in the states. Getting 100 million together is
| drastically easier there than anywhere else.
|
| Similar to silicon valley's tech scene, you get migration
| of people who want to work at a level where that sort of
| capital is required.
| idiotsecant wrote:
| >That's why not one startup has hacked healthcare in America,
| not one.
|
| This post makes a lot of points, but in general I think they
| boil down to the above statement : the belief that large,
| complex systems are just run by stupid and/or malicious
| people and that a sufficiently clever 'hack' will fix all the
| problems. I think that is an attitude that is common on HN,
| but wrong.
|
| Most big problems are not technology problems, they are
| People problems with a capital P. Technology problems can be
| fixed with 'one simple hack they don't want you to know
| about!!!' People problems are complex and messy and cause and
| effect can be intermingled vertically and horizontally with
| other seemingly unrelated factors as well as temporally with
| things that don't even exist yet or used to exist but don't
| anymore!
|
| The way we fix these messy, complex People problems is by
| respecting that they are real problems, that the people
| acting on those systems are (mostly) reasonable people just
| doing what reasonable people do, and slogging through
| solutions a day at a time with the oldest technology around -
| political power. These problems resolve if you can get enough
| people to agree they need solved.
| throwaway122385 wrote:
| lhl wrote:
| While the US health care system as a whole is mess, I don't
| think it's intractable if you can attack it from the right
| directions. GoodRX did with pharmaceuticals, and Mark Cuban
| seems to be doing a good job with CostPlus Drugs. (Valisure
| is also pretty interesting, doing in-house validation of
| generics).
|
| Marty Makary wrote a book a couple years back, The Price We
| Pay on the wicked knot of a problem that is US health care.
| For those mildly interested, there was a Peter Attia Podcast
| inteview a while back that covers the gist of it:
| https://peterattiamd.com/martymakary/
|
| For those that want to get some color from some of the
| biggest problems from a clinician/practioner's perpective, I
| found some of these podcast episodes to be pretty great/eye
| opening: https://zdoggmd.com/podcasts/
| rg111 wrote:
| Hi. Some great projects. What's more commendable is your
| dedication towards your projects and seeing them through to
| end- to the point that they are actually useful. This is what I
| truly admire.
|
| I have a question for you. What is the tech stack that you use?
|
| And if it is not too much: What resources did you use to learn
| Deep Learning?
| dclowd9901 wrote:
| As a "professional" programmer, I'm humbled by your
| accomplishments. I really must find ways to contribute more to
| the world. It seems there's a lot of opportunities in AI to do
| it.
| llaolleh wrote:
| Your story was inspirational. It's really cool to run this
| project to help others without expecting any payment.
| ska wrote:
| WBRT is pretty brutal. Am I right in thinking you are focusing
| on multiple site treatment/palliative treatment of metastatic
| presentations? High site count also or sticking to say < 5?
| coolwulf wrote:
| Exactly, I'm working on a workflow platform for multiple
| brain metastasis stereotactic radiosurgery. This will greatly
| benefit patients with more than 5 BMs.
| samstave wrote:
| ELI5, please.
| ska wrote:
| Metastatic disease is when a cancer spreads to multiple
| locations. This can make it
| difficult/impossible/impractical to treat effectively,
| especially surgically.
|
| Whole brain radiotherapy works by killing everything a
| little bit in the hopes that the tumors die first (e.g.
| like chemo). There are good reasonswhy this tends to
| mostly-sort-of be true, but getting the balance right is
| hard and too much dose will definitely cause other
| problems.
|
| SRS is a way of targeting radiation directly to locations
| to kill cells, with less effective dose (hence damage) to
| other parts of the brain.
|
| It's all pretty harsh stuff, and you can die from the
| necrotic tissue caused by it, also.
|
| Often with this kind of disease you know you aren't going
| to cure someone, but you can get rid of symptoms and make
| people more comfortable (palliative care).
| ska wrote:
| Cool stuff! I've done some work in adjacent areas - there
| are huge challenges (not only technical) but great to feel
| you are making an impact.
| pen2l wrote:
| Oh, it's you!
|
| What a beacon of light and inspiration you are. Thanks for your
| work.
|
| That said, I welcome you to publish your work so it can become
| even better after a formalized peer-review process.
| coolwulf wrote:
| For the multiple brain mets SRS project, we will be
| presenting at this year's AAPM annual meeting.
| FpUser wrote:
| I am not a religious man at all but God Bless you. You are an
| amazing human being and a source of inspiration.
| YeGoblynQueenne wrote:
| >> Furthermore, the accuracy of tumor identification has reached
| 90%.
|
| How is this accuracy calculated? Further in the article it is
| noted that there is no patient data saved by the project:
|
| >> He said that he's not sure actually how many people have used
| it because the data is not saved on the server due to patient
| privacy concerns. But during that time, he received a lot of
| thank-you emails from patients, many of them from China.
|
| Considering user privacy is laudable in my opinion, but I'm still
| curious to know how accuracy is known.
| Iv wrote:
| Probably based on a test set from the original dataset.
| YeGoblynQueenne wrote:
| The expression "has reached" makes it sound more like an
| extrinsinc evaluation process has taken place (i.e. not on a
| test set).
| ghoomketu wrote:
| Reading this article only makes me realise how crypto industry
| has crippled the progress independent researchers like this would
| badly need gpus for AI.
|
| How many really useful, cool and meaningful projects are stuck
| because such authors can't find or afford gpus - as they are
| being used to calculate meaningless hashes instead :/
| westcort wrote:
| My key takeaways:
|
| * The free AI breast cancer detection website took coolwulf about
| three months of spare time, sometime he had to sleep in his
| office to get things done, before the site finally went live in
| 2018
|
| * The website also gained a lot of attention from the industry,
| during which many domestic and foreign medical institutions, such
| as Fudan University Hospital, expressed their gratitude to him by
| email and were willing to provide financial and technical support
|
| * Afterwards, he and Weiguo Lu, now a tenured professor at
| University of Texas Southwest Medical Center, founded two
| software companies targeting the radiotherapy and started working
| on product development for cancer radiotherapy and artificial
| intelligence technologies
|
| * But in 2022, he returned with an even more important "brain
| cancer project"
|
| * coolwulf (Hao Jiang) (right) He told us that his parents are
| not medical professionals, and his interest in programming was
| fostered from a young age
|
| * A reliable AI for tumor detection can enable a large number of
| patients who cannot seek adequate medical diagnosis in time to
| know the condition earlier or provide a secondary opinion
|
| * He said that he's not sure actually how many people have used
| it because the data is not saved on the server due to patient
| privacy concerns
|
| Link to the technology: http://mammo.neuralrad.com:5300/
| dekhn wrote:
| that's an unadorned http link. Really?
| ramraj07 wrote:
| > A reliable AI for tumor detection can enable a large number
| of patients who cannot seek adequate medical diagnosis in time
| to know the condition earlier or provide a secondary opinion
|
| Citation Required?
| oversocialized wrote:
| green-salt wrote:
| Amazing work. I'm glad this is going to help so many people.
| latchkey wrote:
| ETH will soon move from PoW to PoS (let's not debate the timeline
| or if it is a good idea). This will put about 32 million GPUs
| worth of compute and millions of CPUs searching for something
| else to do (or just flood the market with used equipment).
|
| I have been searching, for years, for alternative workloads for
| these GPUs beyond just PoW mining and password cracking. Many of
| them are on systems with tiny cpus, little memory, little disk,
| little networking so the options are heavily limited.
| AI/ML/Rendering/Gaming actually make bad use cases.
|
| If anyone has thoughts on this, I'd appreciate hearing them. Let
| it all die is certainly an option, but it also seems just as
| wasteful as keeping it going. Maybe we can find a better use
| case, like somehow curing cancer...
| redisman wrote:
| Why isn't there a folding coin? Productive mining and you
| reward the new protein folds or whatever
| bufferoverflow wrote:
| There's Folding Coin (FLDC)
|
| https://foldingcoin.net/
|
| But it never took off.
| fancyfredbot wrote:
| Because folding is too hard to verify. For a Blockchain to
| work it needs to be very easy to prove that the miner has
| actually done the work. With a folding problem it's very hard
| to prove the answer a miner gives is actually a solution to
| the problem rather than a quick guess. It's a shame!
| quickthrower2 wrote:
| Just make it centralized (like most cryptocurrency
| effectively is anyway). You would still need some checks
| and balances sure.
|
| Make a token. Altruists buy the token to fund the "miners".
| They may also make a profit but they buy knowing most
| likely they wont and it is for a good cause.
| latchkey wrote:
| That is what gridcoin is. Unfortunately, it doesn't work.
| PartiallyTyped wrote:
| What about federated learning to deal with the little memory
| issue?
| VHRanger wrote:
| Proof of Stake has been 6-18months away for 5 years now.
|
| As far as I'm concerned it'll release along with Star Citizen
| dekhn wrote:
| folding@home has been doing this for 20+ years. They already
| did all the smart research and tech development. Just use that
| until somebody comes up with a workable DrugDiscoveryAtHome or
| CureCancerAtHome.
| latchkey wrote:
| This is a very dismissive answer which seems odd coming from
| someone with a lot of karma. Running GPUs at scale isn't easy
| or cheap.
|
| There is no incentive to run this other than good feelings.
| Unfortunately, that isn't enough in the business world to
| spend millions on cap/opx.
|
| What I'm looking for is incentivized options. Even better if
| they come from a web3 situation where a business can operate
| without having actual customers.
|
| "Mining", but with not such "wasteful" work.
| dekhn wrote:
| only good feelings? f@H has made fundamental contributions
| to our understanding of biophysics of folding proteins!
| latchkey wrote:
| Sorry, it was just a way to say "not financially
| incentivized".
| passivate wrote:
| I am aware of domain applications from F@H, but not core
| science work. What are those fundamental contributions?
| dekhn wrote:
| We conclusively demonstrated that kinetic models of
| folding are critical to do better drug discovery against
| GPCRs and other target classes.
|
| Or did you mean something more fundamental, like "the
| biophysics of protein folding is primarily determined by
| entropic-driven hydrophobic collapse, not enthalpic
| contributions from hydrogen bonding?"
| quickthrower2 wrote:
| The dark side is you need a ponzi scheme to fund it. When
| that ponzi collapses the work stops. Good feelings are
| better IMO!
| latchkey wrote:
| It is odd that you think of crypto as a ponzi while
| promoting a referral network in your HN profile.
| dekhn wrote:
| yeah, everybody knows crypto is a _pyramid_ scheme, ponzi
| schemes are specifically about a type of coupon fraud.
| bryans wrote:
| It's ironic to be declaring the GP as dismissive while
| you're flippantly dismissing the work of everyone involved
| with F@H. You're conveniently ignoring the incentive of
| expanding our understanding of biology, which has very real
| applications and results[1] that benefit the entire world
| for the rest of human existence, instead of benefiting a
| single participant in the short term.
|
| [1] https://www.hpcwire.com/2020/10/14/how-foldinghome-
| identifie...
| latchkey wrote:
| What? I'm not dismissing their work.
|
| I'm just saying there is no direct financial incentive
| for 32 million GPUs to move over to it. If there was,
| they'd be on that instead of ETH.
| bryans wrote:
| You literally said "no incentive [...] other than good
| feelings." Regardless, even adding the words "direct
| financial" is still incorrect. For one of many examples,
| insurance companies investing in F@H infrastructure would
| be beneficiaries, albeit via savings and not revenue.
| latchkey wrote:
| I literally clarified my statement since you came to an
| incorrect conclusion.
|
| "direct financial" is correct when you're giving examples
| that are not direct.
| bryans wrote:
| You're moving the goal posts, on top of changing the
| definition of "direct" to fit your false narrative. First
| you claim there was no incentive at all, then there was
| no "direct" incentive, and now direct apparently means
| "immediate" instead of linear.
|
| If a company invests in research specifically because the
| fruits of that research will reduce expenses, that is
| direct financial benefit.
| daniel-cussen wrote:
| Oh you know what an alternative use is? Oaths. Works with old
| ASICs as well...well I think. So you take a document, like this
| comment, you append a nonce (you'll see) and you hash it until
| you get a lot of zeroes in the front. Same as bitcoin, but
| you're not hashing the bitcoin protocol. Then, you know the
| document has been sworn, as a cryptographic oath, to that
| extent. Nonce: 38943
| latchkey wrote:
| This is effectively timestamp.com
|
| There is also very little incentive structure.
| daniel-cussen wrote:
| So then go to https://geraintluff.github.io/sha256/
|
| > oath = "Oh you know what an alternative use is? Oaths.
| Works with old ASICs as well...well I think. So you take a
| document, like this comment, you append a nonce (you'll see)
| and you hash it until you get a lot of zeroes in the front.
| Same as bitcoin, but you're not hashing the bitcoin protocol.
| Then, you know the document has been sworn, as a
| cryptographic oath, to that extent. Nonce: 38943"
|
| > sha256(oath) 00009ea9ab415b7f60cd43571c159d1bf1e01de4bae6a7
| 06ec9053ceb94d385c
|
| Note the leading 0's. That's no timestamp, that's an oath.
|
| In reply to the sibling comment: no. I like timestamp.com,
| and in fact I could have never found out about it other than
| by talking about the oath concept, but this is not just
| including it in the blockchain. It's proving its value to the
| author to bother doing the work of getting a good nonce for
| it. Literally putting my money where my mouth is. And
| swearing an oath to that extent, I could cryptographically
| swear it more, with more work, or use a smaller less
| impressive nonce if I'm not as sure.
|
| And incentives? There is an incentive for me. At the same
| time it is effectively burning money, swearing by burning
| money. Took like seven seconds of compute, too. I had to wait
| human time for that. It's collateral, it's an oath. And it's
| an impediment to forgery, and in addition, an impediment to
| eg news sites telling different people different things. With
| oaths they have to tell everybody the same thing.
| ephbit wrote:
| Could you kindly point me/others to some info about this
| oath concept?
|
| I mostly get pointed to Oauth stuff when searching for
| "oath sha256 nonce".
| daniel-cussen wrote:
| There's nothing out there beyond what you've read in my
| two comments above yours.
|
| Well I suppose I can still point you to it:
| https://news.ycombinator.com/item?id=31451260
|
| You know what? I'll make a post about it and link it
| here.
| netsharc wrote:
| Crypto is <valley-girl>literally</valley-girl> stopping us from
| finding the cure for cancer!
| mwt wrote:
| Folding@home would love to take a swing at a sliver of that
| compute
| jseliger wrote:
| It's peculiar to me that Folding@home never managed to get
| GPUs working for MacOS, given the platform's popularity:
| https://foldingforum.org/viewtopic.php?f=83&t=32895
|
| As a consequence, it seems not worth installing on MacOS:
| https://stats.foldingathome.org/os
| dekhn wrote:
| It's not worth implementing for MacOS because it woulnd't
| increase the overall folding rate very much given the
| relative weakness of the Mac GPUs compared to the existing
| f@h fleet.
| PragmaticPulp wrote:
| > ETH will soon move from PoW to PoS (let's not debate the
| timeline or if it is a good idea). This will put about 32
| million GPUs worth of compute and millions of CPUs searching
| for something else to do (or just flood the market with used
| equipment).
|
| Crypto markets crashing together could do this, but ETH's
| switch isn't going to do much for old cards.
|
| Checking https://whattomine.com/ shows that ETH mining isn't
| even in the top 5 most profitable things to mine with a 1080Ti
| right now. The miners looking to squeeze every bit of
| profitability out of old hardware switched away from ETH a long
| time ago.
| RealityVoid wrote:
| I hear people say this, but I am absolutely certain their
| assumptions are wrong.
|
| 1) The sum total of rewards is fixed for POW 2) Introducing
| extra hashing power will increase the difficulties of these
| mining ops up to the profitability equilibrium point.
|
| After the overall "free" hashing power increases to a point,
| GPU's will start flooding the market at dumping prices.
|
| It will be incredibly rad!
| latchkey wrote:
| Wrong. Why?
|
| 1) The cards have already paid for themselves. They are 100%
| ROI positive and even at the current low amounts very
| profitable. Regardless of what W2M says, ETH is still the top
| most profitable coin. Large miners don't sell immediately,
| they wait for the market to go up or the option against their
| ETH holdings.
|
| 2) ETH doesn't require latest hardware because the algo is
| memory hard, which means that the bottle neck is in the
| memory controller, not in the speed of the GPU chip itself.
| https://www.vijaypradeep.com/blog/2017-04-28-ethereums-
| memor...
|
| 3) The actual consumable is electricity price, which really
| hasn't changed much in the last few years for large miners
| who have contracts.
| zamadatix wrote:
| There are plenty of good uses, projects like BOINC have been
| using GPUs for good for over a decade. The problem is the
| incentive system disappears, it's a lot easier to get people to
| run 32 million GPUs when it makes them money instead of costs
| them money.
| latchkey wrote:
| Well, exactly. It has to be incentivized.
| whoisterencelee wrote:
| Please please check gridcoin.world
| latchkey wrote:
| This seems like an incentivized BOINC built on top of an
| inflationary shitcoin. There is no utility in the coin
| itself and that is reflected in its price history.
|
| What we need is something that has utility... like run
| BOINC, earn tokens that can be used in the real world for
| something other than just dumping on the market.
| notfed wrote:
| What does "90% accuracy" mean? Is this before or after applying
| Bayes' theorem?
| 1-6 wrote:
| Goes to show that you don't need to be a long time programmer to
| have impact in most areas of society.
|
| Why is Python so good? It democratizes by lowering the bar to
| coding.
| redeyedtreefrog wrote:
| In the UK the NHS don't do screening for breast cancer for under
| 50s because it's believed that it would do more harm than good by
| leading to unnecessary treatment for cancers that would never
| have actually caused any harm, and even where no treatment is
| carried out it causes great distress. Though there are arguments
| that the age cut off is too high, and should be set at 40.
|
| The above is with regard to a well-funded and regulated screening
| program that presumably has much better precision/recall than
| this website. I wonder what the cut off age is for this website
| before the diagnoses cause more harm than good? 60? 70?
|
| This is getting lots of upvotes because it's confirmation bias
| for the large segment of HN readers who believe that problems
| would easily be solved by a small number of brilliant
| technologists, if only it weren't for governments and big
| organisations with all their rules and regulations.
| laingc wrote:
| A lot of people, including myself, don't believe that central
| health authorities have the right to make that call.
|
| Moreover, I personally don't have confidence in their ability
| to make those kinds of decisions, and I believe the abysmal
| performance of the NHS supports my view.
| Gatsky wrote:
| The NHS has finite resources. They have to decide if
| implementing a screening program is worthwhile or not, versus
| spending the money elsewhere. You can still go to your doctor
| and get a mammogram (or even a more useful test) if you have
| other reasons why this might be justified in your situation.
| They aren't 'banning' mammograms for young women.
| KennyBlanken wrote:
| It's not just a matter of finite resources.
|
| Any sort of treatment is invasive. Almost all form of
| medical treatment has side effects and risks.
|
| Finding out you have "cancer" is traumatic and extremely
| emotional, though breast cancer is one of the most
| survivable (in part because, well, everyone loves boobs.
| Prostate cancer, on the other hand...)
|
| Putting these tools in the hands of medical professionals
| is one thing. Putting them in the hands of the general
| public is beyond irresponsible.
|
| People physically assaulted doctors and nurses for not
| being given ivermectin; imagine how insufferable people
| will get when some website examined their mammogram and
| said they have cancer.
| ramraj07 wrote:
| I share your disdain for a central authority in making these
| judgement calls, but I have even less confidence in the
| majority of people who think they can solve everything with
| AI. Signed, a data scientist with a PhD in biomedical
| engineering.
| Waterluvian wrote:
| I think central authorities absolutely must make that call.
| Who else is going to decide how to dole out a scarce social
| resource? Americanizing healthcare is obviously not a good
| choice given how much worse it does overall by basically
| every measure (unless you're rich and don't give a toss about
| other people).
|
| I certainly agree that central authorities can be better. But
| that's kind of a truism.
|
| What alternative options do you have in mind? Admittedly I'm
| short on alternative ideas.
| webmobdev wrote:
| Thanks for the different perspective. What did you mean by
| "unnecessary treatment" though? If you have cancer, doesn't it
| need to be treated? Doesn't cancer anywhere always cause harm
| to the body?
| dekhn wrote:
| For many detectable tumors, the best answer is "wait and
| see", not "immediately remove". There are many reasons for
| this- surgery itself is risky, the tumor itself might not
| ever become harmful.
|
| See for example https://jamanetwork.com/journals/jamaoncology
| /fullarticle/27... for some more context.
| latortuga wrote:
| Breast cancer for example is diagnosed by increasing levels
| of invasiveness. First a mammogram, then possibly a 3D
| mammogram, then an ultrasound, then a biopsy. There are
| possibilities for false positives all along this path and
| increasing levels of possible complications when performing
| procedures. If a false positive gets to a biopsy and you get
| an infection from it, you would not have ever gotten that
| infection if they didn't start testing you so young. False
| positives are _very_ common with breast cancer screening.
| markdown wrote:
| Not to mention the fact that getting a biopsy can cause the
| cancer to spread all over the body where it might never
| have grown beyond its original position had it been left
| untouched.
| gregsadetsky wrote:
| 1) I just downloaded the "The Mammographic Image Analysis Society
| database of digital mammograms" [0] and ran it against the tool
| [1] image by image. Results below, code here [2]:
| true_pos 36 true_neg 207 false_pos 63 false_neg
| 16 total 322
|
| 2) How is it true when the site [1] says "We will not store your
| data on our server. Please don't worry about any privacy issues."
| when you can find all analyzed mammograms under the "static"
| directory?
|
| http://mammo.neuralrad.com:5300/static/mamo.jpg
|
| http://mammo.neuralrad.com:5300/static/mammo.jpg
|
| (trying file names at random)
|
| [0] https://www.repository.cam.ac.uk/handle/1810/250394
|
| [1] http://mammo.neuralrad.com:5300/upload
|
| [2] https://github.com/gregsadetsky/mias-check
| coolwulf wrote:
| Thank you for your efforts for validation and I appreciate
| that. There is a script running in the background to auto clean
| the files in static folder every day.
| dekhn wrote:
| You just admitted you _do_ store images.
|
| Also, you're serving up on http. Don't do that.
| [deleted]
| bitshaker wrote:
| This is impressive. Wonderful work to OP.
|
| I'm currently working with Digistain (S21) and we're using AI to
| predict breast (and eventually other) cancer recurrence.
|
| The tests are performed using infrared spectroscopy to measure
| protein synthesis and then fed into AI in order to make proper
| measurements and predictions.
|
| We've shown we're able to predict better than any other known
| method and are beginning our partnership and rollout to many
| hospitals around the world.
| ArtixFox wrote:
| did they win?
| transfire wrote:
| Sadly, this would be illegal in the USA and get shut down pretty
| quickly.
| giantg2 wrote:
| More like a patent holder would usurp all the work someone else
| did and make a fortune off of it after taking 5 years to get
| through the red tape.
| markus92 wrote:
| Na, this is all well-known work. This field is seriously big,
| lots of publications on it dating back to the early '90s.
| renewiltord wrote:
| unlessI'm wrong, he's in Michigan.
| charia wrote:
| OP is probably talking about the legality of American
| hospitals using this software in an official capacity like
| some Chinese hospitals seem to be doing.
|
| I'm completely unfamiliar, but it wouldn't surprise me if for
| diagnosing? software like this to be used in an official
| medical capacity in America it would need to go through some
| sort of particular vetting process because if it isn't it
| might leave hospitals who use it open to lawsuits.
| caycep wrote:
| that would be a potential YC idea. A company that enables
| smaller groups to do clinical research w/o needing an army
| of people to wade through the regulatory red tape. That
| also isn't in and of itself a giant predatory CRO type
| organization.
| dekhn wrote:
| that's literally what pharma is now- companies that exist
| to help smaller groups get their research through the
| clinical and approval process. It would be hard to buidl
| that level of expertise in a smaller company.
| codingdave wrote:
| What exactly is illegal about this? If you are thinking HIPAA
| laws, they don't apply when you are sharing your own medical
| information/images.
| dekhn wrote:
| This is a regulated industry. If you don't jump through the
| necessary hoops, the US government _will_ shut you down and
| there are many laws on the books they can use for this. There
| are many laws beyond HIPAA (which exists to make it easier to
| share data, not harder) which apply.
| codingdave wrote:
| We may be talking about different things - I was referring
| to writing the software and having people upload their own
| data to it. You guys must be referring to more formal usage
| in the healthcare industry.
| dekhn wrote:
| who operates the servers for the software? Each
| individual at home is going to have a machine that does
| inference on their own images?
| Flankk wrote:
| The FDA may or may not attempt to classify it as a medical
| device and then shut it down. Otherwise legal if it includes a
| disclaimer.
| pentium wrote:
| way cool, no need to label yourself amateur, software, hardware,
| radiology, and real impact. Hats off to you.
| [deleted]
| somethoughts wrote:
| Admittedly I just skimmed the article but I feel like the title
| should be more - "This "amateur" medical...". His primary
| expertise is more physics/CS/programmer related than it is human
| biology/medicine.
| jonplackett wrote:
| Is 90% correct rate considered good enough for this kind of use?
|
| Seems like 1/10 wrong would be bad, how does that compare with a
| doctor doing it?
| latortuga wrote:
| According to the American Cancer Society
|
| > About half of the women getting annual mammograms over a
| 10-year period will have a false-positive finding at some
| point.
| OJFord wrote:
| 'Amateur' oughtn't be scare-quoted because it's not a slur, many
| of the finest programmers were amateurs for many years before
| they were old enough to be given a job in the profession.
| ant6n wrote:
| If u used to be a paid software programmer and got a different
| job, but continued doing programming side projects without pay,
| are u an amateur or not?
| NHQ wrote:
| It literally means "for the love".
| Wohlf wrote:
| Yes, but you'd also be a former professional. Amateur also
| isn't meant to be a negative term, it just means you do it as
| a hobby rather than as a profession.
| jxramos wrote:
| I had an art teacher affectionately remind me the etymology for
| amateur
|
| > borrowed from French, going back to Middle French, "one who
| loves, lover," borrowed from Latin amator "lover, enthusiastic
| admirer, devotee," from amare "to have affection for, love, be
| in love, make love to" (of uncertain origin) + -tor-, -tor,
| agent suffix https://www.merriam-
| webster.com/dictionary/amateur#etymology...
|
| changes the feeling of it all when you get that context,
| someone who loves a subject pretty much--no qualifications
| skill wise or regarding depth but they love it and should
| presumably take things seriously to some degree as any lover
| would.
| gist wrote:
| Using 'amateur' (quoted or not) is click bait. It's an
| embellishment to the rest of the headline. For that matter even
| though it's true the graphics cards are as well. Only thing
| that could have made it more click bait would be to also put in
| AI in the headline.
| OJFord wrote:
| True, rather like age (or any discrimination category
| actually) is often used to make something sound more of an
| achievement, even though it's actually just about right place
| right time, experience (which you could happen to have at
| almost any age), etc.
| vmception wrote:
| > In short, it is to let the AI help you "look at the film", and
| the accuracy rate is almost comparable to professional doctors,
| and it is completely free.
|
| In the US, the issue is getting the Xray
___________________________________________________________________
(page generated 2022-05-20 23:00 UTC)