[HN Gopher] Improbable Inspiration: Bayesian Networks (1996)
___________________________________________________________________
Improbable Inspiration: Bayesian Networks (1996)
Author : 1e
Score : 77 points
Date : 2020-12-05 16:53 UTC (6 hours ago)
(HTM) web link (www.cs.ubc.ca)
(TXT) w3m dump (www.cs.ubc.ca)
| dmarchand90 wrote:
| I like how in the mid 90s neural networks were almost a write-
| off. "But the neural nets won't help predict the unforeseen. You
| can't train a neural net to identify an incoming missile or plane
| because you could never get sufficient data to train the system."
| nextos wrote:
| They are almost orthogonal concepts in some regards. Bayesian
| models (and in particular Bayesian networks or graphical
| models) and neural networks are about different things. The
| former try hard to capture uncertainty and causality. The later
| are all about non-linearity.
|
| For example, Pyro implements tons of facilities to have
| Bayesian models augmented with neural networks.
|
| It makes a lot of sense from a modeling perspective to model
| the big picture using a Bayesian model (generally a graphical
| model) and then use neural networks for some components. You
| capture the overall causal structure, but you are also
| outputting really precise predictions. For example, a deep
| markov model.
|
| There are tons of unexplored ideas combining both, and in
| general I think this is the future of deep learning and one
| component towards AGI.
| radomir_cernoch wrote:
| Indeed. Mathematically speaking, a graphical model merely
| formalizes conditional independence. Their advantage is a
| statistical interpretation, which is also a factor that makes
| algorithms (like belief propagation) harder to parallelize on
| GPUs.
| sixdimensional wrote:
| I actually really think that the way Bayesian probability
| factors in subjective probability is key, in that even if an
| algorithm spits out a result, it is still subject to human
| interpretation as well. I think some kind of composite
| decision support with both purely objective results (e.g.
| neural networks or other models that are purely machine
| based) as well as subjective beliefs could be really
| interesting and I still haven't seen much that does this.
|
| I think maybe reinforcement learning where human feedback
| becomes part of the loop is about as close as I could think
| of. But that is different than factoring in human input to
| probability calculations.
| new23d wrote:
| > Then, in the late 1980s--spurred by the early work of Judea
| Pearl, a professor of computer science at UCLA, and breakthrough
| mathematical equations by Danish researchers--AI researchers
| discovered that Bayesian networks offered an efficient way to
| deal with the lack or ambiguity of information that has hampered
| previous systems.
|
| The "mathematical equations by Danish researchers", for those
| interested, are most likely this paper:
|
| Lauritzen, S.L. and Spiegelhalter, D.J. (1988), Local
| Computations with Probabilities on Graphical Structures and Their
| Application to Expert Systems. Journal of the Royal Statistical
| Society: Series B (Methodological), 50: 157-194.
| https://doi.org/10.1111/j.2517-6161.1988.tb01721.x
|
| Direct PDF Link:
| https://www.eecis.udel.edu/~shatkay/Course/papers/Lauritzen1...
| CapriciousCptl wrote:
| Interesting to see this underpinned the Office Help System one
| way or another-- probably being the infamous paperclip.
| mensetmanusman wrote:
| Way ahead of their time. They just needed 25 more years of
| Moore's law...
| cultus wrote:
| There's been enough progress in approximate Bayesian methods
| that many things can be done thousands of times faster than
| back then, as well. The reputation of Bayesian methods as being
| slow is undeserved nowadays.
| p1esk wrote:
| Can someone point me to any examples where Bayesian neural
| networks are successfully used for any practical
| applications? Like where they are better than regular non-
| Bayesian NNs? By better I mean better accuracy.
| emavro wrote:
| Not NN, just simple BN. A risk assessment application. More
| specifically, calculation of financial risk of climate
| change-related risks for the mining sector.
| Link:https://www.mdpi.com/2412-3811/4/3/38
| RavlaAlvar wrote:
| Bayesian network is not Bayesian neural network
| minkowski wrote:
| Not answering your question, but just to point out to
| readers that this article is about graphical models, not
| Bayesian neural networks.
| tachyonbeam wrote:
| It's kind of unfortunate that ML has become completely
| synonymous with neural networks in many people's mind.
| nextos wrote:
| Very small datasets and/or where a good uncertainty
| estimate of predictions is really important.
| tirthapatel wrote:
| AFAIK, Bayesian networks are extensively used in biological
| sciences and economics. Not sure if this will be useful,
| but I found a survey that discusses these applications: htt
| ps://www.frontiersin.org/articles/10.3389/fncom.2014.0013..
| .
___________________________________________________________________
(page generated 2020-12-05 23:00 UTC)