[HN Gopher] Explainable Deep Learning: A Field Guide for the Uni...
___________________________________________________________________
Explainable Deep Learning: A Field Guide for the Uninitiated
Author : BERTHart
Score : 100 points
Date : 2020-05-03 17:53 UTC (5 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| mindgam3 wrote:
| Lost me at the first sentence.
|
| > Deep neural network (DNN) is an indispensable machine learning
| tool for achieving human-level performance on many learning
| tasks.
|
| Not to be pedantic, but words matter. Is anyone actually claiming
| that deep learning achieves true "human-level performance" on any
| real world open-ended learning task?
|
| Even the most state of the art computer vision/object
| classification algorithms still don't generalize to weird input,
| like familiar objects presented at odd angles.
|
| I get that the author is trying to write something motivating and
| inspirational, but it feels like claiming "near" or "quasi"-human
| performance, with disclaimers, would be a more intellectually
| honest way to introduce the subject.
| kahnjw wrote:
| Words matter, concretely define "open ended"? Did you just add
| that phrase to preemptively nullify evidence to the contrary?
|
| Deep learning has surpassed human level performance on many
| tasks [1][2]... (could add more you get the point).
|
| [1]
| https://www.sciencedirect.com/science/article/pii/S2215017X1...
| [2] https://arxiv.org/pdf/1502.01852v1.pdf
| rwilson4 wrote:
| "Many learning tasks" is a wiggle term. Sure, edge cases exist,
| but the methods do work impressively well in many cases.
| svara wrote:
| > Is anyone actually claiming that deep learning achieves true
| "human-level performance" on any real world open-ended learning
| task?
|
| No, but the text you quoted doesn't say that.
|
| Human level performance in this context means humans perform no
| better than some algorithm on some specific dataset.
|
| Incidentally, that's also how you get to claim superhuman
| performance on classification tasks. Just include some classes
| that aren't commonly known in your dataset, e.g. dog breeds,
| plant species, or something like that. ;)
| mindgam3 wrote:
| > No, but the text you quoted doesn't say that [deep learning
| achieves human-level performance
|
| Uh, it says DNNs are indispensable for achieving human level
| performance. That clearly implies that this level of
| performance is achievable, despite all evidence to the
| contrary.
| jdminhbg wrote:
| This is a weird interpretation of that sentence. There are
| lots of fields where human-level performance has been
| achieved. See Go, for example.
| asah wrote:
| Maybe you need an RNN to help parse that sentence!! :-)
|
| Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo
| buffalo
|
| etc
| troelsSteegin wrote:
| This is a review paper. It's long. Past the first sentence I find
| it readable and well organized. It mentions some of the work on
| interpretability I'd expect to see, Finale-Velez, Rudin, Wallach,
| and LIME, but does not appear to mention Shapley. The bottom line
| conclusion is "In the end, the important thing is to explain the
| right thing to the right person in the right way at the right
| time." That's both an obvious truth and a differentiating mindset
| in research-first space. It's worth a skim.
| cs702 wrote:
| I'm surprised there is no mention of capsules and capsule-routing
| algorithms.
|
| Capsules are groups of neurons that represent _discrete entities_
| in different contexts. For example, a 4x4 pose matrix is a
| capsule representing a particular object in different
| orientations seen from different viewpoints. Similarly, a subword
| embedding can be seen as a capsule with vector shape representing
| a particular subword in different natural language contexts. More
| generally, a capsule can have any shape, but it always represents
| only one entity in some context.
|
| In certain new capsule-routing algorithms -- e.g., EM routing[a],
| Heinsen routing[b], dynamic routing[c], to name a few off the top
| of my head[d] -- each capsule can activate or not _depending on
| whether the entity it represents is detected or not in the
| context of input data_.
|
| Models using these algorithms therefore make it possible for
| human beings to interpret model behavior _in terms of capsule
| activations_ -- e.g., "the final layer predicts label 2 because
| capsules 7, 23, and 41 activated the most in the last hidden
| routing layer."
|
| While these new routing algorithms are not yet widely used, in my
| humble opinion they present a promising avenue of research for
| building models that are explainable and/or enable assignment of
| causality at high levels of function composition.
|
| --
|
| [a] https://research.google/pubs/pub46653/
|
| [b] https://arxiv.org/abs/1911.00792
|
| [c] https://arxiv.org/abs/1710.09829
|
| [d] If you're aware of other routing algorithms that can
| similarly activate/deactivate capsules, please post a link to the
| paper or code here.
| Eridrus wrote:
| It should be pretty obvious why, they don't work as well as
| what is standard. This is about ways to explain the models we
| get good performance with.
| orionr wrote:
| For those interested in this area for PyTorch models, take a look
| at Captum (https://captum.ai/). Still a lot of work to do, but
| we've provided a number of algorithms described in this field
| guide in the library. Always looking for collaborators and
| contribution of others.
|
| Disclosure: I support the team that developed Captum.
___________________________________________________________________
(page generated 2020-05-03 23:00 UTC)