[HN Gopher] Reinforcement-learning AIs are vulnerable to a new k... ___________________________________________________________________ Reinforcement-learning AIs are vulnerable to a new kind of attack Author : magoghm Score : 37 points Date : 2020-02-28 23:34 UTC (23 hours ago) (HTM) web link (www.technologyreview.com) (TXT) w3m dump (www.technologyreview.com) | curiousgal wrote: | Non paywalled version: https://outline.com/ekbXnD | tastroder wrote: | The actual paper seems to be this: | https://arxiv.org/abs/1905.10615 | | PDF: https://arxiv.org/pdf/1905.10615.pdf | | Website with videos: https://adversarialpolicies.github.io/ (that | would make a better submission imho) | | Github: https://github.com/HumanCompatibleAI/adversarial-policies | | You have to stretch the definition of "new" somewhat to come up | with the title TR chose, adversarial effects in all kinds of | learning settings certainly aren't, the paper itself seems to | contain quite interesting thoughts on how to assess them though | (as opposed to just using them to steer the training process). | colsmit wrote: | This article is not good, I encourage reading the paper its based | on instead: https://arxiv.org/pdf/1905.10615.pdf | | "In some ways, adversarial policies are more worrying than | attacks on supervised learning models, because reinforcement | learning policies govern an AI's overall behavior.If a driverless | car misclassifies input from its camera, it could fall back on | other sensors, for example." TIL fail-safe components are 1) | ubiquitous 2) work 3) only an option for supervised learning | components. | | "A supervised learning model, trained to classify images, say, is | tested on a different data set from the one it was trained on to | ensure that it has not simply memorized a particular bunch of | images. But with reinforcement learning, models are typically | trained and tested in the same environment." First, a RL | environment is not equivalent to a supervised learning data set. | Second, the train validate test paradigm is not thrown out in RL | research, its why OpenAI put their Starcraft agent on public | ladders. | | "The good news is that adversarial policies may be easier to | defend against than other adversarial attacks." This sentence | refers to Graves et al. adversarially training their agents. | Adversarial training is, of course, also conducted frequently in | supervised learning. ___________________________________________________________________ (page generated 2020-02-29 23:00 UTC)