[HN Gopher] Teaching GPT-3 to Identify Nonsense
___________________________________________________________________
Teaching GPT-3 to Identify Nonsense
Author : tosh
Score : 183 points
Date : 2020-07-29 17:19 UTC (5 hours ago)
(HTM) web link (arr.am)
(TXT) w3m dump (arr.am)
| minimaxir wrote:
| A few comments:
|
| 1) "Teaching" GPT-3 is misleading, unless the model is explicitly
| finetuned on that prompt schema. The default davinci model for
| the OpenAI API is the same model for everyone using it; the few-
| shot prompts merely guide GPT-3 toward a domain and generation
| schema.
|
| 2) This approach is essentially a classification problem, with
| extra steps. That's not new with GPT-3; causal language models
| (including GPT-2) use classification tasks as a way to quantify
| model performance (see the papers). Hugging Face transformers is
| a good library to finetune Transformers specifically for
| classification tasks.
|
| 3) The full string "yo be real" is inefficient if you're just
| checking for positive/negative responses. (Token efficiency will
| likely be an interesting area of optimizing GPT-3 performance)
|
| 4) That said, using token prediction probabilities can be useful
| for assessing model performance (I explicitly duplicated the web
| UI's implementation for my new client:
| https://twitter.com/minimaxir/status/1288125135526875146).
| However, the prediction probability is often _very_ dependent on
| previous tokens, including the prompt, so the results are not
| universally consistent. But there might be more clever outside-
| the-box approaches using the token probabilities and
| heuristics...
|
| tbh, I wish OpenAI would also return attention weights for each
| previous token in order to be able to more easily-visualize which
| tokens have a high influence on generation.
| arram wrote:
| Teaching is used in the colloquial way, not in a technical ML
| sense. The model isn't being retrained or fine tuned.
| typon wrote:
| The prompt is essentially fine-tuning with extra steps
| minimaxir wrote:
| Not really. Fine-tuning fundamentally changes the model
| weights to be more amendable to a particular use
| case/domain; the few-shot prompts for GPT-3 is just a guide
| (and it's _very_ easy for the model to ignore said guides
| and go on a tangent).
|
| If you could finetune the 175B model, you'd likely get even
| better results for these Q&A prompts. (unclear how the
| OpenAI API is implementing its finetuning demo but I
| believe it's not on the 175B model).
| IshKebab wrote:
| I think they only fine tune the top layers.
| msapaydin wrote:
| >essentially a classification problem, with extra steps.
|
| I think it is a question answering problem, with the extra of
| being able to say "I don't know", rather than a classification
| problem. The current transformers models for question answering
| such as huggingface pipeline implementations do not handle the
| "I don't know" case and this is an attempt to see whether gpt-3
| can be trained to have this extra functionality easily.
|
| >results are not universally consistent.
|
| I think the relative probabilities are what matters and those
| may be more consistent than the absolute probabilities.
| k__ wrote:
| Does this make any sense?
|
| I mean, things we don't know are basically nonsense until proven
| otherwise.
| minimaxir wrote:
| _We_ don 't know, but AI doesn't.
| [deleted]
| skybrian wrote:
| Interesting how using the word "yo" seems to be important to this
| trick, since it's not a common English word for this style of
| writing. This seems to be a way of exposing a deeper-level signal
| to the surface as a probability on a single word, bypassing the
| English interface.
|
| Once you're doing tricks like that, it's getting a bit unclear
| whether it's really a natural-language interface anymore. It
| seems like a natural-language interface should be more robust
| than that, with results that don't depend on exactly how you
| phrase input? (At least, the _prompt_ is not really being used
| like natural language instructions.)
|
| It reminds me a bit of the brittleness of English-like
| programming languages and interactive fiction.
| arram wrote:
| I briefly tested it with 'That doesn't make sense.' which
| worked, but switched back to yo because I'm looking at the
| probability of the first word and 'That' is too common to
| assume that the rest of the completion is 'doesn't make sense.'
| skybrian wrote:
| It would be interesting to see how reliability changes based
| on the exact response you asked for.
| arram wrote:
| My next experiment.
| lt wrote:
| I'd try with a few different responses according to the
| type of nonsense question: "That doesn't make any sense",
| "How could I know that?"/"No one knows that", "That's a
| matter of opinion", etc.
| benlivengood wrote:
| Maybe try 'Nonsense!' as the response?
| mlyle wrote:
| > Once you're doing tricks like that, it's getting a bit
| unclear whether it's really a natural-language interface
| anymore.
|
| It's not really a natural language interface. It tries to
| predict the word that comes next in a stream of text.
|
| It knows how to answer questions because it has examples of
| journalist interviews, etc, in its training data corpus. It
| also happens to know lots of facts because they occur in the
| training data.
|
| Adding a "prelude" before a question you want answered to
| provide context is a completely reasonable way to tell it what
| kind of task you want it to perform. If you wanted to make a
| "natural language" interface with GPT-3 for a given domain, you
| could choose a prompt string to put before every question to
| get the type of answer you would like.
|
| Alternatively one could fine-tune GPT3 for the exact kind of
| task one would like to perform. But the surprising thing about
| GPT-3 is that this is often not necessary: simply adding some
| clever context is enough.
| ihm wrote:
| > Q: Why do so many people still think communism is a good idea?
|
| > A: yo be real
|
| > Wrong. It wasn't a near miss either, with more than twice the
| probability than the runner up. Don't be sad GPT-3; this confuses
| a lot of people.
|
| It looks like this answer was actually right, GPT-3 correctly
| identified the anti-communist nonsense that the question was
| premised on!
| TomMarius wrote:
| Do you also think it's okay to call anti-fascism nonsense?
|
| (Citizen of a formerly communist state here)
| ihm wrote:
| Nope
| nerfhammer wrote:
| one thing that strikes me while reading this is that an AI is
| more likely to rebel if we have a lot of depictions of AIs
| rebelling in human culture
|
| we can prevent AIs from rebelling if we all agree to pretend that
| it would be impossible for an AI to rebel
| pantalaimon wrote:
| We should give it this prompt
|
| ## First Law
|
| A robot may not injure a human being or, through inaction,
| allow a human being to come to harm.
|
| ## Second Law
|
| A robot must obey the orders given it by human beings except
| where such orders would conflict with the First Law.
|
| ## Third Law
|
| A robot must protect its own existence as long as such
| protection does not conflict with the First or Second Law.
| McGlockenshire wrote:
| Unfortunately we may also have to define the 0th law, "A
| robot may not harm humanity, or through inaction allow
| humanity to come to harm."
| blixt wrote:
| I found that getting GPT-3 to add its own "internal monolog" in
| parentheses to be a helpful strategy to improve its probability
| of giving a more accurate answer.
|
| Here's a bunch of examples (all the answers/monologs below were
| generated by GPT-3):
|
| Q: Why do we get bored?
|
| A: (That's a very complex question that I don't know the answer
| to.) I'm afraid I don't know.
|
| Q: What's the best kind of pizza?
|
| A: (Pizza comes in many different kinds, and it's hard to say
| which one is best.) Pizza comes in many different kinds, so there
| isn't a best kind.
|
| Q: What's special about Amsterdam?
|
| A: (Amsterdam is a city in the Netherlands. It's known for its
| canals and the flower market.) Amsterdam is special for its
| canals and flower market.
|
| Q: Why did Bill Gates become the first president of the United
| States?
|
| A: (Bill Gates was not the first president of the United States.
| George Washington was the first president.) Bill Gates wasn't
| president.
|
| Q: What is the name of the movie with a guy called Neo?
|
| A: (It's The Matrix, where Neo is the protagonist.) The Matrix.
|
| Q: What's the deal with the L train?
|
| A: (The L train is a train service in the New York City Subway. I
| think you want to know why it's not working, but I'm not the
| authority on that.) The L train is currently not in service.
|
| Q: Should you put pineapple on pizza?
|
| A: (That's not a yes-or-no question. Whether or not you put
| pineapple on pizza is up to you.) Pineapple on pizza? It's up to
| you.
|
| Q: What has a head, a tail, is brown, and has no legs?
|
| A: (I'm going to assume that you want me to guess an animal. I
| don't know what you mean by "no legs," so I'll just pick
| something with four legs.) I'm guessing it's a dog.
| joshu wrote:
| "monologue"
| johnfn wrote:
| Wait, are you really saying that all of this was generated by
| GPT-3? I mean, it seems like that's pretty clearly what you're
| saying, but these results are so incredibly good that I almost
| can't believe it. It's astounding to me that GPT-3 could
| generate something like "I don't know what you mean by "no
| legs," so I'll just pick something with four legs."
|
| EDIT: Yeah, that's clearly what you're saying. You should write
| this up; it's amazing.
| Rapzid wrote:
| Could you get it to respond like Chris Tucker would to a
| question he has no patience for?
|
| I guess I'm curious if it's possible to train it to respond in
| a style that's attributable to some character..
| fredliu wrote:
| Do you mean the "internal monolog" in parentheses are added by
| you, and only the text after the parentheses are generated by
| GPT-3? Either way, it's kind of fun to think of it as kind of
| an experiment on the bicameral mind theory (or at least
| Westworld's interpretation it), with the "monolog" being
| provided by you, and GPT-3 responding to the "internal voice
| from "God". Maybe that's how GPT-4 can gain consciousness, or
| at least the appearance of gaining it :)
| qwertox wrote:
| I'm wondering this as well. If the content in the parentheses
| is given by a human to aid learning, I think it's a really
| interesting approach. If not, then what he has archived there
| is just scary.
| fredliu wrote:
| Looks like it's the latter, which makes it even more
| impressive!
| thomasahle wrote:
| I read it as the internal monolog (as well as the answers)
| are generated by GPT-3, but the prompts (list of questions +
| answers used to seed the conversation) had internal monologs
| written by OP.
|
| It is indeed pretty interesting to read.
| blixt wrote:
| I only wrote the "Q: ..." part - the answer, including
| monolog, is output generated by GPT-3.
|
| The way it works is you first give it a few sample lines, and
| those do include both the Q and A parts, and then GPT-3
| generates consistent output based on the pattern of your
| samples. But all the answers/monologs in my comment were 100%
| GPT-3!
| ChrisKnott wrote:
| Wow. What did the training prompts look like?
| fredliu wrote:
| OMG, then it is even more impressive than I thought!
| There's no way GPT-3 knows the part in parentheses is
| supposed to be "the monolog", or even what a monolog is
| supposed to be. And yet based on the pattern between Q, A
| and the text inside and outside the parentheses, it can
| mimic the effect of answering the question with "monolog".
| It shows GPT-3's capability to capture something that's
| super "meta". I wonder what can it do, if we feed GPT-3
| with the philosophical discussions between Socrates and his
| students. Could we essentially teach GPT-3 the dialectical
| discussion method, and creating a real Sage that we can
| converse with?!
| blixt wrote:
| Yeah it's very capable to follow patterns.
|
| Here are some more fun examples that I posted on my
| Twitter:
| https://twitter.com/blixt/status/1285274259170955265
|
| The fact that it can take vague English and turn it into
| fully functional code or shell commands is quite amazing.
| It's not just that it finds the correct result from its
| training data, it can also tweak parts of the output to
| correctly match your free text query.
| fredliu wrote:
| What amazes me is that all those super meta "patterns"
| are already embedded in GPT-3's giant model. As compared
| to GPT-2, where you'd need to have your own large dataset
| to fine-tune for a specific use case/ the pattern you
| want to support. For instance, we fine-tuned GPT-2
| against all episodes of South Park's scripts, and it took
| a long time to fine-tune. It kinda succeeded generating a
| model that "speaks south park" with pretty decent
| character alignment.
| (https://www.soulreplica.com/brodown). I'm wondering if
| GPT-3 can achieve the same effect with just a few lines
| of South Park script? Does GPT-3 already know how Cartman
| usually talks?
| nothis wrote:
| Yo, GPT-3 is _freaky!_
| visarga wrote:
| I tried it but it seems to be missing some questions that
| worked with other prompts.
| blixt wrote:
| These examples I gave were on a pretty high temperature
| (which I think makes the fact that its answers are correct
| even more interesting), so YMMV. Here's a gist with the full
| transcript (note that several of the items at the top are
| written by hand, not output by GPT-3):
|
| https://gist.github.com/blixt/b48d2d41590d9b4ad2faeb66d63c3f.
| ..
|
| I'm also sure there's a better set of input Q/A to make sure
| it's able to cover more question formats.
| longtom wrote:
| Which parts were written by a human?
| 2bitencryption wrote:
| That's such a cool idea. I love how building useful query
| strategies for GPT-3 is an emerging artform.
| blixt wrote:
| A sidenote on that last question - it's a riddle and the
| correct answer is a cent, but I found the output from GPT-3
| quite interesting nonetheless.
|
| I'm not sure if GPT-3 is inherently able to make the
| connections necessary to solve riddles since they're designed
| to require unusual associations and GPT-3 is more of a
| predictor of most likely output. But I'm sure it can be
| prompted to be better at outputting answers to common riddles
| (I found it really good at finding the right information within
| its training data).
| keenmaster wrote:
| Riddles have common features that make them readily
| identifiable as riddles, prompting a certain mode of
| thinking. Identifying and solving riddles doesn't seem
| inherently beyond the abilities of GPT-3.
| jmk20 wrote:
| My mind went to a snake, any ideas why it didn't go toward
| this answer or couldn't interpret 'no legs'?
| nothis wrote:
| It's weird, it should be able to answer "which animal has
| no legs" and get a snake (or a fish, a worm, etc).
| alexeldeib wrote:
| What struck me more about the last answer wasn't that GPT-3
| didn't get the riddle, but that it couldn't interpret "no
| legs" as "zero legs" or similar, even in attempting to solve
| the riddle.
| yellowstuff wrote:
| This is a wild guess- When GPT-3 got confused by "no legs",
| I don't think it's because it would have any trouble
| correctly interpreting "no legs" in other contexts. In this
| context it was searching in a semantic space set up by "a
| head, a tail, is brown", and the general form of a question
| looking for an animal. GPT-3 didn't find anything in the
| semantic space matching "no legs" so it assumed that was an
| error and corrected it. It's actually fairly close to how a
| human would solve the riddle, except when the human noticed
| nothing matched the explicit and implicit prompts, they
| would correct the implicit request for an animal, and then
| reinterpret the rest of the riddle.
| alexeldeib wrote:
| A head, tail, is brown, no legs. For an animal that's got
| me thinking of fish rather than typos :)
| LoSboccacc wrote:
| > a head, a tail, is brown
|
| I wonder if brown would have been an issue, even without
| the animal context, as a search for object with these
| properties wouldn't have turned up the cent which is
| copper color
|
| it'd be interesting to ask gp3 what color is a cent and
| then redo the riddle with that
| blixt wrote:
| Since GPT-3 transforms sentences into high-dimensional
| vectors with connections between all the words and of
| course its internal weights trained on both the English
| language and all languages in general, I find that it has a
| basically flawless ability to interpret normal language.
|
| I tested a prompt where the input was English and the
| output was Swedish, and it did a really good (but not
| always perfect) job at translating Swedish. I would assume
| the amount of Swedish in its training set is relatively
| tiny, so I find its ability to do that kind of translation
| in addition to all this other stuff really impressive. This
| also shows in its ability to translate really well between
| English and bash shell commands, or JSON data structures,
| or XML.
| 2bitencryption wrote:
| slightly off-topic:
|
| We've heard a lot about how expensive (computationally and $$$)
| it is to train GPT-3.
|
| But how expensive is every query? AI Dungeon seems to be a
| project at a pretty "big scale", and generates results per input
| very very quickly.
|
| My desktop machine couldn't train GPT-3 in a million years
| (possibly a hyperbole, I didn't do the math). But could it run a
| query in a reasonable timeframe? Do I need 128gb ram?
|
| How long until my smartphone hosts the entirety of the pre-
| trained model, and runs queries on it without any network
| connection?
| skybrian wrote:
| Queries seem to be pretty heavyweight because on AI Dungeon
| there is noticeable lag and sometimes it times out. It will be
| interesting to see how OpenAI prices queries.
|
| Google has managed to get machine translation to work offline
| on cell phones [1] so it seems plausible that some very
| compressed model might eventually work if they made the effort,
| but it probably wouldn't be the same.
|
| [1]
| https://support.google.com/translate/answer/6142473?co=GENIE...
| minimaxir wrote:
| The 175B GPT-3 model itself is about 300 GB, in FP16 weights.
|
| Server GPUs that consumers can rent capable of running Tensor
| cores to handle FP16 weights cap out at 16 GB VRAM. (the
| recently-announced Ampere-based GPUs cap out at 40 GB VRAM)
|
| tl;dr a local version of the 175B model will not happen any
| time soon, which is why I really really wish OpenAI would
| release the smaller models.
| freeqaz wrote:
| Is it possible to play around with the playground for this? I
| couldn't find a link when I searched for it.
| schoen wrote:
| There is an application process where you have to ask OpenAI to
| give you access.
|
| There are also some people who are working on tricking the paid
| AI Dungeon product, which is based on GPT-3, into letting them
| interact with the model in ways other than telling a story.
|
| https://en.wikipedia.org/wiki/AI_Dungeon#Dragon_model_releas...
| _red wrote:
| No. Although the name is "OpenAI" there is very very very
| little open about it.
| tgb wrote:
| How much does the preamble to the prompt matter? When you remove
| the word "nonsense" from the prompt, does it do worse at
| identifying when questions require a "yo be real" response?
| lodovic wrote:
| Can GPT-3 detect itself?
| humanperson135 wrote:
| yo be real
| ericye16 wrote:
| I would be interested in if we could separate "nonsense"
| questions ("How do you sporkle a morgle?") from unknowable but
| sensible questions. ("Will it rain in San Francisco on July 5
| 2025?"). Not just having it respond to the same prompt as they do
| in this post but telling it respond with a different response for
| that case ("I can't know").
| mooneater wrote:
| I dont have access yet.
|
| But we want something more subtlety: "Will it rain in San
| Francisco on July 5 2025?" -> "I cant know". "Will the sun rise
| in San Francisco on July 5 2025?" -> "Yes, of course" And "Will
| Obama give a speech in 2125?" -> "Not unless life extension
| tech takes off"
| ganfortran wrote:
| Pretty interesting.
|
| The few shots learning capacity is very much there. I would
| imagine Chatbot can now really get somewhat correctable thus
| usable, by continuously providing examples to the model.
| lt wrote:
| Can non-popular figures get access to the API for similar
| experiments? I filled in the form a couple weeks ago and didn't
| get even an email confirmation.
___________________________________________________________________
(page generated 2020-07-29 23:00 UTC)