[HN Gopher] A Multi-Level View of LLM Intentionality
___________________________________________________________________
A Multi-Level View of LLM Intentionality
Author : zoltz
Score : 40 points
Date : 2023-09-11 16:59 UTC (5 hours ago)
(HTM) web link (disagreeableme.blogspot.com)
(TXT) w3m dump (disagreeableme.blogspot.com)
| lukev wrote:
| I'm not sure the definition of "intention" the article suggests
| is a useful one. He tries to make it sound like he's being
| conservative:
|
| > That is, we should ascribe intentions to a system if and only
| if it helps to predict and explain the behaviour of the system.
| Whether it _really_ has intentions beyond this is not a question
| I am attempting to answer (and I think that it is probably not
| determinate in any case).
|
| And yet, I think there's room to argue that LLMs (as currently
| implemented) cannot have intentions. Not because of their
| capabilities or behaviors, but because we know how they work
| (mechanically at least) and it is incompatible with useful
| definitions of the word "intent."
|
| Primarily, they are pure functions that accept a sequence of
| tokens and return the next token. The model itself is stateless,
| and it doesn't seem right to me to ascribe "intent" to a
| stateless function. Even if the function is capable of modeling
| certain aspects of chess.
|
| Otherwise, we are in the somewhat absurd position of needing to
| argue that all mathematical functions "intend" to yield their
| result. Maybe you could go there, but it seems to be torturing
| language a bit, just like people who advocate definitions of
| "consciousness" wherein even rocks are a "little bit conscious."
| Icko wrote:
| > Primarily, they are pure functions that accept a sequence of
| tokens and return the next token. The model itself is
| stateless, and it doesn't seem right to me to ascribe "intent"
| to a stateless function. Even if the function is capable of
| modeling certain aspects of chess.
|
| I have two arguments against. One, you could argue that state
| is transferred between the layers. It may be inelegant for each
| chain of state transitions to be the same length, but it seems
| to work. Two, it may not have "states", but if the end result
| is the same, does it matter?
| haltist wrote:
| Ascribing human properties to computers and software has always
| seemed very bizarre to me. I always assume people are confused
| when they do that. There is no meaningful intersection between
| biology, intelligence, and computers but people constantly keep
| trying to imbue electromagnetic signal processors with
| human/biological qualities very much like how children
| attribute souls to teddy bears.
|
| Computers are mechanical gadgets that work with electricity.
| Humans (and other animals) die when exposed to the kinds of
| currents flowing through computers. Similarly, I have never
| seen a computer drink water (for obvious reasons). If
| properties are reduced to behavioral outcomes then maybe
| someone can explain to me why computers are so averse to water.
| version_five wrote:
| He was inspired by lesswrong which from my scan more
| mysticism and philosophy (with a handful of self importance)
| than anything about how computers work. Advanced technology
| is magic to laypeople. It's like how some people believe in
| homeopathy. If you don't understand how medicine works, it's
| just a different kind of magic.
| haltist wrote:
| I guess that might be tied up with human biology, the need
| to attribute agency to inanimate objects. That one is a
| worthwhile puzzle to figure out but most people seem more
| mesmerized by blinking lights and shiny gadgets than any
| real philosophical problems.
| version_five wrote:
| Not because of their capabilities or behaviors, but because we
| know how they work (mechanically at least) and it is
| incompatible with useful definitions of the word "intent."
|
| I've never seen this deter anyone. I can't understand how
| people that know how they work can have such ridiculous ideas
| about llms.
|
| I'd add though that inference is clearly fixed but there is
| some more subtlety about training. Gradient descent clearly
| doesnt have intelligence, intent (in the sense meant),
| consciousness either, but it's not stateless like inference and
| you could argue has a rudimentary "intent" in minimizing loss.
| og_kalu wrote:
| The most useful definitions have predictive power.
|
| When you say upsetting things to bing chat, you'll find the
| conversation prematurely end.
|
| You can cry all you want about how bing isn't _really_ upset.
| How it doesn 't _really_ have intention to end the chat but
| those are evidently useless defitions because the chat _did_
| end.
|
| A definition that treats Bing as an intentful system is more
| accurate to what happens in reality.
| version_five wrote:
| That might be useful in helping a child learn to use it, it
| has no value when actually studying neural networks. You
| could equally pretend the sun sets every night because it's
| upset from shining all day.
| og_kalu wrote:
| >That might be useful in helping a child learn to use it
|
| It is useful for anyone looking to use such systems. A
| LLM piloted robot could potentially pick up a knife and
| stab you because you obstructed some goal or said mean
| words and pretending it didn't have intent to do so won't
| bring you back to life. Acting like it does could help
| avoid such a scenario.
|
| >You could equally pretend the sun sets every night
| because it's upset from shining all day.
|
| No you couldn't.
|
| The conversation ended prematurely because of your input.
| There is zero ambiguity on the relation.
|
| But because you ascribe intent to bing, you can predict
| (accurately) that saying nice things will not end the
| conversation.
|
| LLMs act like they have intent. This is a matter of
| conceding they do or not. Conceding so is more useful
| because it has more accurate predictive power than the
| alternative. This becomes plain when LLMs are allowed
| more actions than just conversation.
|
| >it has no value when actually studying neural networks.
|
| On the contrary. Now you know that certain things are
| unnecessary to build a system that acts like it has
| intent.
| lawlessone wrote:
| >You could equally pretend the sun sets every night
| because it's upset from shining all day.
|
| That would describe quite a lot of ancient religious
| beliefs about the Sun
| jedharris wrote:
| The article provides a very clear reason for using the idea of
| "intention": that framing helps us understand and predict the
| behavior. In contrast framing a mathematical function as having
| "intention" doesn't help. The underlying mechanism isn't
| relevant to this criterion.
|
| Clearly the system we're understanding as "intentional" has
| state; we can engage in multi-round interactions. It doesn't
| matter that we can separate the mutable state from the function
| that updates that state.
| lo_zamoyski wrote:
| A mathematical function isn't a mechanism. It has no causal
| power.
| nomel wrote:
| Sure, but we're talking mathematical functions that have
| been given physical form, by attaching them to input and
| output devices. Or, am I missing something? For other
| examples, see anything automated with a motor, and some
| decision tree, semi chaotic or not.
| lo_zamoyski wrote:
| Uh, hold on. That's not what's meant by intentionality. No one
| is talking about what a machine _intends_ to do. In philosophy,
| and specifically philosophy of mind, "intentionality" is,
| briefly, "the power of minds and mental states to be about, to
| represent, or to stand for, things, properties and states of
| affairs" [0].
|
| So the problem with this guy's definition of intentionality is,
| first, that it's a redefinition. If you're interested in
| whether a machine can possess intentionality, you won't find
| the answer in interpretivism, because that's no longer a
| meaningful question.
|
| Intentionality presupposes telos, so if you assume a
| metaphysical position that rules out telos, such as
| materialism, then, by definition, you cannot have "aboutness",
| and therefore, no intentionality _of any sort_.
|
| [0] https://plato.stanford.edu/entries/intentionality/
| jedharris wrote:
| Some philosophers take that position. Dennett, explicitly
| cited in the article, wrote _The Intentional Stance_ (1987)
| about exactly the approach to intentionality taken in the
| article. His approach is accepted by many philosophers.
|
| As you point out, the approach you cite can't be used in a
| materialist metaphysical position. That's a pretty severe
| problem for that definition! So Dennett's approach, or
| something like it, has major advantages.
| jedharris wrote:
| Also, you are obviously wrong (or rhetorical?) when you say
| "No one is talking about what a machine intends to do." We
| certainly do! You can say "No one should" or other normative
| condemnations but then we're arguing on different territory.
| og_kalu wrote:
| >Unless you think that there is some fundamental reason why LLMs
| will never be able to play chess competently, and I doubt there
| is, then it seems that we could with the right prompts implement
| some sort of chess AI using an LLM.
|
| You can play a good game of chess (or poker for that matter) with
| GPT.
|
| https://twitter.com/kenshinsamurai9/status/16625105325852917...
|
| https://arxiv.org/abs/2308.12466
|
| There's also some work going on in the eleuther ai discord
| training LLMs specifically for chess to see how they shape up.
| They're using the pythia models. so far:
|
| Pythia 70M, est ELO 1050
|
| Pythia 160M, est ELO 1370
| lawlessone wrote:
| I've found they fall apart after a couple of moves and lose
| track of the game.
|
| Edit: This might not be the case anymore it seems, my below
| point doesn't actually contradict you, seems it matters a lot
| how you tell the model your moves. Also saying things like
| "move my rightmost pawn" completely confuses them.
| og_kalu wrote:
| Not had it lose track with the format in the first link
| (GPT-4, not really tried 3.5)
| lawlessone wrote:
| Yeah i was wrong. I think it has gotten better since i
| tried this.
| pixl97 wrote:
| The token model of LLMs doesn't map well into how human
| experience the world of informational glyphs. Left and right
| is a intrinsic quality of our vision system. An LLM has to
| map the idea of left and right into symbols via text and line
| breaks.
|
| I do think it will be interesting as visual input and
| internal graphical output is integrated with text based LLMs
| as that should help correct their internal experience to be
| based closer to what we as humans experience.
| lawlessone wrote:
| " An LLM has to map the idea of left and right into symbols
| via text and line breaks."
|
| Oh yeah that's i suggested it :)
|
| I do wonder though if we give the LLMs enough examples of
| texts with people describing their relative spatial
| position to each other and things will it eventually
| "learn" to work things these out a bit better
| og_kalu wrote:
| >I do wonder though if we give the LLMs enough examples
| of texts with people describing their relative spatial
| position to each other and things will it eventually
| "learn" to work things these out a bit better
|
| GPT-4's spatial position understanding is actually really
| good all things considered. By the end, 4 was able to
| construct an accurate maze just from feedback about the
| current position and possible next moves after each move
| by GPT-4.
|
| https://ekzhu.medium.com/gpt-4s-maze-navigation-a-deep-
| dive-...
|
| I think we just don't write much about moving through
| space and that is why reasoning about it is more limited.
| [deleted]
| labrador wrote:
| The authors of the text the model was trained on certainly had
| intentions. Many of those are going to be preserved in the
| output.
| passion__desire wrote:
| Can we say ChatGPT or its future versions would be like an
| instantiation of the Boltzmann Brain concept if it has internal
| qualia? The "brain" comes alive with the rich structure only to
| disappear after the chat session is over.
| SanJoseEngineer wrote:
| [dead]
| jedharris wrote:
| Cool way of putting it. Let's run with that. A good actor can
| be seen as instantiating a Boltzmann Brain while on stage --
| especially when improvising (as always may be needed). Maybe
| each of us is instantiating some superposition of Boltzmann
| Brains in everyday life as we wend our way through various
| social roles...
|
| From now on I'll listen for the subtle popping sounds as
| these BBs get instantiated and de-instantiated all around
| me...
|
| Of course a philosopher can object that (1) these BBs are on
| a substrate that's richer than they are so aren't "really"
| BBs and (2) they often leave traces that are available to
| them in later instantiations which again classical BBs can't.
| So maybe make up another name -- but a great way to think.
___________________________________________________________________
(page generated 2023-09-11 22:00 UTC)