[HN Gopher] 2022 Letter
___________________________________________________________________
2022 Letter
Author : oli5679
Score : 63 points
Date : 2022-12-30 18:18 UTC (4 hours ago)
(HTM) web link (zhengdongwang.com)
(TXT) w3m dump (zhengdongwang.com)
| mdorazio wrote:
| Do any mainstream AGI researchers believe that GPT-style ML
| methods will plausibly get us to AGI? I have only a very shallow
| understanding of the state of the art, but from the outside
| playing with the tools it seems much more likely that we'll get
| to a local maximum far below AGI and require a different approach
| altogether. I'd love to read some discussion on the topic if
| anyone has good non-hype/biased fanboy links to share.
| 0x008 wrote:
| Actually they don't. You can listen to the podcasts by Lex
| Fridman with Yan Le Cun or Andrej Karpathy regarding that
| topic. But basically what Le Cun is saying is that the
| information density in text is not high enough in order to
| learn a realistic representation of the world from it.
| [deleted]
| readonlybarbie wrote:
| As a professional programmer and a relatively optimistic AGI
| enthusiast, why would the current ML methods not work, given
| sufficient CPU/RAM/GPU/latency/bandwidth/storage?
|
| In theory, as long as you can translate your inputs and outputs
| to an array of floats, a neural network can compute _anything_.
| The required number of neurons might not fit into the world 's
| best RAM, and the required number of weights and biases for
| those neurons might not be quickly calculated by a CPU/GPU
| however.
| tmoertel wrote:
| One big gap is causal learning. A true general intelligence
| will have to learn how to intervene in the real world to
| cause wanted outcomes in novel scenarios. Most current ML
| models capture only stastical knowledge. They can tell you
| what interventions have been associated with wanted outcomes
| in the past. In some situations, replaying these associations
| seems like genuine causal knowledge, but in novel scenarios
| this falls short. Even in current day models designed to make
| causal inferences, say for autonomous driving, the causal
| structure is more likely to have been built into the models
| by humans, rather than inferred from observations.
| azinman2 wrote:
| Yes but that doesn't mean you won't need new architectures or
| training methods to get there, or data that doesn't currently
| exist. We also don't know how many neurons / layers we'd
| need, etc.
|
| The brain itself is infinitely more complex than artificial
| neural networks. Maybe we don't need all of what nature does
| to get there, but we are so many orders of magnitude off its
| redonk. People talk about number of neurons of the brain as
| if there's a 1:1 mapping with an ANN. Real neurons have
| chemical, physical properties, along with other things
| probably not yet discovered going on.
| readonlybarbie wrote:
| This is an interesting comment. I agree that I hear the
| "all we need is 86 billion neurons and we will habe parity
| with the human brain", and I feel it is dubious to think
| this way because there is no reason why this arbitrary
| number _must_ work.
|
| I also think it is a bit strange to use the human brain as
| an analogy because biological neurons supposedly are
| booleans and act in groups to achieve float level behavior.
| For example I can have neurologic pain in my fingers that
| isn't on off, but rather, has differences in magnitude.
|
| I think we should move away from the biology comparisons
| and just seek to understand if "more neurons = more better"
| is true, and if it is, how do we shove more into RAM and
| handle the exploding compute complexity.
| jiggawatts wrote:
| The current AI approach is like a pure function in
| programming: no side effects, and given the same input you
| always get the same output. The "usage" and "training" steps
| are seperate. There is no episodic memory, especially there
| is no short term memory.
|
| Biological networks that result in conscious "minds" have a
| ton of loops and are constantly learning. You can essentially
| cut yourself off from the outside world in something like a
| sensory deprivation bath and your mind will continue to
| operate, _talking to itself_.
|
| No current popular and successful AI/ML approach can do
| anything like this.
| readonlybarbie wrote:
| Agreed, but I also wonder if this is a "necessary"
| requirement. A robot, perhaps pretrained in a highly
| accurate 3d physics virtual simulation, which has an
| understanding of how it can move itself and others in the
| world, and how to accomplish text defined tasks, is already
| extremely useful and much more general than an image
| classificiation system. It is so general, in fact, that it
| would begin reliably replacing jobs.
| jimbokun wrote:
| But it's not AGI.
| readonlybarbie wrote:
| Ok, so now we just have to define "AGI" then. A robot,
| which knows its physical capabilities, which can see the
| world around it through a frustrum and identifies objects
| by position, velocity, rotation, which understands the
| passage of time and can predict future positions for
| example, which can take text input and translate that
| into a list of steps it needs to execute, which is
| functionally equivalent to an Amazon warehouse employee,
| we are saying is not AGI.
|
| What is an AGI then?
| phphphphp wrote:
| An Amazon warehouse worker isn't a human, an Amazon
| warehouse worker is a human engaged in an activity that
| utilises a tiny portion of what that human is capable of.
|
| A Roomba is not AGI because it can do what a cleaner
| does.
|
| "Artificial general intelligence (AGI) is the ability of
| an intelligent agent to understand or learn any
| intellectual task that a human being can."
| readonlybarbie wrote:
| I think the key word in that quote is "any" intellectual
| task. I don't think we are far from solving all of the
| mobility and vision-related tasks.
|
| I am more concerned though if the definition includes
| things like philosophy and emotion. These things can be
| quantified, like for example with AI that plays poker and
| can calculate the aggressiveness (range of potential
| hands) of the humans at the table rather than just the
| pure isolated strength of their hand. But it seems like a
| very hard thing to generally quantify, and as a result a
| hard thing to measure and program for.
|
| It sounds like different people will just have different
| definitions of AGI, which is different from "can this
| thing do the task _i_ need it to do (for profit, for fun,
| etc) "
| dinkumthinkum wrote:
| Well, I would put it back like why would it? When you
| understand how these things work, does it sound anything like
| what humans do? When prompted with a question, we do not
| respond by predicting words that come next based on a
| gigantic corpus of pre-trained text. As a professional
| programmer, do you think Human intelligence works like a
| Turing machine?
| isthisthingon99 wrote:
| Did mainstream AGI researchers predict something like GPT would
| exist 15 years ago? I would listen to those who did on their
| opinion.
| [deleted]
| abecedarius wrote:
| There's a 2018 book of interviews of many well-known
| researchers where they're asked about future prospects:
| http://book.mfordfuture.com/ (list of interviewees on that
| page). The actual interview dates weren't specified but don't
| seem to be earlier than 2017, in my reading. Almost all of
| them greatly underestimated progress up to now, or refused to
| say much. (I'm hedging a little bit because it's a year since
| I read it, and memory is fuzzy.)
|
| Shane Legg of DeepMind wrote a blog post at the opening of
| the 2010s where he stuck his neck out to predict AGI with a
| time distribution peaking around 2030. He thought the major
| development would be in reinforcement learning, rather than
| the self-supervised GPT stuff.
| isthisthingon99 wrote:
| Sounds about right.
| hiddencost wrote:
| I don't think anyone serious thinks or talks in terms of AGI.
| The feverishly simplistic idea of the singularity is quite
| silly.
|
| Most notably, neural networks alone will not reach any kind of
| AGI.
|
| Start adding the capacity to read from massive knowledge
| stores, and a place to keep long term information (i.e.,
| memory, probably also in a database), plus a feedback loop for
| the model to learn and improve? Plus the ability to call APIs?
| Now you're talking. I think all of those pieces are close to
| doable right now, maybe with a latency of 5s. If one of the big
| players puts that in place in a way that is well measured and
| they can iterate on, I think we'll start to see some really
| incredible advances.
| localhost wrote:
| The gpt_index project looks very promising in this area.
|
| "At its core, GPT Index is about:
|
| 1. loading in external data (@NotionHQ, @Slack, .txt, etc.)
| 2. Building indices over that data 3. Inputting a prompt ->
| getting an output!"
|
| https://twitter.com/jerryjliu0/status/1608632335695745024
| amelius wrote:
| Interesting. How are these indexes stored and how are they
| fed into the transformer model so that GPT can use them?
| Does this require an additional training step?
| hiddencost wrote:
| One really exciting place things might improve is in data
| cleaning. Right now preprocessing your data and putting it in
| a format that can be learned efficiently and without bias is
| a huge pain // risk. This next generation is allowing us to
| largely ignore a lot of that work.
|
| Similarly, transfer learning is finally good.
|
| And the models are generalist, few shot learners.
|
| As a consequence, individuals with minimal expertise can set
| up a world class system to solve niche problems. That's
| really exciting and it's going to get easier.
| hiddencost wrote:
| Cross-language learning (what was referred to as an
| "interlingua" in the 90s) means we're seeing some stunning
| advances in low resource languages. It used to be that
| everyone ignored languages other than English, and then
| provides them with mediocre support.
|
| I think we're at a point where there's very little excuse
| not to launch in many languages at once.
| ripe wrote:
| Sort of the opposite of what you want, but Gary Marcus says we
| need to cross a few hurdles first:
|
| http://rebooting.ai/
| alsodumb wrote:
| I am a PhD student working in learning and autonomy space and
| every researcher I know thinks Gary Marcus is a joke. I'm not
| saying he doesn't know things, but all I am saying is machine
| learning at scale is not his area of expertise although he
| pretends it is. Period. He passes on very generic, obvious
| statements about the future without any details and when
| someone does something in that direction he claims 'I told
| you so!, you should have listened to me in the past!'. Look
| at the entire chain of discussion between Gary Marcus and
| Yann LeCun in this thread you'll get a sense of I am talking
| about: https://twitter.com/ylecun/status/1523305857420824576
|
| Gary Marcus is an academic grifter and to me he is no
| different than crypto bros who grift non-experts.
| hiddencost wrote:
| Seconding reports that Gary Marcus is almost as big a waste
| of your time as Jurgen Schmidhuber.
|
| Marcus has been writing some variant of exactly the same
| article multiple times a year for the last 15 years.
| Symmetry wrote:
| We still seem to be missing an equivalent of explicit memory
| formation, serializing digested perceptions into working then
| short term and long term memory. The however many thousand
| tokens in a GPT's buffer can span a much larger span of time
| than the second's worth of sense impressions your brain can
| hold without consciousness[1] and memory getting involved but
| the principle seems to be the same.
|
| This isn't to say that there wouldn't be some simple hack to
| allow memory formation in chat agents, just that there's at
| least one advance we need besides simple scale.
|
| [1] As in not subliminal, not anything to do with philosophical
| notions of qualia.
| mikepurvis wrote:
| I'm completely a bystander, but I feel like one flag for me
| with current approaches is the ongoing separation between
| training and runtime. Robotics has been through a similar thing
| where you have one program that does SLAM while you teleop the
| robot, and you use that to map your environment, then afterward
| shut it down and pass the static map into a separate
| localization + navigation stack.
|
| Just as robots have had to graduate to the world of continuous
| SLAM, navigating while building and constantly updating a map,
| I feel like there's a big missing piece in current AI for a
| system that can simultaneously act and learn, that can reflect
| on gaps in its own knowledge, and express curiosity in order to
| facilitate learning-- that can ask a question out of a desire
| to know rather than as a party trick.
| ben_w wrote:
| I think that depends on which definition of AGI you prefer. It
| knows more than I do about most topics (though I still beat it
| at the stuff I'm best at), so I'd say it's got the A and the G
| covered, and it's "only" at the level of a university student
| at the stuff I've seen it tested on, so it's "pretty
| intelligent" in the way toy drone submarines are "pretty good
| swimmers".
|
| It's not as fast a learner (efficient with samples) as humans
| are; it doesn't continuously learn from interactions with
| others like we do; and it's certainly not superhuman at
| everything (or probably anything other than _breadth_ of
| knowledge)...
|
| ...but Yudkowsky recently criticised Musk for taking that too
| far and limiting the definition of "AGI" to normality meant by
| "ASI" (Yudkowsky was also saying that no, GPT isn't AGI):
| https://twitter.com/ESYudkowsky/status/1600362288149856256?c...
| fmajid wrote:
| Yes, Dan Wang's letters are outstanding.
___________________________________________________________________
(page generated 2022-12-30 23:00 UTC)