[HN Gopher] OpenAI API
___________________________________________________________________
OpenAI API
Author : gdb
Score : 327 points
Date : 2020-06-11 15:05 UTC (7 hours ago)
(HTM) web link (beta.openai.com)
(TXT) w3m dump (beta.openai.com)
| agakshat wrote:
| It's been a long time coming, but I am curious to see how
| OpenAI's research output is directed and impacted by market
| forces.
| minimaxir wrote:
| Since the demos on this page use zero-shot learning and the used
| model has a 2020-05-03 timestamp, that implies this API is using
| some form of GPT-3: https://news.ycombinator.com/item?id=23345379
| (EDIT: the accompanying blog post confirms that:
| https://openai.com/blog/openai-api/ )
|
| Recently, OpenAI set the GPT-3 GitHub repo to read-only:
| https://github.com/openai/gpt-3
|
| Taken together, this seems to imply that GPT-3 was more intended
| for a SaaS such as this, and it's less likely that it will be
| open-sourced like GPT-2 was.
| wildermuthn wrote:
| But since the resources required for training such a model are
| only available to well-funded entities, it seems like offering
| the model as an API while releasing the original source-code is
| the best practical method of getting the model into the hands
| of people who would otherwise not have access?
| minimaxir wrote:
| That depends on _which_ GPT-3 model they 're using, and from
| both the API and the blog page, it's unclear.
|
| Easy access to the 175B model would indeed be valuable, but
| it's entirely possible they're using a smaller variant for
| this API.
| ericlewis wrote:
| Exciting!
| wildermuthn wrote:
| In one of their examples, they note "They saw ratings hover
| around 60% with their original, in-house tech -- this improved by
| 7-8% with GPT-2 -- and is now in the 80-90% range with the API."
|
| Bloomberg reports the API is based on GPT-3 and "other language
| models".
|
| If that's true, this is a big deal, and it epitomizes OpenAI's
| namesake. The largest NLP models require vast corporate resources
| to train, let alone put into production. Offering the largest
| model ever trained (with near-Turing results for some tasks) is a
| democratization of technology that would otherwise have been
| restricted to well-funded organizations.
|
| Although the devil will be in the details of pricing and
| performance, this is a step worthy of respect. And it bodes well
| for the future.
| grizzlemeelmo wrote:
| Here's the Bloomberg story
| https://www.bloomberg.com/news/articles/2020-06-11/trillions...
| denster wrote:
| Agreed on the democratization front!
|
| We saw this OpenAI demo:
| https://player.vimeo.com/video/427943452
|
| and were just _blown away_. Very cool!!
|
| I guess a spreadsheet is never too old [1] to learn new tricks
| :)
|
| [1] Founder of https://mintdata.com here, so a bit biased (&
| opinionated about) spreadsheets, take the above with a pound or
| 10 of salt.
|
| [2] I've sent them this example how we'd invoke their APIs,
| hopefully they'll let us into the beta, fingers crossed :)
| https://mintdata.com/docs/learn/core-mechanics/work-with-dat...
| azinman2 wrote:
| It's only Open(tm)[?] if I can run the API on my own machines.
| sudosysgen wrote:
| Indeed. I really don't understand how proprietary SaaS is
| "Open". It's just as locked down as IBM Watson and even
| moreso than Google's WaveNet-aaS.
| madcowd wrote:
| If big LM's are the future then even if you had the model you
| couldn't run it on your own machines without having a DGX or
| two laying around.
| [deleted]
| eggsnbacon1 wrote:
| OpenAI started as a non-profit, went for-profit. Still owned by
| the big players.... Something isn't right.
|
| Is OpenAI just a submarine so the tech giants can do unethical
| research without taking blame??? Its textbook misdirection,
| nonprofit and "Open" in the name, hero-esque mission statement.
| How do you make the mental leap from "we're non-profit and we
| won't release things too dangerous" to "JK we're for-profit and
| now that GPT is good enough to use its for sale!!". You don't.
| This was the plan the whole time.
|
| GPT and facial recognition used for shady shit? Blame OpenAI. Not
| the consortium of tech giants that directly own it. It may just
| be a conspiracy theory but something smells very rotten to me.
| Like OpenAI is a simple front so big names can dodge culpability
| for their research.
| etaioinshrdlu wrote:
| I think it's simply because OpenAI is fundamentally created and
| controlled by venture _capitalists_ , and the tech they created
| turned out to be just too juicy an opportunity to not monetize.
|
| I can't say I blame them, when they realize they are sitting on
| the technological equivalent of a mountain of gold. What would
| you do?
| [deleted]
| seph-reed wrote:
| > sitting on the technological equivalent of a mountain of
| gold. What would you do?
|
| Greed is not justified. I get that people are weak, selfish,
| they can't stop themselves. Some feel sympathy because
| they've been weak too. "Maybe it's justified," they like to
| think. "Everybody lies." But seriously, those who care so
| much about money and power they can't do things in a
| civilized respectable way: they are not yet an adult and must
| be hard barred from the upper tiers of capitalism until they
| learn that life does not revolve around them.
|
| I blame them for being shitty, and blame everyone around them
| for letting it happen.
| Voloskaya wrote:
| > so the tech giants can do unethical research
|
| I know it's trendy (and partly justified) to look down on
| OpenAI, but can you actually give any basis for this claim?
|
| What kind of research is OpenAI doing that all the other big AI
| players (Google/DeepMind, FB, Microsoft) aren't also invested
| in? And even if others are doing the same, what part of
| OpenAI's research do you consider unethical?
|
| > It may just be a conspiracy theory
|
| Yea, it very much looks like that to be honest.
| eggsnbacon1 wrote:
| > What kind of research is OpenAI doing that all the other
| big AI players (Google/DeepMind, FB, Microsoft) aren't also
| invested in? And even if others are doing the same, what part
| of OpenAI's research do you consider unethical?
|
| I believe all of them are doing unethical research,
| especially facial recognition. Notice the public backpedaling
| this week from all the big tech companies on this too. By
| directing their cash through OpenAI they can avoid whatever
| fallout comes from unleashing things like GPT3 on the world.
|
| The most straightforward use case for GPT3 is generating fake
| but believable text. AKA spam. That's what it was designed to
| do. If you think fake news is a problem now, wait till
| someone is generating a dozen fake but believable news
| articles per minute by seeding GPT3 with a few words and
| hitting a button.
|
| Its a conspiracy theory with some circumstantial evidence. We
| will probably never know either way, because who would admit
| to it if it was true.
| wrsh07 wrote:
| Interestingly, by serving gpt3 as an API like this, they
| can actually monitor to see if companies are using it to
| generate spam
| Voloskaya wrote:
| > I believe all of them are doing unethical research,
| especially facial recognition.
|
| Yes all of them are doing facial recognition research,
| except... OpenAI, so how exactly is OpenAI used as a
| scapegoat to be able to do that kind of research without
| public backlash?
|
| > By directing their cash through OpenAI they can avoid
| whatever fallout comes from unleashing things like GPT3 on
| the world.
|
| GPT-3 si not unethical research. It is what you decide to
| with it and how you decide to release it that can
| potientially be unethical.
|
| Also, OpenAI is just ahead of other labs because they have
| an insane compute budget and really talented people, but if
| you have been following a little bit the NLP news, you will
| see that your theory of OpenAI being a front for unethical
| research just makes no sense. OpenAI release GPT-2, 1.5B
| billions parameters, then NVIDIA realeased Megatron, 8B
| parameters, Google released T5 at 11B and recently
| Microsoft did turing-nlg at 17B. So they are clearly
| working on this in their own names and very much
| publicizing their work.
| aivosha wrote:
| conspiracy hypothesis
| markshepard wrote:
| Well said. I think they need to change the name. It is
| misleading on many levels.
| codekilla wrote:
| A reshuffling might give: NopeAI or PeonAI
| solinent wrote:
| Based on my experience with non-profits, they are just like
| regular corps except they don't pay taxes, and they're always
| attached to a for-profit interest. The real community
| organizations don't tend to incorporate, as then you have to
| hire people to manage the corp or do it yourself.
|
| This OpenAI work is almost certainly a way for these bigger
| corps to collude. Proving that would be impossible, though.
| zitterbewegung wrote:
| GPT-2 is hard to do "shady" things right now (speaking from
| experience)[1] but maybe GPT-3 might do better?
|
| I could get poems to generate well. Tweets were a bit harder
| but I don't think we are at the point where you could just use
| a generative model to fool people that would be cheaper than
| actually hiring someone to write fake news. (Also shameless
| plug below)
|
| [1] 1400 - TALK.8 - "A way to make fake tweets using GPT2" -
| Joshua Jay Herman https://thotcon.org/schedule.html
| gobengo wrote:
| It looks like a similar organizational structure as Mozilla
| Foundation + Mozilla Corporation.
| eggsnbacon1 wrote:
| They redefined the org from non-profit to "capped profit",
| whatever that means.
|
| They're directly selling GPT 3 even though they originally
| said they wouldn't release it because of potential bad uses.
|
| They paid MS a ton of money for hardware and got a huge
| equity investment from them.
|
| And lets be honest here, the easiest and most straight-
| forward use of GPT3 is generating spam and low quality
| clickbait. Its the only use case that requires zero effort.
| The whole thing is built to generate fake but believable
| text. Its DeepFakes for text.
|
| I'm not saying the whole thing is nefarious and evil, just
| suggesting that OpenAI may not be what it seems. There's a
| lot of odd things going on with it. They should have done
| what universities do, spin off the technology into a
| different for-profit company and sell it. Instead of
| redefining their entire org structure to make money.
| mrfusion wrote:
| Couldn't you generate fake support for issues on social
| media with this?
| pphysch wrote:
| Already being done on an industrial scale, though this is
| further progress.
| ape4 wrote:
| If that's the case, they need to change they're name.
| swyx wrote:
| wow you just made the connection for me. GPT2 was too dangerous
| to release, and now GPT3 is so much better - is there no point
| at which things become too dangerous anymore? what was the
| conclusion on that one?
| simonkafan wrote:
| GPT2 being "too dangerous to release" was a marketing stunt
| from the very beginning.
| deep_etcetera wrote:
| Who are you quoting here?
| minimaxir wrote:
| The blog post directly addresses this question:
| https://openai.com/blog/openai-api/
|
| > What specifically will OpenAI do about misuse of the API,
| given what you've previously said about GPT-2?
|
| > We will terminate API access for use-cases that cause
| physical or mental harm to people, including but not limited
| to harassment, intentional deception, radicalization,
| astroturfing, or spam; as we gain more experience operating
| the API in practice we expect to expand and refine these
| categories.
| swyx wrote:
| ah i've been caught not reading the linked post
|
| hmm i dont love this. either OpenAI has implicitly promised
| to monitor all its users, or has adopted a "report TOS
| violations to us when they happen and we will judge"
| stance. neither are great roads to go down.
| bhl wrote:
| With Amazon having a moratorium of their rekognition API, I
| wonder if a Cambridge Analytica type event could happen to
| OpenAI where someone abuses and escapes the terms of
| service.
| throwaway7281 wrote:
| When I learned that Sam Altman (sorry Sam) was involved, I
| understood the direction, you mentioned.
|
| And yes, there is often no need to call something open
| explicitly, if it really is. Is into OpenOS, or just Linux?
| aivosha wrote:
| More fake news and generated AI content there is more people
| would stop trusting social media. It will saturate to that
| tipping point that humanity will need to find more genuine ways
| to communicate. So I say bring it on.
| m_ke wrote:
| I guess Sama plans on manufacturing growth metrics by forcing YC
| companies to pretend that they're using this.
|
| Generic machine learning APIs are a shitty business to get into
| unless you plan on hiring a huge sales team and selling to
| dinosaurs or doing a ton of custom consulting work, which doesn't
| scale the way VCs like it to. Anybody who will have enough know
| how to use their API properly can jus grab an open source model
| and tune it on their own data.
|
| If they plan on commercializing things they should focus on
| building real products.
| wildermuthn wrote:
| I imagine they're considering offering GPT-3, which would be
| cost prohibitive to fine-tune for most people. I also I heard
| inference was too slow to be practical. Perhaps they have some
| FPGA magic up their Microsoft sleeves.
| m_ke wrote:
| Nobody is putting these huge models in production, even the
| smaller transformer models are still too expensive to run for
| most use cases.
|
| With the way the field is moving, GPT-3 will be old news in a
| month, when more advances are made and open sourced.
| wildermuthn wrote:
| Precisely my point. If they could put a model as large as
| GPT-3 into production (at a reasonable price to the
| consumer), wouldn't that be a 10x improvement?
| krallistic wrote:
| GPT-3 isn't a 10X improvement. (At least from everything
| we know so far.)
| wildermuthn wrote:
| If the OP is right that nobody is putting the largest
| models into production (which I think is in inaccurate
| statement), then GPT-3 in production would be a 10x (ok,
| 5x?) improvement over the small GPT-2s and BERTS in
| production? So 10x in practice, if the hypothesis is
| correct? Which like I said, I don't believe to be the
| case.
| fongitosous wrote:
| i don't understand. if they run it for you and you apply
| transfer learning and fine tuning on your specific use case
| that would reduce drastically the costs hence why their
| offer make sense
| [deleted]
| dang wrote:
| > I guess Sama plans on manufacturing growth metrics by forcing
| YC companies to pretend that they're using this.
|
| That's wrong in almost too many ways to list. Sam left YC over
| a year ago, nor would he do such a thing. Nor does YC have that
| kind of power over companies, nor would it use it that way if
| it did have. That would be wrong and also dumb.
| m_ke wrote:
| Sorry, that was supposed to be sarcastic. What I meant to say
| is that Sam has a huge network and is a phone call away from
| pitching any CEO in the valley. One of the biggest benefits
| of YC these days is the huge network of companies in your
| portfolio, which makes getting intros and pilots a lot
| easier, leading to "traction" and more VC dollars.
| antris wrote:
| Not everyone wants to be an admin to their infrastructure. Real
| existing services like Heroku and Squarespace exist as useful
| services because even though you might know how to design and
| build a website from scratch, sometimes you just need something
| done quickly without too much worrying about details of the
| system that do not matter for your project at this point. I
| really don't see how this wouldn't apply to AI projects as
| well.
|
| I could make a much better site coding my own website from
| scratch and setting up servers myself, but for some projects I
| wouldn't even think about it that way, because using Heroku or
| Squarespace I can save a LOT of time and get the results I need
| much quicker.
| m_ke wrote:
| That's true, but machine learning models are not twilio or
| sendgrid, you have to tune them for your use case, monitor
| their performance and handle the uncertainty of their
| outputs. Doing that well requires a data scientist and if you
| have one they will be much more productive iterating on their
| own models instead of depending on a 3rd party black box.
| Grimm1 wrote:
| Except the point of these larger transformer models is they
| generalize well over a wide range of domains or only
| require a small amount of transfer learning for really
| specific domains.
|
| I'd say they're perfect candidates for the API as a service
| model.
| zoopdewoop wrote:
| I'm pretty sure people said the exact same thing about
| Algolia when it was getting started (you have to tune
| search for your use case! How could you possibly use a
| search provider?!?)
|
| Truth about the situation: - Transformers generalize well
| and don't need much fine tuning - OpenAI can probably fine
| tune for your use case better than you can - Getting new
| models into production takes 6 months to a year at
| companies of this size, if you did have Data Scientists in
| house, it might just be better to go with a solution like
| this for velocity - Not every company has the talent to
| make an in house ML program successful.
| antris wrote:
| Not a data scientist myself, but plenty of data scientists
| in a consultancy company that I used to work in said that
| they have to implement variants of a limited set of models
| over and over again, because they couldn't reuse code and
| infrastructure. The project contracts demanded that all IP
| created by the consultant is the property of the client.
| This even caused some of the data scientists to lose
| motivation, because the job wasn't challenging to them
| intellectually as it involved setting up the same stuff
| again and again. Very rarely would their actual expertise
| be needed in the job.
|
| I am not sure if this particular service solves the problem
| for them in any way, but to my ear it sounds like there is
| a need for code and infrastructure reuse in the data
| scientists domain that is ripe for innovation.
| krallistic wrote:
| I wonder if there are any legal complications in the transition
| from a non-profit to a regular company (especially from a tax
| perspective)
| lerax wrote:
| Natural Language Shell seems fun
| mcrider wrote:
| Whoa -- Speech to bash commands? That's a pretty novel idea to me
| with my limited awareness of NLP. I could see this same idea in a
| lot of technical applications -- Provisioning cloud
| infrastructure, creating a database query.. Very cool!
| nickswalker wrote:
| Cool indeed! While language-to-code (where code is a regular,
| general-purpose language) has only recently started to be
| workable, text-to-SQL has been a long running
| application/research area for semantic parsing.
|
| Some interesting papers and datasets:
|
| NL2Bash: https://arxiv.org/abs/1802.08979
|
| Spider: https://yale-lily.github.io/spider
| jorgemf wrote:
| It is not a novel idea and I don't think it is practical. If
| the natural language was practical for bash we would already
| have already "list directory" instead of "ls" and so on. "ls"
| is just 3 keystrokes while the natural language option is 15, 5
| times more.
| chabad360 wrote:
| It could be useful for learning tho (but at that point it
| could also become a crutch).
| dreamer7 wrote:
| But the character length would matter less when you can move
| to the speech domain.
|
| ls is 2 syllables list dir is also 2 syllables with more
| meaning.
|
| Ultimately, with natural language, the effectiveness seems to
| be when it is coupled with speech-to-text
| kredd wrote:
| I was imagining more of a "list of files that contain word
| "hello" in them at least 5 times". Would be useful to easily
| write longer and pipe-chained commands, especially for people
| that don't use bash-like scripting on a daily basis.
| mcemilg wrote:
| From AGI to money machine...
| gumby wrote:
| I miss the opposite: the old openAI gym and other testbeds. I
| still don't know why they shut those down.
|
| What alternatives do people like?
| owenshen24 wrote:
| Seems potentially more simple to get up and running then the
| Azure and Google Cloud alternatives which seemed involved when I
| last tried them.
| say_it_as_it_is wrote:
| OpenAI started off wide-eyed and idealistic but it made the
| mistake of taking on investors for a non-profit mission. A non-
| profit requires sponsors, not investors. Investors have a
| fiduciary responsibility to maximize profits, not achieve social
| missions of open AI for all.
| gdb wrote:
| OpenAI LP, our "capped-profit" entity which has taken
| investment, has a fiduciary duty to the OpenAI Charter:
| https://openai.com/blog/openai-lp/
| LockAndLol wrote:
| It'd be great if OpenAI also introduced CAPTCHA. I'd be much more
| willing and understanding to resolve those than anything Google
| makes.
| andyljones wrote:
| Concrete numbers from the various pullouts:
|
| > They saw ratings hover around 60% with their original, in-house
| tech -- this improved by 7-8% with GPT-2 -- and is now in the
| 80-90% range with the API.
|
| > The F1 score of its crisis classifier went up from .76 to .86,
| and the accuracy went up to 96%.
|
| > With OpenAI, Algolia was able to answer complex natural
| language questions accurately 4x as often as it was using BERT.
|
| I think the most informative are the first two, but the most
| _important_ is the final comparison with BERT (a Google model). I
| am, uh, a little worried about how fast things will progress if
| language models go from a fun lil research problem to a killer
| app for your cloud platform. $10m per training run isn't much in
| the face of a $100bn gigatech R&D budget.
| grogenaut wrote:
| $10m per training run gets me a lot of engineering time to
| build our own version of this system and lease it to other
| customers. Just skip one training run and I've got a pretty
| good team.
| ganstyles wrote:
| Putting aside the question of whether it would ever be a
| choice between spending $10M on a training run and hiring a
| team for $10M, GPT transformers were the end result of
| decades of language research and innovations. You're making
| it sound as though you can build the next iteration past
| GPT-3 for $10M, which I don't think is the case.
| brainless wrote:
| This is what I submitted for beta list:
|
| I want to create a software that can generate new code given
| business case hints, by studying existing open source code and
| their documentation.
|
| I know this is vague, but sounds like what we eventually want for
| ourselves right?
| anaganisk wrote:
| Remember how Microsoft trained their bot from reddit comments
| and it went anti human? Well I guess I have to start dropping
| hints for the skynet in all my repos.
| mrmonkeyman wrote:
| If said AI will also maintain it.
| historyremade wrote:
| "Powered by Azure" -- Elon clearly distrust Amazon.
| jfoster wrote:
| Elon is no longer part of OpenAI. Microsoft invested $1b.
|
| https://en.wikipedia.org/wiki/OpenAI
| benatkin wrote:
| Does this mean Microsoft isn't going to sue their customers
| for patent infringement?
| google234123 wrote:
| Why would you say this?
| jfoster wrote:
| I presume it's reference to OpenAI's patent pledge:
|
| > Researchers will be strongly encouraged to publish
| their work, whether as papers, blog posts, or code, and
| our patents (if any) will be shared with the world.
|
| I'm not sure if it's ever been publicly elaborated on.
|
| https://openai.com/blog/introducing-openai/
| jfoster wrote:
| Microsoft's stake in OpenAI doesn't seem to be publicly
| known.
|
| > Exactly what terms Microsoft and OpenAI have agreed on
| with this $1 billion investment isn't clear.
|
| https://www.theverge.com/2019/7/22/20703578/microsoft-
| openai...
| sytse wrote:
| An API that will try to answer any natural language question is a
| mind blowing idea. This is a universal thinking interface more
| than an application programming one.
| kamikazehosaki wrote:
| OpenAI seems like a completely disingenuous organization. They
| have some of the best talent in Machine Learning, but the
| leadership seems completely clueless.
|
| 1) (on cluelessness) If Sama/GDB were as smart as they claim to
| be, would they not have realized it is impossible to run a non
| profit research lab which is effectively trying "to compete" with
| DeepMind.
|
| 2) (on disingenuity) The original openAI charter made OpenAI an
| organization that was trying to save the world from nefarious
| actors and uses of AI. Who were such users? To me it seemed like,
| entities with vastly superior compute resources who were using
| the latest AI technologies for presumably profit oriented goals.
| There are few organizations in the world like that, namely FAANG,
| and their international counterparts. Originally OpenAI sounded
| incredibly appealing to me, and a lot of us here. But if their
| leadership had more forethought, they would perhaps not have made
| this promise. But given the press, and the money they accrued, it
| has now become impossible to go back on this charter. So the only
| way to get themselves out of the whole they dug into was by
| making it into a for profit research lab. And by commercializing
| perhaps a more superior version of the tools Microsoft, Google
| and the other large AI organizations are commercializing, is
| OpenAI any different from them?
|
| How do we know OpenAI will not be the bad actor that is going to
| abuse AI given their self interest?
|
| All we have is their charter to go by. But given how they are
| constantly "re-inventing" their organizational structure, what
| grounds do we have to trust them?
|
| Do we perhaps need a new Open OpenAI? One that we can actually
| trust? One that is actually transparent with their research
| process? One that actually releases their code, and papers and
| has no interest in commercializing that? Oh, that's right, we
| already have that -- research labs at AI focused schools like
| MIT, Stanford, BAIR and CMU.
|
| I am quite wary of this organization, and I would encourage other
| HN readers to think more careful about what they are doing here.
| chillee wrote:
| Why is it "impossible"? Academic labs are non-profit, and they
| are also effectively trying "to compete" with DeepMind.
| dna_polymerase wrote:
| Have a look at this discussion and the article from earlier
| today [0]. Of course, a singular lab could compete with
| something DeepMind does, but not without massive amounts of
| money in their pockets. The state of the art has become
| pretty expensive, really fast.
|
| [0]: https://news.ycombinator.com/item?id=23486163
| Yajirobe wrote:
| State of the art can be (and usually is) born in academic
| labs.
| Grimm1 wrote:
| Awesome! Just signed onto the wait list.
| typon wrote:
| This is incredible. I can't tell how much this is cherry-picked
| examples vs. revolutionary new tech.
| spookyuser wrote:
| Yeah I can't tell exactly which ones but I really feel like
| some of the OpenAI demos of products could be potentially huge
| if fleshed out.
| gdb wrote:
| Sign up for the beta if you'd like to be the one to flesh
| them out :)!
| dmvaldman wrote:
| AGI in text is < 3yrs away.
| Barrin92 wrote:
| there's zero understanding in any of this. This is still just
| superficial text parsing essentially. Show me progress on
| Winograd schema and I'd be impressed. It hasn't got anything to
| do with AGI, this is application of ML to very traditional NLP
| problems.
| gwern wrote:
| > Show me progress on Winograd schema and I'd be impressed.
|
| The paper evaluated Winograds:
| https://arxiv.org/pdf/2005.14165.pdf#page=16
| dmvaldman wrote:
| i think you are assuming that what is happening under the
| hood is that a human-inputted sentence is being parsed into a
| grammar. it is not.
| Barrin92 wrote:
| I know that it isn't. That's part of the problem. There is
| no attempt to generate some sort of structure that can be
| interpreted semantically and reasoned about by the model.
| The model just operates on the input superficially and
| statistically. That's why there has been virtually no
| progress on trivial tasks such as answering:
|
| _" I took the water bottle out of the backpack so that it
| would be [lighter/handy]"_
|
| What is lighter and what is handy? No amount of stochastic
| language manipulation gets you the answer, you need to
| understand some rudimentary physics to answer the question,
| and as a precondition, you need a grammar or ontology.
| FeepingCreature wrote:
| Have you tried feeding this to GPT and seeing if it
| continues it in a way that reveals understanding?
|
| It sounds like you're saying "It doesn't work because it
| can't work", but you haven't actually shown that it
| doesn't work.
| Barrin92 wrote:
| yes, I have. You can paste these into the website of the
| Allen Institute for AI, yourself here.
| (https://demo.allennlp.org/reading-
| comprehension/MjE1MzE1Mg==)
|
| In the example above it guesses wrongly, but again this
| is not surprising because it can't possibly get the right
| answer (other than by chance). The solution here cannot
| be found by correlating syntax, you can only answer the
| question if you understand the meaning of the sentence.
| That's what these schemas are constructed for.
| azinman2 wrote:
| What breakthrough occurred?
| dmvaldman wrote:
| Zero shot and few-shot learning in GPT-3 and lack of
| significant diminishing returns in scaling text models. Zero-
| shot learning is equivalent to saying "i'm just going to ask
| the model something that it was not trained to do"
| azinman2 wrote:
| And how do we get from zero shot to AGI? You're making
| gigantic leaps here.
| dmvaldman wrote:
| what is the difference between zero-shot learning in text
| and AGI? not saying there isn't one, but can you state
| what it is?you can express any intent in text (unlike
| other media). to solve zero-shot in text is equivalent to
| the model responding to all intents.
|
| many people have different definitions for AGI though.
| for me it clicked when i realized that text has this
| universality property of capturing any intent.
| azinman2 wrote:
| Zero-shot learning is a way of essentially building
| classifiers. There's no reasoning, there's no planning,
| there's no commonsense knowledge (not in a comprehensive,
| deep way that we would look for it call it that), and
| there's no integration of these skills to solve common
| goals. You can't take GPT and say ok turn that into a
| robot that can clean my house, take care of my kids, cook
| dinner, and then be a great dinner guest companion.
|
| If you really probe at GPT, you'll see anything that goes
| beyond an initial sentence or two really starts to show
| how it's purely superficial in terms of understanding &
| intelligence; it's basically a really amazing version of
| Searle's Chinese room argument.
| dmvaldman wrote:
| I think this is generally a good answer, but keep in mind
| I said AGI "in text". My forecasting is that within 3
| years you will be able to give arbitrary text commands
| and get the textual output of the equivalents of "clean
| my house, take care of my kids, ..." like problems.
|
| I also would contend that there is reasoning happening
| and that zero-shot demonstrates this. Specifically,
| reasoning about the intent of the prompt. The fact that
| you get this simply by building a general-purpose text
| model is a surprise to me.
|
| Something I haven't seen yet is a model simulate the mind
| of the questioner, the way humans do, over time (minutes,
| days, years).
|
| In 3 years, I'll ping you :) Already made a calendar
| reminder
| azinman2 wrote:
| Pattern recognition and matching isn't the same thing as
| reasoning. Zero shot demonstrates reasoning as much as
| solving the quadratic equation for a new set of variables
| does; it's simply the ability to create new decision
| boundaries leveraging the same set of classifying power
| and methodology. True agi isn't bound to a medium -- no
| one would say Helen Keller wasn't intelligent for
| example.
|
| I look forward to this ping :)
| sytelus wrote:
| Natural language search is approximately $100B business. This
| might be first AI application that changes the search landscape
| from 1990s and finally puts an end to the question "where is
| money in AI?".
| nick_araph wrote:
| It seems like a step towards OpenAI becoming something like a
| utility provider for AI capabilities
| alphagrep12345 wrote:
| Interesting to see this. Is this similar to Google and Azure's ML
| apis?
| zitterbewegung wrote:
| Looks like OpenAI is going head to head with huggingface.
|
| This makes a lot of sense and it seems they are telegraphing to
| monetize what they have been doing. It also seems like this is
| why they don't release their models in a timely manner.
| minimaxir wrote:
| The notable difference is that the base Huggingface library is
| open source, so you could in theory build something similar or
| more custom to the OpenAI API internally (which then falls into
| the typical cost/benefit analysis of doing so).
| zitterbewegung wrote:
| So its like Github vs Gitlab which makes more sense. I can
| see huggingface have a hosted version because now you can
| share your models on their platform.
___________________________________________________________________
(page generated 2020-06-11 23:00 UTC)