[HN Gopher] On "Learning to Summarize" ___________________________________________________________________ On "Learning to Summarize" Author : luu Score : 24 points Date : 2020-09-12 04:55 UTC (18 hours ago) (HTM) web link (nostalgebraist.tumblr.com) (TXT) w3m dump (nostalgebraist.tumblr.com) | simonebrunozzi wrote: | I initially thought this would be about "humans" learning to | summarize longer texts in a smart way. It's a topic dear to my | heart; in fact, I even launched a little side project a few years | back, called MNMN [0], with the aim of providing summarization of | interesting articles or discussions on the web. I started with my | own itch to scratch: Hacker News homepage, which I religiously | read pretty much every day. | | Here's an example of how the Hacker News home page would be | summarized: [1]. | | Instead, this "learning to summarize" article is about another | interesting topic, which is how you teach an AI (GPT-2 and GPT-3) | how to summarize text. It's also - bear with me - something dear | to my heart, as I used to teach compilers at a CS course in an | Italian university in 2004-2006, and I developed an interest for | languages in general (not just computer ones). | | This one below is the central point of the article, and it is | indeed a crucial part of having success with an AI (note: LM | stands for Language Model): | | > IMO there are two almost unrelated ideas going on in OpenAI's | preference learning work. | | > First, the idea of collecting binary preference annotations on | LM samples, and (in some way) tuning the LM so its samples are | better aligned with the preferences. | | > Second, a specific method for tuning the sampling behavior of | LMs to maximize an (arbitrary) score function defined over entire | samples. | | We are, IMHO, at the cusp of a true revolution in linguistics. | Can't wait to see what happens in the coming 18-24 months. I | expect to be blown away on at least a few fronts. | | [0]: https://github.com/simonebrunozzi/MNMN | | [1]: https://github.com/simonebrunozzi/MNMN/blob/master/Weekly- | Su... ___________________________________________________________________ (page generated 2020-09-12 23:00 UTC)