[HN Gopher] On "Learning to Summarize"
       ___________________________________________________________________
        
       On "Learning to Summarize"
        
       Author : luu
       Score  : 24 points
       Date   : 2020-09-12 04:55 UTC (18 hours ago)
        
 (HTM) web link (nostalgebraist.tumblr.com)
 (TXT) w3m dump (nostalgebraist.tumblr.com)
        
       | simonebrunozzi wrote:
       | I initially thought this would be about "humans" learning to
       | summarize longer texts in a smart way. It's a topic dear to my
       | heart; in fact, I even launched a little side project a few years
       | back, called MNMN [0], with the aim of providing summarization of
       | interesting articles or discussions on the web. I started with my
       | own itch to scratch: Hacker News homepage, which I religiously
       | read pretty much every day.
       | 
       | Here's an example of how the Hacker News home page would be
       | summarized: [1].
       | 
       | Instead, this "learning to summarize" article is about another
       | interesting topic, which is how you teach an AI (GPT-2 and GPT-3)
       | how to summarize text. It's also - bear with me - something dear
       | to my heart, as I used to teach compilers at a CS course in an
       | Italian university in 2004-2006, and I developed an interest for
       | languages in general (not just computer ones).
       | 
       | This one below is the central point of the article, and it is
       | indeed a crucial part of having success with an AI (note: LM
       | stands for Language Model):
       | 
       | > IMO there are two almost unrelated ideas going on in OpenAI's
       | preference learning work.
       | 
       | > First, the idea of collecting binary preference annotations on
       | LM samples, and (in some way) tuning the LM so its samples are
       | better aligned with the preferences.
       | 
       | > Second, a specific method for tuning the sampling behavior of
       | LMs to maximize an (arbitrary) score function defined over entire
       | samples.
       | 
       | We are, IMHO, at the cusp of a true revolution in linguistics.
       | Can't wait to see what happens in the coming 18-24 months. I
       | expect to be blown away on at least a few fronts.
       | 
       | [0]: https://github.com/simonebrunozzi/MNMN
       | 
       | [1]: https://github.com/simonebrunozzi/MNMN/blob/master/Weekly-
       | Su...
        
       ___________________________________________________________________
       (page generated 2020-09-12 23:00 UTC)