[HN Gopher] Claude Shannon's research laid foundations for moder...
       ___________________________________________________________________
        
       Claude Shannon's research laid foundations for modern
       communications (2020)
        
       Author : ColinWright
       Score  : 94 points
       Date   : 2022-12-30 23:16 UTC (1 days ago)
        
 (HTM) web link (www.quantamagazine.org)
 (TXT) w3m dump (www.quantamagazine.org)
        
       | 2devnull wrote:
       | Was rereading "Fortune's Formula" this morning after dreaming
       | about Shannon last night. A name that should be more widely
       | recognized.
        
       | dang wrote:
       | Discussed at the time (of the article):
       | 
       |  _Claude Shannon Invented the Future_ -
       | https://news.ycombinator.com/item?id=25507627 - Dec 2020 (68
       | comments)
        
       | rdxm wrote:
       | [dead]
        
       | amelius wrote:
       | Not just communication, but many other fields. E.g. deep learning
       | is rooted in information theory.
        
         | aborsy wrote:
         | Well, the notion of bit as the unit of information, as well as
         | digitization and digital communication, came primarily out of
         | Shannon's work.
        
         | LudwigNagasena wrote:
         | The Kelly criterion that is widely used in finance was invented
         | at the Bell Labs. It was based on the Shannon's theory.
        
           | mhh__ wrote:
           | > widely used
           | 
           | A nice idea but not really.
        
             | User23 wrote:
             | I use it every time I'm in Vegas.
        
       | bee_rider wrote:
       | Shannon's work has already produced a whole lot of course, but I
       | don't think we're anywhere near seeing the full outcome of his
       | contributions yet.
       | 
       | Currently our computers operate by filling the transistors with
       | charge and/or dumping it to ground. Who even cares about in
       | information-theoretic efficiency in that case? The cost of the
       | actual work done is dwarfed by the ancillary cost of running the
       | machine.
       | 
       | If we ever move on to something less brute-force, like reversible
       | quantum cellular automata, I think we'll see him as an invaluable
       | part in the chain of formalizing what information and computation
       | mean physically.
       | 
       | Kelvin/Maxwell -> Shannon -> Landauer -> maybe Bennett
        
       | notlukesky wrote:
       | The suggestion to use the word entropy by von Neumann is a great
       | story:
       | 
       | My greatest concern was what to call it. I thought of calling it
       | 'information,' but the word was overly used, so I decided to call
       | it 'uncertainty.' When I discussed it with John von Neumann, he
       | had a better idea. Von Neumann told me, 'You should call it
       | entropy, for two reasons. In the first place your uncertainty
       | function has been used in statistical mechanics under that name,
       | so it already has a name. In the second place, and more
       | important, no one really knows what entropy really is, so in a
       | debate you will always have the advantage.'
       | 
       | https://mathoverflow.net/questions/403036/john-von-neumanns-...
        
         | mitthrowaway2 wrote:
         | It's a funny story, but now that we do understand better what
         | entropy is it's clear that information and thermodynamic
         | entropy are the same concept, so it was a very good call.
        
           | User23 wrote:
           | Von Neumann was incredibly clever.
        
             | mitthrowaway2 wrote:
             | Yes, it's likely he also understood that they were the same
             | concept, and his recommendation was less of a joke than his
             | words suggest.
        
             | bee_rider wrote:
             | von Neumann was an alien sent to Earth when we were about
             | to get/were in the process of getting nukes, to make sure
             | we advanced fast enough to not blow ourselves up. And the
             | wild thing is that you, the reader, are only like 75% sure
             | I'm kidding and don't actually believe this.
        
       | rogerkirkness wrote:
       | Certainly it seems like Shannon is turning out to be the lindy
       | Nicola Tesla equivalently impactful person but in the realm of
       | bits.
        
         | 082349872349872 wrote:
         | > _...in the realm of bits._
         | 
         | Especially as an OG (first author to use the word?):
         | https://people.math.harvard.edu/~ctm/home/text/others/shanno...
         | 
         | > _If the base 2 is used the resulting units may be called
         | binary digits, or more briefly_ bits _, a word suggested by J.
         | W. Tukey._
         | 
         | [Edit: also, consider the section "3. THE SERIES OF
         | APPROXIMATIONS TO ENGLISH" to be an early exploration of MLMs:
         | Microscopic Language Models]
        
           | kkylin wrote:
           | John Tukey also invented FFT and apparently was the first to
           | use "software" in a publication:
           | 
           | https://en.wikipedia.org/wiki/John_Tukey
        
       ___________________________________________________________________
       (page generated 2023-01-01 23:00 UTC)