[HN Gopher] Claude Shannon's research laid foundations for moder... ___________________________________________________________________ Claude Shannon's research laid foundations for modern communications (2020) Author : ColinWright Score : 94 points Date : 2022-12-30 23:16 UTC (1 days ago) (HTM) web link (www.quantamagazine.org) (TXT) w3m dump (www.quantamagazine.org) | 2devnull wrote: | Was rereading "Fortune's Formula" this morning after dreaming | about Shannon last night. A name that should be more widely | recognized. | dang wrote: | Discussed at the time (of the article): | | _Claude Shannon Invented the Future_ - | https://news.ycombinator.com/item?id=25507627 - Dec 2020 (68 | comments) | rdxm wrote: | [dead] | amelius wrote: | Not just communication, but many other fields. E.g. deep learning | is rooted in information theory. | aborsy wrote: | Well, the notion of bit as the unit of information, as well as | digitization and digital communication, came primarily out of | Shannon's work. | LudwigNagasena wrote: | The Kelly criterion that is widely used in finance was invented | at the Bell Labs. It was based on the Shannon's theory. | mhh__ wrote: | > widely used | | A nice idea but not really. | User23 wrote: | I use it every time I'm in Vegas. | bee_rider wrote: | Shannon's work has already produced a whole lot of course, but I | don't think we're anywhere near seeing the full outcome of his | contributions yet. | | Currently our computers operate by filling the transistors with | charge and/or dumping it to ground. Who even cares about in | information-theoretic efficiency in that case? The cost of the | actual work done is dwarfed by the ancillary cost of running the | machine. | | If we ever move on to something less brute-force, like reversible | quantum cellular automata, I think we'll see him as an invaluable | part in the chain of formalizing what information and computation | mean physically. | | Kelvin/Maxwell -> Shannon -> Landauer -> maybe Bennett | notlukesky wrote: | The suggestion to use the word entropy by von Neumann is a great | story: | | My greatest concern was what to call it. I thought of calling it | 'information,' but the word was overly used, so I decided to call | it 'uncertainty.' When I discussed it with John von Neumann, he | had a better idea. Von Neumann told me, 'You should call it | entropy, for two reasons. In the first place your uncertainty | function has been used in statistical mechanics under that name, | so it already has a name. In the second place, and more | important, no one really knows what entropy really is, so in a | debate you will always have the advantage.' | | https://mathoverflow.net/questions/403036/john-von-neumanns-... | mitthrowaway2 wrote: | It's a funny story, but now that we do understand better what | entropy is it's clear that information and thermodynamic | entropy are the same concept, so it was a very good call. | User23 wrote: | Von Neumann was incredibly clever. | mitthrowaway2 wrote: | Yes, it's likely he also understood that they were the same | concept, and his recommendation was less of a joke than his | words suggest. | bee_rider wrote: | von Neumann was an alien sent to Earth when we were about | to get/were in the process of getting nukes, to make sure | we advanced fast enough to not blow ourselves up. And the | wild thing is that you, the reader, are only like 75% sure | I'm kidding and don't actually believe this. | rogerkirkness wrote: | Certainly it seems like Shannon is turning out to be the lindy | Nicola Tesla equivalently impactful person but in the realm of | bits. | 082349872349872 wrote: | > _...in the realm of bits._ | | Especially as an OG (first author to use the word?): | https://people.math.harvard.edu/~ctm/home/text/others/shanno... | | > _If the base 2 is used the resulting units may be called | binary digits, or more briefly_ bits _, a word suggested by J. | W. Tukey._ | | [Edit: also, consider the section "3. THE SERIES OF | APPROXIMATIONS TO ENGLISH" to be an early exploration of MLMs: | Microscopic Language Models] | kkylin wrote: | John Tukey also invented FFT and apparently was the first to | use "software" in a publication: | | https://en.wikipedia.org/wiki/John_Tukey ___________________________________________________________________ (page generated 2023-01-01 23:00 UTC)