[HN Gopher] Information Theory: A Tutorial Introduction ___________________________________________________________________ Information Theory: A Tutorial Introduction Author : teleforce Score : 172 points Date : 2021-06-26 15:52 UTC (7 hours ago) (HTM) web link (arxiv.org) (TXT) w3m dump (arxiv.org) | jostmey wrote: | I think of information theory as an entirely new branch of | mathematics because it represents a new way of quantitatively | thinking. Calculus provided a different way to think about | problems separate from geometry. Information theory let's me | think about problems _differently_. | phonebucket wrote: | Another fantastic (and free!) Information Theory resource is | David MacKay's book, which also addresses aspects of Bayesian | Statistics and Machine Learning: | http://www.inference.org.uk/mackay/itila/ | | It's among my favourite textbooks: you can feel his enthusiasm | and personality through the pages. The world is a poorer place | without him around. | vmilner wrote: | See also his video course: | | https://www.youtube.com/watch?v=BCiZc0n6COY | elromulous wrote: | Written some 60 years ago, Information Theory and Coding by | Abramson[1] is an absolute gem for those looking to get into info | theory. The cover book [2] being the more complete resource (and | somewhat of a grad level defacto standard text). | | [1] https://www.amazon.com/Information-Theory-Coding-Norman- | Abra... | | [2] https://www.amazon.com/Elements-Information-Theory- | Telecommu... | mcguire wrote: | " _To mistake a binary digit for a bit is a category error. In | this case, the category error is not as bad as mistaking marzipan | for justice, but it is analogous to mistaking a pint-sized bottle | for a pint of milk._ " | | That's well put. | chrispeel wrote: | Shannon's original paper "A Mathematical Theory of Communication" | which introduced information theory is so accessible as to be | useful as a tutorial itself: | | http://people.math.harvard.edu/~ctm/home/text/others/shannon... | Swizec wrote: | +1 for the original paper. I used it to pass my information | theory class in college because it made more sense than the | official materials and my lecture notes. | | It's a fantastic explanation of the theory. | m-hilgendorf wrote: | The edition with Weaver's introduction is also very good, | especially if you would like the context and ramifications of | Shannon's theory. | | https://pure.mpg.de/rest/items/item_2383164/component/file_2... | haolez wrote: | Except that some of the notation has changed a little. But it's | insanely good and impactful for a piece that is only 8 pages | long, if I remember correctly. | anon_tor_12345 wrote: | it's 55 pages | haolez wrote: | His master's thesis was much smaller, if I remember | correctly. Gonna check with my ex teacher :) | duped wrote: | His master's thesis was on applying boolean algebra to | the design of switching circuits, which was similarly | groundbreaking (although discovered in parallel by | another). ___________________________________________________________________ (page generated 2021-06-26 23:00 UTC)