[HN Gopher] Jacob Ziv has died
___________________________________________________________________
Jacob Ziv has died
Author : tkhattra
Score : 469 points
Date : 2023-03-26 17:11 UTC (5 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| k_eshav wrote:
| Rest in peace! <3
| superdisk wrote:
| RIP. I just got done implementing a modified version of LZSS
| which is crucial to fitting all the music into a game I'm
| writing. Respect.
| gotoeleven wrote:
| To everyone who is wondering, there is a black bar its just one
| pixel.
| floodle wrote:
| I love this :)
| phkahler wrote:
| I remember my first LZW implementation. It was so amusing to me
| the the one case where something didn't exist in the dictionary
| yet, you knew what it was and could add it and keep going!
| helpfulclippy wrote:
| It is fitting that such a large name is expressed in so few
| characters. RIP.
| fortran77 wrote:
| It's even one fewer in his native Hebrew: "y`qb zyv"
| golemiprague wrote:
| [dead]
| DeathArrow wrote:
| RIP. I remember when I was a young guy learning programming,
| reading the source code of various DOS based LZ, LZH or LZW
| compression utilities and implementing myself a LZH algorithm
| thinking it will bring me great fame and fortune. Lempel, Ziv and
| Eugene Roshall were my heroes.
|
| At that time many young programmers regarded compression,
| compiler writing and anti-virus writing as very important. Now
| people just learn React. :)
| jacquesm wrote:
| Compression and compiler writing are just as important today as
| they were back then. And the closer to the real end of cheap
| hardware improvements we get the more important they will be.
| They are where real progress lies, the rest was just a
| temporary freebie.
|
| Take cars: cars have a reasonable upper limit in physics so low
| that the difference between 'mild' (100 kph on the highway) and
| insane (300 kph+, no matter where) is a mere factor of three.
| And it took a good 100 years before that barrier was broken. It
| won't progress much beyond the standard of 100 or so kph under
| normal circumstances. There are just too many things that work
| against you when you go much faster than that including energy
| consumption, braking distance, the willingness of your vehicle
| to stay on the ground, attention requirements (ok, AI...
| someday...) and so on. So even if it is possible in exceptional
| cases it likely won't be the norm, even on closed over and
| evacuated freeways.
|
| Aircraft did a little better, but even there the supersonic
| passenger jet came _and_ went. Though it may be back one day.
|
| In comparison computers have gotten _many_ orders of magnitude
| faster in a much shorter time. But there is a physical limit,
| and we are very likely to run into that soon. And once that
| happens the only way to progress is to make more efficient use
| of the resources you 've already got.
| DeathArrow wrote:
| > In comparison computers have gotten many orders of
| magnitude faster in a much shorter time. But there is a
| physical limit, and we are very likely to run into that soon.
| And once that happens the only way to progress is to make
| more efficient use of the resources you've already got.
|
| Good points. And I will even argue that while we didn't hit a
| physical limit in hardware yet, it's still important to have
| a reasonable performance because not having performance means
| resources are wasted, money are wasted and it might lead to
| user frustration.
|
| But it seems to not be the majority's opinion or at least not
| most of the business owners opinion.
| freedomben wrote:
| Unfortunately until crappy and slow performing software
| starts losing business to well-implemented alternatives, it
| will never change. The current PM model is purely based on
| shipping fast, and any thought to performance is
| immediately dismissed as "pre-mature optimization." I've
| been consulting for companies that had to rolling reboot
| their entire cluster of app servers every couple of hours
| because a host of memory leaks (that also added O(n)
| runtime) would slowly exhaust resources. Rather than have
| someone fix the problem, they chose to invest in devops to
| automate the rolling restarts. This is not all that unusual
| in my experience, because customers will tolerate very
| sluggish response times so it's not worth fixing to the
| business.
| DeathArrow wrote:
| Entirely my experience. I worked for a very large company
| and since our microservices were running in Kubernetes it
| was totally OK for the services to crash. We just logged
| the things in elastic stack, used data dog to monitor,
| used some SRE to restart services when things were messy.
|
| I've lost days to solve a memory bug in one of the
| services and no one cared.
|
| The original architect and programmers were gone, the
| engineering managers were pushing Clean, SOLID and design
| patterns hopping that will help and the domain was hard
| because we had to deal with real money while respecting
| tens of different laws and locales.
|
| We tried at our best to do the impossible, we were
| understaffed by a factor of 5X.
|
| And the services were crashing like crazy but it looked
| like a no downtime because we provisioned another POD in
| Kubernetes.
|
| Provisioning was like: how many pods do we need? 3? Let's
| make it nine. How much memory do we need? Half a gig?
| Let's make it 4 gig to be sure.
|
| So it was a mess and it worked and that mess continues to
| work somehow.
|
| Not sure if mess driving engineering is a sound
| bussineses idea, though.
| colechristensen wrote:
| I don't agree. Aircraft had their growth phase earlier, first
| flight in 1903 to landing on the moon in 1969. Airliners look
| like they did in the 60s because there's nothing left to
| improve on with regards to efficiency. How many orders of
| magnitude are there between the wright flier and a 747?
|
| We're reaching the diminishing returns portion of the
| logistic curve for computing with regards to both things like
| hardware and compiling, compression, etc. It's just hard to
| compare industries which are done growing with ones that
| still feel like they might be exponential forever.
| DeathArrow wrote:
| > It's just hard to compare industries which are done
| growing with ones that still feel like they might be
| exponential forever.
|
| Some industries never evolved too much. I don't see a big
| functional difference between cars a 100 years ago and cars
| now. And there were electric cars in 1900s.
|
| A car is constraint by lots of things just like GP comment
| affirmed. Putting more gadgets in a car won't change what a
| car is or how it works at a fundamental level.
|
| From the very first microprocessors to today's there is a
| big leap. If we apply that leap to cars they should be
| using nuclear fusion and take us to moon and back in
| seconds. And it's not like trillions of dollars weren't
| invested in car R&D in more than a century.
|
| Software growth seems exponential only because of hardware
| and the Moore's Law. Which will come to an end soon.
|
| Thanks to Moore's Law, we programmers, afforded to be
| careless, not think about performance or use unnecessary
| abstractions just out of esthetical or ideologic
| considerations. It allowed businesses to say that time to
| market trumps everything and customers should simply buy
| more powerful boxes or more boxes. Now this will come to an
| end soon and we will have to think about efficiency, speed
| and performance.
|
| Or maybe chips will move from silicon to some alternative
| processes where growth is still possible.
| MontyCarloHall wrote:
| >At that time many young programmers regarded compression [...]
| as very important.
|
| They still do regard it as important, just in a different
| sense. Shannon formally proved that compression is congruent to
| statistical inference -- an efficient compression algorithm
| seeks to minimize the statistical entropy of the scheme used to
| encode a message. Compression is thus loosely congruent to a
| branch of statistical inference you may have heard a thing or
| two about called "artificial intelligence" [0], which many
| young programmers are very much engaged with.
|
| Take the Hutter Prize [1], which awards money for the ability
| to compress the entire English Wikipedia, or the Large Text
| Compression Benchmark [2], the latter of which's current top
| performer is a transformer-based neural network [3]. As the
| LTCP's rationale says [0],
|
| >Given a probability distribution P over strings s, the Shannon
| capacity theorem states that the optimal (shortest average)
| code for s has length log 1/P(s) bits. If the distribution P is
| known, then such codes are in fact easy to find and compute.
| The problem is that for natural language, we do not know how to
| explicitly compute the model P(s) for a given string s.
|
| >However humans must implicitly know P(s) because our speech
| and writing, by definition, follows this distribution. Now
| suppose that a compute can compute P(s) for all s. Then for any
| question q (or dialog ending in a question) and any answer a,
| the machine can compute P(a|q) = P(q,a)/P(q), and use this
| distribution to answer any question q posed by the judge. By
| definition, this distribution would be identical to the
| distribution of answers given by the human, and the two would
| be indistinguishable.
|
| [0] http://www.mattmahoney.net/dc/rationale.html
|
| [1] http://prize.hutter1.net/
|
| [2] http://www.mattmahoney.net/dc/text.html
|
| [3] https://bellard.org/nncp/
| DeathArrow wrote:
| While compression is congruent with some cases of statistical
| inference, no one is trying to do statistical inference using
| compression. Even the example you gave of Fabrice Bellard
| (which I highly admire) does the converse and uses neural
| networks to compress.
|
| Even for realizing what you've said one needs a bit of
| theory, basic computer science and some math. And most people
| today have a "do" philosophy and are result oriented. They
| are not learn oriented, they hate CS and math and they hate
| low level programming.
|
| I don't think compression is highly relevant today, but at
| the time I started learning it was a good tool to sharpen
| one's mind. Learn algorithms, learn about memory management,
| learn about hardware, learn low level constructs, learn a bit
| of math.
|
| Whenever a young person wanting to undertake a programmer
| career is asking me for advice I'll ask what their motivation
| is. If they just want money I advice them to learn the tools
| in fashion today, maybe Javascript and React, maybe Python,
| maybe C#. But if they respond they want to do it for both
| enjoyment and money, I put them on the hard path. I ask they
| enroll an BS CS program, or if that is not possible, give
| them long lists of books, tutorials, courses which cover CS
| basics, math used in CS basics, hardware, low level
| programming and the programming hot topics du jour. In that
| last case I also design their learning paths to accommodate
| their goals and I am always taking time to answer them
| questions in the future if I can, if not I will at least
| indicate where should they look or which is the right person
| to ask.
| vintermann wrote:
| > no one is trying to do statistical inference using
| compression
|
| Well, maybe no one sensible? I still think it's quite cool
| that you can take any general purpose compression
| algorithm, and abuse it to do e.g. image classification.
| (Just append the image you want to classify to each of the
| classes' sets in turn, and see which compresses best!)
|
| And actually I do remember a paper that tried to use ideas
| from PAQ to do online learning. Gated Linear Networks, out
| of DeepMind, in 2019:
|
| https://arxiv.org/abs/1910.01526
|
| All compression is related to AI, but especially sample
| efficient online learning is basically what data
| compression is all about.
| DeathArrow wrote:
| Well stochastic processes are a much wider category than
| mere algebraic operations applied on group theory that
| losseles data compression use.
|
| I'd think that calculus or functional theory or category
| theory can find more bijections towards statistics or
| even congruences than mere arithmetics or algebra ever
| will. Ok, you can explain or derive any mathematics
| construct using only algebra, and there were efforts to
| do so, but does it makes sense?
| CyberRabbi wrote:
| Oh no!
| theandrewbailey wrote:
| RIP.
|
| This is worthy of an HN black bar.
| mdaniel wrote:
| I'd bet that an email to hn@ would fix it; I don't believe the
| black bar is programmatic and thus maybe dang hasn't seen this
| yet
| einpoklum wrote:
| Two Professors Emeritus from my alam mater, the Technion - one
| from EE, the other from CS (Prof. Ziv). I distinctly remember
| noticing them in the old group photos of the first few graduating
| cohorts of the B.Sc. in CS.
|
| In Hebrew (or rather Aramaeic) we say:
|
| khbl `l dbdyn vl mshtkkhyn
|
| A pity due to those who absent and not present.
| golemiprague wrote:
| [dead]
| pfoof wrote:
| Lempel, Moore, Ziv... This year is very unlucky
| dogma1138 wrote:
| I'm a bit surprised that neither Lempel nor Ziv were worthy of a
| black bar here.
| Andrew_nenakhov wrote:
| Sad news. Just recently Abraham Lempel [0] died, just days after
| I tasked my 16 yo daughter to implement an lzw compression as a
| part of her programming education. I chose this particular task
| for her because many years ago I have implemented an lzw
| compression myself, years ago, learning a lot about bits, masks
| and gifs in the process. R.I.P.
|
| [0]: https://en.m.wikipedia.org/wiki/Abraham_Lempel
| userbinator wrote:
| I noticed your username after reading your comment, and was not
| surprised.
|
| For some reason, it seems the former USSR is highly prominent
| in compression algorithms. RAR, PAQ, encode.su, etc. and of
| course Andrey Markov.
| xattt wrote:
| > For some reason, it seems the former USSR is highly
| prominent in compression algorithms.
|
| I'd take a wild guess that it has to do with making more with
| having less, such as access to larger storage volumes or
| broadband.
|
| Specific to compression, when you're "tech poor", the time
| you spend waiting for data to get processed is essentially
| free.
| Andrew_nenakhov wrote:
| Incidentally, the guy who wrote RAR (and an incredibly great
| file manager for Windows, FAR Manager), Eugene Roshal, has
| studied at the very same faculty/specialty as I did. However,
| he had graduated before I entered the university.
| qup wrote:
| > I tasked my 16 yo daughter to implement an lzw compression as
| a part of her programming education
|
| Kudos, to the both of you.
|
| I'm sad we're starting to rapidly lose so many of our titans.
| londons_explore wrote:
| In other industries, the death of the inventor of a
| technology often marks the start of a further wave of
| innovation.
|
| For all kinds of reasons, the inventor (who is sometimes by
| now rich and/or powerful), may have a specific vision or path
| they want to proceed down. When they die, other people start
| investigating other paths, often with more success.
|
| I wonder if tech will see the same?
| colechristensen wrote:
| No. In tech when the creator quits or dies they're replaced
| by MBAs that squeeze every penny out of the creation. (Like
| authors whose grandchildren direct the trusts which own the
| copyright a century after the work was written)
|
| New creations in tech have to wait for patents to expire
| and the incumbents to crumble to the point where they don't
| have the cash to buy out the competition any more.
| qup wrote:
| To be fair, Ayn Rand said basically the same thing about
| industry long before tech bros existed.
| kortilla wrote:
| A bit tone deaf considering the thread is about
| algorithms that are open.
| MontyCarloHall wrote:
| I hope that an LZW compressor was something she would have
| naturally wanted to hack on herself, rather than something
| she's only coding because she was "tasked" to.
|
| The most surefire way to kill a young hacker's spirit is to
| force them to hack on things they don't care about. This goes
| with anything, not just computing. To whit, a childhood friend
| of mine was a gifted pianist who really loved jazz. His parents
| forced him to study with a classical teacher. Guess who doesn't
| play piano anymore?
| Andrew_nenakhov wrote:
| Well, she seems to be interested in programming and I don't
| force it on her. So far I think she does quite good. I have
| chosen lzw for her since my strategy is to give her quite a
| few diverse tasks to give her broad (albeit not very deep)
| understanding of general areas of development. I have covered
| this topic in more detail in a sibling comment here
| https://news.ycombinator.com/item?id=35318995
| serf wrote:
| another anecdote to balance :
|
| as a kid who played chess, the kids who were 'forced' to play
| chess -- the ones who came from a long chess playing family
| and had different scenarios drilled into their head from the
| day they could talk -- were much more talented at chess than
| I.
|
| Their talent persuaded them to continue pursuing it far
| longer than I ever did, with many of them still involved.
|
| The most involvement I have now is a friendly lichess game
| every few months.
|
| It was their talent that allowed them to find more enjoyment
| within the pursuit than I ever did, and that talent was
| essentially forced by their immediate family.
|
| So, essentially, I am just saying that forced persuasion
| thing that parents' sometimes do is a double-edged sword; it
| can create animosity and bitter hatred but it can also create
| the drive and motivation for boundless talent and personal
| satisfaction.
| hsbauauvhabzb wrote:
| I'm pretty good at computers but had no real guidance. I
| now hire young people with a genuine interest, then gift
| them equipment and guidance that will last them their
| entire lifetime (Even if they get a career in something
| else).
|
| There's such thing as pushing too hard, but nobody got buff
| without being pushed by those around them
| copymoro wrote:
| [flagged]
| _sword wrote:
| How do I delete someone else's comment
| jjice wrote:
| If you're daughter can knock out lzw at 16, she's going to be
| an incredible programmer.
|
| How do you go about teaching your kids? I don't have kids yet,
| but if they do have in interest in programming, I'd love to
| give them a strong head start like you have.
| Andrew_nenakhov wrote:
| Thanks, I think she has a potential to become a great
| developer, unless we all aren't replaced by that chatGPT
| thing everyone's talking about.
|
| Regarding teaching, I understood early that while I can
| program, I have very little understanding on how to teach it.
| So first, she underwent two Python courses (basic and
| advanced) on stepik.org [0], which had automated tests and
| have her an understanding of the language.
|
| Then, I planned to start throwing tasks of increasing
| complexity at her until she solves them, giving her a more or
| less complete coverage of all things programming. So far, she
| has written such things as
|
| - Tetris, in the console, with keyboard input
|
| - Snake game, with competing snakes which use wave algorithm
| to find path
|
| - LZW algorithm to learn bits and compression and other
| things
|
| Next my plan for her follows roughly this path:
|
| - some GUI programming, using GTK and Windows Forms.
| Calculator or maybe Checkers or something.
|
| - some web programming, backend and frontend. Most likely
| Django / jQuery
|
| - maybe some neural networks since are all so hot now
|
| - some hardcore assembler stuff, like maybe some simple game
| or visual demo
|
| - some mobile development, Android or iOS
|
| Once she'll be done with all that, I think she'll have quite
| a clear picture of most general areas of development and will
| be free to pursue the area she liked the most.
|
| [0]: https://stepik.org
| eointierney wrote:
| You are awesome and your daughter is awesomer
| userbinator wrote:
| _some hardcore assembler stuff, like maybe some simple or
| visual demo_
|
| Demoscene/sizecoding might be interesting, especially if
| she also likes artistic stuff.
| AnnikaL wrote:
| May his memory be a blessing.
| [deleted]
| tkhattra wrote:
| LZ77 and LZ78 are ubiquitous, forming the basis of compression
| schemes used in ZIP, PNG, GIF, and Yann Collet's Zstandard.
|
| [0] https://en.wikipedia.org/wiki/LZ77_and_LZ78
| Retr0id wrote:
| HN served this webpage to me using `content-encoding: gzip`,
| which is using LZ77 (combined with huffman coding) if you look
| far enough under the hood.
___________________________________________________________________
(page generated 2023-03-26 23:00 UTC)