[HN Gopher] The End of Big Data
___________________________________________________________________
The End of Big Data
Author : swyx
Score : 23 points
Date : 2022-05-26 08:13 UTC (14 hours ago)
(HTM) web link (benn.substack.com)
(TXT) w3m dump (benn.substack.com)
| bigcat12345678 wrote:
| Can someone summarize what the article want to say?
|
| I am not happy about spending 10 minutes on it but realized I
| need another 10 minutes, and in the end would still be confused,
| and would need post this comment anyway.
|
| So I just stopped there and wrote this comment.
|
| Rip writing clarity, and the effective communication...
| PLenz wrote:
| Big data didn't lose. It became so common that now what was once
| big data is just data.
| mr_toad wrote:
| > But we never connected these dots during our evaluation because
| the story Databricks told was buzzwords. Snowflake's was boring,
| and all we wanted--at least at first--was boring.
|
| How do you describe a chocolate gateau to someone who has only
| ever eaten rice?
|
| If you've never worked with tools like Pandas, or R, or SAS,
| you're just not going to understand Spark and Databricks.
| nonrandomstring wrote:
| > the end of an over-hyped era
|
| Hype is part of what drives technology and I don't see it going
| away any time soon - because a lot of people make a buck purely
| by milking hype cycles.
|
| I've come to see there are two modes of intelligence. The ability
| to understand something and say 'yes' to it is complemented by
| bullshit detectors, a more mature ability to see through other
| people muddying the waters trying to look smart, and reject their
| nonsense.
|
| Every cycle has its real innovators and hype-pedlars. At the
| height of blockhain madness a decade ago they had worked
| themselves into a frenzied fetish of near-supernatural woowooism.
| Once all the smoke and hullabaloo has cleared, and all the
| grandiose promises have fallen to the wayside, what remains is a
| core of genuinely useful and relatively simple core concepts that
| enter the canon, along with standards and protocols.
|
| This line in TFA sums it up:
|
| > I needed a database.
|
| The same happened with "data". The hype problem there has always
| been a lack of telos reminiscent of the underpants gnome's flawed
| "profit?" scheme. Somewhere is a crucial missing step, where we
| ask "Why?". And the answer is "don't worry, the data plus magical
| "AI" fairy dust will reveal why". There is a quite religious
| (faith-like) flavour to this.
|
| I blame the NSA. The idea that "collect it all" is anything but a
| thugish brute force excuse to waste of billions dollars building
| data-centres, has led to commercial obsession with "data" as a
| panacea.
|
| It isn't "big data" _per se_ that 's a problem. Some
| applications, like in medicine or environmental science,
| absolutely thrive on large sets and sophisticated analytics. But
| the fact is that in most applications it's mostly useless,
| burdensome, energy-consuming, and space-wasting. But "data-
| hoarding" and over-analysis is pushed by those with sledgehammers
| to sell for cracking nuts.
| Swizec wrote:
| > Once all the smoke and hullabaloo has cleared, and all the
| grandiose promises have fallen to the wayside, what remains is
| a core of genuinely useful and relatively simple core concepts
| that enter the canon, along with standards and protocols
|
| _has_ this happened with blockchain yet? I 'm not seeing it.
|
| Maybe I haven't been around the block enough because blockchain
| is the first hype tech I've seen that keeps not doing anything
| that existing tech can't do better. And yes I've looked, a lot.
|
| The underlying tech sounds super promising and I hope someone
| figures _something_ out. Not much luck so far.
| phil21 wrote:
| > has this happened with blockchain yet? I'm not seeing it.
|
| No one wants to admit it for some reason, but Bitcoin is the
| killer app for Blockchain.
|
| > The underlying tech sounds super promising and I hope
| someone figures something out. Not much luck so far.
|
| Plenty of luck for me at least. Bitcoin solved exactly the
| problem it told me it would when it was created -
| permissionless digital "cash" transactions that cannot be
| reversed. It's a very specific use case, but one I am quite
| happy exists when I need it.
|
| Those who have never had their money stolen by the banking
| system or denied the ability to use their money as they see
| fit likely won't understand my take. Those that do, are using
| the currency today.
___________________________________________________________________
(page generated 2022-05-26 23:00 UTC)