[HN Gopher] A Database for 2022
___________________________________________________________________
A Database for 2022
Author : tosh
Score : 99 points
Date : 2022-04-01 20:33 UTC (2 hours ago)
(HTM) web link (tailscale.com)
(TXT) w3m dump (tailscale.com)
| musjleman wrote:
| So to recap their timeline: * Instead of an
| actual database use a JSON file. * Write a blog post about
| how that didn't scale. * Instead of an actual database
| hand-roll something else. * Write a blog post about how
| that didn't scale. * Instead of a database with built-in
| replication which is tailor built for key-value storage use
| SQLite with a single table containing key-value pairs and some
| fresh glue to make it replicate.
|
| I'm sorry, but what the fuck? Just writing the blog posts alone
| might have consumed more work-hours than what it would have taken
| to set a database up in the first place.
|
| Are we supposed to expect another blog post somewhere around a
| year down the road titled "SQLite didn't scale" or "The new
| library for replication that was released a few months ago
| contained a bug that took our prod down"?
| ithkuil wrote:
| > This gives you near real time backups (*or with a couple deft
| modifications, lets your app block at critical sections until the
| backup is done*)
|
| anyone has an example of how to do this?
| ramenmeal wrote:
| I think I'm missing some context. Using a text file and using
| etcd as a DB for a production system seems like a terrible
| engineering decision. It seems like something you'd do as a proof
| of concept or side project. It's interesting that they're
| blogging about this, as if they're proud of it. I guess I'm just
| missing the point. This is their 3rd DB Migration, something that
| I prefer to avoid at all costs. I guess they have different
| priorities and values. But I'm just confused, why would an
| engineer want to join a company that is making these decisions?
| Why would the company want their users to know about these
| decisions? They started using etcd knowing that he would become a
| bottle neck, that should have been a non-starter, right?
| rudasn wrote:
| This is an April fool's joke, right?
|
| Edit: no it's not..
|
| > Footnote: coworkers point out it's April Fool's today and
| request that I clarify this isn't a joke. Joke's on them: every
| day's April Fool's in the Tailscale Database Engineering
| department.
| maximilianroos wrote:
| I see a dozen comments here, all critiquing their proposal.
| Surely this is April Fools?
|
| > I'm sorry, but what the fuck? Just writing the blog posts alone
| might have consumed more work-hours than what it would have taken
| to set a database up in the first place.
|
| > Using a text file and using etcd as a DB for a production
| system seems like a terrible engineering decision. It seems like
| something you'd do as a proof of concept or side project. It's
| interesting that they're blogging about this, as if they're proud
| of it.
|
| > This is their 3rd DB Migration, something that I prefer to
| avoid at all costs. I guess they have different priorities and
| values. But I'm just confused, why would an engineer want to join
| a company that is making these decisions?
| endisneigh wrote:
| I'm so desperate for a SQL database where I can just put it on a
| bunch of commodity hardware via Docker, connect them up and never
| worry about this again. Ideally it'd monitor may queries and
| create indexes for me. pleeeeeeaaaaaaaaaaaassssseeeeeeeeee
|
| /dream
|
| FoundationDB is similar but you have to do so much yourself. It's
| more or less what I'm describing for a key value store though.
| Not sure why it's not more popular.
| btgeekboy wrote:
| FoundationDB is an Apple thing, isn't it? Might explain why. It
| feels kinda like they've put it out there and it just hasn't
| had anyone with a marketing budget to push it along.
| Nican wrote:
| I am having quite a good time with CockroachDB. It has been
| mostly set-and-forget, and it auto-balances, and does rolling
| upgrades pretty smoothly.
| AtlasBarfed wrote:
| ... this is just replication of a database? As in the full
| database per node/replica?
|
| Hasn't AWS's SQL db-as-a-services had this for years now?
| butlerm wrote:
| Pretty sure you could track down instances of this technology
| going back decades now. Nothing is new under the sun. Doesn't
| mean you can use it.
| detaro wrote:
| I don't think they are claiming to have invented database
| replication, so what?
| timcavel wrote:
| infogulch wrote:
| They're using litestream [1] to replicate a SQLite database,
| which streams additions to the SQLite WAL file to object storage
| and can replay it back. This is fairly hands-off from SQLite's
| perspective. There's also the SQLite Session Extension [2] that
| is a built-in way to support generating and applying "patches".
|
| I'm curious how these tools will mature, it seems like a good
| match for microservices.
|
| [1]: https://github.com/benbjohnson/litestream
|
| [2]: https://www.sqlite.org/sessionintro.html
| silisili wrote:
| I watched an interview of Mr. Hipp, creator of SQLite, that I
| can't find now but was pretty interesting. Aside from being way
| different than I expected for some reason, in a good way - very
| down to earth and friendly, he was asked specifically about
| that, and more or less answered that his job was to write a
| solid DB, and replication can be done elsewhere. That's a
| pretty honest answer, and looks like someone took up the
| challenge.
| Xeoncross wrote:
| The litestream project was created by
| https://github.com/benbjohnson who wrote
| https://github.com/boltdb/bolt (a key value store) which has
| been instrumental (from my point of view) in the Go community
| as one of the original choices for an embedded database as it
| was not only fast, but had transactions with stable snapshots.
|
| It was used by https://github.com/blevesearch/bleve,
| https://github.com/etcd-io/etcd, and number of other projects.
|
| These days, https://github.com/dgraph-io/badger is often
| favored because of it's improved write throughput.
| otterley wrote:
| It should be noted that BoltDB has been retired as it
| suffered from a number of congenital defects. It was
| implicated in the major Roblox outage that happened a couple
| months ago, and Consul doesn't use it anymore.
|
| https://news.ycombinator.com/item?id=30015913
|
| (This comment is _not_ to suggest that the Litestream project
| is of poor quality or anything like that.)
| benbjohnson wrote:
| BoltDB author here. Just to clarify, the project was
| retired because of maintenance burden. CoreOS wanted to
| make some changes but I didn't have the bandwidth to test
| and maintain it all. Since this was before Go had good
| version management, we decided that they could fork the
| project as "bbolt" and users could move over as needed.
| They did a good job maintaining it so I eventually archived
| the original project.
| infogulch wrote:
| Another one of Ben's projects is on the front page right now:
|
| Postgres wire compatible SQLite proxy -
| https://news.ycombinator.com/item?id=30875837
| benbjohnson wrote:
| Bolt author here. Badger is a good database but it's mostly a
| tradeoff of write versus read throughput, specifically, range
| queries. Badger is an LSM so it doesn't perform as well (last
| time I checked) with iterating over ordered key/value pairs.
| LSMs have bloom filters to speed up point queries so those
| aren't as much of an issue.
| KwisaksHaderach wrote:
| _The obvious candidates were MySQL (or one of its renamed
| variants given who bought it) or PostgreSQL, but several of us on
| the team have operational experience running these databases and
| didn't enjoy the prospect of wrestling with the ops overhead of
| making live replication work and behave well. Other databases
| like CockroachDB looked very tempting, but we had zero experience
| with it. And we didn't want to lock ourselves into a cloud
| provider with a managed product like Spanner._
|
| What about managed mysql/posgresql? no lock-in and installing
| them locally is trivial.
| butlerm wrote:
| If you install it locally, that is pretty much the definition
| of an unmanaged database. Other than by the user of course.
___________________________________________________________________
(page generated 2022-04-01 23:00 UTC)