[HN Gopher] The history of Berkeley DB
___________________________________________________________________
The history of Berkeley DB
Author : yarapavan
Score : 76 points
Date : 2021-11-20 17:36 UTC (5 hours ago)
(HTM) web link (queue.acm.org)
(TXT) w3m dump (queue.acm.org)
| tbrock wrote:
| I would have given almost anything to be there and work on this
| with them or sleepycat's successor wired tiger.
|
| So much fun to implement this stuff and see it provide so much
| value.
| dboreham wrote:
| It was the best of times.
| dalke wrote:
| "after I quit a miserable job"
|
| I was there. He was my boss. It was miserable.
| junon wrote:
| Because of him? Or because of the job?
| dalke wrote:
| The job was miserable. He was VP of engineering. He, I, the
| VP of marketing and I think someone else left within the same
| week because things weren't working out. (It was only a 50
| person company.)
|
| The company got bought out, mostly as acqui-hire for the
| scientific staff.
|
| The main tech was by a Stanford prof. and one of his ex-grad
| students. An X-Windows/IRIX-only(!) based tool for molecular
| modeling. It added support for integrating bioinformatics
| analyses, gathered from public web servers. Which was cool,
| but companies wouldn't buy it because sending proprietary
| data to public sites as a big N-O.
|
| Many companies had similar resources in-house. The program
| could call out to awk or shell as a configuration language
| for doing those searches. But our customers were all
| switching to Perl and didn't care for those languages. So,
| late in the cycle we added Perl integration support.
|
| There was only one person who could really work on the core
| tool. (I remember fixing ~10,000 compiler warnings, nearly
| all because it used a home-brew GUI based on void* function
| pointers.) There had been a push to port it to the Mac, but
| that took a year and the fork was unmergeable.
|
| We developed an intranet server product for basic
| bioinformatics tools in perl that would mirror the core
| bioinformatics databases, process them for FASTA, BLAST, and
| text searches. Mind you, this was 1998, so CGI.pm and a home-
| brew templating language I threw together.
|
| We needed this because we were ramping up science consulting
| (I don't recall why any more), and needed our own internal
| tools because proprietary data.
|
| So, Mike came in as VP of engineering, with a product that
| was hard to maintain, nor could it quite pivot to the new
| hotness of bioinformatics, nor pivot to the new hotness of
| the web. (Get this: there was a proof-of-concept version that
| used a persistent version of the tool running on the
| developer's desktop, in an off-screen display buffer, which
| presented a clickable image map to the user. Each click would
| round-trip another screen shot to the running program.) And
| without the ability to really hire the people to get out of
| that hole, because funding was running out.
|
| On the organization level, there was also then-new CEO, who
| sales and marketing referred to as a tornado, because she
| would come in and everything would get blown around and mixed
| up. Not a good boat to be in.
|
| This was also Mike's first VP-level management job, so rather
| like jumping into the deep end.
|
| Mind you, I was a fresh-behind-the-ears 27 year old, with
| only 2 years of professional experience in an academic lab. I
| know little about what the politics was with upper-
| management. I can agree that it was miserable.
|
| Going back to Sleepycat, I actually evaluated BDB vs GDB vs.
| a few other technologies for one of our projects, and ended
| up having problems with all of them. Only after I left, when
| I met Mike after he started with Sleepycat, did I point out
| that I he probably should have told me about his connections
| with BDB. He agreed. ;)
| hbarka wrote:
| Wow thanks for sharing that
| 0xdky wrote:
| I used BDB as a Win32 profiler backend. The profiler was
| lightweight and would write a flat file with profiling data and
| function addresses to keep the captured data small.
|
| A post processing tool would read the profiler data and create a
| BDB file with support for extracting call graphs and topN sort of
| analysis.
|
| The final GUI was implemented in Visual Basic since other
| developers would not use the TUI/CLI based tools in console.
|
| The next project used BDB to store file system metadata on
| embedded NAS storage. We implemented a fast 'find' like service
| based on file metadata (stat fields) stored in BDB with support
| for user defined file metadata.
| bch wrote:
| Oh goodness - I'm just getting Bostics Sleepycat -> WiredTiger
| play now. Well done Keith.
| buserror wrote:
| I've used Berkeley DB for decades, in many projects, many time
| having to have a yelling match with other people who wanted to
| use one of the large spaghetti plate "database".. this was before
| the "noSQL" trend, and even after. How many projects /need/
| network access etc, in a LOT of cases you just want a file
| backed, very quick library and that does the job.
|
| For more complicated setups, sqlite also works wonder.
| kstrauser wrote:
| BDB is brilliant. SQLite is my go-to now for new projects as
| it's nearly as good as BDB in most ways, and much better the
| second you need rich queries.
| ozgune wrote:
| > Chris eventually went to Amazon and brought Berkeley DB with
| him there, so we became one of the first backing stores for
| Amazon's Dynamo key-value store.
|
| BDB was impressive in that you'd file a bug with them & they
| would have a patch for you by the next day. If you did the same
| thing with Oracle, stars would have to align even to get an ack.
|
| I think that's because they kept BDB pretty simple. I happened to
| sit in a meeting with Margo Seltzer where the (later
| distinguished) engineer from Amazon was asking for new features.
| I think Margo must have said no to every feature request because
| it didn't align with their roadmap. I've never seen anyone do
| that. It was impressive to watch.
|
| Then again, Oracle acquired Sleepycat in the end.
| zealsham wrote:
| Berkeley DB is such an important piece of software today . Its
| one if the essential parts of bitcoin
___________________________________________________________________
(page generated 2021-11-20 23:00 UTC)