[HN Gopher] HSE: Heterogeneous-memory storage engine designed fo...
___________________________________________________________________
HSE: Heterogeneous-memory storage engine designed for SSDs
Author : caution
Score : 121 points
Date : 2020-04-27 20:34 UTC (2 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| [deleted]
| tiernano wrote:
| GitHub repo: https://github.com/hse-project
| haneefmubarak wrote:
| Looks pretty cool when you make it to the GitHub
| (https://github.com/hse-project). Order of magnitude performance
| gains! I imagine most of that come from skipping the Filesystem
| layer and just hitting the raw Block layer directly.
|
| I am curious about the durability and how well tested all of that
| is though. On the one hand, filesystems put a lot of work towards
| ensuring that bytes written to disk and synced are most likely
| durable, but OTOH Micron is a native SSD vendor so they've
| probably thought of that.
|
| I'm also curious whether RAIDing multiple SSDs together at the
| block layer and then running HSE on top of that will be faster or
| whether running multiple HSE instances (not the right word, it's
| a library, but you get what I mean) with one per drive and then
| executing redundantly across instances would be faster. Argument
| for the former is that each instance would have to redo the
| management work, argument for the latter is that there's probably
| synchronization overhead within the library so running more in
| parallel should allow for concurrency and parallelism gains.
| junaru wrote:
| Why is this press release getting massively upvoted?
| shockinglytrue wrote:
| It claims to fix MongoDB
| shockinglytrue wrote:
| Much better link: https://github.com/hse-project/hse
|
| PR is insane hot air referring to another hot air product (can
| you even buy their 3D Xpoint devices yet?)
| dang wrote:
| Ok, we've changed the URL to that from
| https://investors.micron.com/news-releases/news-release-
| deta....
| aloknnikhil wrote:
| > https://github.com/hse-project/hse
|
| Their benchmarks show significant gains compared to RocksDB.
|
| > https://github.com/spdk/rocksdb
|
| But what I'd really like to see is a comparison against RocksDB
| using SPDK
|
| > https://dqtibwqq6s6ux.cloudfront.net/download/papers/Hitachi...
| Based on these results, SPDK performs significantly better than
| the kernel requiring only 1-2 cores to saturate IOPS on an NVMe
| SSD (compared to the kernel requiring 16)
| wtallis wrote:
| I'd also like to see comparison to Toshiba/Kioxia's fork
| TRocksDB: https://github.com/KioxiaAmerica/trocksdb
| erulabs wrote:
| "World's first" Open-Source storage engine for SSDs? I believe
| Aerospike has advertised itself as that for years, and certainly
| most MongoDB instances are backed by SSD these days. Heck,
| conceptually etcd is a key-value storage engine built for SSDs.
|
| > HSE optimizes performance and endurance by orchestrating data
| placement across DRAM and multiple classes of SSDs or other
| solid-state storage.
|
| Orchestrating data placement? Isn't that what all storage engines
| do?
|
| What am I missing here? Is this a block level rather than file-
| system level driver?
| cryptonector wrote:
| Sounds kinda like a ZFS.
| elihu wrote:
| So, is this open-source firmware that runs directly on Micron
| SSDs, or is it an upper-layer thing that runs on the host system?
| buildbot wrote:
| I thought it was firmware too, but it appears to be more of a
| key value store block level access engine, and improves mongo
| performance.
|
| I got really excited thinking it was an open source nvme fpga
| core.
| drenvuk wrote:
| This is unbelievably cool. It is a multi segmented key prefix Key
| Value store. Can someone just strap paxos or raft to this and
| call it a day please? Pretty please?
| haivri wrote:
| I wonder how close of an API this provides compared to
| RocksDB... If close, CockroachDB might be a good trial
| candidate
| fortran77 wrote:
| Will this be a replacement for something like the overpriced and
| under performing proprietary products from Pure Storage?
| jandrewrogers wrote:
| PR copy aside, the claimed performance differences relative to
| RocksDB and WiredTiger are typical of many storage engines, the
| performance doesn't stand out. I don't think either RocksDB or WT
| has made a serious claim to prioritizing performance in their
| designs in any case.
|
| Also, I have to wonder how narrowly "open-source storage engine
| for SSDs" is being defined here such that it excludes so many
| earlier storage engines in claiming the title of "first".
| g14i wrote:
| Many techniques already used by Aerospike on their KV database,
| which also bypass the OS file system/cache.
|
| I'm a long time Aerospike user with no connection to Aerospike.
|
| Edit: I would love to see a benchmark with Aerospike.
___________________________________________________________________
(page generated 2020-04-27 23:00 UTC)