[HN Gopher] Replacing Elasticsearch with Rust and SQLite (2017) ___________________________________________________________________ Replacing Elasticsearch with Rust and SQLite (2017) Author : nethunters Score : 81 points Date : 2021-05-16 17:24 UTC (5 hours ago) (HTM) web link (nickb.dev) (TXT) w3m dump (nickb.dev) | lsb wrote: | I wonder how effective this would be with the SQLite with a new | virtual file system fetching pages via XHR that someone posted | last month. Then you would be able to do full text search on a | huge database only sending a few KB per search | | https://news.ycombinator.com/item?id=27016630 | [deleted] | polyrand wrote: | I think replacing Elasticsearch with SQLite is a great idea. Even | more if you use the full-text search functions that SQLite | includes. | | I recommend checking out scout[0], which, I think, can be a good | replacement for Elasticsearch in some cases. I'm also working on | an Elasticsearch replacement built on top of SQLite for my | litements[1] project, but it will still take a few weeks to have | a working version. | | [0] https://github.com/coleifer/scout [1] | https://github.com/litements/ | fabbari wrote: | It definitely depends on what you're doing with it. In the | article the author is using it to parse only nginx logs to | report page hits counts - I probably would have not gone with | Elasticsearch to begin with. I find it strange that the issue | with InfluxDB was the lack of 'Group by'; you can use | 'Distinct' with other aggregation functions with the same | result - and without having to write and maintain your own | stack. | | I appreciate - and often indulge - in writing stacks of | software that are better implemented other open source, but I | wouldn't say that the article describes a solution that is | viable as a replacement for Elasticsearch. | nethunters wrote: | I think it's best put as a replacement for ElasticSearch for | those who don't really need ElasticSearch. | | Edit - i.e. where ES is overkill for the use case, however | for an appropriate use case ElasticSearch is amazing | xbar wrote: | I didn't read the article, but I think this is a sizable | audience. | Anonymous4272 wrote: | Similar to litements litecache is DiskCache[0]. It is a pure | python Redis/Memcached alternative which is faster than them | for reads. Under the hood it uses shared Sqlite. | | [0] https://github.com/grantjenks/python-diskcache | polyrand wrote: | Yes, I'm aware of DiskCache. Wonderful library and also very | useful, I enjoyed reading the source code too. I have taken a | few ideas from it that I would like to implement in the | future. | RcouF1uZ4gsC wrote: | > This query can't be represented in Diesel's DSL (a sample of | the DSL is demonstrated in Diesel's getting started). | | I feel like most of the time, SQL DSLs end up being bad. SQL is | already a very high level language. There are a ton of examples | of how to do stuff with SQL queries. DSLs, even if they have a | way to express the SQL, are not nearly as widespread as SQL and | then you will have to spend time figuring out how to translate | the SQL into the DSL. | | In addition, with libraries like sqlx for Rust, you can also get | the type safety of a DSL using regular SQL. | | I would say that as a developer, the time spent getting familiar | with SQL is a good investment that will likely pay off across | many projects and programming languages. | pvillano wrote: | also if you can, set up your logging to output JSON/GELF. | leetrout wrote: | I'm always surprised when people haven't heard of Xapian which is | also in this space. | | I tell people Xapian is the SQLite of search. | | https://xapian.org/ ___________________________________________________________________ (page generated 2021-05-16 23:00 UTC)