[HN Gopher] Datomic is Free ___________________________________________________________________ Datomic is Free Author : xmlblog Score : 780 points Date : 2023-04-27 13:40 UTC (9 hours ago) (HTM) web link (blog.datomic.com) (TXT) w3m dump (blog.datomic.com) | sdfjkl wrote: | Where's the source? Why does it say "Pricing" on the website if | it's free? This isn't free, it's just an attempt at free | publicity. | wartijn_ wrote: | The pricing page mentions it's free. | | > This section only applies to Datomic 990-9202 and lower. | Newer versions of Datomic Cloud will be free of licensing | related costs, and you will only pay for the hardware that you | use to run the system. | CrimsonCape wrote: | Since the conversation seems to be focusing on the Apache 2.0 | license, what would you do? Clearly there isn't a lot of | precedent for "closed-source, free-to-use" licenses. | | In this case Datomic maintains development control over their | product and "source of truth" is still themselves, and the | implicit assumption is that you enthusiastically use their | product for free with no strings attached because you respect | them as the source of truth. | avodonosov wrote: | If you work with Datomic, my little helper functions may be | useful: https://github.com/avodonosov/datomic-helpers | augustl wrote: | Datomic's is perfect for probably 90% of small-ish backoffice | systems that never has to be web scale (i.e. most of what I do at | work). | | Writing in a single thread removes a whole host of problems in | understanding (and implementing) how data changes over time. (And | a busy MVCC sql db spends 75% of its time doing coordination, not | actual writes, so a single thread applying a queue of | transactions in sequence can be faster than your gut feeling | might tell you.) | | Transactions as first-class entities of the system means you can | easily add meta-data for every change in the system that explains | who and why the change happened, so you'll never again have to | wonder "hmm, why does that column have that value, and how did it | happen". Once you get used to this, doing UPDATE in SQL feels | pretty weird, as the default mode of operation of your _business | data_ is to delete data, with no trace of who and why! | | Having the value of the entire database at a point in time | available to your business logic as a (lazy) immutable value you | can run queries on opens up completely new ways of writing code, | and lets your database follow "functional core, imperative | shell". Someone needs to have the working set of your database in | memory, why shouldn't it be your app server and business logic? | | Looking forward to see what this does for the adoption of | Datomic! | brundolf wrote: | Something I've been curious about: how well (or badly) would it | scale to do something similar on a normal relational DB (say, | Postgres)? | | You could have one or more append-only tables that store | events/transactions/whatever you want to call them, and then | materialized-views (or whatever) which gather that history into | a "current state" of "entities", as needed | | If eventual-consistency is acceptable, it seems like you could | aggressively cache and/or distribute reads. Maybe you could | even do clever stuff like recomputing state only from the last | event you had, instead of from scratch every time | | How bad of an idea is this? | koreth1 wrote: | That's a pretty common pattern in event-sourcing | architectures. It is a completely viable way to do things as | long as "eventual-consistency is acceptable" is actually | true. | augustl wrote: | Datomic already sort of does this :) You configure a storage | backend (Datomic does not write to disk directly) which can | be dynamodb, riak, or any JDBC database including postgres. | You won't get readable data in PG though, as Datomic stores | opaque compressed chunks in a key/value structure. The chunks | are adressable via the small handful of built-in indexes that | Datomic provides for querying, and the indexes are covering, | i.e. data is duplicated for each index. | brundolf wrote: | Interesting! I assumed Datomic was entirely custom | | Now I'm even more curious if you could skip Datomic and | just do something like this directly with a relational DB | in production | kamma4434 wrote: | Yes, but you end up rewriting Datomic! | brundolf wrote: | Well, except it sounds like Datomic is closed-source :) | JBiserkov wrote: | But why?! The whole point of Datomic is that it | implements this entire immutable framework for you, on | top of mutable storage. | | So YOU can focus on building your own specific business | logic, instead of re-implementing the immutable DB wheel. | epolanski wrote: | > Datomic's is perfect for probably 90% of small-ish backoffice | systems that never has to be web scale (i.e. most of what I do | at work). | | So is any cloud-managed db offering and at that scale we | talking very small costs anyway. | | Why datomic instead? | augustl wrote: | Because of the reasons I list :) Anything in particular that | wasn't clear/relevant? | xpe wrote: | > Datomic's is perfect for probably 90% of small-ish backoffice | systems that never has to be web scale (i.e. most of what I do | at work). | | I don't think I agree with this as stated. It is too squishy | and subjective to say "perfect". | | More broadly, the above is not and should not be a cognitive | "anchor point" for reasonable use cases for Datomic. Making | that kind of claim requires a lot more analysis and persuasion. | augustl wrote: | I agree, I mostly phrased it that way for effect. My | "analysis" is 100% subjective, opinionated and anecdotal :) | spariev wrote: | One thing which is quite hard to do in Datomic is simple | pagination on a large sorted dataset, as one can easily do with | LIMIT/OFFSET in MySQL for example. There are solutions for some | of the cases, but general case is not solved, as far as I | remember (it's been a while I used it extensively) | augustl wrote: | It depends! If you want to lazily walk data, you can read | directly from the index (keep in mind, the index = the data = | lives in your app), or use the index-pull API which is a bit | more convenient. | | However, if you want to paginate data that you need to sort | first, and the data isn't sorted the way you want in the | index, you have to read all of the data first, and then sort | it. But this is also what a database server would need to do | :) | convolvatron wrote: | there must be a relational sort? ah yes, there is, and the | ability to feed a relational output into a Clojure sort | Scarbutt wrote: | _Datomic 's is perfect for probably 90% of small-ish backoffice | systems that never has to be web scale_ | | How do they scale it for Nubank? (millions of users) | ithrow wrote: | I don't how they do it, but the obvious answer is probably | sharding. Their cloud costs must be no joke. Peers require | tons of memory and I can only guess they must have thousands | of transactors to support that workload and who knows how | many peers. Add to this that they probably need something | like Kafka for integrating/pipelining all this data. | outworlder wrote: | > Peers require tons of memory | | As do most distributed databases. Even when you don't store | your entire database (or working set) in memory, you'll | likely still have to add quite a bit of memory to be used | as I/O cache. | augustl wrote: | Good question! I don't have any personal experience in that | regard. I would probably have paid up for enterprise support | (or bought the entire company ;)) | electroly wrote: | > Someone needs to have the working set of your database in | memory, why shouldn't it be your app server and business logic? | | This one confused me. The obvious reason why you don't want the | whole working set of your database in the app server's memory | is because you have lots of app servers, whereas you only have | one database[1]. This suggests that you put the working set of | the database _in the database_ , so that you still only need | the one copy, not in the app servers where you'd need N copies | of it. | | The rest of your post makes sense to me but the thing about | keeping the database's working set in your app server's memory | does not. That's something we specifically work to avoid. | | [1] Still talking about "non-webscale" office usage here, | that's the world I live in as well. One big central database | server, lots of apps and app servers strewn about. | augustl wrote: | It's definitely a trade-off! If you have 10s or 100s of app | servers that has the exact same working set in memory, it's | probably not worth it. | | But if you have a handful of app servers, it's much more | reasonable. The relatively low scale back-office systems I | tend to work with typically has 2, max 3. Also, spinning up | an extra instance that does some data crunching does not | affect the performance of the app servers, as they don't have | to coordinate. | | There's also the performance and practicality benefits you | get from not having to do round-trips in order to query. You | can now actually do 100 queries in a loop, instead of having | to formulate everything as a single query. | | And if you have many different apps that operates on the same | DB, it becomes a benefit as well. The app server will only | have the _actual_ working set it queries on in memory, not | the sum of the working set across all of the app servers. | | If this becomes a problem, you can always architecture your | way around it as well, by having two beefy API app servers | that your 10s or 100s of other app servers talks to. | xmlblog wrote: | > If you have 10s or 100s of app servers that has the exact | same working set in memory, it's probably not worth it. | | The introduction of intelligent application-level | partitioning [1] and routing schemes can help one balance | cost and performance. | | [1] https://blog.datomic.com/2023/04/implicit- | partitions.html | electroly wrote: | SQLite provides a similar benefit with tremendous results | using its in-process database engine, although the benefit | there is more muted by default because of the very small | default cache size. We do have one app where we do this. | There's no database server, the app server uses SQLite to | talk directly to S3 and the app server itself caches its | working set in memory. I can definitely see the benefit of | some situations, but for us it was a pretty unusual | situation that we might not ever encounter again. | | All that said... can't Datomic also do traditional query | execution on the server? I thought it had support for | scale-out compute for that. AIUI, you have the option to | run as either a full peer or just an RPC client on a case- | by-case basis? I thought you wouldn't need to resort to | writing your own API intermediary, you could just connect | to Datomic via RPC, right? | carry_bit wrote: | AIUI, the full peer _is_ Datomic; the RPC server is just | a full peer that exposes the API over http and is mainly | intended to be used with clients that don 't run on the | JVM (and so can't run Datomic itself in-process). | dimitar wrote: | Consider this use case - in addition to your web app, you | have a reporting service that makes heavy duty reports; if | you run one at a bad time, bad things might happen like users | not being able to log in or do any other important work, | because the database is busy with the reports. | | So in a traditional DB you might have a DBA set up a | reporting database so the operational one is not affected. | Using Datomic the reporting service gets a datomic peer that | has a copy of the DB in database without any extra DBA work | and without affecting any web services. This also works | nicely with batch jobs or in any situation where you don't | want to have different services affect each others | performance. | | Its true that a lot more memory gets used, but it is | relatively cheap - usually the biggest cost when hosting in | the cloud being the vCPUs. But usually in Clojure/Datomic web | application you don't need to put various cache services like | Redis in front of your DB. | | Thea assumption here is that the usual bottleneck for most | information systems and business applications is reading and | querying data. | electroly wrote: | I appreciated this insight into other people's use cases, | thank you for that! This architecture brings RethinkDB to | mind, which also had some ability to run your client as a | cluster node that you alone get to query. (Although there | it was more about receiving the live feed than about | caching a local working set.) | svieira wrote: | > which also had some ability to run your client as a | cluster node that you alone get to query | | FoundationDB does this as well. | thewataccount wrote: | Is RethinkDB still around? | | They actually have recent commits, and a release last | year. | electroly wrote: | The company is gone, but the open source project lives | on. We still use it in production. | thewataccount wrote: | How's it been for production? | | Would you recommend using it, or would it be better to go | with a safer option? | electroly wrote: | We've never had any issue with it on a typical three-node | install in Kubernetes. It requires essentially no ongoing | management. That said, it can't be ignored that the | company went under and now it's in community maintenance | mode. If you don't have a specific good use for Rethink's | changefeed functionality, which sets it apart from the | alternatives, I'm not sure I could recommend it for new | development. We've chosen not to rip it out, but we're | not developing new things on it. | thewataccount wrote: | Interesting thank you! | | I remember back when it came out it was a big deal that | it could easily scale master-master nodes and the object | format was a big thing because of Mongo back then. | | That was before k8's wasn't a thing back then, and most | of the failover for other databases wasn't a thing just | yet. I'm too scared to use it because they have a | community but they're obviously nowhere as active as the | Postgres and other communities. | tyre wrote: | With AWS you can create a replica with a few clicks. | Latency is measured in milliseconds for the three-nines | observed usage. | | I don't think this is a huge challenge (anymore) for | Postgres or whatever traditional database. | augustl wrote: | A few milliseconds per query adds up if you're doing one | query per item in a list with 100s or 1000s of elements | :) | dimitar wrote: | That is a read replica, datomic peers can do writes as | well, which further expands the possible use cases. | [deleted] | xmlblog wrote: | Having the working set present on app servers means they | don't put load on a precious centralized resource which | becomes a bottleneck for reads. The peer model allows app | servers to service reads directly, avoiding the cost of | contention and an additional network hop, allowing for | massive read scale. | NovemberWhiskey wrote: | I think the point is that treating your database as an arms- | length, RPC component that's independent from your | "application" isn't necessarily the only pattern. | foobiekr wrote: | Strong agree. there are vast, massive cost savings and | performance advantages to be had if the model is that a | shard of the dataset is in memory and the data persistence | problem is the part that's made external. The only reason | we are where we are today is that doing that well is hard. | bruiseralmighty wrote: | This is true, but the tradeoff is that now your central DB is | a bottleneck that is difficult to scale. | | Having the applications keep a cached version of the db means | that when one of them runs a complex or resource intensive | query, it's not affecting everyone else. | fulafel wrote: | > Someone needs to have the working set of your database in | memory, why shouldn't it be your app server and business logic? | | This is Ions in the Cloud version, or for on-prem version the | in-process peer library. | pachico wrote: | You seem to describe the Event Source paradigm rather than a | database :) | augustl wrote: | The main difference between event sourcing and datomic are | the indexes and the "schema", which provides full SQL-like | relational query powers out of the box, as well as point-in- | time snapshots for every "event" (transactions of facts). | | So, "events" in Datomic are structured and Datomic uses them | to give you query powers, they're not opaque blobs of data. | JimmyRuska wrote: | > doing UPDATE in SQL feels pretty weird, as the default mode | of operation of your _business data_ is to delete data, with no | trace of who and why! | | It's a good idea to version your schema changes using something | like liquibase into git, that gets rid of at least some of | those pains. Liquibase works on a wide variety of databases, | even graphs like Neo4j | | I got the same feeling in Erlang many times, once write | operations start getting parallel you worry about atomic | operations, and making an Erlang process centralize writes | through its message queue always feels natural and easy to | reason about. | samuell wrote: | Question to people having used Datomic: | | Based on experience with Prolog, I always thought using Datalog | in a database like Datomic would mean being able to model your | data model using stored queries as a very expressive way of | creating "classes". And that by modeling your data model using | nested such queries, you alleviate the need for an ORM, and all | the boilerplate and duplication of defining classes both in SQL | and as objects in OO code ... since you already modelled your | data model in the database. | | Does Datomic live up to that vision? | mtnygard wrote: | You can definitely use Datomic in the way you describe, in at | least a few different ways. I haven't often seen queries | stored in the database itself. It's more common to have the | queries as data in the application. Since queries are | ordinary Clojure data structures, it's even more common to | see functions that build queries from inputs or functions | that transform queries (e.g., adding a tenant-id filter | clause). | | Datomic also support rules, including recursive rules. I | wrote a library to do OWL-style inference about classes of | entities using rules. You can see an example here | (https://github.com/cognitect- | labs/onto/blob/master/src/onto/...). This is a rule that | infers all the classes that apply to an entity from an | attribute that assigns it one class. | | I would also say that building an "entity type definition" | system as entities in Datomic is almost the first thing every | dev tries after the tutorial. It works... but you almost | never _need_ it later. | gleenn wrote: | Clojure in general is all about passing around little maps of | data and in particular not using OO to model. So Datomic | naturally continues that by returning maps of nested | structures to represent your query results and does side-step | the ORM completely. | samuell wrote: | Thanks for the comment! | | I was more thinking of the means to define your data | "classes" (or whatever it is called on this context) | though, rather than how it is passed around. | PreachSoup wrote: | Question: What's the difference between nested oop objects | (composition over inheiritance) vs maps of nested | structures? | blatant303 wrote: | "It is better to have 100 functions operate on one data | structure than to have 10 functions operate on 10 data | structures." | | Alan Perlis. | gleenn wrote: | Rich Hickey has a great talk about how Objects are data | structures with unique interfaces that are unnecessary | complexity. He showed and example of a web server with a | web request object with request headers etc etc. Doing | simple things like collecting information out of that | nested object structure is bespoke and harder than it | should be for no real gain. If everything is a map or | list or set, it becomes completely trivial to extract and | manipulate data from a structure. It's a subtle | difference at first but when everything is like that it | makes your system far simpler. Unfortunately I can't | remember exactly where the part of this talk is. | bavell wrote: | > Unfortunately I can't remember exactly where the part | of this talk is. | | If you or anyone else remembers, I'd love to watch | [deleted] | janosos wrote: | https://youtu.be/aSEQfqNYNAc | | The full video: [Time 0:49:06] | https://www.youtube.com/watch?v=VSdnJDO-xdg | | Transcript here: https://github.com/matthiasn/talk- | transcripts/blob/master/Hi... | sharms wrote: | Here is this specific part cut out from the larger talk | "Clojure made simple": | https://www.youtube.com/watch?v=aSEQfqNYNAc | greg7mdp wrote: | This? https://www.youtube.com/watch?v=aSEQfqNYNAc | PreachSoup wrote: | Most of the time, the class name is used just as a | nominal type identifier. That's what we do at least | tyre wrote: | "No real gain" | | I don't agree with this. iirc Rack ultimately uses and | array to represent HTTP responses. It has three members: | the status code, the headers, and the response body. | | If you're shipping a new change, is it easier to mistake | response[0] or response.headers? | | This is a trivial example, but the general class (ha) of | trade-off is amplified with more complex objects. | | I love clojure and lisp but the blindness behind a | statement like "no real gain" has always kneecapped | adoption. | outworlder wrote: | > If you're shipping a new change, is it easier to | mistake response[0] or response.headers | | False dichotomy. There are many options other than | arrays. Clojure in particular is fond of hashmaps. You | can have your response.headers without OOP. | augustl wrote: | In Clojure, response.headers is still data :) You just | use the built-in ways of reading named keys, such as | (:headers response) or (get headers :response). | [deleted] | kaba0 wrote: | I think there is a need for objects. An active | connection, talking to the GPU, etc are not data -- | identity is essential for their operation. | | OOP is probably the best way to model such,.. well | objects, allowing them a private, encapsulated state, and | making it only modifiable, or even viewable through a | well-defined public interface that enforces all its | invariants. | gleenn wrote: | Clojure uses objects for connections and things, but | POJOs are harmful IMHO because it makes manipulation and | collecting data a bespoke task every time. Every time you | change the data, you have to change a class to represent | the JSON and the ORM class and .... this is all just data | kaba0 wrote: | Well, that's what Java records are for. As for having to | change the type description, that's more of a static | typing discussion, though it can be generated -- having a | single source of truth and generating others is imo the | best approach. | ledauphin wrote: | Java records weren't even a gleam in the eye of James | Gosling when Clojure was solving these problems at its | inception. | filoeleven wrote: | Connection pools exist precisely because the code outside | of the connection management piece shouldn't have to care | much whether or not there is an active connection. It's | all boilerplate, except for handling the "unable to | connect" case. | | When you call a connection or connection pool object, | you're querying its current state. This is absolutely | data. | waffletower wrote: | I think OOP (object oriented programming) is abused and | is not the optimal paradigm for most software services. | It succeeds best at providing inter-operability structure | for API design. "Objects", as mentioned here, are an | abstraction. Stateful data can be organized and | manipulated elegantly without use of the OOP paradigm. | Many small systems that employ OOP hamper their | maintenance and extendability by unnecessary dependence | upon encapsulation and data ownership. Mutability is a | villain here as well -- when data structures are | immutable, there is little fear of panoptic architectures | designed without ownership constraints. Here software is | no longer corralled into walled gardens of "objects"; | large complex types and their brittle method associations | are avoided, greatly simplifying software architectures | as a result. | beders wrote: | Every single day I wish the architects at my current job had | chosen Datomic instead of Postgresql. It would have saved us so | so much time and trouble. The time traveling ability alone would | have been so useful so many times. | | Also the ability to annotate transactions is awesome. | | So many goodies. | | Here's a good summary: | | https://medium.com/@val.vvalval/what-datomic-brings-to-busin... | hoffs wrote: | I am almost certain you don't. | kgwxd wrote: | I really like Clojure and the ideas behind Datomic but free | without source is a trap, every time. They have to make money | somehow, but they already sold to a bank. If that bank wants devs | willing to work on their systems after the current generation | moves on, I think they'd be better off going open source and to | continue paying good devs to work on it. Everyone already knows | lock-in is bad for businesses. Devs will seek non-proprietary | solutions first, if they can't find it, there are already plenty | of proven proprietary solutions they'll settle on way before | Datomic. Open the source, sell the support. | charesjrdan wrote: | You could look into http://xtdb.com/ if you want an open source | alternative | fulafel wrote: | > Datomic Cloud will be available on AWS Marketplace with no | additional software cost. | | This is cool as well. It's a CloudFormation template based | product you can deploy from AWS Marketplace. | [deleted] | espeed wrote: | Is SQLite Wasm on the horizon? | chrisjc wrote: | Curious why you're asking this question when it seems to have | little to do with Datomic going "free"? Did you mean Datomic | WASM on the horizon? Or am I missing some other connection | between SQLite and Datomic? | espeed wrote: | Datomic Wasm backed by SQLite. | chrisjc wrote: | Interesting! | | What are you using SQLite for? If it's analytics, perhaps | DuckDB WASM might be an option? | hombre_fatal wrote: | Aside, I remember HN in 2009 or so where Clojure was a daily | homepage staple and Rich Hickey was putting out his talks about | Clojure and code design. | | I watched a lot of that and used Clojure fulltime for five years. | Wonder what he's up to these days. | xmlblog wrote: | Attending https://2023.clojure-conj.org/schedule/ | kgwxd wrote: | Oh, that's today and tomorrow. I had been waiting for that, I | could have sworn earlier this year they said there was going | to be a live stream available. Maybe it didn't pan out. | | Edit: Oh, there are streaming tickets for $20. | stonemetal12 wrote: | >used Clojure fulltime for five years. | | How did that work out for you? Usually following a hype cycle, | there is a negative hype cycle i.e. Mongo is webscale, then | Mongo is a buggy mess. | | Clojure seemed to just fade away. Did it turn out well or are | there interesting pitfalls that make it not as great as | advertised? | hombre_fatal wrote: | Clojure is just really niche. Even an ecosystem that stays | the same size overtime is dwarfed by the continually growing, | continually polished competitor ecosystems that will pull | people from the small niches. | | The best things about Clojure are things you don't really | appreciate until you've already done the work to learn them. | | For example, I never would have known how amazing it was to | evaluate code inside the editor until I did the work of | learning Emacs + evil-mode + nrepl/cider + whatever so that I | could spin up my http server in-process and then modify code | without restarting everything. Even today I'm doing `nodemon | index.ts` like a goofball. | | I stopped using Clojure simply when I met someone who wanted | to build some big projects with me and, despite appreciating | that Clojure was probably amazing, they simply couldn't be | bothered to learn it. Fair enough. It was when Javascript was | just getting `yield` coroutines (before async/await) which | finally made Javascript bearable for me enough to switch to | it for server work. | | Clojure just has increasingly compelling languages and | ecosystems to compete with, yet it has a huge ramp up, being | a lisp, that make it hard for people to choose it. | | Just consider how, to even write Clojure comfortably, you | really need something like Paredit. Else, what exactly are | you going to do if you want to nest or unnest a (form) deeper | in the ast structure? Manually rebalance parens? Cut and | paste it? Only Paredit lets you easily move a (form) around | the ast. And it's amazing but yet another tool you have to | learn just to truly evaluate Clojure. | blatant303 wrote: | You don't need paredit. I use Sublime and ctrl+shift+space. | Been writing Clojure for 10 years or so. | mschaef wrote: | I've used Clojure personally for >10 years and done a bit | with it commercially along the way. (I also have a personal | affection for Lisp-like languages that goes back to at least | the mid-90's.) | | > How did that work out for you? | | For the personal projects, it's been incredibly useful. The | language fits the way I think, and being built on the JVM, it | has both a performant runtime and lots of access to a wide | ecosystem of libraries. | | The Clojure-specific ecosystem library has been accused of | being stagnant. I tend to take a more charitable view. The | libraries I've used have tended to be smaller in scope and | more well defined in terms of feature set. This makes it | easier for them to converge on a feature-complete state, and | many of them have done just that. If you don't mind | assembling multiple smaller libraries into the useful whole | you need, this can provide a stable platform on which to | build and develop expertise and higher level libraries. | | For larger scale commercial work, it's a harder sell. As | you've pointed out, Clojure is not hugely popular, so it's | fundamentally a minority language. This can make VC's touchy | about funding. This is true to the extent I'm aware of at | least one organization that started moving away from Clojure | for that reason. | | There's also the shape of the learning curve. It can be hard | to get started with Clojure because of the issues around the | syntax and associated tooling. The more piecemeal aspect of | the library ecosystem can then make it harder to get to hit | the early successes a larger framework-oriented approach can | give you out of the box. You can get there, but it at least | takes more initial effort. The same is true for all the | abstractive power of Clojure (and other Lisps). Abstractions | are nice, but they take time to develop and the payoff is on | a considerable lag. The useful rule about waiting to abstract | until after you see 2 or 3 instances of a pattern means you | need to at least have spent enough time to see those 2 or 3 | instances (and maybe a few more) before you really start to | see the payoff in your own code. | | The net of all this is that it's a language that may make it | more difficult to get funding, will be initially somewhat | confusing to most developers, and the payoff may well be | deferred to the point you don't see it before you give up | (either out of frustration or due to external factors). All | in all, a considerable set of headwinds. | | So what does that mean? It's probably better for projects on | a longer time horizon that have a team willing and able to | put in extended effort to effectively use the language. (And | if the team is not self-funded, good to have a funder with | some ability to accept the risk of a non-conventional | solution). Not saying these projects don't exist, just that | they're not common enough to build a 'popular/mass-market' | ecosystem on. | askonomm wrote: | As far as the metrics state, Clojure never faded away. It's | on an upward trajectory still, albeit slowly. Perhaps it | faded away in terms of "HN hype of the day". | | That said, I've been (and currently am) a Clojure engineer | for the past 5 years and loving it. Quite a lot of jobs out | there, more and more each time I look, healthy ecosystem and | friendly community. It doesn't hurt that it's the most paid | programming language as well. | vaylian wrote: | Probably spending a lot of time in hammocks[1] | | [1] https://www.youtube.com/watch?v=f84n5oFoZBc | puredanger wrote: | Working on Clojure and Datomic! | ilrwbwrkhv wrote: | The whole clojure ecosystem and the wonderful tools around it | never really took off due to unclear documentation, poor | onboarding and too few evangelists. Datomic and other products | are really cool but are now being given away as scrapware due | to this lack of effort to make the whole ecosystem more | palatable, colorful and easy to get into for new audiences. | cellularmitosis wrote: | Even as a Clojure hobbyist I feel like all of these points | are off? Between 4clojure, clojuredocs and the slack channel | and the surprising number of books available, the onboarding | and docs are great. And when I think of my favorite lang | evangelists, hickey and nolen are absolutely #1 and #2, and | have influenced me heavily, despite my day job not involving | Clojure at all. | lib-dev wrote: | Those talks inspired far more than Clojure itself. It sort of | started this movement toward simplicity as a value. | frou_dh wrote: | If that's the case then all the people running around | praising the ""simplicity"" of golang got the wrong end of | the stick completely from Rich's classic presentations. | [deleted] | moomin wrote: | The thing is, Hickey was entirely right to reject this idea | of "Simplicity" being the same thing as "Easy". But he then | decided to conflate in "comprehensible" which it turns out | is very much a matter of aesthetics. | | Turns out if you really focus on composability above other | concerns you get Haskell. | frou_dh wrote: | > But he then decided to conflate in "comprehensible" | which it turns out is very much a matter of aesthetics. | | Did he? I seem to remember a quip in one of the | presentations about German not being comprehensible to | him being his own problem, because he never learned | German. | sooheon wrote: | In Hickey's world comprehensible is a subset of easy. | tikhonj wrote: | "Easy" vs "comprehensible" is exactly the trade-off | Haskell makes: it might be difficult to learn, but it's | easy to reason about once you learn it. (Of course, both | sides of that equation come with their own caveats...) | bvanderveen wrote: | From experience: | | Datomic Cloud is slow, expensive, resource intensive, designed in | the baroque style of massively over-complicated CloudFormation | astronautics. Hard to diagnose performance issues. Impossible to | backup. Ran into one scenario where apparently we weren't quick | enough to migrate to the latest version, AWS had dropped support | for $runtime in Lambda, and it became impossible to upgrade the | CloudFormation template. Had to write application code to | export/reimport prod data from cluster to another--there was no | other migration path (and yes, we were talking to their | enterprise support). | | We migrated to Postgres and are now using a 10th of the compute | resources. Our p99 response times went from 1.3-1.5s to under | 300ms once all the read traffic was cut over. | | Mother Postgres can do no wrong. | | Still, Datomic seems like a cool idea. | lgrapenthin wrote: | As someone who is using Datomic Pro in production for many | years now I must agree with you. One time I began a project | with Datomic Cloud and it was a disaster similar to what you | described. I learned a lot about AWS, but after about half a | year we switched to Datomic Pro. | | There were some cool ideas in Datomic Cloud, like IONs and its | integrated deployment CLI. But the dev workflow with Datomic | Pro in the REPL, potentially connected to your live or staging | database is much more interactive and fun than waiting for | CodeDeploy. I guess there is a reason Datomic Pro is the | featured product on datomic.com again. It appears that | Cognitect took a big bet with Datomic Cloud and it didn't take | off. Soon after the NuBank acquisition happened. That being | said, Datomic Cloud was not a bad idea, it just turned out that | Datomic Pro/onPrem is much easier to use. Also of all their | APIs, the "Peer API" of Pro is just the best IME, especially | with `d/entity` vs. "pull" etc. | ithrow wrote: | I guess this is why datomic.com front page now defaults to | datomic pro and not cloud. | JulianWasTaken wrote: | I don't doubt your story of course, and I love Postgres, but | comparing apples to oranges no? | | Datomic's killer feature is time travel. | | Did you simply not use that feature once you moved off Datomic | (and if so why'd you pick Datomic in the first place)? Or are | you using Postgres using some extension to add in? | ithrow wrote: | Datomic's 'time travel' is an audit feature, not something | for your application/business logic to depend on. Performance | reasons make it impractical, unless you only have like 10 | users and very little data. | JulianWasTaken wrote: | That's certainly not how it sells and markets itself. | | The first feature on benefits (and the only reason I've | ever heard Datomic brought up and/or considered it myself | for production workflows) is using that stuff in | application workflows: | https://docs.datomic.com/pro/time/filters.html#history | | Could be you're saying it in fact doesn't work well | performance-wise, that'd (surprise me but) certainly | explain why it's not more popular -- but I think it's clear | it _wants_ you to use this as an application feature. | newlisp wrote: | Welcome to sales tactics ;) | | Datomic is great but as another commenter said, is good | for "small-ish backoffice systems that never has to be | web scale". You almost probably can rely on querying | history for internal applications. I think their primary | market was for companies to use it internally but they | never made this clear. | xmlblog wrote: | > "small-ish backoffice systems that never has to be web | scale". Doesn't production use of Datomic by Nubank and | Netflix (to mention just two examples) belie this | assertion? | bvanderveen wrote: | We implemented it in Postgres with 'created_at' and | 'deleted_at' columns on everything and filtering to make sure | that the object 'exists' at the time the query is concerned | with. Changes in relationships between objects are modeled as | join tables with a boolean indicating whether the | relationship is made or broken and at what time. | | Our data model is not large and we had a very complete test | suite already, so it was easy to produce another | implementation backed by postgres, RAM, etc. | wwweston wrote: | Yeah, it seems you could be able to substitute thoughtful | schema design avoiding updates/deletes for time-travel as a | feature. | | I wonder if anyone has made a collection of reference | examples implemented this way (and in general think that a | substantial compendium good examples of DB schema and | thinking behind them could be worthwhile). | twic wrote: | I'm moderately confident you could mechanically transform | a time-oblivious schema into a history-preserving one, | and then write a view on top of it which gave a slice at | a particular time. Moderately. | ysleepy wrote: | That is essentially what MVCC does. | twic wrote: | That is an extremely good point. | pjot wrote: | It's called a slowly changing dimension. In this example, | it's a type-2. | | https://en.m.wikipedia.org/wiki/Slowly_changing_dimension | rehevkor5 wrote: | Maybe search around on bitemporal database table | modeling. | btown wrote: | https://www.postgresql.org/docs/11/contrib- | spi.html#id-1.11.... discusses a model for implementing time | travel in Postgres <12 using SPI. https://git.postgresql.org/ | gitweb/?p=postgresql.git;a=commit... discusses why it was | removed in Postgres 12 - it seems logical that it's more | maintainable to implement in plpgsql, though as far as I can | tell there aren't off-the-shelf implementations of this. | | We use https://django-simple- | history.readthedocs.io/en/latest/ (with some custom tooling | for diff generation) for audit logs and resettability, and | while you can't move an entire set of tables back in time | simultaneously, it's usually sufficient for understanding | data history. | avodonosov wrote: | Why backups were impossible? Couldn't you backed up the storage | resources? | outworlder wrote: | Are they _forcing_ you to use CloudFormation? Or is it just the | officially supported mechanism? | | > Mother Postgres can do no wrong. | | I'll say that Postgres is usually the answer for the vast | majority of use-cases. Even when you think you need something | else to do something different, it's probably still a good | enough solution. I've seen teams pitching other system just | because they wanted to push a bunch of JSON. Guess what, PG can | handle that fine and even run SQL queries against that. PG can | access other database systems with its foreign data wrappers(ht | tps://wiki.postgresql.org/wiki/Foreign_data_wrappers). | | The main difficulty is that horizontally scaling it is not | trivial(although not impossible, and that can be improved with | third party companies). | ggleason wrote: | > Datomic Cloud is slow, expensive, resource intensive, | designed in the baroque style of massively over-complicated | CloudFormation astronautics. Hard to diagnose performance | issues. Impossible to backup. | | You should give TerminusDB a go (https://terminusdb.com/), it's | really OSS, the cloud version is cheap, fast, there are not | tons of baroque settings, and it's easy to backup using clone. | | TermiusDB is a graph database with a git-like model with | push/pull/clone semantics as well as a datalog. | elesbao wrote: | nice. now everyone can experiment the pain of running s3+pgsql+a | huge blob of binary using container ram ! /rant - it's an awesome | piece of software regardless. | jchw wrote: | Not complaining about the actual announcement itself here: seems | pretty sweet all things considered, But: the "Is it Open Source?" | section should lead with "No." It's not a complicated question, | and it's not a complicated answer. I think it's weird to talk | about having "all the same rights" without explaining why that | matters particularly (it does matter, it's just not explained | much!) but it is somewhat tangential to the question being posed | which has a very clear and straightforward answer. | | I hope more companies consider this unusual arrangement at least | as an alternative to other approaches. Permissively licensed | binaries can come in handy, though it certainly comes with it's | risks. For example, Microsoft released the binaries for its | WebView2 SDK under the BSD license; this is nice of course, but | the side-effect is that we can (and did) reverse engineer the | loader binary back to source code. I suspect that's unlikely to | happen for any substantially large commercial product, and I am | not a lawyer so I can't be sure this isn't still legally dubious, | but it's still worth considering: the protections of a EULA are | completely gone here, if you just distribute binaries under a | vanilla permissive open source license. | daveguy wrote: | Given the recent bait and switch moves by many companies from | free-for-everyone open source to free-for-users-only (eg apache | to agpl), it seems like doing this with "binaries only" | practically admits to a bait and switch plan. | charesjrdan wrote: | There are also similar datalog immutable databases with fully | open source codebases out there too like https://www.xtdb.com/ | or https://datahike.io/ | xpe wrote: | Saying "no" also would provide a nice opportunity for Nubank to | explain why. | | To some, the answer is "open source" no matter the question. | Hello wagging tail, meet dog. | thomastjeffery wrote: | They lead with, "The Datomic binaries are being released under | the Apache 2.0 license." | | I can feel the internal open vs. closed source argument from | here. | kevincox wrote: | I felt the exact same thing. I was pretty sure the answer was | "No" but they were too chicken to write it. | | Were they hoping that people wouldn't notice? | a2800276 wrote: | You'd think they could have just left that question out of | their FAQ and licensed it with a "free-as-in-beer" click- | through license. | nerdponx wrote: | Probably. They were hoping that the less-informed (like me) | would see "The Datomic binaries are being released under the | Apache 2.0 license" and think "Datomic is relicensed under | Apache 2.0". | sneak wrote: | It worked for Docker Desktop and BSC. | | The products even have github repos! There's just no source | in them. | | It's open source cosplay. | unionpivo wrote: | > It's open source cosplay. | | I am stilling that phrase. It's perfect. | MrGilbert wrote: | I'm not sure, but I think from a marketing perspective, you do | not want to write "No" anywhere. I might be wrong, and I'd love | if someone with knowledge in this area could answer. It's | something I recall from the back of my mind... | JoshTriplett wrote: | Dishonesty and subterfuge are _bad marketing tactics_. | newswasboring wrote: | Is it though? Seems to me that most marketing today is | subterfuge. | thomastjeffery wrote: | Is most marketing good? I don't think so. | newswasboring wrote: | I don't know what you are calling "good". That is a super | loaded word. I would say it is effective though. The | baseline evidence for that is how much money gets spent | in that industry, I guess. | JoshTriplett wrote: | > I would say it is effective though. | | Really depends on your metrics. Very little of it builds | long-term trust that didn't already exist. Much of it | abuses long-term trust. | | See the discussion on | https://news.ycombinator.com/item?id=35590734 | thomastjeffery wrote: | > The baseline evidence for that is how much money gets | spent in that industry, I guess. | | I can't imagine a clearer example of circular reasoning. | | I can lift exactly as much weight as I decide to put on | the barbell. | newswasboring wrote: | While I see your point, please note people actually | spending money don't just look at how much money is in | the industry. They have internal metrics and can track | performance of individual marketing campaigns. I called | it a baseline argument because I'm not a marketing expert | so this it the only real KPI __I__ understand. | switchbak wrote: | Marketing to developers is also decidedly different than | for non devs. We've been baited and switched, and screwed | over a million times (thanks Google). It takes a lot of | trust building to get me to trust any organization. | | I especially don't trust free shit if I don't have access | and control of the future of the code. Even then it's not a | sure thing. | Timwi wrote: | Being a marketing tactic makes it more scummy, not less. | nerdponx wrote: | I didn't realize it was possible to release binaries under a | different license from the source code that generated them. In | this case, is the "source code" just the physical machine code | or bytecode in the binary? | jchw wrote: | Disclaimer; I'm not a lawyer, nor do I play one on TV. | | A copyright license is a copyright license: in theory, all a | copyright license does is give you additional rights to use | something. Using a license like Apache 2 for binaries is | somewhat unconventional, but it's totally possible. It | (obviously) does not give you access to the source code, and | I think this could never work with the GPL and other copyleft | licenses because they use wording that implies you need to | distrubute the source code, which you don't have. | | The copyright owner, of course, has ownership, so their | obligations don't really change by virtue of giving someone a | copyright license. As far as I know, they could give someone | a license to use something that is completely invalid and | could never actually be used, and they can definitely do | things like stop distributing under one license and switch to | another. They own the source code, and they own the binaries | (I believe the binaries would be considered a sort of | derivative work in copyright terms, but again, not a lawyer.) | So when they distribute a _binary_ under a given license, it | 's unrelated to any particular distribution of _source code_. | The only time this gets complex is when the ownership of a | asset is split among many disparate parties, at which point | everyone is pretty much beholden to the copyright licenses; | like open source projects without CLAs. But if they own the | source code entirely, they could, for example, distribute | some source code under GPL, but then distribute modified | binaries under a commercial license with a EULA, and not | redistribute the modified source code, since it 's their code | to license to others, not a license they are subjected to | themselves. | remram wrote: | You can attach any license text to anything, but most open- | source license make little sense when applied to binaries. | Like in this case, the Apache 2 license doesn't make | distinction between the source and binary, referring to both | as "the Work": | | > You may reproduce and distribute copies of the Work or | Derivative Works thereof in any medium, with or without | modifications, and in Source or Object form, provided that | (...) | | Applying this only to their binaries directly contradicts | what the license says. | squeaky-clean wrote: | I suppose you could modify the binaries directly and | redistribute and it would be legally allowed? Not like | anyone is going to do that though. | TheCoelacanth wrote: | If you think about it, that's happening every time that you | get a closed-source binary distributed to you. They're giving | you a license to the binary, but not to the source. | | It's certainly weird for the binary license to be Apache, | rather than some proprietary EULA, though. | assbuttbuttass wrote: | I am not a lawyer, but doesn't the Apache license specifically | grant the right to redistribute the source code? | | > You may reproduce and distribute copies of the Work or | Derivative Works thereof in any medium, with or without | modifications, and in Source or Object form | mbreese wrote: | But what if you never had the source form? You can't | redistribute what you never had... | | This is actually an interesting question. But I can't see how | a binary only distribution would be in the spirit of the | Apache license. | jrmg wrote: | It's runs on a JVM, right? Presumably it could be | decompiled and cleaned up (might be a massive task - but | possible), and the reconstructed source would fall under | the Apache license. | | What a weird choice to make. | ilc wrote: | It is the very reason the license exists. | | It is in the spirit and the letter of the license to do | binary only distribution. | | Why do you think so many companies are pushing the Apache | License 2.0 over the GPL? | mbreese wrote: | I thought the reason was the explicit patent language and | lack of license virality. | | The Apache license 2 is pretty clear that binary only | distribution is allowed, but I think it's also clear that | the assumption is that source is available in some form. | Otherwise, why would you care about derivative works? | | As is, it would be possible to decompile the JVM code | into _something_ resembling source code and then | distribute that with or without modification. Which just | seems odd to me. | erichocean wrote: | Eh, it's Java bytecode and it's Apache 2. So you can patch | Datomic and extend it without breaking the license. | slim wrote: | You can also probably decompile it | ithrow wrote: | I wonder if public benchmarks are now allowed with the | binaries. | mst wrote: | I guess NuBank (Cognitect's owners) have concluded that the paid | licensing business wasn't worth the hassle compared to having the | developer time involved spent on other things. | | Releasing only binaries, while I understand people being grumpy | about it, seems like an interesting way of keeping their options | open going forwards. Since it was always closed source, it now | being 'closed source but free' is still a net win. | | The Datomic/Cognitect/NuBank relationship is an interesting | symbiotic dynamic and while I'm sure we can all think of ways it | might go horribly wrong in future I rather hope it doesn't. | motoboi wrote: | Very probably they understood that having a property database | makes hiring and onboarding more difficult than necessary. | | Open sourcing the database helps on that. | elteto wrote: | They didn't open source the DB, just the binaries. | remram wrote: | How can you "open source" something that doesn't include | the sources? | Thorrez wrote: | Yeah, they didn't open source anything. They just made it | free. | elteto wrote: | Ah! Yes, but not quite! It's not freeware. The binaries | are technically open sourced, you can do with them as you | please within the confines of the Apache license. | elteto wrote: | Well, _technically_ you are now free to modify the binary | and redistribute the result as per the Apache 2.0 | license. That's different than giving something as | freeware, which would not allow/cover | modification/redistribution. | croes wrote: | That's parent's point, they didn't. They only made the | binaries available under the Apache 2.0 license | bananapub wrote: | they haven't open sourced the database though? | jcadam wrote: | The licensing was the biggest thing hurting Datomic adoption. | This is a smart (albeit late) move. | rbalicki wrote: | If I recall correctly, Datomic gives you the ability to query the | database at a given timestamp. Are there other DBs with this | feature that folks are aware of? | dangets wrote: | Link comment in a similar HN discussion: | | https://news.ycombinator.com/item?id=14715279 | activitypea wrote: | What is the benefit of having it closed source? | | My view is that Datomic is a novel upstart in the persistence | space. Most of their competition - Postgres, Mongo, Cassandra - | is open-source, so they're just shooting themselves in the foot. | The "pay us extra for convenient hosting and consulting" model | isn't threatened by open-source in the slightest. | | The only thing I can think of is that they're trying to compete | with Oracle/Db2/SQL server, but I can't imagine an enterprise | eyeing any of those solutions ever giving Datomic a chance. | kragen wrote: | sounds like they're not releasing the source code, just the | binaries, even though the binaries are under the apache license | | so, less than useful if you want to study and modify datomic; you | may have the _legal_ "right to repair" but not the practical | possibility | nXqd wrote: | amazing piece of work, it's nice to see it's free. | warthog wrote: | Wow I wonder what their incentive is in making this public and | free | spapas82 wrote: | It seems that they give the binaries for free but they won't | release the source code. Can somebody explain to me what's the | point of keeping the source closed in this case? I really can't | think of any reason | stefncb wrote: | Licensing issues, keeping the door open to making it not free | again, greed, lack of understanding from | management/lawyers/whatever, not wanting to deal with | contributions (though here you can do what SQLite does), false | sense of security, etc. | harperlee wrote: | Perhaps security by obscurity, it being now a proprietary | component of a bank? | xpe wrote: | There are valid reasons not to release the source code that | have nothing to do with "security by obscurity": legal, | various notions of "control", and more | harperlee wrote: | And I'd add that security by obscurity is also a valid | reason. It's bad as a standalone strategy, but good as a | complementary strategy. | | Related: https://news.ycombinator.com/item?id=24444497 | xpe wrote: | Nubank's goal to keep the Datomic source code private | remains secret is based primarily on IP law and internal | security controls (on employees, contractors, and | possibly obfuscating compilation). Disagree? | [deleted] | harperlee wrote: | > Disagree? | | No idea! Just speculating... | xpe wrote: | > And I'd add that security by obscurity is also a valid | reason. It's bad as a standalone strategy, but good as a | complementary strategy. | | As the thread you link mentions, the phrase "security by | obscurity" historically means (more of less) "security | primarily by obscurity". But sometimes this point gets | lost. The thread you mention is interesting. | | Wikipedia: | | > Security through obscurity (or security by obscurity) | is the reliance in security engineering on design or | implementation secrecy as the main method of providing | security to a system or component. | | Summary: | | Layers of security (which can include a wide range of | techniques, including obfuscation, etc): useful, because | delaying attacks and/or making them less likely is | useful. | | Obscurity as a main method: theatre, because it often | leads to self-deception about the true risks involved | harperlee wrote: | Sorry, we're both editing at the same time :) I added a | related link to the parent comment. | xpe wrote: | Yep :) That's a great thread, thank you! | xpe wrote: | > I really can't think of any reason | | What have you thought about or read so far? | vrglvrglvrgl wrote: | [dead] | dustingetz wrote: | is Datomic Cloud used by Nubank? | _bohm wrote: | Nubank bought Cognitect/Datomic a few years back, so they own | it I believe | casion wrote: | Heavily according to people who work there. They appears to be | a conj talk about it today. | AzzieElbab wrote: | Congratulations to Rich Hickey's children!! I hope your college | experience was excellent. Disclaimer: that is how Rich explained | why Datomic stayed closed source. | rektide wrote: | It notably powers Roam, a kind of interesting notes/mini Notion | product. | | There's a reasonably interesting writeup of the tech details that | helps show off Atomics value some, | https://www.zsolt.blog/2021/01/Roam-Data-Structure-Query.htm... | https://news.ycombinator.com/item?id=29295532 | moorg wrote: | For those who haven't followed the story: | | 2007 - the Clojure programming language is announced by Rich | Hickey and gains quite a bit of traction over the next 5 or 6 | years. It never becomes a "top 5" language, but it could still | today be arguably considered a mainstream language. It's been | endorsed as "the best general purpose programming language out | there" by "Uncle" Bob Martin[1] (author of Clean Code) and Gene | Kim[2] (auther of The Phoenix Project, the seminal DevOps book). | The fact that Rich spent two years working on it without pay and | without the commercial backing many other languages enjoy is a | real testament to his commitment and his vision. A Clojure- | related emacs package[3] quotes Rich when it starts a REPL: | "Design is about pulling things apart." | | 2012 - the Datomic database is announced by Rich Hickey's | company. The database is praised for its ingenuity and its "time | travel" features. It was designed to be deployed anywhere in the | beginning, but, over time, it became difficult to deploy outside | of AWS environments and even the AWS deployment path was quite | cumbersome--the Datomic marketing page used to feature a maze- | like diagram of all the AWS-specific features needed to make the | thing work (it would be nice to find a link to that picture); I'd | think most companies would have trouble digesting that and | integrating it into their technology "stack". | | 2020 - Nubank (a Brazilian fintech backed by at least one US | venture firm and a large production user of Datomic) acquires | Rich Hickey's company. It appears Datamic never gained much use | outside of a handful of companies. Making it free of charge | (2023) may be the cost-effective thing to do in such a situation | if it costs more to handle billing and payments than are brought | in. The reason they're not releasing the source code could be | legal one or simply the fact that open sourcing a large piece of | software takes a lot of effort--something a for-profit financial | services company like Nubank doesn't prioritize (rightly so). | | 1: https://blog.cleancoder.com/uncle- | bob/2019/08/22/WhyClojure.... 2: | https://itrevolution.com/articles/love-letter-to-clojure-par... | 3: https://github.com/clojure-emacs/cider/blob/master/cider- | uti... | lgrapenthin wrote: | The 2012 section seems not correct. In the time between 2012 | and 2020 I deployed Datomic in various non AWS environments. | Datomic was never particularly tied to AWS. I think your | timeline also misses Datomic Cloud, which was an AWS exclusive | product that launched much later than 2012. | NickBusey wrote: | Free as in beer. Not free as in speech. | | > Is it Open Source? | | Datomic binaries are provided under the Apache 2 license which | grants all the same rights to a work delivered in object form. | | Datomic will continue to be developed at Nubank, where it is a | critical piece of our infrastructure. | LukeEF wrote: | There are already some open source alternatives to datomic. | TerminusDB (https://github.com/terminusdb/terminusdb) for | example is implemented in prolog (and Rust) so has the datalog | variant query power that makes datomic so powerful. If you want | free as in speech (thou I love free beer). | filoeleven wrote: | XTDB is also worth mentioning, especially since they're on | the HN front page with a v2 early access announcement. There | are differences in how they do things. I can't meaningfully | comment on business usage of either or what the trade-offs | between them are. | | https://news.ycombinator.com/item?id=35733515 | mqus wrote: | > perpetual, worldwide, non-exclusive, no-charge, royalty-free, | irrevocable copyright license to reproduce, prepare Derivative | Works of, publicly display, publicly perform, sublicense, and | distribute the Work and such Derivative Works in Source or | Object form. | | Doesn't this mean, that, as soon as I (somehow) get hold of the | source code, I can distribute it as I want? | oh_sigh wrote: | Probably not, because the source is not a derivative. Having | said that, if you decompile the binary I bet you could | distribute that source. | hlship wrote: | That would be very ugly source, as Datomic is written in | Clojure and AOT compiled to Java bytecode. Due to the | architecture of Clojure (especially, the use of macros) it | is not exactly possible to work backwards from JVM bytecode | to anything that looks like the original source code. It's | not like Java where a clever decompiler can exploit output | patterns generated by the Java compiler to make reasonable | guesses at the structure of the source code. | | But this is all besides the point; Datomic is now free (as | in beer) with a great license (Apache 2.0). You can use | this amazing tool for free, and you have as much need to | look at the source to do so as you might need to look at | PostgreSQL's source. | | Some of us have been hoping for this day since Datomic was | first announced, but even as an insider (I have been | working at NuBank NA for less than a year) I was stunned at | the speed with which this decision was made and | implemented. | cjbprime wrote: | "distribute the Work [..] in Source [..] form" | oh_sigh wrote: | I took that to mean the form provided by the authors - eg | the "binary source" | stonemetal12 wrote: | They didn't license the source form to you, so still no. | montroser wrote: | So...no, it is not open source. I wish they would just answer | that in a straightforward way. | | "No, the source is not available, and the product will continue | to be developed by us, internally. However, binaries are | provided..." | wcerfgba wrote: | Yeah the answer in the page is a total meme. Kind of | disappointing, since I like Datomic a piece of tech. :( | simonw wrote: | I find that answer confusing. I don't think I've ever seen | binaries being licensed under Apache 2 without the relevant | source code before. | chii wrote: | It gives the licensee the ability to distribute the binary, | use or include it in their products in the same way as an | open-source product. It just merely prevents modification | without decompiling (which i assume is not easy given it's | clojure, not to mention obfuscation?). And presumably it | makes it less likely someone would just produce a competing | product if they should choose to re-monetize it? | ingenieroariel wrote: | I saw Fabrice Bellard do it this year (MIT): The CPU version | is released as binary code under the MIT license. The GPU | version is commercial software. Please contact fabrice at | bellard dot org for the exact terms. | | https://bellard.org/ts_server/ | a2800276 wrote: | Yeah, it's a complete bullshit move. Mongoose OS (an embedded | iot Plattform not the db) does something similar. It's | extremely weasely and doesn't instill trust at all. | xpe wrote: | Trust is not the same for everyone. One model of trust is: | A trusts B to do C. | | Nubank is releasing the binary permissively. You might | _want_ more, but this is not a breach of trust. | | Datomic has been around for more than 10 years, so there is | ample data to base expectations. | frou_dh wrote: | [flagged] | Pet_Ant wrote: | There is nothing wrong with saying "It is not open source | but you can freely use the binaries". That is the same | thing, but upfront about it, but this feels like open- | source-washing. | a2800276 wrote: | Nothing wrong with that at all. But that's not what was | said. Answering the question (that you posed to yourself | in your own FAQ): "Is it Open Source?" by stating: "we've | licensed the binary using an Open Source license" seems a | bit disingenuous. A more matter-of-factly answer would | have been: "No". | [deleted] | adamfeldman wrote: | What is Datomic, you ask? It's a database written in Clojure. | https://hn.algolia.com/?q=datomic Datomic is an | operational database management system - designed for | transactional, domain-specific data. It is not designed to be a | data warehouse, nor a high-churn high-throughput system (such as | a time-series database or log store). It is a good fit for | systems that store valuable information of record, require | developer and operational flexibility, need history and audit | capabilities, and require read scalability. | | (via https://docs.datomic.com/pro/getting-started/brief- | overview....) | [deleted] | mbesto wrote: | For 90% of the web devs that just spin up Postgres/MySQL, why | would you use Datomic over that? | fulafel wrote: | A tangent but it would be interesting to see survey data of | how many devs default reach for SQL first these days. A lot | of people use various other kinds of DB models which are | preceived to have smoother learning curves. | [deleted] | roguas wrote: | You have to add a lot of scaffolding to postgres to make it | semi-immutable. datomic just is, wanna know previous user | email, just go back and see. Out of the box, without thinking | about it. | evantbyrne wrote: | Immutability is certainly tempting for certain kinds of | data. Does it handle use-cases where data needs to be | deleted though? i.e., privacy compliance. | [deleted] | mcbits wrote: | It looks like they call that excision, which leaves | behind a breadcrumb saying something was deleted and | which can't itself be deleted. | dgb23 wrote: | Temporality in general becomes super handy if you have | something like reports that need to be consistent across | time. Or if you want to ask questions about the past. Or | questions about the future without affecting the present. | panick21_ wrote: | Because its a different model of integrating your database | and your app. | | It allows you to write queries in a pull style, it can be | trigger based, datalog or raw index access. Its by default | immutable and allows historical query. It allows meta data on | the transaction themselves. | | A lot of the time the user builds much of that himself or | relays on frameworks to do it. | dgb23 wrote: | That's not easy to answer. It's a question of: | | - mutable data vs immutable data | | - tables, row based vs tripple store, attribute based | (EAV/RDF) | | - table schemas vs attribute schemas | | - relational connections vs graph connections | | - SQL vs datalog | | - nested queries vs flat queries and rules | | - remote access vs client side index | | etc. | alecco wrote: | Just the temporal properties alone make it very useful for | anything where it matters like billing, finance, inventory. | Else you are in views/schema/indexing hell to do it on top of | SQL. | | There is some SQL temporal support but it's not great and | varies a lot. Also since it's not native to the storage it | has a lot of complexity issues under the rug making it not | great. | | Many financial systems use Event Sourcing (OOP + ORM). I had | to suffer this at a previous employer. | | See https://vvvvalvalval.github.io/posts/2018-11-12-datomic- | even... | slaymaker1907 wrote: | The temporal support seems handy, but time is still going | to be really tricky for financial systems. Datomic only | covers what the physical state of the database was at a | particular time, but there's also the effective legal time | (maybe a payment was dated a day before the system actually | processed it) as well as requirements to remove data after | a period of time (including point in time stuff). | augustl wrote: | Indeed, it depends a lot on the domain. Datomic only has | "technical" database time, and doesn't have any built-in | way of modelling domain time. You can set the transaction | timestamp manually when you write, but you can't set it | to be earlier than the latest transaction that was | committed. So, if you want your domain modelling to | piggyback on Datomic time travelling, you can only do | things like delaying writes for, say, an hour, and hope | you have all the data by the time you commit to db. | nlitened wrote: | There's a bitemporal Datomic-like database from JUXT that | does exactly that, I believe https://www.xtdb.com | riku_iki wrote: | > Just the temporal properties alone make it very useful | for anything where it matters like billing, finance, | inventory. | | you can easily create datamodel to have this in SQL dbs: | create table transaction_history(..., execution_time | timestamp); | augustl wrote: | Fyi, Datomic lets you look at the entire database at any | point in time, as an immutable value. Also, you can | annotate transactions with metadata, and query for "which | tx wrote this specific value for this row/column" and | look at custom metadata you added to the tx to reason | about your system. Doing all of that in SQL is not | trivial, to say the least. | tasuki wrote: | This is not an answer, it's the beginning of a question. | Yes sure, we know `create table` and we know it's a good | idea to record the execution timestamp. What exactly do | you put in the place of the three dots? | endisneigh wrote: | The price is certainly right, but has anyone used this in | production? What was your experience like? | Naomarik wrote: | https://sayartii.com/ is using Datomic stored on postgres that | I have set up on Linode. That was all done back in 2020 and | haven't needed to touch it. Site now gets ~180M monthly reqs | and I store an enormous amount of analytic data on Datomic (was | supposed to be temporary) so users can see impressions/clicks | per day for each advertisement. I'm surprised it's still | working. | | Development experience is extremely nice using clojure. I've | used it for two other projects and has been very reliable. My | latest project didn't really need any of its features compared | to a traditional rdbms but I opted for it anyways so I don't | have to write sql. | mardifoufs wrote: | Was it expensive to run? Now that it is free, I guess that's | less of a concern! | ddellacosta wrote: | My personal experience was using Datomic backed by DynamoDB, at | the second Clojure company I worked at. In particular I | remember feeling like it was hard to anticipate and understand | its performance characteristics in particular, and how indices | can be leveraged effectively. Maybe if we had chosen Postgres | as a backing store that would have been better? I dunno. | | Using it was pretty nice at the scale of a small startup with a | motivated team, but scaling it up organizationally-speaking was | a challenge due to Datalog's relative idiosyncrasy and poor | tooling around the database itself. This was compounded by the | parallel challenge of keeping a Clojure codebase from going | spaghetti-shaped, which happens in that language when teams | scale without a lot of "convention and discipline"--it may be | easier to manage otherwise. All of that said, this was years | ago so maybe things have changed. | | At this point I'd choose either PostgreSQL or SQLite for any | project I'm getting started with, as they are both rock-solid, | full-featured projects with great tooling and widespread | adoption. If things need to scale a basic PostgreSQL setup can | usually handle a lot until you need to move to e.g. RDS or | whatever, and I'm probably biased but I think SQL is not really | that much worse than Datalog for common use-cases. Datalog is | nice though, don't get me wrong. | | EDIT: one point I forgot to make: the killer feature of being | an immutable data store that lets you go back in time is in | fact super cool, and it's probably exactly what some | organizations need, but it is also costly, and I suspect the | number of organizations who really need that functionality is | pretty small. The place I was at certainly didn't, which is | probably part of the reason for the friction I experienced. | hlship wrote: | Although it is true that "time traveling" queries are | relatively rare for production needs, the basic architecture | supports things that many applications really need: | | - It is possible to make queries against the database PLUS | additional data not yet added, that is, "what if" queries | | - Having a stable database-as-value is really useful for | paginating results; you don't have to worry about new values | being inserted into your results during execution, the way | you do with traditional databases no longer how long | (minutes, hours, even days) you take to traverse the data | | - Reified transactions makes it possible to store extra data | with each transaction, trivially, such as who made the update | and why | | - Immutability is amazing for caching at all layers | xmlblog wrote: | Newer releases have improved significantly in this area. It's | now possible to understand perf implications with the | addition of io-stats[1] and query-stats[2]. | | [1] https://docs.datomic.com/pro/api/io-stats.html [2] | https://docs.datomic.com/pro/api/query-stats.html | raybb wrote: | I haven't used it but I guess Nubank uses it. | https://www.youtube.com/watch?v=qIdrT6r77gA | adamfeldman wrote: | One example: https://www.datomic.com/nubanks-story.html | dagw wrote: | Worth keeping in mind that Nubank owns the company that makes | Datomic, so that might colour their opinion. On the flip side | they probably wouldn't have bought the company if they | thought their product was crap. | Scarbutt wrote: | My guess is they bought the company because they were | already too invested in Datomic. So it was kind of forced | to minimize risk. | hoffs wrote: | Also helps them to tune it for their use cases where | other users have to rely on closed sourceness of it | jackrusher wrote: | Yes, it's used by many companies in production. There's a | partial list here: | | https://www.datomic.com/customers.html | endisneigh wrote: | to be honest the testimonials are terrible. | | > "Datomic added to DynamoDB was the only option that didn't | force us to sacrifice features or add the expense of | developer time. Without it, we would have had to push back a | lot more, as the features would have been too difficult." | | (https://www.datomic.com/the-brigades-story.html) | | like, what? effectively useless information. | | some of the other testimonials mention keeping revision | history, which is neat, but why Datomic vs. others? it's | pretty easy to keep revision history with other databases | too. | simtel20 wrote: | It's not simply revision history, it's a complete record of | everything with time, without re-architecting your data or | app. IIRC datomic structures your data so that all | transactions and state have a time dimension so you can go | forward or back in time trivially (no special query, no | temporal sql, etc.) | motogpjimbo wrote: | What happens when you have a legal obligation to delete | or anonymise data for some reason? | fulafel wrote: | There's | https://docs.datomic.com/pro/reference/excision.html - | but like in other data models you also might choose to | not store sensitive infromation like PII in cleartext in | the main DB at all. At least in earlier versions excision | wasn't supported in the Datomic Cloud version. | lebski88 wrote: | Excision still isn't supported for cloud, it is / was on | their plan but there was never any movement in that | direction. | Timshel wrote: | Not a good look when four out of the six companies behind the | "customer stories" do not exist anymore (or at least their | website is dead ...) | xmlblog wrote: | Netflix, Facebook (Meta), Nubank, and many others. | dgb23 wrote: | https://www.datomic.com/get-datomic.html still seems to show | licensing fees at the time of this comment. | thatwasunusual wrote: | > Datomic's is perfect for probably 90% of small-ish backoffice | systems that never has to be web scale (i.e. most of what I do at | work). | | I will also argue that 90% of those don't need this. Just by | seeing the term "web scale" makes me shy off. | augustl wrote: | Anecdotally, all the ones I've worked it where I've used SQL | have needed it. I've always ended up wondering when, why and | who changed an attribute to its current value, but that's not | knowable unless you jump through hoops and manually implement | it. | thayne wrote: | > Is it Open Source? | | > Datomic binaries are provided under the Apache 2 license which | grants all the same rights to a work delivered in object form. | | That doesn't answer the question at all. I assume the answer is | no, because otherwise they would just say yes, and have a link to | the source code somewhere. But that is such a weird, and possibly | duplicitous way to answer. | LukeEF wrote: | There are already a few open-source alternatives that run datalog | variant query languages. I'd point the curious towards TerminusDB | [1] and TypeDB [2]. TerminusDB is implemented in prolog (and | rust) so an alternative with datalog in the heart. | | [1] https://github.com/terminusdb/terminusdb [2] | https://github.com/vaticle/typedb | martypitt wrote: | Only the binaries are made available, not the source, which is | interesting. | | I guess they don't claim to be open source, they're claiming to | be free, which is - in itself - awesome. | | Last time I checked, you couldn't push binaries to maven central, | without also releasing the source. That may have changed. | casion wrote: | Maven actually has a tutorial on publishing binaries without | source. So I assume it's ok when they tell you how to do it. | martypitt wrote: | Sure, Maven makes this possible. | | But Maven Central has strict rules around what can be | published there. I just double checked and it's a requirement | to publish the source as well as the binaries: | | https://central.sonatype.org/publish/requirements/#supply- | ja... | BaculumMeumEst wrote: | it seems you're right, but it also says the following, so | i'm confused on whether it's a hard requirement? | | "If, for some reason (for example, license issue or it's a | Scala project), you can not provide -sources.jar or | -javadoc.jar , please make fake -sources.jar or | -javadoc.jar with simple README inside to pass the | checking. We do not want to disable the rules because some | people tend to skip it if they have an option and we want | to keep the quality of the user experience as high as | possible." | miroljub wrote: | They say it's under the Apache 2 licence, so it is open source. | | EDIT: I was wrong. They actually released _binaries_ under the | Apache licence, not the source code. Which is, mildly said, | deceptive. I don 't even have an idea what that actually means. | martypitt wrote: | They say the binaries are being made under Apache 2. | | They don't say anything about the source code being | published. That's why (to me) this is so interesting. I've | never seen binaries released without source code before. | stefan_ wrote: | What is even the point of releasing binaries under Apache | 2? When I patch the binaries, do I need to release a | hexdiff too to fulfill my Apache obligations? Very weird. | politician wrote: | I suppose you could run it through a Java decompiler and | clean up the results with an LLM. How long would that | take? 5 years to make it useful? | pjmlp wrote: | We used to call it Public Domain and Shareware (with | variations like Coffeeware, Beerware, Postware,...) | a2800276 wrote: | They licensed the binary under Apache. It's a publicity | stunt. | lolinder wrote: | Making your product available for free isn't a publicity | stunt, it's a huge step for a business. And, in practice, | it's not that much different for the average user if only | the binaries are Apache licensed. When was the last time | you needed to open up the Postgres source code and modify | something? | amluto wrote: | Never. On the other hand, I have considerable confidence | that I could do so, and that if something goes wrong with | upstream development, someone is likely to do so. | | If I use a free-binary-but-no-source product, I'm much | more likely to get stuck. | | (Of course, as a regretful MySQL user, I am pretty stuck, | but largely because MySQL is, in many respects, a | terrible product. It does, quite reliably, get security | updates at the kind of general maintenance that keeps it | working no worse than it ever did.) | ingenieroariel wrote: | Today I looked up pgvector's NixOS availability. For the | past 15 years I have relied on postgis source being | available and improved by the community for my day to day | business. | | My point is that the option to modify the source results | in software bein available and community maintained in a | way that binary only isn't. Even if I change the source | myself just twice a decade. | a2800276 wrote: | If it wasn't a publicity stunt, it certainly had the | effects of one: I've never heard of Datomic before and | here they are at the top of hackernews! | | > And, in practice, it's not that much different for the | average user if only the binaries are Apache licensed. | When was the last time you needed to open up the Postgres | source code and modify something? | | Sure, if you're playing a game it probably doesn't make a | difference. If I'm building my IT infrastructure on a | product, tt makes a huge difference if I get a an open- | source-licensed "binary" or access the to source: | | - the package they distribute contains no less than 960 | different jars. Most of those are the standard apache- | project-everything-and-the-kitchen-sink-style | dependencies. Say I'd like to update log4j because it | contains a catastropic vulnerability that datomic decide | not to fix. (not that that sort of thing ever happens) | | - or say Datomic decides to abandon the product | altogether or goes out of business | | - or say I'm not happy with their quality of service | contract around their DB they support and would like to | work with a different company | [deleted] | lolinder wrote: | Again, with your hypotheticals--when was the last time | you needed to do any of that with Postgres or another | FOSS DBMS? | | For the vast majority of use cases, a FOSS DBMS and a | free-as-in-beer DBMS are indistinguishable. If you're in | a category where they're not, then don't use Datomic, but | this is still far more than a publicity stunt. | a2800276 wrote: | We must be working in a different world. In all my career | I've not once worked with a serious business that did not | have a support contract for their database system open | source or not. | | Most of those had escrow agreements for central closed | source components with vendors in case the vendor went | out of business. (obviously only for things perceived as | critical and from companies with some perceived risk of | failure). | | And god knows how many times have I experienced companies | biting themselves because they bought into a product that | turned out not to deliver what was promised after the | contracts were signed. | xmlblog wrote: | Free beer binaries are not mutually exclusive of | Enterprise support agreements featuring all those things | you mentioned above _for people that need that_. | a2800276 wrote: | Completely agree. I'm fine with a free beer license. The | context of the post is that the binary is licensed using | an Open Source license which leads to confusion. | nightski wrote: | Rich Hickey started Datatomic (along with Stuart Halloway | & Justin Gehtland). He also created the Clojure | programming language and has been on Hacker News numerous | times with many popular talks. In fact they all have made | famous contributions. | | Many businesses use Microsoft SQL Server or Oracle and | don't need access to the source. I'm not saying open | source isn't nice, but it is absolutely not a requirement | for IT infrastructure. | | I'd imagine people rely on many cloud services that are | in fact, not open source. | [deleted] | eternalban wrote: | Someone (forget who but he worked there) was giving a | presentation of Datomics in some downtown (NYC) bank circa 2014 | iirc. Per the presenter -- iirc someone asked a specific | technical question -- even people working for the company don't | get to see the full source. Only a small team has access to the | full source, and he said he wasn't one of them. | JimmyRuska wrote: | RDFox is worth a try as an alternative, also datalog but C++ | based, has incremental reasoning, and explainability. It's a | database but also a rules engine that can chain any number of | rules. As far as I know datomic is unique for its "query the | database at any point in history" and incremental tracking of | schema changes, easy to use UDFs, it really shines above other | databases in that context. | rafaelturk wrote: | How datomic is Better/worse that symply using DynamoDB? | brianwawok wrote: | Datomic LOOKED cool 10-12 years when it first came out. But they | started from day 1 with a price, so most people, me included, | just passed over it. | | I think they went way too fast to commercial, and needed to go a | freemium model to actually get market share. | xpe wrote: | This doesn't quite reflect the history. Datomic had various | free/trial options. They evolved a little bit. Someone who | watched the pricing and licenses very closely probably could do | a better timeline than I could. | brianwawok wrote: | Right but it was always very clear "you can have it in dev | for free, but prod is $$$$$". It was not something like "use | it free in prod as much as you want, or pay us for support" | | I had a few projects it would have been cool on, but I just | did postgres instead and won in the long run. | blatant303 wrote: | Datomic is an event-sourced db, and it makes it hard to introduce | retroactive corrections to the data when your program's semantic | already rely on using datomic's time travelling abilities: at one | point you'll need to to distinguish between event time and | recording time as explained in this excellent blog post: | | https://vvvvalvalval.github.io/posts/2018-11-12-datomic-even... | | This is why I' rather use XTDB [1], a database similar to datomic | in spirit, but with bitemporality baked in. | | [1] https://www.xtdb.com ___________________________________________________________________ (page generated 2023-04-27 23:00 UTC)