[HN Gopher] Automerge-Repo: A "batteries-included" toolkit for l...
___________________________________________________________________
Automerge-Repo: A "batteries-included" toolkit for local-first
applications
Author : gklitt
Score : 105 points
Date : 2023-11-08 17:19 UTC (5 hours ago)
(HTM) web link (automerge.org)
(TXT) w3m dump (automerge.org)
| scotttrinh wrote:
| Super excited to see Automerge getting this high-level API out.
| Been following since before 1.0 and I can't wait to play around
| with the latest incarnation! Congrats to the Automerge team.
| pvh wrote:
| Thanks, Scott. This API should make it much, much easier for
| folks to build with Automerge and kind of just encapsulates
| everything we've been doing in-house over the last few years.
| zyang wrote:
| Last time I looked into CRDT, automerge was not as fast/efficient
| as yjs, but the team was actively improving the algorithm. Is
| there any benchmark to show the progress.
| heathermiller wrote:
| still not as fast/efficient as Yjs. there are some benchmarks
| here from late September'23: https://arxiv.org/abs/2212.02618
|
| disclaimer: i'm a co-author and the paper is focused on a
| different CRDT framework, but point is that it measures Yjs and
| automerge side by side
| pvh wrote:
| The benchmarks Matt Weidner has been working on are great and
| outside scrutiny is always welcome, but I should note that I
| find there's an element of artificiality to them. In
| particular, testing the performance of the sync system while
| simulating many users typing into the same document doesn't
| really measure behaviour we have observed "in the wild". In
| our research, we've found that editing is usually serial or
| asynchronous. (See https://inkandswitch.com/upwelling for
| further discussion of our collaboration research.)
|
| The benchmark that concerns me (and that I'm pleased with our
| progress on!) is that you can edit an entire Ink & Switch
| long-form essay with Automerge and that the end-to-end
| keypress-to-paint latency using Codemirror is under 10ms
| (next frame at 100hz).
|
| While these kinds of benchmarks are incredibly appreciated
| and absolutely drive us to work on optimizing the problems
| they uncover, we try to work backwards from experienced
| problems in real usage as our first priority.
| heathermiller wrote:
| Ouch Peter. Massive offense taken.
|
| > In our research, we've found that editing is usually
| serial or asynchronous.
|
| Medium-to-large-size company with a town hall = many people
| editing a document at the same time. Workshop at a company
| or a university with a modest size classroom = many people
| editing a document at the same time. I can't tell you how
| many times our web-based collaborative code editors would
| fall over during talks with small audiences we would give
| back in the days when I led the Scala Center.
|
| Just because one of the benchmarks you have seen (of a
| multitude of benchmarks) breaks automerge by stressing it
| in what we believe is the most stressful scenario possible-
| multiple concurrent users, which is sort of the point of
| concurrency/collaboration frameworks, does not make it
| artificial or worth so flippantly discarding.
|
| > long-form essay with Automerge and that the end-to-end
| keypress-to-paint latency using Codemirror is under 10ms
| (next frame at 100hz)
|
| Not at all what we measured.
|
| I'd just like to register here that Yjs is the framework
| most widely used "in real usage" (your words) and not
| automerge (for many reasons, not just performance.)
| pvh wrote:
| Please accept my unreserved apologies, Heather! No
| offense is intended. I can speak for everyone working on
| Automerge when I say that we've very much appreciated
| Matthew's work and have indeed spent quite a lot of time
| studying and responding to it. We spoke about it in
| person last week, in fact.
|
| As for the use-cases, I do not mean to exclude live
| collaboration from consideration, just to note that it
| hasn't been our focus or come up often in the use-cases
| we study. Live meeting notes are definitely a real use-
| case and I don't dispute the performance results you
| show.
|
| As for Y-js, it's a wonderful piece of software with
| excellent performance and a vibrant community made by
| exceptional people like Kevin Jahns. We simply have
| slightly different goals in our work, which undoubtedly
| reflect where our engineering investments lie.
|
| Indeed, your paper did not measure the same things we
| look at, and that's why it found new results. Hopefully
| in time we will join the other systems in performing well
| on your benchmarks as well.
| anglinb wrote:
| This is super powerful, been playing around with the previous
| releases for the past few days. It works really well, but still
| needs a few dx tweaks to make it performant for large
| applications. You have to watch the callbacks yourself to update
| slices of state and unless your app is small enough that the
| whole thing can re-render every update.
|
| That being said, I love everything automerge is doing and hope
| this pace will keep up!
| pcl wrote:
| Cool stuff!
|
| What do you suggest is the sweet spot for document size and
| "hotness"? Your cookbook [0] says "We suspect that an Automerge
| document is best suited to being a unit of collaboration between
| two people or a small group." Does that mean tens of kilobytes?
| Hundreds? More? And how much concurrent contention is viable? And
| is the "atom of contention" the document as a whole, or do you
| have any plans for merging of sub-parts?
|
| Also, do you have support for juggling multiple transports,
| either concurrently or back-to-back? In particular, I'm thinking
| about synchronizing via the cloud when connected, and falling
| back to peer-to-peer when offline. In that peer-to-peer case, how
| many peers can I have, and can my peer network behave as a mesh,
| or must it stick together to some degree?
|
| And finally, it looks like your tutorial [1] doesn't actually
| exist! You refer to it in a blog post [2], but it's a dead link.
|
| [0] https://automerge.org/docs/cookbook/modeling-data/
|
| [1] https://automerge.org/docs/tutorial/introduction/
|
| [2] https://automerge.org/blog/automerge-2/
| pvh wrote:
| The way I think about it is that if the data should always
| travel together it should be in one document. For example -- if
| your TODO list always goes as a unit, then make it an array of
| objects in a single Automerge document. On the other hand, if
| you want to build an issue tracker and to be able to link to
| individual issues or share them individually then a document
| each is the way to go. Does that help?
|
| As for network transports you can indeed have multiple at once.
| I usually have a mix of in-browser transports (MessageChannels)
| and WebSocket connections. I suspect we'll need to do a little
| adjusting to account for prioritization once people really
| start to push on this with things like mDNS vs relay server
| connections but the design should accommodate that just fine.
|
| As for the docs, my apologies. The "tutorial" was merged into
| the quickstart as part of extensive documentation upgrades over
| the last few months. We should update the link in the old blog
| post accordingly.
|
| Here's a link to save you the effort:
| https://automerge.org/docs/quickstart/
| xrd wrote:
| If you are interested in this, check out the video from
| StrangeLoop 2023:
|
| https://www.youtube.com/watch?v=Mr0a5KyD6BU
|
| Also, check out the unconf for localfirst that happened right
| after 2023:
|
| https://github.com/LoFiUnconf/stlouis2023
|
| Ink & Switch is doing such interesting stuff. Their after party
| at StrangeLoop was so cool.
| bryanlarsen wrote:
| It's too bad the unconf was full, I didn't get in. Hopefully
| they do it again.
| idosh wrote:
| How is it compared replicache, watermelondb and the rest?
| bomewish wrote:
| Seems we have a really great technical spec -- but aren't y'all
| gonna build a product on it and let us pay you to use? A google
| docs for markdown/quarto documents would be brilliant but
| apparently does not yet exist...
| pvh wrote:
| Automerge is a library that anyone can adopt, and we are a
| research organization, not a product company.
|
| We have built a variety of projects with Automerge, both
| publicly and for use in private, including recently the
| markdown-with-comments editor we call Tiny Essay Editor
| (https://tiny-essay-editor.netlify.app/) by Geoffrey Litt.
|
| That said, sponsoring the Automerge team helps us build faster
| and is always welcome. (Thanks to our current and past sponsors
| for their support!)
| satvikpendem wrote:
| Nice, I use automerge with Rust via autosurgeon [0] which is
| their Rust wrapper, but looks like it hasn't been updated
| recently, any updates on that? I'm guessing with a small team
| that web support is taking priority right now, as I'm running
| this on my Rust client (technically Flutter but via the FFI
| package flutter_rust_bridge [1]) and server (via the Axum web
| server crate).
|
| [0] https://github.com/automerge/autosurgeon
|
| [1] https://github.com/fzyzcjy/flutter_rust_bridge
| izak30 wrote:
| We're using the same stack, along with automerge-repo-rs, we
| haven't needed much in the way of updates, what are you hoping
| for that doesn't exist?
|
| Edit: Typo `autosurgeon-repo-rs` to `automerge-repo-rs` and
| link. https://github.com/automerge/automerge-repo-rs
| satvikpendem wrote:
| Is autosurgeon-repo-rs separate from autosurgeon? I can't
| find anything by that specific name on Google.
|
| The API is still a little clunky, with hydrating and
| reconciling, and it's not as clean as the automerge-repo one,
| especially with those React examples.
| izak30 wrote:
| Sorry for the typo. Updated. I get what you mean but maybe
| I've gotten used to it. We also added a little sibling
| library to it for partial hydrating and reconciling that
| fits our patterns better.
| https://github.com/bowtieworks/automerge_orm
| parhamn wrote:
| Anyone have any info on who is behind this project, how reliable
| it is (will it be around in 2 years), etc? Considering using it
| for one of my projects.
| pvh wrote:
| Ink & Switch is behind it; or more expansively mostly Orion
| Henry, Alex Good, Martin Kleppmann, and myself. As an
| organization, we have been working on Automerge for about six
| years now. We also have a wonderful community of other
| contributors both in industry and research.
|
| Automerge is not VC-backed software. Indeed, for a number of
| years Automerge was primarily a research project used within
| the lab. Over the last year, it has matured to production
| software under the supervision of Alex Good. The improved
| stability and performance has been a great benefit to both our
| community and internal users. Our intention is to run the
| project as sponsored open source for the foreseeable future and
| thus far we have done so thanks to the support of our sponsors
| and through some development grants.
|
| Ink & Switch's research interests drive a lot of Automerge
| development but funding from sponsors allows us to work on
| features that are not research-oriented or to accelerate work
| that we'd like to do but that doesn't have current research
| applications. If you adopt Automerge for a commercial project,
| I'd encourage you to join the sponsors of Automerge to ensure
| its long-term viability.
___________________________________________________________________
(page generated 2023-11-08 23:00 UTC)