[HN Gopher] Show HN: Multiplayer Demo Built with Elixir
___________________________________________________________________
Show HN: Multiplayer Demo Built with Elixir
Hey HN, I'm an engineer at Supabase [0] and one of the creators of
this demo. My team and I have been working hard to bring developers
the next version of Supabase Realtime. The current version of
Realtime [1] is a Change Data Capture (CDC) server for a PostgreSQL
database that broadcasts changes via WebSockets to authorized
subscribers. It's written in Elixir/Phoenix. The server utilizes
PostgreSQL's logical replication functionality, which writes
database changes to Write-Ahead Logging (WAL) segment files, and a
replication slot, responsible for managing and retaining WAL files.
Database changes are polled from WAL by the server using
PostgreSQL's replication function pg_logical_slot_get_changes and
changes converted to JSON objects using the wal2json [2] extension
by setting it as the output plugin. Security is enforced through
two checks - each check ensures only authorized client subscribers
are sent database changes. The first check validates a JWT that is
sent by clients subscribing to database changes. This JWT must
contain an existing database role and optional claims, both of
which can be referenced in Row Level Security (RLS) policies. Every
valid client subscription is then inserted into the
realtime.subscription table with an assigned UUID, database role,
and claims. The second check calls the realtime.apply_rls SQL
function from Write Ahead Log Realtime Unified Security (WALRUS)
utility lib [3]. This function takes the database changes, executes
a prepared statement to verify if the database role and claims have
SELECT permissions on the changes, and outputs an array of
authorized UUIDs. Then, the server finds all the subscribers whose
UUIDs are in that array and broadcasts the changes to them. The
next version of Supabase Realtime will offer three features:
Broadcast, Presence, and Extensions. Broadcast, our Pub/Sub
offering, can be used to pass ephemeral data from client to client
such as cursor movements. This runs on a distributed cluster of
nodes built on top of Phoenix PubSub + Channels. Presence, can be
used for tracking online/offline users and their state. This is
built into Phoenix, and uses replicated state across a cluster
using an Observe-Remove-Set-Without-Tombstones (ORSWOT) CRDT [4]
which prefers adds over removes when resolving conflicts.
Extensions, are a way for the community to add additional
functionality to take advantage of the WebSocket infrastructure. We
have converted the existing Change Data Capture system to an
extension that supports connecting to multiple customer databases
(multi-tenancy). Other possible extensions include listening to
other databases like MySQL and getting stock market events server-
side [5], then broadcasting them to connected clients. This demo
is built using a Supabase project, Supabase Realtime, and Next.js
and deployed on 20 Fly [6] nodes located around the world. You can
find an introduction and walkthrough of the demo here [5].
Supabase Realtime is entirely open source and you can find the demo
code here [7]. Once we have stabilized the release we will add it
to the self-hosted offering [8]. This demo is a way to highlight
the upcoming features and gather feedback/ideas. Feel free to ask
me anything and let me know what you think! [0]
https://supabase.com [1] https://github.com/supabase/realtime [2]
https://github.com/eulerto/wal2json [3]
https://github.com/supabase/walrus [4]
https://gsd.di.uminho.pt/members/cbm/ps/delta-crdt-draft16ma...
[5] https://supabase.com/blog/2022/04/01/supabase-realtime-with-...
[6] https://fly.io [7]
https://github.com/supabase/realtime/tree/multiplayer [8]
https://github.com/supabase/supabase/tree/master/docker
Author : wenbo
Score : 125 points
Date : 2022-04-11 18:15 UTC (4 hours ago)
(HTM) web link (multiplayer.dev)
(TXT) w3m dump (multiplayer.dev)
| kbyatnal wrote:
| If I understand this correctly, it's basically transmitting
| database changes over a websocket? If so, how does that work re:
| performance when it comes to something like live cursor
| positions?
|
| Writing to the DB every 50ms for each cursor on the screen
| doesn't seem great for performance...or am I missing something?
| kiwicopple wrote:
| This example is built with our refactored Realtime server which
| solves that issue. (We had a lot of developers trying exactly
| that, and it ended poorly for their database)
|
| The new version adds "Channels", which are used for sending
| ephemeral/high-frequency events to other connected users.
| wenbo wrote:
| It depends on the task at hand. If you wish to share cursor
| positions then you'll be using the new Broadcast feature where
| data is not written to the DB but forwarded to other clients.
| That's how we've implemented the multiplayer cursor positions
| in this demo.
| seanwilson wrote:
| This is great. Do you need to be cautious of overloading your
| Supabase database and impacting performance with heavy use of
| multiplayer?
| michelpp wrote:
| Supabase dev here but not a developer of this particular
| project, so I don't have the exact details or numbers, but for
| sure be considerate using a tool like this, it can add loading
| challenges in a number of dimensions.
|
| In particular when it comes to the database side you're going
| to want to limit individual realtime event writes. I suggest
| write to an append only log table and process the data in
| batches using a background worker.
|
| If you intend to insert every event one by one and index them
| various ways and have foreign keys to and from, inserting is
| going to incur a high cost. This can be mitigated for the most
| part with temporal partitioning, but even then it might be fine
| for you use case or analytic case to do background updates over
| an append only log and keep your appends quite fast.
|
| As for the network traffic and loading of the HTTP front end
| someone else on the team can maybe chime in on that.
|
| EDIT: Note this is very broad advice for any realtime app and
| not specific to this product.
| chasers wrote:
| Yeah this is one of the reasons for these new features. So
| you don't have to send things like cursor positions,
| currently online, typing, etc through your database.
| wenbo wrote:
| > cautious of overloading your Supabase database
|
| I think just being mindful about disk space and CPU usage will
| be helpful when setting up a database for logical replication
| because of changes being recorded to WAL files and having
| Realtime poll the database for changes.
|
| Our initial version of Realtime streams database changes to the
| server and those get broadcast to clients. However, the only
| security offered is checking whether or not a JWT is valid when
| client attempts to connect.
|
| The second and current version of Realtime, which I discussed
| in this post, checks database role and claims of subscriptions
| for every database change against RLS policies. This comes with
| a performance tradeoff as it prioritizes security. We are
| benchmarking this version in our new Supabase Realtime version
| and our team will try and optimize performance.
|
| > impacting performance with heavy use of multiplayer
|
| Realtime only needs a single connection to the database, and
| once a node gets the changes, it'll broadcast them to all other
| nodes in the cluster which will then be forwarded to all
| clients. This is highly scalable and nodes can be added in
| different regions experiencing higher loads. We're going to add
| in rate limiting to make sure that the cluster remains healthy,
| but those rate limits can be customized depending on the use
| case.
| kiwicopple wrote:
| > overloading your Supabase database
|
| It's also worth mentioning that "Presence" and "Channels"
| don't put any additional load on your database. You can
| broadcast messages (like the cursor movements in this demo)
| without it touching your database.
| wenbo wrote:
| I want to add that each room is capped at five players. Whenever
| you go to https://multiplayer.dev, you're assigned to a room with
| the fewest players so that no one is alone for too long. If you
| feel stuck in a room try navigating back to
| https://multiplayer.dev.
| alberth wrote:
| Hi wenbo - would you mind explaining a bit more on what the
| user is suppose to experience in this demo. Maybe I'm the only
| person confused but I just see people chatting in the bottom
| right bubble. And occasionally, someone moves their cursor. I
| don't feel like I'm try grasping what is being "demo'ed".
| wenbo wrote:
| The purpose of this demo is to show off three features of the
| new Supabase Realtime.
|
| - Broadcast: Cursor movements and messages (floating bubbles
| next to cursors) sent directly to other players in the room.
|
| - Presence: Whenever someone joins the room, you'll be able
| to see that person represented as a colored circle in the
| upper right hand cover. When they leave, then their circle
| disappears.
|
| - Change Data Capture / Replication: When a player sends a
| message, that message is inserted into the database, and that
| change is then broadcast to all players in the room. You can
| see these message in the chat box in the lower right hand
| corner.
|
| - Latency: You can see how long it takes your client to send
| a message to the nearest node in the cluster and have the
| node respond with a message. This will vary depending on your
| internet.
| cupofpython wrote:
| in contrast, I had a pleasant experience with people engaging
| with each other by leveraging the text bubble next to the
| cursor and then multiple people putting the cursor at the
| same place and shaking it for fun
| kiwicopple wrote:
| wenbo recorded a short video earlier which might be easier
| for mobile users to see what's going on:
| https://youtu.be/BelYEMJ2N00?t=103
|
| more details and full video in this blog post:
| https://supabase.com/blog/2022/04/01/supabase-realtime-
| with-...
| FractalHQ wrote:
| Oh wow- I'm a huge fan of Supabase and have loved using it in a
| few Sveltekit projects so far! I gotta ask- how straightforward
| would it be to power a collaborative code editor with this?
|
| I have a YJS powered collaborative Svelte REPL I built using
| y-monaco and y-webrtc, all "serverless" on Vercel using CFWorkers
| to connect/verify users. Supabase already houses the private REPL
| loading/saving/sharing features. Native Supabase CRDTs could be
| perfect for the next version of the app!
| kiwicopple wrote:
| > CRDTs
|
| definitely. it's one of the primary use-cases we had in mind
| when developing this, and something we've wanted to offer for a
| long time. I'm happy to see it popping up early in the
| comments. We wouldn't offer our own CRDTs, but Realtime can be
| a nice transport layer for other CRDT implementations (which
| can then be serialised and persisted to your database)
|
| > YJS powered
|
| I'm also glad to see you're using Yjs - it's very cool. We hope
| that this implementation can be another Yjs Provider[0] if
| Kevin is onboard with that. Once that's implemented, you would
| be able to use it with all the same bindings (i.e. y-monaco).
|
| [0] https://github.com/yjs/yjs#providers
| [deleted]
| jacquesm wrote:
| Very interesting, I can see a whole raft of applications for this
| tech that are outside of the game scene, various collaboration
| tools use very similar tech under the hood and this would be a
| very nice way to abstract some of that out and to lower the bar
| to launch collaborative applications.
| wenbo wrote:
| Great observation and exactly how we've envisioned this new
| Realtime.
| nitinagg wrote:
| Neat demo. What are the scaling limitations?
| wenbo wrote:
| For the Change Data Capture functionality, I mentioned that
| there's a tradeoff between performance and security in another
| comment (https://news.ycombinator.com/item?id=30994062).
|
| For Broadcast, we can scale by adding additional nodes to the
| cluster but up to a point. The current architecture works
| because every node is aware of all other nodes in the cluster,
| but this strategy becomes untenable after a certain point. The
| good news is that we're nowhere near that point and we'll
| optimize and develop other strategies to circumvent this
| eventual limitation. We'll also need to carefully manage and
| process the inflow and outflow of messages on each node to make
| sure we're not delaying message delivery or using more memory
| than absolutely necessary.
|
| For Presence, it syncs the joins and leaves from node to node
| every 1.5 seconds (default) and there will be a point where
| this work will take longer than that interval. Fortunately, the
| underlying ORSWOT CRDT implementation can be refactored to make
| syncing more efficient.
| jacquesm wrote:
| That sounds like you might take good advantage of machines
| with more than one network card, send the local updates via
| broadcast to all the machines in a cluster via one network
| and do the internet side via the other.
| Kinrany wrote:
| Since you already have cursors, make it into a game of catch by
| showing small explosions when cursors collide :D
| wenbo wrote:
| I like that!
| Mizza wrote:
| This is awesome. I recently built a multiplayer Phoenix LiveView
| experience - https://internetfight.club - and in the process had
| the idea for a fully DB-reactive dev experience, seems like you
| scratched the itch first! Well done. I think this is the future
| of web development and I'm quite happy about it. That being said,
| I wonder how far you could get on ETS alone..
| wenbo wrote:
| Would love to learn more about how you built Internet Fight
| Club! What were your design decisions and what challenges did
| you encounter? Also, what do you mean by ETS alone?
| Mizza wrote:
| It was mostly a learning project for me, but I'm planning to
| do a write-up soon as a guide for Python developers moving to
| Elixir.
|
| Re: ETS, I was just imagining something like Realtime but
| without Postgres, just using ETS. Most of the use cases I can
| think of for this type of thing don't require relational
| data, so I was thinking of just using ETS as a KV/document
| store to hold user and application state and then reacting to
| changes in to that the way you are here. I think you'd lose
| the benefits of relational queries and persistence but gain
| some function in overhead and decentralization. I'm moving to
| Elixir after spending a lot of time doing "serverless" and
| event-driven architectures, so now I'm spending a lot of time
| thinking about how to get the benefits of both worlds.
| wenbo wrote:
| > write-up soon as a guide for Python developers moving to
| Elixir
|
| Awesome initiative!
|
| > ETS as a KV/document store to hold user and application
| state and then reacting to changes in to that the way you
| are here
|
| This is actually pretty interesting. I can't speak to ETS
| but Mnesia has replication and you can expose the
| replication log using something like
| https://github.com/emqx/mria. I've only had a cursory look
| at this so I could be wrong about its capabilities but it
| would be an awesome extension to the new Realtime if
| possible.
| yumraj wrote:
| I believe OP meant https://www.erlang.org/doc/man/ets.html by
| ETS
| tough wrote:
| Trying to run the demo I get a blank loading page on my nextjs
| app
|
| ```realtime-realtime-1 | Transport: :websocket realtime-
| realtime-1 | Serializer: Phoenix.Socket.V1.JSONSerializer
| realtime-realtime-1 | Parameters: %{"apikey" => "nokey", "vsn" =>
| "1.0.0"} realtime-realtime-1 | 2022-04-11 22:30:17.543 [warn]
| Ignoring unmatched topic "room: _" in RealtimeWeb.UserSocket
| realtime-realtime-1 | 2022-04-11 22:30:17.556 [warn] Ignoring
| unmatched topic "room:NAejZl5PKVbAwUsxCucM-" in
| RealtimeWeb.UserSocket realtime-realtime-1 | 2022-04-11
| 22:31:11.144 [warn] Ignoring unmatched topic "room:_" in
| RealtimeWeb.UserSocket realtime-realtime-1 | 2022-04-11
| 22:31:11.166 [warn] Ignoring unmatched topic
| "room:jQCdG-h3NT8nFVk7BpCAc" in RealtimeWeb.UserSocket ``` Any
| idea what I might be missing?
| chrisco255 wrote:
| Nice job, but the demo rendered extremely weird on my mobile
| device. Some weird jankiness forced half the site off screen.
| Hard to see what's happening.
| wenbo wrote:
| Sorry about that! We've been prioritizing desktops but will be
| working on making it mobile-friendly in the future.
| burggraf wrote:
| Supabase developer here -- I'm excited to use this myself!
| wenbo wrote:
| Have you signed up for the waitlist? ;)
| abc3erl wrote:
| :D
___________________________________________________________________
(page generated 2022-04-11 23:00 UTC)