[HN Gopher] Show HN: I rewrote the 1990's LambdaMOO server
___________________________________________________________________
Show HN: I rewrote the 1990's LambdaMOO server
I got my start on the Internet in the very early 90s playing with,
authoring in, and programming on LambdaMOO
(https://en.wikipedia.org/wiki/LambdaMOO) and similar systems.
Shared virtual social spaces, with a persistent object oriented
authoring / scripting language. They can be classified as MUDs
(depending on who you talk to) but the focus is social, creative /
authoring, and shared programming not RPG gaming. I've always
wanted to see this kind of thing modernized and further developed.
Over the last 25 years or so I've worked on similar but novel &
improved things, but never finished. So I decided to just re-
implement LambdaMOO and use that as a base, instead and keep
compatibility as a goal, but build it out on a more modern
foundation that takes advantage of multiple core machines, newer
network protocols, newer connectivity methods, uses MVCC
transactions for the shared database etc. LambdaMOO is a somewhat
extensive system in that it is composed of compiler, a virtual
machine, an object database, user permissions system, network
runtime. In some ways it's kind of like a shared, text-based
Smalltalk image/runtime... So quite a bit to implement and get
right before it all works together. The big challenge throughout
has been slavishly maintaining backwards compatibility so existing
"cores" (databases) work. It's not done, but it's darn close.
Would like for people who are into this kind of thing to check it
out, and maybe even help. Many of the technical aspects here are
still provisional, but this is the start. Constructive assistance
welcome. (Yes, it's a rewrite in Rust, but that's not really the
point, even though that's a cliche that's fun.)
Author : cmrdporcupine
Score : 115 points
Date : 2023-09-23 15:56 UTC (7 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| [deleted]
| mindcrime wrote:
| _I 've always wanted to see this kind of thing modernized and
| further developed._
|
| I've had similar thoughts, so let me say "Congrats, this is
| awesome"! It definitely warms the cockles of my heart to see work
| like this. I just wish I had the free cycles to work on something
| similar (or even contribute to your project). But there are too
| many demands on my time right now. :-(
| gilmore606 wrote:
| How are you handling concurrency/multithreading? I wrote a lot of
| MOOcode back in the day and often explicitly relied on the fact
| that property values couldn't change out from under me during a
| verb's runtime.
| cmrdporcupine wrote:
| Like a database (because it is one). MVCC. Full transactional
| isolation. Nobody sees your changes until successful commit.
|
| This is what MOO was effectively mostly doing already, through
| a big ol' single threaded interpreter we all took turns on.
| Only one task was running at a time (sort of, you could
| suspend() and return later, and then all bets were off on
| consistency). It's just... that doesn't really scale too well.
| trollied wrote:
| Hands up if you know what tinyfugue is :)
| tromp wrote:
| I spent a good fraction of my years of MUD playing time on
| writing ever more elaborate tf macros to automate as much MUD
| play as possible. $ wc ~/tfrc 878
| 6634 53336 /Users/tromp/tfrc
| stavros wrote:
| That's how I learned to program. At one point all my fighting
| was done for me.
| blipvert wrote:
| Now that is a name that I've not heard in a long time ...
| cmrdporcupine wrote:
| I almost used it in the screenshot, but the live editing of
| code with emacs is way hotter :-)
|
| Check out blightmud. It's kind of tinyfuge-the-next-generation.
| smirya0 wrote:
| Nice. I still use VMOO (anyone remember VMOO)! Is that
| rmoo/emacs on Fedora?
|
| There's a nice collection of MOO resources:
| https://lisdude.com/moo/
|
| It is too bad some of the old MOO dbs like ScrabbleMOO and
| NYC MOO are lost to history...
| stavros wrote:
| MUSHclient or nothing.
| kevinsync wrote:
| TinTin++ over here
| VadimPR wrote:
| Mudlet represent :)
| [deleted]
| nrr wrote:
| This makes me smile a lot. Whenever I pick up a new programming
| language, one of the first projects I tackle is writing a port of
| a similar MU* server because of the same breadth of concerns, so
| it's very nice seeing someone else recognize the complexity and
| take it on as a worthy challenge. (Though, this goes the extra
| mile and modernizes things; I've seldom accomplished that.)
|
| I hadn't looked at doing this in Rust yet, so it'll definitely be
| interesting to give the code a read, particularly with the focus
| toward modernization.
| smirya0 wrote:
| Very cool! Is it still using cooperative multitasking? My
| understanding was MOO verbs/tasks run until the verb does
| suspend/fork or an input() call happens or something like that,
| or until the verb runs out of "ticks" and is killed by the VM.
| How do you do it?
|
| Does it use async I/O / is it one thread still?
|
| You might find this conversation interesting about multi-
| threading in LambdaMOO: https://groups.google.com/g/moo-
| talk/c/omF68ZM9rZc/m/B3f-jj4...
| cmrdporcupine wrote:
| It's multithreaded with multiversion concurrency control over
| shared state. Each command invocation is a new thread, in a new
| transaction, and the database attempts to resolve any shared
| state conflicts at commit time.
|
| There's still ticks and time limits, but they're there to
| impose time / resource limits, not used to control access to
| shared state.
|
| Tick-based time-slicing to the database is one reason why
| LambdaMOO (and its offshoots like toaststunt, etc.) has
| intrinsic scalability limits. (In the 90s, LambdaMOO-the-
| actual-MOO used to lag like a son of a bitch under load...)
|
| (This is not to say the actual impl of MVCC I have right now is
| ideal. It's a work in progress. I just defer to RocksDB's
| conflict resolution, and it is built on assumptions of "single
| writer, multiple readers" at its foundation, which isn't ideal
| for what I'm doing... )
| miki123211 wrote:
| Can this transactional model be made to handle in-transaction
| I/O?
|
| I've seen MOOs that had verbs for performing HTTP requests,
| through shelling out to curl IIRC. If you have a verb that
| reads a value, sends it out and updates it based on the
| response, what do you do if the value changes in the
| meantime? When is the transaction committed exactly?
| cmrdporcupine wrote:
| I won't be providing outbound HTTP (or inbound) requests
| via MOO code. The only I/O is what is provided through the
| Session interface which is, yes, transactional; all I/O
| "effects" are buffered and committed or reverted after the
| same at the DB level. Inbound HTTP is provided at the Rust
| layer in front of a standardized RPC interface over 0MQ and
| goes through the same set of transactional semantics as all
| other commands:
|
| The transaction gets committed at the end of command
| execution, and then all the I/O that "occurred" in the
| narrative log gets output to its destinations. If a
| serialization conflict occurred, the output buffers (and
| all other mutations) are discarded, and everything retried,
| again. (In theory. Right now it just rolls back and barfs,
| because I have not finished this layer yet :-) )
|
| Where this falls over is with MOO's synchronous "read"
| builtin, which is used in places for e.g. password changes
| or prompts. This is something I have not tackled yet, but
| will probably just have to deprecate, and will be one of
| the only things breaking compatibility with existing cores.
|
| Alternatively I could or might provide a transaction mode
| which is "softer" for tasks that need that kind of
| dependent input.
|
| It will require some thought. But on the whole MOO's model
| is similar to a database: In a DB it's query in, commit,
| see results; in my system in it's command parse, execute
| verb, commit, flush I/O results out.
| skulk wrote:
| I love seeing projects like this! I have one too, but it
| completely throws away compatibility and is very simplistic. I
| opted for a lisp-like language instead of a lua-like language
| (https://github.com/sid-code/nmoo but don't look at the code,
| it's embarrassing).
|
| I have some questions: - Does each verb call
| create a whole new world state that it mutates, and later commits
| to the database? That's the impression I get from reading the
| code. Does this come with a lot of overhead? - What is that
| emacs mode you're using to edit verb code?
| cmrdporcupine wrote:
| I've done the "whole new thing" before, too. 20ish years ago,
| tho I only have a few fragments of what I worked on back then:
| https://github.com/rdaum/mica being one of them I found on an
| old drive. Not complete.
|
| But sticking with compatibility has allowed me to enforce
| development discipline, basically. And then I'll move it
| onwards from there. I have many ideas. And a product concept
| (if I could find an investor...). But I felt it important to
| start with a thing with existing foundation and use cases
| before doing the exciting stuff/changes.
|
| Re: world state / transactions -- yeah, basically all I/O and
| mutations happen in a transactional context, and then at commit
| time conflicts are resolved; if they're not resolve-able, the
| transaction is retried in a new state. As for overhead, yes
| potentially maybe a lot, but it's also a solvable problem; this
| is how an MVCC SQL database (like, even Postgres) works. TLDR
| it's likely inefficient now, but I believe I can make it
| efficient. And I think it's the best way to solve the shared
| world state problem and still meet user's expectations of
| consistency. Original LambdaMOO had a global interpreter lock
| and only one user was mutating the world at a time, in their
| given tick slice. Not gonna scale.
|
| Re: the MOO client, it's `rmoo.el`:
| https://github.com/lisdude/rmoo -- it's been around for a long
| time (25, 30 years?) and it and/or MOO.el (another emacs one)
| are how/why I learned emacs in the first place. I had to minor
| patch my local copy to make it work with emacs 29.1, though I
| don't know elisp well enough to say whether my patch is the
| right thing to do or not, so...
|
| EDIT: I just re-read and saw you asked whether each new verb
| call creates a new state, and actually it's not that. It's each
| new command or network transaction, which can in fact lead to a
| chain of verb (method) calls. Recall a MUD has a command loop,
| so basically a transaction begins at command entry, and commits
| when all the invoked methods/verbs provoked from that complete.
| derefr wrote:
| > Original LambdaMOO had a global interpreter lock and only
| one user was mutating the world at a time, in their given
| tick slice. Not gonna scale.
|
| I've previously seen (abandoned) attempts to convert the
| Erlang runtime directly into a MOO runtime. Each command
| being a concurrent actor-process would make a lot of sense to
| me. (Though a semantics where all commands are MVCC against
| the at-inception world-state and will rollback or retry on
| conflict, kind of moots having any kind the advantage of a
| runtime based on message-passing, since you're not going to
| be passing messages between separate concurrent "sessions.")
| bcrosby95 wrote:
| > Each command being a concurrent actor-process would make
| a lot of sense to me.
|
| I've tossed around doing MUDs in Elixir before. I'm not
| familiar with MOO though.
|
| State has to be owned by a process - who owns the state of
| the world? Process per "thing" is too fine grained - you're
| going to have race conditions and deadlocks. Process per
| "room" can work, but it's still not easy. There are still
| some cross-room coordinating actions you need to tighten
| up, and this makes general scripting very hard.
|
| Keep in mind, when data is passed "between" processes it is
| deep copied. This can be a big problem depending upon your
| process model and how many things your players can carry
| around.
|
| As an experiment I made my own software transactional
| memory. It worked, _assuming_ no network partitions, but
| designing and testing it to work in an environment with
| those was beyond me.
| derefr wrote:
| > State has to be owned by a process - who owns the state
| of the world? ... Keep in mind, when data is passed
| "between" processes it is deep copied.
|
| The Erlang runtime has other stateful things besides
| processes, and manipulating data within these and/or
| getting a handle on data in these places doesn't
| necessarily involve copying.
|
| * ETS tables, which hold state privately to themselves
| (with copying in/out) in a similar way to how processes
| do. If you manipulate data inside ETS tables by "sending
| compute to data" (think Redis INCR) rather than by
| "sending data to compute", then no copying happens.
|
| * "Globals" in the form of data compiled into read-only
| versioned modules loaded into the modules table and
| available to be referenced from any/all active processes,
| only copied if the module gets unloaded before the
| process dies. (Originally this was just a design pattern
| -- https://github.com/discord/fastglobal -- but it
| eventually became its own runtime feature in Erlang 22,
| https://www.erlang.org/doc/man/persistent_term).
|
| * Large binaries (anything over 64 bytes) aren't
| allocated in an actor's memory arena, but instead are
| allocated in a special global-per-node binaries heap, and
| then ref-counted, where each actor-process holds one
| reference to each large-binary it's using, and then each
| read-only slice of that binary, in turn holds a reference
| to the per-actor reference handle for the binary. One
| clever technique for sharing a large "database" of data
| between many actors, is to store the data encoded in a
| large binary in an encoding that is efficient to
| partwise-decode; and have the "lookup" operation just
| parse+decode the appropriate data out of the binary.
| (This is how erlang:module_info/2 used to work -- the
| global modules-table itself holding a set of references
| to the loaded modules' binaries, which module_info would
| then parse metadata out of on demand.)
| intelVISA wrote:
| > As an experiment I made my own software transactional
| memory. It worked, assuming no network partitions, but
| designing and testing it to work in an environment with
| those was beyond me.
|
| BFT STM is a noble goal and, tbh, building performant
| fault tolerant atomic transaction systems outside of C++
| must be quite challenging with all the abstractions atop
| it
| cmrdporcupine wrote:
| Yeah the whole actor thing is kind of orthogonal to the
| question of how you handle the shared world state (which is
| what makes a MOO a MOO.) There are parts of my runtime that
| look Erlang-ish, Actor-ish, because that's a concurrency
| pattern that Rust does well with, in general.
|
| In the end a MOO is just a database with fancy stored
| procedures. And the original code is not a particular...
| good one. Handling serialization conflicts is what
| databases have to do as one of their core abilities, and
| LambdaMOO just tries to punt on the whole question.
| smirya0 wrote:
| how did you come to pick Rust (just curious, vs Elixir,
| Java with Loom, etc)?
| cmrdporcupine wrote:
| These days Rust is both my day job language (I work on
| systems for autonomy in tractors) and my hobby language.
| It's my preferred tool.
|
| I have attempted similar systems in C++ before, and got
| fairly far along at times. But Rust really solves a lot
| of problems, I tear my hear out a lot less.
|
| This is a "systems" type of application; this is a
| database and compiler and a virtual machine and language
| runtime. I'm not super concerned with optimization at
| this point, but I will likely have to be at some point.
| Being closer to the metal is key for me on a project like
| this.
|
| I'm likely to e.g. rip out RocksDB and replace it with my
| own database engine at some point (something I work on
| here and there as another side project, unpublished
| still.) I will also likely need a better garbage
| collection solution. With Rust I have options.
|
| The community of libraries available for Rust is good.
| The right tools are there. And the language is expressive
| enough to make it work well.
|
| That said, I have mulled writing the "host" portion --
| the layer that manages network sockets and web endpoints,
| in Elixir/Erlang:
|
| The 'daemon' portion of the system sits behind an RPC
| layer built around ZeroMQ -- so that layer can be
| restarted/upgraded without losing active network
| connections. Or can sit on a separate machine for
| scalability reasons, etc. So it's entirely possible to
| split what I'm calling the "host" (network facing layer,
| what we used to call a "frontend" when I worked at
| Google, but most people think of that term as referring
| to a UI layer) from the backend "daemon."
|
| Anyways for me this project ties together a bunch of
| interests in systems programming I've had for years. It
| becomes a kind of synthesis project.
| dang wrote:
| Related:
|
| _LambdaMOO Takes a New Direction (1992)_ -
| https://news.ycombinator.com/item?id=35856654 - May 2023 (15
| comments)
|
| _LambdaMOO takes a new direction (1992)_ -
| https://news.ycombinator.com/item?id=22680965 - March 2020 (29
| comments)
|
| _Lambda MOO Programming_ -
| https://news.ycombinator.com/item?id=20405167 - July 2019 (1
| comment)
|
| _Lambda MOO Programming Resources_ -
| https://news.ycombinator.com/item?id=16327975 - Feb 2018 (1
| comment)
|
| _Exploring 3-Move - A LambdaMOO inspired environment_ -
| https://news.ycombinator.com/item?id=14075439 - April 2017 (15
| comments)
|
| ---
|
| The "new direction" article was made famous by Clay Shirky in "A
| group is its own worst enemy". I won't list those here since it's
| a generic tangent but there are past threads at
| https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que....
| Alex3917 wrote:
| Also, the article A Rape in Cyberspace is about something that
| happened in LambdaMoo, and is one of the most famous articles
| ever written about the Internet:
|
| https://www.villagevoice.com/a-rape-in-cyberspace/
|
| It looks like it's been submitted to HN several times, but has
| never gotten many upvotes.
| DicIfTEx wrote:
| "A Rape in Cyberspace" was later collected into a book called
| _My Tiny Life: Crime and Passion in a Virtual World_ , all
| about the author's relationships and experiences in LamdaMOO.
| Highly recommend.
| clouddrover wrote:
| To see the original LambdaMOO in action:
|
| telnet lambda.moo.mud.org 8888
| tromp wrote:
| Is there still a connect-4 game somewhere in a games room?
| Shekelphile wrote:
| Good luck. As someone who used to work on a lambda-derived MUD
| it's great to see a little life left in that community. The
| original server was horribly outdated even 20 years ago.
|
| I also found this project randomly on github at one point, not
| sure how far along it got to being usable:
| https://github.com/verement/etamoo
| cmrdporcupine wrote:
| Yes it's funny I came across that one only a couple weeks ago
| -- I may have had wind of it before but it didn't sink in --
| and it's basically what I'm doing (well at least the first part
| of what I'm doing)... but in Haskell. It looks like the author
| got pretty far along, too. But no new commits in some time.
|
| If I'd seen that a year ago, I'm not sure I would have started.
| Hard to say.
|
| I admire Haskell, but I could never really grok it.
| NelsonMinar wrote:
| Wow neat! LambdaMOO was such an influential thing for. MUDs in
| general had a lot of influence, both in social media and gaming.
| And then LambdaMOO had the specific brief of being a social media
| experiment. Neat to see the tech reborn!
|
| Pavel Curtis is still out there, LinkedIn says he recently left
| his job at Microsoft after 20 years. He also runs a puzzle shop:
| http://www.pavelspuzzles.com/site/
| sshumaker wrote:
| I talked to Pavel about taking a role in my team earlier this
| year (I ended up leaving Microsoft myself since). He was
| passionate about making the developer experience excellent -
| code quality, clean APIs, etc. Thats a tall order for some
| parts of Microsoft with 30+-year old codebases. He mentioned he
| was interested in rockets so I hope he found a gig doing that.
___________________________________________________________________
(page generated 2023-09-23 23:00 UTC)