Jan 14, 2020 - Linkulator Release

This post announces the release of a piece of software developed by
myself and two others for use on the increasing number of pubnixes
(including tildes) and especially micro-pubnixes these days.  The
software is called Linkulator [1] and it is a mini, commandline-driven
link aggregator for multi-user servers.  You have probably seen the
Y Combinator News or Lobste.rs websites.  Linkulator is similar, but
used on the command line instead of through a web browser.  And intead
of just aggregating web links, Linkulator also works with gopher links,
local filepaths, or (theoretically) many other resource types.

The operation of Linkulator is very simple.  Start it up and it
will show you a category menu of links that have already been posted.
You may browse the links in these categories, post a new link, or leave
short comments in response to other links.  Links in Linkulator can
be viewed in a user-configurable browser, lynx by default -- meaning
that both html and gopher resources can be viewed.  That's about it.
The UI really is that simple.

Using Linkulator does not require an admin to install anything (other
than a recent Python version).  All you need to do is clone the 
Linkulator repo and start using it.  Of course, Linkulator is most
fun if other users on your system also use it so others should
download it too, or a single copy should be put in a location 
accessible to others.

-- Design

A fun part of developing Linkulator was some of the architectural
decisions we made.  In particular, we wanted some of the advantages
of a setuid program or a server-client architecture, but without
the opacity of a compiled language and without the need for an
admin to run a server or to grant Linkulator elevated privileges.
Linkulator does this by using a distributed content management model;
user contributions (links and comments both) are stored in each
user's home directory.  When the program is started, Linkulator scans
home directories and aggregates all content into the browseable menu
described above.  This would be a costly and noticably slow process
on a big multi-user system like sdf.org, but it is snappy on smaller
systems.  I have extensively tested it on rawtext.club and a bit on
tilde.town as well.  (I have also seen it choke on the thousands of
home directories in SDF and Grex.)

The decentralized data store model yields some interesting aspects of
Linkulator's behavior.  For one, anyone can delete their contributions
at any time and this sometimes impacts others.  If a user deletes
a link that they posted, the link disappears for all users.  If the
deleted link had comments from other users associated with it, those
other comments are not deleted, but they stop appearing in the display
because they no longer have a parent link to associate with.

Another result of the decentralized model is that Linkulator does not
support anonymous posts. This is a difference from pubnix bulletin
boards like SDF's bboard which has an anonymous board or BBJ which has
an anonymous mode.  All of a Linkulator user's content is stored in
a world-readable file in their home directory, so it would be trivial
for other users to unmask anonymous posts anyway.

And finally, the decentralized model makes it easy for users to go
back and edit posts or replies they've left in the past.  On the one
hand, this means users can fix any typos they may have made.  But on
the other hand, a mischeivious user could change their past posts with
resulting change to the meaning of commen threads, causing confusion
or embarassment for others.

The nice thing is that these tradeoffs don't really matter. In most
small pubnixes, users are pretty respectful of each other and the
likelihood of antisocial behavior is pretty low.  If someone does
cause trouble though, Linkulator has an 'ignore' file where you can
list usernames to exclude from display.

One design decision we were forced into because of the decentralized
model was the method for managing category labels.  Without a central
data store, there could be no reference set of categories without hard
coding them into the program.  Instead of hard coding the categories,
we let them be defined dynamically within each link submission. If
a user submits a link that they categorize as "news", then "news"
becomes one of the categories in Linkulator's display menu.  If another
user submits a second link also categorized as "news", then the "news"
category simply as two links under it.

This dynamic category system could potentially yield a very cluttered
Linkulator menu, and this is a risk we thought about.  Ultimately we
decided that this creates an opportunity for conventions to develop
within the user community.  Users are encouraged to look at existing
categories before categorizing their own link post, and try to adhere
to existing categories if possible.

-- Development History

Some of you may be aware that Linkulator is actually version 2.0.
The original Linkulator was written in bash and used a centralized data
store model, a combination that led to challenges for development.
Development on Linkulator 1.0 eventually ground to a near halt,
largely as we struggled with the challenge of a centralized tool
that was still built on a non-compiled langauge.  Late last year,
we threw in the towel on using bash and switched to Python 3.0.
This and the refactoring into a decentralized model quickly led to
the release we're announcing now.

The two people who helped with Linkulator's development are ~asdf
who I first met on tilde.town and ~sloum who runs colorfield.space.
They didn't just help; they made hugely valuable contributions to
the code base and provided very thoughtful input on many different
design choices.  It has been a lot of fun working on Linkulator with
them and I've definitely improved my coding skill and understanding
about the structure of collaborative projects from working with them.

-- Development Future

Linkulator is an ongoing project, already with a number of additional
development goals, some modest and some potentially big changes.
It is an open collaborative project too, and you are more than welcome
to submit feature requests, bug reports, general ideas, or code.
The git repo link is at the bottom of this post, or email me with
any inquiries.

[1] https://tildegit.org/cmccabe/linkulator2