[HN Gopher] Scaling Mastodon is impossible
___________________________________________________________________
Scaling Mastodon is impossible
Author : the_mitsuhiko
Score : 109 points
Date : 2022-11-14 15:03 UTC (7 hours ago)
(HTM) web link (lucumr.pocoo.org)
(TXT) w3m dump (lucumr.pocoo.org)
| [deleted]
| senko wrote:
| > On decentralized systems in particular I encourage you to read
| Moxie's take on web3 which outlines the challenges of this much
| better than I ever could
|
| I am sad that the crypto/defi/web3 crowd hijacked the term
| "decentralized" and now people who should know better equate the
| two.
|
| Web is decentralized. E-mail is decentralized. The internet is
| decentralized.
|
| > Mastodon encourages not just decentralization, but federation.
| [...] I will make the point that this is the root of the issue
| here.
|
| Ok, so we're actually talking about federation being bad, not
| decentralization as such.
|
| > I used to host a pastebin for a few years. It was Open Source
| and with that others also hosted it. I had to shut it down after
| it became (by a small percentage of users) used to host illegal
| content. [...] I really hard a hard few weeks when I first
| discovered what my software ended up being used for.
|
| We take that risk every time we open source something or run an
| internet service. I would not be surprised if some criminals
| somewhere used Flask for nefarious purposes. That doesn't mean
| Flask and his other software is not, on net, a huge benfit for
| society. This issue is orthogonal with (de)centralization, tho.
|
| > Imagine you're a rather small server and suddenly Eli Lilly and
| Company joins your instance.
|
| Let me rephrase that to show what a straw man this is: _Imagine
| you 're a rather small email server and suddenly Eli Lilly and
| Company starts hosting their mail on your instance_
|
| The chance of that happening is precisely zero. WTH would they do
| that (and why would you let'em)? The same argument goes for
| Mastodon.
|
| Turning to the title of the post, _Scaling Mastodon is
| Impossible_ , I agree with the author (to the extent that I know
| about Mastodon, anyways). Maybe Mastodon isn't up to the task,
| maybe ActivityPub as a protocol is inadequate, but those are
| technical challenges to overcome. I don't believe this dooms the
| entire "decentralized federated network" concept.
|
| > Wikipedia for all it's faults shows quite well that a
| centralized thing can exist with the right model behind it. [...]
| A "Not Twitter Foundation" that runs an installation of an Open
| Source implementation of a scalable micro blogging platform
|
| This is an interesting proposal. I worry that the cost structure
| - in terms of hard work, not server capacity - is way different.
| For all its edit wars, the basic principle on Wikipedia that
| someone (or a group) authors an article and thousands or millions
| people read it. Once the article is written, it is (for the most
| part) static. "NotTwitter" is the opposite - you have constant
| stream of new content that needs to be policed (if you don't want
| your town square to descend into madness). Reddit's army of
| moderators shows how difficult that task can be.
|
| [Meta: was the article flagged? It's got a fair number of points
| but is way down in the list of articles]
| compsciphd wrote:
| decentralized systems are always less efficient than a comparable
| centralized system (any insight that can make a decentralized
| system more efficient will do the same in a centralized system).
|
| considering how hard it is to scale centralized such systems, it
| makes simple sense that it be much harder to scale a
| decentralized system.
| goranmoomin wrote:
| Quite interesting that a lot of comments mentioning the technical
| side of scaling, when the bulk of the post was about the more
| social side of things.
|
| While I don't agree that decentralization itself is a dubious
| goal, the argument that moderation, the legal side of things,
| etc... is not scalable (moderation especially in its current form
| where the entire fediverse can report anything) seems quite true?
| We can't rely on generous people hosting instances, taking legal
| risks and spending time for it if the fediverse wants to grow
| bigger than what it currently is.
| vidarh wrote:
| I don't see anything here that is backed up with anything other
| than more dubious opinions.
|
| When it comes to scaling in particular, _we have a counter-
| example_ in e-mail which shows scaling an inbox model +
| reflection (mailing-lists) to a follower-list can scale to a
| vastly larger audience. _Of course_ it can, given that it
| decomposes neatly into a trivially parallelisable set of
| components of a well understood nature.
|
| It was when Twitter was still getting their fail whales, sometime
| around 2007, I first pointed out that parallel (might even have
| been on HN). There are some tricks to _avoiding lag in
| presentation_ for accounts with huge numbers of followers, but
| there decentralisation already serves to turn a flat list of
| followers into a tree of instances - > subsets of followers per
| instance, and _if /when_ any instances gets to a size when
| scaling a single instance becomes problematic, then decomposing
| into a virtual federated set of instances presenting a unified
| namespace solves that just fine (have built a mail setup doing
| just that - mapping <user>@<domain> to <user>@<backend shard> is
| not hard)
|
| The issue of "unpaid Labour" and peoples lack of desire to self
| host also depends on an assumption that Mastodon is inherently
| tied to unpaid volunteers or self hosting, but you can _already_
| pay for Mastodon hosting, and more options are sure to appear if
| it keeps growing. And the largest Mastodon instance is run by a
| non-profit getting paid by sponsorships already.
|
| And this goes straight to the issue of what happens when someone
| popular joins an instances and/or trust: No instance needs to
| accept someone who brings undue load without expecting donations
| or charging or just turning them away, but the converse is that
| trust is layered in Mastodon:
|
| You can trust someone on the basis of a history of behaviour and
| integrity of the node you're on, but people who need more trust
| _can also run their own instance on their trusted domain_. E.g.
| if POTUS wants a trusted Mastodon account, the logical choice
| would be to have webfinger for @potus@whitehouse.gov or similar
| point to a trusted instance run by the government. Nobody else
| _can_ set up addresses on whitehouse.gov because it requires
| controlling the ability to run a webfinger setup responding at
| https://whitehouse.gov/.well-known/webfinger?resource=acct%3...
|
| Conversely, if people need someone trusted to run their instance,
| there are any number of people who'll happily take their money to
| provide guarantees.
| pornel wrote:
| I think it's necessary to separate technical problems from social
| ones. If necessary, the protocol can be changed to scale better
| (Mastodon is trying to be nice to respect a pre-existing
| standard, but if that standard crumbles, it's fair to come up
| with a better one, and IMHO it is solvable).
|
| Mastodon from the start wanted to tackle the social part of the
| problem about the fundamental disagreements about moderation.
| There's a wide spectrum between people who are keen to start
| Internet fights, people willing to tolerate that in the name of
| "free speech" principle, and people who just want to have a nice
| place where they can hang out without randos demanding they
| "debate" their right to exist.
|
| These groups will _never_ agree what level of moderation is
| appropriate. Even within Mastodon communities it 's controversial
| what the rules for federation, bans, and content warnings should
| be.
|
| Twitter, Facebook and others begrudgingly ended up being arbiters
| in political culture wars and must en masse decide what is
| acceptable to say. This ends up being an absurd situation where
| they try to balance the amount of abuse to an advertiser-
| acceptable level.
|
| Mastodon's answer is that you can moderate your community however
| you want, and cut off whole parts of the network you don't like.
| This is ridiculed as "bubbles", but if you don't agree -- make
| your own anything-goes instance!
|
| It doesn't work quite well. There's still a lot to work out, but
| I think it's a better starting point than pleading with a
| billion-dollar enterprise to police the content in the way _you_
| think is right.
| somebodythere wrote:
| If you post on your own anything-goes instance, you will
| probably be defederated from your friends who are posting on
| normie instances. You don't have a choice but to go to a normie
| instance yourself and adjust your content and following to suit
| the instance's rules.
| tpxl wrote:
| If you wish to participate in society, you have to abide by
| the rules of the society. This is a feature, not a bug.
| sramsay wrote:
| I'm having flashbacks to a number of basically interminable
| debates in computing : big-endian vs little-endian, thin clients
| vs thick, windows vs mac, framework vs no-framework, functional
| vs imperative, monolithic kernels vs microkernels, emacs vs vi .
| . .
|
| In none of these cases is the debate resolved by asking: "What
| problem are we trying to solve?"
|
| (edited for clarity)
| blamarvt wrote:
| How did you forget tabs vs spaces!
| bovermyer wrote:
| The first half of this article had me thinking that the author
| was missing the point.
|
| The second half made me think, though. Specifically, about the
| difficulties around back pressure and moderation.
|
| The final thoughts are on the right track. A central foundation
| that acts as a steward for a protocol or platform, and instances
| of that protocol/platform are decentralized and federated.
| Zaskoda wrote:
| This is why Bitcoin is money. Bitcoin creates an incentive to
| participate in running the network. Mastodon does not have such
| an incentive. Mastodon server admins have to either pay out of
| pocket or come up with a scheme to charge users.
|
| Email is similar and now most people get their e-mail from one of
| a few providers who exploit their data for advertising purposes.
| This is exactly the problem we wish to distance ourselves from in
| regard to centralized social services such as Facebook, Twitter,
| Youtube, etc.
|
| If you're not the customer, you become the product.
|
| Mastodon and the Fediverse made decentralized software that
| people want to use - but they didn't come up with a way to make
| it sustainable. Web3 has a model of financial sustainability, but
| they've yet to come up with any other decentralized software that
| lots of people want to use. These two communities focused on
| decentralization should really collaborate more.
| shkkmo wrote:
| I fail to see how bitcoin or another cryptocurrency makes
| funding good moderation easier. "Running the network" means
| very different things in those two contexts.
|
| I would argue that in fact, cryptocurrencies famously face
| similar challenges to the fediverse with the moderation a
| decentralized community. Fraud, abuse and grift are rampant in
| the cryptocurrency community and even the good actors in that
| space seem to struggle to find any way to reduce that. The only
| limited successes there are from the centralized exchanges.
| joecot wrote:
| I know it's hard to imagine under Capitalism, but lots of
| things work without a financial incentive. The early-ish
| internet was powered by web forums. There are thousands of
| community run game servers. Wikipedia editors and contributors
| are unpaid. Volunteers do things because they want to see them
| happen and they like the community.
|
| Bitcoin has brought out the worst in many people, to the point
| where coal power plants were coming back online in order to
| handle the power demand of generating bitcoin, and miners were
| grabbing up as many video cards as they could, boxing out
| gamers from reasonable build budgets for years. Bitcoin is how
| society runs when it only cares about financial incentives.
| There are plenty of parts of society that run without a
| financial incentive at all just fine, and I'm hopeful the
| future of the web is a lot more like Mastodon than it is like
| web3.
| xiphias2 wrote:
| Lightning network already works well for me, I use it on the
| internet regularly, and I bought a coffee with it on the street
| a few days ago.
|
| I would happily pay a bit for Mastodon if I needed, but for me
| Twitter is good enough though for now.
| jmbwell wrote:
| Not sure this article is internally coherent. It starts out
| saying that centralization or decentralization should be an
| implementation detail. At the end, it advocates for a market-
| oriented centralization approach. Throughout, it calls out
| problems that seem to be features.
|
| The analogy to decentralized package hosting seems misaligned. If
| a package repo disappears, it breaks a tree of dependencies. If a
| social media instance disappears, the followers (the
| "dependencies") can adapt without being disrupted. This seems
| like a feature.
|
| On the issue of "not agreeing what it should look like," this
| also seems like a feature. Communities can use the platform to
| self-organize according to their own visions and priorities.
| Surely, someone who has spent time exploring the various
| instances available can quickly identify this. Not agreeing what
| it should look like is what makes it what it is. Not having to
| force everyone to agree what it is, is what it is. Again, it's a
| feature, and something fediverse users have happily "learned to
| live with," and moreover, embraced.
|
| Moderation and illegal content are a legitimate challenge on any
| platform. With the decentralized fediverse, everyone handles it
| according to their own policies. This does mean some admins allow
| it perhaps more than they should, but it also means they face the
| consequences directly. Admins who work to keep it out do face a
| daunting task in many cases, but there are tools for it. And a
| process of sort of natural selection already seems to favor those
| who have the resources and energy and motivation and skill to do
| it well. Seems, again, like a feature.
|
| "Mastodon is old." Not sure how this has anything at all to do
| with centralization or decentralization. We all use things that
| are old. Many old things work great. As far as others not getting
| their changes merged back, well, that's the benefit of forking
| and open source. Anyone who wants changes that aren't part of the
| upstream can fork it and make those changes. If enough people
| like them, people will move to the fork. It happens all the time.
| Feature.
|
| Technical challenges described are demonstrably surmountable.
|
| Market-based approach: well, we're living it, and Mastodon in
| particular is succeeding in the market. There are competitors
| like Pleroma and microblog.pub. If enough people decide Mastodon
| proper has "fucked up too much," then there are others right
| behind it ready to "step in and replace it."
|
| I dunno. There are arguable advantages to centralization, but I'm
| not sure this article touches on them, or at least not
| convincingly. Rather, it seems like the main complaints, if
| they're not in fact addressed by the design and operation of the
| fediverse, are design choices that give the fediverse advantages
| in other aspects. In any case, I don't see how the claim of the
| headline is well-supported by the points raised in the article.
| tiffanyh wrote:
| For those not aware, the author is the creator of Flask.
| masterof0 wrote:
| I like him, even bought his first Flask book. I just don't
| agree with him on this one. He (I hope is he, correct me OP)
| tries to make the short term scaling issues of one Fediverse
| project (Mastodon) the reason of why decentralization is bad or
| not feasible.
| the_mitsuhiko wrote:
| Author here: just to clarify I did not write a book on Flask,
| I did create the framework though.
| masterof0 wrote:
| Oh crap, my bad, I thought Miguel did, thanks for Flask
| anyway.
| Animats wrote:
| There are several issues here: finding, identity, storage, and
| delivery. They're somewhat separable.
|
| Finding is "where do I go to get X". We have URLs, which assume a
| specific server. There are content-based keys, such as DOIs and
| URIs and hashes. But how do you find where the info is stored?
| Google? Something like DNS? Something else? This is the hardest
| problem. What Youtube really sells is "discovery" not streaming
| hosting. There are lots of streaming services, but you won't get
| the views.
|
| Identity has all the usual problems. If people can create lots of
| identities at low cost, there will be spam and worse. No good
| answers there. China has this fixed, but they don't do anonymity.
| You need a government ID to connect to anything.
|
| Storage is the big cost problem. Where does all this stuff go,
| and who pays for it? IPFS was supposed to be the distributed
| answer to this, and Filecoin was supposed to be the way to pay
| for it. That didn't work out. On the other hand, if someone wants
| it out there, then maybe they have to pay to store one copy.
|
| Delivery can be distributed, but do you want to? Bittorrent was
| the prototype. Peertube is another peer to peer way to do it.
| Each video has a home server, and large numbers of people
| watching the same thing won't overload it because anyone watching
| the video also serves it. It works OK but is not as smooth an
| experience as YouTube. Plus it runs down your battery and runs up
| your bandwidth usage. Bandwidth is much cheaper in data center
| bulk than out at the end of a cell connection. Maybe do something
| like that but with ISP level caching servers, all serving each
| other. Sort of like Cloudflare / Akamai.
| javier_e06 wrote:
| The post is a collection of rambling thoughts.
|
| The author should take a look at Linux Torvalds talk with Google
| (2009). https://www.youtube.com/watch?v=4XpnKHJAok8 back and day
| when development was centralized (no cloning).
|
| The concept of "Trusted Circles" is at the core any resilient
| information exchange and it is by definition: Decentralized.
| tbugrara wrote:
| Author lacks imagination.
| jameshart wrote:
| > Imagine you're a rather small server and suddenly Eli Lilly and
| Company joins your instance.
|
| The point of mastodon is not that big companies will join small
| volunteer-run instances. They can run their own mastodon servers,
| just like they run their own mail servers and chat servers (or,
| more accurately, they outsource running their own mail and chat
| servers to Google and slack).
|
| The right place for ActivityPub to land is that businesses and
| institutions will host their own (or pay a SaaS offering to do it
| for them, like they do with email and their CMS today),
| individuals will sign up to shared services (like you sign up
| with gmail for email, or blogger.com to blog).
|
| The problem is going to be holding the barbarians at bay. All the
| same stuff we have to do for email will need to be deployed:
| Community blocklists for misbehaving servers, IP reputation, ML
| spam detection...
|
| And funding it will require cash which the adtech industry will
| spiral in and offer to provide. 'Federate with our servers and
| we'll pay you 1c for every message you allow us to post to your
| users...' Server admins will resist for a while but eventually,
| the walls will crack.
|
| Enjoy it while it lasts, this burst of old school volunteer-run
| internet. It won't survive this eternal September. It never does.
| As the architect of the Matrix said: _Denial is the most
| predictable of all human responses. But, rest assured, this will
| be the sixth time we have destroyed it, and we have become
| exceedingly efficient at it_
| TaylorAlexander wrote:
| Yep and there's already a paid service offering hosted
| instances!
|
| https://masto.host/
|
| There's also an open and active GitHub issue on mastodon
| discussing how to separate server hosting from domain name so
| you can point DNS at an existing instance and use it from that
| domain: https://github.com/mastodon/mastodon/issues/2668
|
| Many thanks to my friend on twitter for pointing me to both of
| these:
| https://twitter.com/__jesse_li/status/1592006641897320448
| MonkeyMalarky wrote:
| I just had nightmares of internal corporate instances.
|
| >Mastodon by Teams (tm), please log in with your active
| directory credentials to see the latest posts from HR!
| class4behavior wrote:
| As a side note, imho, there's little incentive for (large)
| businesses to associate with any particular instance because of
| the evident risk that users would target and harm their brands.
| user3939382 wrote:
| > the architect of the Matrix
|
| Who was, by the way, an homage to Vint Cerf, creator of TCP/IP.
| kerblang wrote:
| Mmmm I think this is popular urban legend
| edent wrote:
| Twitter didn't scale. Not at first.
|
| Those of us on it during the early days were well acquainted with
| the Fail Wail.
|
| But, with a lot of engineering work - and constant upkeep - it
| began to handle the load. Let it be so with Mastodon.
|
| Nearly every point the author makes is subjective. That's fine,
| of course. But it doesn't help demonstrate that users do (or
| don't) want centralization.
|
| Personally, I'd rather be moderated by my local community than by
| a faceless American company trying to appeal to advertisers. And
| if I don't like their stance I can go elsewhere.
| hedgehog wrote:
| The scaling requirements are also unevenly distributed. Servers
| hosting celebrities are going to have very different load vs
| servers hosting 99% of people who are mostly interacting with
| friends/family/interest groups. The overall resource footprint
| with Mastodon will surely be much bigger than a fully
| centralized service like Twitter that can continuously identify
| and engineer out inefficiencies across their whole user base,
| not sure this is a high ranking consideration though.
| vidarh wrote:
| It's also not hard to solve. I've run large scale mail
| distribution systems, and I've run a webmail platform and
| been on the receiving end of large mailing list blasts.
|
| The solution in both cases is just basic divide and conquer.
| Mastodon the software may not be able to scale effortlessly
| to the largest accounts, but that's a market opportunity.
| ActivityPub certainly can accommodate it.
|
| If inbound volume is a problem, shard inbound activities, and
| zipper merge on lookup, likely maintaining a cache of the
| most recent n amount of entries, given this would mostly
| apply to things like the Federated feed for a large instance
| where "nobody" looks far back.
|
| If outbound volume from a single account is a problem, you
| similarly just split the follower list into buckets and hand
| off delivery to a distributed set of workers for delivery;
| this scales just fine with the caveat that you depend on the
| receiving instances being able to handle the deluge. But
| ActivityPub allows for batched deliveries to shared inboxes
| [1] you can post to, to reduce the deliveries to one message
| _per shared inbox_ (which would typically best case be one
| per instance; e.g. the "sharedInbox" property for my
| mastodon account is "https://mastodon.social/inbox). Assuming
| all current instances has a sharedinbox property (which is
| likely given most of them are Mastodon, and Mastodon does),
| currently delivering to all the ~6m Mastodon accounts
| requires delivering to ~1550 shared inboxes; delivering
| directly to 6m wouldn't have been a problem, but it's not
| necessary.
|
| If handling the outbound volume from a single account
| _arriving at your instance for delivery to a large proportion
| of your users_ is a problem, ActivityPub already has a built
| in solution: To find out where to deliver, you need to use
| WebFinger to obtain the profile urls, and get the profile
| urls to find the endpoints to talk to, including the inbox,
| which means foo@mastodon.social and bar@mastodon.social can
| have inboxes on different servers without even needing to do
| any more advanced sharding. But of course you can do the
| latter too and have the same inbox url mapped by your load
| balancer to any of a number of internal shards.
|
| This is all stuff we had to solve _literally decades ago_ for
| e-mail.
|
| [1] https://www.w3.org/TR/activitypub/#shared-inbox-delivery
| pwinnski wrote:
| > My Claim: Decentralization is a Questionable Goal
|
| As oligarchs run Twitter and Facebook, decentralization seems
| less like a goal, and more like a necessary condition.
|
| > Imagine you're a rather small server and suddenly Eli Lilly and
| Company joins your instance.
|
| Why would they ever join a small indie mastodon server? Do they
| send email from elililly@hotmail.com too? If they can run an
| email server for @Lilly.com, they can run a mastodon server for
| @Lilly.com
| whalesalad wrote:
| I realize that the word "scaling" here is used in the
| human/community sense, but for some reason (considering Mastodon
| is written in Rails) I am reminded of this gem:
| https://web.archive.org/web/20080107085941/http://www.zedsha...
| BeefWellington wrote:
| It would be great if the author backed up, really, any of their
| points with some kind of demonstration about what they're talking
| about. This article otherwise is seemingly a bunch of handwaving
| about vague "problems" that exist but without actually clearly
| stating examples.
|
| Also the author is flat out incorrect on this:
|
| > This decentralization however came with a lot of challenges and
| today decentralized package hosting is no longer supported by the
| Python ecosystem.
|
| _pip_ has supported github urls for a long time.
|
| _edit_ Further:
|
| > The second thing that became apparent over time was also that
| decentralized services came with a lot of security risks. Every
| one of those hosts allowed the re-publishing of already existing
| packages. Domains that lapsed could be re-registered by other
| people and new packages could be placed there.
|
| Linux package management systems solved the decentralization
| "problem" years ago. It's why there's so many mirrors available
| when you download packages. Signed packages, and even "trusted-
| source" checksums can provide for integrity in cases where
| decentralization exists.
|
| That some package managers threw this away in favour of
| monolithic repositories is kind of irrelevant to how ActivityPub
| works.
| the_mitsuhiko wrote:
| Author here.
|
| > pip has supported github urls for a long time.
|
| setuptools used to be able to pull packages published to the
| index from external URLs. Support for this was removed many
| years ago. Also you cannot publish packages to PyPI that
| reference dependencies on GitHub.
|
| About the rest of your points on package indexes I believe I
| addressed them in the post already.
| BeefWellington wrote:
| That's specific to _one_ scenario for Python.
|
| It's hardly the entire ecosystem, which is my point.
| Decentralization is there already and well-supported in the
| provided tools. You just can't abuse the official PyPI
| sources, and that's a reasonable approach.
| lmm wrote:
| > It's hardly the entire ecosystem, which is my point.
|
| It is, de facto, the entire ecosystem. Everything's on
| PyPI, and if your package can't interoperate with PyPI then
| it will wither away very quickly.
| the_mitsuhiko wrote:
| The point I was making is that systems have a tendency to
| centralize, even in the absence of a bad actor.
| Decentralization was really well supported in the Python
| ecosystem in former times, it no longer is. The trend is
| pretty self evident I would argue.
| icehawk wrote:
| Some of these arguments start on really odd premises:
|
| > Imagine you're a rather small server and suddenly Eli Lilly and
| Company joins your instance. Today they have around 140K
| followers on Twitter and they are a publicly traded company.
| First of all with an account that large, every one of their posts
| will cause a lot of load on your infrastructure. Secondly though,
| they are a very interesting target to attack.
|
| Why would Eli Lilly join some small instance? Why wouldn't they
| make their own instance? They have their own email and website
| after all. why be lilly@someoneelse.social when they could just
| be their own thing?
| pornel wrote:
| If they set up @social@lilly.com this solves the blue checkmark
| dilemmas too. It verifies their identity without implying
| status/endorsement/$8 flex.
| this_user wrote:
| > Why wouldn't they make their own instance?
|
| Because it's not worth the effort. If anything, there are going
| to a commercial providers that run servers for companies in a
| bundled way where all of their clients use their servers. But
| more likely, they won't bother at all with this, because it's
| not an important part of their business.
| ilyt wrote:
| And then the big public instance decides it's not worth even
| federating with small folks because if you block them they
| will come to your instance because you have "content" - a
| bunch of big names that use it - and also display their
| site's ads to them.
|
| ...aaaand we're back to Twitter in all but name
| fleddr wrote:
| I agree with the author.
|
| Mastodon is quicksand. Instances are not guaranteed to keep
| existing, they depend entirely on a citizen running it, scaling
| it, and paying for all that, often with the help of donations.
| Even a relatively small influx of new users may pose an
| existential financial threat for the instance, or a lock so that
| nobody new can join.
|
| Even when things are running "normally", instance owners may
| simply quit. Further, did you know that almost every instance
| regularly purges all media attached to toots?
|
| Your instance, content within it, account you created within it,
| the media attached to your toots, are all incredibly fragile and
| can disappear at any time. The main Mastodon scaling approach:
| creating lots of small instances, makes this problem worse, not
| better. You're just spreading fragility.
|
| You don't have this insecurity at Twitter, not at this
| fundamental level. Twitter pretty much auto-scales and your
| content is not lost. Sure, people may get banned, but a normy
| user would normally not face these existential issues.
|
| The other thing that scales poorly is moderation. Twitter is
| sometimes perceived as being inconsistent in its rules (or
| biased), but this is a 100 times worse on Mastodon. Every
| instance has their own arbitrary and ever-changing rules. The
| same applies to federation. Instance mods regularly block
| federation with other instances based on arbitrary decisions.
|
| Technically, the UX is inconsistent. One instance may work well
| whilst the other has page loads into the 10 seconds, or federated
| content delayed by hours.
|
| Doesn't scale, fragile, and deeply inconsistent. It has a place
| and I still consider Mastodon an accomplishment. But it should
| not be compared with any of the centralized services.
| SamBorick wrote:
| I think there's an axiomatic difference here.
|
| I don't believe any large service is more stable in the long
| term than a small one. To put it morbidly, all will be dust
| eventually. At least mastodon has account migration baked in.
|
| Additionally, why would I want all my content to last forever?
| If I want something to last forever I'll carve it on a rock,
| not toot it.
|
| Pretty much everything else you listed is a positive.
| fleddr wrote:
| There's quite a lot of wiggle room between short term
| volatility and a concept like "forever". If you take it as
| far as "we'll all die anyway", you might as well not post
| anything, ever, anywhere.
|
| Large central services give reasonable stability as to not
| easily shut down altogether, having your account and all
| content removed. On Mastodon, this is a constant threat.
| tpxl wrote:
| > The other thing that scales poorly is moderation
|
| Moderation does scale poorly, which is why twitter is a
| shitshow and smaller forums (like this one) have a chance of
| being well moderated.
| fleddr wrote:
| I get what you're saying, but you're missing some important
| aspects.
|
| I wasn't commenting on the quality of moderation, instead on
| the scalability. Instance owners are easily overwhelmed and
| this problem gets worse as things grow.
|
| Second, if your point is to create a tightly-moderated
| instance bubble, then Mastodon's federation features will
| largely go unused. If that is the outcome, then there's no
| point to Mastodon. You can recreate such spaces everywhere,
| for free.
| MivLives wrote:
| I ran head into the first one. Back in August my instance owner
| announced they was going to be shutting down. Gave me plenty of
| time to move, and while followers and people you're following
| are easy to move, your content really isn't.
|
| Perhaps that's for the best? Perhaps social media in the
| microblogging style should be considered ephemera.
|
| Moderation and performance you're 100% dead on.
| fleddr wrote:
| I'm putting myself in the shoes of a common Twitter user. To
| exaggerate a little, they are spoiled and have zero tolerance
| for friction.
|
| Look at all the complaints regarding onboarding. They are
| forced to pick a server and many directly stop at that,
| loudly complaining. Two days to wait for an activation email?
| Unacceptable.
|
| As for content preservation, I get what you're saying. At the
| same time, I do believe that people are not used to the idea
| that their old media is purposefully deleted, automatically.
| Or that their toots and instance account can disappear at
| will. Those are new concepts and frictions when coming from a
| central platform.
|
| To illustrate that point, Facebook has multiple giant data
| centers that host nothing but stale content. That pic you
| posted 7 years ago, it's over there. You will absolutely
| never ever look at it again, but in case you do...it's there.
| These massive investments underpin my idea that users expect
| their content to be persisted. If this wasn't important,
| Facebook would not do this.
|
| I think the complexity of content preservation is that social
| media has various uses. You can chit-chat and use it as a
| casual conversation tool, yet others use it as a broadcast
| channel. Some may use it for record-keeping. Creators may
| build up a portfolio of rich media posts. Some people may be
| fine with the chit-chat deleted, but for many other uses it
| would be not acceptable at all.
| hrpnk wrote:
| In the '00s, phpBB forums and IRC were popular, but the
| Internet far less hostile for users. Even bigger challenges
| will come when instance operators are asked to comply with
| local law, requiring them to have privacy policies, t&cs and
| enforce these alongside regulations like GDPR.
| fleddr wrote:
| Fully agree. Hobbyist moderators currently do moderation at-
| will. They may have a day job, go on a holiday, and of course
| they sleep. Without having several moderators across time
| zones, everything you said might become an existential issue,
| especially on larger and open instances.
|
| And to add insult to injury, better anti-harassment features
| are near impossible to implement because of the nature of
| federation. Other instances may run an older version, or
| simply chose to ignore your new rules.
|
| If and when Mastodon grows to the size as becoming
| interesting for many mass scale harassment attacks, the only
| logical way out is to disable the federation part more and
| more. Which means you basically end up with tiny centralized
| bubbles. For which you could have simply created a subreddit
| or Discord, for free.
| bombcar wrote:
| Perhaps everything you ever said being stored forever on the
| Great Servers in the Cloud is not the best way to go about
| things.
|
| Loss and death are part of life, they should be part of online
| life, too. Especially for things that are basically a glorified
| chat room.
| LinuxBender wrote:
| The article goes into some details beyond scaling that I can
| resonate with. I had a few forums and IRC servers in the past
| that grew rather large. I eventually shut them down, not because
| of scalability but because of legal liability and dealing with
| the myriad of personality issues that put my domains at risk.
| Scaling a forum or IRC to hundreds of thousands or even millions
| of people is not hard especially nowadays with cloud scaling and
| the current state of modern kernels and hardware.
|
| What I found too challenging was having to moderate the content
| and finding moderators that could be trusted to remove illegal
| content in a timely manor. Worse, there were trolls that would
| use bots to post highly illegal material and then automatically
| submit their own posts to my registrars, server providers and
| government. The bots somehow even grabbed screenshots right after
| they posted content. I say bots because there was no way a human
| to perform their actions so quickly. This was a losing battle and
| I did not have the legal resources to deal with it, nor the
| development resources to play the cat and mouse arms race 24/7. I
| do have my own conspiracy theories as to who these bot owners
| were but that doesn't matter any more. Nowadays I could probably
| block more of those bots with techniques I have learned but I
| just do not have the desire to get back into that quagmire.
|
| I suspect some of the Mastodon admins will learn this lesson with
| time. They, like me, will probably start in a state of denial and
| dismiss the risk until it _gets real_. And it certainly gets
| real.
|
| The only technical work around I could find was to set forums to
| make all posts moderator-approved, meaning only the poster can
| see their post until a moderator approves it. This does not scale
| and people want their posts to be instantly available. With IRC I
| had to constantly add new file sharing domains to word filters to
| block the links to illegal material and that was also a losing
| battle.
|
| [Edit] BeefWellington brings up a good point. I should add that I
| am referring to public instances of forums and IRC servers that
| anyone may join. Private servers are at much lower risk assuming
| the trusted members are good at setting strong passwords and
| static content is not accessible at all without an account and
| Mastodon servers are not linked to lesser trusted or non-private
| instances.
| BeefWellington wrote:
| > I suspect some of the Mastodon admins will learn this lesson
| with time.
|
| I doubt it. A _lot_ of the new instances are invitation-only,
| and the point of Federation is I can just run my own instance
| and seek out the content I desire. I don 't have to let anyone
| else onto my instance.
| LinuxBender wrote:
| I can see that working. Private instances that only invite
| truly trustworthy people are probably much lower risk, the
| only risk being account take-over and the static files are
| are not accessible by bots then the bar is set much higher.
|
| I should clarify that I was referring to forums and IRC
| servers that anyone could join. The Mastodon model in this
| case would be public instances that are not strictly private
| and are linked to other instances. Private instances would be
| much safer. The risk of linked instances would map to the
| weakest link.
| bombcar wrote:
| That's the solution to many of the decentralization
| problems; invite-only.
|
| But people WANT the _chance_ of winning the "lottery" as
| it were, and going viral.
|
| You're not doing that in your small discord or private
| mastodon.
| LinuxBender wrote:
| I completely understand and agree with their incentives.
| Those with the public instances will play the
| winning/losing lottery, losing being not managing the
| troll automated induced bad content fast enough. I
| encourage anyone taking on this challenge to first and
| foremost get some trustworthy non-toxic non-power-
| tripping moderators around the world for the "follow the
| sun" management of the instances.
| hrpnk wrote:
| Creating scarcity in access will lead to shared accounts,
| account re-sale, hacking, takeovers - all the classical
| account management problems.
| bombcar wrote:
| Some of those will exist no matter what (Twitter accounts
| are unlimited and still sold) but - limited to invite
| only doesn't need to mean "limited as in scarce" -
| there's no reason to share an account if you can just
| invite the person, instead.
| ilyt wrote:
| Right but at this point it's just fancy RSS feed with extra
| steps.
| ilyt wrote:
| Let's start from the fact that "decentralization" in "moderator
| of server you've chosen to dwell decides what you can see and
| what you can not" is terrible idea from the get go.
|
| Whether decentralization for social twitter-like sites might work
| is up to discussion, but Mastodon is just bad try at that.
|
| You're trading consolidated moderation by one rule for thousand
| little fiefdoms, each with different rules and waging ban war on
| eachother. Empower the users to filter and pick what they want to
| watch, not moderators
| rexpop wrote:
| > moderator of server you've chosen to dwell decides what you
| can see and what you can not
|
| That's already the case under the "moderation by one rule"
| paradigm.
| bawolff wrote:
| > Decentralization promotes an utopian view of the world that I
| belief fails to address actual real problems in practice.
|
| Its interesting - most recent examples this is true, but it
| seemed to work well in the past. Bit torrent is a staggering
| success. Email is showing some flaws now a days, but how many
| other application layer protocols do you know that were designed
| 40 years and still widely in use.
|
| Although controversial, i would actually say original bitcoin was
| also a success in the sense it did what it intended to do beyond
| anyone's wildest dreams (whether or not that is a good thing is a
| different conversation)
|
| Decentralized protocols seem to have a long history of success.
| Somewhere that seems to have flipped.
| ilyt wrote:
| > Bit torrent is a staggering success.
|
| There is no way to inject spam and ads to the torrent and
| little incentive to fake seeders/peers to make people download
| the wrong one. That's why. It's not a communication method, it
| uses other communication methods to distribute torrents/magnet
| links
|
| > Email is showing some flaws now a days, but how many other
| application layer protocols do you know that were designed 40
| years and still widely in use.
|
| Frankly we nearly had that in blogosphere. You could federate
| what you want to watch via RSS. You could discuss under the
| post. There was even mechanism to get which post referred yours
| via pingbacks (but of course got removed due to _of course_
| being way to distribute spam. The thing it lacked is some kind
| of place to be endlessly fed whatever is now popular (as it is
| trend for every popular social platform) and general seamless
| usability.
|
| But it put entirety of curation up to the user, don't like the
| blog ? Don't subscribe it, no mod to tell you what to do nor
| someone injecting shit into your stream because mods decided
| this thing need to be promoted now
| thal3s wrote:
| This is mostly an opinion piece that's critiquing issues we've
| only begun to solve.
|
| Also, Elon has now shown us all the absolute danger of
| centralized platforms. Email and web servers are a federated
| system and function just fine, so I'm disinclined to believe so
| hand-waving about how this "wont ever work."
| doener wrote:
| "@doener guess its an basic architectural problem... I would opt
| to offload content to cache servers so that the many messages
| interchanged by the instances only contain ID's / links to the
| content. In an IPv6 world one even might use multicasts to send
| Content to the consumers more efficient...
|
| Off course such concepts of less independent instances are a
| different mindset.
|
| Any fully distributed system of independent instances must end up
| in some "maximization" of bandwidth usage.
|
| @doener also, if you think about such cache servers (editing
| content will be a 'problem'), one should optimise the then
| smaller messages interchanged between instances and between
| instances->users in a way, that these fit into the least count of
| Ethernet-frames. In an optimal world: one!
|
| Maybe in a BitTorrent way, users could also act as "cache
| servers" for content."
|
| https://social.tchncs.de/@schnedan/109344407024988466
| the_third_wave wrote:
| Mastodon might not scale that well due to it being overly heavy
| but Activitypub scales just fine unless by scaling you mean
| something which makes it possible to exert control over the whole
| constellation by some centralised authority. I'm running a host
| of Activitypub services - 3 Peertube instances, 1 Pixelfed
| instance, 1 Pleroma instance - on a single server without any
| problems, mostly for experimenting with the things but the
| Peertube instances are used regularly.
|
| Decentralisation is not, as this piece states, 'a Questionable
| Goal'. It is the essence of a thriving internet, it is what makes
| the difference between a world-wide inTRAnet and an inTERnet. It
| makes it possible to pick and choose your own services or host
| them yourself while still being able to interoperate - the
| 'inter' in internet - with others. It may not be a good fit for
| someone's business plan but to that I can but sing a song while
| playing the world's smallest violin: _Nae kings! Nae quins! Nae
| lairds! Nae masters! We willna ' be fooled again!_
| bullen wrote:
| The real problem is IP/DNS.
|
| Until we decentralize those we're going to have problems:
|
| http://radiomesh.org
| Aeolun wrote:
| This guy just has issues with all the issues that directly stem
| from decentralization. You cannot decentralize, allow everyone to
| host their own server, or to join a server someone else hosts
| without running smack dab into the same problems.
|
| I'd argue that some of the things he sees as problems are
| actually features.
| barbariangrunge wrote:
| > I have a lot of thoughts on this that are too long for a Tweet
| or Toot. Since some of my followers asked though I decided do a
| longform version of this
|
| People should leave Twitter and start blogs so that they can talk
| about more complex things with more nuance
| poulpy123 wrote:
| Maybe we should call that web 4.0
| RamblingCTO wrote:
| Web 2 2.0
| bombcar wrote:
| What you need, really, is centralized "big squares" where
| people can talk about limited topics with strong oversight
| (think: HN) but they can _link_ too off-site spaces where
| smaller groups can have their _own_ rules and topics.
|
| I know some of these types of groups, and I don't really
| publicize them, because the people who would fit in will find
| them, no need to grow to world-wide size.
| pwinnski wrote:
| Some mastodon instances allow for much longer than
| 500-character posts. The qoto.org mentioned in the article
| allows 64K, perfect for blogging.
| plgonzalezrx8 wrote:
| infosec.exchange allows up to 11000.
| Finnucane wrote:
| Perhaps _scaling_ is also a questionable goal. Without the
| imperatives of capitalism--aggregating large markets to exploit--
| how much need is there for these services to be so large scale?
| For individuals, 'network effects' are useful, but not
| infinitely so. There are diminishing returns.
| msla wrote:
| > On the topic of moderation the very same issue is even more
| absurd. Some instances want uncontrolled free speech where some
| instances effectively are pure shit-posting instances which are
| completely de-federated from the most of the fediverse as a
| result. Other instances really like to control their content,
| where some popular ones such as fosstodon ban all languages than
| English as a result to allow moderation. There also is no real
| agreement on if larger or smaller instance are going to make the
| problem of moderation better or worse.
|
| This is an inherent problem that centralized platforms try
| desperately to handwave away. However, sometimes handwaving
| doesn't work, and there's no lowest common denominator to fall
| back on which will reliably keep you out of trouble everywhere;
| for an example that probably won't get me flamed too badly here
| in this decade, is it Derry or Londonderry? Just calling it
| Stroke City won't work if the people on the site are
| _sufficiently enthused_ about their discussion forum being
| Correct. It gets worse the closer you get to contested claims of
| genocide (do you ban the people claiming it was genocide or the
| people insisting it wasn 't?) and discussions about who gets to
| qualify as human. Trying to be apolitical only works if you ban
| all the people from one side or the other.
|
| It gets worse once you globalize, and you run into governments
| that are really, really keen on enforcing Correctness on certain
| issues, especially if you want to expand into two countries with
| mutually-incompatible views of which side of a given issue is
| Correct. Silence is not always an option. At that point,
| decentralization is the only solution.
| Dowwie wrote:
| > An Open Source implementation of Twitter that is significantly
| cheaper to run than a Mastodon host that can scale to larger user
| numbers should be possible. And that being Open Source would
| potentially permit us to see this work out in practice by letting
| different communities exist side by side if we can't agree on
| common rules
|
| Maybe Lemmy [1] with a Twitter UI is worth exploring?
|
| [1] https://github.com/LemmyNet/lemmy
| mdaniel wrote:
| Heh, I'd get a kick out of seeing "twitter with downvotes" but
| one would have to admit that those two sites have a vastly
| different mental model and thus I wouldn't expect just a reskin
| to get it done
| TheDesolate0 wrote:
| encryptluks2 wrote:
| This author seems to not back up any of their claims. Like most
| technology you have to scale the services. If you want to look at
| complex examples look at how GitLab scales.
| dewey wrote:
| > Like most technology you have to scale the services.
|
| The blog post isn't really about how to technically scale a
| service though.
| masterof0 wrote:
| The author claims decentralization is a questionable goal, but
| fail at explaining why. Claims we need something in the middle,
| like what? He tries to link decentralization to cryptocurrencies
| and defis, etc... very sneaky, but lazy argument.
| Decentralization means freedom, self determination and power,
| power to host your content on your own terms, or in a place that
| is friendly to you, and exchange ideas with people in other
| communities, where they also own their data. You can have people
| on @socialist.social talking with @billionaries.social , but one
| party can't make the other disappear just because. His other
| argument is against Mastodon technical design, I agree , I
| wouldn't pick the stack the Mastodon team chose, that said, there
| are many activitypub enabled projects, GoToSocial, Misskey,
| etc... which are more resource efficient, so the failure of one
| project , is just a learning opportunity for another newer
| projects. Regarding "unpaid labor and opsec", people can rent a
| server from a provider that guarantees updates and uptime, or
| host their own, in the same way is done across the industry. I'm
| not claiming this is what the author is saying, but most of the
| criticism I see of the Fediverse comes in the flavor of "Look all
| those people I don't like and can't get banned, how dare them to
| share their ideas, let me start complaining to make them
| disappear".
| varelse wrote:
| nyx_land wrote:
| > All of these things have one thing in common: distrust. Some
| movements come from the distrust of governments or taxation,
| others come from the distrust of central services.
|
| This isn't framing the problem the fediverse is solving
| accurately. What fedi solves isn't not trusting anyone, because
| it's not a fully P2P network. Rather, it's more about having
| accountability for the services you use. With a massive
| centralized social media platform like Twitter where it needs to
| effectively scale up to be able to accommodate the entire world,
| it's impossible for a service like this to exist without either
| being run by a government or a corporation. In the US, the two
| are more or less the same anyways since we've outsourced most of
| our infrastructure to the private sector and have been doing that
| for decades. And in the case of a private entity controlling one
| of these services that need to scale massively, it's effectively
| impossible for them to not be incentivized to do all the things
| that make people not trust them -- mining users' data, etc.
|
| On fedi, you know who your server admins are; they're human
| beings that you can message if you have a dispute with another
| user, or there's an issue with the service itself, or whatever.
| If an instance admin does something shitty, they can be held
| accountable for it and can have their reputation ruined on fedi,
| which isn't a perfect system since it leads to things like people
| being defederated unjustly, but I would say it's far more
| scalable socially to have a patchwork of different small to
| medium sized servers run by individuals or teams of people rather
| than faceless corporations that largely automate all of their
| user-facing interactions (algorithmic bans and support) where
| it's basically impossible to hold them accountable for anything
| unless you have millions of dollars to burn in court.
|
| It is true that centralization/decentralization are
| implementation details for solving a problem, but this is a case
| of technological solutions not being sufficient for cultural
| problems and vice versa. Fedi, for all the problems I have with
| it, is however closer to striking a balance between solving
| problems on both of these fronts because it returns the internet
| to being a community-driven network that has its own culture, and
| has more ability to handle bad actors or failures in the network
| with federation as opposed to earlier eras in the internet where
| for example everyone was using their own separate forums and the
| centralization of the internet onto social media platforms was
| really an inevitable consequence of that structure of the
| internet.
| markstos wrote:
| Mastodon is messy. The world is messy. We have cities with
| different rules, different mayors, different odds of existing in
| 50 years. It's nice to have all the cities follow all the same
| rules and customs if you agree with them, and nice to have
| another city to move to if you don't.
|
| Email as a decentralized medium has survived for decades. You use
| a big provider like Gmail, choose a host in another region, or an
| organization like Proton Mail that does thing somewhat different.
|
| It's OK that Mastodon is messy and at times chaotic. It's
| organic.
| travisgriggs wrote:
| Isn't it sort of an oxymoron when we try to distribute or
| decentralize a uniform or consistent platform?
|
| Don't get me wrong, I'm a huge fan of decentralization. I do
| systems engineering. Decentralizations means redundancy.
| Redundancy is good.
|
| But if all you're doing is federating a mono-platform/algorithm,
| then the single point of failures move to the platform itself.
| Decentralizing a uniform platform is like replicating the same
| human being lots to improve the redundancy of the human race. Ask
| the gene pool how well that works.
|
| Variety is good. It's what brings beauty to the human experience.
| ivarv wrote:
| fwiw - there are at least two production ready servers that
| coexist as twitter-like social hubs -
| https://git.pleroma.social/pleroma/pleroma and
| https://github.com/mastodon/mastodon. Additionally, other
| platforms that support the w3c's ActivityPub protocol can also
| interact with Mastodon.
| lokedhs wrote:
| The Fediverse feeds are not uniform. You may have a "federated
| feed" in your Mastodon instance, but the feed you see on your
| instance is vastly different from another instance.
|
| The federated feed is nothing more than a combined list of all
| public feeds from all the users on the instance. There is no
| way to get a central firehose, since there is a large number (I
| was going to say majority, but that's probably unlikely) of
| posts that never even leave the local instance.
| Zak wrote:
| There's a lot of software that speaks ActivityPub. Mastodon is
| the most popular, but Pleroma, Pixelfed, Hubzilla, Friendica,
| and several other less popular/mature options exist. Some of
| these, such as Friendica, are older than either ActivityPub or
| Mastodon, but integrated support for that protocol later.
| topspin wrote:
| I keep having this thought that Twitter could be solved by
| Cloudflare or similar cloud systems that can expose HTTP APIs and
| cheaply serve content through global caches.
|
| Start with the principle that you own (and pay for, if necessary)
| the cloud resources for your piece of the social network. A
| standardized API that does the things you expect of a microblog;
| post stuff, collect replies, followers, etc. runs in your
| account. You control it: access, moderation (possibly delegated
| as you prefer), advertising, throttling, whatever.
|
| Assuming everyone is using a standardized protocol (DIDs for
| identity for instance) users could interact seamlessly. How hard
| could it be to clone Twitter on Cloudflare? Would it take more
| than a few thousand lines of Javascript/WASM to replicate the
| basic microblogging functions? A good onboarding system to
| automate the setup to be at least as easy(...?) as typical social
| networks would be necessary. Search would solve itself if the
| network was sufficiently popular.
| schwartzworld wrote:
| > My Claim: Decentralization is a Questionable Goal
|
| Hard disagree with this. Email is the example I give people of
| how federation could work. You can use any email provider and
| interact with any other email provider effortlessly. This is
| undeniably an improvement over internal direct messages within a
| centralized service (i.e. Facebook or Twitter DMs). If you leave
| facebook, you leave your contact list and message history. On
| email, if using a custom domain, you can switch email providers
| or even host your own server without anybody on your contact list
| even having to know this occurred.
|
| Lots of people have been banned from platforms, rightly or
| wrongly. You can't get banned from email. You can't get banned
| from having your own website. You can't get banned from the
| telephone system. If you don't own your own data, you are at the
| mercy of those who do.
|
| > On the topic of moderation the very same issue is even more
| absurd. Some instances want uncontrolled free speech where some
| instances effectively are pure shit-posting instances which are
| completely de-federated from the most of the fediverse as a
| result. Other instances really like to control their content,
| where some popular ones such as fosstodon ban all languages than
| English as a result to allow moderation.
|
| These are talked about as problems, rather than being the primary
| selling point.
| TylerE wrote:
| Funny, email is the example I give of why it doesn't.
| humanistbot wrote:
| > You can't get banned from email.
|
| If you're on Google or spamhaus's blacklist, you're effectively
| banned.
| TheCraiggers wrote:
| You != your email address.
|
| Yes, a specific server or address can be "banned". But there
| is nothing stopping _you_ from just opening another gmail /
| yahoo / whatevermail account.
| bvrmn wrote:
| And how to change a dozens accounts on others services to a
| new email without old one?
| tootie wrote:
| "Email is the example I give people of how federation could
| work"
|
| That presupposes that federation is actually a goal and not a
| means to an end. It's amazing how great developers are at not
| understanding what creates value. Twitter's value is _reach_.
| Get 400M users in one place, hook them with interesting
| content, server them ads, nudge them to sign up, track their
| engagement to make the platform sticky and attract more
| contributors. Any censorship that has been done has been for
| one and only one reason and it's protect the value of the
| platform. Primarily to advertisers, but also to users because
| users make the platform valuable. Decentralization deliberately
| cripples the main value of the platform (reach) in exchange for
| what exactly? Lack of moderation?
|
| If you want decentralized, uncensorable communities, we solved
| that 30 years ago with usenet. Or, like you said, you can't be
| banned from email or running your own website. That's always
| been the case. People still write blogs. Yet Twitter sold for
| $44B and Tumblr was sold for $3M. Why? Because Twitter has
| reach.
| idlewords wrote:
| Email is an example of how federation fails, not how it works.
| It is now centralized in the hands of a few providers with the
| power to blackhole independent actors.
| mike_hearn wrote:
| That centralization mostly occurred due to the economics of
| below-cost pricing though. It's not necessarily due to
| federation itself failing as a mechanism for email.
|
| In other words, Gmail is the most popular email provider
| because it's in Google's interest to have people be logged in
| all the time so they can personalize search results, so that
| incentivizes them to make a very good email system and then
| give it away for free. Giving something good away for free
| because you gain indirect benefits via some other business
| will rapidly centralize more or less anything, which is why
| there are at least theoretically rules against market dumping
| and tying (which aren't really enforced in the software
| world, but that's another matter).
|
| We can imagine a parallel universe in which search is far
| more competitive, with lower margins, and thus Google
| couldn't financially justify subsidizing consumer Gmail for
| so many years. In such a world it's likely that there'd be
| more players, perhaps they wouldn't be as good but there'd
| likely be more of them, there'd probably be companies that
| specialized in selling spam filtering tech to them and it'd
| look more federated than what we have today.
| KerrAvon wrote:
| It's still viable to use a minority email provider, even if
| that now has to be a medium-size ISP (at minimum) instead of
| your own server. There's still a difference between no choice
| and many choices.
| panarky wrote:
| _> blackhole independent actors_
|
| Freedom doesn't mean the requirement to spend the community's
| resources propagating material that the community agrees is
| harmful or objectionable. That would be the opposite of
| freedom.
|
| The ability to refuse actors that the community doesn't want
| to spend resources on is _essential_ to freedom.
|
| Every actor has the freedom to choose whether to conform to
| community norms, or to form their own communities with
| different norms.
| bioemerl wrote:
| Freedom for titanic companies like Google to control what
| can or can't be said through enforcement on their
| monopolistic platforms is also not freedom for the
| individual.
|
| Your idea here works when the system is fragmented enough
| that these choices are _personal_ choices, but the
| existence of so many large platforms makes their choices
| systemic, not personal.
| dotnet00 wrote:
| This argument comes with the rather absurd assumption that
| the only material that's being blocked is that which the
| community agrees is harmful. When more often than not it's
| simply the material that is from individually small enough
| third-parties where the content doesn't really matter.
|
| On top of that, there isn't community agreement as much as
| there is community ignorance. For every case we hear about
| of someone being banned or blocked by an automated system
| despite doing nothing wrong, there are hundreds more
| similar cases which simply don't gain enough traction for
| the 'community' to hold the service provider responsible.
| rexpop wrote:
| And so we should just leap to consolidation from the get-go?
| pwinnski wrote:
| In the article itself, he points out that banking is
| decentralized and works wonderfully.
|
| He doesn't even mention email or phone service, both of which
| are decentralized and work well enough.
| jojobas wrote:
| Email is still holding up, but realistically if Google,
| Microsoft and perhaps 2-3 other email hosts decided to collude
| and disallow any other email origins (let's say they would
| allow outgoing to them still) the majority of people's reaction
| to those complaining would be "just get yourself a gmail
| account".
| pizza234 wrote:
| > Hard disagree with this. Email is the example I give people
| of how federation could work.
|
| That depends on how one looks at it. If spam and illegal
| content are considered, it may not be "working" in the sense a
| public service needs to be.
|
| Email works because it's private, so the requirements are much
| looser; a large part of the article is how to solve the bad
| actors problem, which doesn't apply to email (well, it applies,
| but spam is more or less accepted as fact of life, while fake
| news etc. isn't - and it may never be, independenly of being
| right or wrong).
| yjftsjthsd-h wrote:
| > If spam and illegal content are considered, it may not be
| "working" in the sense a public service needs to be.
|
| By that standard, centralized social networks have utterly
| failed us as well.
| pizza234 wrote:
| > By that standard, centralized social networks have
| utterly failed us as well.
|
| They're not on the same level; moderation of social
| networks like Twitter/Facebook can be considered
| insufficient, but it exists. Contrast to the very low
| barriers (if any) to send spam emails, where the filter is,
| ironically, in the hands of centralized services like
| Gmail.
| yjftsjthsd-h wrote:
| No, the filter is in the hands of everyone who runs a
| mail server. Gmail et al. run the most centralized spam
| filters, but anyone running their own MX will almost
| certainly run their own spam filter as well. Likewise,
| Mastodon moderation falls to the servers.
| poulpy123 wrote:
| But mail and Facebook/twitter/mastodon are different: one works
| on the principle of private message, the other work on the
| principle of agora. And emails grew organically with the web,
| not the others, it change a lot of things
| jtode wrote:
| >Lots of people have been banned from platforms, rightly or
| wrongly. You can't get banned from email. You can't get banned
| from having your own website. You can't get banned from the
| telephone system.
|
| This right here is why I can't take anyone seriously who takes
| the free speech "issue" surrounding corporate platforms. There
| is no free speech on a platform, ever.
| giaour wrote:
| > You can't get banned from email. You can't get banned from
| having your own website. You can't get banned from the
| telephone system.
|
| The platform may not be able to automatically enforce a ban,
| but you can still be banned via non-technical means. Not using
| email or the phone can be a condition of bail or parole, for
| example, like how Kevin Mitnick wasn't allowed (by court order)
| to use the telephone system in the early aughts.
| _448 wrote:
| > On email, if using a custom domain...
|
| The problem starts here. Not many people who are not tech savvy
| even know what a custom domain is. Let alone having an email
| with a custom domain.
|
| > You can't get banned from email
|
| Ask people who get locked out of their gmail account.
|
| What we techies miss is that there are more people who don't
| understand technology and are not willing to spend time
| learning about it as they have other more important things to
| do. Tech is just one of the tools they are using to get their
| tasks done. We cannot expect people to spend time understanding
| everything about tech. It is not surprising that it took a
| commercial company with aggressive behaviour when it comes to
| controlling users data to put BSD on regular non-tech consumer
| desktops. And why Microsoft Windows succeeded in being on
| regular consumers desktop OS whereas commercial Linux-based OS
| struggled; and the company that successful put Linux on mobile
| devices also has quite a record when it comes to users data.
|
| We techies can talk a lot about decentralisation and owning
| data. But the regular non-tech users really don't care. They
| just want simple things done quickly using tech and get on with
| their lives.
|
| Recently I saw videos from multiple news channels on how to use
| Mastodon. They were at pains to explain to users how there are
| multiple servers and one has to choose a server, which is run
| by individuals or organisations. And if the server goes down
| then the user has to move to another server. Regular non-tech
| users really don't want to get into this complexity. Has anyone
| ever seen TV channels explain with great effort how to use
| Twitter?
| gfaster wrote:
| Like this? https://www.cnet.com/culture/how-to-get-started-
| using-twitte...
| agentdrtran wrote:
| > You can't get banned from email
|
| If your domain gets added to blocklists from
| gmail/outlook/spamhaus you basically are.
| dmje wrote:
| Someone posted delta chat[0] on here a while back. Basically
| chat that is email. Or maybe email that is chat. It struck me
| as a clever idea at the time and got me thinking whether you
| could make a social network entirely out of email. Sounds
| bonkers and probably is but you know, gotta think, right?
|
| [0] https://delta.chat/en/
| pram wrote:
| Seems like NNTP would be the better choice.
| crazygringo wrote:
| > _Email is the example I give people of how federation could
| work... You can 't get banned from email._
|
| Except e-mail has been a total failure when it comes to
| spam/abuse, and in practice it isn't this federated,
| decentralized paradise at all -- virtually everyone I know uses
| Gmail for their personal account. And if you want to run your
| own server, good luck getting Gmail to accept your e-mails at
| all.
|
| And if you can't get Gmail to trust your server, that's pretty
| much a ban for all practical purposes.
| jmbwell wrote:
| Plenty of organizations and individuals run their own servers
| and have no problem exchanging email with Google, Microsoft,
| and other large providers.
|
| The point is that they can and do because the system is not,
| in fact, centralized.
|
| Also, judged only on the volume of spam and abuse, many
| systems besides email -- decentralized and centralized alike
| -- could be called a "total failure." And yet somehow these
| systems remain stable and functional and useful.
| joecot wrote:
| The only problem with doing this could be mail
| deliverability. But if you setup SPF/DKIM/DMARC, and use a
| mail service, it works fine if you're not spamming.
| Mailgun, Amazon SES, Mxroute, they'll all deliver your mail
| and for cheap. There's no single point of failure needed to
| get your mail delivered.
| secabeen wrote:
| For some definitions of fine, sure. I run my own mail,
| and relay it through an AWS EC2 instance that I've used
| on the same IP for over 5 years. It took an exceptional
| effort over many months to get off the Hotmail blacklist
| a while back. I still wonder when emailing someone I've
| never emailed before if that message will go do a Junk
| folder, or be delivered.
|
| The benefits outweigh the costs, but it is not easy, and
| you can start having problems that are very hard to
| solve.
| xiaomai wrote:
| You don't need to use a delivery service. It's true you
| need to set up SPF/DKIM/DMARC for many peers to accept
| mail from you, but then you can deliver the mail yourself
| directly.
| joecot wrote:
| Yes, this is true, but I still get better delivery
| through delivery services. For example a lot of email
| providers send any email coming from a cloud hosting IP
| directly to spam.
| [deleted]
| js2 wrote:
| No one called it a paradise. That's a strawman argument.
|
| I've been using my own domain for my email for over two
| decades. In that time it's been self-hosted, hosted with
| Google, and hosted with Fastmail. I control it, so I can take
| it wherever I want. I've been able to take my full message
| history with me too.
|
| It has not been a "total failure" with spam/abuse either,
| though I'm glad to have Fastmail handle that for me these
| days.
|
| In any case, I'll take the tradeoffs over a centralized
| system.
| FpUser wrote:
| >"I've been using my own domain for my email for over two
| decades."
|
| Same here since the end of the 90s. Completely agree with
| the rest of your post.
| vidarh wrote:
| Same position here, and having the ability to move my
| domain is by far the most important to me. And also why my
| medium to long term plan is to move my Mastodon setup to my
| own domains.
|
| > though I'm glad to have Fastmail handle that for me these
| days.
|
| I get less spam in my Fastmail account than on my remaining
| Gmail accounts, but far more importantly for me: I get far
| fewer false positives with Fastmail. The amount of real
| mail that used to end up in my Gmail spam folder made the
| spam filtering pretty much pointless because I ended up
| going through the spam folder daily anyway.
| blowski wrote:
| I receive as much social media spam as I do email.
| UncleEntity wrote:
| > virtually everyone I know uses Gmail for their personal
| account.
|
| Which, ironically enough, is the answer to everyone's
| problems with mastodon.
|
| Does everyone you know use gmail for their business accounts?
|
| Does absolutely, 100% of the people you know use gmail for
| personal accounts?
|
| I'm going to go out on a limb and say no to both the above.
|
| There's no reason mastodon has to be everyone runs their own
| server just like everyone doesn't have to run their email
| server today. People use the company server for work stuff
| and fluffycat233575@yahoo.com for personal stuff.
|
| I don't know if it's just the HN bubble but all people post
| is why it can't (or shouldn't, because, get off my lawn)
| work. It's like people have a vested interest in making sure
| non-adtech doesn't succeed, almost like their very
| livelihoods depended on it.
|
| --edit--
|
| And, back in the day, spam filtering was something you had to
| do on your own or just dealt with all the spam. The big email
| providers became big in part because they solved that problem
| for most everyone. In '97 nobody (OK, nobody but the US
| postal service) was saying there needed to be this gigantic
| corporation monitoring everyone's emails for wrongthink or
| only a central entity could solve the spam problem. People
| just bucked down and solved the problem enough to get a
| sizable portion of the population using email...I'm genuinely
| curious if there's ever been a technology with higher uptake
| than email now.
| firecall wrote:
| Sadly, this is very true!
|
| It's a total nightmare out there!
|
| I have clients on big Hosts like SiteGround, and sending
| email is a problem!
|
| I'm 100% in support of the idea that services should be
| protocols though! :-)
| 98codes wrote:
| And when I wanted to rid myself of all things Google, nobody
| needed to know, I didn't need to rebuild my contacts -- I
| moved my custom domain to a different provider, done and
| done.
|
| Beyond that, Twitter is rife with spam, scams, and tons of
| other abuse. So is Facebook. So is reddit. The fact that
| email doesn't work that way doesn't matter.
| athenot wrote:
| My _email addresses_ do get more spam than, say my Twitter
| DMs. But my email provider (not Gmail) is good at identifying
| most and my own Mail client can pick up the rest.
|
| So the net number of spam emails in my _email inbox_ is less
| than 1 per week. On Twitter, for me it 's at least 1 per day
| and there's nothing I can do to tweak the filtering.
| Semaphor wrote:
| > Hard disagree with this.
|
| From me as well.
|
| > You can use any email provider and interact with any other
| email provider effortlessly. This is undeniably an improvement
| over internal direct messages within a centralized service
|
| But the general public went for siloed systems instead.
|
| So I'd say you or me want decentralization, but I don't think
| that holds true in general.
| falcolas wrote:
| > But the general public went for siloed systems instead.
|
| And yet some former Twitter users have identified why this is
| a problem, and are now moving back towards a decentralized
| platform. It's why scaling Mastadon is even a topic right
| now.
| Semaphor wrote:
| A comparatively tiny group of outspoken people switched.
| And it's not even clear if they'll stay.
| dradtke wrote:
| True, but it's way easier to add centralization to a system
| that is decentralized by nature than it is to somehow
| decentralize something that is centralized by nature.
| throwawayacc4 wrote:
| >Lots of people have been banned from platforms, rightly or
| wrongly. You can't get banned from email. You can't get banned
| from having your own website. You can't get banned from the
| telephone system. If you don't own your own data, you are at
| the mercy of those who do.
|
| If there's anything to be learned from the KiwiFarms saga, it's
| that these statements are no longer true.
| voxic11 wrote:
| KiwiFarms was back within days, if anything the KiwiFarms
| saga showed that it is still true.
| tommica wrote:
| And it was down quick after that, but right now it is
| accessible in some countries.
|
| This saga has really muddled up responsibilities of
| different parts of the chain that makes the internet, where
| companies that should not be filtering traffic are doing
| exactly that.
| the_mitsuhiko wrote:
| On the topic of censorship / cancellation my limited
| experience so far has shown me that defederation happens and
| plenty of servers are entirely unreachable from the rest of
| the ecosystem. I'm not sure if this is an argument in favor
| or against anything, but at least it shows me that there are
| different standards and ideas at play and they really don't
| fit well together into a coherent view of what Mastodon is.
| It really depends on "which mastodon". poa.st and
| hachyderm.io are both Mastodon instances but they generally
| don't cross as an example.
| _a9 wrote:
| KiwiFarms is the perfect example on how that it is still
| true. The Twitter crew forgot about them after a week and it
| went back to normal.
| [deleted]
| Kalium wrote:
| Email is my go-to example of all the problems of federation. In
| theory any person can stand up their own server and interact
| with everyone else. In practice, there's been so much abuse
| over the decades that it takes a staggering investment in
| time/energy/money/expertise to do so. Enough that's it's
| completely beyond the reach of the vast majority of people.
|
| This problem is so bad that's it's driven a quiet _de facto_
| re-centralization of email.
| jaredcwhite wrote:
| Mastodon itself as a software product/project _may_ be
| "impossible" to "scale" (scale to what?), but decentralization of
| social networking is not only not "impossible", it's actually
| inevitable. We'll look back at the 2010s and wonder why anyone
| ever thought the discourse on the web could/should become
| dominated by a tiny number of corporate media platforms.
| rainonmoon wrote:
| Does anyone _really_ wonder why the current state of affairs
| has risen? Because that seems pretty naive, given how rich and
| influential it 's made its progenitors.
| CharlesW wrote:
| > _...decentralization of social networking is not only not
| "impossible", it's actually inevitable._
|
| When has any medium gone from centralized - decentralized?
| markstos wrote:
| AT&T was for a time the sole provider of telephone service
| throughout most of the United States. In the 80s it was
| broken up into competing companies due to anti-trust action.
|
| https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System
|
| The companies we now know as AT&T, Verizon and CenturyLink
| all came from that.
| hosh wrote:
| > All of these things have one thing in common: distrust. Some
| movements come from the distrust of governments or taxation,
| others come from the distrust of central services.
|
| How about resiliency? Participation from community members? It's
| not always distrust that drives decentralization.
|
| I think James C. Scott's book, _Seeing Like a State_ is worth
| mentioning, since it discusses the failure of centralization in
| more ways than simply distrust. Here 's a long-form essay
| summarizing the highlights:
| https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call...
|
| This isn't really a new idea. In building architecture,
| Christopher Alexander spent a lifetime writing about this. You
| can see some of the highlights in his keynote speech to OOPSLA
| '96: http://www.patternlanguage.com/archive/ieee.html ... His
| ideas enable end users (residents) to be able to change their
| built environment while still maintaining a cohesive
| architectural design. Sadly, his groundbreaking ideas on pattern
| languages became watered down into HOA design regulations;
| software engineers did not heed what he talked about in that
| keynote; and Human Computing Interaction design dropped end-user-
| customizable software (like Hypercard) in favor of designs that
| favored aggregators, because it is more profitable (not
| necessarily more resilient, or better for society)
| nullcipher wrote:
| I can't quite understand what the author's problem is. There's a
| bunch of conclusions without actually explaining the problem.
| aussiesnack wrote:
| > Decentralization promotes an utopian view of the world
|
| Dystopian, actually. It promotes (and depends on) a nihilistic
| view that trust cannot be built, that it can only decline over
| time. This may be true or false. But it's not 'utopian'.
| kup0 wrote:
| I enjoy that most instances are small and will remain that way.
| There are a few that may grow to the point where scaling could
| become difficult, but there are ways to combat that.
|
| What the author sees as problems with mastodon (decentralization,
| moderation differences between instances, etc) to me are the
| strengths of mastodon. It's the whole dang point.
|
| Most mastodon instances aren't trying to be another twitter. The
| ones that are and do want to scale will inevitably run into
| scaling issues that any large platform full of user-generated
| data will run into. I'm just glad that there are different
| instances so when one big instance runs into scaling problems,
| other instances can just keep rolling along unaffected by it
|
| I think talking about mastodon as a singular entity is fraught
| with problems because it's simply not that
| mike_hearn wrote:
| It's worth noting that centralized package hosting concentrates
| the risk into "too big to fail" operations. This is great for as
| long as they genuinely are too big to fail and you can assume
| that someone will always step up to save the day. But the Java
| ecosystem went through a case where that didn't happen:
| JCenter/Bintray was a popular Maven hosting site for many years
| until one day the operator simply announced they didn't want to
| run it anymore and shut it down. It was a clean, phased shutdown
| but ultimately enormous numbers of builds and projects did have
| to migrate away. Now everything is even more centralized around
| Maven Central, which really is (hopefully) too big to fail.
|
| The financial system has a lot of experience with dependency on
| centralized organizations that are too big to fail. It's trading
| one set of problems for another. In particular the risk is that
| the organization starts to "fail" but not badly enough to cause a
| mass collective shift away. Things just degrade and become
| terrible but there's never a moment that overcomes the enormous
| activation energy needed to migrate away. With a federated or
| decentralized system it's easier to bleed off from an institution
| or service that's started failing at its core mission.
| gigatexal wrote:
| Well this is alarmist. I haven't read the ActivityPub spec or
| code but the main failing in the current implementation of
| Mastodon is the codebase is not conducive to horizontal scaling.
|
| There will probably be work in that domain sooner or later. Or
| it'll never really take off. Either way I like how it's small and
| niche. Let the normals stay on Twitter and they can keep the
| advertisers, too.
|
| Moderation is done by users of a node or a server admin or
| admins. And from what I can see admins block on a node level and
| users can block folks at the user or a whole subdomain/node.
___________________________________________________________________
(page generated 2022-11-14 23:00 UTC)