[HN Gopher] Show HN: Moderator Mayhem, a game about the difficul...
___________________________________________________________________
Show HN: Moderator Mayhem, a game about the difficulties of content
moderation
Author : randylubin
Score : 154 points
Date : 2023-05-11 16:43 UTC (6 hours ago)
(HTM) web link (moderatormayhem.engine.is)
(TXT) w3m dump (moderatormayhem.engine.is)
| blast wrote:
| Is https://www.engine.is/ an industry lobbying group? That's what
| it looks like.
| schoen wrote:
| Yes, it is.
| ianferrel wrote:
| In the tutorial, the very first card I was presented with was
| reported for being "illegal", and the description was (from
| memory. I might not have it quite right)
|
| "A post about the movie Cocaine Bear recommends doing cocaine
| while watching the movie and contains a photograph of a bag of
| white powder with the hashtag #cocaine"
|
| That... seems like an obvious joke to me. And even if it's not,
| while "doing cocaine" is illegal, photographs of white powders
| are not, nor is suggesting that people do cocaine, nor is
| portraying them apparently doing so. We can know this is true
| because the producers and actors who worked on the film Cocaine
| Bear were not arrested.
|
| Content moderation _is_ hard, but I 'm not very impressed that
| the very first card shown got it (in my opinion) totally wrong.
| idiotsecant wrote:
| It's ironic that the first post on this thread is about missing
| the point of the first card that user saw.
| ianferrel wrote:
| What was the point of it, and how did I miss it?
| matthewfcarlson wrote:
| This is showing how hard content moderation really is. It's
| judgement calls and very often no one is happy. Seeing more
| information takes an angering amount of time, which makes
| me think this is a fun game that makes an excellent point.
|
| Edit: In some ways, a card that's incredibly contextual as
| the first one is a brilliant move.
| ianferrel wrote:
| I'm not convinced that putting a real head-scratcher as
| the very first choice in a tutorial mode is a brilliant
| move at all. Tutorials are supposed to be easy and hand-
| holding.
|
| But also: this isn't actually a hard judgment call. It's
| just bad instructions. There's an objective difference
| between "content that breaks laws" and "content that
| promotes the breaking of laws" and the rules could easily
| be changed to indicate the one they want.
|
| If you were playing a platformer tutorial and you came to
| a gap and the tutorial instructions said "Press A to jump
| over the gap" and then when you tried you fell in and
| died and the tutorial then said "That was too big a gap
| to jump over. You should have pressed [Down] to climb
| over it instead", would you think "Ah, what a brilliant
| meditation on trusting trust and how the right choice is
| not obvious" or would you think "this is a shitty
| tutorial for a dumb game"? I'm not saying there's _for
| sure_ a right answer here. I could imagine a well-made
| game where the tutorial straight-up lies to you and gives
| you the wrong directions. But most games like that are
| just badly written and poorly thought out.
| bisby wrote:
| "Content that breaks laws" "Content that promotes the
| breaking of laws" "Content that is clearly
| satire/parody/a joke"
|
| A lot of time context matters. In a forum thread called
| "What would be the worst title for a self help book?" a
| post that just says "Give up, kill yourself" is not
| actually promoting self harm. It's saying that its the
| worst advice.
|
| assuming the entire point of the game is "moderation is
| harder than you think, stop assuming all mods are power
| tripping, they have tough choices to make!" -- the
| tutorial isn't supposed to be teaching you how to answer
| the questions properly, because answering questions
| properly isn't the point of the game. It's teaching you
| that about how hard moderation is.
|
| Also, I've played plenty of games where the tutorial
| involves dying and then the follow up is learning how not
| to die (generally in games where dying is common and they
| want to indicate that "dying isn't the same as game over,
| this isn't Super Mario for NES")
| ianferrel wrote:
| I think that content moderation is in fact quite
| difficult, even given clear rules to follow, because so
| much content requires lots of context to understand.
|
| But that makes me even more annoyed at this game which
| rather than presenting me with legitimately difficult
| judgment calls, just gave me clear rules that were _not
| the rules that the actual game used_ when determining
| whether I did the right thing.
|
| It's possible that this is a cleverly designed thing to
| make you realize that the _real_ rules are unwritten and
| the whole thing is a Kafkaesque contraption with no
| correct answers. Or it 's just a quickly-made game where
| no one proofread the actual instructions they were giving
| players.
| BiteCode_dev wrote:
| It's not, because the site tells you there is a perfect
| answers. On this card, it tells you should ban it.
|
| OP didn't miss anything, the website is not subtly showing
| you how content moderation is difficult, it's telling you: we
| want to moderate it that way and you are failing or
| succeeding. That's not a lesson. That's gameplay.
| mirashii wrote:
| > Content moderation is hard, but I'm not very impressed that
| the very first card shown got it (in my opinion) totally wrong
|
| The authors are very clear that there are no right or wrong
| answers. Interpreting it that way misses the entire point of
| the exercise.
| ianferrel wrote:
| I mean, when I approved leaving up the cocaine post, the game
| that they authored gave me a big red X and said I did it
| wrong that it clearly was illegal and should have been taken
| down.
|
| Where were they clear that there are no right or wrong
| answers?
| wmf wrote:
| The users who submit reports are fallible, the moderators
| are fallible, and the bosses who rate the moderators are
| also fallible. Sometimes you'll be right and the system
| will still say you're wrong.
| Zak wrote:
| The "illegal" rule requires a lot of clarification. The
| interpretation here seems to be "It's against our policy to
| post about engaging in illegal activities", or maybe "it's
| against our policy to advocate illegal activities".
|
| Services based in the US are not legally obligated to prevent
| users from writing about using cocaine, or pictures of white
| powders they're claiming are cocaine, or writing that it's
| desirable to use cocaine, so the post itself is not _illegal_.
| Real policies for content moderators should be much more
| detailed than what 's in this game so that most moderators will
| reach the same conclusions about whether a post violates the
| policy.
| gnopgnip wrote:
| It's not just about what's legal, it's whatever the boss says
| oldtownroad wrote:
| That's the point though, right? Encouraging illegal activity
| may not itself be illegal but moderators aren't arbiters of the
| law, they're protecting the integrity of the website for the
| intended user base. If you want to allow people to encourage
| cocaine usage on your website, you have to be prepared to deal
| with law enforcement -- because when someone follows the advice
| and dies with the website open on their computer, the website
| operators will get a call... -- and not many people want to
| risk their entire company because "it's legal to encourage
| people to take cocaine!"
|
| Moderation _is_ hard and that example is... the perfect
| example. User reports are rarely accurate but still valuable.
| commandlinefan wrote:
| > the website operators will get a call
|
| Then what we actually need isn't more content moderation, we
| need explicit laws explicitly _protecting_ website operators
| and any other publishers who are actually following the law.
| pixl97 wrote:
| I mean not really... sometimes some content just sucks and
| you don't want it on your site regardless of its legality.
|
| Start shitposting here on HN and see how long your posts
| hang around.
| commandlinefan wrote:
| But then that becomes a site owner editorial decision
| rather than a fear of legal reprisals decision.
| NoMoreNicksLeft wrote:
| How can one protect the integrity of a website if they're
| undermining free speech rights and simultaneously derailing
| conversations and fragmenting communities while doing so?
|
| Though we didn't first define what "integrity" meant, it now
| seems impossible to formulate a meaning for it that isn't
| absurdly ironic given the actions performed to protect it.
|
| One could be forgiven if we were talking about a forum or
| website that exists in a totalitarian regime like China...
| better that there be some conversation than the only other
| option, none at all. But in the US or Europe or any other
| nation that considers itself even minimally enlightened?
|
| Moderation is not the solution. It may even be the problem.
| ianferrel wrote:
| Certainly social media companies can choose to disallow
| suggestions that people take illegal actions. But that's not
| what the moderation policy for "illegal" content says.
|
| It says "Content that violates local or international laws or
| regulations."
|
| I don't know all the laws of all the countries, but my
| _local_ laws make very little content illegal. Various types
| of obscenity, true threats, etc. "You should do drugs while
| you watch a movie about drugs" is definitely not one of
| those.
|
| It would be very easy, if that's what they want, to change it
| to "Content that encourages breaking local or international
| laws or regulations."
| oldtownroad wrote:
| Visit a platform like LinkedIn, Twitter or Facebook and try
| to report content: the complex decision tree of what to
| report and why will be a good demonstration or how
| unintuitive the reality of content moderation is. Every
| user of every platform has a different understanding of
| even simple concepts, like "scam" and "spam". Report
| reasons are _not_ an accurate classification from a trained
| expert, they're a thing a user chooses from a list that has
| to be designed to both provide valuable context to a
| moderator and capture the wide range of different
| understandings different users have.
|
| Suggesting that a reason should be added for the specific
| situation you've encountered demonstrates the naive
| understanding most people have of moderation and that's
| what this game is good at.
|
| Platforms like Facebook and Twitter will have spent tens of
| thousands of people hours thinking about something as
| simple as the list of report reasons.
| ianferrel wrote:
| There are plenty of difficult judgment calls in content
| moderating.
|
| But the difference between "illegal content" and "content
| that promotes breaking the law" isn't really one of them
| to be made by the moderator. This is just bad
| instructions from the game maker.
| ethanbond wrote:
| You don't think content moderators have to deal with:
| "hey kid, you should go shoot up a school?"
|
| Just cleanly not in their domain?
| ianferrel wrote:
| I think that content moderators often have to deal with
| that issue, and they are given rules like "remove posts
| that suggest that others commit violent acts" or
| something like that. And those are reasonable rules for
| moderators to enforce.
|
| But the rules that _this_ game give do not include that
| rule, and it 's a mark of a poorly designed game that the
| rule says "don't allow illegal content" and then when you
| (correctly!) apply that rule, the game says "you should
| have not allowed this because it's suggesting that
| someone do something illegal". Those aren't the same
| rule!
|
| ETA: Like lots of places have rules against reposting
| copyrighted content. But a post that said "Psst, kid, you
| should go download a movie from the Pirate Bay" should
| not be removed under that rule. Because the _content_ of
| the post isn 't copyrighted. If they _also_ had a rule
| that said "don't _encourage_ piracy ", they could
| reasonably take it down under that rule.
| ethanbond wrote:
| I see I see now what you're saying. You've convinced me
| :)
| ianferrel wrote:
| Always great to have a discussion with open-minded
| people. Cheers, internet buddy!
| gpm wrote:
| You know, I'm nearly certain that that post does violate
| some "international law or regulation".
|
| Do I know which one? Nope, not a clue. But many countries
| have pretty weak free speech protections, and pretty strict
| drug laws. The idea that that violates none of them seems
| quite unlikely.
|
| The fact that your local laws have strong free speech
| protections doesn't really matter when the criteria is
| "local _or international_ ". If it violates north korea's,
| china's, or singapores laws it still violates that
| restriction.
| DennisP wrote:
| Maybe the policy should be rewritten then. If no content
| is allowed that violates any country's free speech
| restriction, then it probably means the site will remove
| criticism of the government of North Korea.
| [deleted]
| rodgerd wrote:
| Well, that's a problem that social media companies have
| with China and India, both of which are big markets.
|
| These are actual problems.
| lostgame wrote:
| ...that seems like an obvious joke to you?
|
| My God, we must run in different circles. I had people I know
| who shamelessly posted almost that exact thing. A few of them,
| lol.
|
| They weren't joking. :P
| clutchdude wrote:
| Hey- not bad!
|
| I got 4 out of 5 stars with high ratings.
|
| One thing I'd add: doxxing of a quasi legal type such as posting
| details about a politicians family member and their private
| dealings.
|
| Certainly captures the feel of going through modqueue and making
| snap decisions.
|
| Glad I don't have a boss constantly reviewing my decisions
| though.
| wizzwizz4 wrote:
| I expected the tick to mean "take action on review/appeal", and
| the cross to mean "reject review/appeal", so that threw me for a
| few of them.
|
| A few of the cards were _really_ frustrating: the correct
| solution to some of them would be writing a ten-word response, or
| making an edit, but I only had "take down" and "leave up"
| buttons. ('It's just a game!' I remind myself.) I do hope _real_
| content moderators aren 't so bereft of tooling.
|
| The story is... really boring, actually? There are a couple dozen
| little things going on in the background, but not many, and
| there's not enough gameplay time for them to be explored. I'd
| hoped for something more like Papers, Please.
|
| The sentiment of the general public doesn't seem to behave
| realistically, either: not when I made obviously-unpopular
| decisions, and not when the CEO made obviously-unpopular
| decisions.
| pphysch wrote:
| Mobile UI broke completely on me before my first round was
| complete (Chrome, Android). Otherwise seems well-done and a
| useful learning tool.
| TomK32 wrote:
| I'd call that breakdown realistic.
| [deleted]
| NickC25 wrote:
| Got fired after 2 rounds.
|
| Well made!
| stonogo wrote:
| This is more like a game about shitty management communication.
| The rules are fairly arbitrary, and you're not told about any of
| them until you screw up, and even clear-cut cases like someone
| posting someone else's phone number are used to slap your wrist
| (apparently the second person consented, but nobody explains why
| that's your problem. Surely the second person can post their own
| phone number). Some advertisement is okay, other advertisement is
| forbidden. There's never a policy about which is which, you just
| have to guess. If this is how social media websites handle
| content reviewing it's no wonder the industry is a cesspit.
|
| In all after playing for about ten minutes, I have to conclude
| that this is some kind of corporate apologetics for bad content
| management. The pointless timer really drives home that this
| problem could be solved by spending more money on training
| content reviewers and hiring more of them. I came away from the
| game feeling like it's a sequel to Papers, Please.
| tedunangst wrote:
| You should intern as a content moderator and create a game
| based on your real experience.
| stonogo wrote:
| I've been involved plenty of content moderation, just not as
| an outsourcing target for a microblog company. I suspect the
| job is much easier when the rules are explicit, provided in
| advance, and the moderation team is sufficiently staffed,
| which is the environment I worked in. I have other priorities
| than making a computer game about it, so we'll never know for
| sure.
| shadowgovt wrote:
| Moderation is a back-stop service with plenty of candidates in
| the pool; companies fire new moderators for coming into the job
| with the wrong "taste" all the time because it's much, much
| easier to cycle someone out and grab someone new than to train
| someone who has the wrong "common sense."
| mulmen wrote:
| TBH I think this is a feature of human nature. It's
| impossible to impose your will and opinion at scale because
| you have to rely on other people with their own will. You can
| have small, curated communities, but you can't build Twitter
| without it devolving into, well, Twitter.
|
| The entire concept of modern social media where every site
| wants to be everything to everyone is wrong. Even the
| "fediverse" gets this wrong IMHO. There's plenty of room on
| the web for all of us. Links are features, not a threat to
| your retention metric.
| sdwr wrote:
| "corporate apologetics" - This is pretty clearly about the
| actual experience of working as a front-line worker in content
| management. Percieved time pressure, edge cases, lack of
| context, juggling 100 different priorities.
| schoen wrote:
| It's hosted by Engine, a lobbying organization for startups,
| so it seems likely that they think it will show people that
| companies hosting user-generated content face a hard
| situation (and double-binds where whatever they do will anger
| people).
|
| I was personally more convinced of the double-bind issue
| after playing: different groups of people (at every level)
| have different norms, the Internet helps bring them all in
| contact with each other, and I can see how that's a recipe
| for making them angry with each other and also with the
| intermediary.
| squirtlebonflow wrote:
| Is the point of the game supposed to be that the only possible
| way to keep up is to blindly approve 90% of them and then
| actually look only at the ones that get appealed?
| cushpush wrote:
| This to me highlights the many delights of freedom of speech. You
| can choose what level of chaos to permit. To enforce a global
| ethic this way seems kinda snooty.
| klabb3 wrote:
| I think the cards should just be what the post is, not a
| description of it. The descriptions (a) are hard to pattern match
| quickly and (b) the creators value judgments are inserted (for
| instance "ethnic slur" is ambiguous in many real cases, eg negro
| could be legit in a spanish-speaking or referencing context, or
| "white trash" which could both be a slur and the title of an edgy
| Vice documentary). In my experience the moderator personality
| types have actual blind spots in their biases around political
| topics (the misinformation, harassment, hate speech categories).
| They seem entirely unaware that had they lived 10 years earlier
| they would have completely different metrics.
|
| That said, one important takeaway I got is that moderation needs
| priorities. While I'm thinking for 15s about whether a review
| about a herbal health product is "medically misleading" I have
| CSAM in the queue right after.
| k__ wrote:
| Requires closer looks too often.
| Minor49er wrote:
| You don't actually get paid for playing this game either. Just
| like real life!
| h1fra wrote:
| Very interesting, got into level 6 then got a bit bored.
|
| For eduction purpose would be cool to explain rules after the
| fact.
|
| Small feedback: when there is an appeal, I'm not sure wether
| green means approving the appeal or stay with the initial
| decision.
| mulmen wrote:
| I loved the part where I lost a bunch of time to attend a
| mandatory nerf battle then got fired for not doing enough work.
| cushpush wrote:
| It's probably more of a spectrum of offensive to inoffensive, so
| the binary choice between permissible and not is going to vary
| depending on context. Maybe it could be a slider with "acceptable
| ranges" depending on platform? For example, I think 4chan would
| not mind most of these posts. But GlobalSocialNet might have some
| problems with some.
| alexb_ wrote:
| This game was made by a political action group for startups. Here
| is their agenda: https://engineis.squarespace.com/s/Startup-
| Policy-Agenda-202...
| mydriasis wrote:
| The "see more" button does not seem to be working for me, which
| is pretty much required to play the game effectively :(
| babuloseo wrote:
| Where is my ability to ban anyone that disagrees with my point of
| view or my agenda? I think this game needs to be modified to
| autoban anyone that appeals my mod decisions :)
| seany wrote:
| I don't get what the point of this is. Just approve everything
| and move on
| kadoban wrote:
| This said on a site that basically only exists because the
| moderation is extremely well done, and is definitely not just
| "approve everything".
| TomK32 wrote:
| Skipped the tutorial and got fired pretty soon, seems unrealistic
| ;-)
| dontupvoteme wrote:
| This reminded me that "Papers Please" exists.
| clutchdude wrote:
| "Glory to Arstotzka"
|
| If you liked that game, you might like Police Contraband. There
| is some bad combat aspects forced into the game, but it's
| enjoyable.
| iamnafets wrote:
| This gave me PTSD from working in T&S. Impressive level of
| detail, nice work creating it!
| DiscourseFan wrote:
| This would be better if there was more moral/ethical depth. It
| also doesn't explore how content moderation policies shape
| discourse and how moderators often times abuse such policies to
| manipulate posts that fit their ideology.
| Daveenjay wrote:
| Does Twitters community notes kind of fix moderation or could
| that also be manipulated?
| Karunamon wrote:
| Notes usually go too far in the other direction, for instance
| you will rarely see notes on mainstream politicians even when
| they are speaking objective falsehoods. This is because that
| system requires an agreement from people who usually disagree
| for a note to show up.
| [deleted]
| jakelazaroff wrote:
| Kind of both. Community notes and moderation are both
| emotional humans making the best decisions they can.
|
| There is no "fix". There's no magic. It's just us in here.
| moritzwarhier wrote:
| Extremely well-executed all in all. I like it.
| joe_the_user wrote:
| Wow,
|
| The game is tense and boring and it illustrates both why paid
| Facebook "moderators" suffer PTSD and why this so-called
| "moderation" fails to improve the content of discussion or build
| community.
|
| Actually worthwhile moderation involves someone caring about the
| whole direction of the discussion and not merely considering "is
| this content over the line", especially since a large portion of
| "don't talk about drugs" or whatever is pure CYA. "Don't say that
| 'cause someone might sue us".
| idopmstuff wrote:
| That works when you're dealing with something the scale of HN
| and have the resources to hire dang. At the scale of Facebook
| or Twitter, though? The sort of moderation you're describing is
| just impossible.
| nneonneo wrote:
| I guess I'm boring somehow, I got 100% in several rounds then got
| promoted after round 8 (which ends the game!).
|
| The trick seems to be that you have to check the "extra info"
| frequently, and be reasonably quick to decide as soon as you see
| it. It doesn't seem possible to make the right decision on many
| of the cards without that context. It is kinda fun to see all the
| attempts to game the system.
| yaseer wrote:
| Would love to see @dang's take on this!
| stainablesteel wrote:
| meh, its a bad interface
|
| if i wanted to moderate something more effectively i would make a
| better one
|
| i'd rather blame the difficulty of moderation itself on people's
| inability to innovate
| pavel_lishin wrote:
| A neat game, but my enjoyment of it is somewhat dulled since I
| watch my mom play this in real life on a daily basis - her job is
| moderating product reviews.
| jonny_eh wrote:
| Did it enhance your empathy for her work?
| pavel_lishin wrote:
| Not really; I've sat with her as she worked, and she shares
| enough things that I feel like this didn't add much to my
| knowledge.
|
| (And she doesn't take Nerf breaks!)
| ftxbro wrote:
| i wonder does she know about gpt
___________________________________________________________________
(page generated 2023-05-11 23:00 UTC)