[HN Gopher] Stop measuring community engagement
___________________________________________________________________
Stop measuring community engagement
Author : rosiesherry
Score : 225 points
Date : 2022-09-08 13:13 UTC (9 hours ago)
(HTM) web link (rosie.land)
(TXT) w3m dump (rosie.land)
| felipelalli wrote:
| But... How? It's missing how to do it (measure value) in the
| article.
| mhall119 wrote:
| I've started writing those answers out as separate blog posts:
|
| Measuring Connections:
| https://www.savannahhq.com/2022/09/06/measuring-your-communi...
|
| Measuring Support:
| https://www.savannahhq.com/2022/09/08/measuring-your-communi...
| slackerIII wrote:
| Early on in a community, there often aren't enough data points to
| cover other possible metrics. With that in mind, starting
| somewhere is better than not starting at all -- measuring
| engagement is a useful way to begin understanding what's
| resonating in your community. As you continue building your
| community, there are many different angles that you should use to
| evaluate its health.
|
| Measuring engagement then becomes one piece of data that's
| valuable, but shouldn't be the only piece. When engagement data
| is combined with qualitative community surveys, enthusiasm from
| members to contribute to a community, clear and timely
| responsiveness to community needs, membership growth over time
| and geographies, depth of member interactions (whether across
| community channels or within specific channels), what's topically
| important to members and why, and overall sentiment and change in
| sentiment over time - that's when community builders can begin to
| better understand the health of their communities and their
| impact on their business.
|
| Engagement is an important piece, but just one of the pieces,
| that helps paint the full picture.
|
| Disclaimer: I'm a co-founder of Common Room [1] and a we've
| invested a lot of energy in solving for this exact problem.
| You're welcome to check out the product (it's free to sign up)
| and would love to hear of ideas or feedback.
|
| 1: https://www.commonroom.io/
| dfabulich wrote:
| This article completely undermines itself by arguing in the
| conclusion ("Measure what you value") that we can and should
| measure "meaningful relationships" instead.
|
| > _They're not as easy to measure as engagement, sure, but they
| can be measured._
|
| I'm afraid that a citation is sorely needed here. There are no
| real measurements for "meaningful relationships."
|
| In fact, the only proxy metric we have for "meaningful
| relationships" is engagement!
|
| And, yes, engagement is not a very good proxy metric for
| meaningful relationships, but since there's no alternative, it's
| all we've got, so this article is pointless.
| nmilo wrote:
| I don't get it. What is a "community manager"? Is it someone that
| runs a company's social media page? And do they think that people
| care about their brand so much that they would join a community
| of people whose only common trait is liking the same brand? And
| in the first place, how do you even build a "community" manually
| and artificially? It's almost a bastardization of the word, as if
| a community is not a set of people with similar goals but instead
| yet another way to convert people's time and desire for social
| interaction into profit.
|
| The article implies that community building should be some kind
| of altruistic purpose, where your only goal is to maximize the
| amount of "meaningful relationships" created? But building an
| artificial community in the first place can never be altruistic
| because the end goal of it all is to guide people to your product
| or conference or whatever. If you were an actual altruistic
| community builder, you would be telling people to get off the
| Internet and go make "meaningful relationships" with people in
| real life.
| [deleted]
| mhall119 wrote:
| > I don't get it. What is a "community manager"? Is it someone
| that runs a company's social media page?
|
| No, that's a social media manager, completely different role.
|
| > And do they think that people care about their brand so much
| that they would join a community of people whose only common
| trait is liking the same brand?
|
| Yes! It happens all the time. Most of the biggest brands have a
| community of people who share that common trait.
|
| > And in the first place, how do you even build a "community"
| manually and artificially?
|
| Again yes, just like you build a garden. The gardener doesn't
| make the plants grow, but they do make sure they have the right
| environment and resources to grow in.
|
| > The article implies that community building should be some
| kind of altruistic purpose, where your only goal is to maximize
| the amount of "meaningful relationships" created?
|
| That's not altruism. Those meaningful relationships help the
| company/project the community was built around. They provide
| feedback, help improve processes, make connections outside the
| company/project, provide support to other users, there are too
| many things that communities do to list them all here.
| JohnFen wrote:
| > That's not altruism. Those meaningful relationships help
| the company/project the community was built around.
|
| Which brings up another aspect of this whole issue -- we're
| talking about companies figuring out how to exploit the human
| need for community in order to increase profits. The
| companies themselves are not encouraging real community,
| they're farming people. Companies are not people, do not have
| the best interests of people in mind, but pretend like they
| are.
|
| It's very distasteful at the least.
| mhall119 wrote:
| That's like saying grocery stores are exploiting the human
| need for food.
|
| A healthy community benefits the people in it as much or
| more than the company behind it. As you identified, humans
| have a need to connect with other humans. And often they
| want to connect around a product, service or hobby that
| they share with other humans.
|
| It should also come as no surprise here on Hacker News that
| humans also really like to tell companies what they think
| about their product, and how to make it better. They also
| like the appreciation they receive from helping others use
| the product better.
|
| A company-backed community serves a need that the members
| of that community have. Nobody is being forced into it or
| exploited. We're giving each other a mutual benefit that,
| as long as it remains mutually beneficial, we are happy to
| keep investing in.
| JohnFen wrote:
| > That's like saying grocery stores are exploiting the
| human need for food.
|
| No, because a grocery store's purpose is to sell a
| product. That product happens to be food. That's very,
| very different from a company leveraging human emotional
| need in order to sell something else entirely.
|
| > A healthy community benefits the people in it as much
| or more than the company behind it.
|
| I would argue that a healthy community doesn't have a
| company behind it -- even if the community is centered
| around a particular company's products.
|
| > A company-backed community serves a need that the
| members of that community have. Nobody is being forced
| into it or exploited.
|
| I think we have a fundamental disagreement here. And
| that's OK.
|
| A company-backed community serves the company. The proof
| of that is what happens when the company-backed community
| starts becoming too critical of the company -- then the
| truth of the relationship rapidly becomes very clear.
| alexb_ wrote:
| >Now imagine if this played out the other way around. Imagine
| that the monsters originally focused on maximizing energy by
| making children laugh as much as possible. Then one day, in a
| horrible twist, someone discovers that a child's scream was a
| much easier and more powerful way of meeting their goals. Their
| company, which built a platform that gave them instant access to
| children around the world, suddenly realizes that they could
| reach their goals by spreading fear rather than joy.
|
| Then they would instantly start using screams? Nobody making
| business decisions in social media companies will prioritize
| "user happiness" over increased production of their main product
| (your attention) for their customers (advertisers).
| egypturnash wrote:
| Congratulation, you grasped the intent of that analogy!
| alexb_ wrote:
| Maybe I'm reading the intent wrong, but no social media
| company will ever take this suggestion. This line in
| particular:
|
| >Let's bring this back to community building. We still want
| our members to be happy, right?
|
| If it doesn't make them more money... no they don't!
| mhall119 wrote:
| That's why it also says "Community isn't social media."
|
| No social media company is going to do this, you are
| correct. But community managers and community-led companies
| absolutely should.
| catherd wrote:
| Some will and some won't. In the absence of rules that penalize
| that type of behavior, usually the ones that will end up
| winning. It has nothing to do with morals, those just seem to
| be the rules of the game as we currently play it.
| DoreenMichele wrote:
| _Even though it's the energy they're after, not the screams, it's
| the screaming that was their primary metric, and everything about
| their company was organized around increasing that._
|
| Missed opportunity to bring it full circle back to this.
|
| Otherwise, not bad.
| pdntspa wrote:
| Hot take: stop building communities around everything.
|
| People are highly tribalist and banding together like that
| produces a lot of ugly outcomes. Communities form and likely
| won't go away, particularly the more people internalize them
| emotionally and integrate them into their identity.
|
| It is exhausting to see a community for literally everything,
| even the smallest products.
|
| Perhaps we should let products simply exist.
| neilk wrote:
| You're raising an interesting point, but you're coming close to
| saying that communities are bad, and in my experience they
| don't have to be. Even ones based around a product.
|
| Near where I used to live in the Valley, there was a real-world
| club for enthusiasts of a particular weird antique car. Not
| toxic, just eccentric. They just liked getting together and
| drinking in a nearby pub and showing off how they'd restored
| this or that aspect of the car.
|
| I think our goals should be to replicate that kind of success.
|
| I think the other posters in this thread have it right, that
| the reason why we usually can't is because our metrics are all
| about selling audiences to advertisers.
| BrianOnHN wrote:
| The other day I was think, it's actually the platforms where
| most of these communities that benefit _directly_ from their
| existence.
|
| And we know those platforms are tools for manipulation.
|
| How much of the desire for these communities might be
| manufactured in the first place?
| pdntspa wrote:
| That's the problem... so much of both the communities and
| desire itself are manufactured. When they grow organically...
| cool, for the most part. But the minute money becomes a focus
| things go to hell.
| wsb_mod2 wrote:
| The profit motive is being blamed here for optimizing for
| engagement.
|
| But how do you assess progress when you remove all desire to
| monetize?
|
| I run r/WallStreetBets and "quality" is an extremely nebulous
| term.
|
| I look for things like "novelty", "thought-provoking / well
| reasoned commentary", "original content", "authenticity", "self-
| awareness" or "primary research". But these are human assessed
| metrics.
|
| Some more easily measurable metrics might include, "length of
| submissions", or "number of outbound links excluding blacklisted
| domains". Or even "number of tickers or quality-correlated
| keywords mentioned".
|
| All these metrics have very clear downsides, and if generally
| well-known, become useless. Interestingly, a score too high can
| also result in something being unlikely to be authentic.
|
| Another challenge is your relationship with users. Surprisingly,
| moderators are not innately adversarial to users, they can also
| promote content through other channels (discord, twitter) or
| sticky threads for a viewership boost.
|
| So, even without a profit motive, what do you do?
| empathy_m wrote:
| Historically, the immortals on MUDs used to optimize for thing
| like "fun" and "clout" - how much do you enjoy interacting with
| the players and being seen as the person in charge of their
| game? How much do you enjoy telling people you volunteer your
| time to work on it?
|
| If WSB members were doing bad stuff like, making mean jokes
| about the leadership of companies Ryan Cohen divests from in;
| or coordinating real-life harassment of family members of
| employees of the Depository Trust Company or something; you'd
| probably want to put a stop to that and focus them somewhere
| else. Because it wouldn't be fun.
|
| If Fox News interviewed you and you sounded silly and all the
| community members got mad at you, you'd probably also quit the
| volunteer job.
|
| But as long as you make choices that make the community a fun
| and engaging place to hang out and feel like you're part of a
| big secret treasure hunt, that seems awesome, and you would
| probably want to make choices that maximize that.
|
| TBH most MUDs were not data-driven and kinda govern by feeling
| -- they'd sort of add steering they thought was interesting and
| keep it in place unless there's a backlash. The tedious
| administrative stuff (bans, moderation, player requests for
| item reimbursement) would always have a backlog and you'd
| recruit junior imms to help out and they would feel like they
| were part of the fun too.
| JAA1337 wrote:
| Just want to give you a shout out for using the acronym MUD.
| I havnt seen it used in like 25 years :). Thank you!
| smeyer wrote:
| What does it stand for?
| solardev wrote:
| Multi user dungeon, basically like early MMORPGs, usually
| text based. MMORPG are massively multiplayer online role-
| playing games, like World of Warcraft.
| yoyohello13 wrote:
| Multi-user Dungeon: https://en.wikipedia.org/wiki/MUD
| jrochkind1 wrote:
| > But these are human assessed metrics.
|
| OK, and? I could guess what the problem with this is, but let's
| spell it out... maybe that they are subjective so different
| assessors can disagree... and this is a problem why?
|
| What if there are some cases where it's actually important to
| use human-assessed metrics as one component, or even as the
| entire thing, cases where no appropriate 'objective' metrics
| are available? (In scare quotes, because these quantitative
| metrics are seldom _quite_ as 'objective' or independent from
| human judgement as assumed. There is usually human judgement
| involved in how the metrics are defined and measured, where
| different people might define and measure a metric different
| ways resulting in different numbers...)
| zinglersen wrote:
| There are definitely some cases where it's important to use
| human-assessed metrics, like cooking a meal.
|
| But if you are the size of Facebook, Instagram, Tiktok, etc.
| it is too risky (and expensive) to let individuals define
| what success looks like - you need something measurable that
| you can spread to multiple projects or even across the
| complete company culture.
|
| I fully agree with your point that metrics are typically not
| objective, like any data once it's used to communicate
| something.
|
| So the question should be what metric(s), if not engagement
| in itself, should we measure for to create a healthy online
| community?
|
| I wonder if it's less about metrics and more about
| principles; how does HN keep the level of quality so high? Is
| it because we share an interest? Because of the quality
| control? The UI/UX of the site? Something else?
| jrochkind1 wrote:
| > how does HN keep the level of quality so high? Is it
| because we share an interest? Because of the quality
| control?
|
| My guess is that dang's moderation is a large part of it,
| and I think dang moderates (creates moderation policy and
| executes it) based on things that are "human-assessed" and
| not quantifiable....
|
| > But if you are the size of Facebook, Instagram, Tiktok,
| etc. it is too risky (and expensive) to let individuals
| define what success looks like
|
| You're _probably_ right, but I think it 's worth
| challenging this conventional wisdom. (Not necessarily
| here, we're not going to work it out, but still, I'll ask
| some rhetorical questions) Why is it too dangerous? Even if
| you have quantitative metrics, are you _sure_ this isn't
| still "letting individuals define what success looks like",
| since individuals decided how the metrics were defined and
| measured? So then, what's the difference? Why,
| specifically, is it too "dangerous" to have non-
| quantitative metrics defining what success looks like?
| Danger of... what? I would say facebook as it is, is
| actually _incredibly dangerous_ to, like, human society.
| So... what kind of danger are we talking about? Danger to
| facebook 's profits instead? Or what?
|
| > So the question should be what metric(s), if not
| engagement in itself, should we measure for to create a
| healthy online community?
|
| Sure. I don't know! I think that's the question the OP is
| meaning to ask too, if not completely answer. The OP
| suggests:
|
| > At the start of this, I said that people join your
| community for support, connection, opportunities to give
| back, and meaningful relationships. Those are the things
| they value, and those are the things you should value. And
| if you value them, you should measure them.
|
| > They're not as easy to measure as engagement, sure, but
| they can be measured. The best part is that maximizing
| these metrics is always going to be good for your
| community.
|
| My point is that i'm not certain this should be short-
| circuited with "Well of _course_ whatever metrics these
| are, they need to be quantitative and have the appareance
| of "objectivity".
| bombcar wrote:
| > I wonder if it's less about metrics and more about
| principles; how does HN keep the level of quality so high?
| Is it because we share an interest? Because of the quality
| control? The UI/UX of the site? Something else?
|
| HN is successful because it can be moderated by one dang.
| Maybe a bonus helper can appear.
|
| But once you get so big that your dang can't do it all
| anymore, it will eventually fall apart.
| marginalia_nu wrote:
| Why does it need to be measurable?
| runnerup wrote:
| Measuring community engagement makes sense when your primary
| mission / core goal is to build a strong community. When I was
| building a large online third-party Discord community for a
| university during COVID, I was very glad that Discord exposed a
| lot of metrics for admins/mods to track. It's obviously
| important to not "game" the metrics by falling into short-term
| gain traps like over-pinging users, or encouraging high-volume
| but low-quality participation, etc. But as long as you are
| laser-focused on "providing real value" to members, then long-
| term or multi-cyclical trends of community engagement is a
| useful benchmark.
|
| But it doesn't make sense to focus on these metrics if your
| mission is to create a great product -- focusing on community
| engagement should be just one small aspect of your overall
| marketing strategy, which should be a relatively constrained
| part of your overall budget (both in labor and money). And even
| within marketing, there are many other important metrics
| besides community engagement.
|
| Soylent had great community engagement but not enough focus on
| scientifically-guided product engineering. Tons of businesses
| that get this wrong.
|
| Generally I interpret overcommitment to community engagement as
| a signal that the founders are overly narcissistic and usually
| apply a discount the expected future value/evolution of their
| product.
| JohnFen wrote:
| > Measuring community engagement makes sense when your
| primary mission / core goal is to build a strong community.
|
| But measuring community engagement doesn't tell you anything
| about the nature of the community. Only that it "engages"
| people. If your goal is to build a strong community, isn't it
| more important that the community be a good one?
| MartinCron wrote:
| Lynch mobs are strong and engaged communities, after all.
| runnerup wrote:
| The most important things (quality of community) are not
| generally quantifiable and proper "measurement" depends on
| the leadership ability of those who choose to foster the
| community.
| wussboy wrote:
| Strong does not mean good. Those are two separate goals.
| The most important developments of the next 100 years will
| be in determining ways to make strong and good communities
| that scale.
| tstrimple wrote:
| > All these metrics have very clear downsides, and if generally
| well-known, become useless.
|
| Good ol' Goodhart.
|
| "When a measure becomes a target, it ceases to be a good
| measure."
| AndrewKemendo wrote:
| So, even without a profit motive, what do you do?
|
| I think there are two options here:
|
| 1. If the community that you have does a "foom" and grows like
| crazy BEFORE it has a set of values, then you're at the whims
| of the mob and where certain vocal and rallying members decide
| they want to take the community. Usually this ends in
| internecine conflict and the inevitable "break offs" into other
| smaller groups
|
| 2. If your community starts with a value vector then you
| structure, nudge and contain the community to a MDP (Markov
| Decision Process) with the value vector as the reward, for
| which the moderators are the critics and the community members
| are the actors in the actor-critic model.
|
| It seems like the vast majority of organizations/movements
| etc... fall into category 1
|
| For me the learning here is, if you are a leader in your
| community it's never too early to nail down what the core value
| and benefit of your community is and stay obsessed with just
| that.
| hammock wrote:
| Hard to scale, but farming out content samples to a panel of
| your target market for them to rate on the attributes of
| "novelty", "thought-provoking / well reasoned commentary",
| "original content", "authenticity", "self-awareness" or
| "primary research" might be an approach
| groby_b wrote:
| > So, even without a profit motive, what do you do?
|
| You acknowledge that human interaction and metrics are not
| compatible, and apply actual judgment. You're a human being,
| not a paperclip optimizer.
|
| Your goal isn't to stripmine anything of value out of a
| community, but to provide a vibrant place people like coming
| to. So, for feedback, you ask them what they like, what they
| miss, what they wish for.
|
| Qualitative assessments instead of quantitative ones. Lean into
| the power of the humanities.
| JohnFen wrote:
| > But how do you assess progress when you remove all desire to
| monetize?
|
| If you ignore the profit motive, presumably you are doing it
| for other reasons. You assess "progress" based on those
| motives. Or perhaps you don't assess "progress" at all, beyond
| "I like what I'm doing here and where this is going".
|
| I've run numerous internet services over the years without the
| goal of them generating an income at all. I've run MUDs,
| websites, discussion groups, etc. I didn't objectively measure
| anything about any of them, because my goals were not ones that
| could be objectively measured. And honestly, even if there were
| some metric that could be used, I would have avoided using it
| because then it becomes about maximizing the metric rather than
| the purpose I started the activity in the first place. If I was
| happy with how they were going, that counted as "success" to
| me.
|
| > So, even without a profit motive, what do you do?
|
| I honestly don't understand this question. Without a profit
| motive, you do it for other motives. If you have no other
| motives, then why are you doing it at all?
| scythe wrote:
| >I didn't objectively measure anything about any of them,
| because my goals were not ones that could be objectively
| measured. And honestly, even if there were some metric that
| could be used, I would have avoided using it because then it
| becomes about maximizing the metric rather than the purpose I
| started the activity in the first place.
|
| There _is_ a middle ground here. Choose a basket of metrics,
| impose the natural partial ordering, and intuit your way
| through pushing up one or the other. If this sounds crazy,
| keep in mind it 's roughly how the Fed (inflation +
| unemployment) runs the monetary system.
| the_snooze wrote:
| >because my goals were not ones that could be objectively
| measured
|
| This is something that technically-minded people can take to
| heart more. There's a lot of value in things that can't be
| objectively measured. For instance, I'm in a bunch of group
| chats with various friends. I don't always interact with it
| (i.e., my "engagement" is fairly low), but I derive a lot of
| enjoyment from those. Why? Because I like those people. Can I
| put a number on how much I like those people? No, because I'm
| not a socially stunted robot.
| wolfofcrypto wrote:
| Mark Cuban recently launched a community at https://biztoc.com
| as sort of a WSB for business people. I wonder if that
| will/could work without an umbrella topic.
| swyx wrote:
| when community engagement is measured badly, its because
| community is being treated as a product like any app where MAU
| and time on site is being prioritized, yet companies dont
| dedicate even 1/100 of the resources to community "product" that
| they do their software product (perhaps because it is not viewed
| as a real job, as perceived in other comments here)
|
| that said, you do need a way to justify to others why you are a
| better community manager than the next, and whether you are
| doinng a good job. that part went unanswered here and the article
| would be better if it had a constructive replacement instead of
| just arguing to be unaccountable to the rest of the org.
| mikkergp wrote:
| I think people enjoy this argument because it's counter
| intuitive, but I also think it's counter intuitive because it's
| wrong. If you build a playground and everyone avoids the teeter-
| totter, there's probably something wrong with the teeter totter.
| Oddly the monsters inc example seems counter to his point since
| it seems like they were measuring something to specific and the
| proper solution would be to something more general (like
| engagement) instead. It seems to me like trying to start with
| something other than engagement would be premature optimization.
|
| Of course engagement is insufficient and of course if you want to
| know something else, measure that other thing, and of course you
| should interrogate your metrics, and maybe it's just this thing
| that bugs me about modern writing where you have to take this
| ridiculous extreme stance to get eyeballs but yeah.
| ChrisMarshallNY wrote:
| That's a great post, and I plan to share it with others.
|
| Sadly, it's pretty difficult to measure for some of the "values"
| that my communities run on, but "engagement" isn't even on the
| map.
|
| I suspect that "engagement" does, indeed, provide real monetary
| value for advertising-based sites, as clicks == money. Since that
| isn't a factor in my communities, we don't worry about that. In
| fact, we try to minimize "engagement."
| jpster wrote:
| This sounds great, but I see a challenge. The author recommends
| measuring e.g. user happiness instead of engagement. How to
| measure user happiness? IMO, some type of survey could be used.
| But the "best" metrics are based on observable behavior. Not what
| users self-report, because they may not know how they actually
| feel about the experience. In general it's better to see what
| users do rather than what they say.
| ItsMonkk wrote:
| You must have an expert or experts. That expert will then make
| decisions, and use metrics and surveys and intuition to drive
| their new world view, but it's the expert that is always at the
| top of the stack.
|
| If you then take that model and try to use them in a new
| domain, that is never going to work out. The environment adapts
| to the incentives that the current system produces, and will
| instantly make your pre-created model worthless[0]. The only
| way to work out long-term is by having an expert stay ahead of
| the pack creating what they think are systems with better
| signal to the noise.
|
| [0]: https://dilbert.com/strip/1995-11-13
| mhall119 wrote:
| I'd recommend reading The Community Engagement Trap[0] by
| Rosie Sherry (which should also be shared here on HN), she
| talks about Community Discovery as the process for getting
| these deeper understandings.
|
| [0] https://rosie.land/posts/the-community-engagement-trap/
| Bjartr wrote:
| Observables are better, but self reported metrics do still have
| value. It's an aggregate version of the problem devs encounter
| when doing usability testing. Users will tell you what their
| experience is, but it's up to you as a software designer to
| distill that feedback into actual needs. Which may not at all
| be what they actually ask for. Survey responses are similar in
| that they don't tell you exactly what to do, but they do
| contain actionable information if you have the skill to
| interpret it.
| tamsaraas wrote:
| > Originally they were there to connect people and let them share
| in each other's joy.
|
| No, they looking for a ways how to easily and faster to fuck.
|
| > They couldn't measure joy, but they could measure engagement,
| and so they did. > That measurement became their goal, and they
| focused on maximizing it.
|
| Because this is social networks profit. Ads, and more people
| attached to some topic where can be shown an ads to these people.
| That's why and only that's why it is works like that. Nobody
| cares about principles. Only money. Power in money, a money is a
| power.
|
| > What they didn't understand was that negative, divisive, hate-
| inspiring posts get more engagement than positive, supportive,
| kind posts do.
|
| For what do your country has intelligence and different defense
| structures? Because it is very easy manipulate by people and do
| wars. Extremely easy to organize that and involve a lot of people
| to mass destruction events. Extremely easy. And i do not know
| (the same as you too) who is behind some kind of negative behind.
|
| > . The algorithm didn't care whether the posts brought joy or
| anger, it only cared about whether they brought engagement.
|
| Because this is about ads and money. More people will watch ads
| more money social network will get. Easy.
|
| > So in order to maximize engagement, the algorithm actively
| encourages and elevates posts that cause unhappiness among the
| platform's users.
|
| You said that? Is a bots with fake votes, SEO thing is not about
| manipulating data, opinions? Its easy to make fake accounts and
| promote hate as a wide interested topic, while it's not. Only
| social network know that.
|
| > They join looking for support, or connections, to give back, or
| find meaningful relationships.
|
| No, they looking for themselves reflected in the community.
|
| > Measuring engagement doesn't tell you if your community is
| happy, or healthy, or even accomplishing any of the things you
| were hoping it would accomplish.
|
| Depends on a case. Some measuring engagement can prevent negative
| business processes.
|
| > Knowing how much engagement you have doesn't tell you how much
| value you're getting
|
| For that you have another metrics. Depends on a case.
| Registration, profit, grow, etc. Final countable results.
|
| > Let's bring this back to community building. We still want our
| members to be happy, right?
|
| Who the fuck am i to let people happy? Happiness is subjective
| thing. I have no clue, no ability to make someone happy. The same
| as you, the same as anybody else. This is very agile thing
| depends on many factors and the most important wish to be happy
| and objective situation.
|
| > So what should we value?
|
| Does your community member even read or know about "values" at
| all? Is it some kind of cult, or what?
|
| ----
|
| I have a well known secret. To get maximum engagement need to say
| that the white color is black color. Water - is not a liquide,
| and planet earth is flat.
|
| And then your "engagement" will go above the sky. Just always
| need to split people, and say that one thing is not the thing
| what you used to think, it's another thing. totally vise versa.
|
| Oh... Darling, exactly this you have done by saying: "stop
| measuring engagement". And by this title invited to engagement a
| lot of people who forced to read all of your nonsense to one more
| time confirm that white is white, black is black.
| burlesona wrote:
| Good article, though I think it missed explicitly making the
| point that the _reason_ "engagement" is the metric is because
| that's what platforms _monetize._ Engagement is a euphemism for
| attention, and social platforms exist to sell your to
| advertisers.
|
| Since the article is written for "community builders,"
| understanding that engagement is a metric to quantify the
| advertising potential of a platform should make it obvious that
| people who _aren 't_ in the business of selling attention to
| advertisers don't _need_ to copy social media and optimize for
| engagement; instead they should target better measurements of
| value creation for their community. Of course Goodhart 's law[1]
| still applies.
|
| 1: https://en.wikipedia.org/wiki/Goodhart%27s_law
|
| (edited for readability)
| throw10920 wrote:
| > the _reason_ "engagement" is the metric is because that's
| what platforms _monetize_
|
| ...and the implication of _that_ is that if we change the
| monetization model, then the metrics _also_ change.
|
| For a company selling a paid product, metrics become things
| like the effectiveness of the sales funnel, how much support
| the average user requires, the rating of the product on app
| stores, etc - which are not ideal, but still _far_ better than
| "engagement".
| crazygringo wrote:
| You might be surprised to discover that even for paid
| products, engagement is often still the #1 metric.
|
| Simply because people who are engaged with a
| product/app/whatever are by far the most likely to be repeat
| customers -- to renew their subscription, to buy the new
| version, to adopt the next product.
|
| Once engagement drops, future revenue dries up. A drop in
| engagement is the canary in the coal mine -- it shows where
| your business is going wrong months/years before it shows up
| in sales stats or renewal rates.
|
| So actually, the metric doesn't change at all. E.g. Netflix
| is obsessed with engagement even though it's subscription-
| based rather than ad-based. Because guess who winds up
| cancelling their subscriptions?
| mhall119 wrote:
| s/surprised/disappointed/
|
| A user's engagement with your product is a great metric,
| because what you value is users using your product.
|
| But for a COMMUNITY it's the wrong metric, because you
| don't value negative comments in your Slack no matter how
| many there are (in fact, the more there are the worse it
| is).
| cyborgx7 wrote:
| Exactly. It took me a while to understand that this article was
| written for community managers of brands, not people developing
| social networks. They keep saying engagement has no value, but
| to social networks and to "influencers" who both make their
| money off of ads, that is exactly what has value.
| hinkley wrote:
| Engagement is just a more sophisticated version of 'eyeballs'.
|
| For community building you want successful events, you want
| interesting questions to be asked, and interesting answers to
| be offered. Someone could write a book on how to make
| communities work. Probably several books.
| mhall119 wrote:
| There are several books on precisely this.
| phkahler wrote:
| >> Good article, though I think it missed explicitly making the
| point that the reason "engagement" is the metric is because
| that's what platforms monetize.
|
| In the case of "community" around a product, the purpose of the
| community is usually to provide free support. That's not so
| much monetization as it is cost savings. For that to work,
| people need to be "engaged" or kept around so they are present
| to field those support issues.
| Theodores wrote:
| You are completely right on Goodhart's Law. I know this from
| experience.
|
| I put the customer first in the belief that 'the rain follows
| the plough' and, if you put the customer first, then all else
| follows.
|
| I see metrics as there to be cracked. If you are measuring the
| right things then you can see where the problems are, fix the
| problems and move on to measuring different metrics that are
| more specific and focused on the remaining problem areas. For
| me it is a campaign where initial reactive firefighting gives
| way to calm, proactive problem solving, fixing problems before
| they are of consequence. Along the way many unexpected things
| are learned about the customer.
|
| I do all this with a focus on the customer, meanwhile my
| colleagues in marketing are out for themselves. They are the
| dead weight keeping progress back as they have to do things
| like buy traffic and hold on to keyword optimised URLs that are
| cargo cult SEO. Their reports for their micromanagers are
| always measuring the same stuff, e.g. 'engagement', yet they
| could be engaging customers solely for the purpose of pissing
| them off for them to never come back.
|
| My favourite is the pop up shown as someone leaves a website
| begging them to stay, or the sign up to the newsletter popup.
| These could get one extra sale at the cost of pissing off a
| million people but if you only measure the former then it all
| seems good.
|
| In the capitalist world only one metric matters to the
| shareholders. Once I worked for a highly successful company
| that sent out sales newsletters with no measurement whatsoever
| of open rates or click throughs. None of that was needed. The
| problem was emptying the warehouse too quickly to have no stock
| left. Customers would be waiting for our email, not ignoring
| them. We had plenty of engagement with the customers as they
| actually bought stuff.
|
| Subsequently I have worked for agencies where they spend a
| fortune on some email service and they get these fancy graphs
| to show the clients of these open rates the world over. Which
| had wow value at the time and 'engagement' stats. But I had
| come from somewhere where we had no need to measure that stuff,
| we weren't looking at these charts, we were coping with a
| deluge of orders and had no time for that.
| bluGill wrote:
| Unfortunately what is really needed here is a suggestion of
| something else to measure that is better. I fight this a lot with
| bad metrics (code test coverage), nobody wants to give them up
| unless I can suggest a replacement to measure.
| mhall119 wrote:
| Here's a couple:
|
| Connections between people:
| https://www.savannahhq.com/2022/09/06/measuring-your-communi...
|
| Support: https://www.savannahhq.com/2022/09/08/measuring-your-
| communi...
| jamesknelson wrote:
| This is an issue with more than just the tech industry. It's an
| issue with modern society itself.
|
| Take the GDP, the number that when it's first derivative goes
| negative, we shout "recession" and use it as an excuse for all
| manner of harmful (and sometimes beneficial) behaviors. What is
| GDP? It's not a measure of wealth; it's a measure of production.
|
| Take the boots paradox, for example. A society making $50 boots
| that need replacing after a few months, will have a higher GDP
| than a society making $100 boots that last for years. And the
| shitty boots society, despite spending more on boots - to
| paraphrase Terry Pratchett - will still have soggy feat.
|
| It's a hard problem, and I can't see any obvious way to solve it.
| Both in social networks, and in economics. For whatever reason,
| consumption seems to be much easier to measure than wealth.
| fullshark wrote:
| I enjoyed the Monsters Inc reference and the turning of it on its
| head central to this piece. But we've said/argued this for years
| and at the end of the day engagement = ad reach = profits and
| until the economics there change any for-profit social media will
| gravitate towards engagement. Hell ANY media gravitates toward
| engagement and outrage bait. An actual community built around
| Mastodon seems possible, and there's community discussions around
| private group chats but they will always remain niche and not
| profitable.
|
| I don't get what https://savannahhq.com is and what the value add
| is there but that seems to be what he's interested in, niche +
| small communities.
| snarf21 wrote:
| Well said! Outrage sells and it is easy to keep it stoked.
| Everyone is addicted to free. If we get rid of ads, how does
| (even the barest bones) Twitter keep the lights on?
|
| I wonder if there is a cross-over between Patreon, Substack and
| Twitter. This would probably work for niches but not writ
| large. But at the end of the day, we need to find a way to pay
| for what we value because that is how we get more of it.
| Interested in new ideas in this space....
| ethbr0 wrote:
| The Medium/Spotify model -- per-user constant subscription
| fee, apportioned to content creators on the basis of their
| views + some time-viewing / read-to-end adjustments to
| discourage clickbaiting -- seems fairly decent, but really
| requires a global platform to work.
|
| But product companies don't like it because they can't scale
| revenue user-transparently by repackaging data (e.g. building
| new / better ad products), and raising prices for users
| sucks.
|
| The original sin is that the web was free, and the
| consequence of that, once post-content-scarcity in a free
| market system, has been continual sleight of hand to make
| money from "free."
|
| If everyone has gotten in the habit of paying for quality
| content... we'd have a very different world now.
|
| (And this said as a hard-charging "information wants to be
| free" type. But the end commercial result nowadays is
| terrible)
| politician wrote:
| I'd prefer not to live in the AOL/Prodigy/Compuserve
| blackhole that was the Internet before "the original sin
| that the web was free." Originally, it wasn't free.
|
| I do agree with you that it would have been better if the
| payment-related features of the HTTP spec weren't just
| dead-on-arrival stubs.
| JohnFen wrote:
| But the internet before AOL/Prodigy/Compuserve was pretty
| awesome. And free (aside from connection costs -- but
| that's no different than today).
| marcinzm wrote:
| Spotify seems to be investing a lot into ads recently.
| JohnFen wrote:
| > The original sin is that the web was free
|
| I disagree. The thing that has destroyed much of what made
| the internet great is that it became ad-driven.
| ethbr0 wrote:
| And how could it not have become ad-driven?
|
| Choices have consequences. We made a choice (free), and
| now we live with the consequences (ad models).
| JohnFen wrote:
| Well, that there was a rich and vibrant internet before
| advertising existed on it seems to indicate it's entirely
| possible.
|
| But I don't think we chose ad-supported. The ads came in
| before anyone was seriously thinking about how to charge
| people for things. The decision was made for us. And even
| if we did pay cash money, the ads would have come anyway
| if history is any indication.
|
| So perhaps the degradation of the internet in this way
| was inevitable. Still, it's a real loss and a real shame.
| kevincox wrote:
| I think the problem is that while engagement may be the source
| of profits that is short-term thinking. If joy based engagement
| drives profits and keeps users on your site long-term it may be
| worth it to take the short-term revenue boost that anger-based
| engagement brings you if it slowly drives people away from your
| site.
|
| Of course it isn't completely clear if anger-based engagement
| actually drives most people away. In that case it is actually
| Facebook's fiduciary duty to capitalize on it which seems to be
| a failing on our society.
| marcinzm wrote:
| Facebook is still not shrinking in the US in terms of users.
| So even if there is a long term cost it seems well beyond the
| time horizon companies care about.
| mhall119 wrote:
| Community isn't social media.
|
| You don't advertise to your community. If you do you're
| actively killing it.
|
| Community is made up of people making connections and building
| relationships with other people. Savannah CRM helps you manage
| those connections and relationships.
| fullshark wrote:
| Yes I agree, I think this is basically the point I was trying
| to make distilled.
| puchatek wrote:
| This. And also this:
|
| > We still want our members to be happy, right?
|
| ... _we_ might want that, but do the people running the
| platforms?
| photochemsyn wrote:
| 'Stop making decisions based on engagment measurement' might be a
| better approach, if what you care about is maintaining the
| quality of your online social group. Some social media sites have
| tons of high-quality original content submitted by people with
| expertise in their domains, and some are 90% fluff. If you want
| to attract people who'll make high-quality contributions, getting
| rid of the fluff is the most important goal.
| robust-cactus wrote:
| Lately - it seems we're all caught between 2 worlds:
|
| 1. If you over focus on impact the quality of your product gets
| torn to shreds. You've built great direction but do people really
| use/like your product?
|
| 2. You build for engagement, people use the thing, but what
| impact did it drive?
|
| This article is great, but it's unbalanced potentially. I think
| directionally, some average in the middle is likely the way - so
| why not both? And while we're add it, add in other indicators.
| Success metrics that balance all of that, plus just a vibe metric
| for good measure.
| efitz wrote:
| The article dances around a topic that is very relevant across
| technology companies- toxic metrics.
|
| I worked for many years at a company that prides itself on being
| metrics driven. To the point that they celebrated having many
| significant figures after the decimal point when measuring length
| of negative events in seconds.
|
| The good side of metrics driven culture is that having a metric
| gives you a concrete goal.
|
| The downside is that really good metrics are often very hard. By
| really good, I mean metrics that measure the desired outcome.
|
| It's much easier to measure, for example, how many operation X
| your service performed, rather than the value the service
| delivered to your business or your customers. So we assume that
| each operation X contributes a uniform, positive quantum to those
| outcomes and we count X's.
| godelski wrote:
| I don't think they dance around the topic so much as outright
| state that it is bad. They mostly talk about engagement because
| it is the primary metric but even state that we should not
| value what we measure but measure what we value. Which it is
| harder to do that, but hey, you got billions of dollars to work
| with, it seems like the resources are there.
| hinkley wrote:
| Toxic metrics are a subset of perverse incentives. Necessary,
| but not sufficient to stay out of trouble.
| JohnFen wrote:
| The software industry has made me very aware that being metric-
| driven can be a terrible thing as easily as a good thing.
| Software has been getting substantially worse over time (in my
| opinion) in large part because of the use of metrics.
| FalconSensei wrote:
| Goodhart's law basically also.
|
| I've seen ppl on Twitter posting 'unpopular opinions' that
| they don't even agree, just to get replies. And then admit on
| the conversation that they don't agree with that.
| yieldcrv wrote:
| It's because investors aren't discerning either
|
| They chose to pay higher prices for shares, divorced from actual
| revenue
|
| The companies, in-turn, found non revenue metrics to show quarter
| over quarter
|
| and thousands of startups copied the model
|
| Just don't rely on the A/B test, be aware of it, but don't rely
| on it. Put the human control back into the process. Users aren't
| staying on your platform 5 seconds longer after you hid the
| escape hatch because they love it, they are frustrated and lost!
| The A/B test doesn't tell you why the engagement is longer, only
| that it is. It takes an empathic human to say "okay, B got us
| that result but not for the reason we want"
| lucideer wrote:
| Nicely written post but can't help noticing the elephant in the
| room being tiptoed around, which really makes me question the
| sincerity of the author.
|
| The monsters need energy to power their civilisation. Why do
| social media companies measure engagement? What are they
| extracting? There is not one single mention of the "p word" in
| the article.
|
| The last two sections are delivered with such heavy doses of
| naivety that this ends up just coming across as Orwellian in
| tone.
|
| > _"We should measure what we value, not value what we measure"_
|
| Who is "we"? Is "we" the shareholders? What do "they" value do
| you think?
| [deleted]
| RicoElectrico wrote:
| So please tell me, what should I optimize for when moderating a
| local OpenStreetMap community. I personally feel it's crucial
| people internalize that's a project with a bottom-up
| organization. A "civil society", if you will. Alas, despite
| 200-250 daily active mappers in Poland [1] only a fraction ever
| posts on the forum, even if just to ask something. Meeting IRL is
| also futile except 2-3 largest cities. We have a forum, Facebook
| group and Discord.
|
| Whatever I'd come up with, I'd like it to be a self-healing
| organism that would continue to thrive even if some important
| members become inactive. And to feel it's not a Sisyphean task.
|
| [1] https://osmstats.neis-one.org/?item=countries&country=Poland
| scoofy wrote:
| I'm confused. I see OSM as a huge success. Firms like Mapbox
| and Strava have an incentive to update that wiki with you. The
| maps are amazing. Any local sub-community will be challenging
| to organize, but maybe the metric should be making it fun for
| you, and if you're having a good time others will join.
|
| I am also starting a wiki, a golf course wiki:
| https://golfcourse.wiki
|
| It's still very early for me, but I see myself in this post.
| It's going to be extremely slow going, as the main consumers of
| the content created have no desire to contribute, but since
| golf courses are mostly static, as long as i plug away, adding
| a little bit all the time, the site will be a success. It costs
| almost nothing to operate. Generating traffic is challenging,
| and i've been trying to find more people willing to contribute,
| but again, I see my product as best placed if i do exactly the
| opposite of what most of the golf course info sites are doing.
| Don't base my success on engagement, no aggregation, no overly-
| prodding to contribute. Just one day at a time, with the
| assumption that the task is endless anyway, having people share
| their passion on their own time.
|
| If i can make that profitable at all, and it shouldn't be too
| difficult as the site grows, i'll see it as a total win.
| tikkun wrote:
| People seemed to like the other summary I made.
|
| So I made a summary of this article too, using GPT-3.
|
| Here it is:
|
| The article discusses how social media platforms originally
| focused on maximizing engagement, without understanding that
| negative posts are more likely to be engaged with than positive
| posts. As a result, these platforms have become full of toxic
| people and ideas. The author argues that engagement is not a
| valuable metric, and that community managers should instead focus
| on metrics that are related to the things that people join
| communities for, like support, connection, and relationships.
| torbTurret wrote:
| You preface all of your posts with, "people seemed to like..."
|
| No. HN does not need the spam. Save that for Reddit.
|
| Ironically given the topic, it's posters like you that ruin the
| quality of online communities and make engagement hard to
| measure.
| tough wrote:
| damn yeah that is good, should build a bot we can summon by
| mentioning or something for the lazy: like @tldrbot
| scanr wrote:
| I wonder if measuring engagement and optimising for engagement
| should be separated. I'm glad that there is lots of engagement on
| HN but I'm also glad that HN chooses not to implement a number of
| engagement optimisation features as well as downranking topics
| that result in high engagement flame wars.
| MoscMob wrote:
| JAA1337 wrote:
| Couldn't agree more.
|
| I believe this falls into the 'Results Oriented' mindset of
| delivering value. This means figuring out your desired outcome
| ahead of time, then measure it over time. In addition, overtime,
| you may determine that your desired outcome has changed and you
| need to measure differently. Yes, iterating is not just for
| software.
|
| IMHO pure quantitative metrics are of little value for customer
| experience. I believe metrics should be a little squishy and
| subjective. The value in the subjectivity is that it spurns
| conversation and actual thought instead of simple counting.
| forbiddenvoid wrote:
| I like the premise of the post. I think it paints a rather overly
| idealistic view of social networks, who exist to make money
| first, not to create value (if they could make money without
| creating value, they certainly would do that).
| mikejulietbravo wrote:
| engagement takes many forms. If you value people connecting with
| each other, measure how many messages they're sending or
| connections they're making.
|
| Saying stop measuring community engagement is a good clickbait
| headline, but it's absurd advice for community managers that even
| your own article doesn't agree with.
___________________________________________________________________
(page generated 2022-09-08 23:01 UTC)