[HN Gopher] Why do we all fall for AI-generated language?
___________________________________________________________________
Why do we all fall for AI-generated language?
Author : azhenley
Score : 73 points
Date : 2022-06-18 15:28 UTC (7 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| FatalLogic wrote:
| Bceuase we asmsume poeple are tyring to mkae snense
| ummonk wrote:
| This is why it's important that the Turing test is adversarial
| and done with a human control. I see way too many people (even
| some smart engineers) reading AI-generated writing that looks
| human and declaring that the AI passed the Turing test.
| nonrandomstring wrote:
| I had to mark about 100 end of term essays written by Indian
| students for a British university. My unwritten instructions are
| not to take language into account much. I must attend mainly to
| the technical content.
|
| At least half were written in what I took to be an authentic
| voice but with such bad grammar and spelling as to render them
| barely readable. Some had clearly been mangled in a laundromat of
| Google translate from Hinglish via Mongolian and Swahili. They
| contained bizarre phrases and comical statements. Many more were
| obviously written by some kind of generator and fudged until they
| read well enough.
|
| Since the student handbook states the threshold for academic
| "plagiarism" is above 20 percent perhaps unsurprisingly the
| Turnitin (an awful tool) score for almost every essays was just
| below 20 percent. An interesting clustering!
|
| Students who cheat have a formidable array of tools now, not just
| GPT but automatic re-writers and scripts to test against Turnitin
| until it passes.
|
| Add to this problem that my time for marking is not paid extra,
| is squeezed tighter every semester, and that students are given
| endless concessions to boost their "experience". The handbook
| also says that if they fail, no worries, they get to try again,
| and again, and again... and I am sure if I actually stuck to my
| guns and failed every single student I'd be fired.
|
| As I wrote in the Times last year, I think the technological arms
| race against GPT (and the economic conditions that mean it's
| used) cannot be won with the time and resources available to
| ordinary human teachers.
| [deleted]
| User23 wrote:
| > As I wrote in the Times last year, I think the technological
| arms race against GPT (and the economic conditions that mean
| it's used) cannot be won with the time and resources available
| to ordinary human teachers
|
| Based on the rest of your post, there appears to be a stronger
| case that your students are setting a rather low bar for GPT to
| stumble over. It's unfortunate that there are so many cultures
| where widespread cheating is condoned, if not outright
| encouraged. They may be able to fool their teachers, but how
| much comfort will that be when the bridges are collapsing, the
| pipelines are exploding, the wind turbines are breaking apart,
| and all the other activities that ultimately report to reality
| and not some human superior who can be bluffed become
| impossible to continue?
| nonrandomstring wrote:
| > It's unfortunate that there are so many cultures where
| widespread cheating is condoned, if not outright encouraged.
| They may be able to fool their teachers, but how much comfort
| will that be when the bridges are collapsing, the pipelines
| are exploding, the wind turbines are breaking apart, and all
| the other activities that ultimately report to reality and
| not some human superior who can be bluffed become impossible
| to continue?
|
| You're so right. But let me add some other feelings, so as
| not to sound like a racist or that British universities are
| some "great white hope" to overseas students. This had little
| to do with them being Indian. It's a generational thing. In
| all cultures we teach young people to game systems. Right
| from the get go they learn that if they can buy powerful
| tools, systems and access then that's fair game. They're just
| doing what they've been rewarded for their whole lives and
| want to make a better life. To them it's not cheating. I am
| the anachronistic throwback here I think.
| schroeding wrote:
| > My unwritten instructions are not to take language into
| account much. I must attend mainly to the technical content.
|
| Interesting, at my non-English, run-off-the-mill university
| there were modules / seminars in CompSci where large amounts of
| language errors in essays (even if written in English, a non-
| native language for the majority of staff and students) could
| ruin the grade. ^^
| gs17 wrote:
| My undergrad even had a "banned error list", which would get
| you kicked down half a grade on any paper.
| matkoniecz wrote:
| > cannot be won with the time and resources available to
| ordinary human teachers
|
| By your description it clearly appears that whoever manages
| your company[1] is not actually interested in detecting
| cheating.
|
| What you describe could be easily combated by giving teachers
| ability to fail blatant cheaters.
|
| [1]At this point it is hard to pretend that it is university
| nonrandomstring wrote:
| I agree with every word you say.
| matkoniecz wrote:
| In such case it seems that GPT-3 main effect is to make
| eroding standards easier. As while rules are not changed,
| cheaters can put less work to pass it by cheating than
| before.
|
| So standards can be lowered while pretending (for now) that
| it has not happened.
| milkey_mouse wrote:
| > automatic re-writers and scripts to test against Turnitin
| until it passes
|
| Like an ad-hoc GAN where Turnitin is the discriminator.
| Interesting.
| RicoElectrico wrote:
| What major was it?
| nonrandomstring wrote:
| I can't say. That would identify the students and that's
| unfair.
|
| But, a technical subject that could be assessed in other,
| better ways [1], and for which written essays are rather easy
| to template and do keyword bingo to get a bare pass.
|
| [1] Making the professor read 100 essays is a cheap option.
| RicoElectrico wrote:
| Yeah, in my EE major I never really had to write an essay.
| I have no idea why English/USA universities are fixated on
| them (or so it seems based on comments from the internet).
| Lab/project reports, seminar talks - I did these instead.
| User23 wrote:
| Humans have a powerful tendency to ascribe human characteristics
| to inanimate objects, including computers. It's a kind of variant
| of the Pathetic Fallacy[1], except for artifacts instead of
| natural objects. The intelligence of an artificial intelligence
| is as real as the characters in our dreams. It's a construct of
| our own consciousnesses. That doesn't mean it should be
| discounted though. Our consciousnesses can do a lot and finding
| artificial ways to stimulate them is powerful to say the least.
|
| [1] https://en.wikipedia.org/wiki/Pathetic_fallacy
| mproud wrote:
| A tweet no doubt generated by an AI.
| krisoft wrote:
| It is such a weird question. People fall for AI-generated
| language because the goal of the people who made the generator
| was to create language like a human would.
|
| Do people wonder why scissors cut paper? Because that is what
| they were made for!
|
| If the AI wouldn't fool the humans the researchers would be
| honing it more. Same way if we couldn't make paper cutting
| scissors there would be people trying to make one.
|
| Am I missing the point here?
| readams wrote:
| I think you're missing the point. The question could be
| possibly rephrased as "what are the tricks that the AI uses to
| fool humans?" The paper goes on to identify some of the
| specific tricks that the AIs appear to use. A related question
| might be "why are we fooled by such simple tricks?"
| FFRefresh wrote:
| If you follow that question through and think through the
| implications, it paints potentially a dark future for digital
| communication.
|
| The 'tricks' AI uses are also 'tricks' that humans use in
| everyday conversation, we just don't call them 'tricks' when
| humans are involved. If we start assuming that first-person
| pronoun usage, mentions of family, etc. are potential signals
| of AI, then I don't see how we don't end up in a state where
| increased dehumanization occurs.
| dustingetz wrote:
| why do we assume that humans are smart? answer: ego;
| corollary: we are not
| lofatdairy wrote:
| The reason why this is interesting isn't because humans are
| smart, but the assumption that human fallibility is both
| predictable and worth investigating (a supposition I'm
| inclined to agree with).
|
| Even if humans are easy to fool (though the fact that we're
| just now achieving this on a generalized scale after at
| least 80 years of theorizing seems to contradict this),
| humans being fooled can result in significant enough
| impacts where we should still investigate the degree to
| which they can be tricked, and what methods of discerning
| are available to us.
| MathYouF wrote:
| We seem obviously to be the smartest living things in the
| universe we know of.
|
| We're also approximately the dumbest possible things that
| could construct the society and technology we have (if we
| weren't, we would've done it sooner).
|
| I agree that some humility of our own intellect as primates
| would do us a lot of good in the coming age of AI.
| jacquesm wrote:
| > We seem obviously to be the smartest living things in
| the universe we know of.
|
| Humblest too.
| lalopalota wrote:
| s/We /A few of us/
| dustingetz wrote:
| civilization was not constructed, it evolved, like our
| biological bodies
| ben_w wrote:
| I'd say it gets more constructed and less evolved the
| further up the hierarchy of civilisation we get. A
| merchant of 1320 had relatively few laws to contend with,
| a corner shop in 2020 could not function without many
| layers of engineered rules for the people who provide
| them with the services they are themselves required to
| use due to other engineered rules for the benefit of
| their customers.
| MathYouF wrote:
| This makes me wonder what laws and tedium of
| administration merchants of the 1320's would complain
| about. I'm sure they still had a few depending on the
| region, and maybe more severe possible outcomes (highway
| robbery, unlawful arrest because of the influence of a
| rival merchant, arbitrary taxation and tariffs, etc.).
| ben_w wrote:
| Yup. I think also languages were much more variable, and
| doing accounting in Roman numerals was so hard they did
| it twice and averaged the answers, and contracts and tax
| receipts were done by carving marks in sticks (hence,
| apparently, the etymology of "stocks").
| dalbasal wrote:
| Taking your point, the title should probably be " _How we
| fall for AI-generated language._ "
| synu wrote:
| Researching semi-obvious things like what elements of AI-
| generated text humans mistake for being human-generated is part
| of the process of how people working on AIs work towards better
| generation. You're just seeing how the sausage gets made.
| mattnewton wrote:
| Maybe a bit, they aren't just asking why scissors cut paper,
| but also why we landed on that design. What about it makes it
| ergonomic to hold and efficient, and more to the point, why
| does being sharp cut it?
|
| In the language model case, why can we model language this way
| so effectively and why does it follow these statistical
| patterns? It turns out that maybe a major reason has something
| to do with self description.
|
| This is also a more interesting question because we understand
| language less than we understand cutting paper, and also
| because the process humans used to design large language models
| is more indirect and alien than traditional industrial design.
|
| My takeaway was that people reading language modeling a person
| writing about themselves imagine it was written by a person,
| more than text written by people not about themselves. That's
| an interesting trick! Describing human experiences in language
| makes people attribute the language to a human right now. Maybe
| this will change after a generation of people knowing about
| this, but that seems important to think about.
| dvt wrote:
| This is actually a pretty good comment. The scissor analogy,
| while a bit on-the-nose, is very accurate. Maybe a better
| example would be: why is our body fooled by artificial hearts?
| Simply because it was built in such a way that it simulates a
| real heart pretty well.
|
| Similarly, these models are built in such a way that they
| simulate real-life conversations pretty well. There's nothing
| really more to it. In my view, this phenomenon has nothing to
| do with intelligence or how smart we are, or whatever.
| ineedasername wrote:
| Lot's of people don't write all that well. A little disorganized,
| awkward phrasing, run on sentences. If AI does a better job than
| even 10% of the population then of course there's going to be a
| sizeable amount of miscategorizing when asking humans to classify
| writing as computer or human generated.
|
| You could probably use a corpus of purely human writing and have
| people attribute a decent portion to computer generated.
|
| Asking why AI writing can fool humans is a bit like asking why a
| computer is better at many tasks often performed by humans.
| akagusu wrote:
| My question is when government will obligate companies to
| identify and label AI generated content so people can distinguish
| it from human generated content?
| legrande wrote:
| We fall for it because although language is a powerful tool, it's
| incredibly bad at conveying nuance, context, and describing
| phenomenons present in nature. Poetry comes close, but still
| doesn't hit the spot, and leaves out so much detail, no matter
| how well written or verbose in its descriptions. Our own mind has
| to fill in the blanks of a well written description. Language
| also can't express the ineffable or the divine. It can hint at
| it, but it won't transmit the phenomenon correctly into another
| mind.
| schroeding wrote:
| From the pre-print paper[1]:
|
| > ... we believe the next generation of language models must be
| designed not to undermine human intuition
|
| Right, but isn't a major reason why we build these huge language
| models to replace _actual_ humans in e.g. Level 1 support with
| chatbots? Almost all chatbots I used in the past (and most were
| not even ML based, someone programmed this in) were weirdly
| personal and tried to be non-robotic, with jokes and human-like
| reactions to inputs like "Thanks!".
|
| Taking a look at some projects that used GPT-3[2], many try to
| imitate humans. For some, like Replika.ai, the whole "being
| human" thing is their entire schtick.
|
| There is obviously a market for text completion AIs that imitate
| humans, so it's doubtful that we'll get this toothpaste back into
| the tube, IMO.
|
| [1] https://arxiv.org/ftp/arxiv/papers/2206/2206.07271.pdf [2]
| https://medium.com/letavc/apps-and-startups-powered-by-gpt-3...
| (caution, 2020)
| trqwerty wrote:
| Most of these AI vs. human tests take very poor or weird looking
| human creations vs. the best AI creations. No wonder that one
| gets the desired results.
| tlhunter wrote:
| I wonder if this spells out the downfall of social media? Or at
| least non-verified users on social media? As time goes on we'll
| have larger and larger armies of bots spewing automated political
| rhetoric everywhere.
| jokethrowaway wrote:
| I think I'll start running my copy through gpt3 to make it more
| human
| matkoniecz wrote:
| Maybe it would be interesting to read if presented as an article,
| but the Twitter thread is unreadable.
| pharrington wrote:
| Their preprint is at https://arxiv.org/abs/2206.07271
| legrande wrote:
| https://nitter.net/maurice_jks/status/1537814372462039043
| matkoniecz wrote:
| oh, that is much better - I survived long enough to notice
| link to their paper.
|
| Thanks!
| [deleted]
| hourago wrote:
| I totally agree. I understand that people do not blog anymore.
| But Twitter is not a good place for lengthy texts, by design!
| hourago wrote:
| Real people does not always speak good.
|
| And everybody has blinds pots, topics that are interesting but
| that we have little knowledge.
|
| > We show that human judgments of AI-generated language are
| handicapped by intuitive but flawed heuristics such as
| associating first-person pronouns, authentic words, or family
| topics with humanity.
|
| And that is a good one. Because we try to understand the others
| when they does not make fully sense.
| andyjohnson0 wrote:
| Because secretly we know that our own consciousness is just a
| model watching its own outputs too.
| midjji wrote:
| Because its not responsive/dialogue.
| secondcoming wrote:
| Is it the case for all languages, or just English?
|
| Native English speakers are typically used to interacting with
| people whose native language isn't English and so easily tolerate
| errors or odd word choice.
___________________________________________________________________
(page generated 2022-06-18 23:00 UTC)