[HN Gopher] Deepfakes, can you spot them?
___________________________________________________________________
Deepfakes, can you spot them?
Author : T-A
Score : 41 points
Date : 2022-03-08 20:19 UTC (2 hours ago)
(HTM) web link (detectfakes.media.mit.edu)
(TXT) w3m dump (detectfakes.media.mit.edu)
| aidanhs wrote:
| Honestly, I struggled with this overall.
|
| For context: I'm mostly unaware of actions of US presidents
| (beyond the broad sweeps based on left vs right) and I've at most
| listened to Trump and Biden maybe a couple of times for a total
| of ~1min in the last few years - just doing this exercise has
| probably at least doubled my exposure.
|
| I found the text and voice mostly impossible unless the content
| clued me in. Video was a little easier because I know to look for
| teeth, video with sound was fairly easy.
|
| Is there a generic trick for recognising faked voices without
| really knowing the original (similar to looking for teeth on
| videos)?
| bobsmooth wrote:
| For the speech clips I thought the disjointedness of the words
| made it obvious. That being said, Trump has a very recognizable
| way of talking so his clips were really easy to sort.
| mywacaday wrote:
| Why doesn't the survey give you a score after completion? Very
| frustrating!
| leto_ii wrote:
| Yes! Or even better, a score distribution together with your
| percentile.
| smdz wrote:
| The audio gives away. The fake audio is either of too monotonic
| or rythmic like singing a song
| dogman144 wrote:
| Been wondering about this in context of UKR and the huge role of
| open source intel via mobile phone videos in it. For
| understandable reasons it's not part of the discussion right now
| but there might be interesting retrospectives about this
| available down the line.
| junon wrote:
| These are super easy compared to some I've seen.
|
| Anyway, THANK YOU site developers for properly handling
| resumption of progress and not abusing the history API. Greatly
| appreciated and unfortunately very rare for these sorts of sites.
| BugsJustFindMe wrote:
| The quality of the fakes is atrocious. Their audio clips don't
| even sound like deep fakes. They just sound like really terrible
| amateur impersonators.
| pqdbr wrote:
| I only did 13, and got 100% right.
| alpb wrote:
| Mildly surprised the quality of this project coming out of MIT.
| Not only the UI was quite hard to use, I could not get beyond
| 10 samples. They were annoyingly obvious. Why would they not
| choose to not to make this as tricky as possible?
| squeaky-clean wrote:
| I only did 12 of them before exiting but got 100% of them. I
| watched them on my phone without turning it landscape to simulate
| how I'd normally watch a brief news clip like this.
|
| They're still not particularly convincing, especially the audio
| ones sound very fake.
|
| But I think these sorts of tests are flawed. The fact that it's a
| test primes you to look for these details in a way you might not
| if these clips were shared on Twitter/Reddit/etc. The deepfakes
| are still pretty obviously fake, so I don't think I'd fall for
| one "in the wild". But I also see this style of test used for
| things like "can you hear the difference between these 2 audio
| compression formats" or "does this virtual guitar amp sound just
| like the real thing", and that's just not how you consume real
| media. For example I'd probably never know Meshuggah switched to
| entirely digital guitar amps years ago if it weren't for them
| saying it in interviews.
|
| When deepfakes one day do get as close to the real thing as
| digital guitar amps have gotten to real amps, I doubt I'd notice
| a random r/worldnews post is faked, even if I could still
| reliably pass an A/B test.
| sebow wrote:
| The video look worse than "professional fakes" long even before
| AI.Text is the hardest for people who don't follow those specific
| people(though it is still easy to spot the fake), and audio is
| very easy aswell.
|
| Not boasting, just reality.I've seen some pretty astounding
| deepfakes, these are not part of those.
| alamortsubite wrote:
| Given how easy the fakes are to spot, the delay the devs put on
| activating the submit button is frustrating.
| NewEntryHN wrote:
| The best deepfake I've seen is the Tom Cruise one
| https://www.youtube.com/watch?v=iyiOVUbsPcM
| hombre_fatal wrote:
| Whoever makes those is an artist. Those impressions alone are
| great.
| seg_lol wrote:
| We should send in Sassy Justice to investigate right now!
|
| https://www.youtube.com/c/SassyJustice/featured
| RosanaAnaDana wrote:
| There is a smoothness to the fake audio and video with audio
| clips that makes them very obvious.
| arez wrote:
| This is too easy, I struggled a bit with no sound but video to
| detect real or fake because the lighting can be a bit weird but
| for the voices you can hear it in a split second that they're
| fake
| Arainach wrote:
| Agreed. The video and text fakes are getting better, but the
| audio (at least here) isn't close. It's not even an uncanny
| valley, it's worse than an SNL parody.
|
| In cases without audio, watching the lips for sync was also a
| reliable giveaway.
| bombcar wrote:
| It's almost so bad it's trying to convince people they're
| better at detecting deepfakes than they actually are ...
| boomboomsubban wrote:
| I only did about half of the samples, but things became
| harder further in. I'm not sure if they were better done or
| I was just becoming bored and losing focus though.
| version_five wrote:
| I didn't do the test. My first thought was that it's the wrong
| question. A "trusted sources" model for believing stuff can
| always be gamed. There should be more focus on independent
| confirmation, and general sanity checking, rather than caring
| about whether a video is fake or not, especially if one is going
| to take some action based on the content.
| cf141q5325 wrote:
| Why the downvote? I think he is making a solid point. Just
| trusting your senses will become problematic fast.
| leto_ii wrote:
| This was an unsatisfying experience. The visual and audio fakes
| are easy to spot, while the texts are short enough and correct
| enough that you don't have enough to go with in making a
| technical decision. I did however get them right by trying to
| figure out if the statement made sense in the context of the
| respective politician's general stance on the issue discussed.
|
| Update:
|
| Come to think of it, IRL text deep fakes don't even make sense.
| You can make up whatever quotes you like, you don't need GPT3 to
| do it for you.
| echelon wrote:
| > Come to think of it, IRL text deep fakes don't even make
| sense. You can make up whatever quotes you like, you don't need
| GPT3 to do it for you.
|
| Exactly. John Oliver pointed out a few weeks ago that the
| Russian media was using stock footage of other nation's
| military actions (Finland?) for propaganda. No need for deep
| fakes. The lowest effort lie can still work.
|
| Deep fake tech is the new photoshop. It's mostly used for good
| (art, memes, jokes with friends, Disney films).
|
| You can imagine a future where deep fakes are utterly
| indistinguishable from real life, but I think that people will
| learn to recalibrate themselves against the new technologies.
| It may even cause us to think more critically: people
| frequently chime in, "how is this not photoshopped?" on social
| media. They'll call out videos of explosions and physics as
| "looking fake". They'll have the same response to purported
| deep faked videos.
|
| All the same, I'm glad and appreciate that academic and policy
| folks are taking a close look at the technology. We need
| frameworks, ethics, and institutional familiarity.
| [deleted]
| Traster wrote:
| Honestly, I think this level of "fakery" is a little... facile.
| Firstly, practically no one judges individual clips, they're
| judging based on who is bringing it to them and who they trust.
| Secondly, Donald Trump could say the most dispicable things and
| still get elected. People seem to forget that Trump got caught on
| a hot mic admitting to sexually assaulting women, and that he got
| elected _after_ that.
| tiborsaas wrote:
| This was very easy. I skipped the text based ones, I can't keep
| up with the random things politicians say :)
| Miner49er wrote:
| I'm pretty certain that in at least one of these Trump is
| standing in front of a green screen, I put fabricated, but I
| guess that was wrong to do because that wasn't fabricated with
| AI? Either way this said it was real, which yeah it was, but the
| setting was faked. Idk.
| radford-neal wrote:
| The test seems flawed. I got the first one (after the attention
| check) wrong - I said it was fake, but it was real. I thought it
| was fake because Trump's hair was clearly very smeared out, not
| at all realistic. But it seems that this must have been an
| artifact of heavy video compression. I don't see how this is a
| meaningful test of anything.
| eropple wrote:
| Most social video on the internet is over compressed junk. I
| was surprised when they used clips that _weren 't_.
| radford-neal wrote:
| Yeah, but how is the test taker supposed to know whether to
| take the smearing as indicating it's fake or not?
|
| Better would be to let them see 10 videos all at once, half
| of which are fake, and ask them to divide into a fake set of
| 5 and a real set of 5, after looking at all of them as many
| times as they like. Asking "fake or real" when there is no
| basis to tell whether flaws should be taken as indicating
| "fake" or just attributed to compression seems meaningless.
|
| Or tell people what aspects of fakeness they're trying to
| assess - eg, forget about video artifacts, just pay attention
| to the audio.
|
| Using clips of Trump and Biden is also a bad idea. They ask
| you to say if you've seen one before, but aren't many people
| going to have seen one, but not clearly remember that, and
| then be influenced to think it's real by sub-conscious
| recognition?
|
| Why not present pairs of videos of the same non-famous
| person, one fake one real, both presented with the same
| amount of compression, and ask one to chose which is the real
| one? Using many different people, of course - why would you
| introduce doubt about the generality of your results by using
| only two people?
|
| Of course, in practice people may be less able to recognize
| fakes when video quality is poor, which would be useful to
| know, but I think one would need to investigate that issue
| separately, not in combination with other reasons that fakes
| might or might not be recognizable.
| qw3rty01 wrote:
| I really wish they took actual things the individual said in the
| past, but used a deepfake for them actually saying it. Most of
| the deepfakes were easy to spot, but a couple weren't clear at
| first that they were fake, only to be given away by the content
| of their speech.
| bufferoverflow wrote:
| The point of a deep fake is usually for the fake to say
| something new and outrageous.
| squeaky-clean wrote:
| It doesn't necessarily have to be. For example what if you
| modified a clip of a political candidate asking for funding
| and only modified the name of the organization/website their
| supporters should go to in order to donate.
| orblivion wrote:
| You know, I sort of don't want to help them figure out which ones
| are more believable.
| fancymcpoopoo wrote:
| maybe the research is really about identifying you as a liar
| kerblang wrote:
| My favorite deep fake is still Sassy Justice
| https://www.youtube.com/watch?v=9WfZuNceFDM
|
| Ha it keeps on giving, note the picture on wall at
| https://youtu.be/9WfZuNceFDM
| marstall wrote:
| any kind of information can contain falsehoods - and every form
| of discourse has its biases. so consider the source - and
| consider multiple sources!
| cf141q5325 wrote:
| For better audio check the channel Vocal Synthesis
|
| >Barack Obama reads the Navy Seals Copypasta (Speech Synthesis)
| https://www.youtube.com/watch?v=-_MZI2YFWgI
|
| Or here him reading Trumps inauguration speech
| https://www.youtube.com/watch?v=ChzEdz7aVVs
|
| All based on
| https://ai.googleblog.com/2017/12/tacotron-2-generating-huma...
| bloaf wrote:
| Yes.
|
| The video fakes are always blurry around the mouth and gestures
| don't match the words.
|
| The audio fakes are too "clean" (too defined breaks between words
| and no background noise) and sound like caricatures.
|
| Text is virtually impossible because politicians say "fake" stuff
| all the time (e.g. speeches someone else wrote) so you can't rely
| on uncharacteristic language.
| bobsmooth wrote:
| I found the grammar of the fake text just off enough to pick
| them out consistently.
| ComradePhil wrote:
| Too easy.
|
| Also, the fake ones with audio or text from Trump are way too
| easy for anyone who has seen Trump over the years to guess (as
| opposed to those who know Trump from "comedy news shows" from the
| last few years)... because Trump wouldn't say some of those
| things (anti-gay marriage stuff, for example).
|
| Biden is a bit more difficult to guess like that because he could
| have said anything which he thought would be popular at a given
| point throughout his career... but then the technology is too bad
| to actually be convincing enough.
___________________________________________________________________
(page generated 2022-03-08 23:01 UTC)