[HN Gopher] Voice clone of Anthony Bourdain prompts synthetic me... ___________________________________________________________________ Voice clone of Anthony Bourdain prompts synthetic media ethics questions Author : andreyk Score : 132 points Date : 2021-07-16 18:20 UTC (4 hours ago) (HTM) web link (techpolicy.press) (TXT) w3m dump (techpolicy.press) | somehnacct3757 wrote: | They probably knew some people wouldn't like it, which is exactly | why they used it, because look at all the free publicity they're | getting now. | alexilliamson wrote: | This statement perfectly describes the last decade in America. | hellbannedguy wrote: | Out of all the celebrities, Bourdain would have viscerally | despised this. | | He hated fake phoney people, or food. | | He hated when his production crew would stovepipe his bits. | | He hated a scene in Greese (I believe) where crew went out and | bought squid, and threw them into the water to made the shoot | better. | | I really liked Bourdain. He was one of the few celebrities that | didn't seem to change with fame, or money. | | I watched him for years, and knew he was unhappy, but never | thought suicide unhappy. | nailer wrote: | The bone luge thing was completely made up. The restaurant | didn't do bone luges. It's mentioned explicitly in World | Travel. | | I have no problem with a fake voice reading Bourdain's own | words. | ceocoder wrote: | Just quick context on the squid bit: it wasn't Tony's crew | (zero point zero productions), it was the combination of local | fixers/boat folks who wanted to make it look like they caught | fish. | holler wrote: | Funny enough, him describing the ploy while sort of mocking | it as it happened led to an enjoyable moment in that episode | (imo). It showcased his humility, and ability to make the | most out of a situation while not taking himself too | seriously. | distrill wrote: | That's one of my favorite scenes. He is so angry drunk and | it's his birthday and you can tell he doesn't like the chef | at all (the chef is the same guy who took them out | "fishing" earlier in the day). | ceocoder wrote: | Thing is, more I rewatch his shows, all of his comments | about depression and anger have taken a really somber tone | now given what we know. I wish someone was there with him | to help him through through that time. | | His death really affected me in a deep way I didn't expect. | I miss him so much. | FireBeyond wrote: | One of the other big scenes like that is up-river in SE | Asia, talking about "how he wished that he could say it | would be difficult [to kill a pig with a spear], but that | time and distance have hardened the person he once was". | [deleted] | wavefunction wrote: | I enjoyed the running gag (based on many episodes) of | Bourdain going on a local fishing/hunting trip and failing to | catch anything, and then having to scramble to some other | local place and make up for the lack of a catch which some | times were some of the highlights of the episode. And then on | rare occasions when they did catch their prey, it made for a | nice juxtaposition. | porknubbins wrote: | That was the Sicily episode- Bourdain wanted to show the ugly, | scammy underbelly of Sicily I think. This kind of thing is | there in a lot of the Mediterranean, but it unfairly makes all | of Sicily look bad. I had a bad impression of the whole island | until I learned more about it years later. Sicily is mostly | nice with a few pockets of unsavoriness like anywhere, not a | mafia island. In my opinion too many episodes are like that- | projection of a certain fantasy about a place while ostensibly | engaging with the real authentic experience. | jacobkg wrote: | Agree 100%. Also just watched this episode last night, it was | Sicily (Season 2) | basisword wrote: | Important to point out it wasn't the crew faking the scene. | The guy that took him fishing had a friend throwing squid in, | unashamedly. They showed the guy on camera. There was no | attempt at trickery from the people producing the show. | andy_ppp wrote: | As I understand it (I read this on Twitter so massive grain of | salt etc.) the family concented to this explicitly. Would you | feel better if it was a voice actor who could read Bourdain's | email in a perfect mimic? What about historical recreations, | would you object to computers being involved in, say, | recreating Abraham Lincoln's voice? Does the age of the subject | matter? | | I think the squeamishness about AI as it becomes more and more | capable will be interesting to define why we are feeling it. | The machine is going to be capable of (and consequently used to | do) these things whether you like it or not. | ineptech wrote: | Any of those are fine - with disclosure. It seems pretty | clear that in this case, they were less than transparent. | | > "If you watch the film, other than that line you mentioned, | you probably don't know what the other lines are that were | spoken by the A.I., and you're not going to know," Neville | told the reviewer, Helen Rosner. "We can have a documentary- | ethics panel about it later." | | ...or we can have it sooner, on Twitter, and you'll get | excoriated, and rightly so. I don't care if they train an AI | to imitate Bourdain rapping the third verse from _Modern | Major General_ , it's a free country, but you have to be | honest about it. You can't call yourself a documentarian and | get cutesy about the authenticity of the material. | briefcomment wrote: | His widow did not provide consent [1]. | | [1] https://twitter.com/OttaviaBourdain/status/14158894550057 | 164... | andy_ppp wrote: | Cool, great to get that clarified but also good to see I | didn't invent the fact they _claimed_ to have asked. | virtue3 wrote: | Historical recreations are fine. | | Creating a digital voice model of abraham lincoln and then | using it for ??? is not. | | All of this is being done for profit means (the orville | redenbacher, fast furious, star wars). | | Just because we can doesn't mean we shouldn't stop and think | if we should. | andy_ppp wrote: | On which timescale does something become historical? Are | the Beatles historical, to you maybe not but possibly to | your children? Where is the line you're drawing? | | If I'm honest humans have never done the moral thinking, we | largely just middle through and hope to get away with it. | tayo42 wrote: | Idk if historical is based on time. I think intention is | more important. if you recreate the Beatles purely for | profit then it seems unethical and disrespectful. if you | want to keep someone's voice for record keeping or | something educational that seems different | patrickthebold wrote: | Time doesn't matter as much as what is said. Fake Anthony | Bourdain voice reading something he wrote seems fine to | me, especially in the context of, say, a documentary. | | John Lennon's voice doing an ad for Vox amps seems in | poor taste. | two2two wrote: | After reading his books, I would agree and was hoping someone | had posted this. If anyone would vehemently oppose such | manipulation, it would be Anthony Bourdain. Is there a | disclaimer before this scene is shown? I'll choose not to watch | such a documentary created by those with skewed ethics. | AutumnCurtain wrote: | There does not seem to be any disclaimer regarding the | manipulation, and the OP article questions whether the | director would even have mentioned it had it not come up | tangentially in the course of this interview. | [deleted] | enriquto wrote: | what is the difference to drawing him on a fictional painting? or | edit a photo? | | People who complain about that sound a bit like those mythical | tribes that are afraid of photographs because they may rob their | soul. | | EDIT: i'm not sure that this is wrong, only confused | tqi wrote: | Is this different than getting a voice actor? | tptacek wrote: | I get why this is newsworthy, but I don't get why it's an ethical | problem. How is it any different from hiring a really good | Bourdain impersonator to read the email? Lots of celebrities have | pitch-perfect impersonators; is this a thing we worried about | when it wasn't AI doing the impersonation? | OhWellLol wrote: | Wait until you learn about using deep fakes to trigger Blackmail | Inflation | | https://m.youtube.com/watch?v=xlmhhh9HYqc | bellBivDinesh wrote: | Putting words in dead peoples mouths is unpopular. Shocking. | causality0 wrote: | I don't get why this is controversial. Documentaries constantly | have someone reading a written document while imitating the voice | of the author, and none of them come out and say "This is not the | actual voice of Abraham Lincoln". Why is having a computer | imitate someone's voice different from having a person do it? | reaperducer wrote: | Because Lincoln has been dead for 172 years, and lived before | audio recording, so no one who could fog a mirror would think | that it was his voice. | | Bourdain didn't die that long ago, and made many recordings. It | certainly is confusing. | usefulcat wrote: | It's pretty obvious why you will never hear any recordings of | Lincoln's voice. In comparison, it most certainly could be | Bourdain's voice here so it's not at all obvious that it isn't. | httpz wrote: | There are ethical concerns but I think we'll just learn to live | with it. Forging signatures are easy and have been around forever | but signatures are still used for very important purposes (even | electronically now!). It may take a while for the legal system to | catch up and using voice recordings as evidence may be tricker | than before. | | But on a more exciting front, I think synthetic voice can be made | like fonts. Celebrities and voice actors will be able to sell | their synthetic voice like how fonts are sold today. You'll be | able to change the voice of your Alexa, Siri to a voice you | downloaded from a marketplace. Netflix may even let you select | the narrator's voice for your favorite documentary. Hundreds of | years later, people may be watching a documentary in Morgan | Freeman's voice while having no idea that he was actually a | famous actor in the past. | BrandoElFollito wrote: | Signatures used for very important purposes are only | decoration. They must be used in conjunction with something | else (a notary, an independent exchange that confirms that the | signature is actually binding, etc.). | | The electronic signature is completely different: there is no | visible artefact anymore, but a process that seals a document | and (under appropriate legislation), certifies that the signer | is who he is. A visual object is sometimes added for aesthetic | purposes. | httpz wrote: | What you listed are just ways we've learned to live with the | forgeable nature of signatures. Now voice recordings (and | even videos) will have a similar fate in the future. | mcculley wrote: | This is an interesting problem. I thought and wrote a little | about this a few years ago: | https://enki.org/2017/03/02/historical-narrators/ | [deleted] | iandanforth wrote: | This recently got worse, they claimed to have the OK from his | widow. Turns out not so much ... | | https://twitter.com/OttaviaBourdain/status/14158894550057164... | minsc__and__boo wrote: | They really need a way for someone to trademark their voice. | yohannparis wrote: | As a viewer of the documentary, I will love that effect instead | of a bland voice-over. But a note should be added on the screen | that the voice is AI-generated. Like when they say a war video is | a reenactment. | pankajdoharey wrote: | It would be interesting to see if recreating songs is possible, | if so i would like to hear the voice of Jim morrison and Curt | Cobain. | rbalicki wrote: | https://www.rollingstone.com/music/music-features/nirvana- | ku... I don't remember if these songs had lyrics, but there | are AI generated songs in the style of Cobain and Jim | Morrison | rbalicki wrote: | Youtube link: https://www.youtube.com/watch?v=r1-B3M_KaRQ | | It does contain lyrics, but the singing is not AI | generated. | 3wolf wrote: | Would you settle for Ronald Reagan? | https://www.youtube.com/watch?v=mAZVp-n-5TM | adventured wrote: | https://www.youtube.com/watch?v=8gvsD11_z1k | caseyohara wrote: | AI-generated Nirvana song: | https://www.youtube.com/watch?v=GogY7RQFFus | genewitch wrote: | I have a machine with two GPUs and a frozen OS with a | tensorflow python app that clones voices, and I'd say the | quality passes if you run it through a phone bandpass filter. | | I've had an idea to use propellerhead recycle to chop the | output cloned voice into syllables, and then "play" the | chopped parts in rhythm, through autotune. | | The issue is you get Eifel 65 sounding autotune if your base | vocals are monotonic or way off key. The only way I can think | of fixing this is to use something like audacity's pitch | changer that doesn't affect the speed of the sample - rough | the lyrical tones in with audacity/recycle, then autotune it | where it needs to go. | | I'd like to say I'm too busy to get this workflow going, but | mostly I'm lazy and someone else will do it first - and | better - I can't improve the AI cloning software. | dkdbejwi383 wrote: | > The only way I can think of fixing this is to use | something like audacity's pitch changer that doesn't affect | the speed of the sample | | Check out Ableton Live's "Simpler" | ethbr0 wrote: | You should message @echelon on HN. ;) | | https://vo.codes/ | echelon wrote: | Oh hi! | | I've had a new version of this in the works for a few | months. It'll launch maybe this weekend? | | It supports user-trained data sets, which should be | pretty neat. | | Your approach sounds kind of like unit selection or | vocaloid. | xkeysc0re wrote: | You should watch The Doors movie by Oliver Stone starring Val | Kilmer. Much of the music, including vocals, was redone with | Val singing. It can be quite unsettling at times | ipsum2 wrote: | https://openai.com/blog/jukebox/ it is, but its not great | yet. | MrMetlHed wrote: | Why would it have to be a bland voice-over? It's an email he | sent to a friend. Have the friend read it. I imagine the friend | would become emotional. That's far more riveting than a | computer recreation, no? | jamestimmins wrote: | Agreed, this whole issue could be solved with a message before | the film runs about how the voiceover is created, and then a | "recreation" label during that scene. | godelski wrote: | I don't think so. There are still other ethical | considerations. Bourdain likely would not have liked this, as | others in the thread point out. So it is weird to honor | someone with a documentary but not honor their wishes | 37r7dyysy wrote: | What makes those considerations significant enough to be | worth the effort of evaluating though? It's too late to | stop the tech from existing, there's no individuals | actually being harmed, and even if you find a framework for | arguing harm that's compelling the end-game is just going | to be updated contracts which demand rights to use the | performer's likeness for these sorts of purposes. The | dialogue just seems like a lot of opining for the sake of | itself with a fashionable hint of 21st century doom-cult | luddism. I guess maybe the unions might have a reason to | worry but I don't have a lot of sympathy for unions | representing millionaires. | ghaff wrote: | Disclaimers help. And it's not like they were making up | words. But I still feel it was probably a poor creative | choice. | KONAir wrote: | I suppose a spoken line at the begining along the lines of | it is not that person, but a generated voice, would be the | best solution. | Grimm1 wrote: | I think with correct context and or the correct say so this would | be fine. But: | | I didn't really follow Mr.Bourdain at all, but given the clarity | from other people who did, and believe he would have hated this; | then it's not the act of making a voice clone that's unethical | it's doing it outside of the wishes, or what would be the | perceived wishes of that person. | | In other words this is unethical because had they gone to ask | permission from the estate/whoever could make that call now that | Mr.Bourdain is no longer with us they would have likely gotten a | no. | | It makes it particularly egregious that if these people were so | interested in making this clone of his voice that they likely | would have followed him closely enough to know he wouldn't have | agreed to this if he were alive. | | I guess you can boil my position down to without permission, | explicit or implied, it's unethical and not only that, legally | I'd put it under impersonation. | robotomir wrote: | After electronic distribution and Amazon's ability to redact the | books on your Kindle, this is the next important thing to notice. | jacquesm wrote: | I'm sure Kenny G. will have some way of making bank on this | technology. | | https://www.jazzguitar.com/features/kennyg.html | AlbertCory wrote: | I think Neville went by a common motto: "All publicity is good | publicity." | | Because he didn't need to do it. Using a voice actor would have | been completely standard and they would get a credit at the end, | which I'm sure they'd appreciate. | | And Bourdain definitely would _not_ have liked it. | RcouF1uZ4gsC wrote: | I actually think getting deep fake stuff to the public is the way | to go. Basically anyone can create a deep fake. It is far better | to get the public used to the idea of deep fakes so they can | develop some immunity to it. | inlikealamb wrote: | We've had Photoshop for 3 decades and airbrushing for even | longer and it's had far-reaching impacts across society... no | one is really immune to it because it isn't obvious and it has | arguably poisoned realistic ideals around body image. | | IMO we _need_ to mandate disclosures around deepfakes. It 's | impossible on a peer-to-peer level, but commercially it should | be clearly disclosed. | autoexec wrote: | We don't get immunity to photoshop in the "I can tell from | some of the pixels" sense, but we gain immunity in the sense | that we no longer accept every image as representing absolute | truth. Plenty of people today still think that video | manipulation more sophisticated than an instagram filter | takes a Hollywood budget or a lot of expertise. | | Once people get more experience seeing and creating deepfakes | on their own they'll be less trusting of random videos they | see in the future | jackpirate wrote: | _we gain immunity in the sense that we no longer accept | every image as representing absolute truth_ | | GPs whole point about body image is that, as a society, we | don't have that immunity. Basically no one has a healthy | ideal of what a good body image is because we never even | get to see them. | autoexec wrote: | The fact that media can so effectively influence our view | of an ideal body has little to do with our awareness | about photo manipulation. We know full well photos are | touched up and fake. In fact we're at the point now where | millions of people are so aware of it that they are | routinely editing their own photos to match whatever the | ideal being pushed at us in the moment is. Maybe that's | "instagram face" or giant asses, hell I remember when it | was heroin chic. The point is that we all know it's fake. | It just doesn't matter because media tells us what to | like/want regardless. | KONAir wrote: | Are there any studies on "I can tell from some of the | pixels"? Over the years I keep coming across both people | (and myself) on internets being correct on 'shopped photos | while masses belivied in. Is this just down to familiarity | with software and a health bit of skepticism? | autoexec wrote: | I think it's the healthy skepticism that's more important | than being able to detect some minute detail in an edit. | For a while photos were considered strong proof, not so | much today. | | There has been research on detecting photo manipulation | and there's plenty of things people can watch out for if | they're trying to "prove" an image was altered, but a lot | of edits I see are so bad/obvious that the people making | them aren't really trying to "fool" anyone with them. | They just think it makes the photo look better. They'll | do things like jack up color saturation to | impossible/unnatural levels, or remove every pore from | their skin, etc. | inlikealamb wrote: | >we no longer accept every image as representing absolute | truth | | That's not true for many people (I'd go as far to say | most), and we're so inundated with it that unedited media | is the _minority_. Ad campaigns get PR for being | "unedited" and even then they're heavily art directed | (casting, lighting, styling, etc) to compensate. | | The effects are so widespread that they're subliminal, even | if you're conscious of the scope that they occur. | Billboards, tv, movies, newspaper, magazines, products on | shelves, menus at restaurants, wedding photos, family | christmas postcards... it's inescapable. | | Even if you're some paragon of mindfulness and truth in | image editing and can somehow isolate yourself from its | influence, you're still subject to it because of how it | impacts the way everyone else behaves and sees the world. | | We should learn from these mistakes. | autoexec wrote: | photo manipulation is pervasive, but I don't take that as | evidence that it is actually being believed as truth. I'm | not really sure what the subliminal effects are. If you | see an ad for a fast food burger on TV it might | successfully make you hungry, and make you want to go to | that restaurant, but nobody really expects the food they | get will look anything like the food in the ad did. | inlikealamb wrote: | > but nobody really expects the food they get will look | anything like the food in the ad did | | Have you ever worked at a restaurant? It's not as unusual | as you think. A lot of us on HN are in bubbles of savvy | people because of our tech-related professions, and most | people are NOT savvy. Many people never consciously think | about the images they're subjected to. | autoexec wrote: | Wouldn't it be a problem for the restaurant if people | were contently sending food back or being disappointed by | the product because it didn't look like the ad? I mean, | the difference is striking! (see | https://i.imgur.com/e9EaVbu.jpeg). If I genuinely | expected the first burger in that image and got served | the second one I'd demand my money back and maybe never | step foot in that restaurant again! Wouldn't most people? | It seems more likely to me that most people accept that | the first burger is a fantasy. | inlikealamb wrote: | I worked restaurants through college and I've probably | had food thrown at me over a dozen times because it | didn't look like the menu! | | It's not the norm, but it's not completely unusual. I've | had many more people complain about the disparity in less | severe terms. It's a very weird world out there, and if | I've learned anything it's that I'm incredibly lucky to | have any amount of self-awareness because a lot of people | are running around out there on pure id... unaware of | just about anything. If you're at all skeptical about | anything you're ahead of the curve. | duxup wrote: | I'm not sure that when you go down that rabbit hole folks will | believe much at all... | | I don't want to go down the rabbit hole here but everyone hears | about 'fake news', they know what it is, they just label things | they don't like 'fake news'. I'm not sure that's helping. | | Either way we are going to find out. | nemo44x wrote: | It will get really fun when one side has the "experts" on | their side and can have them vouch for the authenticity of | something. | nsxwolf wrote: | We are there already with "fact checks". When someone sees | a "fact check" pop up on something they post on Facebook, | they just say "See? I knew I was right" | tejtm wrote: | I expect it will go as well as our collective immunity to | advertising and other propaganda. | xwdv wrote: | Personally I think this is interesting, ethics be damned. It is | obvious the direction we are going with these kind of | technologies. We will be able to create entirely new content | using the personalities of deceased people out of whole cloth. | | Some day we may be reduced to nothing more than a pile of ashes | and a USB stick with our personality reincarnated into it. That's | far better than simply being a pile of ashes, and may be the | closest we get to some kind of immortality, a way to keep our ego | wandering the digital world long after we've died and gone to | wherever you choose to believe. | | Combined with advanced AR technology, all it may take is putting | on some glasses to see the ghosts of your ancestors wandering the | real world and interacting with it. Well, maybe not _our_ | ancestors. _We_ will be the ancestors, and our descendants will | be the ones watching us, tending to our digital souls. | GeorgeTirebiter wrote: | If they had used a voice actor to recite the Bourdain letter --- | would that be the same sacrilege as using an AI? | grenoire wrote: | Sacrilegious bit is that it's not disclosed. It's made to be | deceptive. | jowsie wrote: | I think it would feel less weird/unsettling, but I'd still want | a disclaimer at least, like any other recreation/reenactment. | code_duck wrote: | My brother and I had a running joke that Tom Araya from Slayer | died in the mid 90s, based on his ashen appearance in the Divine | Intervention sleeve. Our "theory" was all of his vocals after | then were bits of previous recordings reassembled into new songs. | It sounded technically possible with manual editing back then, | but it has already gone so far as to automatable. | | In this case, for a documentary, I see how that's a defensible | used case. They could hire an actor to read the email, imitating | Bourdain's voice as closely as possible, to the same effect. | Other uses certainly could be problematic. Discussing them and | working out legal and cultural rules is very relevant right now. | We've already had posthumous celebrity event appearances over the | past couple of years. Famous actors could be in movies with | needing to be involved very much - mainly an IP license. I have | no doubt record companies would love to create new music with all | the artists of the 60s-80s, dead or alive. | jjcm wrote: | I don't see this as any different than getting an actor to play | the part. This happens all of the time in docos for a more | immersive experience. JFK voice comes to mind quite a bit as an | example. Is it unethical to get an actor to play his voice? | mywittyname wrote: | The major difference I see is that the viewer is aware that | they are watching an actor. Or at least, they should be aware. | It has always been standard practice to note when something | being shown is a reenactment of an event, and not the actual | one. | | Even if they used a voice actor to mimic Bourdain, they should | have informed the audience that an actor was reading the | script. | nemo44x wrote: | > It has always been standard practice to note when something | being shown is a reenactment of an event, and not the actual | one. | | Aren't the lines being read by robo-Bourdain actual things he | write and/or said? | | I feel like there's a difference there even if it is a bit | weird to not credit robo-Bourdain in those instances. | thebean11 wrote: | Voice has a lot more information than text. With a | deepfake, all the information (despite his general sound | and rhythm) is completely made up. | nsomaru wrote: | A deepfake audio + video with mask of the mayor of ethekwini was | used to encourage people to continue looting during recent riots | in KwaZulu-Natal, South Africa. | | It's scary. | jsonne wrote: | Do you have a source? That's wild if true. | ipsum2 wrote: | Googling "Deep fake ethekwini" brings up... this comment. | borski wrote: | To be fair, that doesn't make it untrue. It just means a | source is required - the parent _may_ be a primary source, | heh, but that has to be established. | | (I agree with you, I just have been corrected in the past | with actual on-ground knowledge that hadn't been reported yet | - don't know if that's the case here :)) | CamperBob2 wrote: | Looks like we have the first recorded instance of a metafake | here. | georgeglue1 wrote: | This is all I could find... | https://www.facebook.com/eThekwiniM/posts/4359537737429640 | | I don't know what the referenced video is, or if it is a | deepfake though. ___________________________________________________________________ (page generated 2021-07-16 23:00 UTC)