[HN Gopher] Yt-dlp - A YouTube-dl fork with additional features ...
___________________________________________________________________
Yt-dlp - A YouTube-dl fork with additional features and fixes
Author : makeworld
Score : 240 points
Date : 2021-08-26 19:42 UTC (3 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Y_Y wrote:
| So how come a fork was necessary?
| ozarkerD wrote:
| Devs on yt-dl have been inactive for a few months iirc
| julienpalard wrote:
| Does not look inactive to me: https://github.com/ytdl-
| org/youtube-dl/graphs/contributors
| nannal wrote:
| Jun 29, 2021 - Aug 26, 2021: 0 commits
| rtkwe wrote:
| It does look like there's little activity in the last
| couple months and none in the last almost 2 months.
| MrDOS wrote:
| For longer than that. 846 open PRs, 3.7k open issues. And
| this long predates the GitHub takedown debacle. This isn't to
| say that they're totally AWOL: they do a really good job
| keeping on top of the boring break/fix work, like keeping on
| top of the ever-changing interfaces of video providers. But
| they're very conservative about expanding functionality, and
| even fixing more minor bugs.
| nonbirithm wrote:
| And they've closed many issues as duplicates with no
| further explanation given:
|
| https://github.com/ytdl-org/youtube-dl/issues/23860
| rovr138 wrote:
| Have they been duplicates?
| nonbirithm wrote:
| The linked issue contains several issues describing the
| same problem, but what I meant to say was that _which_
| duplicate they were of was never made clear. The
| maintainers seemed to assume that everyone knew what it
| was and never provided a link to the original issue.
| SevenSigs wrote:
| my youtube-dl still works fine for youtube.com but I haven't
| tested many other supported sites...
| the-dude wrote:
| Wasn't the repo taken down for a while?
| banana_giraffe wrote:
| Yes, but it was brought back on November 16, 2020, and
| there was plenty of activity after that, till this recent
| drought of dead air.
| kunagi7 wrote:
| youtube-dl has been inactive for the last 2 months.
|
| Most things still work but support of different services is
| something that needs daily updates to not break. Even if most
| popular websites still work, pages like Newgrounds are
| breaking.
|
| There's 3.7k open issues right now and nothing gets merged [0].
|
| [0] https://github.com/ytdl-org/youtube-dl/issues
| jrm4 wrote:
| Love it.
|
| The question, especially for the folks around here.
|
| When Github _eventually_ takes a stronger stand against this sort
| of thing, because it is 100% going to happen, are folks going to
| properly fight it?
| Macha wrote:
| They were DMCAed, they got bad press out of it, they eventually
| stood up for the developer in exchange for a token concession
| (deleting test data that referred to copyrighted music and was
| named in the complaint):
| https://github.blog/2020-11-16-standing-up-for-developers-yo...
| , and reversed the takedown unilaterally (which would mean the
| claimant could take github/microsoft to court if they felt like
| it).
|
| Whether it's a matter of principles or just the bad press from
| the initial takedown is less clear, but I think it'll be a
| while before they re litigate this issue.
| totetsu wrote:
| I still can't access my fork of YouTube DL with all my custom
| plugin work on GitHub
| nebula8804 wrote:
| Thanks for this post! First I am hearing about this fork!
|
| Darn...unfortunately still can't download Joe Rogan Spotify.
| Guess they haven't found a way around the widevine
| encryption...(Right now it seems like only a few podcasts are
| encrypted so other Spotify streams work)
| nerdponx wrote:
| It's always strange to think that there is encrypted data
| flowing through my software, through my operating system, into
| my hardware, that somehow I, the user, I am unable to access,
| but that the hardware can handle just fine.
|
| How does this work?
| kuschku wrote:
| You can access it just fine. It's just processed by very
| obfuscated code that constantly changes.
| gillytech wrote:
| This is going to cost me karma but don't waste your time with
| Joe Rogan.
| _arvin wrote:
| Don't tell people what to do. People don't watch JRE for Joe
| (mostly), but for more his guests. Not only that, Joe is
| great and you're a hater. Try harder. He's not perfect but
| you likely sure aren't either. Rogan podcast is fine and I
| recommend it.
| gillytech wrote:
| > Try harder.
|
| See my response to nebula8804
| nebula8804 wrote:
| I enjoy some of his podcasts. Yeah some are a waste of time
| but many have been spectacular.
| gillytech wrote:
| To be fair I would agree with you that some of his guests
| are great (e.g. Quentin Tarantino) and I do enjoy the long-
| form interviewing. But Rogan himself, to me, gives off bad
| vibes. Not really sincere or actually interested in his
| guests. I find he'll say he's interested and come back with
| some show of how cultured and learned he is by bringing
| something up on his own. I think he should be interested in
| his guest and put them in the spotlight. Joe himself is an
| unaccomplished stoner and promotes lifestyle choices that
| are repugnant to me.
|
| Others do a much better job of the long form interview and
| have themselves provided some value to society on their own
| merits. Tim Ferriss, Dax Shepherd and Jordan Harbinger to
| name a few.
| Ansil849 wrote:
| So does anyone know what happened to youtube-dl? Development
| seems to have just abruptly ceased with no explanation.
| dwrodri wrote:
| I wouldn't be surprised if the drama surrounding the DMCA
| takedown issued on the main git repository scared away a lot of
| contributors.
|
| Here's some coverage from the EFF, in case you missed it:
| https://www.eff.org/deeplinks/2020/11/github-reinstates-yout...
| arp242 wrote:
| Quite the opposite; number of contributions increased:
| https://github.com/ytdl-org/youtube-dl/graphs/contributors
|
| Just seems a case of the same what usually happens: main
| author loses interest, has other stuff to do, etc. and no one
| really takes up the slack.
| polote wrote:
| Why don't they split youtube-dl in two parts:
|
| - The extractors part (all the scrappers basically)
|
| - The cli tool
|
| What would be wonderful is that the extractors part is splitted
| out, so that anyone can use it without using youtube-dl. That
| would be much easier to update it too, each extractor is
| independent, so it is only a question, does this scrapper
| (extractor) works or not.
|
| The next step would be to have a multi-language format to
| describe a scrapper, instead of being coded in Python. But no
| idea if this is possible or maybe that would make things much
| more complex
| FiloSottile wrote:
| That is mostly already the case, you can invoke the extractors
| through the Python API or through the CLI and do the download
| yourself.
|
| Extractors regularly require custom logic, so they can't be
| described in anything else than a programming language.
| polote wrote:
| Being two different projects has advantages, especially when
| one part (need to) moves much faster.
|
| The fact that some requires custom logic doesn't prove that
| you need a programming language, it is just custom logic
| compared to the standard framework.
| Scaevolus wrote:
| > NEW FEATURES
|
| > Cookies from browser: Cookies can be automatically extracted
| from all major web browsers using --cookies-from-browser
| BROWSER[:PROFILE]
|
| Very nice. Having to manually copy cookies to get past login
| walls is annoying.
| pluc wrote:
| There's this too: https://github.com/deepjyoti30/ytmdl
| TillE wrote:
| A neat tool, but YouTube is generally an abysmal source for
| music. That's partly deliberate (from record labels) and partly
| an inevitable consequence of recompression with different lossy
| formats.
|
| Unless something is completely unavailable elsewhere, you'll
| find far better audio quality from other (original) sources.
| themodelplumber wrote:
| It's not so much about availability or even quality as it is
| convenience for a lot of us. Soviet funk here, favorite TV
| news theme there, C64 remixes over here, NPR tiny desk over
| this way, and a rendition of the full original vocal lyrics
| to the M:I theme which is basically only available on YT
| itself...
|
| I can't imagine trying to source this stuff independently and
| keeping up with it, and if some commercial music crept in I
| imagine it'd be easier to simply stay on YT and ask what
| level of quality one subjectively needs for dental drilling,
| or mindless work, or throne-in-lair-sitting, or whatever it
| is...
| Macha wrote:
| > Unless something is completely unavailable elsewhere
|
| I do run into this from time to time. Let's say I want the
| soundtrack to one of my favourite anime, from 2006.
|
| Some items from it are also on the source game soundtrack, so
| are available on iTunes. This is actually kind of rare for
| older anime which usually don't bother releasing in iTunes
| outside Japan, but being a game adaptation helps it here, I
| guess. Still, anything composed for the anime are not
| included.
|
| Some of it is there on Spotify. Actually, at one point it all
| was, and I can still see the tracks are there but grayed out
| in other people's playlists, so I assume there exists some
| region in which it was available, and the availability in my
| region initially was a mistake by a licensee who forgot to
| limit it to the regions they had rights for. Either way, it's
| no longer legally available for me.
|
| So I could.... VPN to Japanese iTunes, thereby breaking the
| terms of service, (assuming it's still there) or I could try
| import some decade old special edition DVDs of the anime
| which contained the OST.
|
| Or I could rip it from YouTube. Eventually, if you make no
| effort to sell to me, I'm going to resort to other options.
| input_sh wrote:
| > So I could.... VPN to Japanese iTunes, thereby breaking
| the terms of service...
|
| In my experience, you'd immediately enter "vacation mode",
| which is limited to two weeks in free version, but
| unlimited in premium. Spotify doesn't even complain when
| you switch countries in a couple of seconds. They're very
| lenient on enforcement, I've used it for like five years
| before Spotify actually became available in my country
| (though I couldn't pay for the premium with a card from a
| different country). Just log in via VPN once, then it works
| for two weeks.
| Fogest wrote:
| That and a lot of music I have found on there has a music
| video version of the song which can sometimes be slightly
| different than the original song. It's usually something like
| a longer intro before the music starts or something like
| cheering from a crowd or whatever to mimic it being a live
| performance music video. So you're getting a subpar sound
| quality combined with a sometimes differing song from the
| original.
| kevin_thibedeau wrote:
| You can usually find a lyric version with the unedited
| album track.
| iainmerrick wrote:
| YouTube has a ton of obscure music that can't be found on
| Spotify, plus things like old TV specials, theme tunes,
| outtakes and the like.
|
| The quality is usually pretty bad, it's true. But in many
| cases, this stuff is almost impossible to buy even if you
| wanted to, so YouTube is your only option.
|
| YT is also fairly unpleasant to use, which is why a lot of
| people go for youtube-dl.
| ChrisArchitect wrote:
| haven't paid attention to this since all the drama last year but
| again - what are all of you using youtube-dl for? Archiving your
| channel subs?
| ronnier wrote:
| I made a tool for myself so I could 1 click download videos to
| my iphone's camera roll. It's based on youtube-dl. I like to
| share raw videos and do not like to share URLs. I use this tool
| to make that easy -- I copy a url and run the iOS shortcut and
| the video auto saves to my camera roll.
|
| https://github.com/rroller/media-roller
| arp242 wrote:
| My internet connection can be flaky, and just downloading stuff
| and playing it locally works a lot better. YouTube tries very
| hard to minimize the buffer size to only what's needed - which
| makes a lot of sense at their scale - but it also means that
| less-than-stable internet connections are difficult because
| it'll never buffer enough to bridge the 3 minute outage.
| formerly_proven wrote:
| My experience is that the internet is really bad at forgetting
| the stuff you want gone, but really good at turning all your
| bookmarks into 404s and parked domains (another reason why
| bookmarks are pretty much worthless). So if you don't save it,
| it'll be gone next time you're trying to look it up. Doubly so
| if it is something even mildly controversial (now or then).
| boomboomsubban wrote:
| Watching videos in my media player rather than my browser.
| Hardware acceleration has better support in mpv than Firefox.
| crazygringo wrote:
| 1) Videos disappear. Anything I think I might want to watch
| again in the future, I always download rather than bookmark.
| Not just YouTube but any video site (e.g. Reddit).
|
| 2) When Internet is flaky or doesn't exist. I've downloaded
| whole sets of tutorial videos for example to watch on a plane.
|
| 3) Stuff in 4K where my video player does a better job with
| hardware acceleration than my web browser does
|
| I mean out of all the video I watch from the internet, 95% is
| streaming, but youtube-dl is for that other 5% that falls into
| the three categories above.
| sralbert wrote:
| I download Twitch vods and Youtube videos at home and copy them
| to my work PC to listen to during the day.
| Aachen wrote:
| Safekeeping, offline viewing, producing derivative works (it's
| all great to offer setting CC license on videos and then be
| hostile to anyone trying to actually use their rights), and
| viewing on platforms where the browser or player doesn't work
| (that's been a few years by now though).
| Larrikin wrote:
| I've started saving anything I ever watch more than once.
| Cooking recipes, music videos, learning materials, etc.
|
| I started when a group I listened to removed all the content
| they had put up over the past is year and said it was
| promotional material for their latest album which was garbage.
|
| Ads have gotten so bad that any videos over three minutes have
| multiple ad breaks so watching longer music videos or sets are
| just completely ruined when watching on YouTube.
|
| I wish there was a good set up for single watch videos since
| the ad algorithm tries to anticipate good places for breaks,
| which often times is right before punch lines to joke and when
| you skip over the ad they tend to put you back a few
| milliseconds ahead of where you stopped which can ruin jokes. I
| rarely watch for enjoyment (as opposed to learning) on a
| computer so I don't get the benefit of using an ad blocker like
| Ad Nauseam.
| Tistron wrote:
| For me, something like this is exactly where Deno would shine. I
| could just run `deno run <script url> --allow-net=YouTube.com
| --allow-write=.` and not worry about that it could do anything
| dangerous. Probably the url list would be a bit more complicated,
| but I could also just blanket allow net without worrying, since
| allow-read isn't needed.
| banana_giraffe wrote:
| One of the nice features added to yt-dlp recently: It integrates
| with sponskrub to call into the SponsorBlock database and with
| the right options will strip the sponsor segments from downloaded
| videos.
|
| Edit: Correct typo
| themodelplumber wrote:
| dlc? Don't tell me that's a different one...
| LeoPanthera wrote:
| dlc is abandoned and dlp integrates its features.
| banana_giraffe wrote:
| Whoops, typo on my part. Sorry about that.
| azinman2 wrote:
| Are the sponsor parts really so bad? To me not only is it easy
| to skip, I want indie people to make enough money to produce
| high quality content; otherwise media is just what a few
| biggies want to fund. They're not enough from YouTube itself
| unless you're in the top percentile.
|
| Also what sponsored content are people downloading versus just
| streaming live? I don't get the use case.
| sen wrote:
| I pay for YouTube premium but still see them. That's not OK,
| especially when the creators make 10x more money off YouTube
| Premium users than from ad-based users.
|
| YouTube needs to make a way for creators to label the sponsor
| sections and it skips it for Premium users.
|
| As for why downloading, I download any/every video I find
| useful so I don't have to go back to YouTube to watch it
| later. Everything from blender tutorials to car build videos
| that taught me something to music video clips and generally
| just anything interesting. It all goes to my NAS and I can
| instantly pull it up for reference and scrub quicker and
| don't have to deal with YouTube's increasingly annoying
| website.
| kortilla wrote:
| Yes, they are just ads in an even worse form because it's not
| clear when to skip to to pass over them.
| lawl wrote:
| > Are the sponsor parts really so bad? To me not only is it
| easy to skip, I want indie people to make enough money to
| produce high quality content; otherwise media is just what a
| few biggies want to fund.
|
| In my experience, it's the same with ads everywhere else. It
| (usually) starts out not being overly obnoxious, with just a
| "this video is sponsored by [garbage tier mobile
| game/earphones/vpn/whatever] more about them at the end of
| the video" and then the pitch at the end.
|
| I don't mind these. They quickly get the name out at the
| beginning and then don't interrupt the video. What really
| annoys me are the ones that interrupt the video. At some
| point a few of them annoyed me enough that I installed
| SponsorBlock. Because I don't _want_ to hear or see these
| ads, but i tolerated them. But once that threshold is crossed
| where I don 't tolerate all of them anymore, why would I not
| just block all of them? I'm not going to unblock specific
| channels that are well-behaved to listen to ads for products
| i will definitely never buy.
|
| It's the exact same thing with regular ad-blocking. Sometimes
| when I'm on a fresh OS I start browsing the web and only
| notice I don't have an adblocker once I visit a page with
| super obnoxious ads (e.g. google on mobile and realize
| there's only ads and no organic results for like the first 5
| screens).
| input_sh wrote:
| > What really annoys me are the ones that interrupt the
| video.
|
| Especially when those interruptions take two minutes. I
| don't mind them up to 30 seconds, but their length does get
| pretty ridiculous from time to time.
|
| There's no medium that's gonna make watch two minutes ad
| without looking away or trying to skip it. You either sell
| it quickly or don't.
| azinman2 wrote:
| But what then do you think is a realistic alternative?
| YouTube costing $20/mo or more? Can't watch videos without
| a patron?
|
| If your answer is nothing -- I expect everything for free,
| then that's both unrealistic and parasitic.
| lawl wrote:
| > But what then do you think is a realistic alternative?
|
| Not my department. I'm perfectly happy being a parasite.
| I used to watch Twitch every now and then. Twitch
| introduced server side ads and made ad-blocking
| unreliable and annoying - so i stopped using twitch.
|
| My life doesn't depend on youtube, and if they decide to
| shut me out or the platform stops existing, that's fine
| with me. Maybe other video platforms can try out other
| models instead of that effective monopoly google has on
| online video currently.
|
| I've also already said that I tolerate non-obnoxious ads.
| But looking at the rest of the web, it doesn't seem to go
| in that direction, so I'll keep blocking until they kick
| me out.
| ithinkso wrote:
| 'This video was sponsored by X' is ok.
|
| And that is ho... 'HEY! Don't forget to check out X!' is not
| Ansil849 wrote:
| > Are the sponsor parts really so bad?
|
| Yes. Some of us don't like being assaulted with sales pitches
| everywhere we look.
| kaladin-jasnah wrote:
| I think for some us pro-sponsor-segment people, I feel like
| regular ads are particularly dry and more of an "assault."
| But sponsor segments... can be creative, funny, and
| sometimes make me actually want to buy the product (...not
| the generic Raid: Shadow Legends ones... but even those if
| they're well done). I appreciate well-made ads, so I don't
| mind sponsor segments.
|
| Here's an example of one that is... nonsensical and perhaps
| a bit humorous (well, at least I liked it):
| https://m.youtube.com/watch?v=0jPLRtEEjhU
| Ansil849 wrote:
| > I appreciate well-made ads, so I don't mind sponsor
| segments.
|
| Then don't use the sponsor segment removal portion of the
| tool. Simple as that.
|
| Those of us who don't want "clever" advertising any more
| than "annoying" advertising can use it. And that's that.
| nerdponx wrote:
| Maybe these creators should post sponsor-free versions of
| their videos on Patreon?
|
| If you are watching for free, and you're that annoyed by
| ads, then maybe you just feel entitled.
| judge2020 wrote:
| Sure, but creators don't have many other avenues for
| revenue when you're already using YouTube-dl(p) to download
| videos without watching ads. I imagine sponsor spots will
| be devalued over time as sponsorblock usage continues to
| grow, especially for creators with audiences that watch
| content on desktop more than mobile.
| Forbo wrote:
| Is it apparent to the sponsor when someone is watching
| using youtube-dlp versus a regular view? I'd imagine most
| sponsored segments are negotiated based on subscriber
| count or view numbers.
| crtasm wrote:
| I don't believe the view counter gets incremented when
| using any unofficial downloader or front-end.
| banana_giraffe wrote:
| For me, yes, they're bad. I get it's not the end of the
| world, but a lot of the ad copy in sponsor segments really
| grates on me, like VPN ones. I try to get rid of ads from my
| life where I can, this is part of it.
|
| And for me, a lot of the videos I download are played back,
| audio only, while I go for walks or am otherwise away from
| the Internet. YouTube lets you download videos, but my audio
| comes from multiple sources, YouTube is just one source. I
| can integrate yt-dlp into my feed reader so I don't need to
| think about the source anymore than I need to worry about
| going to a random podcast's website to listen to those.
|
| In the end though, a tool like SponsorBlock is just another
| way I remove the annoyances from the modern Internet.
| arghwhat wrote:
| Ugh, the blatant lies in VPN ads.
|
| "MAKES IT SO YOUR ISP CAN'T SEE YOUR TRAFFIC" - yeah,
| congrats, now it's just another ISP that sees your traffic,
| which was probably protected by TLS anyway...
|
| Heck, they even keep recommending the utility of Terms of
| Service violations like accessing contents from other
| regions.
|
| VPN companies are scammy as hell.
| ajdude wrote:
| I always remember the HN comment[0] where developer explained
| which video was the breaking point that pushed them into
| creating this app, and it was kurzgesagt of all authors.
|
| I am personally wondering if Yourube themselves are going to
| start implementing some thing like this sooner or later;
| after all, you are paying for an ad free experience on
| YouTube premium, just to watch all of those new videos become
| filled with in-video advertisements.
|
| [0] https://news.ycombinator.com/item?id=20781415
| nebula8804 wrote:
| Man I thought I was in heaven when I accidentally stumbled upon
| SponsorBlock. Now you're telling me about this?! Amazing! How
| does the video quality fare after stripping? Is it re-encoding
| the video or somehow stripping it without altering the quality?
| banana_giraffe wrote:
| It re-encodes. Actually, I think it defaults to marking the
| sponsor bits ... somehow. Whatever it's doing wasn't enough
| for my player to notice.
|
| That said, for this stuff, I've only downloaded audio, and of
| that mostly talking head type stuff, where I let it cut out
| the segments. Whatever re-encoding is done would have to be
| pretty bad before I'd care.
___________________________________________________________________
(page generated 2021-08-26 23:00 UTC)