[HN Gopher] Youtube.js - full-featured wrapper around YouTube's ...
___________________________________________________________________
Youtube.js - full-featured wrapper around YouTube's private API
Author : mahnouel
Score : 185 points
Date : 2022-04-13 23:31 UTC (23 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| trinovantes wrote:
| I'm always nervous about 3rd party API wrappers. It's basically
| saying "Here's the keys to my (users) Google account, please
| don't do anything bad". Even if it's open source, there's no
| guarantee there's no malicious change in version x.x.N+1
| chmod775 wrote:
| Any software you use might steal your YouTube credentials.
|
| It might be this API wrapper or it might be any other
| dependency. It might even be the scientific calculator you
| installed that had nothing to do with that project.
|
| What makes this especially scary?
| trinovantes wrote:
| Technically nothing can be trusted e.g. can anyone trust
| their silicon, wires, device drivers, compilers, OS, routers,
| SSL CAs, etc.? Trust has to happen at some point.
|
| The difference is that it's trivial for this developer to
| insert a backdoor to steal Google credentials since they know
| exactly how and where the oauth tokens are located. It's
| significantly harder for e.g. a webpack developer to insert a
| backdoor to steal Google credentials since they would have to
| first determine the code it's processing is handling oauth
| tokens and then figure out where they are stored.
|
| The barrier to entry is the key in assessing your threat
| model. 3 letter agencies may not care about the cost if the
| target is valuable but a bored kid on the other side of the
| world will give up pretty quickly.
| [deleted]
| sieabahlpark wrote:
| You could just pin to a version like a good dev and rid
| yourself of the problem.
|
| Just do a quick diff check when you upgrade. If you're too lazy
| to do that then you never really cared about security all that
| much in the first place.
| Firmwarrior wrote:
| WHAT? How am I supposed to make a social media calculator
| flashlight app without pulling in 200 random libraries and
| their 5000 dependencies?
| bragr wrote:
| Cool project but I wonder if the name will catch flak from the
| lawyers. Trademarks and all that. At they very least a big "This
| is not affiliated with Google/Youtube" seems like a wise
| precaution.
| ushakov wrote:
| they have a disclaimer below
|
| https://github.com/LuanRT/YouTube.js#disclaimer
|
| edit: why am i getting downvoted?
| freedomben wrote:
| IANAL but I've worked with some of them and I'd guess that
| isn't sufficient.
|
| My advice to avoid any negative attention from YT/Google (but
| again IANAL) would be to do two things:
|
| 1. Move the "this is an unofficial" disclaimer at the top and
| bold and/or italic it so it's totally unmissable. I would use
| the same approach as MarshallOfSound:
| https://github.com/MarshallOfSound/Google-Play-Music-
| Desktop...
|
| 2. Praise Youtube/Google for having such a great API that
| empowers and enables their users to get the most out of their
| subscription. Make sure to point out the (true) benefits that
| _YT_ gets from having a library like this that is available
| but they don 't have to maintain and promise nothing to.
| saurik wrote:
| The disclaimer doesn't really matter, as 1) we wouldn't
| really expect that many people to read it and 2) it is going
| to get used in a ton of contexts without that disclaimer...
| such as in the headline of this Hacker News post. I can't
| make a company "Apple Computer Backstage" and just put a
| little note on my website that says "Apple Computer Backstage
| has no involvement with Apple Computer". This project simply
| _should not_ have "YouTube" in its name... or, if it
| absolutely _must_ , it _at least_ needs to be on the other
| side of some kind of preposition. This insistence upon using
| other peoples ' trademarks in the names of third-party
| clients based on adversarial interoperability--something that
| the company often has every reason to "throw the book at" as
| it is inherently hostile--is what ends up killing the vast
| majority of these projects over time. This isn't "in for a
| penny, in for a pound": you need to carefully choose your
| battles, and flagrantly violating someone's trademarks is
| going to be your weakest link as it is such low hanging legal
| fruit and doesn't actually buy you anything.
| wvenable wrote:
| The should immediately rename it so as not to be caught the way
| YouTube Vanced was. A disclaimer is not a sufficient response
| to trademark issues.
| Nextgrid wrote:
| They should rename it to Innertube.js (which is the official
| name of the private API, and as far as I know not trademarked
| not used in any user-facing resources).
| delusional wrote:
| Valve.js would also work, since it allows access to the
| innertube.
| RobertRoberts wrote:
| The reason I stopped integrating any API based system (FB,
| Twitter, etc...) into my code bases for services I don't pay for
| (or my customers) is because they all changed willy-nilly and
| broke on a regular basis.
|
| This is more likely to break (be broken by google) than an
| official API, and those are bad enough. (hard pass on even trying
| this out, especially if it's good/nice I'll want to use it and
| kick myself later for being an idiot.)
| stingraycharles wrote:
| Unfortunately it's rarely that these kind of decisions are
| based on technical merit, but instead because they're part of
| the business proposition, or requested by other parts of the
| business (sales/marketing).
|
| Other than that, I fully agree that you should try to minimize
| your dependence on them, it's not good a good position to be
| in; what's in the best interest for YouTube today may not be
| the case in one year.
| matheusmoreira wrote:
| Awesome!! A custom client for YouTube that bypasses all their
| front end code. I wish there was something like this for every
| single web site!
| cphoover wrote:
| I'm surprised they still support video dislike via API, but have
| removed it from the user interface... I understand they are
| likely worried about backwards compatibility with the abundance
| of client-devices, and not inadvertently breaking some app
| somewhere, but why not just make it a noop...
|
| Unrelated: My treadmill has the absolute worst YT client I've
| ever used.
| freedomben wrote:
| My understanding was that they still want to hoover up that
| sweet sweet preferences data (of which dislike is a useful
| metric). It's just not going to be exposed externally.
| Tenoke wrote:
| They do and you can install extensions to see the dislike count
| though sadly they slow down my experience every time Ive tried
| them (at least on Windows Chrome).
| etra0 wrote:
| You cannot longer see the true like/dislike ratio. They
| removed the dislike count even from the API since December
| 13th [0]. The extension does some guesswork to calculate that
| for you [1].
|
| This library also doesn't give you the dislikes anymore (just
| tested it).
|
| [0] https://support.google.com/youtube/thread/134791097/updat
| e-t...
|
| [1] https://github.com/Anarios/return-youtube-dislike#what-
| it-do...
| Tenoke wrote:
| Ah I wasn't aware. For what is worth the gueswork does seem
| accurate enough to be useful on the videos I tested last
| time.
| tshaddox wrote:
| YouTube hasn't removed the dislike button. They have removed
| the dislike _count_.
| grammers wrote:
| Nice, thanks for sharing!
| timoteostewart wrote:
| Love this Q&A: Do I need an API key to use
| this? No, YouTube.js does not use any official API
| so no API keys are required.
| endisneigh wrote:
| If you're YouTube or any site, and want to stop these sort of
| wrappers - what's the easiest way to do so without breaking your
| own site?
|
| I find this task to be an interesting engineering problem.
|
| A related question is if there's an unspoofable way to detect a
| client.
| smrtinsert wrote:
| Require request tokens. Authorized clients would have some sort
| of generator, anything else would fail to execute.
| samisupset wrote:
| Keep changing the implementation, keep changing names, keep
| changing the API formats.
|
| I'm definitely curious if there's a way to do a rotation that
| resists easy automatic code analysis.
| FrenchDevRemote wrote:
| you don't even have to look at the script, just at the
| network requests
|
| you'd basically have to make your own stealthy video format,
| otherwise you can just catch network requests
| comprev wrote:
| Facebook does something similar to combat adblockers. They
| mangle the names of div elements to make sponsored posts
| indistinguishable from friends/group posts. I'm not aware of
| any browser plugins which are effective at blocking FB ads.
|
| Anyone know if other websites put as much effort into anti-
| adblock engineering?
| lostmsu wrote:
| FB Purity seems to be keeping up. But it is not a general
| purpose adblocker.
| wantlotsofcurry wrote:
| I think Workday does the same. Trying to scrape data from
| there was a nightmare (team didn't have api access).
| freedomben wrote:
| It's probably impossible to stop it, but you could make it
| pretty painful by aggressively breaking/changing the API
| contract. You could maximally engage human ID and
| fingerprinting techniques and CAPTCHAs, or do machine learning
| on usage patterns to find likely bots and ban them if you're
| willing to accept that false positives will hurt innocent
| customers. Whatever you decide to do, it won't be free because
| it will also hurt your own devs (such as making it harder and
| more complex to work on). Obviously your own devs can have
| advance notice for changes, but it's still a pain to deal with.
|
| Overall though, I would seriously ask why? Anti-cheat for a
| game maybe? It will cost you time/money to prevent, and it will
| hurt people that like your product enough to hack on it. As a
| user who rejoices in having APIs I can use to automate products
| I like, I'd be far more likely to pay you if you have an API I
| can use.
| xyst wrote:
| So a form of DRM on the APIs themselves? It wouldn't be easy.
| How would you determine if the request is originating from an
| official YT developed app (iOS, Android, ...) or client
| (YouTube.com); or a non-approved client?
|
| I suppose you can have a system which works like TOTP except
| for machine-to-machine. Although it would probably be broken
| since anything on the client side can be disassembled. The UX
| would likely suffer as a result as well.
| freedomben wrote:
| I think the best you could do is attempt
| detection/fingerprinting, but I don't think it's possible to
| stop it since you never have full control of the client.
|
| I once saw a certificate-based implementation once where the
| server issued a temporary x509 cert to the client and then
| used mTLS[1]. It did reduce "unauthorized" clients by raising
| the bar, but it ended up making life way harder for their
| devs and the people they really wanted to stop just
| implemented the cert strategy and moved on.
|
| [1]: https://freedomben.medium.com/what-is-mtls-and-how-does-
| it-w...
| [deleted]
| ec109685 wrote:
| The demos have Node.JS examples. If that's the case, it doesn't
| seem possible to block.
|
| If it's running in a browser, Google can simply disallow those
| domains to make API calls.
|
| Also, at least Apple will block apps that make unauthorized API
| calls to third parties.
| Bilal_io wrote:
| The agent can be faked.
|
| There are CORS rules, but those are enforced by the browser,
| a backend cannot prevent you from calling it, except by
| requiring an access token or something similar.
| guessbest wrote:
| Every few requests put up a catchpa. That is how google did it
| for their search
|
| https://support.google.com/websearch/answer/86640?hl=en
| nomel wrote:
| Wouldn't there be all sorts of human detection that they
| could do, similar to how game cheat engines work? A human is
| going to move the mouse across elements, drag, poke the
| screen, be slow, etc, and all in fairly predictable ways.
| Some API calls almost certainly require human interaction,
| where some interaction graph could be feed as a key to the
| API. It's cat and mouse, but at some point the mouse is going
| to get tired.
| FrenchDevRemote wrote:
| Given the power of Google they may be able to force you to sign
| in after watching a few videos.
|
| Or only serve videos if they detect some kind of physical
| input(mouse/keyboard)
|
| But they don't seem to care that much for non-mainstream tools,
| or this would be blocked already
|
| For small scale sites it's definitely not easy to block
|
| They may be some paid libraries that promises to do it, but
| I've never seen one that seemed really unbreakable
|
| Even if you go all the way and block EVERY possible way of
| doing this, you can make a puppeteer script that could watch
| and record sound/video directly on the screen in like 15 lines
| of code, even if it would be really slow to get long videos
| Nextgrid wrote:
| The only bulletproof way is to change your business model to
| make these useless, irrelevant or no longer threatening. Yours
| and the user's incentives should be aligned.
|
| If you serve content, put it behind a paywall and rate-limit
| based on the maximum amount a human can reasonably consume and
| stop caring whether the user uses your own client or something
| like this - after all you're getting paid either way.
|
| The only businesses that are threatened by unofficial API
| clients are cancerous "growth and engagement" crap where the
| "value" is the wasting of the user's time. Don't be such a
| business and you'll be fine.
| motoxpro wrote:
| What about something like Craigslist? I would rather that not
| be behind a paywall. I would think they would also enjoy not
| having their site scraped and re-created.
| heavyset_go wrote:
| Seems like a use case for remote attestation.
| mediocregopher wrote:
| There's no perfect solution, but you can make it painful. One
| solution I've seen, which only works in a server-side rendered
| site, is for the server to generate a random name for each form
| field being rendered. The mapping of random id to real field
| name is kept in the user's session information server-side, so
| the translation is then done server-side as well whenever the
| user performs an action.
|
| At that point anyone writing a library like this would need to
| actually pull in the rendered page on which the user is
| supposed to be navigated, scrape the field names off of that
| (which won't be easy), and only _then_ could they perform the
| form action.
|
| But if you're a big enough site, someone will likely still take
| the time to do it.
| sschueller wrote:
| Provide a public API (charge for it if you have too). The
| videos on YouTube are the property of the creators, not
| YouTube.
| charcircuit wrote:
| YouTube already has a public API
| freedomben wrote:
| I've been wanting to use the Youtube Music API to automate some
| personal chores (like building/cleaning playlists, etc) and was
| very discouraged. I actually switched to Spotify (trial)
| partially over it but there were a couple of other (off-topic)
| reasons I didn't want to stay with Spotify.
|
| This looks like a wonderful tool! And it's not in Python :-D
| (sorry python people). Like others I'm a little concerned about
| breakage as youtube APIs churn. Does anyone know what Youtube's
| approach to backwards compatibility is for internal APIs? Some
| companies just wait until 98% of user's are on the new clients
| and then rip stuff out, but others I've worked with basically
| don't allow breaking the API except in important circumstances
| and they stick around deprecated for a while.
| heavyset_go wrote:
| Ths drove me nuts because Google Music had an actual public
| API. The service was deprecated in favor of YT Music, which has
| less features and no equivalent API.
|
| That deprecation was a catalyst to move my music collection
| elsewhere, so it's not threatened by the whims of Google
| anymore.
| freedomben wrote:
| Makes a ton of sense. I almost bailed for the same reason. It
| was such a ridiculous downgrade. I just _barely_ (today
| actually) got back a super important feature to me that Play
| Music had years ago: Save a queue as a playlist. If this
| private API gets a C &D or gets aggressively broken, I
| absolutely will bail too.
| qzx_pierri wrote:
| +1 Social Credit Points for posting "SPOTIFY BAD" in a public
| forum
|
| /s
| [deleted]
| jokoon wrote:
| Sadly, I don't think it will let users view video that require
| login.
|
| For example, some 6min show that goes on live french TV everyday
| got flagged and requires login to be viewed, for age reason.
|
| Maybe there was curse words, or some butt-shaped thing in it?
| Quentak wrote:
| https://github.com/zerodytrash/Simple-YouTube-Age-Restrictio...
| imiric wrote:
| It would be great if this had a CLI tool, so that it could be
| used as an alternative to yt-dlp. Or a web frontend as an
| alternative to Invidious, which breaks more often than not.
|
| That said, I wouldn't be surprised if Google issues a C&D, or
| just inevitably breaks it, especially if it uses undocumented
| APIs.
| skanga wrote:
| Its VERY easy to make one. 1. Install NodeJS
| and NPM (if you don't already have it) 2. Create a new
| folder 3. Run "npm install youtubei.js@latest" in that
| folder 4. Create a new file in this folder called ytdl.js
| (or whatever you like) 5. See section "Downloading
| videos:" on the github page. Make the contents of the file
| exactly like that. i.e. just cut/paste that example. 6.
| Replace line 'Looking for life on Mars - documentary' with the
| name of the youtube video you want to download (ideally this
| should come from args) 7. Run "node ytdl.js" and it
| should download
___________________________________________________________________
(page generated 2022-04-14 23:01 UTC)