[HN Gopher] Show HN: API to query catalogs of 20 streaming servi...
___________________________________________________________________
Show HN: API to query catalogs of 20 streaming services across 60
countries
Author : Saepirist
Score : 281 points
Date : 2022-01-09 12:13 UTC (10 hours ago)
(HTM) web link (www.movieofthenight.com)
(TXT) w3m dump (www.movieofthenight.com)
| Cyph0n wrote:
| Small typo in the list of countries: "United Emirates" -> "United
| Arab Emirates".
| marban wrote:
| https://reelgood.com/ is really good. (US-centric)
| bcook wrote:
| Account required?
| michaelmcmillan wrote:
| Cool! Do you login with one account bouncing off 60 IP addresses?
| simlevesque wrote:
| They say "streaming" but mean "video streaming". I thought it
| included music too but it does not.
| hotgeart wrote:
| You should add an order by: Rating, Year, etc.
| Saepirist wrote:
| It's already supported! Different plans have different order by
| fields supported but if you have Ultra or Mega plan you can
| order by the following fields:
|
| original_title, year, imdb_vote_count, imdb_rating
| capableweb wrote:
| Great website! One thing that is missing: Availability is not the
| only criteria people use. One example: Last month I wanted to
| watch Superhero Movie, but I was in Spain, where a lot of movies
| are dubbed. I found out it was available on Amazon Prime (via
| another similar website to the one in the submission) but when I
| tried to watch it, it was only available in Spanish voice with
| English/Spanish subtitles, so obviously wouldn't work. Instead I
| had to result to using the holy pirate cove again to be able to
| watch what I want.
|
| So if also audio/subtitles language could be added to this
| service, it would make it 100% better and more useful for solving
| the problem it's trying to address!
| Saepirist wrote:
| Thank you! Yes, I'm planning to add more details like these.
| MailleQuiMaille wrote:
| How often is the api updated ? And how did you collect the data ?
| Very impressive
| Saepirist wrote:
| Thank you! The data is updated daily by automated tools that
| checks the services regularly.
| ape4 wrote:
| This seems like the hard part. Does it need accounts with all
| the services, use a VPN to appear to be in each country, use
| a fake/rotating User Agent, etc
| Saepirist wrote:
| Depends on the service any of these might be needed.
| toomanydoubts wrote:
| What an uninteresting answer. People are interested in
| the technichal details, come on.
| Saepirist wrote:
| Haha, sorry. But actually it's pretty much it. You need
| VPNs to check catalogs in different countries, which is
| tricky for some services (like Netflix) as they try to
| block VPNs. So you gotta always have a large pool.
| Catalogs of services like Prime Video is just there
| without an account but for Netflix you need accounts to
| see the catalogs (and not to send too many requests
| otherwise they block you).
|
| Rest of it is just writing some regular crawlers.
| toomanydoubts wrote:
| Cool, thanks for expanding and congrats on the project!
| chrismeller wrote:
| I too would love more technical details on the harvesting
| process. Something I've definitely thought about doing in
| the past but wasn't patient enough to implement.
| Readywater wrote:
| I would kill for language options on this. Having been studying
| Arabic while living in Northern Europe, none of the streaming
| services have Arabic dubbed shows. The only thing I've found is
| the expanse, which itself is in a different dialect than what I'm
| studying. Being able to easily search by language availability
| and subtitle availability would be wonderful.
| salamandersauce wrote:
| Only works for Netflix but unogs.com let's you search by
| language and subtitle availability.
| dzhiurgis wrote:
| Not streaming but https://yts.mx/browse-
| movies/0/all/all/8/latest/0/foreign has pretty good catalogue
|
| IMDB itself lets you filter by language:
| https://www.imdb.com/search/title/?title_type=feature&releas...
| but it's PITA to then go and search for these films
| Saepirist wrote:
| Thanks for the suggestion, planning to add more details about
| these kinds of things.
| edmcnulty101 wrote:
| Very cool!
| _of wrote:
| Great project!
|
| Trying to find something good to watch on Netflix, my weekend
| project consisted out of extracting all movies on Netflix and
| sorting them by IMDB score. Here is the result:
|
| https://oscarfranzen.com/netflix-by-score/
| arthur_sav wrote:
| You might want to include a "new releases" section. Sometimes
| they release good stuff that is not rated yet.
| grumpwagon wrote:
| Do you support searching which country a movie is available to
| stream in?
|
| For example the movie It is available in Netflix Austria (but not
| many other Netflix regions), but to find that, I searched It,
| then randomly picked countries from the dropdown list. It looks
| like the API has country as a required field as well, so to do
| this programmatically it would be similar.
| Saepirist wrote:
| It's available on the API, you can use /get/ultra endpoint to
| see in which countries a show is available. Planning to add
| that feature to the web app as well.
| flippingbits wrote:
| One of our customers is using your API to automatically generate
| news for upcoming/changing/leaving shows. Keep up the good work!
| freewizard wrote:
| awesome tool! fount some interesting data explore the Compare
| Streaming Services section [1]
|
| For example, Netflix and Prime Video are almost equal on total
| number of shows (~5.5k movie + series) but on Netflix only 4%
| shows are from pre-2000, while Prime has 20%. Also HBO Max has
| half size of Netflix library, but when filter by IMDb rating and
| voters, the gap is much smaller, and in certain cases HBO even
| wins out.
|
| In terms of language diversity, Netflix is doing the best with
| 45% content not in English, that is far more than anyone else
| (except Zee5 which is primarily Hindi). However when it comes
| down to a specific language, the number is usually much smaller
| and around 100, which is less than ideal for someone primarily
| consumes French/German/Chinese/Korean/etc. (tweeted more about
| this [2])
|
| [1] https://www.movieofthenight.com/compareservices
|
| [2] https://twitter.com/freewizard/status/1480246184887398409
| Saepirist wrote:
| Thank you! Nice to see the compare page is being useful to
| people.
| mixmastamyk wrote:
| Would be nice if it supported Kanopy, it has an eclectic catalog
| for free if accessed with a library card.
| nkrisc wrote:
| I appreciate that this is available, so thank you. But it's
| frustrating that this even needs to exist in the first place.
| ju-st wrote:
| My ublock origin adblocker is blocking ShareButton.js because of
| "Fanboy's Annoyance" filter list (a default one) and not loading
| ShareButton.js leads to a blank page.
| [deleted]
| Saepirist wrote:
| Thanks for letting me know! Gonna check it.
| hutrdvnj wrote:
| Thx for explaining why I see a blank page, I have the same
| problem.
| shp0ngle wrote:
| I'm in Vietnam, the results are wrong.
|
| I put the movie "Twilight" (the vampire cheesy movie) and the
| results show me that there is no option, while it is on Netflix.
|
| https://www.movieofthenight.com/movie/Twilight/1912
|
| https://www.netflix.com/search?q=twilight&jbv=70099113
| Saepirist wrote:
| Thanks for letting me know! Gonna check how it went wrong.
| tcmb wrote:
| Also 'Succession' is available in Germany on Apple TV+, the
| search says otherwise (not available at all in Germany).
| Saepirist wrote:
| Hey, just checked, in Germany it's available to buy on Apple
| TV but not it's not part of Apple TV+. Currently the API only
| supports Apple TV+.
| tcmb wrote:
| Oh ok, I misunderstood that. Thanks.
| lotsofpulp wrote:
| Apple TV+ is only content made by Apple.
| ricardo81 wrote:
| Very nice. Maybe you could build a recommendation engine on top,
| Netflix seems to have a knack for recommending things I've
| already seen. Relies in me telling you which films I like of
| course, or an existing similar dataset.
| Saepirist wrote:
| Thank you! There's already a recommendation engine I've made
| actually, it's the home page of the link I put:
| https://www.movieofthenight.com/
| ricardo81 wrote:
| Indeed, excellent work. Was thinking more of personal
| customisation based on previous selections/ratings rather
| than filters.
| Saepirist wrote:
| You can have that by signing in. Once you have couple of
| stuff in your favorites list, in Feed page a new row called
| "Movies for You" and "Series for You" will appear. Though
| right now it's updated every 24 so it might take up to a
| day for them to appear.
| ricardo81 wrote:
| Ah, excellent. I will most definitely give this a go.
| BlueTemplar wrote:
| I would assume that's quite a hard project, considering that
| even for Netflix it took two decades of research and world-
| class AI competition(s?) with a million dollar prize(s?) :
|
| https://netflixtechblog.com/netflix-recommendations-beyond-t...
|
| (Note that was for the much more USA-focused DVD by mail
| Netflix, so the current problem might be even harder to figure
| out !)
| mason55 wrote:
| https://www.tvtime.com/
|
| TV Time does this somewhat - track what you've watched and
| it'll recommend other stuff for you. It also shows where to
| watch but needs some work on identifying what's free vs paid
| streaming.
| tantalor wrote:
| Pricing:
|
| https://rapidapi.com/movie-of-the-night-movie-of-the-night-d...
|
| Free/Basic: 100/day
|
| Pro: 30k/mo @ $9.90/mo
|
| Ultra: 100k/mo @ $29.90/mo
|
| Mega: 1M/mo @ $99.90/mo
| rusinov wrote:
| Neat idea. But, I am checking Snowfall TV show
| (https://www.movieofthenight.com/series/Snowfall/643) for Russia
| and site says it is not available to stream in Russia, but it is
| in reality, via Kinopoisk
| (https://www.kinopoisk.ru/series/909854/).
| beprogrammed wrote:
| kinopoisk doesn't appear to be one of the 20 services it
| searches.
| rusinov wrote:
| Got it. They get part of their content from Hulu, and if I
| select US, results are correct.
| slickdork wrote:
| This is well done, thank you. As someone who's currently learning
| a language using the refold method[0], this is really really
| helpful.
|
| My one suggestion would be linking to letterboxd [1] in addition
| to imdb -- the user scores on letterboxd are more accurate imo
| than imdb as for what movies i'll like, and i can see what
| reviewers I follow rated a movie, etc.
|
| [0]https://refold.la/ [1]https://letterboxd.com/
| benmanns wrote:
| This is cool! At my day job I work on a lot of entity
| resolution/record linkage and have wanted to do something like
| this for TV/movie content. Maybe also linking against (ahem) non-
| official sources. How are you disambiguating different shows and
| movies and linking them to IMDb/TMDb?
| Saepirist wrote:
| I do have my algorithm to match them with IMDb/TMDb ids, though
| of course it's not always 100% correct.
| gurchik wrote:
| I typed in Mad Men and it shows no streaming services for the
| United States but I know it's available on Prime Video at least.
| Saepirist wrote:
| Currently it only shows free (included with Prime) shows on
| Prime Video. I just checked it and Mad Men is only available
| for buying or via AMC+ channel subscription.
| gurchik wrote:
| Interesting. I've been watching it and it's free (ad-
| supported) with Prime. I have not bought it and don't have
| the AMC+ channel subscription.
| Jnr wrote:
| Nice, maybe someone could finally make a site that shows how many
| and which specific streaming services are needed to cover top20
| movies and tv series of the year.
| jkukul wrote:
| How would you define top 20 movies of the year? What would be
| the criteria?
| y4mi wrote:
| Pick a site and use their list.
|
| I.e. IMDb, trakt.tv, rotten tomatoes etc.
| _jstreet wrote:
| I've been looking for this for some time. It's really useful to
| be able to easily find out which streaming service a film/TV-show
| is available on (or not on!).
|
| Are you making any profit off of this, or is there any way to
| donate/contribute towards the upkeep of the project?
| Saepirist wrote:
| Hi people,
|
| I've been working on this API for sometime now. Thought I'd share
| it here too.
|
| This API allows you to query what's available on 20 streaming
| services (including Netflix, Prime, HBO Max, Disney+, Hulu)
| across 60 countries. You can get the list of shows with filters
| such as release year, genre, IMDb rating etc. You can also get a
| show's global streaming availability via its IMDb or TMDb id. The
| output includes show's IMDb/TMDb ids/ratings, direct url to
| show's page in streaming apps, projected leave date from the
| service if there's one, and other details such as genres,
| directors, cast, runtime, overview, poster etc.
|
| There's also a free plan that allows you to make 100 requests per
| day.
|
| Hope you'll find it useful. And let me know if you have any
| comments or requested streaming services for me to add next.
| ianbutler wrote:
| This is great! This is exactly what I've been waiting for
| someone to do. Google has done this on their "Google TV" device
| for some time, you search for a show and it shows you results
| across all of your streaming apps, it's not perfect but it
| knows where most things are, and I was wondering when I'd see
| an independent person release either an API or an App that does
| something similar. Good stuff!
| Saepirist wrote:
| Thank you!
| jacobwg wrote:
| Looks great! How have you sourced the underlying metadata? From
| past experience, access to such metadata, especially IMDb, was
| rather heavily restricted, requiring a paid licensing agreement
| for access.
|
| Does the Streaming Availability API use properly licensed data
| sources? If so, you might mention this on the marketing site,
| this would be a selling point for certain commercial projects.
| For that matter, it might be worth mentioning if the API can be
| used for commercial projects at all.
| Saepirist wrote:
| Hey, those are some valid concerns.
|
| Scrapping is still a gray legal area but recently there were
| some court decisions in favor of scrapping already publicly
| available data.
|
| The API can be used for commercial projects, it's stated in
| Terms of Use on RapidAPI page but I'm gonna make it more
| visible, thanks for feedback.
|
| Apart from that, obviously I'm not a lawyer and everyone has
| to take their own decision but my two cents is that websites
| that use streaming availability provided in this API should
| be legally fine. In the end all you are providing to your
| users are some links (that are already publicly available) to
| streaming apps.
| e40 wrote:
| Reminds me of https://www.justwatch.com/ which is a non-API
| version. Did they roll their own? Do they use your service?
| dewey wrote:
| It's all in-house, available in 84 countries and supports
| thousands of streaming services. There is an API but it's
| mostly available to larger partners:
| https://www.justwatch.com/us/JustWatch-Streaming-API
|
| Source: I work at JustWatch.
| dzhiurgis wrote:
| Can I use it to find foreign films? Right now my best
| option is yts torrent tracker[0] but there's a ton of hindi
| films which I don't really like.
|
| Would be cool to find all non-english films from europe and
| america.
|
| 0: https://yts.mx/browse-
| movies/0/all/all/8/latest/0/foreign
| atonse wrote:
| I love JustWatch, it's the first place I go when I discuss
| any show with friends, family, kids, anyone.
|
| My kids now, before we buy an episode, say "Dad check on
| JustWatch if we can stream it for free anywhere"
|
| Do you have public examples of others that use your API? I
| have a few ideas that can be used but no time at all to
| build things :)
| dewey wrote:
| One public example would be TMDb (The metadata source
| that also OPs project is using):
| https://www.themoviedb.org/movie/568124-encanto/watch
| elondaits wrote:
| Letterboxd uses their API. I track my watchlist on
| Letterboxd and can filter movies available to stream in
| my subscriber services thanks to their use of the
| JustWatch API.
| timmg wrote:
| Not sure it matters much, but I _had_ been using justwatch
| often. It stopped working for me a while back: search just
| wouldn 't do anything.
|
| I just cleared all my cookies and now it works again.
|
| Not sure how big the problem is, but at least for me, your
| website seemed to be choking on its own cookies.
| e40 wrote:
| What a great website. I use it all the time. Thank you!
| sborra wrote:
| JustWatch is great.
|
| There's a trend in my country (Italy) where old movies are
| being remastered in HD but not published on Blu-ray.
| Instead, they're only made available on one out of a half
| dozen or so streaming services. JustWatch helps me
| immensely to spot new releases like that!
| tcmb wrote:
| I already commented elsewhere, but just wanted to add that this
| is amazingly useful, especially via your 'Movie of the night'
| service. I've bookmarked it and foresee to use it often! Best
| of luck to you, I sincerely hope you get rewarded in some shape
| or form for building this!
| Saepirist wrote:
| Thank you!
| chris_st wrote:
| I'm curious how you keep the results up to date -- found a
| similar thing (without an API), and found lots of wrong results
| as services add and drop movies.
| endymi0n wrote:
| I'm curious as well, as for so many countries ingesting and
| matching that catalogue including error correction is
| currently a full time job for more than a dozen people over 6
| years and 10k+ per month in infrastructure costs.
|
| Will check for canary traps tomorrow as I'm suspecting
| they're just scraping us or our competitors. This is far too
| expensive to build for how cheap they're selling it.
|
| Source: I work at JustWatch.
| chinathrow wrote:
| > Will check for canary traps tomorrow as I'm suspecting
| they're just scraping us or our competitors.
|
| I bite: Aren't you also just scraping others?
| Saepirist wrote:
| Hey, it's nice to see you people under this post.
|
| Regarding your suspicion, surely this is not the first time
| someone making a smaller-scale, more humble and cheaper
| version of a future that some established companies already
| have.
|
| There are two reasons for prices to be cheap:
|
| First one is that I'm a solo developer who has been working
| on this for a couple of years now (I've launched the
| website in 2017, and since 2019 I've been working on to add
| streaming availability information). As I don't have a
| dozen people working on this but only me, it's only natural
| that my expenses are less than an established company.
|
| Second one is that when I decided to add streaming
| availability information to my website back in 2019, all
| the available APIs were either: 1-not available for small
| developers like me, 2-too limited (like Netflix-only APIs),
| 3-too expensive for my student budget (as I was still a
| university student at that time). Due to that situation I
| was in, one of my aims from the start was to make this
| essential data available to small developers/teams/blog
| sites.
| Saepirist wrote:
| The data is updated daily, so new/removed movies shouldn't be
| a problem.
| gukoff wrote:
| Can I somehow get the list of all shows available on Netflix in
| country X but not in country Y?
| Saepirist wrote:
| API doesn't offer that feature out of the box but I believe
| you can pull off that with a bit of extra coding on top.
___________________________________________________________________
(page generated 2022-01-09 23:00 UTC)