[HN Gopher] "YouTube-dl" and "Pirate Bay" back on DDG
       ___________________________________________________________________
        
       "YouTube-dl" and "Pirate Bay" back on DDG
        
       Author : ikt
       Score  : 326 points
       Date   : 2022-04-17 14:41 UTC (8 hours ago)
        
 (HTM) web link (fosstodon.org)
 (TXT) w3m dump (fosstodon.org)
        
       | fortran77 wrote:
       | I wonder if it was unintentional. They rely on Bing's database
       | for their search engine; if these sites disappeared from their
       | data source, might it have disappeared from DDG without any
       | direct intention?
        
       | mlindner wrote:
       | Is there a search alternative that isn't just using Google or
       | Bing underneath? DDG is just Bing and all the recent filtering
       | and what not that Bing has been doing also applies to DDG. Many
       | things that used to show up on DDG no longer do.
        
         | freediver wrote:
         | There are plenty - Kagi, Brave, Mojeek, Yandex, Rightdao,
         | Gigablast...
        
           | mlindner wrote:
           | Doesn't Brave also use Bing underneath?
        
             | xigoi wrote:
             | If their about page isn't lying, about 90% of their results
             | are original.
        
           | sylware wrote:
           | I guess we need an up-to-date wikipedia page about all those
           | alternatives.
        
           | vvf1 wrote:
        
           | DanHulton wrote:
           | Been using Kagi for the last week and the search results have
           | been SURPRISINGLY good. Like, in the "I don't have to scroll
           | to find it" category of good, and no having to deal with spam
           | sites that just copy the actual answer but somehow rank
           | higher than the original, like currently plagues Google.
        
             | AndrewVos wrote:
             | This is probably because Kagi uses Google and Bing indexes.
        
             | ghostly_s wrote:
             | That revenue model is crazy, no way will they last.
        
           | AndrewVos wrote:
           | Kagi uses both Google and Bing index.
        
         | theandrewbailey wrote:
         | > Brave Search is built on top of a completely independent
         | index, and doesn't track users, their searches, or their
         | clicks.
         | 
         | https://brave.com/brave-search-beta/
        
       | Pigalowda wrote:
       | I had never used youtube-dl until the story happened. I
       | downloaded for windows and its speeds were throttled to around
       | 50kB/s. Posters on Stack recommended the ytdlp fork which i tried
       | and it was 5-10 MB/s. Just fyi
        
         | 77pt77 wrote:
         | I started getting systematic
         | 
         | > Connection reset by peer
         | 
         | in a script that I have that downloads podcasts and immediately
         | transcodes to low bitrate opus using ffmpeg.
         | 
         | It's a real bummer...
        
         | burnte wrote:
         | I always find jwz's youtubedown works flawlessly for me.
         | https://www.jwz.org/hacks/youtubedown
        
           | Aissen wrote:
           | How often do you update it ? Because it seems to have seen
           | quite a few update from the beginning of 2022.
        
           | js2 wrote:
           | Do not direct link to jwz.org from HN. Here's a click-safe
           | link:
           | 
           | https://dereferer.me/?https%3A//www.jwz.org/hacks/youtubedow.
           | ..
        
             | Aissen wrote:
             | Does not work either because apparently bit.ly's 301
             | preserves the Referer.
        
               | js2 wrote:
               | Weird, I had tested it. I've updated to a de-referrer
               | site instead.
        
               | Aissen wrote:
               | Might be a browser difference (firefox here). Thanks for
               | the update, it works better.
        
             | bscphil wrote:
             | It genuinely shocks me that there are still people who
             | don't disable sending the referer header cross-origin in
             | the browser: I have not encountered a _single_ website that
             | breaks when setting `network.http.referer.XOriginPolicy` to
             | 1 in Firefox, and only 2 or 3 sites that break when setting
             | it all the way to 2.
             | 
             | It not only completely prevents stuff like this, it
             | profoundly increases your privacy on the web by preventing
             | sites from tracking which domain you came from. There is no
             | good reason any site needs to know that. I am surprised
             | that Mozilla hasn't simply made this the default setting
             | for all users.
        
               | [deleted]
        
             | 77pt77 wrote:
             | > Do not direct link to jwz.org from HN
             | 
             | Why?
        
               | psyc wrote:
               | Because the admin detects HN referer explicitly and
               | presents a joke page.
        
               | mnd999 wrote:
               | Perhaps I'm in the minority, but I find this kind of
               | subversion amusing. Like all good jokes, it's pretty
               | close to the mark.
        
               | psyc wrote:
               | I agree, so it's a safe bet you _are_ in the minority.
        
               | 77pt77 wrote:
               | I've since noticed...
               | 
               | Just copy the link to not send any referrer information.
        
         | UberFly wrote:
         | FireDM (https://pypi.org/project/FireDM) is an awesome front
         | end for ytdlp for those interested.
        
           | belter wrote:
           | GitHub Links from the project page all get a 404.
           | 
           | https://github.com/firedm/ shows no public repos and of
           | course https://github.com/firedm/FireDM gets a 404
        
         | bspammer wrote:
         | Yeah youtube-dl is missing workarounds for throttling
         | implemented by google. youtube-dl is pretty much unmaintained
         | compared to yt-dlp:
         | 
         | https://github.com/ytdl-org/youtube-dl/graphs/commit-activit...
         | 
         | https://github.com/yt-dlp/yt-dlp/graphs/commit-activity
        
           | vmoore wrote:
           | And yt-dlp uses an 'Android API' to stop throttling
        
             | heavyset_go wrote:
             | I expect the ability to use APIs like this will be hindered
             | once remote attestation becomes the norm.
        
             | 1vuio0pswjnm7 wrote:
             | One has to execute functions from base.js^1 to modify the
             | "n" URL parameter and the "sig" parameter to get the
             | fastest download speeds. (One can still download videos
             | with the original n parameter, or without the n parameter,
             | but download speeds will be slower.)
             | 
             | A website that forces users to run Javascript in order to
             | get faster download speeds. This is not a new idea.
             | 
             | 1. https://www.youtube.com/s/player/{player_version}/player
             | _ias...
        
             | [deleted]
        
           | elcomet wrote:
           | This is why I come on HN, I always learn something useful.
        
         | AdmiralAsshat wrote:
         | I noticed this recently, too. I frequently scrape concerts from
         | YouTube by adding them to a private playlist and then running a
         | script on my HTPC to pull down everything from that playlist.
         | As recently as six months ago, pulling down a 2-3GB playlist
         | file (maybe a 30-40 minute concert at 720P) took ten to fifteen
         | minutes. The last time I tried it, it estimated several hours.
        
           | soheil wrote:
           | If you're downloading a playlist you could run it in parallel
           | to achieve your max network bandwidth speed:
           | function ytp() { youtube-dl --get-id "$1" | xargs -I '{}' -P
           | 200 youtube-dl -i --embed-thumbnail --add-metadata -f
           | 'bestaudio[ext=m4a]' -o '%(title)s.%(ext)s'
           | 'https://youtube.com/watch?v={}'; }
        
           | rane wrote:
           | FWIW, just recently wrote a go program that watches a Youtube
           | playlist and downloads new videos to a configurable path:
           | https://github.com/raine/ytdlwatch
           | 
           | I use it to download videos into a Plex library.
        
         | mgdlbp wrote:
         | The workaround for vanilla youtube-dl is to use it with aria2,
         | with options like:                 --external-downloader aria2c
         | --external-downloader-args "--continue --max-concurrent-
         | downloads=3 --max-connection-per-server=3 --split 3 --min-
         | split-size 1M"
         | 
         | (possibly in your config file)
         | 
         | Also, neither --format best nor --format bestvideo chooses the
         | best encoding in all cases; they use bitrate as a heuristic for
         | quality, and a less efficient codec can have higher bitrate but
         | worse quality, resolution, or framerate. The workaround for
         | this is specifying --format with an enumeration of every
         | combination of codec, resolution, and framerate in preferred
         | order, which goes like this:                 --format "(bestvid
         | eo[vcodec^=av01][height>=4320][fps>30]/bestvideo[vcodec^=vp9.2]
         | [height>=4320] ...
         | 
         | Here's a full example (hmm... they're using it with yt-dlp,
         | which I thought had fixed this?):
         | 
         | https://github.com/TheFrenchGhosty/TheFrenchGhostys-Ultimate...
         | 
         | I think there's a bit of variation in the exact order among the
         | config files found online. If you're goals are archival,
         | consider also retrieving metadata, thumbnail, and subtitles in
         | all languages; I also have in my config the options:
         | --verbose       --download-archive ./ytdl-archive.txt
         | --cookies ./ytdl-cookies.txt          --merge-output-format mkv
         | --add-metadata --all-subs --embed-subs
         | --write-info-json --write-thumbnail
         | --no-overwrites --continue
         | --force-ipv4
         | 
         | (the only remaining workaround for age-restricted videos is to
         | give it cookies extracted from a browser with a real Google
         | account logged in)
        
         | altcognito wrote:
         | Am I the only one totally chill with the throttling? Look, I'm
         | not sure if it's the best idea for their own resource
         | utilitization (there are probably peak times when more
         | throttline is better, and quiet internet times when throttling
         | doesn't make sense) As long as they are being gracious hosts
         | and allowing downloads, I'm good with being throttled I guess?
        
           | emteycz wrote:
           | 50 kB/s means you'll be downloading a 10min video like 10
           | hours.
        
             | mdp2021 wrote:
             | > _50 kB /s means you'll be downloading a 10min video like
             | 10 hours_
             | 
             | Very misleading phrasing: you would download in 10 hours _a
             | ~10 hours long video_ , which (of course) could have been
             | downloaded in a fraction of the time.
             | 
             | The throttling has the user download at a speed similar to
             | that required to viewing the video.
        
       | [deleted]
        
       | [deleted]
        
       | nomilk wrote:
       | Why does DDG de-list things in the first place; isn't it in the
       | interest of search engines to be as useful as possible to users,
       | and thus maximise the results provided?
       | 
       | Also curious to know the extent to which Google de-lists things?
       | 
       | Long term, perhaps a decentralised search engine could get around
       | de-listing and provide a more reliable and rigorous search
       | experience.
        
         | RedBeetDeadpool wrote:
         | DDG CEO is clearly biased. Its no longer a neutral platform:
         | 
         | https://twitter.com/yegg/status/1501716484761997318?s=20&t=9...
         | 
         | Before someone chimes in, yes I understand the humanitarian
         | perspective. That's not my point. My point is that DDG is not
         | neutral, and is politically biased.
        
           | RedBeetDeadpool wrote:
           | BTW, if you wanted a biased platform, just use Google, its
           | significantly better in every way.
           | 
           | Ultimately what I'm getting at is: There's no market for DDG.
           | Use Google for biased searches, and use other search engines
           | that are not biased(which excludes DDG) for unbiased
           | searches.
           | 
           | What are the use cases for DDG?
        
             | pmoriarty wrote:
             | DDG respects privacy, or at least claims to. Google has
             | nothing but contempt for it, and thrives on spying on you.
        
         | notriddle wrote:
         | > Why does DDG de-list things in the first place
         | 
         | I don't think anyone wants to use a search engine that never
         | delists anything. Ransomeware, Markov chain junk, plagiarism. A
         | search engine that never delists anything is useless.
         | 
         | The problem is when delisting is used _against_ the end-user's
         | interests.
        
         | notatoad wrote:
         | DDG uses the Bing index. If Bing de-lists something, then it
         | disappears from DDG.
         | 
         | Microsoft (and any other big company) has many competing
         | interests other than just being helpful to users.
        
           | nomilk wrote:
           | > [big companies have] many competing interests
           | 
           | Are the main ones i) reducing competition and ii) managing
           | their reputation?
           | 
           | If so, the case for a decentralised search engine got
           | stronger.
        
             | [deleted]
        
         | jbay808 wrote:
         | I wonder how a decentralized search engine would fight its
         | inevitable SEO war, should it become successful.
         | 
         | I'd expect that all sites wanting to draw traffic would attempt
         | to grab the reins of the search engine to point toward
         | themselves, and the result would be search results ordered by
         | rein-grabbing power.
         | 
         | Not that centralized search engines are immune to this; they're
         | almost as vulnerable (seeing as sponsored search results exist)
         | but the maintainer at least has to balance that with the
         | utility of the search engine overall, to prevent the search
         | engine from falling out of favour.
         | 
         | With a decentralized engine, parties that have deeply invested
         | in manipulating the results will still want the engine to be
         | popular too, but I'm not sure how you resolve the prisoners
         | dilemma there as a whole.
        
           | forgotmypw17 wrote:
           | > I wonder how a decentralized search engine would fight its
           | inevitable SEO war, should it become successful.
           | 
           | > I'd expect that all sites wanting to draw traffic would
           | attempt to grab the reins of the search engine to point
           | toward themselves, and the result would be search results
           | ordered by rein-grabbing power.
           | 
           | I would venture to say that a combination of allow-lists and
           | block-lists from trusted parties, ranked using some kind of
           | distributed web-of-trust system would work reasonably well.
        
             | KMag wrote:
             | Ages ago, I was a professional P2P developer, and I vaguely
             | remember some of the research papers on P2P censorship-
             | resistant reputation systems. Generally, you have public
             | signing keys that sign ratings (say, -1.0 to 1.0) for both
             | content and other raters. These signed ratings collections
             | are then pushed into a distributed hash table.
             | 
             | The basic idea is when you rate something, your client also
             | looks up in the DHT other people who have rated the same
             | content with similar ratings. Your client then pulls the
             | latest ratings collections from those people, and computes
             | the cosign distance between your ratings and their ratings
             | (over the intersection of content that both of you have
             | rated). Periodically, your client signs and publishes an
             | updated ratings document, where the rating for other raters
             | is the cosign distance. The cosign distance, the size of
             | the ratings intersection set, and maybe some other factors
             | go into deciding which raters get published out in your
             | ratings update.
             | 
             | When you query for the rating for a given piece of content,
             | your client grabs the list of ratings for that content from
             | the DHT. It then pulls the latest ratings published by
             | those raters, computes cosign distance, and then does
             | something similar to Djikstra's shortest-path algorithm to
             | recursively search the DHT using these cosign distances as
             | weights. In general, the DHT wouldn't have many signatures
             | stored under the content's hash, but by recursively
             | following the graph of other raters, your client hopefully
             | finds other raters that rate things similarly to you and
             | have rated this content. The path weight to a given rater
             | is the product of cosign distances, and so by using a
             | priority queue for querying, you get something close to a
             | breadth-first search of the ratings graph. Once your client
             | has accumulated enough weight of ratings for the given
             | content, it stops and shows you the weighted average of the
             | ratings (and maybe the weighted std. dev. is displayed as a
             | confidence score to power users who have enabled it).
             | 
             | Presumably, the UI for the ratings system maps 0 to 5 stars
             | to 0.0 to 1.0 (probably not linearly, more likely the
             | client locally keeps a histogram of the user's ratings and
             | then maps the star rating back to a percentile rating), and
             | the "spam" button rates the content as -1.0.
             | 
             | The tricks come down to the metrics used for how the DHT
             | decides priorities for cache eviction of the per-content
             | ratings and also the per-rater ratings. You don't want
             | spammers or other censors to be able to easily force cache
             | eviction. Getting cache eviction metrics right is the key
             | to having the system scale well while also preventing
             | spammers/censors from evicting the most useful sets of
             | ratings.
        
         | tdiff wrote:
         | Banning Russian news outlets from search results was generally
         | supported here on HN.
        
           | gscott wrote:
           | There is no war just a special operation.
        
             | tonguez wrote:
             | we must condemn russias unprovoked genocide of neo nazis
        
               | suction wrote:
               | Wait, did Russians start committing suicide?
        
           | ghostly_s wrote:
           | That's certainly not how I remember that thread going.
        
         | matheusmoreira wrote:
         | Copyright industry pressure.
        
       | mrtweetyhack wrote:
        
       | PaulKeeble wrote:
       | The youtube-dl homepage has returned to the listings as has
       | thepiratebay. However they still aren't indexing thepiratebay or
       | youtube-dl.org's contents so you can't search within the sites
       | you only get the homepage. The complaint the other day was about
       | the indexes too, so its only partially fixed.
        
         | yegg wrote:
         | That's not actually true. Our site search is having issues, so
         | better to just add the site name to the search, but note that
         | youtube-dl.org is just one page and for other sites (that are
         | essentially vertical search engines) you're better off going
         | directly to them since their index is going to be more up to
         | date.
        
       | ok123456 wrote:
       | I'm probably going to stop using DDG now. I don't want my results
       | filtered in anyway because of pearl clutching over 'piracy' or
       | whatever.
        
         | vvf1 wrote:
        
           | ok123456 wrote:
           | Getting a blank "Please reload the page, something went
           | wrong" page instead of rendering anything.
           | 
           | Seems to have an uncaught exception trying to use the beacon
           | API (which I have disabled).
           | 
           | You.com looks like junk.
        
           | mdp2021 wrote:
           | Discussed a few months ago at
           | https://news.ycombinator.com/item?id=29165601
        
         | mysterydip wrote:
         | What are you going to use instead?
        
           | ok123456 wrote:
           | yandex
        
             | CharlesW wrote:
             | Yandex is the worst alternative you could pick.
             | https://www.protocol.com/policy/yandex-gershenzon-qa
        
               | tonguez wrote:
               | "...tech giant Yandex has a handshake deal with
               | government authorities to limit what news outlets the
               | site will pull onto its homepage"
               | 
               | wow that sounds so incredibly different from the
               | alternatives
        
               | ok123456 wrote:
               | Just don't get all your news from yandax??
        
           | s__s wrote:
           | Brave has a new search engine.
        
       | rubyist5eva wrote:
       | Switched to Brave Search - not looking back. I don't even care if
       | the index sucks, it's been "good enough" and it has bangs.
        
         | vvf1 wrote:
        
       | yoyopa wrote:
       | if all you nerds stop talking about it maybe it will stay up
        
       | wyager wrote:
       | DDG has been de-listing a ton of stuff recently. They recently
       | de-listed rdrama, a reddit-trolling website that came up on HN
       | last year. What, exactly, do they think people use them for? If I
       | wanted "result curation" or whatever euphemism for censorship, I
       | would just use Google.
        
         | Firmwarrior wrote:
         | Man, that's interesting, they removed rdrama.net but left
         | KiwiFarms up, and now the main result for "rdrama" is the
         | KiwiFarms thread about it
        
         | Marsey wrote:
         | Luckily searching "Marsey the Cat" on Bing still yields the
         | correct knowyourmeme and Twitter pages.
        
         | scarburato wrote:
         | Bing delisted rdrama too. I think it is related somehow (?)
        
           | BurdensomeCount wrote:
           | Yep, I think the delisting comes from Bing, not DDG since DDG
           | just pulls all their results from Bing since they stopped
           | also using Yandex.
        
           | mgdlbp wrote:
           | I noticed some time ago that Google refuses to offer up
           | Encyclopedia Dramatica unless you use the site: operator.
           | 
           | There was some drama in 2010 over ED being censored in
           | Australia, but it looks like Google has since quietly
           | delisted it completely.
        
         | kevin_thibedeau wrote:
         | They've removed Yandex and are now at the mercy of Bing's
         | censorship.
        
           | BurdensomeCount wrote:
           | Amazing that nowadays if you want the best "uncensored"
           | results you have to go to the Russian Yandex (at least on all
           | topics unrelated to Russia). How did our society get to such
           | a point...
        
             | markdown wrote:
             | If that isn't a rhetorical question... capitalism.
        
               | mpalczewski wrote:
               | > capitalism
               | 
               | If capitalism caused censorship you would expect the
               | least capitalist places to have the least censorship but
               | the opposite is true.
        
               | kenoph wrote:
               | If capitalism causes censorship, that doesn't mean it's
               | necessarily the worst at causing censorship.
               | 
               | Additionally, we don't live in a stationary society, so,
               | whatever capitalism did or did not cause in the past
               | might not apply as-is today.
        
               | stjohnswarts wrote:
               | Capitalism (as practiced by democracies) allows you to
               | start an alternative site that doesn't censor results as
               | well. Totalitarianism (Communism) would not have. Choose
               | your poison from amongst the various governments that
               | have succeeded (for a while) in the past as I don't think
               | Utopia is a possibility with humans as messed up as we
               | are.
        
               | kenoph wrote:
               | Related:
               | https://en.wikipedia.org/wiki/Inverted_totalitarianism
        
         | ldiracdelta wrote:
         | They've "memory-holed", 1984-style, other conservative sites as
         | well. Just another narrative-enforcing search-engine.
        
           | uuyi wrote:
           | I prefer to say "shite removal" after the last half a decade.
           | 
           | At least TPB adds value to peoples lives.
        
             | ldiracdelta wrote:
        
             | ldiracdelta wrote:
             | Yes, the Overton Window must be enforced. We have always
             | been at war with East Asia.
        
             | somenameforme wrote:
             | It depends on what you want out of a search engine. Do you
             | want a search that tries to give you what it thinks you
             | want? Or do you want a search that functions more like a
             | library index where you're going to get what you searched
             | for, even if that might ironically not really be exactly
             | what you wanted - so the onus of creating a more correct
             | search term is on you.
             | 
             | This isn't a rhetorical question of course. There are major
             | arguments for both. But I think a pretty good chunk of DDG
             | users were more often after the library index than what
             | ideally would be a librarian recommendation, but in reality
             | is more like a stereotypical used car salesman style
             | recommendation.
        
               | fallingknife wrote:
               | I would be fine with either. But what I really don't want
               | is a search engine that gives me what it thinks I should
               | want.
        
             | ta8903 wrote:
             | Removal of TPB was a positive too. Can't imagine how much
             | loss of revenue these pirate sites have caused over the
             | years.
        
             | colordrops wrote:
             | The whole point is to have a place where you can see shite
             | if you want. I don't need an online nanny.
        
               | HeckFeck wrote:
               | I might agree with shite. Or I might wish to read it for
               | my own amusement. Or perhaps I want to argue against the
               | shite, in which case, I need to know what shite is out
               | there. In any case, it doesn't matter. I only need the
               | search engine to be an intermediary, not a curator.
               | 
               | Besides, if someone accidentally searches for shite, he
               | can revise his search terms. Just like how you might
               | reword something if a listener was confused.
               | 
               | This "curation" just doesn't need to exist. But that's a
               | moral argument and those applying censorship aren't
               | moral, so won't be partial to it.
        
         | asdff wrote:
         | Isn't DDG just reskinned bing anyhow?
        
       | pkdpic wrote:
       | Sorry to be that guy but what was going on? This is alarming as
       | someone who uses DDG and youtube-dl kinda too much probably.
       | 
       | Also if anyone else is slightly hung over and searches DDG in DDG
       | and gets super confused for three seconds because DDG is a rapper
       | apparently just know you aren't alone. I'm right here with you,
       | whoever you are.
        
         | [deleted]
        
         | jt2190 wrote:
         | From an April 17 Twitter thread from Gabriel Weinberg at
         | DuckDuckGo:
         | 
         | > ... [W]e are not "purging" YouTube-dl or The Pirate Bay and
         | they both have actually been continuously available in our
         | results if you search for them by name (which most people do).
         | Our site: operator (which hardly anyone uses) is having issues
         | which we are looking into.
         | 
         | (Note that "site:" in his comment is how you restrict DDG
         | searches to a specific domain e.g. "site:example.com")
         | 
         | https://twitter.com/yegg/status/1515636218691739653
        
           | naoqj wrote:
           | Did the original complaints of those sites disappearing
           | mention the usage of the site: filter?
        
             | jt2190 wrote:
             | Yes.
             | 
             | > For example, searching for "site:thepiratebay.org" is
             | supposed to return all results DuckDuckGo has indexed for
             | The Pirate Bay's main domain name. In this case, there are
             | none.
             | 
             | > This whole-site removal isn't limited to The Pirate Bay
             | either. When we do similar searches for ["site:1337x.to",
             | "site:NYAA.se", "site:Fmovies.to", site:"Lookmovie.io"],
             | and ["site:123moviesfree.net"], no results appear.
             | 
             | https://torrentfreak.com/duckduckgo-removes-pirate-sites-
             | and...
        
           | OJFord wrote:
           | That's incomplete though, when I tried it (in response to
           | seeing it posted here a couple of days ago) I couldn't get
           | any yt-dl.org search results, i.e. adding site: appeared to
           | function correctly - no results.
        
           | codetrotter wrote:
           | >> Our site: operator (which hardly anyone uses) is having
           | issues which we are looking into
           | 
           | I noticed problems with their site: operator too earlier
           | today, and still now as well. In my case, when I used it in a
           | search I saw that the word "site" itself was also bolded in
           | some results. So it looks like it is using the operator
           | itself also as a search term, which it shouldn't.
           | 
           | I find it surprising that hardly anyone uses it though.
        
             | whoopdedo wrote:
             | When you handle 10 million searches a day it's easy for
             | even large groups of users to get lost in the crowd.
        
         | [deleted]
        
         | fuckcensorship wrote:
         | These links should provide the additional context you're
         | looking for:
         | 
         | https://news.ycombinator.com/item?id=31044587
         | 
         | https://nitter.net/i/status/1515635886855233537
        
       | icare_1er wrote:
       | Maybe DDG understood that they have no purpose if all they have
       | to propose is Google's bad sides and censorship, without Google's
       | search power...
        
         | pmoriarty wrote:
         | DDG's selling point was always privacy.
        
       | whoopdedo wrote:
       | Still missing from Bing. So this is DDG inserting an override.
        
         | bilkow wrote:
         | I also find that searching Bing does not result in youtube-
         | dl.org, only the repository, while DDG returns both, but Bing
         | is not the ONLY source for DDG results, at least according to
         | them.
         | 
         | https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
         | 
         | "We also of course have more traditional links in the search
         | results, which we also source from multiple partners, though
         | most commonly from Bing (and none from Google)."
         | 
         | I've tried comparing Google, Bing and DDG on a private window
         | before, and I didn't find Bing and DDG more similar than Bing
         | and Google. Searching for monkey:
         | https://news.ycombinator.com/item?id=27598329
        
         | alyandon wrote:
         | That is really strange - I just searched for both on bing.com
         | and got relevant results back. Search bubble?
        
           | whoopdedo wrote:
           | Indeed. My search was done in a fresh private window on a
           | computer I never typically use Bing on.
           | 
           | Also, throughout this whole situation I always got the Github
           | page as the first result for "youtube-dl".
        
           | jug wrote:
           | Or search region? I've had different results before based on
           | your Bing/DDG region setting.
        
       | behnamoh wrote:
       | Sometimes when the damage is done, it's done. I'm never going to
       | use DDG ever again.
       | 
       | If this decision was because of legal pressure by Google, I don't
       | see how that got resolved in a matter of days. Which means it
       | wasn't because of Google, but rather a poor decision made by DDG
       | management. How can people trust their product now?
        
         | vmoore wrote:
         | > I'm never going to use DDG ever again.
         | 
         | The removals were a rarity. It's not as if you can't add a `!g`
         | bang query to redirect to Google if you can't find something.
         | And DDG is rampant with all sorts of stuff that shouldn't be
         | there, so I don't think they're hellbent on censorship.
        
           | behnamoh wrote:
           | !g leads to Google, who also blocks yt-dl and other similar
           | tools.
           | 
           | Edit: I was wrong.
        
             | oefrha wrote:
             | Uh, what? youtube-dl.org is the first result for youtube-dl
             | on Google, followed by GitHub repo, ytdl-org.github.io,
             | Wikipedia, etc.
             | 
             | TPB is the first result too. IIRC at one point searching
             | for TPB only returned proxy sites, but that doesn't appear
             | to be the case now.
        
             | bqmjjx0kac wrote:
             | Any source on that? I just searched both "youtube-dl" and
             | "pirate bay" and got reasonable-looking results.
        
               | ttybird2 wrote:
               | You don't get "reasonable-looking" results for "pirate
               | bay" on google.
        
               | ghostly_s wrote:
               | My first result is https://thepiratebay.org and the rest
               | of the page is proxies.
        
               | ttybird2 wrote:
               | Weird, I only get shady proxies and the wikipedia page.
        
           | spiderice wrote:
           | > if you can't find something
           | 
           | You don't always know what you don't know.
           | 
           | If I don't know about YouTube-dl and I search "download
           | YouTube command line", how am I going to know that ddg is
           | hiding the best result from me?
           | 
           | This definitely isn't a small annoyance kind of problem, in
           | my eyes. It's a deal breaker. I'll never use ddg again. If
           | they're going to censor like Google does, then I'm going to
           | use Google because it generally has better results. Ddg needs
           | to offer something beyond what Google offers to make up for
           | their bad results, and they aren't doing that.
        
             | lamontcg wrote:
             | > Ddg needs to offer something beyond what Google offers
             | 
             | If you're searching on google for medical terms it builds a
             | profile on you, and you should be concerned that they could
             | be selling that information to insurers, directly or
             | indirectly.
        
               | 14 wrote:
               | I've been curious about this as a health care worker. The
               | number of times I've looked up a client condition to
               | better help the person I can't count. So is googles
               | profile of me tainted?
        
               | zedadex wrote:
               | They probably have a good idea of what those profiles
               | look like, even if they don't already know you work in
               | healthcare.
        
               | lamontcg wrote:
               | I would be somewhat worried that information could be
               | abused now or in the future. Since that data is
               | necessarily going to be noisy, insurers probably aren't
               | going to make black and white decisions based on it, but
               | they could score you somewhat worse over it. You could
               | maybe trust them that their algorithms would be able to
               | determine that since you work in health care and are
               | probably in the top 5% of people who search health terms
               | that your individual data is polluted by your job, but I
               | would never trust an insurer's black-box algorithms. They
               | only need to be statistically correct over the
               | population, they can always fuck you over individually
               | and still make a fantastic profit for themselves.
               | 
               | Is this really going on? Maybe not, but is it worth it to
               | ignore the risk? Can you just use DDG so it isn't a
               | question?
        
               | 14 wrote:
               | I have long ago switched to ddg but was curious about it
               | as a past google user
        
               | Brian_K_White wrote:
               | probably not, because probably they also know your
               | occupation, or you crossed a threshold of too many to be
               | normal/direct etc.
               | 
               | (except in reality it's less simple than "they know your
               | occupation", it's a huge cloud of data points that an ai
               | makes correlations and assosciations that no human
               | actually knows. It means the searches would also be
               | weighted by indirect things like, not only your
               | occupation but your assosciations. Say you don't have the
               | medical occupation, but your computer makes a lot of
               | medical searches, because your roomate in your college
               | dorm has a medical major etc. And right now, the ais are
               | still pretty stupid and absolutely making a lot of
               | obvious unsafe conclusions, but they also do get more and
               | more spooky every day.)
               | 
               | But this doesn't make it any better. If they did a
               | perfectly accurate job of profiling you, that is not
               | better than doing an inaccurate job.
               | 
               | That much insight is like being married to someone, where
               | they intimately know all your biases and motivations,
               | know all your buttons, know how to manipulate you, know
               | how to weigh any opinions you might express against their
               | knowledge of where you got every idea you ever had,
               | 
               | except it isn't a marriage, and they aren't subject to
               | all that same vulnerability to you, and they aren't even
               | a human but a corporation, and they have this intimate
               | knowledge of everyone not just one spouse or sibling or
               | best friend.
        
         | ufo wrote:
         | The official stance according to DDG was that it was a bug and
         | they fixed it.
        
           | fallingknife wrote:
           | Funny how none of these random "bugs" never delist major
           | mainstream media.
        
             | tedunangst wrote:
             | Would you even know if DDG stopped returning results for
             | site:cnn.com?
        
       ___________________________________________________________________
       (page generated 2022-04-17 23:00 UTC)