[HN Gopher] How to bypass Cloudflare bot protection
___________________________________________________________________
How to bypass Cloudflare bot protection
Author : simonpure
Score : 342 points
Date : 2021-04-04 10:53 UTC (12 hours ago)
(HTM) web link (jychp.medium.com)
(TXT) w3m dump (jychp.medium.com)
| luckylion wrote:
| As mentioned in the article, it's easy to mitigate by not only
| checking that the request is arriving at your endpoint from an
| official CF IP, but to also check that it does not contain a CF-
| Worker header (or contains one with your domain).
|
| The limits on workers' sub requests (e.g. requests are done by
| host name only, can't use IPs with a different host name) make it
| somewhat cumbersome to set up reverse proxy workers (which was my
| use case), but the tech itself is just great, works very reliably
| and the delay, while noticeable, is tiny and never bothered us.
| freedomben wrote:
| I'm glad I'm not the only one that can't type Cloudflare
| reliably. Couldflare is an important part of my architecture.
| Same with Cloudlfare. No CloudFalre or Cloudfalre yet but there's
| still time ;-)
| p2detar wrote:
| Cool. I'd be interested in any hints about bypassing Akamai's
| protection bot. They use some annoying js machinery to detect
| browser sensors and stuff.
|
| Stumbled on that some days ago. Quite annoying and there doesn't
| seem to be a easy hack.
| userbinator wrote:
| More precisely, "bot and minority browser/system" protection...
| when I come across a site that has it, I will just go somewhere
| else, or use Google's cache if absolutely necessary.
| minitech wrote:
| This just sounds like a way to make a proxy with Cloudflare
| workers, which aren't considered by Cloudflare's filters to be as
| suspicious as Tor yet. You could do the same with any new
| VPS/cloud function provider. Am I missing some actual bypass?
|
| If the X-Forwarded-For value from the request is forwarded by
| Cloudflare's proxies unaltered, giving special privileges to
| Cloudflare workers, that's a problem, but it's not clear whether
| "a site who display your headers" is protected by Cloudflare
| _and_ displaying an unaltered X-Forwarded-For. I don't expect
| Cloudflare to have made this oversight, and the e-mail response
| from them matches: "All workers subrequests go through Cloudflare
| again, and therefore you won't be able to bypass any restrictions
| directly."
| bassdropvroom wrote:
| > You could do the same with any new VPS/cloud function
| provider.
|
| I'm not so sure this is true. Not long ago Cloudflare's CEO was
| making fun of people trying (and failing) to bypass
| Cloudflare's anti-bot measures. This allows to bypass this
| because as far as Cloudflare is concerned, the requests are
| coming from a trusted source. Setting up a proxy with a
| provider that isn't Cloudflare, means the requests aren't
| coming from a trusted source any more.
| jsnell wrote:
| > Cloudflare's CEO was making fun of people trying (and
| failing) to bypass Cloudflare's anti-bot measures
|
| Do you happen to have a link?
| bassdropvroom wrote:
| https://mobile.twitter.com/eastdakota/status/12732778391026
| 5...
| judge2020 wrote:
| The thing is that it isn't being seen as a trusted source.
| The same global rate limits apply, and the `Cf-Connecting-Ip`
| (which is the one you should use for security) request header
| still can't be forged and is passed to the end website
| correctly.
| tyingq wrote:
| I have seen workers code that appears to remove Cf-
| Connecting-Ip prior to a fetch(), though no idea if that
| works. Maybe it just gets replaced with the workers ip?
| Like: router.route('/proxy/:url', '*',
| async (event) => {
| event.request.headers.delete('cf-connecting-ip');
| event.request.headers.delete('x-real-ip'); return
| await fetch(event.parameters.url,event.request);
|
| Edit: Tried several variations of this. It doesn't work.
| The cf-connecting-ip header is always in the fetch request
| with the right data. It's a bit confusing because some of
| the Cloudflare tools don't show you all the headers.
| SergeAx wrote:
| VPS providers are not free, and so is traffic produced by VPS.
| On the other hand, 100k req/day is a generous limit, easily
| expandable with additional accounts.
| minitech wrote:
| GitHub Actions and other CI, Heroku, AWS, GCP, Azure, Fly.io,
| etc. have free tiers. This is another one of those that, when
| misused enough, will develop a bad reputation too. (Except in
| the case of Cloudflare workers to Cloudflare bot blocking,
| they're in potentially an even better position to distinguish
| legitimate traffic because they have information from both
| ends.)
| spzb wrote:
| > Am I missing some actual bypass?
|
| No, you're not. Which is why Cloudflare were correct to say
| this isn't a security issue.
| MaheshC wrote:
| If somebody from cloudflare is readying this. This is great.
| Please do not fix it, or if you do, provide a way for legitimate
| limited bypassing when, for instance, using puppeteer from an AWS
| instance.
| SergeAx wrote:
| > At this point you can request any site using Cloudflare
|
| I don't see why I can't request any site on the internet
| whatsoever, having legit Cloudflare IP address. Is there some
| limitation of Cloudflare workers?
| judge2020 wrote:
| 1. They limit subrequests to 50 per workers request
|
| 2. they have abuse alarms for obvious "spray requests by
| utilizing Workers" behavior (although we don't know how well
| these might work)
|
| 2. The Cf-Connecting-Ip header is passed with the correct
| originating request IP, so if a website at least partially
| trusts CF (without directly using CF's proxy) they can match
| based on this header if the REMOTE_ADDR is Cloudflare. They
| could also block requests with the header Cf-Worker[0].
|
| 3. You can't forge the HOST header on CF, so if a website does
| use CF you can't bypass the actual CF proxy firewall by
| fetching the IP and setting the HOST header yourself (you can
| do it within your zone though)[1].
|
| 0: https://community.cloudflare.com/t/is-the-cf-worker-
| header-f...
|
| 1: https://developers.cloudflare.com/workers/runtime-
| apis/reque... (resolveOverride)
| toredash wrote:
| I believe you can with Enterprise plan, override the Host
| header that is.
| kentonv wrote:
| Hi, I'm the tech lead of Cloudflare Workers.
|
| This article contains several misunderstandings.
|
| If a Cloudflare customer has configured their origin server to
| respond only to Cloudflare IPs, then they MUST also verify that
| the "Host" header on any request actually matches their domain
| name. If they do not verify the Host header, then anyone can sign
| up for Cloudflare and simply configure their DNS to point to the
| victim's origin IP address, and requests will be routed there --
| but will have the attacker's domain in the Host header. Workers
| is not needed for such an attack. This attack has always been
| possible and is common to basically all CDNs. Fundamentally, the
| CDN has no way of knowing if the origin server that a user has
| configured really belongs to them -- the CDN can only _tell_ the
| origin (via the Host header) what customer it _thinks_ it is
| serving, and expect the origin not to accept requests that were
| on behalf of a different customer.
|
| Instead of IP-based authentication, we strongly recommend using
| mTLS-based authenticated origin pulls (with a zone-specific key
| pair) or Argo Tunnel, as these methods are much more secure.
|
| As long as the origin server is verifying via one of the above
| means (IP+host header, AOP, or Argo Tunnel) that the request was
| processed by Cloudflare on behalf of the customer's zone, then
| the attack described in the article doesn't accomplish anything.
| If a Worker makes a request to a hostname that is on Cloudflare,
| then all of the target host's Cloudflare security settings will
| apply to that request the same as if the request came from an
| external client.
|
| Additionally, as mentioned in the article, any requests coming
| from a Worker will have the CF-Worker header identifying the zone
| which sent the request. If a customer suspects abusive requests
| coming from Workers, they should report it to us.
|
| ~~The article claims that the X-Forwarded-For header can be
| forged, but this is not true if the target host is itself a
| Cloudflare customer.~~
|
| EDIT: What I said about X-Forwarded-For seems to be incorrect.
| CF-Connecting-IP has always been the header we use to identify
| the client IP, and it cannot be forged. X-Forwarded-For seems to
| have more complicated and subtle behavior that I'm not familiar
| with. I'm investigating.
| aeyes wrote:
| > If a Cloudflare customer has configured their origin server
| to respond only to Cloudflare IPs, then they MUST also verify
| that the "Host" header on any request actually matches their
| domain name.
|
| The worker code sends the correct host name.
|
| https://github.com/jychp/cloudflare-bypass/blob/dd7c1cfc2e6b...
| kentonv wrote:
| Right. If the hostname requested by the worker is a host that
| is itself protected by Cloudflare, then all of that host's
| usual security settings apply. Nothing is bypassed.
|
| The author of the blog post seems to have been confused by
| the fact that different behavior happens if the target host
| is on Cloudflare vs. not on Cloudflare.
| r1ch wrote:
| > If a Cloudflare customer has configured their origin server
| to respond only to Cloudflare IPs, then they MUST also verify
| that the "Host" header on any request actually matches their
| domain name.
|
| This could be improved considerably if Cloudflare removed
| support for insecure TLS options. "Full" SSL bypasses
| certificate hostname verification on the origin server which
| could prevent such an attack at the TLS level. Every time I've
| added a new zone to Cloudflare, Full or Flexible (ie, no SSL)
| has been the default. The only safe option is Full (Strict) but
| I'd bet good money that the majority of Cloudflare customers
| aren't using that.
| abiro wrote:
| > If a Cloudflare customer has configured their origin server
| to respond only to Cloudflare IPs, then they MUST also verify
| that the "Host" header on any request actually matches their
| domain name.
|
| Where is this documented?
| toredash wrote:
| I'm 100% sure it isn't. Cloudflare docs are not great, and as
| a Enterprise customer I spend more time with support than
| needed as a result.
| MuffinFlavored wrote:
| From the article:
|
| > X-FORWARDED-FOR: 1.2.3.4 (yes, we can override this header
| with whatever we want !)
|
| Why aren't you guys disallowing that field from having any
| external value?
| modoc wrote:
| Because it's legitimate to have values before the request
| gets to CloudFlare, for instance if it was routed out through
| a corporate proxy, etc..
| MuffinFlavored wrote:
| Ouch. I wonder if it's time to cryptographically sign those
| values or add a checksum or something.
| acdha wrote:
| Basically this is why you want to go full zero trust: if
| you use mutual authentication over TLS, this becomes much
| less of a concern since you're not trying to craft
| policies based on IP addresses with multiple
| intermediaries.
| kentonv wrote:
| So we're _supposed_ to be appending the client 's IP to
| whatever the client sent for X-Forwarded-For, documented
| here:
|
| https://support.cloudflare.com/hc/en-
| us/articles/200170986-H...
|
| For better or worse, this is how X-Forwarded-For works, by
| convention: each forwarder is supposed to append a value.
| This means that, even in a correct implementation, an
| attacker can specify an arbitrary prefix of values. The
| consumer must be very careful about which part of the chain
| they trust.
|
| But the report here seems to be suggesting that we're
| mishandling this appending in some cases. We're investigating
| and if true will fix ASAP.
|
| Because X-Forwarded-For is surprisingly complicated, we
| recommend using CF-Connecting-IP instead. As the docs say:
| "To restore original visitor IP addresses at your origin web
| server, Cloudflare recommends your logs or applications look
| at CF-Connecting-IP or True-Client-IP instead of X-Forwarded-
| For since CF-Connecting-IP and True-Client-IP have a
| consistent format containing only one IP."
| orf wrote:
| > If a Cloudflare customer has configured their origin server
| to respond only to Cloudflare IPs, then they MUST also verify
| that the "Host" header on any request actually matches their
| domain name.
|
| Out of interest do you provide any automatic checks for this?
| It seems like it would be trivial to add some kind of
| verification step that simulates a request with a bogus host
| header and checks the status code of the response.
| kentonv wrote:
| I don't think we do, but that's a good idea! I think it could
| be automatic. We'd have to first detect that the origin
| server appears to be blocking non-Cloudflare IPs, by seeing
| if it responds to requests from some random non-CF IP
| address. Then we'd verify that it refuses to respond to a
| request for some dummy domain. If it serves the site's normal
| content when the host name is from the wrong domain, then we
| could alert the customer that they seem to have an insecure
| configuration. I will suggest this internally...
| orf wrote:
| The "normal content" bit is possibly quite hard to detect.
| You could do some kind of distance calculation between
| "safe" responses and "unsafe responses", but it seems like
| there would be a bunch of edge cases given the variety of
| sites you host.
|
| Anyway, thanks for hearing my suggestion!
| kentonv wrote:
| Yeah, I think I'd just look for HTTP status code.
| r1ch wrote:
| Assuming you're using HTTPS (and if not, why?!), this is
| checked automatically as part of the TLS handshake. The only
| downside is Cloudflare makes it easy to accidentally disable
| this check as "Full SSL" bypasses certificate verification.
| meowface wrote:
| Yeah, I do feel like there should be more visible
| encouragement to use "Full (Strict)" when someone adds a
| new domain for the first time. Especially now that it's so
| easy to provision free SSL certificates via Let's Encrypt
| etc.
| EB66 wrote:
| What would CloudFlare automatically check the Host header
| against? The Host header would be set to the attacker's
| domain (which is in CloudFlare) and, per OP, CloudFlare does
| not verify ownership of the origin server IP.
|
| It would be a nice feature if CloudFlare customers could opt-
| in for origin server IP verification. Once the origin IP is
| verified, it would prevent other CloudFlare users from using
| the IP as their own origin. However, I understand why
| CloudFlare doesn't verify origin IPs -- it would require a
| verification process that doesn't rely solely on uploading a
| verification file to a web accessible directory. It'd be a
| bit difficult to keep the verification process simple for
| end-users if they aren't running a web server. Or maybe I'm
| wrong and non-HTTP use cases don't need to be considered and
| a verification file uploaded to a web accessible folder would
| work fine.
| EE84M3i wrote:
| I could understand verifying domain names, but verifying IP
| addresses seems like it would be an extremely complex
| feature for very little benefit.
|
| You'd have to have implement it in-line with the dns
| resolution in the server before talking to the origin,
| right? There's no other time you could reasonably do it
| (DNS changes), and it would have to be checked on _all_
| customers requests, not just the ones that opted into the
| feature (because those other customers could be the
| "attacker"!). Of course, you would cache it and use an
| efficient lookup scheme, but in general cloud services
| really don't like creating features that have non-localized
| effects like this.
|
| Not to mention this sounds like a support nightmare foot
| gun for enterprise customers.
| orf wrote:
| Somewhere in the interface you'd have a button that when
| pressed would send a request to the specified origin server
| with a Host header set to "foobar.com". If the status code
| is 200 you'd display a warning saying "perhaps you should
| re-read our documentation about verifying host headers".
|
| I can think of a whole bunch of corner cases to this, but
| that's the general jist.
| EB66 wrote:
| But in order to safely enable that check for all requests
| that go from CloudFlare to the origin IP, you would first
| need to verify ownership of the origin IP.
|
| You wouldn't want users who don't own the origin IP to be
| able to turn on that sort of check and potentially cause
| a service disruption. It could be the case that the
| legitimate owner already has a Host header mismatch in
| their HTTP requests/responses.
| floatingatoll wrote:
| Presenting a warning about possible misconfiguration to
| the customer need not create a service disruption.
| EB66 wrote:
| We're talking about a situation where an attacker would
| create the service disruption by enabling Host header
| checks on an origin server that the attacker doesn't own.
| So a warning wouldn't help because that warning would be
| displayed to the attacker.
|
| The logic is tricky, but in the end origin server
| ownership verification is what's required to safely
| support Host header checks.
| floatingatoll wrote:
| If the attacker is logged into the Cloudflare control
| panel, where presumably the warning would be displayed --
| where _else_ could it be displayed? certainly not on the
| content served to end users -- then, yes, the attacker
| could clear the warning. They could also change the
| origin servers, or do countless other things to disrupt
| service, that do not require modifying an origin server.
| I consider that an acceptable failure case for the
| warning.
|
| I'm not arguing for or against ownership verification,
| but there is opportunity to improve here that does not
| depend on the question of ownership verification.
| thesis wrote:
| This seems like the same kind of reason we're not using
| Cloudflare Access to protect some internal resources.
|
| We'd love to use Argo Tunnel but unfortunately it breaks with
| an OPTIONS request for us. We've tried opening tickets with
| Cloudflare.
| Craighead wrote:
| Medium using Cloudflare with articles explaining how to abuse
| Cloudflare is interesting. Hopefully now that the position is
| tested, Cloudflare won't pressure medium to take down this
| article, even if it does contain misrepresentations.
| njbooher wrote:
| Once upon a time: https://njbooher.github.io/blog/cloudflare-
| workers-ip-spoofi...
| kentonv wrote:
| Yep, I worked on that one. Thank you for reporting it.
|
| Ironically, the purported problem in this case is the
| opposite: Because we _don 't_ attribute the request as coming
| from the original client (in order to avoid the security
| problem you reported), a client that has bad reputation
| associated with their IP can proxy through a worker to hide
| that. The answer is to assign reputation to the Worker
| itself.
| njbooher wrote:
| Yeah, I don't envy you. Finding loopholes is more fun than
| trying to account for everything upfront.
| zamalek wrote:
| > This attack has always been possible and is common to
| basically all CDNs.
|
| I strongly recommend you play around with Azure CDN, because
| this is not entirely true.
| meowface wrote:
| >If a Cloudflare customer has configured their origin server to
| respond only to Cloudflare IPs, then they MUST also verify that
| the "Host" header on any request actually matches their domain
| name. If they do not verify the Host header, then anyone can
| sign up for Cloudflare and simply configure their DNS to point
| to the victim's origin IP address, and requests will be routed
| there -- but will have the attacker's domain in the Host
| header.
|
| What exactly could such a misconfiguration enable? Just that
| the attacker could configure lower levels of Cloudflare
| security settings for their domain, to bypass the origin's
| intended Cloudflare security settings? Would this also bypass
| I'm Under Attack Mode?
| kentonv wrote:
| A couple other notes:
|
| * The IP address 2a06:98c0:3600::103, mentioned in the article
| as appearing in the CF-Connecting-IP header, is a special IP
| address that is used for all cross-zone requests that come from
| Workers. This is _not_ the actual address of any Cloudflare
| machine. Workers fundamentally don 't have distinct IP
| addresses. Instead, the CF-Worker header needs to be used to
| identify which worker sent the request. We intentionally do not
| identify the original client's IP here because the Worker
| itself could be working against the client, and could have
| maliciously modified their request to be something entirely
| different. Hence, the request can't be considered to have come
| from the original client.
|
| * Any cloud service that lets you make HTTP requests can,
| obviously, be used to set up a proxy that hides your IP
| address. Workers is actually less useful for this compared to
| many other cloud hosts since we tag all outgoing HTTP requests
| from a Worker with the sender's domain name (in the CF-Worker
| header), which makes it relatively easier to track, report, and
| block abuse.
| jeroenhd wrote:
| That's a nice trick, with some automation you should be able to
| register new domain names and Cloudflare accounts to get past the
| 100k limit as well.
|
| Cloudflare's anti bot tooling is terribly annoying while using
| Tor. I understand why they do it, though. Using Tor is a good way
| to show how terrifyingly much control Cloudflare has over the
| internet.
| hedora wrote:
| > _I understand why they do it_
|
| Care to elaborate? Why block access to static pages by default?
| Tor's aggregate bandwidth is tiny compared to CF's aggregate
| bandwidth, so DOS attacks surely aren't the reason.
| jchw wrote:
| It's not bandwidth/DoS attacks, but it _is_ malicious
| traffic. Even for websites that don't explicitly block Tor
| exit nodes, they quickly end up banning most Tor exit nodes
| just by virtue of blocking malicious traffic that crops up.
|
| Malicious traffic takes many forms. As an attacker trying to
| attack a website, you are probably not going to come in from
| a residential IP address, unless it's one you have access to
| illegally. It is much more likely you will use Tor.
|
| And because of the nature of Tor, you literally, full stop,
| cannot ban individual Tor users to any meaningful degree. So
| if you offer services not requiring sign up, or services
| where the sign up is unobtrusive, users abusing the service
| via Tor are difficult to stop.
|
| Of course there are counter points... such as [1]. But let's
| be honest, from the perspective of a tired sysadmin trying to
| stop ongoing vandalism, your options are limited, especially
| if you don't _want_ an intrusive sign up option.
|
| As a counterpoint to the counterpoint, if Tor users were
| treated like any other user malicious parties might use the
| Tor network even more often than they already do.
|
| Edit: also, though it is not necessarily malicious _per-se_ ,
| there are some types of traffic like scraping that may be
| undesirable from the PoV of the website. I am not going to
| make a value judgment regarding the merit of blocking
| scrapers, but it is nonetheless a goal where you can see the
| reasoning behind blanket banning Tor.
|
| [1]: https://blog.torproject.org/the-value-of-anonymous-
| contribut...
| BlueTemplar wrote:
| Search engines use scraping. I would guess that most
| websites want to be featured on the search engines?
| jchw wrote:
| Not all websites want to be scraped even by search
| engines, and some only want to be scraped by search
| engines. Unwanted scrapers may refuse to follow
| robots.txt or do things with the scraped data that is
| either undesired to their business model or possibly even
| illegal, though that is complex. The point is, for better
| or worse, as far as I know, websites have the right to
| block scrapers while allowing search engines, and as long
| as it remains viable to prevent scraping, it remains
| likely that sites intending to do this will end up
| blocking or obstructing Tor traffic. Is it a good thing?
| I would argue it's too abstract to judge in such a
| generalized fashion. Whether or not it is a good thing or
| a bad thing, though, is not important. It is the world we
| live in right now.
| colinmhayes wrote:
| I wrote a search engine in school. Since I was
| naive/lazy/wanted a large corpus it didn't follow
| robots.txt. It ended up DOSing multiple sites, including
| Duke's law school during class sign up. I don't think
| those sites wanted to be featured on my search engine.
| jchw wrote:
| Not to detract from the point but I find Cloudflare to be on
| the more reasonable side when it comes to blocking Tor. A lot
| of sites end up effectively banning all Tor IPs, Cloudflare
| merely requires CAPTCHA. And they _do_ offer Privacy Pass as a
| sort of approach to make it a little less annoying.
| m-p-3 wrote:
| And I found hCaptcha to be way less annoying than ReCaptcha
| when using Tor.
| ffpip wrote:
| > Cloudflare merely requires CAPTCHA.
|
| Cloudflare doesn't "require" captchas for TOR users. By
| default, they treat TOR IPs it like any other IP. However
| since a lot of traffic comes out of a TOR IP, it looks like
| bot traffic
| Sebb767 wrote:
| They at least monitor TOR IPs. See this blog post [0] from
| 2016.
|
| https://blog.cloudflare.com/the-trouble-with-tor/
| capableweb wrote:
| > Cloudflare merely requires CAPTCHA
|
| I do agree that Privacy Pass is in theory a good idea,
| although to say that CloudFlare "merely" requires captchas to
| be filled out is a bit disingenuous. You're often required to
| complete 5-10 captchas, and sometimes even after that, you
| get denied. Had that happen to me multiple times.
| mikequinlan wrote:
| If you can't pass a CAPTCHA, you need to stop and ask
| yourself -- are you really a human being, or have you just
| been programed to believe that you are?
| capableweb wrote:
| Actually a good point as I never even considered that.
| Maybe I'm the one who's at wrong here?
|
| Seemingly I am passing the captchas, they are not giving
| me any errors. Simply seems to accept my input, reload
| the page and ask me to do more.
| colinmhayes wrote:
| There are some really crazy captchas out there. I've had
| some where I'd need a microscope and enhance button to
| get.
| iaml wrote:
| Not a valid question when google literally has a patent
| on unsolvable captcha.
| https://patents.google.com/patent/US9407661
| kazinator wrote:
| If you have been trying to solve the same captcha for
| several hours, you need to stop and ask yourself -- are
| you really a human being, or have you just been programed
| to believe that you are?
| arbol wrote:
| This patent has expired due to non payment of fees
| jchw wrote:
| I don't use Tor for _all_ of my traffic, but sometimes I do
| use it just because I can. And my experience lately is that
| I _usually_ get hCaptcha and I usually pass it once and get
| in. It's Recaptcha that is a serious problem.
| croutonwagon wrote:
| Google owned domains, namely Youtube, are even worse.
| They often redirect to different domains for blocks so
| just changing the circuit does nothing.
| akalsz wrote:
| Yeah after years of suffering from reCAPTCHA I'm somewhat
| thankful for hCaptcha. It _is_ still annoying (especially
| Cloudflare 's integration which seems barely compatible
| with the Tor Browser's cookie and circuit management),
| but at least I don't have to switch exit nodes twenty
| times just to have a chance of my solution being accepted
| (like with reCAPTCHA).
| 0df8dkdf wrote:
| > You're often required to complete 5-10 captchas, and
| sometimes even after that, you get denied. Had that happen
| to me multiple times.
|
| Glad you mentioned it. This also has happened to me many
| time. It is unfortunate botnet and spam has give IP
| addresses bad names.
| deanclatworthy wrote:
| Try signing up to gitlab and then signing in. It's completely
| broke with TOR because of cloudflares crappy redirect and
| nobody has done anything about it. The captcha also breaks
| after you spend about thirty mins trying to find a relay
| which doesn't trigger the redirect page protection loop.
| t0astbread wrote:
| As far as I know, Cloudflare transparently redirects some Tor
| users to their own hidden services and then uses the circuit
| ID for rate limiting: https://blog.cloudflare.com/cloudflare-
| onion-service/
|
| "Some" because they seem to have internal heuristics for
| detecting Tor Browser users before they enable that feature
| for a connection, so it doesn't apply to all connections made
| through Tor. YMMV I guess but I haven't seen a Cloudflare
| CAPTCHA over Tor in a long time.
| akalsz wrote:
| I still wonder how that detection thing works. My custom
| Firefox setup with requests proxied through Tor passes as
| the TBB, but copying the same request as a curl command
| somehow doesn't.
| rattray wrote:
| I hadn't heard of "Privacy Pass" before.
|
| https://blog.cloudflare.com/cloudflare-supports-privacy-
| pass...
|
| Sounds like it allows you to browse without tracking in a way
| that more reliably signals you're a human, which seems very
| useful.
|
| Why would a human browsing the internet with Tor _not_ want
| to use Privacy Pass?
| dotancohen wrote:
| > And they do offer Privacy Pass as a sort of >
| approach to make it a little less annoying.
|
| The Privacy Pass extension requires, as it mentions when
| installing it, access to all the data on all websites that
| you visit. So in order to stop getting so many Captchas, one
| has to give this company unfettered access to all sites that
| one visits? Why even bother with an HTTPS connection when the
| browser is potentially leaking the information away with
| explicit user consent?
|
| And I don't even use Tor. But every few weeks I'll have a two
| or three day spat where every webpage that I visit requires
| at least two captchas to be solved. Maybe another user from
| my ISP is scrapping, but I'm a good citizen on the 'net.
| grishka wrote:
| Discriminating IP addresses is a terrible idea to begin with.
| You're breaking the internet by doing it. Please treat all
| IPs equally.
| pepemon wrote:
| No, we won't. Many of those IPs are the origins for spam,
| botnets, crawlers, DDoS, exploitation engines, etc. The
| only feasible solution even with all modern heuristics and
| ML is still good ol' IP scoring and/or banning.
| catblast01 wrote:
| It's just silly to suggest this when quite simply not all
| IPs behave equally and the differences are not subtle. It's
| annoying when "bad" IPs get reassigned and change, but that
| is a relatively infrequent event - and certainly CloudFlare
| is at least adaptive. It's just not a principal aligned
| with reality.
| grishka wrote:
| The problem that no one seems to acknowledge: one IP
| doesn't necessarily mean one person or entity behind it.
| Residential ISPs often use a single public IP as an exit
| point for many subscribers simultaneously. So if one,
| just one, of these subscribers does something that our
| internet overlord Cloudflare considers nefarious,
| everyone else sharing this IP gets punished for doing
| nothing wrong.
| BlueTemplar wrote:
| This specific issue is going to be solved by IPv6 though.
| grishka wrote:
| In 50 years when half the internet users will finally
| have access to IPv6. Or, I'm pretty sure some people, or
| Cloudflare, will start banning entire /32 or even bigger
| subnets because _why not_. Oh you 're from Russia? Here,
| have a 403 for no apparent reason.
| BlueTemplar wrote:
| Considering the current adoption rates, I'm eyeballing 5
| years :
|
| https://www.google.com/intl/en/ipv6/statistics.html
| jbluepolarbear wrote:
| This specific problem happens all the time. That's the
| ISPs problem, not cloudflare. The ISP has a user
| negatively impacting other users, best course of action
| would be remove that user and notify cloudflare after.
| grishka wrote:
| Do ISPs actually do any of this? Does _any_ ISP ever do
| any of this? I don 't think so. Also, no one knows what
| was the particular thing that pissed off Cloudflare,
| because it's a black box.
|
| I'll say it again: centralizing the internet around a
| single company is a terrible idea. Especially so when
| said company terminates TLS and thus has access to
| unencrypted traffic.
| jbluepolarbear wrote:
| ISPs don't because they don't care. Until there's a legal
| or monetary reason to do so ISP providers will do the
| bare minimum. My spouse worked for a large American ISP
| for would hear all the time that apartment complexes
| would not be able to access certain services because one
| user did something that got the IP blocked and nobody
| cared because it wasn't their problem.
| grishka wrote:
| ISPs don't care so they don't do anything. Website
| operators don't care so they offload the responsibility
| to Cloudflare. Cloudflare doesn't care so it bans and/or
| humiliates unrelated people however it sees fit. _Nice._
| jbluepolarbear wrote:
| 100% of the bad traffic to my servers comes from TOR. Not
| all IPs are equal, some aren't playing nice. Cloudflare
| makes it so I don't have to worry about it.
| zelphirkalt wrote:
| That's how responsibility is shifted always to the next
| person or institution.
|
| You make your life simple by saying "Cloudflare makes
| that decision." Cloudflare says "We are merely offering a
| service here! You don't have to use it, if you think it's
| wrong." And ISPs say (or should say): "We are here to
| sell Internet access, not to watch what everyone on the
| Internet." and may by law not be allowed to look into the
| traffic.
|
| In the end the user suffers and is discriminated against
| and no one wants to be responsible.
|
| I would take the position, that each person, who uses
| Cloudflare, has to live with being responsible for
| whatever Cloudflare imposes upon users / visitors. The
| mass of people using Cloudflare services makes the
| difference. Each person using it adds a little bit to it.
| It is a collective responsibility.
| jbluepolarbear wrote:
| Then what's the alternative? If I don't use a service
| like Cloudflare I have to invest a lot of time and money
| into making sure my servers accept the traffic I want and
| reject the traffic I don't. This isn't an ethics dilemma,
| bad actors wreck a lot of innocent people's lives all the
| time it wasn't going to stop on the internet.
| bombcar wrote:
| The alternative is simple - ban tor ips entirely as it's
| not worth the hassle.
| Tepix wrote:
| You're saying RBLs don't work? In my experience they work
| rather well.
| flas9sd wrote:
| not only Tor, but other network resources that are legit, but
| can be abused because of their no-cost nature like Hurricane
| Electrics ipv6-in-ipv4 tunnel broker. Not sure how I feel about
| cloudflares role as arbiter. I think instead of relying on a
| service, a good nginx module that has ratelimiting abilities
| easily surfaced with some grasp of ASN origin could satisfy the
| need of most sites out there.
| xwdv wrote:
| Also worth noting it's a good time to buy Cloudflare stock,
| it's down from recent highs and may be on its way back up in
| the next month. This is a good long term hold.
| dcow wrote:
| TL;DR: somebody learned how proxies work and is naively mad at
| Cloudflare because they offer a cloud worker product that lets
| you proxy requests and their "bug" report is actually a feature.
| klaudius wrote:
| They say there are multiple ways to do IP spoofing. Does anyone
| know some other ways to do it?
| Tepix wrote:
| Find a provider that doesn't prevent you from doing it.
| CFA178B wrote:
| Disclaimer: Throwaway account.
|
| There is a way to bypass CF bot protection, which mostly uses
| basic HTTP features, nothing fancy (and especially not a security
| issue).
| foolmeonce wrote:
| I'd wondered about the openness of the workers do connect to
| unrelated resources myself..
|
| But I agree with CF, they don't inherently have a security
| problem. They have the normal situation of a proxy again. If a
| lot of new domains show up to bot proxy then the Baysian for an
| unknown worker domain is going to become bot..
| oefrha wrote:
| > As you can see, X-FORWARDED-FOR can be set to an arbitrary
| value, so you can bypass IP limitation requests during scrapping
| or more dangerous IP verification during a login procedure ...
| The origin IP is not forwarded to the website, so the only way to
| block this kind of request on your server is to filter on CF-
| WORKER header.
|
| X-Forwarded-For should always be treated as forged at least for
| security purposes (except the hops set by reverse proxies under
| your control). If you do IP whitelisting based on untrusted
| X-Forwarded-For you're doing it terribly wrong.
| daurnimator wrote:
| A lot of people just verify that X-Forwarded-For matches a
| Cloudflare-owned IP.
|
| As this post demonstrates, you need to verify against the
| cloudflare origin pull CA (see
| https://support.cloudflare.com/hc/en-us/articles/204899617-A...
| )
| userbinator wrote:
| For a very long time, XFF-spoofing was a trick to get past
| journal subscriptions and other paywalls. I don't feel so
| guilty about disclosing this here now, as a lot of them have
| (unfortunately) blocked that route. No doubt a few popular
| bookz/journalz sites owe at least some of their content to this
| technique. But it's something to try and might still work on a
| few of them...
| tgsovlerkhgsel wrote:
| I suspect that the idea here is that XFF is supposed to be
| trusted, because it is supposed to come from Cloudflare.
| [deleted]
| xPaw wrote:
| So is there any reason not to block any requests on your server
| if they contain the CF-WORKER header?
| minitech wrote:
| Yep - any legitimate tool that makes requests on behalf of a
| user could do so using Cloudflare workers. (A webpage
| translation service, for example.) It's the same as any other
| provider. You could block based on the _value_ of the CF-Worker
| header if you find it's being abused.
| oefrha wrote:
| I've built legit CORS proxies for web-based community tools
| using CF Workers when the first party doesn't set CORS headers.
| winrid wrote:
| Their bot protection is so annoying for integrations, I may just
| use this...
___________________________________________________________________
(page generated 2021-04-04 23:00 UTC)