[HN Gopher] Turns out half the internet has a single-point-of-fa...
___________________________________________________________________
Turns out half the internet has a single-point-of-failure called
"Cloudflare"
Author : StuntPope
Score : 762 points
Date : 2020-07-20 13:42 UTC (9 hours ago)
(HTM) web link (easydns.com)
(TXT) w3m dump (easydns.com)
| Uhhrrr wrote:
| > But if you want to use a preferred DNS provider, such as
| Cloudflare, who use their DNS responses to optimize your website
| proxy. That works best most of the time, so then you want to go
| with an active/passive model that will step back when things are
| going according to plan, and then when these periodic network
| cataclysms do occur (and they will), they step into the breach
| and update your nameservers so that you at least stay up until
| the crisis is over.
|
| Copy editors are cheap and your reputation shouldn't be.
| louwrentius wrote:
| Who at a C-level position is going to tell anybody that the
| potential risk of Cloudflare (or Amazon/Azure/GCP) going down
| should be protected against?
|
| I would applaud them, but I wonder.
| divbzero wrote:
| There's a lot of truth to this. Cloudflare for now has achieved
| IBM status in that no one will be fired for choosing
| Cloudflare, in spite of any issues that arise.
| Nasrudith wrote:
| My guess is Wall Street HFT or other financial areas with very
| strict unscheduled downtime penalties where they are
| effectively incentivised to be batshit paranoid as it would
| take decades for any penny pinching to remotely pay off. I
| don't know if many of them use Cloudflare for their domains
| though.
| miki123211 wrote:
| Cloudflare is horrible for blind people.
|
| Screen readers, the programs that use synthesized speech to tell
| us what's on the screen, cannot read images. Good captchas
| usually have audio equivalents (which come with their own set of
| problems), but this one doesn't. If you're blind and flagged by
| Cloudflare for some reason, you're cut off from accessing half
| the internet, potentially critical
| banking/governmental/medical/communications/educational services.
| We rely on the internet way more than our sighted peers, so this
| is very important. This has recently happened to me on a few
| sites, fortunately not critical ones, but it was not a pleasant
| experience nonetheless. CF engineers, please fix this ASAP. I'm
| surprised there still isn't a huge lawsuit over this, as this is
| clearly violating all sorts of laws.
| strombofulous wrote:
| I use audio captchas. Google will usually only let you do 2 or
| 3 before banning you and making you do image-based ones. I'm
| pretty sure the button is just there to make it seem
| accessible.
| [deleted]
| solotronics wrote:
| Maybe it would make sense to have multiple independent sections
| of backbone at the BGP level. Instead of having one public
| AS/backbone, break it down into regions at least so that it is
| more confederated.
| nonbirithm wrote:
| Question: is it even possible to have DDoS protection without
| using a provider of it which becomes a single point of failure?
| Or is it maybe possible to decouple this single feature from
| everything else that Cloudflare provides that could take out all
| the sites in the future from an unrelated misconfiguration?
|
| I don't see the centralization as a positive, but I'm wondering
| what percentage of the websites that were taken offline see
| themselves as having no choice but to use Cloudflare in order to
| prevent themselves from being taken down anyway from malicious
| actors instead of by accident.
| [deleted]
| sschueller wrote:
| Sure, Arbour and others sell devices to deal with ddos and many
| isps have clusters of these which you can use. Of course this
| is a service that costs money.
| divbzero wrote:
| I think you could use Cloudflare as your primary DNS provider
| and benefit from their DDoS protection, but also specify backup
| DNS name servers with a different DNS provider in case
| Cloudflare fails.
| dentemple wrote:
| If more companies are willing to provide the same level of
| service and price as Cloudflare, then they can get in on the
| game, too.
| raverbashing wrote:
| If only DDOS attacks were taken seriously and their perpetrators
| punished accordingly (and maybe if the network had better ways of
| self-defense) instead of companies and websites having to fend
| for themselves (or having to resort to solutions like
| Cloudflare).
| ericlewis wrote:
| I am not sure I understand this comment, in the context-
| cloudflare misconfigured some routes and it was quickly
| resolved. was this a DDoS?
| t_sawyer wrote:
| His point is why the internet has a single point of failure
| -> Cloudflare. They offer DDoS protection for free.
| jaywalk wrote:
| I think the point was that so many companies wouldn't have to
| rely on anti-DDoS protection from Cloudflare.
| jgrahamc wrote:
| It was not.
| xwdv wrote:
| Turns out no one got fired for choosing Cloudflare.
| black_puppydog wrote:
| What I find really shocking is the abundant use of CF on _piracy_
| websites of all things. Not the serious ones of course, SciHub
| and library genesis are mirrored differently.
|
| But a lot of small torrent websites and such simply won't load
| without JS and specifically CF code. It's pretty crazy. Luckily I
| don't use any of those bEcAuSe IlLeGaL but still, I find it
| really depressing, especially when webtorrent, IPFS etc are
| available, and frankly many of those pages will never have to
| bear a load that makes CF a requirement.
| jdc0589 wrote:
| its not about caching or handling normal traffic. its about
| ddos protection. sites like that are frequent targets.
| black_puppydog wrote:
| and there should be enough of them that that shouldn't
| matter.
| citizenpaul wrote:
| I love all the comments about fail-over for DDOS/DNS protection.
| What is your budget? Well we are looking at around $0.00 for our
| maximum allowance. Ok so single point of failure it is I guess we
| are done here. Companies only say they care when there is a
| problem, the reality is that they dont.
| FireBeyond wrote:
| Precisely. I remember when the CEO of my old company came to me
| and said (with respect to moving from a on-prem model to SaaS),
| "What's our SLA going to be?"
|
| "Well, what do you want it to be?"
|
| Give me a number and I'll tell you how much it will cost and
| how long it will take to get there.
| divbzero wrote:
| It obviously takes more than $0.00 but not that much more. It's
| a matter of adding a second DNS provider and making sure you
| replicate DNS records manually or with AFXR.
|
| What's really puzzling is if the same companies spend money on
| active/passive failover for application and database servers
| while overlooking DNS single point of failure.
| crazygringo wrote:
| I guess this is off-topic, but the stock photo they're using is
| _cracking me up_.
|
| Guy at work... coffee cup on a _tablet he 's using for a
| coaster_... except he's also _drinking whiskey_ from a beautiful
| crystal glass... there 's a folded paper airplane... there's just
| so much to unpack here, it's pretty hilarious.
| hinkley wrote:
| > except he's also drinking whiskey from a beautiful crystal
| glass
|
| And he has left the stopper off of the decanter like some sort
| of animal.
|
| I think we are supposed to believe that the person here was
| just dicking around online, fiddling with paper, finishing his
| morning coffee, when all of a sudden he gets an email asking if
| anyone knows what's going on with the website.
|
| So he stops what he was (not) doing, puts down his coffee, and
| starts poking around. At which point he realizes he needs
| something stronger than coffee. As he is pouring his glass he
| is confronted with the true horror of the situation, drops the
| stopper on the floor and just holds his face wondering why the
| Universe hates him.
|
| I wonder if we can get JJ Abrams to option the movie rights.
| mountainboy wrote:
| More importantly, CloudFlare is the world's largest man-in-the-
| middle.
|
| You think your TLS connection is e2e between you and the website
| you are visiting? Not so much... because the website has given
| their certs to CF. Worse, sometimes/often the connection between
| CF and the website is not encrypted at all.
| JohnTHaller wrote:
| Many of us don't even know it. My DNS for PortableApps.com is run
| through Digital Ocean, which uses Cloudflare for DNS.
| gripfx wrote:
| On an unrelated note. Thank you so much for PortableApps.com!
| It was invaluable at university when studying in the library.
| To this day, I still use it for utilities that don't need to be
| installed on my desktop.
| mrsalt wrote:
| I am also grateful for PortableApps.com. Along with Scoop and
| Homebrew, they make using a system without root or admin
| privileges a really nice experience, in both Windows and
| Linux.
|
| My sincere thanks to John and all the PortableApps.com
| contributors.
| JohnTHaller wrote:
| You're welcome! I'm glad it's helping you be more
| productive!
| JohnTHaller wrote:
| You're welcome, I'm glad it's helped and helping you! I try
| to keep it growing and relevant with a more synced cloud
| folder/work or school laptop/keep work and personal separate
| bent these days.
| neycoda wrote:
| When Cloudflare causes the problem they were built to solve.
| johnklos wrote:
| I think people are Cloudflare fans simply because so many other
| services suck more.
|
| I gave Cloudflare a fair shake. But after hearing their lies way
| too many times, I'm calling them out for being deceptive and
| unscrupulous.
|
| After being told that sites pretending to host Adobe Flash
| updaters and pretending to be Bank of America can't be taken down
| by Cloudflare because of their rights to free speech, I knew
| their attempts to pretend to be one of us, attempts to pretend to
| care, were nothing but bullshit.
|
| They claim they don't host. If hosting DNS is not hosting, then
| what do you call DNS hosting? You literally CANNOT use their
| abuse web form to report domains which use Cloudflare just for
| hosting DNS. They do not handle abuse sent to their abuse email
| address (they simply send a form response saying to spend ten
| minutes filling out their crappy form that has all sorts of
| problems).
|
| Of course, their web proxy services are also "not hosting", even
| though they're protecting all sorts of scammers.
|
| So why should we think they're not bullshitting us when they run
| 1.1.1.1 and tell us they're not logging? Why should we trust them
| more than our ISPs by running DoH through them?
|
| They WANT us to be dependent on them, because the more control
| they have, the more money they can make. It's dangerous, and
| they've shown they have no honor.
|
| I've genuinely tried to correspond with them on Twitter, and they
| excel at not answering the question asked but instead just
| diverting. It's scummy, unprofessional behavior and I encourage
| everyone to consider whether they deserve anyone's data or
| business.
| james412 wrote:
| > They WANT us to be dependent on them
|
| I think this is an important point about CloudFlare that can't
| be made often enough. It's been some years since I first
| noticed, and it seems as true today as it was then: wedging
| themselves into core Internet services and data flows seems to
| be an intentional part of CloudFlare's strategy.
|
| There is no case given the architecture of the Internet where
| one company need be exposed to so many traffic flows from
| millions of people. The search engines got there first and we
| eventually stopped complaining, but this does not make it any
| more justifiable to copy the model.
|
| What reason would a company have for desiring this outcome? We
| know Google can detect and predict flu outbreaks. Imagine what
| is possible when you have every click on every target web site.
|
| There is a fair chance their data is already approaching the
| comprehensiveness of Google, and I'd be surprised, if not
| disappointed to learn they were not already working on
| unannounced (now or eternally) internal intelligence products
| based on that data. There are simply too many pockets who would
| be willing to pay for it.
| jariel wrote:
| I thought 'we' were for 'Net Neutrality'?
|
| Remember, carriers and service providers not allowed to decided
| what to do based on content?
| tyingq wrote:
| It's interesting to me that Cloudflare doesn't really have any
| competition with a similar business model. I suppose the free
| plan requires quite a lot of spending before the upgrades offset
| it.
| neurostimulant wrote:
| Sucuri seem to have pretty similar business model, which is a
| proxy in front of your site that handle security and cdn.
| tyingq wrote:
| I meant the _" CDN with a meaningful free tier"_ part of the
| business model. That's why _" half the internet"_ has CF as a
| single point of failure.
| nerdponx wrote:
| I don't need a free tier. I just need a "basic bitch" tier for
| my personal usage. Who are some Cloudflare alternatives for
| this?
| corford wrote:
| Depends on what you want but a very solid free CDN and DNS
| option is: host your site on netlify and use dns.he.net for
| your nameservers.
|
| Another good DNS option is dnsimple.com or, indeed, EasyDNS.
| For even more redundancy, use one provider as your domain
| registrar and another for your nameservers (and set short
| TTLs for your zones so you can re-point IPs quickly if you
| need to).
|
| For the other things Cloudflare offers on their free tier,
| I'm not sure what good alternatives exist (there must be
| some, I'm just not familiar with them outside of the obvious
| AWS alternatives).
|
| Edit: one caveat with above advice, I have no idea if netlify
| use cloudflare behind the scenes...
|
| Edit 2: For other options, checkout https://www.cdnperf.com/
| and https://www.dnsperf.com/
| KirinDave wrote:
| If all you're doing is hosting and CDNing static content, the
| US cloud providers can do this very, very cheaply.
| kijin wrote:
| There are other large CDNs like Akamai. They just don't compete
| with Cloudflare in the consumer sector. It probably doesn't
| matter for them because enterprise contracts are where all the
| money is at.
| [deleted]
| KirinDave wrote:
| AWS, Azure and Google Cloud all offer competitive alternatives
| for the small business sector. Personally, I find the actual
| cost of CloudFlare very difficult to reason about in advance of
| actual use. This is in part because it's easy to miss a bit of
| the a la carte model you need.
|
| Full disclosure, I work at Google on the SRE team for Cloud
| CDN. If you want a credible alternative for the CDN part of
| Cloudflare, our product is extremely fast. We were all very sad
| for Cloudflare and watched the whole affair closely, as we've
| got a few customers that use our services alongside
| Cloudflare's.
| tyingq wrote:
| The jist of my comment is why nobody else has tried the _"
| liberal free tier"_ model for a CDN, as it seems to be
| working for CF.
| KirinDave wrote:
| Sure, but all 3 cloud providers have free tiers for this
| stuff, don't they?
| tyingq wrote:
| Not that would work for a CDN use case as far as I know.
| The cloud providers are notorious for high egress costs.
| KirinDave wrote:
| The pricing for Google Cloud CDN doesn't look outrageous
| to me.
| jonplackett wrote:
| Similar problems are happening with crypto - yeah it's
| distributed but so many people are using Coinbase that if they go
| down it's going to cause a lot of problems
| OutsmartDan wrote:
| Is it realistic for a small-medium sized business to have more
| than one DNS provider?
| throw0101a wrote:
| > _Is it realistic for a small-medium sized business to have
| more than one DNS provider?_
|
| Yes: as the weblog post points out, you can have EasyDNS as
| your master with their multiple DNS servers, and then _also_
| have (e.g.) Route53 slaved to EasyDNS and have those _in
| addition to_ EasyDNS in your records.
|
| DNS servers have had replication for decades.
| sukilot wrote:
| It's easy enough if you sign up with a DNS provider provider.
| [deleted]
| michaelbuckbee wrote:
| Honest question: I've never really understood the back of the
| napkin math of how Cloudflare functions economically, which I
| feel would go a long way towards my understanding of why/how they
| were able to become such an integral and generally positive part
| of the Internet.
|
| Did they have some crazy in to get cheap bandwidth? Did they bet
| big on bandwidth prices falling? Did they figure something else
| out that nobody saw? Do they just to a tremendous job of
| migrating sites from free to paid plans?
| rasengan wrote:
| > Did they figure something else out that nobody saw?
|
| I may not have a full scope of the history, but my own
| experience with DDoS protection was quite different. Whilst
| providers offered anti ddos protection through GRE tunnels and
| dedicated machines behind DDoS appliances and a heavy null
| route hand, Cloudflare had a simple few-click solution that
| worked at the web application level making things a lot easier
| and, also, allowing for features like caching, and thus, CDN
| benefit from a global network. Further, they've maximized
| performance on their machines, and as a result, Cloudflare is
| wicked fast.
|
| Cloudflare does what it does really well and has built
| additional services on their global network that make a lot of
| a sense and provide a lot of value.
|
| Hats off to CF.
| aclelland wrote:
| This has been my experience too, CF is just so easy to
| migrate to and to configure once you're there.
|
| If AWS was able to offer a single click "DDOS protection and
| CDN" feature with similar pricing and features as Cloudflare
| then I'd consider it since most of our infrastructure is on
| AWS but at the moment they don't offer anything nearly as
| competitive. Just the Cloudfront bandwidth costs alone would
| dwarf our total infrastructure costs.
| michaelmior wrote:
| Indeed. I think I understand the argument claiming Cloudflare
| is a SPOF. However, if you're actually dealing with DDoS
| attacks with any kind of regularity, you're likely to have
| better uptime with them than without.
| henriquez wrote:
| Cloudflare is a relatively cheap "lite CDN" for developers who
| want caching without putting in any work. This becomes a
| gateway drug for Cloudflare's more expensive plans once you
| outgrow the free plan. It quickly adds up; I worked for a
| company that wasn't even on an enterprise plan and was spending
| hundreds of dollars a month on Cloudflare just because they had
| a lot of domains.
|
| My reservation with Cloudflare is the concept of letting a
| third party MitM my SSL traffic. That and it's more expensive
| than a cheapo CDN like Stackpath if all you really care about
| is CDN (and Cloudflare isn't even really a good CDN, just a
| quick hack to speed up small static files).
| Kalium wrote:
| In my experience, it's extremely rare to find a CDN that
| doesn't expect to do TLS termination. My understanding of TLS
| is that it's exceptionally difficult to cache content if you
| cannot see into the requests.
|
| Perhaps I have overlooked something?
| duskwuff wrote:
| > My understanding of TLS is that it's exceptionally
| difficult to cache content if you cannot see into the
| requests.
|
| Not even "exceptionally difficult", but flat-out
| impossible. From the perspective of an observer, TLS
| sessions are random data. The protocol is specifically
| designed to defeat attempts to replay data -- a CDN is
| indistinguishable from an attacker in that sense.
| henriquez wrote:
| Right, with Stackpath they do TLS but not necessarily on
| the primary domain. You don't have to point your
| nameservers if you don't want to. So you can set them up on
| a subdomain and use their fake SSL to serve purely static
| files (if the file doesn't exist on CDN at the time it's
| requested then their system will pull it directly from your
| server via a private subdomain, serve it to the client and
| store it for next time)
|
| So in this way it's possible to setup CDN with shared SSL
| for purely static files but not the app server itself; you
| don't have to give the keys to the whole kingdom so to
| speak and it's cheaper than Cloudflare at the basic level.
| Kalium wrote:
| Let me see if I follow. Auto-provisioned TLS
| (misleadingly termed "fake SSL") on the front-end for
| delivering static contents and caching. A private
| subdomain with a pinned cert _not_ managed by the CDN to
| deliver static contents to the CDN. And a third subdomain
| for the application itself that 's not going through the
| CDN.
|
| I was under the impression that the same result could be
| achieved with Cloudflare, or indeed nearly any CDN. Was I
| mistaken? Though you may not actually need a secret,
| private subdomain for static files with all CDNs.
|
| Again, please let me know if I've made a mistake
| somewhere. I'd love to learn something this morning.
| henriquez wrote:
| You're right about that. So it might look like this
|
| static.domain.com (CDN subdomain with auto provisioned
| TLS)
|
| static-uncached.domain.com (private pass-through
| subdomain when CDN is missing a file)
|
| www.domain.com (app server hosted wherever)
|
| You're right that you could do something similar with
| other CDNs including Cloudflare (you can just set the www
| subdomain to "bypass Cloudflare" to accomplish a similar
| result), but I'm not aware of any way to use Cloudflare
| on a domain without forwarding your nameservers to them,
| effectively giving them complete control over the domain.
| At least with Stackpath I can host DNS wherever and
| simply point the subdomains I want at them.
|
| Also, by the time you do the work to split static files
| into separate subdomains you might as well go with a
| dedicated CDN. One of the selling points of Cloudflare is
| for sites serving everything on one subdomain that they
| can forward to Cloudflare and get caching without any
| work.
| edaemon wrote:
| They have a CNAME setup where authoritative DNS stays
| outside of Cloudflare:
| https://support.cloudflare.com/hc/en-
| us/articles/36002061511...
|
| It requires at least the Business level plan, though.
| henriquez wrote:
| Ah my bad. That's probably why I never knew about it. The
| $200/month entry price is steep there.
| AlexandrB wrote:
| > My reservation with Cloudflare is the concept of letting a
| third party MitM my SSL traffic.
|
| I wonder what the Venn diagram of people who insist every
| website must use HTTPS for privacy reasons and Cloudflare
| users looks like.
| pradn wrote:
| Hundreds of dollars a month doesn't seem material for a
| company.
| basch wrote:
| Plus that means hundreds of unique domains with unique
| content. If you have hundreds of domains all pointed back
| to one site with a single forwarding rule, 99/100 could be
| free tier accounts.
| henriquez wrote:
| Nope, but I'm sure lots of companies are in this situation,
| where they just barely need the features of the lowest paid
| plans, end up scaling across several domains and Cloudflare
| makes a ton of money off a vast majority of customers that
| will never need their more advanced features.
| snowwrestler wrote:
| Unless you're sending all your traffic back to physical
| machines that you own locked into a cage in a datacenter, you
| are probably letting someone MITM your SSL traffic. For
| example if you are hosting on AWS, Amazon has access to your
| keys. If you are hosting on a hardware server leased from
| Hetzner, Hetzner has access to your keys.
|
| When a 3rd party has access to your keys, their
| responsibilities to you are spelled out in your contract with
| them. That's true for CDNs as well as hosting companies.
| sjy wrote:
| There's a difference between a VM host with the technical
| ability to carry out a targeted MITM attack against its
| customers using hardware-level access, and a provider that
| sells MITM as a service.
| zzzcpan wrote:
| It's more complicated.
|
| For most websites today if someone can intercept traffic
| somewhere close to the server they don't even need the
| keys, they can just fake responses to pass CA validation
| and issue valid certificates with their own keys and MITM
| like there is no encryption.
|
| And coldboot attacks performed by a hosting provider staff
| of dumping memory and finding keys isn't that realistic of
| a threat, just like putting servers into a locked cage on
| someone else's property isn't much of a protection.
| henriquez wrote:
| I send traffic for my sites and apps to physical machines
| that I own and operate in a secure location, but I doubt
| most people are doing this.
| ThePhysicist wrote:
| As far as I understand it it's a freemium B2B model: If you're
| small you probably can get away using the free tier. Then when
| you get bigger you outgrow the free plan, either because you
| want specific features or because your bandwidth becomes too
| high.
|
| That said bandwidth really isn't that expensive, at least if
| you're buying it at the scale that Cloudflare does. Many people
| seem to be used to the bandwidth prices of the large cloud
| hosters, which are really insane and have been marked up by a
| large multiplier to disincentivize people to transfer their
| data elsewhere for processing.
| jgrahamc wrote:
| Read our S-1, it's all in there.
|
| https://www.sec.gov/Archives/edgar/data/1477333/000119312519...
| morrbo wrote:
| II love cloudflare. It has really helped out with several
| sites/projects that I have worked on and the service is top
| notch. I am also an investor. I tend to invest in stuff which
| I use a lot or trust/respect the employees. Weirdly what made
| me really invest is the level of geekiness on the company. I
| remember seeing you guys using a lava lamp wall to generate
| entropy and just thought "that's awesome". I just wanted to
| say don't change. Your lz4 implementation, aes gcm golang
| optimizations have directly benefitted me. Coupled with
| really high quality post mortem articles, articles on
| interesting things like compression, encryption, networking,
| a few random articles (the privacy focussed file system
| recently comes to mind, written at the same time I was making
| my own distributed fs) just leave me with a lot of respect
| for the culture there.
|
| I was worried a bit when I saw the initial IPO that the free
| tier would leave (despite promises it wouldn't) but that
| doesn't seem to be the case. Literally the only bad thing
| I've seen on the site is recently you switched from recaptcha
| to a new one that I had a real tough time with logging in
| today - it was a bit glitchy on my pc. The only suggestion I
| thought of as well would be a simple "maintenance mode"
| similar to the "I'm under attack mode" which would allow
| those of us without super-ha to quickly toggle on something
| to pop up a "sorry server"/site is down form maintenance page
| without having to mess with our proxies/web servers.
|
| Anyway I know this comes across as totally kissing-ass but I
| just wanted to say thanks to someone who actually works
| there. Everyone fat fingers stuff every now and again,don't
| sweat it.
| DrJosiah wrote:
| I work at hCaptcha, we run CF's captcha. If you're having
| problems in the future, popping open a debugger and
| capturing the results can help us figure out what's going
| on. But also: browser, OS, site, ...?
|
| Right now: there is an issue with Safari users on the most
| recent iOS and OS X, where 3rd party cookies have now been
| disabled by default. We're working on a solution.
|
| If that's your issue, you can fix on your side in the
| short-term by not using Safari, or by enabling 3rd party
| cookies.
| Kudos wrote:
| I'd love to see your reply to the blind person commenting
| on your product here.
| benmanns wrote:
| If you use Cloudflare workers, you could put together a
| maintenance page with a simple script + Cloudflare Workers
| KV as a toggle switch. It would cost $5/mo if you're not
| already using workers.
| renewiltord wrote:
| It's a bit of a sad story, but maybe you'll like this read
| about one of the guys who laid the foundation for their
| tech and his sad decline: https://www.wired.com/story/lee-
| holloway-devastating-decline...
|
| I really liked the stories of his skill when he was in his
| prime. Very inspiring.
| eigenvalue wrote:
| Wow, thanks for linking to the article. What an awful
| story. Makes you think about the wisdom of undertaking
| major elective surgery if it's not absolutely required in
| the short term.
| janee wrote:
| Sounds lame but this story really touched me. I recently
| visited a family member who's nearing the end and it
| broke my heart to see them in that condition.
|
| I'm not sure what to take from this. But thank you for
| sharing it
| rhizome wrote:
| > _I remember seeing you guys using a lava lamp wall to
| generate entropy and just thought "that's awesome"._
|
| To be sure (and for the sake of internet rando
| completenessism), it does look like CF waited until the
| original SGI patent on the technique ran out. :)
|
| https://patents.google.com/patent/US5732138
| michaelbuckbee wrote:
| Hey John, thanks for taking the time to reply.
|
| Respectfully, the info in the S1 (flywheels, etc.) seem to be
| what sustains you _now_.
|
| Maybe a better question is "how did you identify and kick off
| that flywheel?"
| eganist wrote:
| Don't take this as being glib, but isn't this the entire
| point of funding rounds? Demonstrate growth and talk about
| a few approaches to creating a business model that you're
| investigating, which then brings investors on as they're
| betting you'll find one that'll succeed?
| Arnt wrote:
| Congratulations with growing so big. Why aren't there dozens
| of lookalikes by now?
|
| I mean CDNs that will let you override the origin's cache
| instructions and do a decent job of DDOS protection, and
| whose feature list otherwise looks a little like
| Cloudflare's.
| aspir wrote:
| One answer - It's much, much harder to build and operate a
| global network infrastructure that it might seem. It's also
| even harder to invent some sort of "killer feature" or
| other genuine innovation on the experience. You're likely
| not using Cloudflare simply for commoditized pipes alone,
| but for other features or designed experiences in their
| offering.
|
| A second answer - there are a bunch of bottom-barrel
| commoditized pipe services. You likely haven't heard of
| them because they're so generic. They've existed before
| Cloudflare, and more will be created in the future
| https://www.citrix.com/products/citrix-intelligent-
| traffic-m...
| amelius wrote:
| Yes, why isn't the CDN a commodity?
| Kalium wrote:
| CDNs are expensive to build, and often not very useful to
| customers until you've built out a large portion of it
| (actual hardware required, you can't just run it atop
| AWS). On top of this, much of the money is in Enterprise.
| So you've got to compete with Akamai, AWS, Fastly,
| Incapsula, Cloudflare, and several other notable ones to
| get any customers to speak of.
|
| There _are_ smaller CDNs out there. You can find them
| readily enough.
| amelius wrote:
| Yes, but for a customer it's trivial to swap out one CDN
| for another, isn't it?
| Kalium wrote:
| Depending on how you integrated it and how deeply, it may
| be trivial. Certainly going from one DNS-based CDN to
| another can be pretty easy - a Cloudflare / CloudFront
| swap could be quick.
|
| Which suggests to me that CDNs are already a commodity in
| some ways.
| [deleted]
| chacham15 wrote:
| The relevant excerpt seems to be here:
|
| Market Opportunity
|
| We believe our platform disrupts several large and well-
| established IT markets. The key markets that are addressed by
| our platform include VPN, internal and external firewalls,
| web security (including web application firewalls and content
| filtering), distributed denial of service (DDoS) prevention,
| intrusion detection and prevention, application delivery
| controls, content delivery networks, domain name systems,
| advanced threat prevention (ATP), and wide area network (WAN)
| technology. From our analysis based on IDC data, $31.6
| billion was spent on those products in 2018, which is
| expected to grow to $47.1 billion in 2022, representing a
| compound annual growth rate of 10.5%. We also are actively
| developing new products to address adjacent markets including
| compute, storage, 5G, and Internet of Things (IoT) that are
| not included in the estimate of our addressable market.
| potency wrote:
| Off topic, but any regrets on getting involved in content
| policing? It always sat with me as wrong and a disturbing
| precedent that an internet backbone service such as
| yourselves would make it their business to shut down unsavory
| yet legal speech.
| nix23 wrote:
| Wikileaks and Piratebay are customers of CF, so your 'legal
| speech' must be really something. Any links?
| potency wrote:
| Cloudflare CEO Matthew Prince spoke to CNBC upon banning
| Daily Stormer: "We were worried that people would say,
| 'We won't work with you anymore,'" Cloudflare CEO Matthew
| Prince told CNBC.
|
| "We had to have the conversation now because at some
| point we'll be a public company. We had to prompt that
| discussion," said Prince, who added "we want to be ready
| internally by July 2018," for a possible stock offering.
|
| CNBC Link: https://www.cnbc.com/2017/09/24/cloudflare-
| ceo-matthew-princ...
|
| Editorials discussing the event:
| https://www.washingtonpost.com/opinions/where-to-draw-
| the-li...
|
| https://www.nytimes.com/2017/09/13/opinion/cloudflare-
| daily-...
| fennecfoxen wrote:
| Yes, and then they banned 8Chan just the same (I think?
| One of those sites where a shooter talked to his friends)
| as soon as they were under pressure.
| LordDragonfang wrote:
| If I'm not mistaken, 8Chan also hosted a lot of _illegal_
| content in addition to that "legal speech".
| yjftsjthsd-h wrote:
| And yet, it was not for anything illegal that they were
| taken down.
| debacle wrote:
| Likely the reddit model. Don't ban anything until it hits
| you in the pocket book.
| transpostmeta wrote:
| So it's literally Nazis.
| jonny_eh wrote:
| There's always a line, and therefore, I'm ok with drawing
| the line at Nazis.
| frandroid wrote:
| slow clap for HNers downvoting this and the parent
| answer...
| zajd wrote:
| Nazism is pretty popular on HN, comes with the territory
| when most of the board's users are well off westerners.
| Doesn't hurt that the mods are way more concerned with
| anti-capitalist rhetoric. Not surprising considering who
| owns the site though.
| [deleted]
| AnthonyMouse wrote:
| > There's always a line
|
| This is where you've gone wrong.
|
| There isn't always a line, some businesses are in the
| business of selling products and services and not in the
| business of policing content. And if some of their
| customers are pedophiles or the like then have the police
| arrest the pedophiles and leave the fry cooks and gas
| station attendants out of it, even if the pedophiles eat
| food and drive cars.
| Certhas wrote:
| Would you have an issue if a printing shop said no to
| doing business with a Nazi newspaper?
| midasz wrote:
| > There isn't always a line
|
| Yes there is. The line is Nazis now. There was a war,
| they lost. They're not welcome anymore.
| ygjb wrote:
| No. The reason why freedom of speech (or freedom of
| expression, if you're Canadian like me) is important in a
| government context is because the government (in theory)
| has a monopoly on violence. Businesses and private
| individuals should not be expected to uphold and enforce
| freedom of speech because requiring them to do so puts
| them at odds with themselves.
|
| An employer expected to uphold freedom of speech must
| then require their employees to work with or for people
| who believe they are lesser people, not even people, or
| should be the victims of abuse, violence, and genocide.
| That is an untenable position from a human rights
| perspective. I as a private individual do not want to do
| business or associate with people who use freedom of
| speech as a platform to preach hate or ignorance, and
| that is my choice.
|
| I am happy to see companies kick bad actors to the curb,
| whether they are fashionable nazis being deplatformed,
| gamergaters harassing women and minorities, and I am
| frustrated when I see legitimate journalists being
| censored by those same platforms. It's not cognitive
| dissonance to support stopping bad actors while rallying
| to support good actors, it's a recognition of the fact
| that our rights are open to abuse and that the current
| system isn't great at coping with abuse.
|
| I certainly don't think that requiring businesses and
| platforms to provide guarantees around freedom of speech
| is a good idea, and most platforms that have attempted to
| or succeeded in doing so have turned into the cesspools
| of the internet.
| Reelin wrote:
| > The reason why freedom of speech ... is important
|
| This view is (IMO) far too narrow. Freedom of expression,
| particularly political expression, is absolutely
| essential to the functioning of western society as it
| currently exists. Personally, I also see it as an ideal
| to be pursued in and of itself regardless of any
| functional need for it.
|
| Government regulation has a high potential for abuse for
| a number of reasons (the monopoly on violence is merely
| one) so it makes sense to take steps to constrain it in
| certain critical cases. Note that this doesn't imply
| anything about private entities; it is simply a logical
| consequence of a functional need or ideal in the context
| of our current system.
|
| As to private entities (businesses, etc) things vary
| based on context. Certainly I don't have any wish (for
| example) to force YouTube to host pornographic content.
| However there may well exist cases where broader freedom
| of expression (either the functional need or the ideal)
| requires protection against private entities.
|
| California actually has such a law - an employer is not
| permitted to take actions that would "influence or tend
| to influence" their employees political activities
| outside of the workplace. This can get very complicated
| (as you might imagine) when an employer takes an official
| political stance on an issue.
|
| US telecoms are also subject to regulation of this sort
| (ie common carrier laws). Personally I think that
| infrastructure companies (Cloudflare and other CDNs, as
| well as those providing the physical layer) ought to be
| subject to something broadly similar. It would help
| protect usage of and access to the underlying
| infrastructure for everyone by shielding related
| companies from negative public opinion (public outrage
| campaigns accomplish nothing if the company is legally
| required to serve all customers).
|
| > ... have turned into the cesspools of the internet
|
| Broadly, I think you are tending to conflate
| infrastructure providers with social networks (and
| similar end user sites). There are important differences
| (for example) between FedEx and an online retailer, even
| though they might both be involved in getting a physical
| product to you.
| [deleted]
| dane-pgp wrote:
| You'll be shocked to hear who use the postal system and
| roads, then.
| jonny_eh wrote:
| Provided by the government.
| jariel wrote:
| Nazis also shop at Macy's and drink at Starbucks.
|
| Cloudflare was purely acting out of market based fear,
| there wasn't a hint of moral impetus. Literally he said:
| "I don't want people saying they won't work with us" -
| which is giving into the mob.
|
| Where is the ACLU on this?
|
| We were all screaming for Net Neutrality just a couple
| years ago.
|
| It's up to communities and governments to make decisions
| on content, it would actually help if the government made
| it _illegal_ for CloudFlare to refuse service to someone
| so long as they were within certain guidelines, thereby
| absolving businesses of this issue.
|
| Imagine literally the marketing and PR teams of Verizon,
| Facebook, Cloudflare, AWS, Google, your rando VPN
| provider, getting to decide if they 'think they might not
| like you' or not, it's just too much.
|
| For marketplaces like AppStore, it's fine. But for other
| services, this is not going to work. It's not the job of
| your Telco or Garbage Pickup do decide if your public
| statements are cool/uncool enough for their Instagram.
| AnthonyMouse wrote:
| So no problem with Nazis using USPS but FedEx is to be
| condemned for not opening all your packages and refusing
| the ones with proscribed literature?
| cellar_door wrote:
| This is a bad analogy. Cloudflare is not inspecting
| individual packets for hate speech. They are refusing to
| do business with an organization that negatively affects
| their brand (The Daily Stormer). They should have the
| right to make that choice as a private entity.
|
| Funny you bring up Fedex:
| https://www.cnn.com/2020/07/02/business/fedex-washington-
| red...
| jariel wrote:
| "They should have the right to make that choice as a
| private entity."
|
| No they should not.
|
| And the analogy works: if you're getting filtered on the
| basis of your content - at the packet level or not - then
| it's fundamentally against Net Neutrality.
|
| Wait until the PR team at Verizon decides they don't want
| to publish your content because you're too vocal about
| BLM. Or, they will only support you if you _do_ support
| BLM, or something rubbish. And now your VPN, your server
| host, caching technology provider, Telco, wireless
| provider, Visa /Amex, Video-conf provider - it's
| completely absurd.
|
| FedEx won't ship 'PlanB' because it's a 'controversial'
| medicine? But they will in 3 states?
|
| USPS will ship condoms everywhere but not in Utah where
| the local Union forbids it?
|
| Alaska Big Oil gets their local VPN owners to ban
| Greentech related sites?
|
| Trump's buddies on the Board of AT&T get them to threaten
| anyone hosting 'fake news' about Trump?
|
| California Teachers Union Pension Fund presses Cloudflare
| to ban all hosting of anything related to law
| enforcement?
|
| And FYI nobody is acting 'morally' - they're scared
| executives just trying to do whatever to hush people up
| and continue making money - a system which hands
| arbitrary power to arbitrary groups. This is not what
| anyone wants.
|
| For services that are inherently 'content neutral' - the
| content should not be allowed to be a basis of
| discrimination.
|
| For Social Media it's different, as there is an inherent
| association between the platform and it's users, but not
| for Cloudflare, or AWS or Verizon, Gmail for example.
|
| There is no end to the insanity otherwise; we need basic,
| smart and clear regulation.
|
| Edit: I should add 'and that's just the US'. Imagine when
| a very vocal, organised group wants to ban Arabs living
| in what is commonly referred to as 'Palestine' from using
| the term 'Palestine'. Or Serbian authorities from hosting
| content using the term 'Kosovo' in any way that reflects
| its supposed 'autonomy'. Or Greek companies ganging up on
| Macedonia's usage of the term 'Macedonia'. Or Greens in
| Germany from banning pro-Nuclear energy content. There
| are at least a handful of Tweeters who would want those
| things. It gets infinitely messy, very quickly.
| dane-pgp wrote:
| I wonder if there are people out there who don't want
| internet infrastructure to be run by the government
| specifically because the government would protect freedom
| of speech "too much".
| [deleted]
| nix23 wrote:
| Yes but you don't have to sell your service to
| Nazi's....roads are a official service provided by the
| government/people's taxes, CF is not.
| Reelin wrote:
| This misses the point to an impressive degree. No one
| here is disputing the _current_ legality and I think
| everyone here understands the difference between
| government and private services.
|
| The argument is that we've come to depend on privately
| owned backbone infrastructure in much the same way we
| depend on publicly owned roads. Furthermore, the
| operators of that infrastructure have shown themselves to
| be vulnerable to public outrage. Therefore, it's
| reasonable to ask if additional regulations might be a
| good idea.
|
| A good analogy here might be to privately operated toll
| roads. Surely such a system shouldn't be allowed to
| discriminate against you based on (for example) your
| bumper stickers?
| nix23 wrote:
| >we've come to depend on privately owned backbone
|
| No, you can use any other service you want. CF has no
| private toll roads, in fact you can make your own 'toll'
| road right know. But if you have your private toll road
| you can forbid any bumper sticker you don't want on YOUR
| road.
| largbae wrote:
| Maybe GP is referring to 1.1.1.2 and 1.1.1.3, the DNS
| resolvers that filter malware and malware+adult content
| respectively? Both are optional alternatives to the
| unfiltered 1.1.1.1 DNS service if so...
| 236dev wrote:
| The only ones I can think of are The Daily Stormer and
| 8chan
| creeble wrote:
| Likewise many DDoS-for-hire services, un-ironically.
|
| Dosninja.com for example (there are dozens, though not
| all Cloudflare protected).
|
| Unlike "free speech" sites, the entire point of a DDoS-
| for-hire site is to _suppress_ speech.
|
| Nice to be able to get it both ways!
| triceratops wrote:
| Can't the free market sort that out?
| IAmEveryone wrote:
| Why is it a precedent? Nobody was under the illusion that
| it isn't technically possible for them to shut down some
| customer they don't like.
|
| Nor was/is there any debate about the legality of doing so,
|
| So the only reason for continuing to work for/with violent
| anti-semites was that they wanted to. Until, at some point,
| they changed their mind.
|
| HN has a strange infatuation with this idea of avoiding
| responsibility by pretending to be powerless. But it's
| neither morally sound nor logically or legally coherent to
| pretend to be bound by some principles that are entirely of
| one's own making.
| amylka wrote:
| It will remain at the politically correct shutdowns. Look
| at the list of countries they are doing business with.
| catsdanxe wrote:
| The NSA/CIA have a huge hidden budget. They are known to invest
| in startups. Why try to break crypto when you can just pay some
| tax payer money to large corps to get them to convince sites to
| mitm themselves.
| NorwegianDude wrote:
| I don't really know, but I guess the biggest feature in the
| past was protection against DoS attacks.
|
| Bandwidth isn't expensive compared to what most people are
| paying for it. Cloudflare is paying for the infrastructure as
| it is to handle attacks, so why not use it? As long as it
| doesn't affect paying customers then it's great marketing.
| zenincognito wrote:
| Worth noting that cloudflare does a poor job of identifying and
| allowing crawlers specially Google. I have a site with 1 mil
| URLs , nothing big and every time you turn on cloudflare it
| reduces the crawler to 1k visits or less per day. If you
| compare your logs for visits for Google and Bing with CF turned
| on/off there is a huge difference.
|
| Suffice to say, that in over 5-6 projects I have added CF to,
| it's never worked out in my favour.
| aneutron wrote:
| Okay here's the thing. I'm okay with people bashing other
| (competing) companies when they do wrong. However, I believe it
| is somewhat childish and uncalled for to bash another company,
| because of a mistake.
|
| First of all "I use easyDNS so I didn't notice it at all tbh" is
| not only a childish assertion, it's borderline a falsehood. You
| DO NOT offer the same services, nor the same scale. (No, VOD
| would not work if your VOD provider used Cloudflare's offering.)
|
| Second of all, as some have noted in other comments, you are very
| welcome to get just as big as them if you can offer similar
| (excellent) service and similar extremely competitive pricing.
| Otherwise, keep working on your offer and stop going for low
| hanging fruit like bashing the competitor for an outage when they
| literally might handle 1000x your traffic, and perhaps offer 20x
| the services your offer.
|
| Just a little rant ...
| bszupnick wrote:
| Honestly I didn't notice the domain name, and I actually
| thought the author was being quite understanding by saying
| things like "This is inevitable and unavoidable and entirely
| excusable. Everybody blows up, every DNS provider in existence
| will experience downtime. No exceptions." and that they use
| Cloudflare themselves.
|
| This is obviously subjective, but to me it didn't come across
| as "they suck use us" but rather pointing out the inherent
| flaws in this quite popular SPOF and cautioning to avoid it.
| csharptwdec19 wrote:
| I think the root cause (which, IMO, you correctly point out)
| is lost on many modern developers.
|
| For whatever reason there's this modern idea that if a
| company A is paying money to company B for a service, that
| company B will handle all the 'hard stuff' for them.
|
| The end result is we have a lot of applications/infra built
| with SPOFs, in some cases known, but in many, swept under the
| rug and abstracted away to passing the buck in case of a
| large failure (i.e. major AWS/Cloudflare/Azure outages).
|
| You also see this at times when vendors pitch internal
| software solutions. I've been at more than one shop where a
| vendor's 'silver bullet' turned into a SPOF time-bomb because
| nobody considered this company's solution could fail. After
| all, the sales presentation said it had %nines%!
| TheCoelacanth wrote:
| "Your app will go down when half the Internet goes down" is
| not that big of a deal to most software companies, because:
|
| 1. no one's going to blame me if my app goes down when half
| the Internet is also down but they are going to blame me if
| my custom solution to the same thing causes an outage,
|
| 2. there's no way my custom solution is going to achieve
| the same uptime. AWS/Cloudflare/Azure are not perfect, but
| whatever I roll for myself is almost certainly going to be
| much less perfect.
| [deleted]
| Sephiroth87 wrote:
| They do blame you though, most people won't be aware of
| the real issue, when cloudflare went down, the trending
| things on twitter were #spotifydown and #applemusic
| mercer wrote:
| I suspect customers complaining on Twitter are not the
| ones cared about to whoever decides to use Cloudflare.
| yjftsjthsd-h wrote:
| I build an app. I use CF for my app. Customers use my
| app. CF has an outage. Customers don't care _why_ I 'm
| down, they care that my app isn't working.
| when_it-rains wrote:
| What is your solution for never going down?
| Semaphor wrote:
| Regarding 2, that's exactly what the blog post was
| advocating for: Redundancy. They weren't saying give up
| on Cloudflare (as they mentioned, they use it
| themselves).
| techslave wrote:
| the tone may have been moderate, but the message is smug.
|
| otoh it's just marketing, and CF is no stranger to it, so i
| think it's fair.
| syshum wrote:
| Well if you are going to rant, you might want to rant and quote
| what they ACTUALLY said, not what you conceived in you mind
| what they were saying
|
| You indicate they said " "I use easyDNS so I didn't notice it
| at all tbh" but NO WHERE IN THAT ARTICLE was that statement
|
| The actual quote is
|
| "We're familiar with Cloudflare's DDoS service for DNS
| providers, because we use it ourselves. Fortunately easyDNS was
| not impacted by the outage (I didn't even notice it, tbh),"
|
| This is a MUCH different statement than you attempt to cast out
| as call "childish". He is stating that the services they use of
| CloudFlare was not impacted by the outage
| throw0101a wrote:
| > _However, I believe it is somewhat childish and uncalled for
| to bash another company, because of a mistake._
|
| The thing that they're bashing is not the mistake, which
| happens to everyone. They're bashing SPOF:
|
| > _EasyDNS was unaffected because while we do use Cloudflare to
| soak up large DDoS attacks against our nameservers, we don't
| use them across all of our nameservers. I think somewhere in my
| book I wrote "DNS providers have a near-pathological aversion
| to SPOFs" (Single Point of Failures). Maybe only we do._
| tannhaeuser wrote:
| Whether this is an ad piece by a competitor or not, the problem
| with monopolies is that "the market" (if there is one) gets
| skewed incentives. Cloudflare has received heavy investment by
| FAANG (+MS) [1] before their IPO, so rather than eg Google or
| others with a vested interest and capability stepping up the game
| and invest into new IP control plane-level DDOS protection
| standards or similar, the situation smells more like a backdoor
| deal, such as an agreement to not go after a particular market
| segment.
|
| Let's also not forget Cloudflare in particular have been accused
| to host/hide the very bad boys that make protection from DDOS
| necessary in the first place. Whether or not that is the case, a
| quasi-monopoly leaves customers with no choice.
|
| [1]: https://petri.com/microsoft-google-and-others-invest-in-
| clou...
| ehsankia wrote:
| Didn't people also say the same thing about AWS a while back when
| that had a downtime? I guess the internet has multiple "Single-
| Point-Of-Failure"s.
| KirinDave wrote:
| That's not actually a contradiction. When an individual website
| is considered as a system, it will have multiple points where
| the failure of a component inhibits the system. It's possible
| to have both cloudflare and your database as "SPoFs" and that
| "single" is not meant to imply everyone only gets one.
|
| It's absolutely true that if us-east-1 in AWS has a bad day, a
| significant fraction of the American digital economy will shut
| down. For some companies, the same is true of Azure and
| Google's various comparable offerings.
|
| I read your post as skeptical. Why would you be skeptical? If
| you care about keeping your product up, you absolutely should
| have a fallback for cloudflare if you're a customer of theirs.
| Now, you might not care (and actually, for most folks I submit
| you need not care), but the folks making sure Ambulances get
| timely push notifications and realtime driving instructions
| probably care quite a bit.
| slazaro wrote:
| A "single point of failure" doesn't mean there's only one of
| them. It means the whole fails when the SPOF fails. But you can
| have many of them.
|
| A steel chain made of links has as many SPOF as links.
| benbristow wrote:
| Ironic since the internet (ARPANET) was specifically designed
| to not have a single point of failure
| KirinDave wrote:
| Doubly ironic since in doing this, they created a system
| where a _protocol_ is the SPoF; namely nonsensical or false
| BGP advertisements can quickly kill the internet as a whole
| if done correctly.
| whinybastard wrote:
| it's a series of single points of failure on different levels,
| so, not multiple points of faiure, but much worse, single point
| after single point, which means the house of cards fails as
| often as any of them. The internet of 2020 is a monolith
| T3RMINATED wrote:
| If only DDOS attacks were taken seriously and their perpetrators
| punished accordingly (and maybe if the network had better ways of
| self-defense) instead of companies and websites having to fend
| for themselves.
| drawkbox wrote:
| That is the problem with massive centralization even if it is
| market level and internally Cloudflare (or any other big fish)
| does decentralization/fail-over of their own. Many of these
| companies should have had fail-over to competitors at least for
| reliability.
|
| The problem with near market monopolization, oligopoly, even the
| singularity, the fail-case is catastrophic and may even wipe out
| decentralized, diffused, dispersed, decoupled system solutions
| that can't make it due to so much relative size from the big fish
| that it squashes them along the way. The bigger the ship the
| longer it takes to turn.
|
| This Cloudflare issue is like the recent Facebook SDK startup
| crashes where everyone has a single point of failure on Facebook
| SDK where people should be using or able to use the OpenGraph API
| directly as they need which is more robust to the app that uses
| it, it won't crash on startup.
|
| In business it is a goal to centralize to grow, in nature and
| robust systems it is more differentiation and decentralization to
| survive. There will always be a push and pull between these two
| forces.
|
| Systems and markets are like gardens. The garden must be
| maintained, new seeds planted and helped to grow from small to
| mid-sized, mid-sized plants the bulk of the garden, and then the
| larger plants need to be culled back when they get too big to not
| take the mid-sized and then all the resources from the new
| seeds/small plants. The problem is we have allowed the top end to
| take over the garden and when they fail they fail spectacularly.
| The bigger the scale the bigger they can fail.
| swayson wrote:
| I enjoyed the analogy to gardening. Made me think of the
| terrific principles portrayed in the book The timeless way of
| building.
| Justsignedup wrote:
| The question is: Can we do better? A natural monopoly is a good
| thing. It just means that the natural monopoly needs to build
| its own redundancy. Cloudflare total failure isn't common.
| nemothekid wrote:
| Cloudflare isn't a monopoly. Cloudflare doesn't even have a
| particularly strong moat when compared to Fastly or Akamai.
| Cloudflare doesn't even have any network effects.
| stevenicr wrote:
| I must disagree, and wish I had more data on how many sites
| host with cloudflare because it's free. I just checked,
| fastly is $50 per month, and Akamai doesn't list pricing so
| let's assume it's more.
|
| Cloudflare has had network affects from integration with
| wordpress and cpanel too I believe for some time now.
|
| without cloudflare your site can be taken offline by any
| random person willing to spend $20 for ddos sellers.
|
| The free plan is a pretty big moat imho, especially if
| 'half the internet can fail..' - I doubt 10% of those sites
| would be using cloudflare if there was a monthly fee pushed
| on them.
|
| Admittedly, my cloudflare usage is only about 30 sites, so
| my data point is small. The few hosting and design clients
| I have and their budget constraints are not indicative of
| 'half the internet' - but I don't believe fortune 500 and
| SV sites are either.
| modo_mario wrote:
| > A natural monopoly is a good thing.
|
| How so?
| Spivak wrote:
| If you have a "monopoly" which is this case just means
| you're the market leader by a wide margin, but in theory
| this could mean near total control of a market but the
| _reason_ you have this total control is because your
| customers choose to buy from you over your competitors
| simply because you 're better and or cheaper then this is a
| good thing. Customers are getting what they want. And the
| market leader in this situation will have a hard time
| abusing their position because any they'll have competitors
| nipping at their heels if they slip up.
| Nasrudith wrote:
| I would personally phrase it pedantically as not that the
| natural monopoly is good but that it exists because they
| are good. A fine distinction and a baked in assumption
| admittedly that if they were to no longer be good they
| would no longer dominate. Technically there are some
| other variables like if an extended stay on top would
| atrophy any competitors or not and the time scale
| operated upon.
|
| On a century scale individual murderers aren't a huge
| concern to society because they are either dead or infirm
| by then.
| ehnto wrote:
| > A natural monopoly is a good thing
|
| I think for the sake of a diverse community and economy,
| monopolies should be difficult to achieve even naturally.
|
| But for this specific example, a robust technical redundancy
| doesn't stop CloudFlare from going out of business. A
| technology company going out of business is pretty much the
| norm. Incumbents are a relatively new phenomenon for
| technology (sans some key exceptions), and I don't think
| CloudFlare is an incumbent. They are an accessory, and your
| business would probably run without them.
| itsangaris wrote:
| I wouldn't classify a natural monopoly as a good thing. It
| mostly signifies high barriers to entry for competitors and a
| market that's more susceptible to failure, as seen here.
| Spivak wrote:
| I guess but there's not really a shortage of commercial
| CDNs. A market with lots of competitors but a clear winner
| says more about consumer preference and network (ha!)
| effects than it does about Cloudflare's moat.
| otherprob wrote:
| The bigger problem is that technology advancements gravitate at
| the demands of the noisiest, with the most social gravity.
|
| Yes there are a lot of smart people at FAANG and Cloudflare
| corp.
|
| There are a lot of just as capable folks not driven by job
| addicted meme.
|
| Technically there's no reason the web couldn't be replaced with
| 1:many via Wireshark key sharing based access control to local
| content.
|
| But via Wall Street, along these very particular rules, is how
| we are told to trade information. How is that not a planned
| economy?
|
| Not just by doing what we're clearly interested in doing
| naturally.
|
| Make no mistake: big corp isn't making us login at gunpoint.
| "They" didn't do this. "We" did this.
| dang wrote:
| Could you please stop creating accounts for every few
| comments you post? We ban accounts that do that. This is in
| the site guidelines:
| https://news.ycombinator.com/newsguidelines.html.
|
| You needn't use your real name, of course, but for HN to be a
| community, users need some identity for other users to relate
| to. Otherwise we may as well have no usernames and no
| community, and that would be a different kind of forum. https
| ://hn.algolia.com/?query=by:dang%20community%20identity...
| [deleted]
| saagarjha wrote:
| > Many of these companies should have had fail-over to
| competitors at least for reliability.
|
| How would you even set such a thing up? I fear that you might
| get a couple of collusionary companies that bail each other out
| and smaller providers might just be left out to dry...
| t0mmyb0y wrote:
| Most companies refuse to handle the accounts/sites cloudflare
| handles.
| Qub3d wrote:
| The article posted actually talks about EasyDNS's
| implementation of exactly this.
|
| > At easyDNS we experienced so much pain from this reality
| that we created a system to automate flipping DNS providers
| at the first sign of trouble.
|
| > We call it Proactive Nameservers, and we're the only
| company in the world doing it for some reason. Maybe this is
| because in order to provide a service like nameserver
| failover, it means a company has to admit to its customers
| the reality that their own nameservers may at some point,
| fail.
| KoftaBob wrote:
| I imagine the implementation could look similar to how cell
| phones can use other carriers networks (for 911) when they
| don't have signal from their own carrier.
| mfkp wrote:
| Many DNS providers have the ability to do AXFR "zone
| transfers", so you can sync your records to a secondary
| provider and you would add a secondary set of nameservers for
| redundancy. Unfortunately Cloudflare doesn't offer this
| unless you pay for their Enterprise plan (they started
| offering it earlier this year).
|
| I do love using CloudFlare for DNS, lots of great features
| and generally works well, but I wish they would support AXFR
| for the lower tiers. I've been working on a solution for this
| using the CloudFlare API, but we'll see how well it works
| out.
| icedchai wrote:
| Amazing that this isn't "free" and folks have to resort to
| syncing records across DNS providers with proprietary,
| vendor-specific APIs. AXFR is standard. It's not just
| Cloudflare... AWS and Azure don't support it either.
| DoctorOW wrote:
| With Cloudflare's popularity, I'll bet if they started
| supporting it, others would too.
| WayToDoor wrote:
| Fwiw, you could use stack overflow DNS Control to manage
| your DNS records and upload them to many providers. Then
| the only thing you'd have to do is flip a line of code to
| fallback to the other DNS provider.
| mfkp wrote:
| Nice, didn't know about that tool. Checking it out now:
| https://github.com/StackExchange/dnscontrol
| FireBeyond wrote:
| Back in the day (I haven't run my own mail server for years
| and years) there was a company called Secondary MX.
|
| That's all they did. They weren't a mail host or provider,
| there was no UI, nothing. All they did was allow you to
| specify them as, well, a secondary MX, so if you were
| offline they'd cache your inbound email until you were
| back. Simple. Efficient.
| 1996 wrote:
| I think they want to price discriminate: the costs from
| AXFR should be minimal, as it is an old technology, very
| optimized and low bandwidth.
|
| However, cloudflare decision turns cloudflare "free"
| offering into a free SPOF. They should extend this offer to
| their free users, who could then use the secondary DNS that
| most hosts/domain name sellers provide for free to the IP
| that is proxified by cloudflare. It could even be limited
| to the case of proxy or cloudflare DNS failure, so that
| cloudflare could still price discriminate (make AXFR fail,
| unless cloudflare is down, like a dead man's switch)
| twblalock wrote:
| There is no CDN monopoly. There are several to choose from.
| Cloudflare is one of the new kids on the block.
|
| There is no cloud monopoly either. Customers can choose from
| AWS, GCP, Azure, and several others.
|
| The problem is not market concentration. There are plenty of
| options. The problem is customers choosing to put all their
| eggs in one basket.
| jonplackett wrote:
| How do you do a fallback from cloudflare failing when they're
| your dns provider too? Any redirection around would take too
| long to change wouldn't it? They'd be back up and running
| before it was implemented. What's the right approach?
| 0xEFF wrote:
| Don't use them for DNS, just point 60 second TTL A records
| at them.
| jniedrauer wrote:
| Where _do_ you put your DNS servers? On-prem is going to
| be less reliable than cloudflare which has a 100% uptime
| SLA. I doubt your local ops team can compete.
| jonplackett wrote:
| my thoughts exactly. still none the wiser what the
| solution is.
| phire wrote:
| I don't think cloudflare supports such a configuration.
|
| The DNS is part of the load balancing, they serve
| different IPs based on location of the DNS query.
|
| Edit: Apparently they do support a CNAME configuration if
| you pay for one of their business plans. That gives you
| the option to quickly switch away (if your TTL is low
| enough) but will impact performance by having to fetch
| the CNAME every 60 seconds.
| EE84M3i wrote:
| Does cloudflare actually do geo-loadbalancing via DNS A
| records now? For years they only did anycast, unlike,
| say, Akamai, which hands out different IPs for each POP.
| phire wrote:
| Actually, I'm not sure if they do any DNS geo-
| loadbalancing. I've seen it report different IPs from
| different locations at times, but that could be something
| else.
|
| But I'm pretty sure they use DNS do other loadbalancing
| and DDoS mitigations.
|
| For example, if a site is under attack, they can send it
| to different IP addresses to keep it away from other
| sites. Or if someone is directly targeting a cloudlflare
| IP with a DDoS, they can redirect all sites to other IPs
| and just blackhole that IP.
| contingencies wrote:
| _The problem is customers choosing to put all their eggs in
| one basket._
|
| Until relatively recently absolutely none, and now almost
| none of the tooling allows effective multi-cloud or hybrid
| cloud/private.
|
| Basically the cloud providers work very hard to prevent the
| commodification of their services with special incompatible
| service offerings, lock-in, interdependency, deep and opaque
| APIs for integration, and networks of training and
| certification that position change as a direct threat to
| people's job security.
|
| Cloud providers today are basically Microsoft in the 90s.
|
| Much as open source challenged Microsoft, I would say that
| the world now needs open infrastructure tooling that
| positions hybrid and multi-cloud as first class
| infrastructure architectural cases in order to displace
| established cloud provider hegemony. Even then we will have
| to fight the hardware and real estate economies of scale
| available to large established cloud providers.
|
| I wrote some observations about this space based
| significantly on HN community comments prior to the rise of
| Docker a few years ago: http://stani.sh/walter/pfcts/ ...
| click 'original' ... the conclusions still seem timely.
| abtinf wrote:
| > Until relatively recently absolutely none, and now almost
| none of the tooling allows effective multi-cloud or hybrid
| cloud/private.
|
| Until relatively recently, the cloud didn't exist.
| icedchai wrote:
| The cloud is marketing speak for what has been going on
| for decades. In the 70's, it was called time sharing.
| contingencies wrote:
| EC2 was launched in 2006. The notes I linked to date from
| 2014.
| david-cako wrote:
| True, however at AWS at least, customers are specifically
| told "multi-cloud doesn't allow you to fully leverage the
| benefits of AWS", whatever that means.
|
| It makes sense that cloud companies are inclined to keep
| customers from giving money to competitors, but they way they
| sell it and structure services, reserved instances, and
| enterprise discounts is such that you basically are putting
| all of your eggs in one basket.
| Nasrudith wrote:
| I thought that was something trivially obvious stated about
| fully leveraging the benefits - that they don't control
| other's clouds and thus they can't use all of the same
| sorts of performance or efficency boosting tricks. (People
| would be way more mad if they did as it would require
| hacking into say Azure to gain root access.) Not a domain
| expert but it is my interpretation.
|
| If you make the engineering decision to go multicloud for
| whatever reason those are inherent trade offs you need to
| be aware of. They have their own agenda of course in
| addition to any actual fundamental "real" in bulk
| efficiencies that price reflects.
| Spooky23 wrote:
| People are given the agency to make their own decisions.
|
| Nobody got fired for buying IBM, until they did.
| blahyawnblah wrote:
| Probably means that if you use a service that only aws
| offers and you failover to a competitor your
| product/service/website will be degraded
| theptip wrote:
| > "multi-cloud doesn't allow you to fully leverage the
| benefits of AWS", whatever that means.
|
| One of the selling points of cloud providers is managed
| services like SQS. If you run a multi-cloud architecture,
| you either can't use managed services, or have to build
| abstraction layers on top of them (and only use the
| features that exist in both cloud providers' versions of
| the managed service).
|
| If you want to use a managed service that only exists on
| AWS, then that's obviously incompatible with a fully multi-
| cloud architecture.
| AnthonyMouse wrote:
| > If you run a multi-cloud architecture, you either can't
| use managed services, or have to build abstraction layers
| on top of them (and only use the features that exist in
| both cloud providers' versions of the managed service).
|
| And this is, of course, why they do everything they can
| do discourage it. Because if you do that, not only are
| you not reliant on them for availability, you can switch
| more of your business to the other provider(s) based on
| current pricing, and they do not want that big time.
| cstejerean wrote:
| Is this GPT-3 or a real comment?
| LoathsLights wrote:
| Calm down there mr buzzwords, this is not a job interview.
| nickreese wrote:
| This is just an advertisement for easydns.
| superkuh wrote:
| The more people that use anything other than Cloudflare, the
| better. I had this conversation with people back in ~2010 about
| Facebook and they all ignored it. They will again, but this
| time the consequences of centralization will be even worse.
|
| It won't just be one single website that goes shitty with
| blockages and manipulation and censorship. It won't even be
| just the web. When Cloudflare achieves their goal of deep
| packet inspection at every peering and transit point it'll be
| the end of the internet as we knew it and the slow transition
| to just another cut apart "China-net (tm)".
| lopis wrote:
| Bingo. Still a good read, and easydns is just trying to profit
| over a screw-up from a competitor, but still essentially an ad.
| toomuchtodo wrote:
| > but still essentially an ad.
|
| Compared to the endless content marketing Cloudflare posts
| [1]? It's an ad, but they're still right. That's just good
| content marketing (informative, relevant, and perhaps you buy
| something because of it).
|
| [1] https://news.ycombinator.com/from?site=cloudflare.com
| badRNG wrote:
| Interesting perspective, but it seems like this is just an ad for
| easyDNS and their "Proactive Nameservers," though I couldn't
| imagine a better time than the misstep of a behemoth of a
| competitor in this space. Not to detract from the more important
| discussion about the internet's dependence on Cloudflare overall.
| nvahalik wrote:
| Hey, never let a competitors misfortune (misstep?) go to waste!
| donmcronald wrote:
| > Proactive Nameservers is a patent-pending system that
| optimizes the nameserver delegation for your mission critical
| domain names.
|
| That's a huge negative and I can't believe they think it's a
| good marketing point. I don't want a patent encumbered, non-
| standard solution for critical infrastructure.
|
| > We must be your domain registrar for this to work.
|
| So they're updating the domain record at the registry level to
| facilitate failover? That's the only scenario I can think of
| where they _need_ to be your registrar. Assuming that's the
| case...
|
| I've always seen 24-48 hours quoted as the worst case wait when
| updating nameservers at the registry. I've never seen an
| explanation of how it works, what's allowed to be cached, how
| long it actually takes to update, etc.. How do they do it in a
| way that's suitable for failover? Do they have a special SLA
| with registries?
|
| How would the registries handle a deluge of nameserver updates?
| Imagine a Cloudflare scale failure and corresponding registry
| updates. Would the registry servers be able to handle it?
|
| I'd love to see a technical explanation of how their proactive
| nameservers system works.
| sradman wrote:
| It may be just an ad for easyDNS' _Proactive Nameservers_ [1]
| product but it provides a roadmap for one possible solution to
| this type of problem. From a quick reading of the marketing
| info, the solution can be summarized as "Provision, Monitor,
| and Fail-Over DNS Name Servers across multiple DNS-as-a-Service
| providers". The question I have is whether the following
| constraint is artificially introduced or not:
|
| > We must be your domain registrar for this to work...
|
| IIRC, Netflix OSS published some tools quite some time ago to
| support multiple DNS providers but I don't know/remember if
| they tackled the availability problem. The question comes down
| to build vs buy and whether the solution is general enough to
| warrant an Open Source Software solution.
|
| [1] https://easydns.com/dns/proactive-nameservers/
| pas wrote:
| They want to be the registrar to be able to update your NS
| records. But ... that's not really important nor needed (So
| the answer to your question yes, it's likely artificial).
| Just use two anycast-ed IPs/domains. (Like Cloudflare.)
|
| The magic happens at BGP level.
|
| I considered CF as a domain registrar, but they don't allow
| setting the NS records. So you must use them. (They basically
| use sane no-nonsense domain registration as a way to gain
| leads for their main product. Pretty smart actually, because
| it's a great high-level add-on for their main product, but
| they just went ahead and made that the bait for everyone.)
|
| Anyway, ideally, if you add 2 separate sets of NS servers to
| your NS records then you eliminated this SPoF, great. Sure,
| it's your job to keep them updated, and in sync (preferably,
| to avoid problems like half of your users landing on a
| different CNAME/IP/etc).
|
| And recursive nameservers will handle the failover.
| StuntPope wrote:
| easyDNS has to be the registrar because only your registrar
| can change your nameserver delegation with the registry.
| This is, in essence, the registrar's job. To maintain your
| domain record and info, including nameserver delegation,
| with the registry.
|
| You could do it with BGP, but it is non-trivial and you
| need your own ASN to do that.
| pas wrote:
| But you can just add multiple DNS providers yourself. I
| mean you can add the namservers of both easyDNS and
| cloudflare. EasyDNS just automates this.
|
| In theory they could simply create a few subsidiaries,
| let's call them saferDNS1,2,3 and have them build
| completely different redundant DNS architectures, and add
| then add the resulting nameservers.
|
| That said, it'd be good to see an actual domain that uses
| this "proactive" feature to see what easyDNS is doing.
| donmcronald wrote:
| I'm not a DNS expert, so... It's not really that simple
| is it? If you have multiple nameservers I thought they
| get equal weight, don't they?
|
| So if you have Cloudflare + (ex:) NS1, and you're using
| Cloudflare for caching, you need your NS1 records to
| return Cloudflare proxied IPs normally, but origin IPs
| under failure conditions. That's a lot of infrastructure.
|
| It also fails completely if you're relying on Cloudflare
| for DDoS protection and IP obfuscation because a failure
| means your origin IPs get exposed. That's assuming
| Cloudflare DNS being down means Cloudflare proxying is
| down too. It might not be the case, but I think you'd
| have to plan for it.
|
| Then there's also Cloudflare's detection of nameservers.
| I haven't tried it with more than Cloudflare's
| nameservers set for a domain, but if your domain doesn't
| actively use their nameserver they'll drop your site from
| their system. So, at the very least, you can't use
| Cloudflare as a secondary DNS provider (at least the last
| time I checked).
| 1996 wrote:
| > Just use two anycast-ed IPs/domains
|
| What are the brands offering DNS at a flat rate over
| anycasted IP over a few continents?
|
| Most of the offers I see are per query at high rates.
| Setting up my own ASN to do this would be too expansive in
| IPv4
| pas wrote:
| I meant that easyDNS should handle the BGP for its
| clients, without requiring their clients to use them as
| registrars.
|
| There's HE.net's free DNS, and though they don't
| explicitly advertise as, it's anycasted. (Check via
| https://tools.keycdn.com/ping , try 216.66.80.18
| [ns5.he.net].)
|
| https://www.cloudns.net/premium/ seems to be quite
| affordable with no query limits :o
| Semaphor wrote:
| From TFA:
|
| > The only requirement to use Proactive Nameservers is that
| we have to be your registrar, because we need to connect to
| the registry to update your nameserver delegation.
|
| So I guess technically this could be achieved with an API for
| your domain settings.
| chronid wrote:
| > IIRC, Netflix OSS published some tools quite some time ago
| to support multiple DNS providers but I don't know/remember
| if they tackled the availability problem.
|
| The classic way of doing this is AXFR (your own DNS server is
| a "hidden master" and the DNS providers are the slaves).
|
| The problem is you won't be able to have redundancy at the
| registrar level, but that has historically at least been less
| of an issue.
| [deleted]
| niutech wrote:
| The solution is the Decentralized Web (DWeb), such as IPFS
| (https://ipfs.io), Freenet (https://freenetproject.org), GNUnet
| (https://gnunet.org) or Hypercore (https://hypercore-
| protocol.org). We should start using them to avoid centralization
| and embrace freedom.
| cortesoft wrote:
| Can any of those handle massive scale?
| nemothekid wrote:
| Can you explain a bit more how this is a solution? Cloudflare
| isn't facebook. They have plenty of competitors, they don't
| have a massive moat, and they are almost exclusively used by
| businesses. Despite all of this, we should ask ourselves how we
| even got here. Why would companies move to DWeb, when they are
| already choosing to use Cloudflare instead of
| Fastly/Cloudfront/Akamai.
| return1 wrote:
| No salespeople to sell dweb
| mattl wrote:
| How do I publish a website on one of these?
| yjftsjthsd-h wrote:
| https://www.dgendill.com/posts/technology/2019-10-15-publish.
| ..
| zelphirkalt wrote:
| Well, I usually block it anyway, because it is not only a single
| point of failure, but also a single point of concentration of
| data, that can be used to track, spy on and profile users. As
| such, I do not blindly trust Cloudflare. I remain sceptical, no
| matter how positive their public image is. Especially I would not
| set my DNS to cloudflare.
| ampersandy wrote:
| Who do you use for DNS then that you do trust?
| yjftsjthsd-h wrote:
| For lookups, it's not that hard to do your own recursive
| resolver.
| phkahler wrote:
| If you keep much of a website simple - plain html and css - won't
| that reduce the need for a CDN in the first place?
| [deleted]
| [deleted]
| foxfired wrote:
| Yes, but cloudflare reach is much deeper then that.
|
| I am not a cloudflare customer but my all websites failed that
| day. The reason was digitalocean uses cloudflare, I use
| digitalocean. So apparently I depend on cloudflare.
| hinkley wrote:
| High availability is an insurance game, and perhaps we need to
| start treating it that way.
|
| Rather than admitting that your customers need to maintain a
| business relationship with your competitors, _you_ need to admit
| you need to maintain a business relationship with your
| competitors. That we need a moral equivalent of underwriting in
| the cloud space.
| yokaze wrote:
| I am not sure, if I would trust anyone who misuses the term
| "Single-Point-of-Failure" on matters of reliability.
| yjftsjthsd-h wrote:
| Why not? It's a single thing, that if it fails, causes your
| app/website/whatever to fail.
| yokaze wrote:
| Because it isn't a single thing, it is a redundant system.
| Redundant systems can also fail, that doesn't make it a
| _single_ point of failure.
|
| You can add another system in parallel as the vendor of the
| product suggests, or you can improve the resilience in the
| redundant system.
|
| To make a hyperbole: my galera cluster is failing, its a
| single point of failure, so I setup a cockroach cluster in
| parallel.
|
| In a way, it is right, as there are failure modes specific to
| the individual systems, but I think, it is incorrect, to
| label that a SpoF.
| surround wrote:
| Instead of trusting any DNS provider with your queries, I
| recommend running your _own_ recursive resolver with Unbound.
|
| https://nlnetlabs.nl/projects/unbound/about/
| everfree wrote:
| But then aren't you just trusting your VPS provider or ISP
| instead?
| surround wrote:
| Unfortunately, ISPs can see the domains you connect to, even
| if you use DNS-over-HTTPS.
|
| https://irtf.org/anrw/2019/slides-anrw19-final44.pdf
| eloff wrote:
| Now that Cloudflare is also a registrar, they could pretty easily
| implement a nameserver failover like EasyDNS. I hope this event
| underscores the importance of that to them.
|
| It's worth noting Cloudflare also supports secondary DNS, but
| only for enterprise customers:
| https://blog.cloudflare.com/secondary-dns-a-faster-more-resi...
| psim1 wrote:
| What responsibility and reparative measures has Cloudflare taken
| for Friday's incident? Was anyone fired for the mistake?
| sujinge9 wrote:
| Why would firing someone make you feel like reparative measures
| have been taken?
| psim1 wrote:
| Reparations would show an actual sense of responsibility.
| Firing someone would be appropriate if they were negligent.
| Other measures might be more appropriate. Is it enough that
| any time there's a Cloudflare incident, all we get are
| lengthy blog posts and sorries from Cloudflare?
| gizmo385 wrote:
| Firing people for making mistakes is just going to foster a
| culture of secrecy and shame. Being so quick to fire is not
| how you retain talent and it isn't how you foster a
| healthy, blameless development culture in your workplace.
| axaxs wrote:
| Firing people for making mistakes is a great way to watch
| productivity dive. The easiest way to prevent mistakes is
| to do nothing at all.
| psim1 wrote:
| I understand the point being made here, but what are
| those affected supposed to take away? Cloudflare made a
| mistake that caused (x millions of dollars of lost online
| commerce revenues, y number of missed telehealth
| sessions, etc.) and since we do not punish mistakes,
| nothing was done. Sorry everyone!
| axaxs wrote:
| Understood. Typically, as a company, you write - 1) what
| went wrong 2) how did it go wrong and 3) what you have or
| will put in place to prevent it from ever happening again
|
| It's a learning process for all involved, really.
|
| From the affected parties point of view, well, they
| should diversify their network a bit better. End users
| should hold those companies feet to the fire, not
| Cloudflare's.
| yjftsjthsd-h wrote:
| > Firing someone would be appropriate if they were
| negligent
|
| Maybe, but making mistakes is far from negligence. Besides
| which, if a single person _can_ accidentally break your
| system, at least at Cloudflare 's scale, that's an
| organizational failure, not a personal failure.
| knorker wrote:
| The other half is on AWS?
| synaesthesisx wrote:
| What's funny about this outage is I'm sure many of of us (myself
| included) used this window to analyze large services and
| determine an increase in major Cloudflare customers and
| presumably, revenue. Even ISPs like T-Mobile faced issues due to
| the Cloudflare outage! The situation has exposed just how
| critical Cloudflare is.
|
| I went ahead and bought calls ahead of NET earnings next month.
| Cloudflare is becoming an increasingly bigger part of the
| internet backbone. Purely speculating here, but I wouldn't put it
| past AWS or another large player acquiring them soon.
| giancarlostoro wrote:
| > I wouldn't put it past AWS or another large player acquiring
| them soon.
|
| I really hope it doesn't come to this sadly. I'd be okay with
| DO or somebody who isn't as massive doing a merger. Maybe they
| can pull resources to make each other even successful and
| maintaining reasonable independence.
| iamnothere wrote:
| > Purely speculating here, but I wouldn't put it past AWS or
| another large player acquiring them soon.
|
| Internet(tm) by Amazon.
|
| I really hope it doesn't come to this. I assume such a move
| would create a vacuum for a competitor. Not everyone wants to
| be completely owned by AWS.
| dharmab wrote:
| > an increase in major Cloudflare customers and presumably,
| revenue. Even ISPs like T-Mobile faced issues due to the
| Cloudflare outage!
|
| Careful about this methodology. Some services at my org were
| impacted despite not being direct CloudFlare customers. They
| had external dependencies that used CloudFlare.
| Fiveplus wrote:
| So it's much bigger proverbial 'blast-radius' lest something
| happen to CloudFlare? Can you elaborate a bit on that part?
| I'm interested in knowing more.
| dharmab wrote:
| Simple case of dependencies failing. Not much to elaborate.
|
| e.g. NPM.js uses CloudFlare DNS, so services which needed
| to talk to NPM.js weren't able to do so.
| tuxninja wrote:
| AWS in 2012, DynDNS after that, now Cloudflare...I wrote about
| this a few years ago, the threat of the singularity of the
| Internet. What was distributed will be centralized again.
| http://tuxlabs.com/?p=430
| ivanvanderbyl wrote:
| Pretty poor form calling out a competitor for something like
| this. Not the first time a DNS provider has done this, and won't
| be the last.
| johnghanks wrote:
| lmfao this is literally a hit piece from a competitor.
| TedDoesntTalk wrote:
| > Turns out half the internet has a Single-Point-of-Failure
| called "Cloudflare"
|
| The other one is called AWS.
| louwrentius wrote:
| So in the end, who cares about single point of failures?
| actuator wrote:
| Does anyone know a dashboard/list for past Akamai outages.
| Surprisingly, I haven't seen a lot of news about Akamai
| downtimes. I searched on Google and the last one I found in a
| news report is from 2011 when its customers like Facebook,
| Twitter were impacted.
|
| They were a PITA to work with when I used them in the past but if
| they are really that good in service availability, you can have
| some justification for their overpriced service.
| EE84M3i wrote:
| AFAIK Akamai only makes their service notifications available
| directly to subscribers.
| valuearb wrote:
| "Cloudflare apparently fat-fingered a routing update and sent all
| of their global traffic to a single POP, vaporizing it almost
| instantly."
|
| Made me chuckle, as it gave me the image of a large server in
| some massive server farm glowing red, then bursting in a massive
| burst of light as dozens of bearded Sysadmins run out of the
| building screaming.
| [deleted]
| arkitaip wrote:
| > We call it Proactive Nameservers, and we're the only company in
| the world doing it for some reason.
|
| Wait, why? [0]:
|
| > Proactive Nameservers is a patent-pending system that optimizes
| the nameserver delegation for your mission critical domain names.
|
| Oh.
|
| [0] https://easydns.com/dns/proactive-nameservers/
___________________________________________________________________
(page generated 2020-07-20 23:00 UTC)