[HN Gopher] Set up a practically free CDN
___________________________________________________________________
Set up a practically free CDN
Author : frogger8
Score : 234 points
Date : 2022-01-15 15:58 UTC (7 hours ago)
(HTM) web link (gist.github.com)
(TXT) w3m dump (gist.github.com)
| modeless wrote:
| I've been using Cloudflare free in front of App Engine's free
| tier for a few years and it really seems too good to be true. I
| can go viral and scale to infinity, for exactly $0.
| anaganisk wrote:
| The other day one of the commenters under a post wanted to
| spend $20 per month to host a static website to handle HN
| traffic, I just LOL'd. I tried to be good and explain why it
| was so unnecessary but nah they would rather bring a canon to a
| water gun fight.
| anderspitman wrote:
| Yeah a static webserver on a low end VPS is more than
| sufficient for front page traffic.
| indigodaddy wrote:
| Cloudflare in front of Replit or Vercel would be same idea
| right
| modeless wrote:
| Sure. Or a full cloud instance running on a free tier
| somewhere. There are tons of ways to host low traffic sites
| for free that can be made to scale to high traffic for free
| by slapping Cloudflare in front. As long as your site is
| mostly static.
|
| In my case, I'm serving data that updates every 10 minutes
| but is otherwise identical for every user. Cloudflare handles
| that use case perfectly.
| boffinism wrote:
| > 440% cheaper than S3 per GB.
|
| However the author thinks percentages work.... They are not
| right.
| guy98238710 wrote:
| The wording is indeed confusing, but not every interpretation
| is wrong. They might mean 440% more efficient, i.e. 5.4x
| cheaper.
| lucumo wrote:
| That's still an innumerate phrasing.
| parsadotsh wrote:
| It does make sense. Consider the phrase "twice as cheap"
| mulmen wrote:
| I have always found that phrasing to be confusing. Consider
| the reciprocal, "half as cheap". Should that be interpreted
| as... double?
|
| Say "half the cost" or "cheaper by half".
| Dylan16807 wrote:
| > Consider the reciprocal, "half as cheap". Should that be
| interpreted as... double?
|
| If the phrasing makes it clear they're talking about a
| higher price, then yes that's a good interpretation. You
| get half as good a deal.
|
| Someone might say "half as cheap" to talk about a price
| drop by 50%, even though that's not really correct, but
| such is language.
| boffinism wrote:
| What is 440% less than 1, then? In my book, it's -3.4.
| [deleted]
| sascha_sl wrote:
| It should be percentage points, because "100% cheaper" would
| already be free.
|
| It probably means AWS is 440% more expensive.
| Rastonbury wrote:
| Technically yes but no person is reading that and thinking
| backblaze is paying you or its free. Its a nitpick
| Blahah wrote:
| It doesn't make sense. Consider the phrase "once as cheap".
| Multiply by two. That's twice as cheap. Meaningless.
| Dylan16807 wrote:
| Do you think "twice as expensive" is meaningless? Because
| by that same logic, if I start with "once as expensive"
| that's not a number and multiplying by two gives me a
| meaningless concept.
| Blahah wrote:
| Once as expensive is a number, it's 1 x the cost. Twice
| as expensive is 2x the cost. Once as cheap might arguably
| be 1x the cost, but twice as cheap is undefined.
| listenallyall wrote:
| codefreakxff wrote:
| My sarcasm detection is not great. Looks like a troll post.
| Maybe it is real? If it is real, 50% down would mean it lost
| half its value. 100% down it lost all its value. Even if it
| was up one billion gillion percent, if it lost 100% of it's
| value it would be worth zero. But, yes, Enron and Madoff and
| many brokers are not out to protect your investments
| ralferoo wrote:
| Then again, the title of the post says "practically free" and
| goes on to say it's actually $5 a month, so I took the rest of
| the figures with a grain of salt...
| [deleted]
| howdydoo wrote:
| B2 itself is not $5 a month. That $5 service is a separate
| consumer-focused "unlimited" backup service from the same
| company. B2 just charges for storage and transfer. Yeah, the
| author was rambling a bit.
| zild3d wrote:
| yeah pretty sure they mean 77% cheaper
| d1sxeyes wrote:
| Driven by the fact that using comparative adjectives in this
| context seems to be at best confusing, I thought I'd play
| around with some numbers to see if I could understand what the
| author was trying to convey.
|
| First, it's important to clarify why comparative adjectives
| don't work here. Look at the following sentence.
|
| - A $5 cookie is _worth twice as much as_ a $2.50 cookie
|
| Here, we establish the baseline (the $2.50 cookie), and express
| how much better the $5 cookie is.
|
| We could also say the inverse:
|
| - A $2.50 cookie is _worth half as much as_ a $5 cookie.
|
| What we can't do is the following:
|
| - A $5 cookie is _double more expensive_ than a $2.50 cookie.
|
| We can make a statement (the cookie is more expensive), or
| share the factor by establishing the baseline (twice _as
| expensive as_ ), but not both at the same time.
|
| The author tries to say 440% cheaper, (not mentioning storage
| or transfer) and I believe their logic was something along the
| lines of the below:
|
| - 440% = 440/100
|
| - Cheaper = Reciprocal of More expensive
|
| - 440% cheaper = 1 / (440/100) = 100/440
|
| - S3 is very difficult to estimate a 'per GB price' because it
| depends on where the data is being transferred to, various
| pricing tiers (S3, S3 _intelligent_ , 'Glacier', 'infrequent
| access', 'One Zone', first 100GB free, egress through
| Cloudfront, and HTTP verb specifics). Fortunately, they have an
| estimate calculator, which gave me 9 USD for 100GB transferred
| out to the internet, or 0.09USD/GB for transfer.
|
| - Storage is a different proposition, with Amazon telling me
| it'll cost 2.30 USD for 100GB as long as I never look at it
| (ie: 0.023USD/GB)
|
| - Backblaze is much easier to reckon with, with $0.005/GB for
| storage, and $0.01 for transfer.
|
| - Therefore, for _transfer_ Amazon 's 0.09 * (100/440) = 0.02,
| which is MORE than Backblaze would charge me.
|
| - For storage, Amazon's 0.023 * (100/440) is indeed 0.005.
|
| So, Backblaze does indeed seem to be [440%, reciprocal of more
| expensive] as Amazon for storage cost alone.
| punnerud wrote:
| Does it support byte-range? so it can be used as a "server free"
| SQLite database: https://phiresky.github.io/blog/2021/hosting-
| sqlite-database...
| anderspitman wrote:
| I'm pretty sure all S3-compatible services support byte ranges.
| Ready for someone to come out of the woodwork to prove me
| wrong.
| onphonenow wrote:
| Some caution with cloudflare:
|
| Because they give a lot away and post blog posts attacking AWS
| folks think they are cheap. But a lot of their free stuff may be
| in the marketing budget. The issue here is:
|
| 1) Their paid offerings for for core product, CDN, does not have
| a public pricing guide. That's a BAD sign if they claim to be
| competing on price. So we can pretty safely assume their paid
| service bandwidth charges are NOT the 80X cheaper than amazon
| despite their claims here.
|
| 2) They hype their bandwidth alliance. I tried following this up
| on Azure, it's basically all call a sales person / maybe you'll
| get a deal type stuff. Smoke and mirrors again.
|
| 3) If you actually try to use their CDN to serve something high
| bandwidth (games / software updates / video streaming) that is
| going to violate some type of clause which says that the free CDN
| is not REALLY unlimited.
|
| 4) Financially they spend a mind boggling amount on sales costs.
| Between sales costs and cost of sales I think they are negative
| relative to revenue.
|
| BTW - If you try any of the other "free" and "unlimited"
| providers (for $5/month etc) you will also find out once you try
| to move some real data volumes (video etc) - it's all a lie over
| and over.
|
| Same things with geo issues. Cloudflare claims to have "one"
| price globally. These are almost alwyas false claims. It's easy
| to test - setup a stream into a high cost area, stream a bunch of
| video there - you'll usually get a call over some tricky TOS
| issue they pull out.
|
| In short, if you want pay and forget, there is a reason the big
| cloud players have a business still.
| frogger8 wrote:
| Looking at your comment history, you mention AWS often. Do you
| work there?
| roblabla wrote:
| They're one of the top, if not the top, cloud providers. It's
| not surprising to mention AWS here, and GP also mentions
| Azure...
| samhw wrote:
| I'm aware that I'm inviting a "bad bot" riposte by saying
| this, but it's:
|
| > one of the top cloud providers, if not the top
|
| rather than:
|
| > one of the top, if not the top, cloud providers
|
| since the latter cashes out to:
|
| > one of the top cloud providers, if not the top cloud
| providers
|
| (Sorry. I know it's annoying ... but it's my life's mission
| to eradicate this particular construct.)
| ani-ani wrote:
| Assuming this is for websites, what's the recommended way of
| handling cache busting with this kind of setup? The article has a
| slightly awkward paragraph that tells you to set long TTLs for
| everything and concludes with "tweak these settings", which kind
| of sounds like "draw the rest of the owl". Might be a good reason
| to use something that integrates all this (eg. Cloudflare Pages,
| since you're already using Cloudflare).
| ricardobeat wrote:
| The usual approach is to add a value to the query string, like
| a last-modified timestamp or deployment version. Doesn't
| require any setup on the CDN side.
| sharps_xp wrote:
| cloudflare let's you purge the cache via API, but frequent
| busting is probably not okay
| samwillis wrote:
| This B2/CloudFlare combination is a good one.
|
| CloudFlare are about to launch their own object/file store with
| native CDN called R2. They have even come up with a wonderful
| trick for migrating to it from S3 and it's looks like it's going
| to have an incredible generous level of free egress. I suspect
| that it will beat out the B2/CloudFlare combination once it
| launches.
|
| https://blog.cloudflare.com/introducing-r2-object-storage/
|
| Edit: Typo
| outloudvi wrote:
| Yes. Cloudflare is doing many of their own stuffs recently:
| serverless, image processing, stream processing, mail
| forwarding, storage etc.
| atlgator wrote:
| Cloudflare stock is cheap right now too.
| dillondoyle wrote:
| What is your opinion on what's driving the movement?
|
| I went in fairly big (for me) and sold near the top with a
| stop loss #gainz. Then bought again when it went down like
| 10% trying to 'buy the dip.' But it keeps falling!
|
| I don't get it. I love their products and ambition. That
| was a huge bump and decline.
| candiddevmike wrote:
| Not the parent but I think Cloudflare's free tier is too
| good and the paid offerings don't compete well if you're
| all in on a cloud provider already. It also may be the
| classic Cloudflare has "features" but doesn't have a
| "product".
|
| Anecdotally, I have yet to work with a company paying
| Cloudflare substantial amounts of money.
| dillondoyle wrote:
| Interesting. I like that whole view of a product. Like
| how do you describe Cloudflare's produce in one sentence.
|
| Maybe something about 'large edge network cloud to reach
| your customer quicker' or something I'm not sure how to
| describe it in a compelling way. Maybe sprinkle on some
| web3 or distributed bs.
|
| I think AWS, now Google is getting there, have that a lot
| of stickyness because it has all the services imaginable
| interconnected (not well sometimes but still). It's a
| platform not individual products. It's hard to pickup and
| leave, especially if Cloudflare doesn't have 1:1
| replacements.
| samwillis wrote:
| I feel they are fairly early on in there "full stack"
| strategy. I believe they are moving quickly from
| launching MVPs for various products to expanding them as
| rapidly as they can. They are also purposely not trying
| to compete 1:1 with the incumbents with the "full stack"
|
| "Workers" is brilliant, but the strategy with WASM on it
| is where I really think it is going to shine. The more
| languages and more of your stack that can be compelled to
| WASM the more you will be able to run on their edge. I
| truly believe WASM is the future in so many places other
| than in the browser.
|
| I also feel the one part of their stack that's missing is
| a transactional acid database (I hope for sql, but nosql
| would be fine). But it highlights a fault with the "edge
| first" strategy. Workers at the edge making multiple
| round trip requests to a central database is slower than
| running your app closer to the db. My wish is that they
| are developing some sort of edge replaced db that
| automatically launches read replicas at the same edge
| locations as where most of your worker load is. You
| probably have to have the main/master in a central
| location though. (I think Fly.io are doing something a
| little like this with Postgres)
| atlgator wrote:
| There may be fundamentals at play related to their
| product offering/pricing too but pure growth companies
| with negative earnings have been getting routed with the
| announcement of Fed tapering and interest rate increases.
| That's the main thing driving Cloudflare's price down.
| schleck8 wrote:
| > CloudFlair
|
| I wanted to look up whether cloudflair.com was already
| registered, and apparently the registrar is cloudflare
| themselves. That's funny
| samwillis wrote:
| Ha! Oops, shouldn't try to do the kids bedtime and write a
| reply at the same time...
| juliosueiras wrote:
| funny enough, B2+Cloudflare will still be cheaper, since
| Cloudflare R2 from the blog you linked is at the cost of
| $0.015/GB/month for stored storage, where as B2 is
| $0.005/GB/month for stored storage, and there is no download
| cost when using the B2+Cloudflare combo
| ggm wrote:
| "if everyone does it, they won't let you do it" -the infinite
| cache life on CF only works because 99.99% of use of the cache
| stays in rational cache lifetimes. If everything had long
| persist, CF would begin cache ejection, and your cost goes up, or
| they change the ToS to forbid it.
|
| LRU exists for a reason.
| DoctorOW wrote:
| This seems more or less like the B2 help document on the matter:
| https://help.backblaze.com/hc/en-us/articles/217666928-Using...
| anduru_h wrote:
| I've been using BunnyCDN - it's been pretty cheap (<$5 month for
| 250GBs of data transfer).
| mike503 wrote:
| +1 for Bunny, if you cannot take advantage of a free tier
| somewhere and have to pay. Setup a client of mine on it for
| video and images, real-time image modifying/branding, smooth
| video playback (without using a special product) all using the
| equivalent of presigned URLs - for a surprisingly low amount of
| money each month. Lots of good stuff going on there.
|
| They also have their own storage options that are tightly
| integrated to their CDN, but basic FTP and simple API for it
| still. No S3 or bulk options there yet. Promised S3 API support
| "soon" though
| destitude wrote:
| Haven't a lot of people on HN warned against using Backblaze for
| backup because of issues restoring from backup? I recall reading
| someone having to pay to have a physical drive sent to them to
| restore a backup because of speed/cost issues trying to restore
| over network.
| hinkley wrote:
| Never underestimate the bandwidth of a station wagon full of
| data tapes hurtling down the highway.
|
| Randall Munroe estimates it'll be 2040 before the Internet has
| more bandwidth than FedEx: https://what-if.xkcd.com/31/
| wrycoder wrote:
| They will refund the drive cost when you mail it back.
| celsoazevedo wrote:
| You're probably thinking about their computer backup service.
| This post talks about B2, an object storage service like S3.
|
| While B2 isn't as fast as S3 and some tasks are way slower (eg:
| deleting files), I'm able to max out my server's 1Gbps
| connection when sending or downloading files from their
| European DC (using rclone and a server also in Europe).
| Rastonbury wrote:
| I think it's their consumer facing Dropbox like desktop tool
| not B2
| magicjosh wrote:
| Is anyone building CDN service on top of IPFS? IPFS is certainly
| too slow for many applications but perhaps there are tiny niches
| where it makes sense today? More like tape drive storage?
| yawnxyz wrote:
| Cloudflare has a CDN gateway on top, but they don't do pinning
| (just kidding, you can add a site to IPFS through CF now --
| edit2: I still don't think they do pinning... really can't
| tell): https://developers.cloudflare.com/distributed-web/ipfs-
| gatew...
| testesttest wrote:
| In my experience their gateway is not great and serves
| captchas occasionally if you're requesting rare content they
| don't have cached. I ended up just running my own node at a
| provider supporting bandwidth alliance and putting cloud
| flare in front of that.
| marcodiego wrote:
| We have a free CDN. It is called bittorrent.
| Dylan16807 wrote:
| Ah yes, 'CDN at home'.
|
| Sadly people's connections are so asymmetrical they're likely
| to have problems or get upset from that kind of thing.
| binarymax wrote:
| I'm looking at wasabi to host some large (500-1000M)
| distributable binary files. Anyone have experience with them?
| breakingcups wrote:
| Quite some caveats. Minimum storage used per month is 1TB. That
| means, if you store 250MB of files on there you will still be
| charged for the whole TB. Also, you will be charged for storing
| those files for at least 90 days even if you only keep them for
| 1 day. Lastly, and this is a very important one, you only have
| free egress equaling the total amount of data you have stored.
| So, say you have 2 TB stored, you can only egress 2 TB that
| month.
|
| If you start to exceed that limit they are free to suspend your
| account.
|
| So it depends very much on your use case, but there were too
| many risks for my use cases.
| cj wrote:
| I was expecting to read about how to set up the infrastructure to
| run my own CDN infrastructure for free (hard to believe is
| possible, so I clicked).
|
| Instead, it's an article about how to use Cloudflare to deliver
| static assets for free, on Cloudflare's free tier.
|
| Edit: I would love to read more about how to set up a cheap self-
| managed CDN, perhaps using cheap VPS servers spread out in
| different regions.
| [deleted]
| lewisl9029 wrote:
| Indeed, I was expecting something like this:
| https://fly.io/blog/the-5-hour-content-delivery-network/
|
| Good thing it already exists!
| vbernat wrote:
| This link may be of interest:
| https://www.linkedin.com/pulse/build-your-own-anycast-networ...
| cmroanirgo wrote:
| I've been wanting the browsers to support SRV dns records for
| quite some time now. Unfortunately, both firefox and chrome
| have bugs in the WONTFIX status. The reasons for these
| statuses given are poor imo, but the implementation could add
| an extremely interesting dynamic to the internet. It would be
| easy to set up vps around the world and let SRV records set
| up in a round robin manner control it. Will it be as good as
| a bgp controlled one? Unlikely, but it would be 100% in your
| control and easy to set up, unlike going cloudflare.
|
| https://bugzilla.mozilla.org/show_bug.cgi?id=14328
|
| https://bugs.chromium.org/p/chromium/issues/detail?id=22423
| teddyh wrote:
| SRV support seems to have been (unfairly, I think) rejected
| by HTTP vendors and standard bodies. There is a new
| standard, however, in the works:
|
| https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-
| svcb-...
|
| https://github.com/MikeBishop/dns-alt-svc
|
| This one might have a larger chance of being accepted,
| since it provides some things which HTTP standard bodies
| and HTTP client vendors want. It doesn't (at this stage,
| anyway) provide for the load-balancing "weight" field from
| the SRV record, but it does support MX-style priority
| numbers, and also port numbers. Very interesting, to say
| the least.
| [deleted]
| ignoramous wrote:
| > _I would love to read more about how to set up a cheap self-
| managed CDN._
|
| See also: https://blog.apnic.net/2021/04/07/building-an-open-
| source-an...
| outloudvi wrote:
| B2-Cloudflare was a great companion when Cloudflare's Bandwidth
| Alliance was a thing and outgress from B2 to Cloudflare didn't
| cost anything, which means that outgress was completely free
| since Cloudflare don't charge on outgress.
| ricardobeat wrote:
| Why "was"? Seems like everything you mentioned is still true.
| It's the whole point of the post.
| ksec wrote:
| This reads to me as B2 / Cloudflare are no longer in Bandwidth
| Alliance?
| vlovich123 wrote:
| Still seems to be listed here:
| https://www.cloudflare.com/bandwidth-alliance/
| timtom39 wrote:
| Bandwidth Alliance isn't a thing anymore? My deployment is
| still serving multiple TB a day free. Do you have a link?
| jcubic wrote:
| You can get literally Free CDN using GitHub repo:
|
| https://gist.github.com/jcubic/a8b8c979d200ffde13cc08505f7a6...
| kondbg wrote:
| Using Cloudflare to proxy B2 content seems like it directly
| violates Cloudflare's ToS.
|
| https://www.cloudflare.com/terms/
|
| > 2.8 Limitation on Serving Non-HTML Content
|
| > The Services are offered primarily as a platform to cache and
| serve web pages and websites. Unless explicitly included as part
| of a Paid Service purchased by you, you agree to use the Services
| solely for the purpose of (i) serving web pages as viewed through
| a web browser or other functionally equivalent applications,
| including rendering Hypertext Markup Language (HTML) or other
| functional equivalents, and (ii) serving web APIs subject to the
| restrictions set forth in this Section 2.8. Use of the Services
| for serving video or a disproportionate percentage of pictures,
| audio files, or other non-HTML content is prohibited, unless
| purchased separately as part of a Paid Service or expressly
| allowed under our Supplemental Terms for a specific Service. If
| we determine you have breached this Section 2.8, we may
| immediately suspend or restrict your use of the Services, or
| limit End User access to certain of your resources through the
| Services.
|
| If this was truly acceptable and not in some grey area, why
| doesn't Backblaze simply route all downloads through Cloudflare
| by default, rather than having each individual customer go
| through the hassle of setting this up?
| cmeacham98 wrote:
| If you only use this for "standard" CDN assets (like pictures
| that are part of your website styling rather than as an image
| host) and you also host your website on Cloudflare, I think it
| should be ok.
| searchableguy wrote:
| Many of the files will violate cloudflare terms of service. You
| cannot put video files behind normal cloudflare cdn as an
| example.
|
| This is a risky setup.
| fivre wrote:
| This part of the Cloudflare ToS only applies to cached content.
| They're fine with simply proxying to your video files.
| magila wrote:
| That is not true. Cloudflare's ToS prohibits serving video
| files via their CDN regardless of whether they're cached or
| not.
| colejohnson66 wrote:
| Why?
| magila wrote:
| Video streaming consumes dramatically more bandwidth than
| the HTML/CSS/JS/JPEG/etc files which Cloudflare's CDN is
| intended to serve. Bandwidth is cheap, especially for
| Cloudflare, but it's not free. Cloudflare's base pricing,
| and especially their allowance for free accounts, does
| not account for such large bandwidth consumption.
|
| Cloudflare does have a product specifically designed to
| stream video. It is priced accordingly.
| sibit wrote:
| If I had to guess I'd say it's because it completes with
| their video streaming service[0].
|
| > Cloudflare Stream is the most affordable and easy-to-
| use streaming platform.
|
| [0] https://www.cloudflare.com/products/cloudflare-
| stream/
| sascha_sl wrote:
| FWIW a partner of mine has been using this setup less for the
| CDN aspect and more for the free B2 egress, and a TB or two of
| large encrypted files (rclone crypt remote) have not resulted
| in any complaints.
| aantix wrote:
| Interesting setup.
|
| And with Rails, it appears that there's a third party
| ActiveStorage adapter for BackBlaze.
|
| https://github.com/jeygeethan/activestorage-backblaze
|
| Direct uploading from client :
| https://github.com/jeygeethan/activestorage-backblaze-javasc...
| dinkleberg wrote:
| I know every use case is different, so this may not matter for
| most. But in my experience, the performance of B2 was quite poor
| in comparison to other S3 compatible options. It seemed like the
| crux of the issue was slowness in their API. Generating pre-
| signed URLs for file upload or download was quite a bit slower,
| resulting in a poor user experience in my application.
|
| If you're just using it to upload public static content for your
| site, this shouldn't be an issue. But if you're dealing with
| private content it probably isn't the optimal solution.
| dotancohen wrote:
| I would love to hear more. I had a gig where we considered S3
| and B2 for serving very private files (financial records). With
| B2 I could not find a good way to ensure that, assuming the URL
| is known, a file could only be accessed by an authorized party.
| poxrud wrote:
| You can set the bucket as allPrivate, then use the API to
| generate a presigned url for the file that you need.
| dotancohen wrote:
| That URL is susceptible to being intercepted in transit,
| though, such as in an email. And validating the request via
| a redirect on my own server invalidates 2/3 of the reasons
| for using an external file host.
| frogger8 wrote:
| I am in the same boat currently in my research. In fact I
| posted this gist link we are discussing. My use case is
| confidential data and this solution won't work if I can't
| guarantee access is limited to specific users around the
| country.
|
| Is free, secure, and fast to much to ask? Joking of course.
___________________________________________________________________
(page generated 2022-01-15 23:00 UTC)