[HN Gopher] Standing on our own two feet
___________________________________________________________________
Standing on our own two feet
Author : gpff
Score : 242 points
Date : 2020-11-06 16:21 UTC (6 hours ago)
(HTM) web link (letsencrypt.org)
(TXT) w3m dump (letsencrypt.org)
| colinclerk wrote:
| Does anyone have experiences with ZeroSSL? Caddy has been
| building in support so I think it could be a drop-in replacement
| for Caddy/CertMagic/ACMEx users.
| regecks wrote:
| ZeroSSL is operated by Sectigo (Comodo). It works fine with
| Certbot and should work fine with any ACME client.
|
| I did run into some random 503s occasionally, but that was
| pretty soon after it was launched. Maybe just a few teething
| issues.
| mholt wrote:
| ACMEz* ;)
|
| Seconding regecks' comment.
|
| We're gradually making ZeroSSL a default CA for Caddy.
|
| (I am currently implementing multi-CA support into Caddy and
| CertMagic, so that Caddy will be able to use both Let's Encrypt
| and ZeroSSL for redundancy. It's the first server to support
| this!)
|
| This is a good thing for the ecosystem.
| yjftsjthsd-h wrote:
| > We're gradually making ZeroSSL a default CA for Caddy.
|
| As in, replacing LE as the default, or supplementing it? (And
| if the former, why?)
| Havoc wrote:
| That seems reasonable. LE is doing good work & hardly their fault
| Android is a fragmented mess.
| lixtra wrote:
| They propose to install Firefox to work around the root
| certificate problem on old android devices. But can't you just
| manually install their root certificate on most phones?
| lights0123 wrote:
| It probably looks a lot less sketchy to your users to tell them
| to install a browser they've probably heard of and may have
| used in the past than install a root certificate. Plus, that
| doesn't fix the problem of other certificates expiring, only
| extends it.
| regecks wrote:
| >Plus, that doesn't fix the problem of other certificates
| expiring, only extends it.
|
| Although I agree fully on the sketchiness part, installing
| the root is a fix.
|
| ISRG Root X1 expires in 2035. Not one of these problematic
| Android devices will be online anymore.
| iudqnolq wrote:
| But to have the whole modern web work you'll need to
| install more than one root.
| fuzxi wrote:
| I'm not sure if it's the same on older versions, but on recent
| Android versions, that requires a rooted device.
| Aissen wrote:
| This might also concern you if your app is running on older
| android devices, and you have a backend that uses letsencrypt
| certificates, while the app is relying on the system root CAs.
|
| A fix for this could be to add Let's Encrypt's root CA to your
| app.
| nevi-me wrote:
| Could Google possibly be able (before were discuss willingness)
| to push an update to root certificate via Play Services?
|
| I'd like to think that anyone not using Play Services (i.e.
| Android with no Play) is likely using a custom browser, and would
| heed a call to switch to Firefox.
|
| The problem with some devices in Africa would be that many people
| will using older phone often don't have enough data for the big
| Play updates to succeed.
|
| Teens often buy 10-100MB of data so they can use WhatsApp. (If
| you're from Southern Africa, and disagree with this, hit me up,
| you probably need to spend some time in a village ;) )
| floatboth wrote:
| Starting with Android 10, Google can push updates to lots of
| system components (media codecs, android frameworks, tzdata,
| ...) https://android-developers.googleblog.com/2019/05/fresher-
| os...
|
| But those on older versions are screwed. It's really a major
| fuck-up that Google didn't do this for root certs since the
| beginning of Android.
| trillic wrote:
| How much does that much data typically cost??
| toast0 wrote:
| No play services on an Android phone in the US probably implies
| willingness to tinker. No play services on an Android phone in
| China only implies it's an Android phone. In the developing
| world, it most likely implies a very low cost Android phone of
| Chinese origin.
|
| Bundling things that need timely updates with the OS with no
| mechanism to update them individually is a design error. Things
| like root certificates, time zone databases, leap second
| information, and even TLS libraries need to be updated on a
| regular basis. These items should be distributed outside of the
| general upgrade process, even if the general upgrade process
| worked (which is clearly not the case). Alternatively, root
| certs and TLS libraries could be bundled with applications as
| needed. You could probably have a stable core x.509 library and
| cipher algorithms bundled with the OS, so that the application
| level TLS library can be kept small. You still need to get tzdb
| updates out though.
|
| In an ideal world, large OS vendors could work with carriers to
| get this small set of updates zero-rated in exchange for making
| sure they are very small and background downloaded only at
| times of low network congestion.
| sandworm101 wrote:
| In an ideal world carriers wouldn't have a say in what
| software updates were installed on my phone. Comcast doesn't
| control the software on the computers it services. Why should
| Telus control what updates are made available for my phone?
| ocdtrekkie wrote:
| Because security and privacy is the compromise Google made
| for dominance: Letting OEMs and carriers do what they want
| is what sold them on Android.
|
| I'd absolutely agree that this is a design error though:
| We'll be better off when Android is dead and gone as a
| platform.
| young_unixer wrote:
| Can't you buy hardware directly and then just put the
| carrier's SIM card?
| freeone3000 wrote:
| Because they're the ones who push updates over the cell
| network. Comcast absolutely controls what software you run
| on your modem. You can update "out of band" manually, at
| least on recent Android Pixel phones. Any other
| manufacturer could also make their updates public, but
| since installing the one not for your carrier band makes
| the phone unusable as a phone, it's not likely to be
| common.
| cassepipe wrote:
| Actually I was surprised that there would be such an easy fix :
| Switching to Firefox. Which is apparently around 70MB,
| apparently affordable from what you wrote and definitely worth
| it if it allows you to unlock a chunk of the internet. So no
| need for an improbable and costly Play update.
| qqii wrote:
| That won't fix any other apps though will it? Anything that
| uses chrome webview for example.
| legulere wrote:
| Now that they're standing on their own feet can they also offer
| S/MIME certificates for email encryption?
| renewiltord wrote:
| > _The remaining 33.8% of Android devices will eventually start
| getting certificate errors when users visit sites that have a
| Let's Encrypt certificate. In our communications with large
| integrators, we have found that this represents around 1-5% of
| traffic to their sites._
|
| This one-third of Android devices only yields 5% of traffic?
| Interesting.
| Havoc wrote:
| Crappy expensive internet means it's not instinct to squeeze a
| quick browsing session into every free minute
| toast0 wrote:
| More interesting that % of traffic in this case would be unique
| users. But it's not surprising to me that people using Androids
| on older devices are using them to access the internet less;
| they're generally not as nice to use, but also if they were
| heavily used, they may have worn out by now.
| degenerate wrote:
| I have a 2010 Android smartphone and it's _painfully slow_ to
| navigate the modern web, almost unbearable. Browsing news
| websites is simply not worth my time of waiting for the phone
| to download and process 22MB of JS, CSS, and graphics. Being on
| wifi makes no difference; it 's the CPU choking to render all
| that cruft.
|
| So yeah, 5% of traffic makes sense.
| GoblinSlayer wrote:
| In firefox (at least in desktop version) you can disable
| javascript and css. Not sure if they are still downloaded.
| notatoad wrote:
| there's a lot of old android phones out there that don't get
| used for anything other than making phone calls or sending
| texts.
| zero37z wrote:
| now the corporates got a valid point, why you dont want to use
| lets encrypt?
|
| still the 33% of the devices is a quite a large number to
| consider.
| juliend2 wrote:
| But like they said in the article, those 33% of Android phones
| represent "1-5% of the traffic" of the "large integrators"
| websites that LE communicated with.
| stefan_ wrote:
| Well that's an easy choice, lose 1-5% of traffic or pay $100
| for a certificate from a vendor whose root doesn't expire
| next year?
| fuzxi wrote:
| 1-5% of traffic that comes from people using devices that
| are at least 4 years old. Someone who can't or won't
| upgrade from a phone that still uses Android Marshmallow is
| probably not bringing in much revenue.
| josephg wrote:
| Exactly. The "easy choice" is to drop support for those
| users. I suspect most web developers won't even notice.
|
| It's a pity - there's no essential reason why old phones
| with new batteries should get worse over time. But
| software kills them in so many ways.
| DanielDent wrote:
| I think a fairly manageable path forward is underway which makes
| this a smaller issue than it first seems:
|
| (1) Firefox already uses its own root store
|
| (2) App developers can include additional roots in addition to
| the system root store:
| https://developer.android.com/training/articles/security-con...
|
| (3) Chrome is migrating to using it's own store: "Historically,
| Chrome has integrated with the Root Store provided by the
| platform on which it is running. Chrome is in the process of
| transitioning certificate verification to use a common
| implementation on all platforms where it's under application
| control, namely Android, Chrome OS, Linux, Windows, and macOS.
| Apple policies prevent the Chrome Root Store and verifier from
| being used on Chrome for iOS."
|
| https://www.chromium.org/Home/chromium-security/root-ca-poli...
| paulpauper wrote:
| i hate how google puts warnings on non-ssl sites. why doe a
| static page that has no forms need ssl? non-ssl worked fine for
| 20 years for webpages and google comes along and says noooo not
| good enough.
| notriddle wrote:
| A MITM can add a "Donate" button to news.ycombinator.com. The
| money doesn't actually go to Hacker News, though. It goes to
| the MITM.
| superdisk wrote:
| The NSA can see what pages you read, and men in the middle can
| modify the page to insert malicious JS or ads or whatever
| without HTTPS.
| gsich wrote:
| As a site owner that is somehow not my problem. It's not my
| fault that you have a shitty ISP. I can understand and
| support that argument.
|
| I do TLS only for my stuff however.
| inquirerofsorts wrote:
| > It's not my fault that you have a shitty ISP.
|
| Really don't think you're grasping this and lack an
| understanding of packet routing.
|
| Please type: traceroute apache.org
|
| It's not just your ISP that's potentially shitty, it's
| every single hop on that list. Even worse if the person is
| using an open wifi connection.
|
| Apache will happily serve you an insecure connection, they
| are the 80th largest site on the internet. This type of
| stuff should be called out by technologists and it's
| incredibly disappointing when its handwaved away.
| kickscondor wrote:
| Ok - next question then: why do browsers block self-signed
| certs? If Lets Encrypt now allows any domain to get a cert,
| what's the harm in a self-signed cert? Seems like a step up
| from plain HTTP.
| ocdtrekkie wrote:
| The theory here is a self-signed cert could be from anyone
| (including the NSA) and you wouldn't know. Unless you
| explicitly trusted the certificate you were using, like
| enterprises do.
| closeparen wrote:
| What you are trusting, when you trust a CA, is that it will
| only issue a certificate for a domain to someone it has
| verified as the owner. A self-signed cert could be from a
| man in the middle attacker.
|
| An active MITM is a relatively more exotic threat than a
| passive observer. In the days when certificates cost money,
| there was a legitimate argument that you shouldn't need the
| whole elaborate active-MITM defense just to get protection
| against passive snooping. But now that you can get both for
| free... just use Let's Encrypt.
| DivineEntropy wrote:
| Small nit, verification is only meant to prove control
| over the domain, not ownership.
| [deleted]
| laughinghan wrote:
| The man-in-the-middle can self-sign their own certs and
| present it to you as the site's own self-signed cert.
| Unless you have some way to verify self-signed certs out-
| of-band, they're useless.
| josephg wrote:
| They're not useless. They stop passive adversaries, like
| the Australian government. They just don't stop active
| adversaries.
|
| Browser security warnings imply https > http > self
| signed https. The correct order of should be https > self
| signed https > http.
| Lammy wrote:
| The NSA can still see that just via the metadata of me making
| the connections necessary to load the encrypted page
| contents.
| traceddd wrote:
| To be fair, they probably can anyways, since the site
| willingly handed over their keys to Cloudflare, or rely on
| some Google CDN, host on EC2, etc.
|
| It's the second or third tier nation state actors that you
| can really make a difference with.
| kibwen wrote:
| Recently there was a browser zero-day observed in the wild that
| operated by MITM'ing HTTP connections and injecting the payload
| into the response. You're thinking of HTTPS as protecting what
| information you _send_ , but it also protects what you receive;
| with an HTTP connection, anyone in the middle can make your
| browser receive anything they want.
| onlinejk wrote:
| Nice TL;DR, thx
| gioele wrote:
| > Without IdenTrust, Let's Encrypt may have never happened and we
| are grateful to them for their partnership.
|
| What I have never understood is why IdenTrust accepted to cross-
| sign Let's Encrypt's root certificate.
|
| With that move, IdenTrust basically broke the CA cartel and
| helped driving the price of basic certificates to zero. How did
| they, as a for-profit organization, justify "doing the right
| thing" when that meant disrupting their own market?
|
| Anyway, kudos to IdenTrust.
| dubcanada wrote:
| IdenTrust doesn't care about random websites, they care about
| HIPPA, enterprise, government, securing documents and emails,
| etc.
| juliend2 wrote:
| Exactly, but the funny thing is that Chrome and Firefox
| (Desktop at least) are no longer showing the differentiating
| green lock for those high-end (EV & OV) certs, but the same
| neutral-looking lock as those Domain-Validated (DV) certs
| that Let's Encrypt is issuing.
|
| I'm very grateful for IdenTrust for having made that move. I
| just hope it won't hurt their business too much because of
| that.
| closeparen wrote:
| Does anyone in enterprise actually need publicly trusted
| certificates for documents and email? Seems like it's an
| inside-the-firewall Exchange server for internal traffic, and
| a white-label "secure messaging center" portal for external
| traffic.
| zinekeller wrote:
| IdenTrust's buisness also spans to managing private CAs for
| companies, which includes managing the HSM and private
| keys. Also, the companies who hire IdenTrust and similar
| companies are not that involved in technology. Also,
| security experts who can manage this safely is a tad harder
| to find and requests higher wages than your standard IT
| staff.
|
| TLDR: yes, but some companies wants another company to
| manage their certs.
| alanfranz wrote:
| Maybe IdenTrust will now offer an ACME compatible endpoint and
| offer signed, paid certs with their CA. Or another CA will.
|
| I wonder whether IdenTrust imagined that a five year cross
| signed root ca would be too little a timespan to get wide
| adoption.
|
| Btw... Wouldn't it be possible to just add a new root ca to
| android? Maybe an app could simplify delivery?
| andymatuschak wrote:
| It seems like a case of "commoditize your complement" to me.
| https://www.gwern.net/Complement
|
| They offer a variety of enterprises services and can now
| capture more of the marginal consumer value relative to
| competitors still fighting for the cert issuing market.
| admax88q wrote:
| Always better to be doing the disruption rather be the one
| being disrupted.
|
| Perhaps they saw the writing on the walls and wanted to boost
| their standing in the post LetsEncrypt world.
| est31 wrote:
| Yeah they likely got a stash of money for it while the other
| CAs got nothing. First mover advantage I guess. It's not that
| they'd have been able to prevent it anyways, as the root
| stores are not under their control, especially when Let's
| encrypt is being supported, even started, by the entities
| which run the root stores (Mozilla, through Firefox, and
| Google, through Android). Ultimately it's the entities which
| run and deploy the root stores who have ultimate say on which
| CA gets to issue certs and which doesn't.
| tothrowaway wrote:
| > As of January 11, 2021, we're planning to make a change to our
| API so that ACME clients will, by default, serve a certificate
| chain that leads to ISRG Root X1.
|
| Ouch. So anyone who wants to support older devices 3 months from
| now needs to add `--preferred-chain "DST Root CA X3"` to their
| certbot command. When that chain is completely retired in
| September next year, certbot appears to fallback on the default
| (even with that argument present).
| miohtama wrote:
| Somewhat related, but in other thread two weeks ago people
| complain Google has too much power over Android ecosystem:
| https://news.ycombinator.com/item?id=24917918
|
| Now, here people are suggesting Google should somehow update the
| old Androids.
|
| Be damned one way or the other.
| gpff wrote:
| Let's Encrypt cross-signature with IdenTrust "DST Root X3" is
| ending on September 1, 2021 but 33.8% of Android devices are
| running versions under 7.1 which don't trust Let's Encrypt new
| root certificate "ISRG Root X1"
| tyingq wrote:
| "https almost everywhere".
|
| Thanks G!
| brendanclune wrote:
| Workaround is Firefox Mobile (because it ships with its own
| root certs), but that's a significant burden to place on the
| user.
| est31 wrote:
| Also the post says that Firefox doesn't work on Androids
| older than 5.0 which according to the dashboard are still
| 5.9% of devices.
|
| For those older devices, the only option is to install the
| new root certificate.
|
| Anyways, there are billions of Android devices out there. 33%
| of those is a large number. You can't just tell all of them
| that they are wrong.
|
| If this happens, people will move away from Let's encrypt in
| masses. They don't realize yet how self-harming this really
| is.
| baggy_trough wrote:
| Not sure about that. Move away from Let's Encrypt to what?
| More likely, most smaller to medium sized sites will say
| forget those old Android guys.
| est31 wrote:
| I found at least Buypass offering a gratis ACME product
| "Buypass Go SSL". They have roots which are deployed at
| least since Android 4.1, which covers way more Android
| devices (according to the Android Studio statistics,
| >99%):
|
| https://android.googlesource.com/platform/libcore/+/andro
| id-...
|
| I'm not sure whether that particular root is being used
| for their Go SSL product. If so, Buypass might be a good
| alternative to migrate to.
|
| From what I read though, they _do_ require an E-Mail
| address, so you 've got to keep that in mind.
| sroussey wrote:
| Too cheap to update can be seen as too cheap to buy your
| product or service, so I can see this happening.
| WorldMaker wrote:
| Root certificate updates are a massive security issue.
| Blaming Let's Encrypt is blaming one of the canaries for
| the coal mine disaster. 33% of Android devices don't and
| can't get up to date root certificates is an impressive
| security crisis that grows worse by the year (look at the
| other root expirations and the crazy workarounds that for
| instance Netflix has been doing to still work on older
| Android devices). Shouldn't the blame squarely be on
| Google, the Android OEMs, and the phone carriers for
| allowing this disaster to happen in the first place?
|
| I realize that is a tough message to get out to users and
| site owners are going to be in the cross-fire, but it seems
| better to try to work for solidarity in pointing fingers at
| the right direction and the right direction certainly isn't
| Let's Encrypt.
| est31 wrote:
| Yes the issue is really severe of most deployed Android
| devices not getting security updates, either at all, or
| the devices are used well beyond the update period.
|
| But this is not up to Let's Encrypt to solve. They market
| themselves to build products for the mass market instead
| of small niches of the market, say, everyone who buys a
| new phone every year. But then they also have to treat
| their product like a mass market product, and if Android
| users still use older versions of the OS, then Let's
| Encrypt should adopt for that.
| jacquesm wrote:
| They will be happy to accept that blame, as long as you
| fork over your $ for a new device. Planned obsolescence
| can have many components, this is just one of them.
| cpach wrote:
| That makes me curious, what workarounds did Netflix
| employ?
| Aissen wrote:
| When you control the client, it's simple, you can do
| pretty much anything: embed your own HTTP stack, TLS
| stack, your QUIC stack, or simply your PKI, or subset of
| the webPKI.
| WorldMaker wrote:
| It was the BBC I was actually thinking about, and it's a
| part of this article (which also mentions this Let's
| Encrypt root change):
|
| https://scotthelme.co.uk/impending-doom-root-ca-expiring-
| leg...
|
| Previous HN discussion on that article:
| https://news.ycombinator.com/item?id=23455463
| fuzxi wrote:
| 33% of devices, but per the article, only 1-5% of traffic
| on sites using LetsEncrypt. I don't see _any_ site owners
| moving to a different CA when this affects less than 5% of
| their visitors, who are likely the poorest fraction of
| their userbase , i.e. probably not many paying users.
| veeti wrote:
| You can still install the last version of Firefox to
| support Android 4.x.
| maxerickson wrote:
| In the post, they advertise that they are going to continue
| to offer a service that provides certificates signed with
| the old one.
| capableweb wrote:
| That's not gonna help as the root will expire, so the
| devices stuck with the older root will still shows errors
| when trying to use them.
| est31 wrote:
| ... which will work until September 2021.
| merlyn wrote:
| How many other root certificates are going to be expiring
| in the next 3 years that older Android won't have updates
| for as well? Probably more than 1.
| TheKIngofBelAir wrote:
| > Also the post says that Firefox doesn't work on Androids
| older than 5.0 which according to the dashboard are still
| 5.9% of devices. For those older devices, the only option
| is to install the new root certificate.
|
| Microsoft Edge still gets updates on Android 4.4 KitKat
| w3ll_w3ll_w3ll wrote:
| I don't think Microsft Edge embeds its own root store on
| Android.
| tacon wrote:
| Is there enough incentive for Microsoft to add a root
| store to Edge by next September? How hard is it to make
| that addition?
| notatoad wrote:
| on androids older than 5, the browser is the old android
| browser instead of chrome. how many sites out there still
| test compatability with that?
|
| even without having to click through security warnings, the
| web is horribly broken on old android devices. the overlap
| of sites using letsencrypt and sites that care about people
| using android <5 has got to be vanishingly small. this
| isn't going to cause a move away from letsencrypt.
| stefan_ wrote:
| A Workaround is not something that _the user_ does.
| rsweeney21 wrote:
| I don't understand the motivation for making this change now. Why
| not keep the universally accepted root certificate as the default
| chain? Why does Let's Encrypt need to switch to their own root
| certificate now, and cause thousands of websites to break on
| older devices?
|
| I don't even control the certificate provisioning process. We use
| Heroku and Webflow. This is frustrating.
| tingletech wrote:
| "the DST Root X3 root certificate that we relied on to get us
| off the ground is going to expire - on September 1, 2021."
| toast0 wrote:
| Their cross cert expires in 10 months. Switching in January
| gives most people time to notice the problem while there's an
| easy temporary fix (switch to the soon to be expiring cross
| cert while you evaluate the root compatability of commercial
| CAs across the devices you support).
|
| I imagine the cross cert cost a bunch of money, and they may
| not have the money to do that again.
| laughinghan wrote:
| Did you miss the part where the "universally accepted" root
| certificate is going to be universally rejected in 10 months?
___________________________________________________________________
(page generated 2020-11-06 23:00 UTC)