[HN Gopher] NIST is announcing that SHA-1 should be phased out b...
       ___________________________________________________________________
        
       NIST is announcing that SHA-1 should be phased out by Dec. 31, 2030
        
       Author : say_it_as_it_is
       Score  : 288 points
       Date   : 2022-12-15 15:49 UTC (7 hours ago)
        
 (HTM) web link (www.nist.gov)
 (TXT) w3m dump (www.nist.gov)
        
       | px43 wrote:
       | Good job SHA-1, you had a good run and didn't get too broken
       | before retirement, which is quite an accomplishment for anyone
       | these days.
       | 
       | Does anyone in the field know if there's a SHA-4 competition
       | around the corner, or is SHA-3 good enough? It would be
       | interesting if one of the SHA-4 requirements was Zero Knowledge
       | Proof friendlyness. MiMC and Poseidon are two of the big
       | contenders in that area, and it would be really great to see a
       | full NIST quality onslaught against them.
        
         | iwillbenice wrote:
        
         | pclmulqdq wrote:
         | The NIST likes having two standardized "good" hash functions. I
         | doubt we will have a SHA-4 until SHA-2 starts getting close to
         | a break. The SHA-3 competition started once SHA-1 started
         | showing some possible weaknesses. Also, if SHA-2 ends up
         | getting retired, Blake3 will likely become SHA-4: Blake2 lost
         | to Keccak because Blake2 was too similar to SHA-2.
        
           | twiss wrote:
           | NIST seems perfectly comfortable with having only one
           | cryptographic primitive for a given category as long as
           | there's high confidence in it. The reason we have two hash
           | algorithms is that back when the SHA-3 competition was
           | created, there was some uncertainty about the long-term
           | security of SHA-2. That uncertainty has since subsided, I
           | would say. But if SHA-2 does end up being considered
           | insecure, as long as there's no reason to suspect that the
           | same will happen to SHA-3, there's no reason to create a
           | SHA-4 yet.
        
         | miga wrote:
         | There is Post-Quantum Cryptography competition, with three
         | e-signature algorithms selected for standardization:
         | https://csrc.nist.gov/projects/post-quantum-cryptography
         | 
         | Given that NIST itself warns PQC algorithm may be unsafe after
         | 2035, this should be considered SHA-4.
        
           | bawolff wrote:
           | Sha-2 is already quantum safe.
           | 
           | Sha is not a digital signature algorithm. That is a different
           | type of crypto primitive.
        
           | adastra22 wrote:
           | Hash functions aren't significantly impacted by quantum
           | computers. You may need to use a longer construction (eg.
           | SHA512 instead of SHA256), but that's it.
        
             | bawolff wrote:
             | Im not a cryptographer, im kind of curious - is it possible
             | to combine a birthday attack with grover's algorithm to
             | attack sha256 in 2^64 time?
        
               | adastra22 wrote:
               | You would get cube root speedup instead of sqrt for a
               | collision (birthday attack), or sqrt instead of brute
               | force for a preimage. So SHA256 is secure from preimage
               | attacks even with a quantum computer, and gives 2^80
               | protection against collisions. SHA-2/384 would be
               | sufficient for 128-bit security against collisions.
        
         | twiss wrote:
         | Not only is SHA-3 good enough, for most applications SHA-2 is
         | still good enough as well. So personally I would be surprised
         | to see a new competition for a general-purpose hashing
         | algorithm anytime soon.
        
           | SilasX wrote:
           | Isn't the entire Bitcoin mining incentive structure
           | equivalent to a huge bug bounty on SHA-2 (256, specifically)?
           | 
           | Edit: Never mind, grand-uncle comment beat me to it:
           | 
           | https://news.ycombinator.com/item?id=34003485
        
           | SAI_Peregrinus wrote:
           | Agreed. Blake3 is super promising as cryptographic hash
           | function family due to its software performance (not sure if
           | anyone has tried making hardware designs for it), but SHA2
           | hardware acceleration is extremely common and makes it good
           | enough. And while SHA3 software performance is poor it's
           | quite good in hardware, so as more chips accelerate it it'll
           | become generally viable. So while Blake3 looks nice, there's
           | not a whole ton of need for it right now, and I doubt it'll
           | become SHA4 ever.
        
             | aliqot wrote:
             | Sha3 is performant, but I'll always give it the stink eye
             | because of NIST selecting the winner, then modifying their
             | solution before standardization without sufficient
             | explanation. Read the room, this is cryptography; one does
             | not simply add mystery padding and rush off to the printing
             | press.
             | 
             | https://www.schneier.com/blog/archives/2013/10/will_keccak_
             | s...
             | 
             | https://en.wikipedia.org/wiki/SHA-3#Padding
        
               | Perseids wrote:
               | It is fair to criticize NIST for enabling rumors about
               | them weakening SHA3, but these are _rumors only_ ,
               | nothing more. Please, everyone, stop spreading them. SHA3
               | is the same Keccak that has gone through extensive
               | scrutiny during the SHA3 competition, and, as far as
               | anyone can tell, it is rock solid. Don't trust NIST,
               | don't trust me, trust the _designers of Keccak_ , who
               | tell you that NIST did not break it:
               | 
               | > NIST's current proposal for SHA-3, namely the one
               | presented by John Kelsey at CHES 2013 in August, is a
               | subset of the Keccak family. More concretely, one can
               | generate the test vectors for that proposal using the
               | Keccak reference code (version 3.0 and later, January
               | 2011). This alone shows that the proposal cannot contain
               | internal changes to the algorithm.
               | 
               | https://keccak.team/2013/yes_this_is_keccak.html
        
               | dchest wrote:
               | All good changes, giving us SHAKE functions and the rest
               | of useful variants based on domain separation in padding.
        
               | WhitneyLand wrote:
               | How much did your opinion change after reading the 10/5
               | update to Schneier's post?
               | 
               | From your first link:
               | 
               |  _I misspoke when I wrote that NIST made "internal
               | changes" to the algorithm. That was sloppy of me. The
               | Keccak permutation remains unchanged. What NIST proposed
               | was reducing the hash function's capacity in the name of
               | performance. One of Keccak's nice features is that it's
               | highly tunable.
               | 
               | I do not believe that the NIST changes were suggested by
               | the NSA. Nor do I believe that the changes make the
               | algorithm easier to break by the NSA. I believe NIST made
               | the changes in good faith, and the result is a better
               | security/performance trade-off._
        
               | a1369209993 wrote:
               | Can't speak for aliqot, but I am now somewhat more
               | confident that the NIST changes were suggested by the
               | NSA, and slightly more confident (or at least less
               | unconfident) that SHA-3 is insecure.
               | 
               | I still think it's _probably_ fine, but I feel better
               | about insisting on SHA-2+mitigations or blake3 instead
               | now, even if the main problem with SHA-3 is its being
               | deliberately designed to encourage specialized hardware
               | accelleration (cf AES and things like Intel 's aes-ni).
               | 
               | (To be clear, the fact that Schneier claims to "believe
               | NIST made the changes in good faith" is weak but nonzero
               | evidence that they did not. I don't see any _concrete_
               | evidence for a backdoor, although you obviously shouldn
               | 't trust _me_ either.)
        
               | booi wrote:
               | Nice try NSA, that's exactly what you want us to believe!
               | 
               | jk I'm sure it's fine... probably
        
               | yarg wrote:
               | Skein always seemed interesting to me:
               | https://en.wikipedia.org/wiki/Skein_(hash_function)
               | 
               | But I've had a soft spot for Schneier since this (pre-
               | Snowden) blog post on Dual_EC_DRBG: https://www.schneier.
               | com/blog/archives/2007/11/the_strange_s....
        
               | nicolashahn wrote:
               | how else do you add a backdoor?
        
               | Dylan16807 wrote:
               | The original padding, as bits, was 100000...0001
               | 
               | The new padding prepends either 01, 11, or 1111 depending
               | on the variant of SHA-3. That way the different variants
               | don't sometimes give the same [partial] hash.
               | 
               | It was weird to toss that in at the last second but
               | there's no room for a backdoor there.
        
             | StreamBright wrote:
             | > Blake3 is super promising as cryptographic hash function
             | family due to its software performance
             | 
             | I am shocked how fast it is. Just tried it on several
             | files. It is blazing fast.
             | 
             | brew install b3sum on mac.
        
             | KMag wrote:
             | As far as Blake3 in hardware for anything other than very
             | low-power smart cards or similar:
             | 
             | Blake3 was designed from the ground up to be highly
             | optimized for vector instructions operating on four
             | vectors, each of 4 32-bit words. If you already have the
             | usual 4x32 vector operations, plus a vector permute (to
             | transform the operations across the diagonal of your 4x4
             | matrix into operations down columns) and the usual bypass
             | network to reduce latency, I think it would rarely be worth
             | the transistor budget to create dedicated Blake3 (or
             | Blake2s/b) instructions.
             | 
             | In contrast, SHA-3's state is conceptually five vectors
             | each of five 32-bit words, which doesn't map as neatly onto
             | most vector ISAs. As I remember, it has column and row
             | operations rather than column and diagonal operations that
             | parallelize better on vector hardware.
             | 
             | SHA-2 is a Merkle-Damgard construction where the round
             | function is a Davies-Meyer construction where the internal
             | block cipher is a highly unbalanced Feistel cipher.
             | Conceptually, you have a queue of 8 words (32-bit or
             | 64-bit, depending on which variant). Each operation pops
             | the first word from the queue, combines it in a nonlinear
             | way with 6 of the other words, adds one word from the "key
             | schedule" derived from the message, and pushes the result
             | on the back of the queue. The one word that wasn't
             | otherwise used is increased by the sum of the round key and
             | a non-linear function of 3 of the other words. As you might
             | imagine, this doesn't map very well onto general-purpose
             | vector instructions. This cipher is wrapped in a step
             | (Davies-Meyer construction) where you save a copy of the
             | state, encrypt the state using the next block of the
             | message, and then add the saved copy to the encrypted
             | result (making it non-invertible, making meet-in-the middle
             | attacks much more difficult). The key schedule uses a
             | variation on a lagged Fibonacci generator to expand each
             | message block into a larger number of round keys.
        
         | als0 wrote:
         | > It would be interesting if one of the SHA-4 requirements was
         | Zero Knowledge Proof friendlyness
         | 
         | What would this requirement look like and why would it be
         | important for a hash function?
        
       | greggarious wrote:
       | Nice, though it's scary many open source projects still just list
       | an MD5 sum for the binaries.
        
         | [deleted]
        
       | jimrandomh wrote:
       | SHA-256 is used for Bitcoin mining, which serves as an enormous
       | bug bounty for both full and partial breaks (if you can
       | efficiently find inputs where the output has lots of leading
       | zeroes that's a partial break, and lets you mine bitcoin more
       | efficiently). That's worth a lot of trust. I don't see any
       | particular reason to think SHA-3 is better (though I'm not an
       | expert) and unless I hear some indication that a problem has been
       | found with SHA-2, I'll probably stick with it forever.
        
         | tromp wrote:
         | > if you can efficiently find inputs where the output has lots
         | of leading zeroes that's a partial break, and lets you mine
         | bitcoin more efficiently
         | 
         | Depends on the type of break. If the break only allows finding
         | a hash with 128+k leading zeroes in 2^{128+k/2} time, that
         | would still be quite useless for bitcoin mining.
         | 
         | The break would have to cover the bitcoin regime of around 78
         | leading 0s.
        
         | mindcandy wrote:
         | In my research into Ethereum, I learned that in the creation of
         | SHA-3, they did a lot of hammering on SHA-256 to see if new
         | weaknesses could be discovered and addressed. The conclusion
         | was that SHA-256 is still solid as far as anyone can tell. The
         | SHA-3 process moved forward anyway so they could have a backup-
         | plan handy in case some problem with SHA-256 pops up out of
         | nowhere.
        
         | [deleted]
        
         | pclmulqdq wrote:
         | Bitcoin mining uses double SHA-256. It tends to be harder to
         | break doubled hash functions, since you don't have tools like
         | length-extension attacks on the second round of hashing. For
         | example, HMAC-SHA-1 is still secure (despite SHA-1 being pretty
         | much broken), and it also uses a two-round hashing
         | construction.
        
           | flatiron wrote:
           | Even if it didn't sha as a hmac is still fine because the
           | length it takes to break. You can't break sha1 in the time it
           | takes for your packet to go through the hmac step.
        
         | anderskaseorg wrote:
         | SHA-256 is vulnerable to a length extension attack. This is
         | well-known to cryptographers; it doesn't matter for some
         | applications and can be worked around in others. But it still
         | catches some developers unaware.
         | 
         | https://en.wikipedia.org/wiki/Length_extension_attack
        
           | adastra22 wrote:
           | Bitcoin mining uses double SHA256 which is not subject to
           | length extension attacks. (Not that it matters, because
           | bitcoin block headers are fixed length.)
        
           | tptacek wrote:
           | There are increasingly few situations in which length
           | extension really matters, because we know more about
           | designing protocols than we did 20 years ago; even code that
           | uses SHA3 or Blake2 tends to still use HMAC. Further, there
           | are standard variants of SHA2 that don't have length
           | extension (they're truncated and don't disclose their full
           | state). It's better to not have length extension properties
           | than to have them, but it's not really a big part of the
           | desiderata.
        
             | flatiron wrote:
             | I'm always confused when people slam sha on hmac. Is there
             | a realtime sha back door I'm missing? Even sha 1 takes
             | days/months to break.
        
           | [deleted]
        
           | phh wrote:
           | I didn't expect such attacks to exist, thanks for bringing
           | that up. However that Wikipedia page seem to say sha-256 is
           | ok since it's truncated?
        
             | Dylan16807 wrote:
             | SHA-256 is not truncated. "SHA-512/256" is truncated. It
             | means you do SHA-512 (with a different starting state) and
             | then throw out half.
        
             | CBLT wrote:
             | Sha-224 and Sha-384 are the truncated versions of Sha-256
             | and Sha-512 respectively.
             | 
             | My boring hash function of choice is Sha-384. The Sha-512
             | computation is faster on Intel hardware, and ASICS to crack
             | it are far more expensive than Sha-256 because of bitcoin.
             | 
             | If you're hashing passwords or something, use a "harder"
             | hash like Argon2 or Scrypt.
        
               | adrian_b wrote:
               | SHA-512 is faster only on Skylake derivatives up to Comet
               | Lake and on older Intel Core CPUs.
               | 
               | On Intel Atom starting with Apollo Lake (2016) and on
               | Intel Core starting with Ice Lake (2019) and on all AMD
               | Zen CPUs (2017), SHA-256 is implemented in hardware and
               | it is much faster than SHA-512.
        
           | AtNightWeCode wrote:
           | It is often worked around. But from what I understand the
           | length extension issue was raised during the design but
           | ignored. Hashing is not encryption but some of the
           | recommended encryption standards are so complicated that it
           | is a risk just by itself.
           | 
           | SHA-384 is a truncated SHA-512. From the claims of sec people
           | it does not offer more security when it comes to length
           | attacks. But from how the algo works I would assume that it
           | does.
           | 
           | Nist is also plain wrong about their calculations. Cause how
           | long it takes to calculate a specific hash depends on the
           | hardware available, not what theory books says. It may in
           | practice be faster to calculate a hash with more bits.
        
       | miga wrote:
       | That is date after which purchase of _new_ software with SHA-1
       | will be forbidden, which seems late given that it takes only two
       | months on GPU to find signature collision with a chosen prefix.
       | 
       | Sounds like the deprecation schedule is too slow and unsafe.
        
       | null0ranje wrote:
       | > NIST is announcing that SHA-1 should be phased out by Dec. 31,
       | 2030
       | 
       | Looks like it'll limp along for a while yet
        
         | dang wrote:
         | We've updated the title with a representative sentence from the
         | article which makes this clearer.
        
       | dreamcompiler wrote:
       | SHA-1 is still perfectly fine for some applications like
       | detecting duplicate files on a storage medium (and it's less
       | likely to produce a false positive than MD5) but it's been a bad
       | idea for anything security related for a decade.
       | 
       | The biggest issue is that git still uses it, which presents a
       | problem if you want to protect a repo from active integrity
       | attacks.
        
         | EGreg wrote:
         | Actually SHA1-HMAC is fine still. Although I had this question:
         | https://crypto.stackexchange.com/questions/102990/what-is-mo...
        
           | oconnore wrote:
           | Since I'm on a roll with truncated hashes: SHA2-512/384, and
           | SHA2-512/256 are not vulnerable to a Merkle-Damgard length
           | extension attack.
           | 
           | Thus, a construct like hash(key + message) can be used
           | similar to SHA3 [1]
           | 
           | [1]: https://keccak.team/keccak_strengths.html
        
           | classichasclass wrote:
           | This is worth emphasizing since HOTP (and thus TOTP) are
           | based on it.
        
             | [deleted]
        
         | amluto wrote:
         | > SHA-1 is still perfectly fine for some applications like
         | detecting duplicate files on a storage medium
         | 
         | If by "perfectly fine" you mean "subject to attacks that
         | generate somewhat targeted collisions that are practical enough
         | that people do them for amusement and excuses to write blog
         | posts and cute Twitter threads", then maybe I agree.
         | 
         | Snark aside, SHA-1 is _not fine_ for deduplication in any
         | context where an attacker could control any inputs. Do not use
         | it for new designs. Try to get rid of it in old designs.
        
           | orev wrote:
           | By "perfectly fine" they mean detecting duplicate image or
           | document files on your local storage, which it's still
           | perfectly fine for, and a frequent mode of usage for these
           | types of tools.
           | 
           | Not every tool needs to be completely resilient to an entire
           | Internets' worth of attacks.
        
             | dagenix wrote:
             | Why would you pick SHA1 which has significant attacks you
             | need to consider as opposed to something like SHA2 that
             | doesn't?
        
               | dreamcompiler wrote:
               | Because SHA1 is faster and that matters if you're hashing
               | several TB of files?
               | 
               | But I don't know for sure that's the case.
        
               | amluto wrote:
               | First the preamble:
               | 
               | DO NOT USE SHA-1 UNLESS IT'S FOR COMPATIBILITY. NO
               | EXCUSES.
               | 
               | With that out of the way: SHA-1 is not even particularly
               | fast. BLAKE2-family functions are faster. Quite a few
               | modern hash functions are also parallelizable, and SHA-1
               | is not. If for some reason you need something faster than
               | a fast modern hash, there are non-cryptographic hashes
               | and checksums that are extraordinarily fast.
               | 
               | If you have several TB of files, and for some reason you
               | use SHA-1 to dedupe them, and you later forget you did
               | that and download one of the many pairs of amusing SHA-1
               | collisions, you will lose data. Stop making excuses.
        
               | VLM wrote:
               | > there are non-cryptographic hashes and checksums that
               | are extraordinarily fast.
               | 
               | Is it still true that CRC32 is only about twice as fast
               | as SHA1?
               | 
               | Yeah I know the XX hashes are like 30 times faster than
               | SHA1.
               | 
               | A lot depends on instruction set and processor choice.
               | 
               | Maybe another way to put it is I've always been impressed
               | that on small systems SHA1 is enormously longer but only
               | twice as slow as CRC32.
               | 
               | For a lot of interoperable-maxing non-security non-crypto
               | tasks, CRC32 is not a bad choice, if its good enough for
               | Ethernet, zmodem, and mpeg streams its good enough for my
               | telemetry packets LOL. (IIRC iSCSI uses some variant
               | different formulae)
        
               | adrian_b wrote:
               | CRC32 is fine for kilobyte-sized data packets.
               | 
               | For files, it is useless. Even if that was expected, I
               | have computed CRC32 for all the files on an SSD. Of
               | course, I have found thousands of collisions.
        
               | jcrawfordor wrote:
               | Deduplication is the kind of application where CRC is a
               | decent approach and CRC has no resistance to attack
               | whatsoever. SHA1 adds the advantage of lower natural
               | collision probability while still being extremely fast.
               | It's important to understand that not all applications of
               | hashing are cryptographic or security applications, but
               | that the high degree of optimization put into
               | cryptographic algorithms often makes them a convenient
               | choice in these situations.
               | 
               | These types of applications are usually using a
               | cryptographic hash as one of a set of comparison
               | functions that often start with file size as an
               | optimization and might even include perceptual methods
               | that are intentionally likely to produce collisions. Some
               | will perform a byte-by-byte comparison as a final test,
               | although just from a performance perspective this
               | probably isn't worth the marginal improvement even for
               | hash functions in which collisions are known to occur but
               | vanishingly rare in organic data sets (this would include
               | for example MD5 or even CRC at long bit lengths, but the
               | lack of mixing in CRC makes organic collisions much more
               | common with structured data).
               | 
               | SHA2 is significantly slower than SHA1 on many real
               | platforms, so given that intentional collisions are not
               | really part of the problem space few users would opt for
               | the "upgrade" to SHA2. SHA1 itself isn't really a great
               | choice because there are faster options with similar
               | resistance to accidental collisions and worse resistance
               | to intentional ones, but they're a lot less commonly
               | known than the major cryptographic algorithms. Much of
               | the literature on them is in the context of data
               | structures and caching so the bit-lengths tend to be
               | relatively small in that more collision-tolerant
               | application and it's not always super clear how well they
               | will perform at longer bit lengths (when capable).
               | 
               | Another way to consider this is from a threat modeling
               | perspective: in a common file deduplication operation,
               | when files come from non-trusted sources, someone might
               | be able to exploit a second-preimage attack to generate a
               | file that the deduplication tool will errantly consider a
               | duplicate with another file, possibly resulting in one of
               | the two being deleted if the tool takes automatic action.
               | SHA1 actually remains highly resistant to preimage and
               | second preimage attacks, so it's not likely that this is
               | even feasible. SHA1 does have known collision attacks but
               | these are unlikely to have any ramifications on a file
               | deduplication system since both files would have to be
               | generated by the adversary - that is, they can't modify
               | the organic data set that they did not produce. I'm sure
               | you could come up with an attack scenario that's feasible
               | with SHA1 but I don't think it's one that would occur in
               | reality. In any case, these types of tools are not
               | generally being presented as resistant to malicious
               | inputs.
               | 
               | If you're working in this problem space, a good thing to
               | consider is hashing only subsets of the file contents,
               | from multiple offsets to avoid collisions induced by
               | structured parts of the format. This avoids the need to
               | read in the entire file for the initial hash-matching
               | heuristic. Some commercial tools initially perform
               | comparisons on only the beginning of the file (e.g. first
               | MB) but for some types of files this is going to be a lot
               | more collision prone than if you incorporate samples from
               | regular intervals, e.g. skipping over every so many
               | storage blocks.
        
               | adrian_b wrote:
               | SHA-1 is implemented in hardware in all modern CPUs and
               | it is much faster than any alternatives (not all
               | libraries use the hardware instructions, so many popular
               | programs compute SHA-1 much more slowly than possible;
               | OpenSSL is among the few that use the hardware).
               | 
               | When hashing hundreds of GB or many TB of data, the hash
               | speed is important.
               | 
               | When there are no active attackers and even against
               | certain kinds of active attacks, SHA-1 remains secure.
               | 
               | For example, if hashes of the files from a file system
               | are stored separately, in a secure place inaccessible for
               | attackers (or in the case of a file transfer the hashes
               | are transferred separately, through a secure channel), an
               | attacker cannot make a file modification that would not
               | be detected by recomputing the hashes.
               | 
               | Even if SHA-1 remains secure against preimage attacks, it
               | should normally be used only when there are no attackers,
               | e.g. for detecting hardware errors a.k.a. bit rotting, or
               | for detecting duplicate data in storage that could not be
               | accessed by an attacker.
               | 
               | While BLAKE 3 (not BLAKE 2) can be much faster than
               | SHA-1, all the extra speed is obtained by consuming
               | proportionally more CPU resources (extra threads and
               | SIMD). When the hashing is done in background, there is
               | no gain by using BLAKE 3 instead of SHA-1, because the
               | foreground tasks will be delayed by the time gained for
               | hashing.
               | 
               | Only when a computer does only hashing, BLAKE 3 is the
               | best choice, because the hash will be computed in a
               | minimal time, by fully using all the CPU cores.
        
               | dylan604 wrote:
               | who is attacking you in this situation though? you're
               | scanning the files on your local system and storing their
               | hashes. you then look for duplicate hashes, and compare
               | the files that created them. if the files are truly
               | duplicates, you can now decide what to do about that. if
               | they are not truly the same, then you claim to have found
               | another case of collisions, write your blog
               | post/twitthread and move on, but does that constitute
               | being attacked?
               | 
               | sometimes, i really feel like people in crypto just can't
               | detach themselves enough to see that just because they
               | have a hammer, not everything in the world is a nail.
        
               | lokar wrote:
               | If you are going to follow up with a full comparison you
               | can use something much cheaper then sha-1
        
               | bawolff wrote:
               | I dont think this is what most people think of when they
               | say deduplication. There are quite a few systems which
               | will just scan for duplicates and then automatically
               | delete one of the duplicates. In such a system sha1 would
               | be inappropriate.
               | 
               | If you are just using sha1 as a heuristic you dont fully
               | trust, i suppose sha1 is fine. It seems a bit of an odd
               | choice though as something like MurmurHash would be much
               | faster for such a use case.
        
               | adrian_b wrote:
               | SHA-1 cannot be trusted only when there is a possibility
               | that _both_ files whose hashes are compared have been
               | created by an attacker.
               | 
               | While such a scenario may be plausible for a public file
               | repository, so SHA-1 is a bad choice for a version
               | control system like GIT, there are a lot of applications
               | where this is impossible, so it is fine to use SHA-1.
        
               | dagenix wrote:
               | Why would you pick a function that is known to have
               | issues when there are other functions that do the same
               | thing but don't have known issues?
               | 
               | Your comparison is flawed. It's more like if you have a
               | nail and next to it a workbench with two hammers - a good
               | hammer and a not as good hammer. This isn't a hard
               | choice. But for reasons that are unclear to me, people in
               | this thread are insisting on picking the less good hammer
               | and rationalizing why for this specific nail it isn't all
               | that much worse. Just pick the better hammer!
        
               | oefrha wrote:
               | Because people already have two decades of SHA-1 hashes
               | in their database and a rewrite + rescan is completely
               | pointless? Hell, I have such a system using md5. So you
               | produced a hash collision, cool, now fool my follow-on
               | byte-by-byte comparison.
               | 
               | Edit: Before anyone lecture me on SHA-1 being slow, yes,
               | I use BLAKE2 for new projects.
        
               | dylan604 wrote:
               | thanks for the reply that did not answer the question
               | asked.
        
               | dagenix wrote:
               | Your question is irrelevant. If you don't care about
               | security, SHA1 is a bad choice because there are faster
               | hash functions out there. If you do care about security,
               | SHA1 is a bad choice because it has known flaws and there
               | exist other algorithms that don't. The only valid reason
               | to use SHA1 is if there is a historical requirement to
               | use it that you can't reasonably change.
               | 
               | Any analysis about how hard it is for an attacker to get
               | a file on your local file system via a cloned got repo,
               | cached file, email attachment, image download, shared
               | drive, etc is just a distraction.
        
               | adrian_b wrote:
               | You would be right, except that there are no faster hash
               | functions (on all modern Intel, AMD and ARM CPUs, which
               | have hardware SHA-1).
               | 
               | BLAKE 3 is faster only in wall clock time, on an
               | otherwise idle computer, because it fully uses all CPU
               | cores, but it does not do less work.
               | 
               | BLAKE 3 is preferable only when the computer does nothing
               | else but hashing.
        
               | VLM wrote:
               | Its worship of the new. Nothing worth anything existed
               | last week, so reimplementing an entire infrastructure is
               | zero cost.
        
               | ShowalkKama wrote:
               | because it's shorter, dealing with long hashes is
               | annoying (I use md5, not sha1)
        
               | bawolff wrote:
               | You could just discard half the sha256 hash. Using the
               | first 16 bytes of sha256 is a lot more secure than using
               | just md5, in which case you might as well just use crc32.
        
             | bawolff wrote:
             | Ask the webkit source code repo how using sha-1 for that
             | purpose went.
             | 
             | It depends what you are doing, but deduplication where a
             | collision means you loose data, seems like an inapropriate
             | place for sha-1.
        
               | bornfreddy wrote:
               | Webkit incident: https://itwire.com/business-it-
               | news/security/fallout-webkit-...
        
           | Pet_Ant wrote:
           | > where an attacker could control any inputs
           | 
           | I believe the GP's point hinges on the word "attacker". If
           | you aren't in a hostile space, like just your won file server
           | and you are monitoring your own backups it's fine. I still
           | use MD5s to version my own config files. For personal use in
           | non-hostile environments these hashes are still perfectly
           | fine.
        
         | 0xbadcafebee wrote:
         | If you're _only_ using the hash for non-cryptographic
         | applications, there are much faster hashes:
         | https://github.com/Cyan4973/xxHash
         | 
         | SHA1 and MD5 are the most widely accessible, though, and I
         | agree it's fine to use them if you don't care about security.
        
         | dragontamer wrote:
         | I feel like the properties of CRC make them superior for that
         | task in most cases though. (CRC8, CRC16, CRC32 and CRC64, maybe
         | CRC128 if anyone ever bothered going that far)
         | 
         | In particular, CRC guarantees detection on all bursts of the
         | given length. CRC32 protects vs all bursts of length 32 bits.
        
           | AlotOfReading wrote:
           | There are only a few examples of anything larger than CRC64
           | being characterized and they're not very useful.
           | 
           | For the sake of the next person who has to maintain your code
           | though, please choose algorithms that adequately communicate
           | your intentions. Choose CRCs only if you need to detect
           | random errors in a noisy channel with a small number of bits
           | and use a length appropriate to the intended usage (i.e.
           | almost certainly not CRC64).
        
             | dragontamer wrote:
             | And when you choose SHA1, does it mean you understood that
             | it's no longer secure? Or is it chosen because it was
             | secure 20 years ago but the code is old and needs to be
             | updated?
             | 
             | CRC says that you never intended security from the start.
             | It's timeless, aimed to prevent burst errors and random
             | errors.
             | 
             | --------
             | 
             | BTW, what is the guaranteed Hamming distance between SHA1?
             | How good is SHA1 vs burst errors? What about random errors?
             | 
             | Because the Hamming distances of CRC have been calculated
             | and analyzed. We actually can determine, to an exact level,
             | how good CRC codes are.
        
               | benlivengood wrote:
               | https://eprint.iacr.org/2019/078 is a good overview of
               | the statistical properties of SHA hashes.
               | 
               | Basically after ~10 rounds the output is always
               | indistinguishable from randomness which means hamming
               | distance is what you'd expect (about half the bits
               | differ) between the hashes of any two bitstreams.
        
               | dragontamer wrote:
               | CRC does better than random vs burst errors. And vs
               | random errors is indistinguishable from any other
               | methodology.
               | 
               | If you are only worried about random errors, might as
               | well use chksum.
        
               | AlotOfReading wrote:
               | You shouldn't choose SHA-1, that's the point of this
               | announcement. Seeing it indicates both that there was the
               | potential for malicious input and that the code is old.
               | The appropriate mitigation is to move to a secure hash,
               | not CRCs. You may not know the bounds and distances
               | exactly, but you know them probabilistically. Bit errors
               | _almost always_ map to a different hash.
               | 
               | The same is true of CRCs over a large enough input as an
               | aside.
        
               | dagenix wrote:
               | If you are choosing between a CRC and SHA1, you probably
               | need to reconsider your understanding of the problem you
               | are trying to solve. Those algorithms solve different use
               | cases.
        
               | dragontamer wrote:
               | If you are choosing SHA1, now that it is retired, you
               | probably should think harder about the problem in
               | general.
               | 
               | CRC should be better for any error detection code issue.
               | Faster to calculate, more studied guaranteed detection
               | modes, and so forth.
               | 
               | SHA1 has no error detection studies. It's designed as a
               | cryptographic hash, to look random. As it so happens, it
               | is more efficient to use other algorithms and do better
               | than random if you have a better idea of how your error
               | looks like.
               | 
               | Real world errors are either random or bursty. CRC is
               | designed for these cases. CRC detects the longest burst
               | possible for it's bitsize.
        
           | IncRnd wrote:
           | > I feel like the properties of CRC make them superior for
           | that task in most cases though.
           | 
           | THIS IS FALSE. Please do not ever do this. Why not? For
           | example, by controlling any four contiguous bytes in a file,
           | the resultant 32bit CRC can be forced to take on any value. A
           | CRC is meant to detect errors due to noise - not changes due
           | to a malicious actor.
           | 
           | Program design should not be done based upon one's feelings.
           | CRCs absolutely do not have the required properties to detect
           | duplication or to preserve integrity of a stored file that an
           | attacker can modify.
        
             | dragontamer wrote:
             | > THIS IS FALSE. Please do not ever do this. Why not? For
             | example, by controlling any four contiguous bytes in a
             | file, the resultant 32bit CRC can be forced to take on any
             | value.
             | 
             | And SHA1 is now broken like this, with collisions and so
             | forth. Perhaps it's not as simple as just 4 bytes, but the
             | ability to create collisions is forcing this retirement.
             | 
             | If adversarial collisions are an issue, then MD5 and SHA1
             | are fully obsolete now. If you don't care for an adversary,
             | might as well use the cheaper, faster CRC check.
             | 
             | ------
             | 
             | CRC is now more valid use case than SHA1. That's the point
             | of this announcement.
        
               | IncRnd wrote:
               | > CRC is now more valid use case than SHA1. That's the
               | point of this announcement.
               | 
               | No. That isn't the point of this announcement. This
               | announcement codifies a transition time for SHA-1 use in
               | SBU and has nothing to do with a CRC.
        
               | Dylan16807 wrote:
               | The point of the announcement is to give a timeline to
               | start treating SHA1 as having no real security.
               | 
               | This is relevant to when you'd choose CRC, because it
               | also has no real security.
        
         | LinuxBender wrote:
         | _SHA-1 is still perfectly fine for some applications like
         | detecting duplicate files on a storage medium_
         | 
         | Absolutely agree, especially when speed is a workable trade-off
         | and accepting real world hash collisions are unlikely and
         | perhaps an acceptable risk. For financial data, especially
         | files not belonging to me I would have md5+sha1+sha256
         | checksums and maybe even GPG sign a manifest of the checksums
         | ... because why not. For my own files md5 has always been
         | sufficient. I have yet to run into a real world collision.
         | 
         | FWIW anyone using `rsync --checksum` is still using md5. Not
         | that long ago _I think 2014_ it was using md4. I would be
         | surprised if rsync started using anything beyond md5 any time
         | soon. I would love to see all the checksum algorithms become
         | CPU instruction sets.                   Optimizations:
         | no SIMD-roll, no asm-roll, no openssl-crypto, asm-MD5
         | Checksum list:             md5 md4 none         Compress list:
         | zstd lz4 zlibx zlib none         Daemon auth list:
         | md5 md4
        
           | tptacek wrote:
           | You're never going to _accidentally_ run into an MD5
           | collision.
        
         | dwheeler wrote:
         | Git no longer uses SHA-1. It instead uses a variant called
         | SHA-1DC that detects some known problems, and in those cases
         | returns a different answer. More info: <https://github.com/cr-
         | marcstevens/sha1collisiondetection>. Git switched to SHA-1DC in
         | its version 2.13 release in 2017. It's a decent stopgap but not
         | a grrat long term solution.
         | 
         | There is also work to support SHA-256, though that seems to
         | have stalled: https://lwn.net/Articles/898522/
         | 
         | The fundamental problem is that get developers assumed that
         | hash algorithms would never be changed, and that was a
         | ridiculous assumption. It's much wiser to implement crypto
         | agility.
        
           | hejaodbsidndbd wrote:
           | It's not called SHA-1DC. It's called "some blob of C whose
           | behaviour is never described anywhere".
        
             | xena wrote:
             | If there's a readily-avaliable blob of C code that does the
             | operation, then by definition it must be described
             | somewhere. Maybe you should get ChatGPT to describe what it
             | does.
        
               | hejaodbsidndbd wrote:
        
           | derbOac wrote:
           | Why isn't Git using something else? Why go to the trouble of
           | implementing something like that?
           | 
           | I don't mean that as some ridiculing criticism, I just am
           | genuinely puzzled.
        
             | chungy wrote:
             | Changing out the hashing algorithm in Git is a very
             | difficult thing to do.
        
               | cerved wrote:
               | not within the git project so much as all the other code
               | that depends on it
        
             | duskwuff wrote:
             | > Why isn't Git using something else?
             | 
             | Because switching to a different hash algorithm would break
             | compatibility with all existing Git clients and
             | repositories.
        
           | woodruffw wrote:
           | > The fundamental problem is that get developers assumed that
           | hash algorithms would never be changed, and that was a
           | ridiculous assumption. It's much wiser to implement crypto
           | agility.
           | 
           | Cryptographic agility makes this problem worse, not better:
           | instead of having a "flag day" (or release) where `git`'s
           | digest choice reflects the State of the Art, agility ensures
           | that every future version of `git` can be downgraded to a
           | broken digest.
        
             | est31 wrote:
             | That's the general anti-agility argument wielded against
             | git, but note that git's use cases require it to process
             | historic data.
             | 
             | E.g. you will want to be able to read some sha-1-only repo
             | from disk that was last touched a decade ago. That's a
             | different thing than some protocol which requires both
             | parties to be on-line, say wireguard, in which instance
             | it's easier to switch both to a new version that uses a
             | different cryptographic algorithm.
             | 
             | Git has such protocols as well, and maybe it can deprecate
             | sha-1 support there eventually, but even there it has to
             | support both sha-1 and sha-2 for a while because not
             | everyone is using the latest and greatest version of git,
             | and no sysadmin wants the absolute horror of flag days.
        
               | miga wrote:
               | It would be safer to forbid broken hashes after certain
               | date, and consider only those earlier hashes that have
               | been counter-signed by new algorithms.
        
             | Dylan16807 wrote:
             | Assuming reasonable logic around hashes, like "a SHA-2
             | commit can't be a parent of a SHA-1 commit", there wouldn't
             | much in the way of downgrade attacks available.
        
               | bornfreddy wrote:
               | Wow, smart! This would keep all the old history intact
               | and at the same time force lots of people to upgrade
               | through social pressure. I'd probably be angry as hell
               | when that happened to me, but it would also work.
        
           | AgentME wrote:
           | SHA-1 was already known to be broken at the time Git chose
           | it, but they chose it anyway. Choosing a non-broken algorithm
           | like SHA-2 was an easy choice they could have made that would
           | still hold up today. Implementing a crypto agility system is
           | not without major trade-offs (consider how common downgrade
           | attacks have been across protocols!).
        
             | fishywang wrote:
             | >SHA-1 was already known to be broken at the time Git chose
             | it
             | 
             | Please pardon my ignorance but could you elaborate on what
             | time (e.g. the year) are you referring to?
        
               | LarryMullins wrote:
               | Since about 2005, collision attacks against SHA-1 have
               | been known. In 2005 Linus dismissed these concerns as
               | impractical, writing:                   > The basic
               | attack goes like this:         >         > - I construct
               | two .c files with identical hashes.                  Ok,
               | I have a better plan.                  - you learn to fly
               | by flapping your arms fast enough         - you then
               | learn to pee burning gasoline         - then, you fly
               | around New York, setting everybody you see on fire, until
               | people make you emperor.                  Sounds like a
               | good plan, no?                  But perhaps slightly
               | impractical.                  Now, let's go back to your
               | plan. Why do you think your plan is any better
               | than mine?
               | 
               | https://git.vger.kernel.narkive.com/9lgv36un/zooko-zooko-
               | com...
        
               | nighthawk454 wrote:
               | I didn't know either. From Wikipedia [1], SHA-1 has been
               | considered insecure to some degree since 2005. Following
               | the citations, apparently it's been known since at least
               | August 2004 [2] but maybe not demonstrated in SHA-1 until
               | early 2005.
               | 
               | git's first release was in 2005, so I guess technically
               | SHA-1 issues could've been known or suspected during
               | development time.
               | 
               | More generously, it could've been somewhat simultaneous.
               | It sounds like it was considered a state-sponsored level
               | attack at the time, if collisions were even going to be
               | possible. Don't know if the git devs knew this and
               | intentionally chose it anyway, or just didn't know.
               | 
               | [1] https://en.wikipedia.org/wiki/SHA-1
               | 
               | [2] https://www.schneier.com/blog/archives/2005/02/crypta
               | nalysis...
               | 
               | EDIT: sibling comment has evidence that Linus did in fact
               | know about it and considered it an impractical vector at
               | the time
               | 
               | https://git.vger.kernel.narkive.com/9lgv36un/zooko-zooko-
               | com...
        
         | VLM wrote:
         | "We recommend that anyone relying on SHA-1 for security migrate
         | to SHA-2 or SHA-3 as soon as possible." --Chris Celi, NIST
         | computer scientist
         | 
         | The emphasis being on "for security"
         | 
         | I've also used SHA-1 over the years for binning and verifying
         | file transfer success, none of those are security related.
         | 
         | Sometimes, if you make a great big pile of different systems,
         | what's held in common across them can be weird, SHA-1 popped
         | out of the list so we used it.
         | 
         | I'm well aware its possible to write or automate the writing of
         | dedicated specialized "perfect" hashing algos to match the
         | incoming data, to bin the data more perfectlyier, but sometimes
         | its nice if wildly separate systems all bin incoming data the
         | same highly predictable way thats "good enough" and "fast
         | enough".
        
           | jraph wrote:
           | > I've also used SHA-1 over the years for binning and
           | verifying file transfer success, none of those are security
           | related.
           | 
           | It could. If you want to verify that the file has not been
           | tempered by someone, it is security related.
        
         | kiratp wrote:
         | As an industry we need to get over this pattern of scoping down
         | usage of something that has failed it's prime directive. People
         | still use MD5 in secure related things because it's been
         | allowed to stick around without huge deprecation warnings in
         | libraries and tools.
         | 
         | SHA1 (and MD5) need to be treated the same way you would treat
         | O(n^2) sorting in a code review for a PR written by a newbie.
        
         | bawolff wrote:
         | But sha-1 is now both slow and insecure.
         | 
         | If you dont care about security, use a faster hash. If you care
         | about security use sha256 (which is about the same speed
         | anyways).
         | 
         | The only valid reason to still use it in non-security critical
         | roles is backwards compat.
        
         | oconnore wrote:
         | If you need a shorter hash just truncate a modern hash
         | algorithm down to 160 or 128 bits. Obviously the standard
         | lengths were chosen for a reason, but SHA2-256/160 or
         | SHA2-256/128 are better hash functions than SHA1 or MD5,
         | respectively. Blake2b/160 is even faster than SHA1!
         | 
         | (I suspect this would be a good compromise for git, since so
         | much tooling assumes a 160 bit hash, and yet we don't want to
         | continue using SHA1)
        
           | kzrdude wrote:
           | Git has already implemented a solution based on SHA-2 with
           | 256 bit output so that's unlikely to be changed for the time
           | being. (But it has not really been launched in earnest, only
           | as a preview feature.)
        
           | danbruc wrote:
           | Just as a note, the primary reason for the truncated variants
           | is not to get a shorter hash but to prevent extension
           | attacks. For variants without truncation, the final hash is
           | the entire internal state, therefore an attacker can
           | calculated the hash of any message that starts with the
           | original message and then has additional content without
           | knowing the original message. Truncating the hash denies
           | access to the complete internal state and makes this
           | impossible.
        
             | cesarb wrote:
             | Another way to prevent extension attacks is to make the
             | internal state different whenever the current block is the
             | last block, as done for instance in BLAKE3 (which has as an
             | additional input on each block a set of flags, and one of
             | the flags says "this is the last block").
        
           | GTP wrote:
           | Or use Keccak (which is the permutation that was chosen for
           | SHA3), which lets you pick the length that you need.
        
         | umvi wrote:
         | What kind of attack against a git repo can be realistically
         | carried out today?
        
           | kadoban wrote:
           | Given a git repo with signed commits I can give you the same
           | repo with some file(s) changed with the signatures intact.
           | 
           | Pretty sure you can anyway, I haven't thought deeply about
           | the git file formats involved and such.
           | 
           | Note that this attack isn't _that_ serious. There's not a lot
           | of cases where this would make sense.
        
             | AnssiH wrote:
             | I don't think so, unless you utilize some not-yet-public
             | vulnerability.
             | 
             | As far as I know, with current public SHA-1
             | vulnerabilities, you can create two new objects with the
             | same hash (collision attack), but cannot create a second
             | object that has the same hash as some already existing
             | object (preimage attack).
        
           | [deleted]
        
         | hannob wrote:
         | > SHA-1 is still perfectly fine for some applications like
         | detecting duplicate files on a storage medium
         | 
         | That's what the developers of subversion thought, but they
         | didn't anticipate that once colliding files were available
         | people would commit them to SVN repos as test cases. And then
         | everything broke:
         | https://www.bleepingcomputer.com/news/security/sha1-collisio...
        
           | TheRealPomax wrote:
           | That changes the parameters quite a bit though. For local
           | digests, like image deduplication of your own content, on
           | your own computers, sha-1 is still perfectly fine. Heck, even
           | MD5 is still workable (although more prone to collide).
           | Nowhere in that process is the internet, or "users" or
           | anything else like that involved =)
           | 
           | You use digests to quickly detect potential collisions, then
           | you verify each collision report, then you delete the actual
           | duplicates. Human involvement still very much required
           | because you're curating your own data.
        
             | dylan604 wrote:
             | If we're talking specifically image deduplication, then a
             | hash comparison is only going to find you exact matches.
             | what about the image deduplication of trying to find alt
             | versions of things like scaling, different codecs, etc?
             | 
             | if you want to dedupe images, some sort of phashing would
             | be much better so that the actual image is considered vs
             | just the specific bits to generate the image.
        
               | TheRealPomax wrote:
               | Depends on the images. For photographs, a digest is
               | enough. For "random images downloaded from the web", or
               | when you're deduplicating lots of user's data, sure, you
               | want data appropriate digests, like SIFT prints. But then
               | we're back to "you had to bring network content back into
               | this" =)
        
               | dreamcompiler wrote:
               | Very true but if the hash matches the images are
               | guaranteed to match too. That's my first pass when
               | deduping my drives. My second pass is looking for
               | "logically equivalent" images.
        
       | charlieyu1 wrote:
       | At least 20 years too late
        
       | upofadown wrote:
       | I wonder what NIST does for a case where SHA-1 is not used for
       | any cryptographic properties? I recently ran into that for
       | OpenPGP. The authenticated encryption mode uses SHA-1 because it
       | was otherwise used in the standard but because of how things work
       | only a non-cryptographically secure hash is required.
       | 
       | I have somewhat jovially suggested that the OpenPGP standard
       | should just rename it if it turns out that the name becomes a
       | problem...
        
         | kzrdude wrote:
         | Yes, they say "Modules that still use SHA-1 after 2030 will not
         | be permitted for purchase by the federal government" so I
         | wonder... does that mean products that can read 2022-era git
         | repos (using sha-1) are they no longer allowed by that date,
         | and so on? There must be exceptions by use case!
        
           | klabb3 wrote:
           | Use is quite vague in this context, at least having a
           | distinction between read/write would be more clear.
        
         | [deleted]
        
         | woodruffw wrote:
         | I'm not sure I understand what you mean by PGP not requiring
         | SHA-1's cryptographic properties. Do you mean that PGP's
         | authenticated encryption mode only requires preimage
         | resistance?
        
           | upofadown wrote:
           | Not even that. It only requires that the hash is not
           | reversible.
        
             | woodruffw wrote:
             | Can you elaborate? This doesn't match my understanding of
             | how "authenticated" encryption works in PGP (by which I
             | assume you mean MDC, which is closer to integrity-without-
             | identity than authentication).
        
               | upofadown wrote:
               | For most PGP use, the MDC only serves as an integrity
               | check [1]. That is the same for the proposed modes as
               | well. In the case of symmetrical encryption it does in
               | fact serve to authenticate the encrypted material based
               | on the passphrase.
               | 
               | It does not use the popular combination of an encryption
               | function acting more or less independently of a MAC
               | (message authentication code). It uses a different
               | method[2]. This seems to cause much confusion.
               | 
               | [1]
               | https://articles.59.ca/doku.php?id=pgpfan:authenticated
               | 
               | [2] https://articles.59.ca/doku.php?id=pgpfan:mdc
        
         | Spooky23 wrote:
         | If you're in a regulated industry and required to run FIPS
         | validated modules in FIPS mode, usually you lose access to
         | system implementations of the removed algorithm.
        
         | twiss wrote:
         | FWIW, the "crypto refresh" of the OpenPGP standard specifies
         | the use of AEAD for authenticated encryption, replacing the
         | SHA-1 "modification detection code". Even if the latter isn't
         | broken, the former is more modern and more performant, and so
         | retconning the use of SHA-1 for optics shouldn't be necessary
         | once everyone moves to that :)
        
           | upofadown wrote:
           | Even assuming that anything real comes out of the "crypto
           | refresh" (it seems to be going the way of the last effort) it
           | would still be irresponsible to promote the use of an
           | entirely incompatible encryption mode for no real reason. The
           | existing standard has been in use for over 20 years now. Even
           | if there were some sort of security weakness discovered it
           | would make more sense to try to resolve that issue in some
           | sort of backwards compatible way. Since there is no weakness
           | then the best course of action seems pretty clear and is
           | really easy to do. The crypto refresh is currently attempting
           | to add no less than three new authenticated encryption modes
           | in addition to the existing one for a total of four. Each and
           | every mode is entirely incompatible with the others. This
           | strikes me as sort of nuts...
        
             | [deleted]
        
             | twiss wrote:
             | Only one of them, OCB, is mandatory to implement. The other
             | two are optional, but there may be performance or
             | compliance reasons to use one of them. E.g., people who
             | care about FIPS compliance may want to use GCM.
             | 
             | In any case, since there are feature flags and algorithm
             | preferences to signal support for these, all of this is in
             | fact backwards compatible. There's little risk that someone
             | will accidentally use an AEAD mode to encrypt a message for
             | a recipient that doesn't support it, since the recipient
             | needs to signal support for it in their public key.
             | 
             | And, offering performance and security benefits for those
             | that care to upgrade their implementations and keys is
             | still a worthy goal, IMHO.
        
               | upofadown wrote:
               | >The other two are optional,...
               | 
               | That just makes things worse. The OpenPGP standard covers
               | the offline, static encryption case. You have an
               | encrypted file or message. If your implementation has
               | implemented the encryption method then you can decrypt
               | it. If it doesn't than you can't. Contrast this with an
               | online, dynamic method like TLS where you can negotiate a
               | method at the time of encryption and decryption.
               | 
               | >In any case, since there are feature flags and algorithm
               | preferences to signal support for these, all of this is
               | in fact backwards compatible.
               | 
               | The OpenPGP preferences are not applicable to symmetric
               | encryption. In that case the implementation has to guess
               | what method will be supported by the decrypting
               | implementation. In most cases it would make sense to use
               | the most widely implemented method. That is always going
               | to be the one that has been used since forever.
               | 
               | The OpenPGP preferences are included in the public key at
               | key generation time. They reflect the capabilities of
               | that implementation. The public key then has a life of
               | it's own. It is quite normal to use that key to encrypt
               | material for other implementations. Having optional
               | methods, again, makes things much worse here. This is not
               | a theoretical problem. I have already been involved in a
               | difficult usability issue caused by an implementation
               | producing one of these new incompatible encryption modes.
               | The file was created on the same implementation as the
               | public key. So the implementation saw that the reciepient
               | supported all the same methods that it did. But the
               | actual implementation that did the decryption did not
               | support that mode. This sort of interoperability problem,
               | even if it only happens from time to time, seriously
               | impacts usability. That is the ultimate issue. Why are we
               | making things harder for the users of these systems for
               | no real reason?
        
               | twiss wrote:
               | > The OpenPGP preferences are not applicable to symmetric
               | encryption. In that case the implementation has to guess
               | what method will be supported by the decrypting
               | implementation. In most cases it would make sense to use
               | the most widely implemented method. That is always going
               | to be the one that has been used since forever.
               | 
               | I agree, if you don't know the capabilities of the
               | implementation decrypting the message then it makes sense
               | to be conservative, and wait until these AEAD is widely
               | implemented before using it.
               | 
               | > I have already been involved in a difficult usability
               | issue caused by an implementation producing one of these
               | new incompatible encryption modes.
               | 
               | I also think that generating AEAD encrypted OpenPGP
               | messages today is irresponsible, since the crypto refresh
               | is still a draft, not an RFC yet. Even if the decrypting
               | implementation _can_ read the message today, the draft
               | could still change (although now that it 's in Working
               | Group Last Call it's unlikely), and then you'd have an
               | even bigger problem.
               | 
               | But I think that's the fault of the implementation, not
               | of the (proposed) standard. If we eventually want to have
               | the security and performance benefits of AEAD, we have to
               | specify it today (well, or yesterday, but that's a bit
               | hard to change now ^.^).
        
               | upofadown wrote:
               | >If we eventually want to have the security and
               | performance benefits of AEAD,...
               | 
               | The thing is, there doesn't seem to be any security
               | weaknesses with the existing AE method. I have looked
               | hard. There also doesn't seem to be any need for the AD
               | (associated data) part.
               | 
               | OCB seems to be the fastest out of all of them. If the
               | proposal was just to add on OCB as an enhanced
               | performance mode then I might be OK with that. Why make
               | the people encrypting multi-TB files wait? I am mostly
               | grumpy with the idea that we have to drop an existing
               | well established standard for stuff like messaging and
               | less extreme file sizes.
        
       | zachruss92 wrote:
       | It's about time haha. Once they found real world conflicts it
       | needed to be phased out for anything security related. At least
       | it isn't very hard to move from SHA-1 to SHA-256.
        
         | px43 wrote:
         | AKA, SHA-2 :-)
        
           | SAI_Peregrinus wrote:
           | SHA-2 is an entire family of hash functions: SHA-228,
           | SHA-256, SHA-384, SHA-512, SHA-512/224, and SHA-512/256.
        
       | snhly wrote:
       | Other than legacy stuff, I think the main reason programmers
       | still use it is the fact that '1' is easier for them to call to
       | mind than '256', or whatever else. So when you're throwing a
       | script together really quickly, you'll prefer to type SHA1
       | instead of anything else. At least that's how my brain works.
        
       | jodrellblank wrote:
       | Who named it "Secure Hash Algorithm" :|
       | 
       | Now the name will be annoyingly and misleadingly wrong _forever_
       | , in a way that was totally predictable.
        
         | GTP wrote:
         | That was the name of the standard, for which they had to pick a
         | "Secure Hash Algorithm". And it's normal to have to update
         | standards over time, regardless of the name. Which name would
         | you have suggested for a standard defining a secure hash
         | function? Usually it's better to not use fantasy names for
         | standards, and "A Hash Function That Is Secure Now But Maybe
         | Not Anymore In 10 Years" it's a bit too verbose in my opinion.
        
           | dolni wrote:
           | Change "Secure" for "FIPS" and you have a decent name. FHA-1
           | -- FIPS hash algorithm, version 1.
        
           | lapetitejort wrote:
           | The same concept applies to telescopes. Every year a
           | "Ridiculously Huge Telescope" gets announced. If I named them
           | I'd start small: "Decently Sized Telescope"; "Mildly Safe
           | Hashing Algorithm".
        
             | Dylan16807 wrote:
             | Huge Telescope 1. Huge Telescope 2. Huge Telescope 3.
             | 
             | If we do the same thing as the hash names, I think it's
             | fine. "SHA-1", specifically with the "1", doesn't cause
             | problems.
        
         | jagger27 wrote:
         | Cryptographers have a long history of doing this. Many of them
         | broken. DES and WEP are two that immediately spring to mind.
        
         | kzrdude wrote:
         | Fortunately it's still true for Fast Fourier Transform. Would
         | be fun to break that one.
        
         | woodruffw wrote:
         | Well, the safest part of any seatbelt is the part that goes
         | over the chest, not the seat :-)
         | 
         | Standards evolve, and SHA has grown new generations to replace
         | ones that have become insecure. See also "Secure Sockets
         | Layer."
        
         | icare_1er wrote:
         | Well, in that case, we should remove "Secure" from any protocol
         | name like HTTPS, FTPS, SSH and pretty much everything, because
         | by definition, whatever is secure at a given time probably wont
         | be anymore 10years later.
        
           | jodrellblank wrote:
           | No because those S's don't mandate any particular algorithm;
           | HTTPS has changed from SSL to TLS, and SSH often throws up
           | mismatched cipher or version errors when connecting to older
           | servers, so they could still be secure 10 years later.
           | 
           | I would however be fine with removing "unlimited" from the
           | name of services which are limited, 'Simple' from protocols
           | which don't need subjective comments about their complexity,
           | 'Ultimate' and 'Supreme' from the names of things which are
           | neither the last nor best, etc.
        
       | yrro wrote:
       | I wonder if Windows' Kerberos implementation will ever support
       | the aes128-cts-hmac-sha256-128/aes256-cts-hmac-sha384-192
       | encryption types. It's been stuck on aes128-cts-hmac-
       | sha1-96/aes256-cts-hmac-sha1-96 since Vista...
        
       | ComplexSystems wrote:
       | Any estimate on how long SHA 2 will last?
        
         | woodruffw wrote:
         | SHA-2 uses the same construction as SHA-1, but my understanding
         | is that there are no practical collision (much less preimage)
         | attacks against full-round SHA-2.
        
         | tptacek wrote:
         | Indefinitely. It's fine. The message expansion in SHA2 is
         | totally different from that of SHA1 (hash message expansion is
         | analogous to cipher key scheduling), and forecloses on the SHA1
         | attacks. JP Aumasson, part of the BLAKE team, has suggested
         | SHA2 might never be broken; that's hyperbolic but gives a sense
         | of how little research is on the horizon to threaten it.
         | 
         | The big issues with SHA2 are MD structure/length extension
         | (which HMAC addresses, and you can also use the truncated
         | versions; length extension matters pretty much exclusively if
         | you're designing entire new protocols) and speed.
         | 
         | I'd reach for Blake2 right now instead of SHA2 (or SHA3) in a
         | new design, but I wouldn't waste time replacing SHA2 in
         | anything that already exists, or put a lot of effort into
         | adding a dependency to a system that already had SHA2 just to
         | get Blake2.
        
           | miga wrote:
           | Then why does NIST warn that we might need PQC algorithms by
           | 2035?
           | 
           | Or is this claim ignoring progress of quantum computing?
        
             | luhn wrote:
             | Quantum computing threatens factorization and elliptic
             | curves, i.e. RSA and ECDSA. Hash functions are considered
             | relatively safe. The NIST's PQC standardization is focused
             | on public key cryptography--I can't find any initiatives
             | from them working with post-quantum hashing.
        
         | dchest wrote:
         | Here's one:
         | 
         | "neither us nor our children will see a SHA-256 collision (let
         | alone a SHA-512 collision)" -- JP Aumasson
         | https://twitter.com/veorq/status/652100309599264768
        
       ___________________________________________________________________
       (page generated 2022-12-15 23:00 UTC)