Hashing and Password Storage

Topics Covered

What Hash Functions Guarantee

The three properties

The avalanche effect

Which functions to use

Hashes are not encryption

Low entropy inputs

Integrity needs a trusted digest

Length extension

Output length and security level

Hashing is not a design on its own

Library choice

Hash functions are among the most widely used cryptographic tools in software, and among the most frequently misused. They appear in password storage, file integrity checks, content addressed storage, deduplication, digital signatures, message authentication and cache keys. Each use relies on specific properties, and choosing the wrong function, or relying on a property the function does not have, produces bugs that are invisible in testing and exploitable in production. This lesson explains what hashes guarantee, which functions to use for which job, and how to store passwords so that a database leak does not become an account takeover.

The three properties

A cryptographic hash function maps input of any length to a fixed length output, called a digest, and it is designed to have three properties. Preimage resistance means that given a digest, it is infeasible to find any input that produces it. Second preimage resistance means that given one input, it is infeasible to find a different input with the same digest. Collision resistance means it is infeasible to find any two different inputs with the same digest. Collision resistance is the hardest to achieve, and it is the property that has failed first for older functions.

Changing one character of the input flips about 128 of 256 output bits. Reversing a digest would take on the order of 2 to the 256 attempts.

The avalanche effect

A good hash function makes its output look random. Changing a single bit of input changes, on average, half of the output bits, so similar inputs produce completely unrelated digests. This property is what makes hashes useful for integrity checking: any change to a file, however small, produces a different digest. It also means that hashes cannot be used to measure similarity between inputs, a common misunderstanding when teams try to use them for fuzzy matching or deduplication of nearly identical documents.

Which functions to use

For general purpose hashing today, the SHA-2 family, particularly SHA-256 and SHA-512, and the SHA-3 family are standard choices. BLAKE2 and BLAKE3 are fast, well studied alternatives used in many modern tools. MD5 and SHA-1 are broken for collision resistance: practical collisions have been demonstrated for both, and they must not be used where an attacker could benefit from creating two inputs with the same digest, such as signatures or certificates. They remain acceptable for non security purposes like detecting accidental corruption, but using them there invites confusion, so most teams simply avoid them.

Hashes are not encryption

A hash is one way: there is no key and no way to recover the input from the digest except by guessing inputs and checking. Encryption is two way: with the key, the original data can be recovered. Teams sometimes describe hashed data as encrypted, or try to hash data they later need to read back, such as an email address needed for sending messages. When the data must be recovered, encryption is the right tool. When only comparison is needed, such as checking that a submitted value matches a stored one, a hash or keyed hash may be enough.

Low entropy inputs

Preimage resistance protects inputs that are hard to guess. It does not protect inputs drawn from a small or predictable space. Hashing a phone number, a national identifier or a date of birth gives little protection, because an attacker can hash every possible value and compare. A ten digit phone number has only ten billion possibilities, which a single graphics card can hash in seconds. This is why hashing personal data is not anonymisation, and why passwords, which people choose from a predictable space, need the special treatment described later in this lesson.

At 10 billion SHA-256 hashes a second on one GPU, every ten digit phone number takes about a second. A random 128 bit token is out of reach.

Integrity needs a trusted digest

A hash proves integrity only if the digest itself comes from a trusted source. Publishing a file and its digest on the same server protects against accidental corruption in transit, but an attacker who can change the file can usually change the digest too. Real integrity protection needs the digest delivered through a separate trusted channel, or signed with a key the verifier trusts, which is the subject of the lesson on signatures. Download pages that show a checksum next to the download link provide much less protection than they appear to.

Length extension

SHA-256 and SHA-512, like other functions built on the Merkle Damgard construction, are vulnerable to length extension. Given the digest of a secret followed by a message, an attacker who knows the length of the secret can compute the digest of the secret followed by the message plus additional data of their choosing, without knowing the secret. Systems that authenticate requests by computing a plain hash of a secret key concatenated with the message are therefore broken. The fix is to use HMAC, described in the next section, or a function such as SHA-3 or BLAKE2 that does not have this weakness.

Output length and security level

The length of a digest sets an upper bound on its strength. Because of the birthday paradox, finding a collision in an n bit hash takes roughly 2 to the power n over 2 attempts, so a 256 bit hash gives about 128 bits of collision resistance, which is far beyond any practical attack. A 128 bit hash gives only 64 bits of collision resistance, which is within reach of well funded attackers. This is one reason 256 bit outputs are the standard choice for new designs, and why shortening digests for convenience, discussed later in this lesson, needs care.

Hashing is not a design on its own

Hash functions are building blocks. Most real problems need a construction around them: a MAC for authentication, a signature for public verification, a password hashing function for low entropy secrets, or a keyed hash for pseudonyms. A useful habit in design review is to ask, whenever a plain hash appears, which of these constructions the design actually needs. In practice, a plain unkeyed hash is the right answer mainly for integrity checks against a trusted digest, content addressing and storing high entropy tokens. Almost every other use benefits from one of the constructions covered in the rest of this lesson, and reaching for them early avoids rework.

Library choice

Cryptographic hashes should come from the platform's standard library or a well maintained cryptography package, never from code copied from a blog or written by hand. Hand written implementations often contain subtle bugs, such as incorrect padding or byte order, that pass simple tests and fail in edge cases. Standard libraries are also optimised, often using hardware instructions, and they are updated when weaknesses are found. Pinning a specific, audited library across the organisation, and documenting which functions to call for which purpose, removes a whole category of decisions from individual engineers.

A short internal page listing the approved function for each purpose, with a code snippet in each of the organisation's main languages, answers most questions before they reach the security team and makes reviews faster, since reviewers can point to the page rather than explaining the reasoning each time.

The page should also say which functions are banned and why, so that an engineer who finds MD5 in old code knows whether it matters there and what to replace it with.

Use the property you actually need

Hashes give preimage, second preimage and collision resistance, and a single changed bit changes half the output. Use SHA-256, SHA-3 or BLAKE2 or BLAKE3, never MD5 or SHA-1 where collisions matter, remember that hashes are not encryption and give no protection to guessable inputs, and never build authentication from a plain hash of a secret and a message.