pk.org: Computer Security/Lecture Notes

Authentication

Paul Krzyzanowski – September 28, 2025

Authentication is one of the core functions in computer security. It sits alongside two related concepts:

A login sequence typically covers all three. Here, we focus on authentication, especially protocols that both prove identity and establish fresh keys for secure communication. We'll cover authorization when we look at access control.


Proving Knowledge of a Secret

The essence of cryptographic authentication is to prove you know a secret without revealing it. A standard mechanism is challenge–response.

Suppose Alice and Bob share a long-term secret key K. This is a pre-shared key. Alice sends Bob a random value, a nonce. A nonce is a number used once. It must be unpredictable and never reused. Bob returns the nonce processed with K (for example, EK(r1) if using symmetric encryption, or a HMAC(K, r1) if using a keyed MAC over the nonce).

If Alice verifies the response, she is convinced that Bob also knows K. If both parties challenge each other, the result is mutual authentication.


Authentication and Key Exchange

Authentication is usually coupled with the creation of a fresh session key. We want to minimize the amount of data we encrypt with long-term keys. A common pattern uses a trusted third party that shares a long-term key with each participant.

High-level idea:

  1. Alice asks the trusted server for a key to talk to Bob.

  2. The server generates a session key KAB.

  3. The server returns KAB encrypted with Alice’s key, and a message containing KAB encrypted with Bob’s key. This message is called a ticket. Alice cannot read Bob’s ticket but can forward it to him.

Why replay attacks are a problem

If an attacker can replay an old ticket or an old message that contains a session key, Bob may accept a stale key as if it were new. Two different risks arise:

A sound protocol must bind tickets to the current run and prove freshness, not just possession of a key.


Security Protocol Notation

When describing authentication and key exchange protocols, we use a compact notation to show who sends what to whom and how messages are protected.

This notation is not programming code. It is a shorthand that makes it easier to write down message sequences, highlight which party can read or create each part, and focus on what the protocol is trying to achieve.


Evolution of Symmetric Protocols

Before diving into the well-known protocols, it helps to start with a very simple design. Suppose Alice wants to talk securely with Bob and relies on Trent, a trusted third party, to create a session key for them. Trent knows a long-term key shared with each user and can generate fresh session keys.

A very simple protocol

  1. Alice tells Trent she wants to talk to Bob.

  2. Trent generates a fresh session key KAB. He encrypts one copy with Alice’s long-term key KA and another copy with Bob’s long-term key KB. The second copy is called a ticket because Alice cannot read it but can forward it. Trent sends both encrypted parts to Alice.

  3. Alice decrypts her part to learn KAB. She forwards Bob’s ticket to him.

  4. Bob decrypts his ticket to learn KAB. Now Alice and Bob can communicate securely with the shared session key.

This works in principle, but it has a fatal flaw: there is no proof of freshness. If an attacker records an old ticket, she can replay it later. Bob will accept it as if it were new. Two bad outcomes follow:

The key idea behind later protocols is to add explicit checks for freshness.


Needham–Schroeder (symmetric key version)

Needham–Schroeder builds directly on this simple design but adds nonces to prove that the session key is new. A nonce is a random number used once. By including nonces in the messages, Alice and Bob can confirm that the key was freshly created for this session and not replayed from the past.

Like the simple version, this protocol allows Alice and Bob, who each share a long-term secret only with Trent, to agree on a new session key KAB.

Steps

  1. A → T: A, B, r1
    Alice requests a session with Bob and includes a nonce r1. She expects Trent to return this nonce in the reply so she can confirm that the message is not a replay.

  2. T → A: { A, B, r1, KAB, { A, KAB } KB } KA
    Trent creates a new session key KAB. He sends Alice her copy encrypted with her long-term key, and also provides a ticket for Bob—{ A, KAB } KB. The inclusion of r1 shows Alice that this message is a reply to her request and not an old message.

  3. A → B: { A, KAB } KB
    Alice forwards Bob’s ticket. Only Bob, using his key KB, can decrypt it and recover KAB.

  4. B → A: { r2 } KAB
    Bob now knows KAB, but he wants assurance that Alice is also active and holds the same key. He sends Alice a fresh nonce r2 encrypted with KAB.

  5. A → B: { r2 + 1 } KAB
    Alice responds with a simple transformation of r2 to prove she holds the session key.

Weakness: If an attacker later learns an old session key, they can replay step 3 (the ticket) and correctly answer Bob’s challenge. Bob has no way to tell the ticket is stale.


Denning–Sacco (timestamp modification)

Needham–Schroeder relied on nonces to prove freshness, but this did not protect against replay of old tickets. If an attacker recorded a ticket and later learned its session key, they could impersonate Alice by replaying it to Bob. Denning and Sacco proposed a modification: use timestamps instead of nonces. If each ticket includes the time it was issued, Bob can verify that it is recent and reject any replayed tickets. Denning-Sacco trades nonces for timestamps.

Steps

  1. A → T: A, B
    Alice requests a session with Bob.

  2. T → A: { A, B, KAB, { A, KAB, t } KB } KA
    Trent generates a new session key KAB and includes a timestamp t. He gives Alice her copy encrypted with KA and creates a ticket for Bob containing (A, KAB, T), encrypted with Bob’s long-term key.

  3. A → B: { A, KAB, t } KB
    Alice forwards the ticket to Bob.

  4. B → A: { t + 1 } KAB
    Bob decrypts the ticket, checks that the timestamp is recent, and replies with t+1 encrypted with the session key to confirm he holds the same key.

Strength: Because the timestamp is inside the ticket, Bob can reject replayed tickets directly. An attacker cannot reuse an old ticket to impersonate Alice.

Trade-off: All parties must have synchronized clocks, which is not always easy in practice.


Otway–Rees Protocol

Otway and Rees proposed another refinement. They wanted to prevent replay without requiring synchronized clocks. Their idea was to attach a unique session identifier to every message in the protocol run. The identifier ties all the exchanges together.

Steps

  1. A → B: n, A, B, { n, A, B, r1 } KA
    Alice begins a new session with identifier n. She also includes her own nonce r1 encrypted with her key, so she can confirm freshness later.

  2. B → T: n, A, B, { n, A, B, r1 } KA, { n, A, B, r2 } KB
    Bob forwards Alice’s message to Trent. He adds his own contribution: a nonce r2 encrypted with his key. Both parts carry the same session identifier n.

  3. T → B: n, { r1, KAB } KA, { r2, KAB } KB
    Trent generates KAB and returns replies for both Alice and Bob. Each reply includes the corresponding nonce to show it belongs to this run. The session identifier n ties the responses to this particular session.

  4. B → A: n, { r1, KAB } KA
    Bob forwards Alice her reply. Alice decrypts and confirms that her nonce r1 and the session identifier n match, proving that this message belongs to her current session and is fresh.

Strength: The session identifier prevents messages from being replayed across different sessions. Unlike Denning–Sacco, the protocol does not rely on clocks.


These three protocols illustrate the evolution of thinking about freshness in authentication:

  1. Needham–Schroeder introduced the use of a trusted third party but failed against replays.

  2. Denning–Sacco used timestamps to block replays but required clocks.

  3. Otway–Rees achieved freshness without clocks by embedding a unique identifier into every message.

Kerberos

Kerberos, developed at MIT in the 1980s, refined these ideas into a practical system. It uses tickets, session keys, and timestamps to resist replays. Passwords are never sent on the network. Instead, once a user proves their identity, Kerberos issues short-lived session keys bound to tickets.

Kerberos splits Trent’s role into two parts:

This separation solves a practical problem. If there were only one server, Alice would have to use her password every time she wanted to access a service. That would expose her password to repeated attacks and increase the chances of compromise. By separating the AS from the TGS, Kerberos limits password use to the initial login. After that, Alice uses the TGS ticket to request additional service tickets without retyping her password. This design is what enables Kerberos to support single sign-on securely.

Steps in Kerberos

  1. AS exchange

    • A → AS: A, { t } KA
      Alice identifies herself and proves she knows her password-derived key by sending a timestamp t encrypted with KA. The AS checks freshness and that Alice possesses KA, which prevents offline password guessing.

    • AS → A: { TGS ticket, KA,TGS, t + 1 } KA
      The AS returns a session key KA,TGS for Alice ↔ TGS and a TGS ticket (opaque to Alice). Echoing t + 1 shows the AS processed a fresh request. The TGS ticket contains Alice’s identity, KA,TGS, and timestamp t, encrypted with the TGS’s key.

  2. TGS exchange

    • A → TGS: { TGS ticket }, { A, B, t } KA,TGS
      Alice asks for a ticket to service B, presenting the TGS ticket and a fresh timestamp encrypted with KA,TGS.

    • TGS → A: { { A, B, KAB, t } KB , KAB } KA,TGS
      The TGS returns a new session key KAB for Alice ↔ B, along with a service ticket for B that contains (A, B, KAB, t) encrypted with B’s long-term key.

  3. Service exchange

    • A → B: { service ticket }, { A, t } KAB
      Alice proves freshness to B by sending her identity and t encrypted with KAB.

    • B → A: { t + 1 } KAB
      B checks the ticket and timestamp, then confirms by returning t + 1, proving B holds KAB.

Properties of Kerberos


Public Key Authentication

The symmetric protocols we studied all required a trusted third party, Trent, to create and distribute session keys. Public key cryptography offers a different model. If Alice knows Bob’s public key, she can generate a session key and send it encrypted with that key. Only Bob, who holds the matching private key, can decrypt it. Bob can also challenge Alice to prove that she holds her private key. This avoids the need for Trent to know or generate the session key.

The challenge is verifying that a public key really belongs to the party it claims to represent. Certificates solve this problem by binding a public key to an identity, and we have already seen how X.509 certificates and certificate authorities provide this trust infrastructure.

In TLS, which we studied earlier, this model is combined with an ephemeral key exchange that produces fresh session keys for each connection. This not only authenticates the server but also ensures forward secrecy: even if the server’s private key is stolen in the future, past sessions remain secure.


Password-Based Protocols

While cryptographic protocols are powerful, many real systems have relied on password-based authentication. These protocols are simple but vary in security.

PAP (Password Authentication Protocol)

The simplest password protocol is PAP. The client sends a username and password to the server; the server checks the credentials against its stored records. PAP is simple but insecure:

PAP is rarely used in production except as a fallback or for debugging. Because PAP exposes the password on the wire, it is especially vulnerable when users reuse passwords across sites or when attackers employ automated credential checks. Always avoid sending plaintext passwords unless the transport is strongly protected.

CHAP (Challenge–Handshake Authentication Protocol)

CHAP improves on PAP by using a challenge–response mechanism. The server sends Alice a fresh random value, a nonce. Alice replies with a hash of that nonce combined with her secret password. The server checks whether the response matches what it can compute.

The password itself never travels on the network. Because the challenge is new each time, replaying an old response is useless. CHAP protects against simple sniffing attacks but still relies on the security of the password and does not prevent offline attacks against a stolen password file.

A note on MS-CHAP: Microsoft developed MS-CHAP and MS-CHAPv2 and these variants were widely deployed historically (for PPP, PPTP VPNs, and inside EAP). However, MS-CHAPv2 has well-known weaknesses, and Microsoft guidance recommends encapsulating it (for example, in PEAP) or replacing it with stronger methods such as EAP-TLS.

Note: This looks similar to challenge-based OTPs, which we discuss later, but there is a key difference. In CHAP, the secret is a static password reused across logins. In challenge-based OTPs, the secret is stored in a device that generates one-time responses, so each login proves possession of the device rather than knowledge of a password. Algorithmically, they operate the same.


Weaknesses of Passwords

Passwords are widely used but deeply flawed. Users often pick weak passwords, reuse them across sites, and forget them. Attackers can guess weak passwords directly against a login service. More dangerously, if they obtain a password file from a compromised server, they can attempt unlimited guesses offline.

How passwords are stored

Most systems do not store passwords in plaintext. If they did, anyone with access to the file -- a rogue administrator, a compromised account, or an attacker who stole the file -- would instantly know every user’s password. Because many people reuse the same password across sites, a plaintext breach in one system could lead to compromises of accounts on other services.

Instead, systems store only a password hash, computed with a one-way function:

Hashing prevents an immediate disclosure of all passwords if the database leaks. However, a stolen set of hashes still enables an attacker to attempt offline guessing: the attacker can compute H(guess) locally and compare it to stored hashes without contacting the server.

Online vs. offline guessing

There are two very different ways attackers can attempt password guessing:

Dictionary and rainbow table attacks

Attackers rarely try passwords at random. Instead, they use smarter strategies:

Salts and password hashing

A salt prevents precomputation attacks. A salt is a short random string generated by the system and stored alongside each password hash. Instead of storing hash(password), the system stores hash(salt, password). The same password chosen by two users will produce different hashes because their salts differ.

With salt, attackers cannot use a single rainbow table for all users; they would need to build a separate table for every possible salt, which is impractical.

For offline guessing, salts force the attacker to recompute candidate hashes per account; the attacker cannot reuse precomputed tables across accounts.

Linux example:
In Linux, hashed passwords are stored in the file /etc/shadow. A salted SHA-512 hash might look like:
$6$mysalt123$uw7/eKvgmWOAR...
Here $6$ indicates SHA-512, mysalt123 is the salt, and the rest is the hash. It isn't feasible for an attacker to precompute a table of hashes of all likely passwords, each with every possible random salt string.

Slow hashing functions

Salts make rainbow tables useless, but attackers can still run dictionary attacks by hashing guesses on the fly. To slow them down, some systems use deliberately expensive hash functions:

The important point is that these functions can be configured to remain usable for legitimate logins while raising the cost of brute-force attacks by orders of magnitude.

Large-scale online attacks: credential stuffing and password spraying

Even without stolen hash files, attackers exploit human reuse and predictable choices at scale. Two common large-scale threats on password-based systems are credential stuffing and password spraying attacks:

Mitigations against password attacks

To defend both online and offline attacks, combine several layers:

To defend both online and offline attacks, combine several layers:


One-Time Passwords

Static passwords can be replayed. One-time password systems generate a new password for each login. The idea is to never reuse a password.

There are four main forms of one-time passwords:

  1. Sequence-based: the password is derived from a sequence number or a previous password.

  2. Challenge-based: the password is derived from a random challenge presented by the server and a shared secret.

  3. Counter-based: the password is derived from a shared counter value and a secret.

  4. Time-based: the password is derived from the current time slice and a secret.

Sequence-based OTPs: S/Key and OPIE

The idea is to let a user prove identity multiple times without ever reusing the same password. This is done by creating a sequence of values with a one-way function. A one-way function (like a cryptographic hash) is easy to compute but infeasible to reverse. If you see the output, you cannot get back to the input.

For example, suppose we start with a random seed R. We apply a hash function repeatedly:

This gives us a chain of values, each derived by hashing the previous one. Because hashing is one-way, knowing x3 does not let you compute x2 or x1.

In S/Key, the server stores the end of the chain, xN. The user keeps the earlier values. At login:

Each login consumes one password. Even if an attacker sees xN−1 on the network, they cannot compute xN−2 because the hash cannot be reversed.

OPIE (One-time Passwords In Everything) was a practical implementation of this idea for UNIX logins and remote shell access. It let users avoid sending static passwords over the network, which could be captured by eavesdroppers. ß

Challenge-based OTPs

In a challenge-based OTP system, the server generates a fresh random challenge for each login attempt. The user’s device — often a hardware token or an authenticator app — computes a one-time password as:

response = f(secret, challenge)

where secret is shared between the device and the server. The user enters the response, and the server checks it. Because the challenge is never reused, the response is valid only once and replay attacks fail.

This may look very similar to CHAP, which we covered earlier. Both involve the server sending a challenge and the client responding with a function of that challenge and a shared secret. The difference is in what the secret is and how the system is used:

Both rely on the same underlying concept — a shared secret and a one-way function applied to a random challenge — but CHAP is still password-based authentication, while challenge-based OTPs are a form of one-time password system and represent a possession factor in multi-factor authentication.

Challenge-based OTPs are less common today than other mechanisms, but are still used in some hardware tokens.

Counter-based: HOTP

HOTP stands for HMAC-based One-Time Password. Both client and server share a secret key K and maintain a counter C.

This provides portable one-time codes but requires the client and server counters to stay in sync.

Time-based: TOTP

TOTP is Time-based One-Time Password. It replaces the counter with time:

TOTP is the most widely used OTP method today. Apps like Google Authenticator and Duo implement it.


Multi-Factor Authentication (MFA)

Multi-factor authentication uses independent categories of evidence:

To count as MFA, the factors must come from different categories. A password plus a PIN is not MFA because both are knowledge factors. A password plus a TOTP code, or a password plus a fingerprint, is MFA.

One-time passwords (OTPs) are the most common “something you have” factor. The systems we discussed earlier -- S/Key, HOTP, and TOTP -- are all used in MFA deployments, with TOTP apps such as Google Authenticator being the most widespread today.

Push notifications are another common factor. When you log in, a prompt appears on your phone asking whether you approve the attempt. This is convenient but vulnerable to MFA fatigue: attackers send repeated requests until the user, annoyed, accepts one.

Number matching strengthens push-based MFA. Instead of tapping “approve,” the user must type the number shown on the login screen into the app. This ties the approval to the specific login attempt and defeats fatigue attacks.


Passkeys

Passwords fail for three main reasons:

  1. People reuse them across sites.

  2. Phishing tricks people into typing them into fake websites.

  3. Even strong passwords can be stolen from databases.

MFA improves security but still relies on passwords as one factor, and phishing can bypass some second factors. Passkeys were designed to eliminate passwords altogether.

Passkeys are based on public key cryptography. For each service, a device generates a public/private key pair.

Passkeys are unique per site, cannot be phished, and cannot be reused elsewhere. They can be synced across a user’s devices to make them portable.


Adversary-in-the-Middle Attacks

An adversary-in-the-middle (AitM) (also referred to as man-in-the-middle or MitM) attack places an attacker between client and server. The attacker intercepts and relays all messages. This lets them capture passwords, OTPs, or even forward a valid login in real time.

Defenses:

These defenses do not make AitM impossible but raise the cost and limit the damage of such attacks.


Summary

Symmetric protocols evolved by adding nonces (Needham–Schroeder), timestamps (Denning–Sacco), or session IDs (Otway–Rees). Kerberos industrializes this with tickets and timestamped authenticators. Public key systems, used in TLS, add forward secrecy via ephemeral Diffie–Hellman. Passwords remain weak but are mitigated with salts, slow hashing, OTPs, MFA, and passkeys. Robust design requires replay protection, authenticated channels, and defenses against adversary-in-the-middle attacks.