Key establishment, the adversary on the network, freshness, and challenge-response authentication

In August 2015, at DEF CON, a hacker conference held each year in Las Vegas, security researcher Samy Kamkar demonstrated a device he called RollJam. It cost about $32 in parts, and it opened cars and garage doors that used rolling codes.

A rolling code system rejects codes it has already accepted. Each press of a key fob sends a new code, computed from a counter and a secret shared with the car. The car accepts valid codes ahead of its current counter, within a limited window. Recording a code that successfully unlocks the car does not let a thief reuse it.

When the owner pressed the button, RollJam jammed the car’s receiver while recording the code, so the car did not respond. The owner pressed again. RollJam jammed and recorded the second code, then transmitted the first one, and the car unlocked. The attacker kept the second code, which remained valid until the car accepted a later code and advanced its counter past it.

The cipher wasn’t broken, and no secret key was stolen. The replayed code was authentic, and the car could not tell that it had been recorded earlier.

Hash functions, message authentication codes, and digital signatures protect data, but they do not decide when a message should be accepted. A security protocol is a sequence of messages exchanged between parties to achieve a goal, such as proving an identity or agreeing on a key. An adversary can interfere with that sequence by replaying messages, reordering them, or mixing messages from different exchanges. A protocol can fail even when its cryptographic operations work correctly.

Descriptions of security protocols traditionally give the parties names. Alice and Bob want to communicate, and Alice usually starts the exchange. Trent is a trusted third party that both rely on. Eve is an eavesdropper who can only read traffic. Mallory is a malicious party who can also modify, replay, or inject messages, either as an attacker on the network or as a dishonest participant. These notes use Alice, Bob, and Trent, and usually call the attacker the adversary.

Getting a Shared Key

Symmetric ciphers and message authentication codes (MACs) efficiently protect large amounts of data. The algorithms are fast but they need both parties to have the same key before protected communication can begin.

Giving every pair of parties its own key does not scale. A group of $n$ parties needs $n(n-1)/2$ keys, so 1,000 employees would need 499,500 keys in total. Adding one employee requires 1,000 more keys. A browser visiting an unfamiliar website also needs a way to begin without a pre-arranged shared secret.

Key establishment is the process of getting a shared secret key into the hands of the parties that need it, and only those parties. It’s sometimes called key exchange. There are four basic approaches:

A pre-shared key is distributed out of band, by some means other than the network it will protect. A Wi-Fi password printed on a card at a coffee shop is an example of this. It works for small groups, and changing it means telling everyone again.
A trusted third party shares a long-term key with every participant and generates a secret key for any pair that asks to communicate. This is how Kerberos, the authentication system in Windows domains, works.
Public key transport has one party generate a key and encrypt it with the other party’s public key.
Diffie-Hellman key agreement lets two parties compute a shared secret from values exchanged in the open. The exchange must be authenticated, using signatures or a pre-shared key, to resist an active attacker.

Long-Term Keys and Session Keys

Keys differ in how long they are meant to last. A long-term key identifies a party and stays in use for months or years. A password-derived key shared with an authentication server is one example, and so is the private key associated with the public key in a web server’s certificate.

A session key protects a single conversation and is discarded when the conversation ends. Protocols go to the trouble of creating session keys, rather than encrypting everything with long-term keys, for three reasons:

A leaked session key exposes only the traffic protected by that key. The consequences of losing a long-term key can extend across many conversations.
Limiting the data encrypted under each key helps systems stay within the cipher’s safe usage limits.
A session key can be erased when the conversation ends. Long-term keys must remain available for future connections until they are replaced.

The Adversary on the Network

A protocol has to be judged against what an attacker can do. The standard assumption treats the network itself as hostile. In this case, the adversary can:

Read and record every message.
Delete, delay, or reorder messages.
Replay a recorded message, either later in the same run or in an entirely different run.
Inject messages of its own, and initiate protocols with any party, including several runs at once.

The adversary may also be a legitimate participant, with its own keys, who uses them to attack other participants. This model was formalized in 1983 and is known as the Dolev-Yao model, after its authors.

The model treats cryptographic operations as secure. Without the required key, the adversary cannot decrypt a ciphertext or create a valid MAC or signature. This model isolates flaws in the message exchange rather than any weaknesses in the underlying cryptography. Password guessing and attacks on weak algorithms fall outside the model and need separate analysis. We will cover password guessing in detail later.

The Adversary in the Middle (AiTM)

One position on the network gives an attacker particular power. An attacker sitting between two parties can relay traffic between them, and each party believes it is talking directly to the other. The attacker reads everything that is not encrypted, changes anything that is not authenticated, and passes the rest along unaltered so that nothing looks wrong.

This is an adversary-in-the-middle (AiTM) attack. Older sources call it a man-in-the-middle (MITM) attack. The position can come from a compromised router, a malicious Wi-Fi access point, a poisoned DNS entry, or a phishing link that sends a user to the attacker’s server instead of the real one. Phishing uses a deceptive message, such as an email or text, to lure a person to a fake site or trick them into revealing a credential. We will cover it in detail later.

For example, an attacker can substitute values during an unauthenticated key exchange, leaving each party with a key shared with the attacker: Alice sets up a key to the attacker while the attacker sets up a key with the server Alice thinks she’s connecting to. This attacker site can also relay a login to the real site while capturing the user’s password.

Replay

A valid MAC or signature authenticates the protected bytes under the key used to verify it. An adversary who records the message can send it again at a future time, and the cryptographic check will still pass. The check alone does not establish when the message was produced or whether it has already been accepted.

For example, suppose a bank’s server accepts signed instructions from a customer’s app without checking for duplicates. An adversary records an instruction to transfer $500 to a landlord and submits ten extra copies. The server transfers an additional $5,000, and every copy passes signature verification.

A replay attack reuses a valid message outside the context in which it was sent. The defense is to give the receiver some way to tell a current (fresh) message from an old one.

Establishing Freshness

A message is fresh if the receiver has evidence that it was created for the current exchange or within an acceptable recent period. Protocols use three common mechanisms to check this:

Nonces. A nonce is a value intended for one use. It is often remembered as “number used once.” For the freshness checks described here, a nonce is generated randomly so that it is unpredictable.

For a freshness check, Alice chooses an unpredictable value and sends it to Bob. She accepts a reply only if it contains that value protected by a MAC or signature she can verify. An earlier reply contains a different nonce, so it fails. An attacker will not have the key to create a valid MAC or signature for a message containing that nonce. A securely generated 128-bit random nonce has a negligible chance of repeating during ordinary use.

The nonce must reach Bob before he can respond, and it can be included in a message the protocol already needs. Alice’s nonce gives Alice evidence of freshness: she sees a response that contains the value she recently generated. By itself, it does not give Bob any evidence that Alice’s messages are fresh since he doesn’t know when the nonce was created.

Timestamps. A sender can include the current time in a protected message, and the receiver accepts it only if the time is recent. Timestamps save a round trip, and they let a receiver judge freshness without having sent anything first.

Timestamps require clocks that agree within a chosen tolerance. A replay inside that tolerance window can succeed unless the receiver also remembers recent messages. An attacker who changes a machine’s clock by tampering with its time source may make old messages look current.

Sequence numbers. Each message has an increasing counter protected along with its contents. The receiver tracks accepted values and rejects duplicates. Transport Layer Security (TLS) uses record sequence numbers in its cryptographic protection, so replaying or reordering protected records within a connection causes verification to fail.

Sequence numbers require both sides to keep state and agree on what happens after a gap, a crash, or a restart. Car remotes allow gaps because pressing a fob out of range advances its counter without the car seeing it. The RollJam attack retained a code that was still ahead of the car’s counter.

Different Uses of Nonces

A nonce is a value intended for one use. Its purpose determines how it must be generated and whether it can be public.

The freshness nonce described above connects an authenticated response to a current request. It can be sent openly, but it must also be unpredictable. If an attacker can predict the next nonce, it can send that value to the legitimate party ahead of time and save the authenticated reply. When the receiver later sends a request with that nonce, the attacker can replay the saved reply without involving the legitimate party.

Encryption algorithms also use nonces. AES in Galois/Counter Mode (AES-GCM) requires a different nonce for each encryption under the same key. Here, a counter works: the nonce must not repeat, but it need not be unpredictable.

Signature algorithms have their own requirements. The Elliptic Curve Digital Signature Algorithm (ECDSA) uses a secret per-signature value, also called a nonce. It must be securely generated and never reused for different messages under the same key. Exposing or reusing it can reveal the signing key.

Challenge and Response

Freshness makes it possible to prove knowledge of a secret without revealing it. Suppose Alice and Bob share a key $K$, and Alice wants to confirm that she is talking to Bob. She sends a nonce $r$ as a challenge. Bob returns a response, such as $\operatorname{HMAC}_K(r)$. Alice computes the same value and compares.

The response demonstrates knowledge of $K$ without sending the key. A recorded response will not answer a future challenge since the nonce will be different.

This exchange is challenge-response authentication. By itself, it does not prevent an attacker from relaying a current challenge to the real key holder and forwarding the answer.

In this exchange, Alice authenticates Bob, but Bob does not authenticate Alice. This is one-way authentication. Mutual authentication gives both parties this assurance. One approach is for Bob to include his own nonce with his response and for Alice to answer it.

The Reflection Attack

Mutual authentication is easy to get wrong. Suppose Alice and Bob use the same key and the same computation in both directions. Alice sends a challenge $r_A$, and expects $\operatorname{HMAC}_K(r_A)$ back along with a challenge of Bob’s.

An attacker without the key can pose as Bob in five steps:

Alice sends the challenge $r_A$ to the attacker, believing it is Bob.
The attacker opens a second session with Alice and sends her $r_A$ as its own challenge.
Alice, following the protocol, returns $\operatorname{HMAC}_K(r_A)$ in the second session.
The attacker sends that response back to Alice in the first session, along with a new challenge of its own.
Alice verifies it and accepts the attacker as Bob.

Alice has answered her own challenge, allowing the attacker to pass the first session’s authentication check. This is a reflection attack.

The fix is to make the two directions distinguishable. The response can include the responder’s identity, as in $\operatorname{HMAC}_K(\text{"Bob"}, r_A)$, or each direction can use a different key. Alice then refuses to produce anything that could serve as an answer in the other direction.

Protecting the Conversation After Authentication

Proving identity at the start of a conversation does not protect later messages. If those messages are unprotected, an adversary in the middle can let authentication finish and then inject commands in the legitimate party’s place. Taking over an authenticated session is session hijacking.

A secure channel binds authentication to keys that protect subsequent messages. An authenticated key exchange establishes shared keys and authenticates one or both participants. Authentication can also run inside an existing protected channel, as with a password login over TLS. In either case, the application must preserve the connection between the authenticated identity and later requests.

Next: Part 2: Key Distribution with a Trusted Third Party

Part 1 - Secure Communication