pk.org: Computer Security/Lecture Notes

Secure Communication

Virtual Private Networks (VPNs)

Paul Krzyzanowski – 2025-11-15

When we connect to services via the Internet, our data travels through multiple routers and networks we don't control. At each hop, someone could potentially intercept, read, or modify our communications. The fundamental protocols that make the Internet work (IP, TCP, UDP, and even the routing protocols themselves) were designed without security in mind. They have no built-in authentication, no integrity checks, and no encryption.

This creates a challenging environment. How do we protect our data as it travels across untrusted networks? How do we prevent unauthorized access to our internal systems? And in a world where the boundaries between "internal" and "external" are increasingly blurred, how do we maintain security at all?

We'll explore three major approaches to these problems: secure communication protocols that protect data in transit, firewalls that control access between networks, and zero-trust architectures that challenge our traditional assumptions about network security. In this part, we will look at protecting data in transit.

The Problem with the Network Layer

The Internet's core protocols have fundamental security vulnerabilities:

On top of these network-layer issues, protocols such as ARP, DHCP, and DNS can be spoofed, allowing attackers to redirect traffic or position themselves as a man-in-the-middle.

We need security mechanisms that work despite these vulnerabilities: protocols that provide confidentiality, integrity, and authentication even when the underlying network is completely untrusted.

Transport Layer Security (TLS)

We've already covered the cryptographic details of TLS, so let's focus on what it provides and how it fits into the security landscape.

What TLS Gives Us

TLS creates a secure channel between two applications. After the TLS handshake completes, the applications can communicate as if they're using regular TCP sockets, but with three critical guarantees:

  1. Authentication: The client can verify it's talking to the correct server (and optionally, the server can verify the client's identity)

  2. Confidentiality: The communication is encrypted, preventing eavesdropping

  3. Integrity: Any tampering with the data will be detected

How TLS Works (Brief Review)

The TLS 1.3 handshake accomplishes several things simultaneously:

  1. The client and server agree on cryptographic algorithms (cipher suites)

  2. They perform a Diffie-Hellman key exchange to establish a shared secret

  3. The server proves its identity using a certificate signed by a trusted Certificate Authority

  4. Both sides derive encryption keys from the shared secret

Once the handshake is complete, all data is encrypted using authenticated encryption with associated data (AEAD), typically AES-GCM or ChaCha20-Poly1305.

AEAD: Quick Reminder

AEAD (Authenticated Encryption with Associated Data) combines encryption and authentication in a single operation. Think of it as generating a keyed MAC concurrently with encryption, but more efficient than doing encrypt-then-MAC separately. The "associated data" part means you can also authenticate some metadata (like packet headers or sequence numbers) without encrypting it. This is why TLS 1.3 doesn't need a separate HMAC step like earlier versions did.

Client Authentication: Why It's Rarely Used

TLS supports mutual authentication where both the client and server can present certificates. However, client certificates are almost never used in practice. Why?

Instead, we typically authenticate clients after establishing the secure TLS connection, using passwords, multi-factor authentication, or other mechanisms. The key insight is that TLS protects the confidentiality of these credentials. Even simple passwords are safe from eavesdropping over TLS.

Benefits and Limitations of TLS

TLS solves many problems:

But it has limitations:

Virtual Private Networks (VPNs)

While TLS provides secure communication between two applications, VPNs provide secure communication between entire networks or between a host and a network.

The Concept of Tunneling

Before we can understand VPNs, we need to understand tunneling. The ideal solution for connecting geographically separated networks would be dedicated private lines, physically separate from the public Internet. Companies like Amazon, Google, and Meta have actually deployed private networks connecting their data centers around the world. But for most organizations, this approach is prohibitively expensive and impractical.

Tunneling provides a virtual alternative. The basic idea is packet encapsulation: take an entire IP datagram and treat it as the payload of another IP packet. Consider two private networks that need to communicate. Network A uses the address range 192.168.1.0/24, and Network B uses 192.168.2.0/24. These are private addresses that cannot be routed on the public Internet.

When a computer at 192.168.1.11 on Network A wants to send a packet to 192.168.2.22 on Network B, the original packet has:

The gateway router on Network A intercepts this packet and encapsulates it inside a new IP packet with public addresses:

This outer packet travels across the public Internet to Network B's gateway router, which strips off the outer headers and extracts the original packet. The gateway then routes the original packet to 192.168.2.22 on its internal network.

From the perspective of the communicating computers, they appear to be on the same private network. The encapsulation is invisible to them. This is the essence of tunneling: creating a virtual point-to-point link between two networks using the public Internet as the transport.

However, basic tunneling does not provide security. The encapsulated data travels in cleartext, so anyone who intercepts the outer packet can read the inner packet. The tunnel endpoints are not authenticated, so an attacker could potentially impersonate a gateway. And there is no integrity protection, so the encapsulated data could be modified in transit.

What Makes It a VPN

A Virtual Private Network builds on tunneling by adding critical security features:

  1. Encryption: The encapsulated data is encrypted so eavesdroppers can't read it. Even if an attacker captures packets traversing the public Internet, they see only encrypted ciphertext.

  2. Integrity: Message authentication codes (MACs) ensure that outsiders cannot modify the data without detection. If an attacker tampers with a packet, the receiving gateway will detect the modification and discard the packet.

  3. Authentication: Authentication mechanisms ensure that you are connected to the legitimate gateway, not an imposter. This typically involves pre-shared keys or public key cryptography (often with X.509 certificates.

VPN = tunnel + encryption + integrity + authentication

This combination creates a secure channel through an untrusted network, allowing private networks to communicate as if they were directly connected.

VPN Deployment Models

Before looking at specific VPN technologies, let's understand the three main ways VPNs are used:

1. Site-to-Site VPNs (Network-to-Network)

This was the original use case for VPNs: connecting geographically separated corporate networks together. Two gateway routers establish a VPN tunnel between them, making multiple remote networks appear as one unified private network.

This model avoids the expense of dedicated private lines while maintaining security. Branch offices can communicate as if they were on the same local network, with all inter-site traffic encrypted and authenticated as it crosses the Internet.

For example, a company might have offices in New York and London. The New York office uses the 10.1.0.0/16 address range, and the London office uses 10.2.0.0/16. Gateway routers at each location establish a VPN tunnel between them. When a computer in New York (10.1.5.20) needs to access a file server in London (10.2.3.15), its packets are automatically encapsulated and encrypted by the New York gateway, sent across the Internet to the London gateway, decrypted and de-encapsulated, and delivered to the file server.

2. Remote Access VPNs (Host-to-Network)

As broadband Internet and remote work became common, VPNs evolved to support individual computers connecting to corporate networks. An employee's laptop runs VPN client software that establishes a secure tunnel to the corporate VPN gateway. Once connected, the laptop can access internal resources as if it were physically in the office.

This deployment model protects remote workers in several ways. Traffic between the remote computer and the corporate network is encrypted, protecting it from eavesdropping on untrusted networks like coffee shop WiFi. The VPN gateway authenticates the remote user before granting access. And the corporate network's firewall can apply the same security policies to remote users as it does to on-premises users.

This is probably the most common corporate use of VPNs today, enabling secure remote work without exposing internal services directly to the Internet.

3. Privacy VPNs (Host-to-Provider)

A third model has become popular with consumer VPN services like ExpressVPN, ProtonVPN, NordVPN, and others. These services allow individual users to establish VPN connections to the provider's network, which then acts as a gateway to the broader Internet.

In this model, the user's traffic is encrypted between their device and the VPN provider. The provider then forwards the traffic to its destination, making it appear to originate from the provider's network rather than the user's actual location.

This serves several purposes:

However, this model has critical trust implications. The VPN provider can see all your traffic: every website you visit, every service you use. You're not eliminating trust; you're shifting it from your ISP to the VPN provider.

Some items users should consider with privacy VPNs:

The dramatic growth of privacy VPNs reflects real concerns about surveillance and content restrictions. When the UK began enforcing age verification rules for explicit content, ProtonVPN reported a 1,400% hourly increase in VPN signups from the UK. These services fill a genuine need, but users should understand both their benefits and limitations.

VPN Protocols

Now let's look at the major VPN protocols. We'll examine them roughly in chronological order, which also shows the evolution of VPN technology.

1. IPsec

IPsec was developed in the 1990s as a standardized way to provide security for IP traffic. It operates at Layer 3 (the network layer) and is typically implemented directly in the operating system kernel.

Unlike TLS, which runs over TCP, IPsec is a separate protocol at the same layer as TCP and UDP. When you look at an IP packet header, the protocol field identifies what comes next: 6 for TCP, 17 for UDP, 50 for IPsec ESP (Encapsulating Security Payload), and 51 for IPsec AH (Authentication Header). This means IPsec can protect any higher-layer protocol, including TCP, UDP, ICMP, and others.

IPsec Protocols

IPsec consists of two security protocols:

  1. The Authentication Header (AH) protocol provides authentication and integrity protection for IP packets.

    It computes a message authentication code over the entire IP datagram (excluding fields that change in transit, like TTL and fragmentation information). AH protects against tampering, forged addresses, and replay attacks (by including a sequence number in the authenticated header). However, AH does not provide confidentiality; the payload remains unencrypted.

  2. The Encapsulating Security Payload (ESP) protocol provides everything AH does, plus encryption of the payload.

    ESP is what most people think of when they think of IPsec VPNs. It encrypts the packet contents, authenticates the packet (including the IP header in tunnel mode), and protects against replay attacks.

You might wonder why AH exists if ESP does everything AH does and more. This is a historical artifact. IPsec was developed in the 1990s when encryption was computationally expensive and subject to export restrictions in some countries. AH provided integrity without the overhead of encryption. Today, there is little (or no) reason to use AH alone; ESP with its built-in authentication is the standard choice.

IPsec modes

IPsec operates in two modes that correspond to different deployment models:

Tunnel mode is used for site-to-site and remote access VPNs. The entire original IP packet (headers and payload) is encapsulated as the payload of a new IP packet. The outer packet has the addresses of the VPN gateways, while the inner packet retains the original source and destination addresses. This allows private addresses to be used on the internal networks.

Transport mode is used for host-to-host communication when both endpoints implement IPsec. The original IP header is preserved, but the payload is protected. This mode is less common because it requires IPsec implementation on both communicating hosts and does not support address translation.

Key Exchange: IKE

IPsec uses the Internet Key Exchange (IKE) protocol to negotiate security parameters and establish keys. IKE handles authentication between peers (using certificates or pre-shared keys), negotiation of encryption algorithms and parameters, generation of session keys using Diffie-Hellman key exchange, and periodic rekeying to maintain forward secrecy.

The original IKE protocol (IKEv1) was complex, with multiple exchange modes and significant implementation variability between vendors. IKEv2, standardized in 2005 and updated in 2014, simplified the protocol considerably. IKEv2 requires fewer message exchanges, has built-in NAT traversal support, and includes MOBIKE (Mobility and Multihoming Protocol) for maintaining VPN connections when IP addresses change, which is particularly useful for mobile devices.

IPsec Cryptographic Components

IPsec supports various cryptographic algorithms for different functions:

IPsec's advantages include:

Its disadvantages:

2. OpenVPN

OpenVPN emerged in the early 2000s as the first widely-adopted open-source VPN solution. It took a fundamentally different approach from IPsec: instead of implementing a new network-layer protocol, OpenVPN runs as a user-space application and leverages TLS for its security.

Architecture

OpenVPN creates a virtual network interface (typically called a TUN device for IP tunneling) that the operating system treats as a regular network interface. When an application sends traffic to a destination that should go through the VPN, the operating system routes that traffic to the virtual interface. The OpenVPN process reads from this interface, encrypts the traffic, and sends it as regular UDP or TCP packets to the remote OpenVPN server.

This design means OpenVPN runs entirely in user space rather than in the kernel. Traffic must cross the boundary between kernel and user space twice (once when received by the virtual interface, once when sent over the real network), which adds some overhead compared to kernel-space implementations.

Control and Data Channels

OpenVPN separates its communication into two logical channels:

The control channel handles initial connection setup, authentication, and key exchange. It uses TLS, the same protocol that secures web browsers. This gives OpenVPN access to TLS's mature (and well-supported) ecosystem: X.509 certificates, certificate authorities, established libraries like OpenSSL, and well-understood security properties.

The data channel carries the actual tunneled traffic. It uses symmetric encryption (typically AES or ChaCha20) and HMAC for integrity protection, with keys derived from the TLS handshake.

This separation allows OpenVPN to renegotiate keys periodically without interrupting the data flow, providing perfect forward secrecy for long-running connections.

Authentication Options

OpenVPN supports two authentication approaches:

Certificate-based authentication (the more common approach) uses TLS with X.509 certificates. Both the client and server present certificates, providing mutual authentication. This is similar to how mutual TLS works, though in practice, many deployments only require server certificates and authenticate clients through other means (like using a username/password over the encrypted channel).

Pre-shared key authentication uses static keys configured on both endpoints. This is simpler to set up but provides weaker security: the same keys encrypt all traffic, and compromising the key compromises all past and future communications.

Transport Options

OpenVPN can run over either UDP or TCP at the transport layer:

UDP is generally preferred for VPN traffic. VPNs already carry higher-layer protocols (including TCP) inside the tunnel, so using UDP for the outer transport avoids the problem of "TCP over TCP." When the inner TCP connection experiences packet loss, it triggers its own retransmission. If the outer transport is also TCP, packet loss triggers retransmission at both layers, creating performance problems.

TCP is useful when UDP is blocked by firewalls. Many restrictive networks allow outbound TCP connections (especially on port 443, which is used for HTTPS) but block UDP. Running OpenVPN over TCP on port 443 makes VPN traffic look like ordinary HTTPS traffic to simple firewalls, though deep packet inspection can still distinguish them.

Advantages and Disadvantages

OpenVPN's advantages include:

Its main disadvantage is performance. Running in user space with separate control and data channels adds overhead compared to kernel-space implementations. For most use cases, this overhead is acceptable, but high-throughput scenarios may notice the difference.

3. WireGuard

WireGuard is a modern VPN protocol first released in 2016, designed with a philosophy of radical simplicity. Its creator, Jason Donenfeld, observed that existing VPN protocols had accumulated complexity over decades and set out to create something dramatically simpler without sacrificing security.

Design Philosophy

The entire WireGuard codebase is approximately 4,000 lines of code, compared to hundreds of thousands of lines for IPsec implementations or OpenVPN. This small size makes the code auditable and has allowed formal verification of its cryptographic properties. The protocol has been included in the Linux kernel since version 5.6 (March 2020), giving it the performance benefits of kernel-space implementation.

WireGuard makes opinionated choices rather than offering configuration options. There is no cipher suite negotiation; WireGuard uses exactly one set of modern cryptographic primitives. This eliminates an entire class of vulnerabilities related to negotiation and downgrade attacks.

Cryptographic Choices

WireGuard uses:

These algorithms were chosen because they are modern, well-analyzed, fast in software, and resistant to timing attacks. WireGuard does not support older algorithms, even for compatibility. The reasoning is that as cryptographic understanding advances, the protocol can be versioned rather than negotiated.

Key Management

WireGuard takes an unusual approach to identity and key management. Each peer is identified by its public key, not by a certificate. To configure a WireGuard interface, you generate a key pair, share public keys with peers you want to communicate with, and specify which IP addresses should be routed to each peer.

This simplicity comes with a trade-off: WireGuard does not handle key distribution. You must have some out-of-band mechanism to securely exchange public keys. For personal use this might be as simple as copying keys manually. For larger deployments, you need additional infrastructure for key management.

Session Establishment and Key Rotation

When two WireGuard peers need to communicate, they perform a cryptographic handshake using their static key pairs and ephemeral keys. This handshake establishes session keys for encryption.

WireGuard automatically rotates session keys approximately every two minutes by performing new handshakes. This happens based on time rather than message count, and it is designed to handle packet loss gracefully. The frequent key rotation ensures perfect forward secrecy: even if an attacker somehow obtained a session key, they could only decrypt a small window of traffic.

WireGuard communicates entirely over UDP, using a single port (typically 51820 by default). Unlike TCP-based VPNs, WireGuard connections are inherently stateless from the network perspective. A peer that goes silent is not "disconnected" in any protocol sense; it simply stops sending packets. When it comes back, the next handshake re-establishes communication seamlessly.

Optional Pre-Shared Keys

For users concerned about future quantum computing attacks on elliptic curve cryptography, WireGuard supports an optional pre-shared key that provides an additional layer of symmetric encryption. This is sometimes called "post-quantum" protection: even if quantum computers eventually break Curve25519, an attacker would still need to obtain the pre-shared key to decrypt traffic.

Advantages and Disadvantages

WireGuard's advantages include:

Its disadvantages are relatively few:

Split Tunneling

When using a remote access VPN, an important configuration choice is whether to use full tunneling or split tunneling.

With full tunneling, all network traffic from your device goes through the VPN tunnel, regardless of destination. If you're connected to your corporate VPN and browse to YouTube, that traffic goes through the corporate network first, then out to the Internet.

Advantages:

Disadvantages:

With split tunneling, only traffic destined for the corporate network goes through the VPN. Traffic to Internet sites goes directly from your device.

Advantages:

Disadvantages:

Many organizations disable split tunneling for security reasons, accepting the performance trade-off. Others allow it but implement additional controls like endpoint protection software.

VPN Performance Considerations

VPNs aren't "free" in terms of performance. They come with overhead:

Encryption overhead is actually quite minimal with modern hardware. Most CPUs have hardware acceleration for common encryption algorithms like AES. The computational cost of encryption is rarely the bottleneck.

Encapsulation overhead adds headers to packets (typically 20-60 bytes depending on the VPN protocol), slightly reducing the effective payload size. This can occasionally cause fragmentation issues, but it's generally not significant.

Routing overhead is usually the bigger issue. When you use a VPN, especially a privacy VPN, your traffic may route through geographically distant servers. If you're in New York connecting to a VPN server in Switzerland to access a website hosted in Virginia, you've added significant latency and distance. The physical laws of network propagation mean your data is traveling much farther than necessary, which noticeably slows down your Internet experience.

For corporate VPNs, the provider's server capacity can also be a bottleneck. If the VPN gateway can't keep up with traffic volume, it becomes a chokepoint for all remote access.

Common VPN Protocols: A Summary

Various VPN protocols exist, each with different characteristics:

Protocol Key Features Primary Use Notes
IPsec Standardized, kernel-level, complex Enterprise site-to-site and remote access Dominant in corporate environments; ESP mode most common
OpenVPN Open source, TLS-based, flexible Remote access, privacy VPNs Can operate over TCP or UDP; highly portable
WireGuard Modern, minimal, fast All VPN use cases Increasingly popular; built into Linux kernel; simple configuration
IKEv2/IPsec Fast reconnection, mobile-optimized Mobile remote access Part of IPsec suite; excellent for smartphones switching networks
L2TP/IPsec Layer 2 tunneling with IPsec security Remote access Widely supported but slower; being displaced by newer protocols
PPTP Legacy Microsoft protocol (Deprecated) Do not use because of known security vulnerabilities
SSL/TLS VPN Browser-based, application-layer Remote access without client software Used by Cisco AnyConnect, Palo Alto GlobalProtect; sometimes called "clientless VPN"

The trend in VPN technology is toward simpler, more secure protocols. WireGuard's rapid adoption shows the appeal of "just use the best current practices" rather than supporting decades of legacy options. Meanwhile, PPTP should be considered completely deprecated due to known cryptographic weaknesses.

Security Considerations

While VPNs provide important security benefits, they are not a complete security solution. Understanding their limitations is essential for using them effectively.

VPNs Protect the Path, Not the Endpoints

A VPN secures traffic while it traverses the tunnel, but it does not protect the endpoints themselves. If your computer is compromised by malware, the attacker can access your data before it enters the VPN tunnel. Similarly, if the remote network you are connecting to is compromised, the VPN does not help.

VPNs Do Not Guarantee Anonymity

Using a VPN changes which parties can observe your traffic, but it does not make you anonymous. The VPN provider (for privacy VPNs) or the corporate network (for remote access VPNs) can see your traffic. Websites can still identify you through cookies, browser fingerprinting, and login credentials.

VPNs Can Create a False Sense of Security

Users sometimes believe that connecting to a VPN makes all their activity secure. In reality, a VPN protects against some threats (eavesdropping on the local network) but not others (phishing, malware, compromised websites). Security requires defense in depth, not reliance on any single control.

Trust Must Go Somewhere

Every VPN deployment involves trusting some party: the VPN gateway operator, the IT department, or the commercial VPN provider. Make sure you understand who you are trusting and whether that trust is warranted.

Summary

Virtual Private Networks extend secure communication from individual applications to entire networks. By combining tunneling with encryption, integrity protection, and authentication, VPNs create secure channels through untrusted networks.

Three deployment models serve different needs. Site-to-site VPNs connect geographically separated networks, making them appear as one unified private network. Remote access VPNs allow individual computers to connect securely to corporate networks from any location. Privacy VPNs provide encrypted tunnels to third-party providers, protecting users on untrusted networks and obscuring their apparent location.

Three protocols have dominated VPN deployments. IPsec, developed in the 1990s, provides standardized security at the network layer with broad equipment support. OpenVPN, emerging in the early 2000s, runs in user space and leverages TLS, providing portability and the ability to bypass restrictive firewalls. WireGuard, introduced in 2016, takes a minimalist approach with a small, formally-verified codebase and modern cryptographic choices.

Each protocol has trade-offs. IPsec offers broad compatibility but complex configuration. OpenVPN offers portability but with performance overhead. WireGuard offers simplicity and performance but is newer and may require additional key management infrastructure.

Regardless of which protocol you use, remember that VPNs are one component of a security strategy, not a complete solution. They protect data in transit but cannot secure compromised endpoints, guarantee anonymity, or substitute for defense in depth.

Next: Study Guide