pk.org: Computer Security/Lecture Notes

Network Security

Protecting Access -- Firewalls, and Zero Trust

Paul Krzyzanowski – 2025-11-15

While TLS and VPNs focus on protecting communication across untrusted networks, firewalls focus on controlling what traffic is allowed to cross network boundaries.

Network Address Translation (NAT)

Before we dive into firewalls proper, let's understand NAT, which provides a basic level of protection while solving a different problem.

The IP Address Shortage

IPv4 addresses are 32 bits, providing about 4 billion possible addresses. That sounds like a lot, but it's far fewer than the number of devices connected to the Internet today.

What's more, addresses are allocated in chunks: Rutgers owns the entire range of IP addresses where the top 16 bits start with 128.6. (the 128.6.0.0/16 network), which means they can have almost 65,536 (216) devices addressable on the Internet (same high-order bits but different values for the lower 16 bits). Hewlett-Packard is sitting on two sets of addresses that allocate the top eight bits and allow it to have two sets of 24 bits to address internal devices: enough for over 33 million devices on the Internet!

Does every device really need a globally unique IP address?

NAT's answer is "no." Within an organization, devices use private IP addresses from special reserved ranges:

These addresses are not routable on the public Internet. A NAT-enabled router maintains a translation table that maps internal address:port pairs to external address:port pairs.

When an internal host (say, 192.168.1.10:5000) sends a packet to an external server, the NAT router replaces the source address with its own public IP and a unique port number (say, 68.36.210.55:4000), recording this mapping. When the response comes back to 68.36.210.55:4000, the router looks up the mapping and forwards the packet to 192.168.1.10:5000.

Note that the dominant form of NAT today is NAPT (Network Address and Port Translation), which modifies both IP addresses and port numbers. This requires the router to examine and modify transport layer headers (ports), not just network layer headers (IP addresses). The router must understand TCP and UDP port numbers and recalculate checksums after modifying the packets.

Security Benefits of NAT

While NAT was designed to conserve IP addresses, it provides an important security benefit: external hosts cannot initiate connections to internal hosts.

An external attacker can't send packets to 192.168.1.10 because that address isn't routable on the Internet. Even if they could, the NAT router has no translation table entry. It only creates entries when internal hosts initiate connections. This effectively blocks all unsolicited inbound traffic.

This isn't perfect security (internal hosts can still make outbound connections that could be exploited), but it's a significant improvement over every internal device having a public IP address directly accessible from the Internet.

First-Generation Firewalls: Packet Filters

Before diving into how packet filters work, it helps to understand their basic goal. A packet filter sits at a network boundary and decides, for each packet that crosses that boundary, whether to allow it or drop it. It does this without remembering past packets and without understanding higher-level application behavior. Its job is simply to enforce a set of rules based on values found in packet headers.

A packet filter (also called a screening router) examines each IP packet and decides whether to allow it or drop it based on rules matching:

How Packet Filtering Works

Packet filters organize rules into chains (also called access control lists -- but don't confuse them with user-based file access control lists). Each packet is evaluated against the rules in order. Each rule specifies:

Common actions include:

The order of rules matters! Once a packet matches a rule with an Accept or Drop action, no further rules are evaluated for that packet.

Basic Firewalling Principle

Definitions:
Ingress = the action or fact of going in or entering.
Egress = the action of going out of or leaving.
Don't blame me for the use of these terms!

Ingress filtering controls inbound traffic (from external networks to internal ones), while egress filtering controls outbound traffic.

The basic firewalling principle is that there should be no direct inbound connections from external systems (Internet) to any internal host: all traffic must flow through a firewall and be inspected.

For ingress filtering, the basic principle is "default deny": block everything except specifically allowed traffic.

You might allow:

You should also block obviously malicious traffic:

Egress filtering is less common but increasingly important. While we generally trust internal hosts, a compromised internal host can:

Fragmentation Attacks Against Firewalls

Packet fragmentation can be used to bypass stateless filters. Examples include:

Overlapping Fragments: Attackers send fragments whose payloads overlap. A firewall may reassemble differently from the target host, causing the firewall to miss malicious content.

Tiny Fragments: Malicious data is split across many small fragments so that key header fields (like TCP flags or port numbers) appear only after the first fragment. A naive firewall may inspect only the first fragment and allow the rest.

Modern stateful firewalls reassemble flows before inspection to mitigate these problems.

Second-Generation Firewalls: Stateful Packet Inspection

First-generation packet filters examine each packet independently. But network protocols like TCP create ongoing conversations between hosts. Stateful packet inspection (SPI) firewalls track the state of these conversations.

Why State Matters

Consider TCP. A proper TCP connection begins with a three-way handshake (SYN, SYN-ACK, ACK). Data packets should only flow after this handshake completes. A first-generation firewall can't enforce this because it just sees individual packets.

A stateful firewall tracks TCP connection state. It can enforce rules like:

Some protocols require tracking even more complex state.

Voice over IP (VoIP) protocols like SIP (Session Initiation Protocol) are a good example. SIP uses one connection to set up a call (typically on port 5060), but then the actual voice/video data flows on different, dynamically negotiated ports using RTP (Real-time Transport Protocol). A stateful firewall can track the SIP signaling, identify which ports will carry the media streams, and create temporary rules to allow those related connections.

Historically, FTP was the classic example. It uses a control connection on port 21 and a separate data connection, with the ports negotiated dynamically. While FTP is rarely used today, the same pattern appears in many modern protocols like SIP, H.323 (video conferencing), and even some gaming protocols.

Benefits of Stateful Inspection

Stateful inspection makes firewalls much more effective:

Security Zones and the DMZ

Most organizations don't just have "internal" and "external" networks. They create multiple security zones with different trust levels, and firewalls control traffic between these zones.

The DMZ Concept

The DMZ (DeMilitarized Zone) is a network zone that sits between the completely untrusted Internet and the fully trusted internal network. Servers that must be accessible from the Internet (web servers, mail servers, DNS servers) live in the DMZ.

The firewall enforces different policies to control traffic between the different networks. The policies differ by traffic direction:

Internet → DMZ:
Allow only specific services (HTTP/HTTPS, SMTP, etc.) to specific servers
Internet → Internal:
Block everything; no direct Internet connections to internal hosts
Internal → DMZ:
Allow specific connections, possibly for administrative services not available from the Internet (like SSH for management)
Internal → Internet:
Generally allow, so users can access Internet services (though may restrict certain protocols or destinations)
DMZ → Internal:
Strictly limit to only what's absolutely necessary (e.g., web server connecting to internal database on specific port)
DMZ → Internet:
Typically restricted to required services (e.g., mail server connecting to other mail servers)

Why We Restrict DMZ Access

The logic is this: servers in the DMZ are Internet-facing, so they're at higher risk of compromise. By placing strict limits on what DMZ systems can access (both internally and externally), we limit the damage if one is compromised.

If an attacker compromises your web server in the DMZ, they shouldn't be able to:

Systems in the DMZ are called bastion hosts. This is a term that is used to refer to systems that are carefully configured, run only essential software, have only required user accounts, and are regularly audited. They're the systems most exposed to attack, so they receive the most security attention.

Network Segmentation

The DMZ is one example of the broader principle of network segmentation. Instead of one flat internal network, organizations divide their infrastructure into multiple segments:

Firewalls enforce policies between segments. Even if an attacker compromises a system in one segment, they face additional barriers to reaching other segments.

The benefits of segmentation include:

This segmentation strategy creates multiple trust boundaries within an organization, with each boundary enforcing the principle of least privilege. Systems only have access to what they absolutely need.

Third-Generation Firewalls: Deep Packet Inspection

Packet filters, even stateful ones, primarily examine packet headers. They look at IP addresses, ports, and protocol types. But what about the actual content of the packets?

Deep Packet Inspection (DPI) examines the application-layer data inside packets. This enables:

Design Challenges with DPI

Deep packet inspection faces serious practical challenges:

First, it must operate at network speeds. Modern networks move data very quickly, and the firewall must keep up. This means DPI hardware can only buffer a limited number of packets and can only store a limited number of patterns to match against.

Second, encrypted traffic can't be inspected. Since more and more Internet traffic uses HTTPS, some organizations deploy TLS-intercepting firewalls that act as a man-in-the-middle, decrypting and re-encrypting traffic. This breaks end-to-end encryption and requires distributing the firewall's certificate to all client machines (e.g., your browser will no longer get your bank's certificate when you connect to your bank).

Deep Content Inspection (DCI)

Deep Content Inspection extends DPI to handle content that spans multiple packets:

DCI is computationally expensive, so it's typically only applied selectively on traffic that matches certain criteria.

Intrusion Detection and Prevention Systems (IDS/IPS)

IDS and IPS systems are specialized forms of deep packet inspection focused on identifying and stopping attacks.

IDS vs. IPS

An Intrusion Detection System (IDS) monitors network traffic and reports suspicious activity. It's passive; it observes and alerts but doesn't take action.

An Intrusion Prevention System (IPS) actively blocks traffic it identifies as malicious. It sits inline between networks and can drop malicious packets before they reach their destination.

Three Approaches to Threat Detection

Either type of system may use various techniques to determine what qualifies as an intrusion. There are three types of mechanisms used.

1. Protocol-Based Detection

This approach verifies that traffic adheres to protocol specifications. For example:

The idea is that many attacks exploit protocol implementations by sending malformed or unexpected input. By enforcing strict protocol correctness, we can block these attacks.

2. Signature-Based Detection

This approach maintains a database of known attack patterns (signatures). Each signature describes the sequence of bytes or packets that make up a specific attack. As with malware detection, signatures are simply patterns and have no relationship to digital signatures.

When traffic matches a signature, it's blocked (in an IPS) or flagged (in an IDS).

Just like with malware detection, the limitation is that signature-based detection only catches known attacks. Zero-day exploits and new malware variants won't match any signatures. This requires constantly updating the signature database.

3. Anomaly-Based Detection

Instead of looking for known bad patterns, anomaly-based systems look for deviations from normal behavior. The system first establishes a baseline of normal activity, then flags anything statistically unusual:

The challenge is distinguishing malicious anomalies from legitimate unusual activity. New employees, new applications, or legitimate changes in usage patterns can all trigger false positives.

Practical Challenges with IDS/IPS

Real-world IDS/IPS deployment faces several challenges:

Next-Generation Firewalls (NGFW)

The term "Next-Generation Firewall" refers to products that combine stateful packet inspection, deep packet inspection, and intrusion prevention into a single platform. The term is widely used in industry and originated as a marketing label, but it describes a real technical shift: firewalls that understand and control traffic at the application level rather than relying solely on ports and protocols.

NGFWs typically include:

The key addition is application identification. Older firewalls assumed that port numbers revealed the application (for example, port 80 meant HTTP and port 25 meant SMTP). Today, most applications use HTTPS on port 443, so port numbers are no longer meaningful. NGFWs classify applications by examining network behavior rather than relying on the operating system to reveal which process opened a socket.

NGFWs do not know which local process generated the traffic; only a host-based firewall can see that. Instead, NGFWs infer the application by analyzing:

For example, Zoom, Slack, Dropbox, and Teams all use HTTPS on port 443, but their signaling behavior, packet size distributions, and TLS metadata differ. An NGFW can use these network-level clues to classify each application and apply policy accordingly.

They're essentially security appliances that consolidate multiple security functions that previously required separate devices, and they add application- and identity-focused policy capabilities that traditional stateful firewalls did not have.

Application Proxies

An application proxy is a different approach to packet-inspecting firewalls. Instead of determining whether to forward packets to a service, a proxy provides the service's external interface: external clients connect to the proxy rather than the service itself.

An application proxy sits between clients and servers, acting as an intermediary. Clients connect to the proxy, and the proxy connects to the real server. A proxy terminates the client’s connection and initiates a separate connection to the server, giving it complete control over both sides of the exchange. This provides several security benefits:

Protocol Validation: The proxy can deeply understand and validate application protocols, blocking malformed or malicious requests. Since the proxy doesn’t implement the underlying application, the proxy developer’s job is largely to implement and verify the protocol.

Content Filtering: The proxy can examine and modify content, removing potentially dangerous elements.

Hiding Internal Structure: External clients only see the proxy, not the internal servers behind it.

Logging and Monitoring: All traffic flows through a single point where it can be inspected and logged.

Common examples include web proxies, email proxies, and VPN concentrators.

The downside is that you need a different proxy for each protocol, and proxies can become performance bottlenecks.

Host-Based Firewalls

All our discussion so far has focused on network firewalls that protect entire networks. Host-based (or personal) firewalls run on individual systems and provide an important additional layer of security.

What Host-Based Firewalls Do

Host-based firewalls control network traffic at the individual system level. They can:

These provide defense in depth. Even if malware gets onto a system, a host-based firewall can prevent it from communicating over the network. Similarly, if one system in a network is compromised, a host-based firewall can prevent lateral movement to other systems.

An important advantage of host-based firewalls is that they know which application is sending or receiving specific packets and can make application-based decisions rather than simply making host-based, content-based, or port-based decisions. The firewall integrates with the operating system’s networking stack, so it knows the specific executable and process that opened each network connection. This allows fine-grained outbound control that network firewalls cannot provide.

Windows Firewall

Windows includes a built-in firewall called Windows Defender Firewall (formerly Windows Firewall). It's been included in every version of Windows since Windows XP Service Pack 2.

Basic Features:

  • Enabled by default on Windows systems

  • Maintains separate profiles for different network types (Domain, Private, Public)

  • Automatically adjusts rules based on network location

  • Provides both inbound and outbound filtering

How It Works: Windows Firewall operates on rules that specify:

  • Program or port to allow/block

  • Protocol (TCP/UDP)

  • Direction (inbound/outbound)

  • Network profile when the rule applies

When you install an application that needs network access, Windows typically prompts you to allow it through the firewall. You can also manually configure rules through Windows Defender Firewall with Advanced Security.

Example Scenarios:

  • When you connect to a coffee shop WiFi, Windows switches to the "Public" profile, which blocks most inbound connections

  • At home, it might use the "Private" profile, which allows file sharing and other local network features

  • In a corporate environment, it uses the "Domain" profile with centrally-managed policies

Advanced Configuration: IT administrators can configure Windows Firewall through Group Policy, creating sophisticated rules that:

  • Block all outbound traffic except to specific destinations

  • Restrict certain applications to only communicate on the corporate network

  • Log connection attempts for security monitoring

macOS Firewall

macOS includes a built-in firewall that works differently from Windows Firewall. It focuses on application-level control rather than port-level control.

Basic Features:

  • Not enabled by default (users must turn it on in System Settings -- not sure why they did that)

  • Application-centric approach

  • Simpler interface than Windows Firewall

  • Supports stealth mode

How It Works: The macOS firewall primarily controls which applications are allowed to accept incoming connections. When an application tries to listen for incoming network connections, macOS prompts the user to allow or deny it.

Configuration: Users can access the firewall through System Settings → Network → Firewall. Options include:

  • Block all incoming connections: Extremely restrictive mode that blocks all incoming connections except those required for basic network services

  • Allow/Deny specific applications: Fine-grained control over which apps can accept connections

  • Stealth mode: Makes the Mac invisible on the network by not responding to probe requests

Example Scenarios:

  • When you first run a web server or file sharing application, macOS asks if you want to allow incoming connections

  • In stealth mode, your Mac won't respond to ping requests or port scans, making it harder for attackers to discover

  • Built-in services like File Sharing are automatically managed when you enable them in System Settings

Differences from Windows: macOS takes a simpler, application-focused approach. While Windows Firewall can create complex rules based on ports, protocols, and IP addresses, macOS firewall primarily asks "should this application be allowed to accept connections?" This makes it easier for typical users but less flexible for advanced configurations.

Third-Party Firewalls:

Both Windows and macOS support third-party firewalls like:

  • Little Snitch (macOS): Provides detailed outbound connection monitoring

  • ZoneAlarm (Windows): Adds additional features beyond Windows Firewall

  • Norton and other security suites that include firewall components

Limitations of Host-Based Firewalls

While host-based firewalls are valuable, they have important limitations:

Administrator Compromise: If malware gains administrator or root privileges, it can disable or reconfigure the firewall. This is why host-based firewalls work best as part of a defense-in-depth strategy, not as a standalone solution.

Complexity for Users: Users may not understand firewall prompts and might allow malicious applications simply to make dialogs go away.

Performance Impact: On some systems, especially with third-party firewalls, inspecting all network traffic can degrade performance.

Limited Visibility: Host-based firewalls can only see traffic to and from their own system. They can't protect against attacks between other systems on the network.

Best Practices for Host-Based Firewalls

To get the most value from host-based firewalls:

Zero Trust Architecture

Traditional network security assumes a "perimeter" model: there's a clear boundary between trusted internal networks and untrusted external networks. Firewalls defend this perimeter. Once you're inside, you're largely trusted.

This model is increasingly problematic.

The Problem of Deperimeterization

The concept of a clearly defined network perimeter is breaking down:

Mobile Devices: Employees work from coffee shops, airports, and home. Their devices constantly move between trusted and untrusted networks.

Cloud Computing: Your applications and data might run on AWS, Azure, or Google Cloud rather than in your own data center. Where's the "perimeter" now?

Insider Threats: Not everyone inside your network should be trusted. Malicious insiders exist, and legitimate users' accounts get compromised.

Compromised Systems: If an attacker gains access to a system within your network, traditional perimeter security provides little resistance to lateral movement (moving from that system to others).

Web Applications: Your employees' browsers connect to countless external services, potentially downloading malware or leaking data.

The idea that "inside equals safe" is fundamentally flawed.

Zero Trust Principles

Zero Trust Architecture (ZTA) operates on a simple premise: Never trust, always verify.

In practice, “verify” means continuously evaluating both the user and the device against current policy rather than assuming that earlier authentication still applies. Zero trust is not just multi-factor authentication; MFA verifies identity, but ZTA also validates device posture, application context, and policy compliance on each access.

Don't assume something is secure because it's on your internal network. Instead, require authentication and authorization for every access to every resource, regardless of where the request originates.

The National Institute of Standards and Technology (NIST) defines seven core tenets of zero trust:

  1. All data sources and computing services are considered resources that need protection.

  2. All communication must be secured regardless of network location. Being on the "internal network" doesn't make communication automatically trusted.

  3. Access to resources is granted on a per-session basis. Each time you access a resource, you must be authenticated and authorized for that specific access. This doesn't necessarily mean entering your password every time (single sign-on can handle authentication), but it means each access decision is made independently based on current policy, current device state, and current threat level. You're not granted permanent access to all resources just because you logged in once.

  4. Access is determined by dynamic policy. Policies can consider identity, device posture (e.g., OS version, installed apps, user-managed or company-managed), location, time, and current threat level.

  5. The organization monitors and measures the integrity and security posture of all assets. This includes both company-owned devices and BYOD (bring your own device).

  6. All resource authentication and authorization are dynamic and strictly enforced before access is allowed. There is no implicit trust based on the device, user, or service.

  7. The enterprise collects information about asset and network state and uses it to improve security posture. Continuous monitoring and improvement.

Microsegmentation in Zero Trust

While we introduced network segmentation earlier as part of a firewall strategy, microsegmentation takes this concept much further as a key component of zero trust.

Traditional segmentation divides the network into a handful of zones (DMZ, internal, guest, etc.). Microsegmentation divides it into very small segments, potentially one segment per application or per individual virtual machine.

In a microsegmented environment:

This is often enabled by software-defined networking and virtualization technologies that make it practical to create and manage many network segments. In many deployments, microsegmentation is enforced by distributed firewalls inside the hypervisor or container runtime rather than by a perimeter firewall. The firewall policies move from the network edge into the virtualization layer itself.

Microsegmentation supports zero trust by ensuring that even if an attacker gains initial access, they're contained within a very limited environment. Combined with the other zero-trust principles, this dramatically reduces the potential impact of a breach.

Implementing Zero Trust

In theory, zero trust means every connection is authenticated, authorized, and encrypted end-to-end. In practice, this is challenging:

Application-Level Security: Ideally, security is built into applications themselves. Applications would authenticate users, verify authorization, and encrypt data end-to-end. But most existing applications weren't designed this way, and there's no standard framework for this.

Zero Trust Network Access (ZTNA): As a fallback, many organizations implement zero trust at the transport layer rather than the application layer. This typically means:

Unlike traditional VPNs, which often grant broad network access, ZTNA restricts users and devices to only the specific applications they are authorized to reach.

Device Trust: Zero-trust systems often evaluate device posture before granting access. Is the device managed by the organization? Does it have updated security software? Is it running from a known location? Has it been jailbroken or rooted?

Identity-Centric Security: Instead of "trust this IP address because it's internal," zero trust focuses on "trust this authenticated user, on this verified device, accessing this specific resource, for this specific purpose."

Challenges with Zero Trust

Zero Trust isn't a simple solution you can just "implement." It's a fundamental shift in security architecture that comes with significant challenges:

Complexity: Zero trust requires coordinating identity management, device management, access control, encryption, and monitoring across all resources. This is organizationally and technically complex.

Legacy Systems: Many existing applications and systems weren't designed for zero trust. Retrofitting them is difficult.

User Experience: Requiring authentication for every resource access can frustrate users if not implemented carefully. Single sign-on and strong authentication methods (like hardware tokens) help, but add complexity.

Insider Threats Still Exist: Zero trust helps limit what authenticated users can access, but it can't completely prevent misuse of legitimate access.

Stolen Credentials: If an attacker steals valid credentials or compromises an authorized device, zero trust controls may not detect the attack.

Performance: Encrypting everything, authenticating every session, and checking policies for every access can impact performance.

Government Zero Trust Initiatives

The importance of zero trust has been recognized at the highest levels of government:

United States: The Office of Management and Budget directed all federal agencies to move from perimeter-based defenses to zero-trust architecture by September 30, 2024. I have no idea if this goal has been put on hold with the current administration and haven't read of any completion state.

European Union: The Network and Information Security Directive (NIS2) has similar requirements, with an implementation deadline of October 2024. Again, I don't know if this happened.

United Kingdom: While not mandating zero trust, the National Cyber Security Centre strongly promotes it and provides detailed implementation guidelines.

Canada: Zero trust is a core part of Canada's national Cyber Security Strategy.

These initiatives reflect widespread recognition that perimeter-based security is no longer sufficient for modern distributed systems, particularly in government environments where legacy systems and heterogeneous infrastructures increase the attack surface.

How the Parts Fit Together

Let's see how these technologies work together in a realistic modern organization:

Perimeter Defense: The organization has firewalls at the network perimeter, implementing stateful inspection and deep packet inspection. These block obvious threats and control what services are accessible from the Internet.

DMZ: Public-facing services (web, email) run in a DMZ, isolated from internal networks. These bastion hosts are hardened and closely monitored.

Network Segmentation: Internal networks are divided into segments based on function and sensitivity. Firewall rules control traffic between segments.

VPN for Remote Access: Employees working remotely connect via VPN. After authenticating (using multi-factor authentication), they gain access to internal resources. Split tunneling is disabled for security.

TLS for Application Security: All web applications use TLS. Internal applications also use TLS, not trusting the "internal network" to provide security.

Zero Trust Principles: Access to sensitive resources requires re-authentication. Device posture is checked before granting access. All access is logged and monitored.

Microsegmentation: Critical applications run in microsegmented environments where only specific, required communication is allowed.

Host-Based Firewalls: All endpoints run host-based firewalls that restrict application network access and prevent lateral movement if compromised.

IDS/IPS: Network traffic is monitored for suspicious patterns. Known attacks are automatically blocked.

This defense-in-depth approach means that:

Conclusion

Network security has evolved from simple packet filters to sophisticated systems that analyze behavior, inspect encrypted traffic, and assume no implicit trust.

Transport Layer Security gives us secure communication between applications, protecting confidentiality and integrity even across completely untrusted networks. VPNs extend this concept to entire networks, creating secure tunnels through the hostile Internet.

Firewalls have evolved from simple packet filters to deep packet inspection systems that understand applications, detect malware, and actively prevent intrusions. Security zones and network segmentation allow fine-grained control over what traffic is allowed where.

Zero Trust Architecture represents a fundamental rethinking of network security, acknowledging that traditional perimeter defenses are insufficient in a world of mobile devices, cloud services, and persistent threats.

No single technology solves all problems. Effective network security requires layered defenses (defense in depth). Use encryption to protect data in transit. Use firewalls to control access between networks. Use zero trust principles to limit the damage from inevitable breaches. Monitor continuously, update regularly, and assume that eventually, something will go wrong.

The goal isn't perfect security. That's impossible. The goal is to make attacks difficult enough, and their impact limited enough, that your organization can continue to function even in a hostile environment.