Memory Vulnerabilities and Defenses

Vulnerabilities and Exploits

Buffer overflow: A condition where a program writes more data into a buffer than it can hold, overwriting adjacent memory such as variables or control data.
Integer overflow / underflow: A mathematical operation that exceeds or falls below the representable range of a data type, often causing incorrect buffer sizes or loop limits.
Off-by-one error: A boundary mistake where a loop or copy operation accesses one element past the intended limit.
Format-string vulnerability: An error where untrusted input is used as a format string, allowing memory reads with %x or writes with %n.
Shellcode: A small sequence of machine instructions injected into memory, historically used to open a command shell when executed.
Return-to-libc attack: An exploit that calls existing library functions instead of injected code, bypassing non-executable memory protections.
Return-oriented programming (ROP): An attack that chains short code fragments, called gadgets, already present in executable memory to perform arbitrary computation.
Use-after-free: A condition where a program continues to use a pointer to memory that has already been freed, allowing corruption or control hijacking.

Non-executable memory (NX / DEP): A hardware-enforced rule that prevents executing code from writable memory regions such as the stack or heap.
Address-space layout randomization (ASLR): A defense that randomizes the location of code, stack, heap, and libraries each time a process starts, reducing predictability of addresses.
Stack canary: A small random value placed before saved control data on the stack; checked before returning from a function to detect buffer overflows.
Heap canary (heap cookie): A small guard value stored before or after a heap allocation, checked when the block is freed to detect overwrites or underflows.
Allocator hardening: Enhancements to memory allocators that verify metadata consistency, randomize allocation patterns, delay reuse of freed blocks, and isolate metadata from user memory.
Safe linking / pointer mangling: A protection that encodes allocator pointers with a secret or address bits, preventing attackers from forging valid links.
Safer library functions: Secure versions of standard C functions, such as snprintf or fgets, that check buffer sizes to prevent overflows.

Compiler instrumentation: Extra checks inserted by the compiler to detect invalid memory accesses, undefined behavior, and arithmetic errors during testing.
Fuzzing: Automated testing that generates or mutates program inputs to discover crashes, sanitizer failures, and edge cases.
Sanitizers: Compiler-enabled runtime checks that detect memory or logic errors, such as out-of-bounds access or use-after-free, during development.

Control-flow integrity: A technique that restricts program execution to legitimate control paths, preventing attackers from redirecting jumps or returns.
Intel Control-flow Enforcement Technology (CET): Hardware support for CFI that provides a shadow stack for return-address protection and indirect branch tracking.
Pointer authentication: A mechanism that attaches a short integrity code to pointers so the processor can detect if they have been modified.
Memory tagging: A hardware feature that assigns small tags to memory allocations and pointers to detect mismatched or stale references.

Defense in depth: A strategy of layering independent security mechanisms so that bypassing one defense still leaves others in place.
Information leak: A vulnerability that exposes memory contents or addresses, undermining protections like ASLR.
Metadata: Control information used by memory allocators to manage heap blocks, often a target for heap corruption attacks.
Quarantining: A heap defense that delays reuse of freed memory to make use-after-free exploitation unreliable.
Integrity check: A validation step ensuring that critical data, such as a pointer or return address, has not been modified unexpectedly.