How branch prediction and speculative execution work — the trick that makes fast CPUs fast (and leaks your secrets)

Speculative execution is one of the most consequential ideas in modern computer architecture: the CPU does work it has not yet been allowed to do, on the assumption that it will be allowed to keep it a few cycles later. The trick buys enormous performance — a typical core would otherwise sit idle for many cycles waiting on memory or branch outcomes — but it also leaves behind physical side effects that are not fully rolled back when the guess turns out to be wrong. Those side effects are the entire foundation of the Spectre and Meltdown hardware leak families.

The heart of the trick is branch prediction, which exists because a huge fraction of every program is conditional branches — instructions that say if x, do this, otherwise do that. The CPU cannot know which path to take until it finishes the dependent load or comparison, but it has to keep issuing instructions every cycle. So the core runs a dedicated branch predictor: a small hardware table that remembers, per branch instruction, which way the branch usually went, and guesses the next outcome using that history. Most modern predictors are a blend of a taken / not-taken history and a branch target buffer that predicts the destination of indirect jumps (function calls through a pointer, switch vtables, and so on) — the same mechanism Spectre variant 2 exploits.

How the pipeline works — and where the guess leaks

• Fetch and decode the next instruction stream; the branch predictor guesses where the next fetch should come from
• Speculatively execute the guessed path ahead of the dependent check — cache lines get touched, store buffer slots get filled, microarchitectural state gets set
• Commit (retire) in program order only once the real result is known and no exception fires; a wrong guess triggers a pipeline flush that discards the committed state
• Side effects that survive the flush — cache residency, translation buffer entries, port contention — are what an attacker can read, and they are the actual leak channel

Spectre, the 2018 class of vulnerabilities, works by training the branch target buffer into a specific mispredict, then running an out-of-bounds or out-of-access read while the CPU has not yet validated the bounds check. The read never commits — but it loads a cache line that encodes whatever the secret was, giving the attacker a cache side-channel to read it back byte by byte. The deeper structural insight — that any mispredict can leave physically observable state behind — is what makes the entire family of Spectre-class leaks possible, from Bounds Check Bypass through Branch Target Injection through GhostWrite and QEMU TSS leak variants. Meltdown is the sibling mechanism: speculative user-mode reads of kernel memory commit the secret into L1 before the page-table permission check fires.

The three big lessons

• Performance and security are in fundamental tension here — the very hardware that makes a core fast also creates a channel that the ISA did not intend to expose, and closing that channel costs real cycles
• Any state that survives a mispredict flush is a side channel — cache, TLB, store buffer, port contention, everything is a potential leak channel, and new variants keep being discovered decades in by researchers
• The trick will not go away — every next-gen CPU from KVM-era cores through speculative execution in 2026 runs deeper branch predictors and deeper out-of-order pipelines; the mitigation stack is the only line of defense

Sources: Communications of the ACM — Spectre Attacks (2023); IEEE Spectrum — How the Spectre and Meltdown Hacks Really Worked; Wikipedia — Spectre (security vulnerability).

Corrections: Report an error at corrections@wireshaman.com.