Pipeline Hazards Explained: Overcoming Stalls in CPU Architecture#

Structural Hazards#

Structural hazards arise when two or more pipeline stages require the same hardware resource at the same time, and the hardware is incapable of serving both requests simultaneously. For instance, if your CPU’s design allows only one instruction memory access per cycle, two instructions that need to fetch or write in the same cycle cannot both proceed. One of them must stall.

An example of a structural hazard might be a design that has a single memory port for both instructions and data. This bottleneck will force the CPU to arbitrate whether an instruction fetch or a data access gets priority. To solve structural hazards, designers can replicate resources, adopt multi-port memory, or restructure the pipeline to avoid conflicting hardware usage.

Data Hazards#

Data hazards occur when instructions depend on the results of prior instructions still in progress in the pipeline. Because pipelining processes multiple instructions simultaneously, a future instruction may need a value that is not yet computed or written by a preceding instruction.

Data hazards come in three categories:

1. Read After Write (RAW) – The most common type, where an instruction tries to read a register before the previous instruction finishes writing to it. This is also known as a true dependency.
2. Write After Write (WAW) – An instruction tries to write to a register before a previous instruction completes a write to the same register. This is sometimes called an output dependency and occurs in more advanced pipelines with out-of-order execution.
3. Write After Read (WAR) – An instruction tries to write to a register before a previous instruction has read from it, also referred to as an anti-dependency. This typically arises in out-of-order execution scenarios.

Data hazards are commonly overcome or minimized using hardware forwarding, pipeline stalls, or more advanced methods such as register renaming.

Control Hazards#

Control hazards, often called branch hazards, arise from branch (or jump) instructions that alter the flow of instruction execution. When a branch is taken, the pipeline may have already fetched or partially processed instructions that should not be executed. Conversely, if the branch is not taken, instructions that should be executed might have been neglected.

The basic pipeline strategy typically fetches and decodes the next instruction sequentially. But when an instruction changes the program counter (PC), the pipeline must discard or flush instructions that are in-flight if the branch is taken. Techniques such as branch prediction, delayed branching, and speculative execution help mitigate these control hazards.

Forwarding#

Forwarding (also called bypassing) is a hardware technique used to resolve data hazards. In a simple pipeline without forwarding, an instruction that produces a result will write the result to a register in the WB stage, but another instruction might need that result in its EX stage just one cycle later. Forwarding routes the result directly from an earlier pipeline stage (often EX or MEM) to the input of the dependent instruction’s EX stage, circumventing the need to wait until the WB stage completes.

By adding specialized hardware paths and multiplexers, the CPU can detect data dependencies and provide the correct operand from an earlier pipeline stage in time for an instruction’s EX stage, eliminating some pipeline stalls.

Pipeline Stalls#

Sometimes, a hazard cannot be resolved immediately via forwarding. In such cases, pipeline stalls (also known as bubbles or no-ops) are introduced. A stall effectively halts the advancement of instructions through certain pipeline stages for a cycle or more. This allows the dependent instruction’s required data to become available before it enters the EX stage.

For example, consider an instruction sequence where the second instruction needs the result of the first. If the CPU’s pipeline and hardware design cannot forward the data immediately, it can insert one or more stall cycles between the two instructions to ensure correct data ordering.

Branch Prediction#

Control hazards are often mitigated with branch prediction. The CPU predicts whether a branch will be taken and proceeds to fetch the next set of instructions accordingly. If the prediction is correct, the pipeline continues without interruption. If it is wrong, the pipeline discards the speculatively executed instructions, incurring some penalty.

Simple branch predictors use heuristics (e.g., always predict taken or not taken). More advanced predictors maintain branch history tables or two-level adaptive schemes to track recent branch behavior. Modern CPUs often combine multiple prediction strategies and have sophisticated branch resolution units to reduce the penalty of mispredictions.

Speculation#

Speculative execution is an extension of branch prediction and other forms of dynamic execution. When the CPU is uncertain about the outcome of a conditional instruction or an address, it “speculates” on the likely outcome and continues executing beyond that instruction. If the speculation proves correct, performance is improved because instructions were processed in parallel. If it proves incorrect, the CPU must flush the speculative instructions, reverting the architectural state to a known good point before continuing.

Out-of-order and superscalar CPUs rely heavily on speculation to keep the pipeline full, thus improving overall throughput. However, speculation also introduces complexity in hardware design, especially around memory consistency and security issues such as side-channel attacks.

Data Hazard Example#

Below is a typical scenario of a Read After Write (RAW) hazard. Suppose we have a simple pipeline that attempts to complete one instruction per clock cycle, without advanced forwarding:

1
; Instruction 1: Load the value at memory address in R2 into R1
2
LW R1, 0(R2)
3

4
; Instruction 2: Add the value in R1 to R3, store in R4
5
ADD R4, R1, R3

If the load instruction requires multiple cycles (e.g., for memory access) to complete, the ADD instruction might need the value in R1 before it has been fully updated. A simplistic pipeline’s timing could look like this:

Without forwarding or stalls, the ADD instruction tries to use R1 in its EX stage (cycle 4), even though R1 is written in cycle 5. This causes a RAW hazard. The solution could be:

1. Insert a stall (or a no-op) between the LW and ADD instructions.
2. Use forwarding hardware to bypass the new R1 value from the MEM or WB stage directly to the ADD instruction in cycle 4.

Control Hazard Example#

Now consider a branch instruction:

1
; Instruction 1: Compare registers R1 and R2
2
CMP R1, R2
3

4
; Instruction 2: Conditional branch if R1 < R2
5
BLT label
6

7
; Next instructions (if not taken) might get fetched speculatively
8
ADD R4, R4, R5
9
SUB R6, R6, R7
10
...

While the CPU determines whether the branch is taken, it may have already fetched the ADD and SUB instructions. If the branch is taken, those instructions are invalid and must be flushed from the pipeline. A branch predictor that guesses correctly improves performance; an incorrect guess wastes cycles.

Out-of-Order Execution#

Out-of-order execution lets the CPU reorder instructions to avoid pipeline stalls whenever possible. Instead of carrying out instructions strictly in the program order, the CPU can dispatch instructions to different execution units as soon as their inputs are ready. This approach deals with parallelizable regions of code more efficiently.

The CPU keeps track of the logical program order so it can maintain the illusion of sequential execution from the programmer’s perspective. Internally, it uses techniques like register renaming and speculation to ensure consistency. Out-of-order execution can significantly reduce stalls from data or structural hazards, but it complicates CPU design and increases hardware schematics.

Register Renaming#

Register renaming is a key technique that helps avoid certain data hazards, particularly WAW (Write After Write) and WAR (Write After Read). When multiple instructions want to write or read the same architectural register, the CPU might assign each instruction a distinct physical register behind the scenes. This means that two instructions do not have to wait on each other if they functionally require different data, even though they might use the same architectural register name.

In essence, register renaming removes false dependencies caused by the reuse of architectural register names. It preserves the correct logical program behavior while allowing more parallelism and fewer pipeline stalls.

Multi-Issue Processors#

A multi-issue processor can dispatch multiple instructions per clock cycle, increasing instruction throughput. Examples include “superscalar” architectures where multiple pipelines operate in parallel. This design intensifies the importance of hazard management, as now you must ensure that data and control dependencies are respected across multiple instructions executing concurrently.

In a superscalar pipeline, structural hazards can become more frequent because multiple instructions may simultaneously request access to functional units like adders, multipliers, or memory ports. Similarly, data hazards multiply in complexity, requiring robust forwarding and scheduling logic.

Hyper-Threading#

Hyper-Threading (HT) is Intel’s brand name for Simultaneous Multithreading (SMT). With SMT, a single physical CPU core aggregates hardware resources but manages simultaneous contexts (threads) to keep the pipeline busy. Whenever one thread encounters a stall (e.g., waiting on data from memory), another thread occupying the same physical core can utilize the idle pipeline resources.

While Hyper-Threading does not directly eliminate pipeline hazards, it helps the CPU remain productive by switching to instructions from another thread if one thread’s pipeline hazards cause stalls. This approach is especially effective when each thread has distinct data and instruction streams, reducing competition for shared resources.

Superscalar vs. VLIW Architectures#

Superscalar designs dynamically schedule multiple instructions at runtime, while Very Long Instruction Word (VLIW) architectures rely on the compiler to bundle parallel instructions into a single, wide instruction word. VLIW systems simplify the hardware but demand more complex compilers and often have more explicit hazard avoidance at the software level.

In VLIW systems, hazard management is largely static—solved by the compiler. The compiler inserts no-ops or reorders instructions to avoid hazards. In superscalar systems, hazard detection and resolution is heavily reliant on hardware. While this can be more efficient in certain workloads, it also adds complexity to the CPU’s design.