Shared Success: How Interconnects Drive Efficiency in Chiplet Architectures#

Key Concepts#

• System Partitioning: Instead of a single, large, monolithic chip, engineers distribute functionality across multiple chiplets. One chiplet might be dedicated to CPU computation, another to GPU acceleration, and yet another to specialized tasks such as networking or AI processing.
• Heterogeneous Integration: Different manufacturing processes can be used for each chiplet, allowing the combination of lower-power processes for memory or analog circuitry with high-performance processes for compute. This ability to mix technologies can provide enormous flexibility and efficiency.
• Scalability: By stacking or tiling multiple chiplets, designers can create systems with more cores, more memory bandwidth, or more specialized accelerators, all while reducing yield loss and production cost.

On-Die vs. Off-Die#

• On-Die Interconnect: Traditional SoCs use on-die interconnect buses or crossbars (e.g., ARM AMBA, AXI) to connect internal blocks. These are extremely high bandwidth and often short in length.
• Off-Die Interconnect: In a chiplet architecture, the communication between dies happens across substrates (2.5D or 3D packaging) or printed circuit boards (PCBs). Achieving high bandwidth with low latency and minimal power overhead is significantly more challenging off-die.

Key Parameters to Consider#

1. Bandwidth: The amount of data that can be transmitted per unit time (Gbps or TB/s).
2. Latency: The time it takes for a signal to travel from one chiplet to another.
3. Power: The energy cost for transmitting data over the interconnect.
4. Scalability: How well the interconnect can grow if you add more chiplets or more lanes.

Interconnect Topologies#

• Point-to-Point: Direct, dedicated links connecting two chiplets. Excellent for low-latency communication but can become unwieldy with many chiplets.
• Star Topology: Central hub or router that all chiplets connect to. Simplifies routing but can become a bottleneck.
• Mesh / Network-on-Chip (NoC): A grid-like structure allowing multiple paths between chiplets, improving scalability. However, it can be more complex to implement.
• Ring: Each chiplet is connected to two neighbors in a circular fashion. Often simpler than a full mesh but might limit peak performance.

2.5D Integration#

In 2.5D packaging, chiplets are placed side by side on an interposer (often made of silicon) that provides high-density routing. The interposer has fine pitch wiring, letting chiplets communicate with each other at high bandwidth and low latency.

• Advantages: High-density routing, relatively mature technology, good thermal characteristics (as chiplets lie side by side).
• Drawbacks: Can be costly due to large interposers, which must be defect-free. Also, routing complexity can be significant.

3D Stacking#

Here, chiplets are stacked on top of each other, creating extremely short connections, which is beneficial for bandwidth and latency. Through-silicon vias (TSVs) connect the layers electrically.

• Advantages: Reduced footprint, lower interconnect power, potentially very high bandwidth.
• Drawbacks: Heat dissipation can be challenging, and stacking yields can affect overall cost.

Embedded Multi-die Interconnect Bridge (EMIB)#

EMIB is an Intel-developed technology that places a small “bridge” embedded within the package substrate, connecting chiplets locally. Unlike a large interposer, EMIB uses smaller silicon bridges, potentially reducing cost and complexity.

• Advantages: Flexible placement of high-density bridge regions, scalable, can be less expensive than full interposers.
• Drawbacks: Still relatively proprietary, and the packaging ecosystem around EMIB is less standardized compared to 2.5D on an interposer.

The choice between 2.5D, 3D, or EMIB depends on design requirements around performance, power, cost, thermal constraints, and time-to-market. Yet, in all of these approaches, the interconnect standard used to connect the chiplets remains a crucial factor.

Example Protocols#

1. Serial RapidIO: Known for its low-latency, high-reliability links. It’s been used in networking and embedded systems.
2. PCI Express (PCIe): A widely adopted standard. Many chiplet solutions use a PCIe-based interface for compatibility. However, its overhead may be higher than specialized interconnects.
3. CXL (Compute Express Link): Built on top of PCIe 5.0, offering coherency and low-latency memory sharing. Strong contender for CPU-GPU or CPU-Accelerator chiplet connections.
4. Infinity Fabric (AMD): AMD’s proprietary technology for linking CPU and GPU chiplets.
5. NVLink (NVIDIA): Allows high-speed, coherent connection between GPUs and CPUs in HPC systems.
6. Cache-Coherent Interconnect for Accelerators (CCIX): Enables coherent memory sharing between processors and accelerators.
7. UCIe (Universal Chiplet Interconnect Express): An emerging standard focusing on simplifying die-to-die communication, aiming to create a vendor-neutral ecosystem.

Coherency Models#

An essential question in deciding on a protocol is whether the system needs coherency. A cache-coherent interconnect lets multiple chiplets simultaneously access shared memory regions without data inconsistencies. While coherence is beneficial for CPU-GPU or multi-core architectures, it adds complexity and potentially higher latency.

UCIe (Universal Chiplet Interconnect Express)#

UCIe is an open specification designed to unify die-to-die connectivity. It encompasses physical layer, protocol, and software stack requirements. By adhering to UCIe, a memory controller chiplet from one vendor can theoretically talk to a CPU chiplet from another vendor.

Key aspects covered by UCIe:

• Physical Layer: Defines the electrical characteristics (signal levels, line encoding, etc.).
• Protocol Layer: Generally built on PCIe/CXL for compatibility, but aims to be flexible.
• Form Factor: Addresses mechanical and thermal design for multi-die packaging.

Bunch of Wires (BoW)#

BoW is another effort focusing on high-bandwidth and low-latency connections. It aims to minimize the complexity and cost of bridging chiplets by defining a simplified interface.

Why Standards Matter#

• Ecosystem Growth: Standards attract more developers, more IP providers, and more system integrators.
• Interoperability: Enables mixing chiplets from different suppliers, fostering competition and innovation.
• Lower Barrier to Entry: New entrants can design a specialized chiplet that plugs into an existing system without creating a proprietary interconnect.

Example 1: A Simplified Verilog Module for a Die-to-Die Link#

1
module die2die_link #(
2
    parameter DATA_WIDTH = 64
3
)(
4
    input  wire                  clk,
5
    input  wire                  rst_n,
6
    // TX interface
7
    input  wire [DATA_WIDTH-1:0] tx_data_in,
8
    input  wire                  tx_valid_in,
9
    output wire                  tx_ready_out,
10
    // RX interface
11
    output wire [DATA_WIDTH-1:0] rx_data_out,
12
    output wire                  rx_valid_out,
13
    input  wire                  rx_ready_in,
14
    // Physical IO
15
    output wire [DATA_WIDTH-1:0] serial_out,
16
    input  wire [DATA_WIDTH-1:0] serial_in
17
);
18

19
    // Simple pass-through logic for demonstration
20
    assign serial_out = tx_data_in;
21
    assign rx_data_out = serial_in;
22

23
    // In a real design, you would have SERDES, encoding, error checking
24
    // We skip these details here for brevity
25
    assign tx_ready_out = 1'b1;
26
    assign rx_valid_out = 1'b1;
27

28
endmodule

In a real-world design, this module would be accompanied by:

1. SERDES blocks (serializer/deserializer) to handle high-speed I/O.
2. Error-correcting logic or link-layer protocols.
3. Flow control signals to prevent buffer overruns.

Example 2: C-Style Pseudocode for Configuring an Interconnect#

1
// Constants
2
const int LINK_BANDWIDTH_GBPS = 256;
3
const int MAX_RETRY_ATTEMPTS  = 3;
4

5
// Configuration structure
6
typedef struct {
7
    int bandwidth;
8
    int retry_limit;
9
    bool enable_crc;
10
    bool enable_flow_control;
11
} InterconnectConfig;
12

13
// Initialize interconnect subsystem
14
void initInterconnect(InterconnectConfig *cfg) {
15
    // Configure bandwidth
16
    setBandwidth(cfg->bandwidth);
17

18
    // Configure retry attempts
19
    setRetryLimit(cfg->retry_limit);
20

21
    // Enable optional features
22
    if (cfg->enable_crc) {
23
        enableCRC();
24
    }
25
    if (cfg->enable_flow_control) {
26
        enableFlowControl();
27
    }
28

29
    // More advanced commands might follow...
30
}
31

32
// Example usage
33
int main(void) {
34
    InterconnectConfig config = {
35
        .bandwidth         = LINK_BANDWIDTH_GBPS,
36
        .retry_limit       = MAX_RETRY_ATTEMPTS,
37
        .enable_crc        = true,
38
        .enable_flow_control = true
39
    };
40

41
    initInterconnect(&config);
42

43
    // Now, the rest of the system can start using the link...
44
    return 0;
45
}

AMD’s Chiplet-Based Zen Architecture#

AMD’s Zen CPU architecture heavily relies on chiplet designs. Each chiplet (often called a “Core Complex Die,” or CCD) contains multiple CPU cores and caches, while an I/O die manages memory interfaces and external communication. AMD’s Infinity Fabric ties all these chiplets together, balancing bandwidth and latency.

Intel’s EMIB for Heterogeneous Integration#

Intel uses EMIB in products such as its Stratix 10 FPGAs and certain CPU-GPU combinations. By embedding small silicon bridges in the package substrate, Intel can combine high-performance logic with memory or specialized accelerators on one package. The interconnect is proprietary but offers extremely high bandwidth.

NVIDIA’s NVLink#

NVLink is a high-speed interconnect primarily aimed at connecting GPUs, enabling them to act as a unified pool of resources. In the context of chiplets, NVLink-like designs could be used to scale out multiple accelerator dies. Although not typically referred to as “chiplets,” the principle is similar: high-bandwidth, coherent links that tie chunks of silicon together.

Apple M1/M2 SoC Family#

While Apple’s M-series chips are monolithic, Apple has used advanced packaging to integrate memory inside the package (also known as “package-on-package” design). This approach shows how short interconnect distances can revolutionize performance and power consumption. Future Apple designs may adopt chiplet-like strategies, especially as they aim to scale up performance while extending battery life.

1. Co-Packaged Optics#

To tackle the bandwidth bottleneck, researchers are exploring co-packaged optics (CPO), where photonic interconnects are placed very close to the chip. This approach promises higher bandwidth over longer distances. While not widespread yet, it could become essential for HPC systems needing exascale throughput.

2. Multi-Chip Modules (MCMs) in HPC#

Large-scale HPC installations often use Multi-Chip Modules where multiple CPU chiplets are placed on a single package. Interconnect technologies like InfiniBand or proprietary HPC fabrics then link these packages together across large clusters.

3. AI and Machine Learning Workloads#

As AI models grow in size, the need for rapid data movement between compute units becomes critical. The future of AI accelerators might be a chiplet design featuring multiple specialized compute dies, each responsible for a segment of the neural network, all tied together by ultra-high-speed interconnects.

4. Security in Chiplet Interconnects#

Security poses a unique challenge in disaggregated chiplet environments. Without proper isolation or encryption, malicious actors could intercept data or inject fraudulent traffic. Industry efforts to develop secure enclaves or encrypted data paths between chiplets are ongoing.

5. Semiconductor Manufacturing Evolution#

As advanced nodes become increasingly expensive, we might see new manufacturing breakthroughs that favor chiplet designs even more. For instance, a main CPU complex could be at 3 nm, while a set of specialized accelerators uses a more mature (and cheaper) 14 nm node. This modular approach could accelerate innovation and reduce risks.

6. Standardization Efforts and Ecosystem Growth#

Standards like UCIe will continue to mature, addressing areas like power management, multi-vendor interoperability, and advanced packaging guidelines. This standardization could lead to a flourishing “chiplet marketplace,” where system designers purchase off-the-shelf chiplets that meet specific performance, memory, or protocol needs.