- Meta and Panmnesia propose a new architecture that views data centers as a single large computer.
- The architecture uses CXL to connect CPUs, accelerators, and memory for improved collaborative efficiency.
- The new system reduces data transmission latency, enhancing the execution efficiency of AI workloads.
- Panmnesia has implemented core components in silicon and is preparing for commercial supply.
The design of artificial intelligence data centers may soon shift from being merely a collection of individual machines to functioning more like one massive computer. This is the vision behind a new architectural design proposed by semiconductor company Panmnesia and Meta. The design utilizes Compute Express Link (CXL) to connect central processing units (CPUs), AI accelerators, and memory across racks while maintaining highly coordinated operations. As AI models grow, this issue becomes increasingly critical. Training models with trillions of parameters may require hundreds or thousands of accelerators to exchange several terabytes of data.
Even if most devices complete tasks quickly, any slower component can hold up the entire operation, making the predictability of latency increasingly important.
The Design Philosophy of the New Architecture
Within racks, accelerators can already communicate through dedicated high-speed connections. However, between racks, data typically travels through traditional network equipment and software layers, introducing variability in the time required for individual requests. Panmnesia and Meta's approach extends the CXL domain beyond individual racks into the broader data center. This architecture aims to make resources across the facility work collaboratively rather than treating each rack as an isolated computing island. Three hardware components are crucial to the design: a high-radix non-blocking CXL switch, a link accelerator unit, and a fabric controller.
The combination of these three is designed to reduce the uncertainty of latency when data is transferred between devices.
The Importance of Reducing Latency
The architecture also leverages optical connections to overcome the physical distance limitations of electrical signals, so CXL-over-optics can extend the fabric to a larger portion of the data center without abandoning the underlying CXL model. The proposed system changes the scale at which computing resources work together. In a reference configuration, one CPU is connected to two accelerators. Under the new architecture, this number rises to 16, while a single coherence domain can encompass up to 960 accelerators.
Latency becomes the new battleground. Researchers say that accesses that typically leave the rack and go through traditional networks may now travel along more predictable paths. Round-trip latency could be reduced from the microsecond range to a few hundred nanoseconds, representing a decrease of up to tenfold. This predictability may be as important as raw computing power. Large AI workloads are typically collective operations, meaning the slowest participant can determine when the entire operation proceeds. Reducing these latencies could enable more accelerators to operate as a single execution environment. The architecture may also reduce the impact of failures.
When problems arise, the proposed system will replace individual devices rather than entire servers.
Panmnesia CEO Myoungsoo Jung said, "As AI systems continue to expand, the ability to connect a large number of accelerators and memory devices quickly and efficiently becomes as important as the performance of individual accelerators. This research outlines the direction for the next generation of AI infrastructure, with CXL enabling the entire data center to operate as a single computing system." Panmnesia stated that it has implemented core components in silicon, completed validation, and is preparing for commercial supply.
The findings of the research have been published in the journal Nature Reviews Electrical Engineering.
Item Specification Number of accelerators connected to CPU 16 Number of accelerators that can be covered 960
The Impact of the New Architecture on AI Infrastructure
As AI technology rapidly advances, the design requirements for data centers are also evolving. Meta and Panmnesia's new architecture not only improves data transmission efficiency but also optimizes overall performance by reducing latency, enabling multiple accelerators to work together more effectively. This transformation is crucial for handling large-scale AI workloads, which typically rely on the collaboration of multiple computing units. The implementation of this new architecture is likely to change the operational model of future data centers and provide more efficient solutions for commercial applications.