Credo Technology Group Holding Ltd (NASDAQ: CRDO) has announced plans to standardize a new interconnect within the Open Compute Project (OCP) community to address the "memory wall." This technical bottleneck occurs when processor speeds outpace memory data supply, leaving expensive compute resources idle. By contributing its OmniConnect lightweight AXI framer specification to the OCP Open Chiplet Economy (OCE) Lightweight Serial Interconnect (LSI) Workstream, Credo aims to foster an open ecosystem for memory disaggregation and chip-to-chip communications, specifically targeting the constraints found in modern AI inference environments.
Credo Leads OCP LSI Workstream Initiative
To combat the limitations of current AI infrastructure, Credo has established the OCP Open Chiplet Economy (OCE) Lightweight Serial Interconnect (LSI) Workstream. This initiative operates under the OCP Server Project and focuses on developing interconnect solutions to overcome memory bottlenecks that currently limit AI inference scalability. Credo intends to contribute its OmniConnect lightweight AXI framer specification to this workstream, facilitating an interoperable ecosystem of products from both Credo and third-party providers. This standardization effort is designed to support the industry's transition toward composable infrastructure, where compute, memory, and storage are optimally combined. By advancing an open approach to lightweight framing and serial interconnect, the goal is to enable flexible chiplet integration, including near-package pluggable configurations. This movement seeks to align the industry around architectures that reduce complexity while meeting the high-performance requirements of next-generation AI infrastructure and modular, power-optimized architectures.
Addressing HBM Constraints via OmniConnect
The industry currently faces significant challenges with High-Bandwidth Memory (HBM), which suffers from high costs, limited availability, and density constraints. In modern AI inference, memory bandwidth and capacity often serve as the primary performance constraints rather than raw compute power. Credo’s OmniConnect versatile AXI-over-VSR (Very Short Reach) SerDes bus is positioned to mitigate these issues by enabling both die-to-die interconnect and scale-up networking. This technology allows for the connection of multiple compute engines via external chiplets. By enabling composable AI architectures, this standardized interconnect could deliver up to 25x greater memory density and 5% higher bandwidth than HBM4. Such advancements allow system designers to flexibly compose domain-specific architectures optimized for evolving AI models and diverse data flows. Ultimately, this approach aims to reduce the heavy reliance on HBM, promoting higher performance, better resource utilization, and improved cost efficiency for large-scale AI deployments.
Key Takeaways
- Credo is contributing its OmniConnect lightweight AXI framer specification to the OCP LSI Workstream to standardize chip-to-chip communications.
- The proposed architecture aims to provide up to 25x greater memory density and 5% higher bandwidth compared to HBM4.
- The initiative targets the "memory wall" to prevent expensive compute resources from sitting idle due to slow memory data supply.
TechInsyte's Take
In our view, Credo’s move to standardize its OmniConnect specification within the OCP is a strategic attempt to pivot the industry away from the HBM monopoly. By addressing the "memory wall" through memory disaggregation and chiplet-based modularity, Credo is not just selling a product but is attempting to define the architectural standard for the next generation of AI inference. This signals a shift toward composable infrastructure where hardware can be tailored to specific workloads. For decision-makers, this suggests that the future of AI scaling may rely less on monolithic, expensive HBM stacks and more on flexible, interoperable, and high-density chiplet ecosystems.
Questions & Answers
How does the OCP LSI Workstream address the "memory wall" in AI inference?
The workstream develops interconnect solutions to bridge the gap between processor speed and memory access speeds. By enabling memory disaggregation and chip-to-chip communications, it prevents compute resources from idling while waiting for data.
What are the specific performance advantages of this new interconnect over HBM4?
The initiative aims to enable composable AI architectures that can deliver up to 25x greater memory density and 5% higher bandwidth than HBM4, while reducing dependence on costly and capacity-constrained HBM.
What role does the OmniConnect specification play in this ecosystem?
Credo’s OmniConnect lightweight AXI framer specification serves as the foundational technology for the standardized interconnect. It enables both die-to-die interconnect and scale-up networking by connecting multiple compute engines via external chiplets.
Why is the transition to composable infrastructure important for AI scaling?
Composable infrastructure allows compute, memory, and storage to be optimally combined. This flexibility enables designers to create domain-specific architectures for evolving AI models, leading to better resource utilization and cost efficiency.
Source: BUSINESSWIRE
