Qualcomm Challenges HBM Dependence in AI Inference

Qualcomm has introduced High-Bandwidth Compute (HBC), a near-memory compute architecture, as part of its Dragonfly data center roadmap. The technology is designed to address the “memory wall,” where Artificial Intelligence (AI) performance is increasingly constrained by memory bandwidth, memory capacity, and the energy cost of moving data between memory and compute.

The architecture separates the AI accelerator from the main System-on-Chip (SoC) and places compute directly beneath a Low Power Double Data Rate (LPDDR) Dynamic Random-Access Memory (DRAM) stack. The HBC accelerator connects to the LPDDR stack via Through-Silicon Vias (TSVs), allowing more data movement within the memory package rather than across the full path between the DRAM and accelerator. Qualcomm frames this as a way to deliver some of the speed advantages of Static Random-Access Memory (SRAM) with the density of stacked DRAM, while reducing reliance on expensive High-Bandwidth Memory (HBM)-based packaging.

Qualcomm says HBC Gen 1 will be used in the AI250 accelerator, which is expected to sample in mid-2027. The company claims that AI250 with HBC Gen 1 can deliver 133 Terabytes per Second (TB/s) of effective memory bandwidth per card, representing an 18X increase versus AI200 with LPDDR5X. Qualcomm also says HBC can deliver 6X higher bandwidth per watt than HBM and 200X higher capacity per watt than SRAM. A second-generation HBC design is planned for AI300 in 2028, with Qualcomm claiming a 54X improvement in effective bandwidth over AI200.

Qualcomm’s HBC announcement reflects a broader shift in AI hardware competition. Earlier AI infrastructure discussions were often centered on accelerator performance, cluster scale, and raw compute. HBC signals that inference hardware is moving toward a different value equation, where the most important constraints are memory bandwidth, data movement, energy per token, latency consistency, and Total Cost of Ownership (TCO).

This is especially relevant for Agentic AI workloads. Inference has progressed from one-shot answers and the relatively lower token usage of earlier models without reasoning capabilities. A single user request may involve multiple model calls, retrieval steps, tool interactions, memory lookups, and agent-to-agent exchanges. These workflows can make memory movement and latency just as important as accelerator performance. In this environment, the winning architecture is beginning to pivot toward a system that can efficiently serve tokens at scale.

HBC is Qualcomm’s attempt to address this problem on the memory side, but more importantly, it is a bet that the industry has been solving the wrong problem for too long. Instead of continuing to scale HBM capacity or inflate on-chip SRAM, Qualcomm is effectively saying that the current Graphics Processing Unit (GPU)-centric architecture is structurally inefficient for inference. Bringing compute closer to DRAM challenges the assumption that more bandwidth and bigger accelerators will solve inference economics. The “memory wall” discussion has mainly focused on the shortage. Still, it is now becoming clear that it is also a symptom of an architecture that incurs excessive off-chip data movement, high memory access latency, and significant energy overhead for each transfer.

The memory crunch makes this more strategically important. HBM remains the default for high-end AI systems, but it is expensive, supply-constrained, and tightly coupled with advanced packaging bottlenecks. For Qualcomm, HBC is, therefore, not only an architectural choice but also a commercial strategy to reduce dependence on a memory technology that may remain constrained for several years. By using LPDDR-based near-memory compute rather than HBM-heavy designs, Qualcomm can potentially lower system cost, improve power efficiency, and protect its AI accelerator roadmap from HBM availability risks. CXMT’s DRAM expansion, Micron’s long-term supply deals, and persistent shortages all reinforce the same market signal: memory access is becoming a control point, and vendors are differentiating by how they secure, use, or even avoid the most constrained memory resources.

The comparison with CXMT highlights this divide more clearly. CXMT is scaling DRAM supply, which helps alleviate capacity constraints but does not fundamentally change how AI systems consume memory. Qualcomm, on the other hand, is trying to redefine that consumption model. One strategy expands supply within the current framework, while the other seeks to fundamentally reshape it. If HBC works, it would place Qualcomm alongside other alternative AI architectures, such as Cerebras’ wafer-scale SRAM approach and SambaNova’s SN50 Reconfigurable Dataflow Unit (RDU), in showing that the industry is beginning to rethink how closely compute should sit to data, rather than simply feeding more bandwidth into GPU-centric systems.

HBC could also reshape demand in ways that are not immediately obvious. If near-memory compute becomes viable, it would redistribute the pressure. Demand could shift toward LPDDR stacks, TSV integration, and more complex packaging ecosystems, potentially creating new bottlenecks, rather than removing old ones. In that sense, HBC may further fragment the memory landscape, forcing suppliers and customers to navigate multiple memory architectures instead of converging on HBM.

The real question is whether HBC can break the industry’s reliance on GPU-centric inference design. Qualcomm’s claims are aggressive, but the harder challenge is ecosystem inertia. Software stacks, developer tooling, and hyperscaler infrastructure are deeply optimized around existing architectures. HBC may be technically compelling for certain workloads like decode-heavy inference or long-context serving. Still, unless it delivers a clear and sustained cost advantage (with the supply capacity to satisfy demand and displace the entrenched incumbents, i.e., NVIDIA/reach scale/momentum and this is dependent on developer uptake of Qualcomm’s relatively immature stack), it risks being treated as a niche alternative, rather than a new standard.

Silicon vendors should treat Qualcomm’s HBC announcement as a signal that AI inference hardware is becoming more memory-centric. Peak performance will still matter, but customers are increasingly evaluating AI infrastructure in terms of tokens per watt, tokens per dollar, latency consistency, memory bandwidth, and deployable efficiency. Vendors that continue to market accelerators only around raw compute risk missing where the inference bottleneck is moving.

The first area to watch is whether near-memory compute becomes a credible alternative to HBM-heavy inference systems. HBC does not need to replace HBM across all workloads to be relevant. If it can reduce cost and power for high-volume inference, it could create a separate design lane for latency-sensitive and power-constrained serving environments. This would reinforce the broader split between training systems optimized for large-scale training runs on interconnected mega-clusters and smaller inference systems optimized for memory efficiency and economics.

The second area to watch is memory supplier strategy. HBC depends on LPDDR, stacked DRAM, TSV integration, and close coordination with suppliers. DRAM suppliers already participate in AI infrastructure through HBM and system DRAM integrated into Central Processing Units (CPUs). Still, HBC could create a more specialized role for LPDDR-based memory inside inference accelerators. This could matter for suppliers seeking to capture AI demand across a wider memory portfolio, especially as conventional DRAM, server DRAM, and HBM markets all tighten.

The third area to watch is hyperscaler adoption. HBC has to prove itself inside real inference infrastructure. Qualcomm is positioning Dragonfly as a rack-scale inference platform, which means adoption will depend on software support, networking, cooling, orchestration, and compatibility with existing cloud deployment models. For silicon vendors, this is the harder test. A new memory architecture may improve bandwidth and efficiency on paper. Still, hyperscalers will adopt it only if it lowers the cost and power of serving AI workloads at scale (for internal and external customer workloads).

The broader takeaway is that AI silicon is moving from compute-centric design toward memory-aware system design. HBM, SRAM, LPDDR, interconnect, packaging, and inference-serving software are becoming part of the same competitive equation. Qualcomm’s HBC is important because it reframes the inference bottleneck as data movement. For the silicon ecosystem, the next phase of AI hardware competition may be less about adding compute and more about putting the right compute closer to the right memory.