Google’s Hesitancy to Compare Willow on Traditional Quantum Benchmarks

When Google unveiled Willow, its next-generation quantum processor, the tech giant made headlines not just for the processor’s advanced capacity and features, but also for its surprising decision to forgo traditional quantum benchmarks in favor of focusing exclusively on Random Circuit Sampling (RCS). This raised questions in the quantum computing community: why wouldn’t Google compare Willow against other well-established benchmarks, such as Quantum Volume, gate fidelity, or coherence times?

A New Era of Quantum Computing

Willow represents a significant leap in quantum processor development, promising breakthroughs in speed and computational capability. Yet, despite its announced features, Google opted not to demonstrate Willow running on traditional metrics that quantum researchers and hardware developers typically use to gauge performance. The company’s decision to stick with Random Circuit Sampling (RCS) as the sole benchmark raised some eyebrows.

To understand why, it’s essential to appreciate how quantum computing operates in fundamentally different ways from classical computing, and how this impacts the choice of benchmarks.

The Challenge with Traditional Benchmarks

1. Quantum Volume: One of the most prominent benchmarks in the quantum world, Quantum Volume, introduced by IBM, combines a variety of quantum properties such as the number of qubits, gate fidelity, and qubit connectivity. This metric is meant to represent the overall computational capability of a quantum processor. Quantum Volume’s intention is clear: it measures the largest random quantum circuit a quantum processor can handle successfully. But for Google, showing Willow’s performance using Quantum Volume might not highlight its most advanced features.Why not Quantum Volume? For one, Willow might not yet be scalable enough to push its Quantum Volume to the levels required for meaningful comparisons. Early-stage quantum processors are still maturing, and benchmarking them against Quantum Volume could potentially make Google’s processor seem underwhelming. With NISQ (Noisy Intermediate-Scale Quantum) devices still in their infancy, focusing on Quantum Volume might risk underplaying the novel approaches Google has taken in designing Willow.
2. Gate Fidelity: Another commonly used benchmark is gate fidelity, which measures how accurately quantum gates are executed on the quantum processor. Fidelity is essential because quantum computations depend on the precision of quantum gate operations. However, quantum computers are inherently noisy, and as quantum systems scale up, maintaining high fidelity becomes increasingly difficult.Why not Gate Fidelity? Google’s reluctance to use gate fidelity as a primary metric could stem from the limitations of current quantum processors, including Willow. While gate fidelity is certainly a crucial measure for quantum system optimization, Willow may not yet achieve the consistency required to be judged favorably on this benchmark. The focus on RCS, however, can still showcase the processor’s potential without getting bogged down by the complexities of error rates or fidelity losses that typically come with large-scale quantum systems.
3. Coherence Time (T1 and T2): Coherence times, both T1 (relaxation time) and T2 (dephasing time), are another crucial set of benchmarks for assessing the stability of qubits. Longer coherence times allow for more complex quantum circuits and deeper algorithms to be run before errors creep in. The longer a qubit can maintain its state, the more powerful a quantum processor can be.Why not Coherence Time? Google’s Willow might not have reached the levels of coherence time that would set it apart from other systems. The performance of quantum processors is still limited by noise and decoherence, so Willow may be at a stage where this metric doesn’t yet fully capture its capabilities. In this early phase, Google might have felt that focusing on a performance task like RCS, which emphasizes quantum computational advantage rather than system limitations, would be more effective.
4. Error Rates: One of the most pressing challenges in quantum computing is quantum error rates. Current quantum processors are highly prone to errors due to the delicate nature of qubits. Error correction protocols are still in their infancy, meaning that high error rates are inevitable at this stage.Why not Error Rates? Discussing error rates in the context of Willow would almost certainly highlight the processor’s imperfections. As quantum error correction algorithms are still evolving, Google may have opted to avoid focusing on error rates to prevent drawing attention to the processor’s susceptibility to noise. Instead, by emphasizing RCS, Google showcases its quantum processor’s potential to execute tasks that are notoriously difficult for classical systems to simulate.

RCS: A Benchmark with Purpose

Random Circuit Sampling (RCS), however, is a benchmark that sidesteps many of the complexities of traditional quantum metrics. The task involves sampling from the output distribution of random quantum circuits—a problem that is exceedingly difficult for classical systems to simulate efficiently as the complexity grows exponentially with the number of qubits. By focusing on RCS, Google could effectively demonstrate Willow‘s ability to handle quantum mechanical phenomena like entanglement and superposition in a way that classical systems simply cannot.

While RCS doesn’t offer a holistic view of the quantum processor’s capabilities, it provides a clear and compelling metric for quantum advantage—the demonstration that a quantum processor can perform a task that no classical computer could feasibly handle. By focusing on RCS, Google highlights Willow’s potential to reach beyond the capabilities of classical hardware without being weighed down by the limitations of current benchmarks.

The Strategic Choice

So, why did Google choose to highlight only RCS? The answer lies in timing and messaging. Google is likely aware that quantum hardware is still very much in the early stages of development. Benchmarks like Quantum Volume, gate fidelity, and coherence time may not yet provide a full or fair representation of Willow‘s capabilities, especially as the processor is still evolving. By sticking with RCS, Google is setting expectations for quantum computing’s unique advantages without overpromising on performance. It also avoids comparisons that could make the quantum processor seem underperforming in areas where improvements are still needed.

Moreover, RCS allows Google to focus on the future potential of quantum computing, rather than be bogged down in the imperfections of current hardware. It’s a way to draw a clear line between the promise of quantum systems and the challenges that remain, all while showcasing something that classical systems cannot replicate.

We can safely assume that Google’s decision to highlight Random Circuit Sampling is less about withholding information and more about strategically focusing on a task that demonstrates Willow’s strengths while sidestepping the shortcomings that still plague the quantum industry.