Quantum Program Testing: Assertions, Circuit Equivalence, and Regression Checks for Qiskit and Cirq
Quantum circuits can change behavior when you refactor gates, alter measurement code, upgrade an SDK, or send a circuit through a different transpiler. This guide presents a reusable quantum circuit testing workflow for Qiskit and Cirq, including deterministic assertions, probability tolerances, circuit-equivalence checks, seeded simulations, property-based cases, and hardware-aware regression tests.
Overview
Testing a quantum program is not the same as checking a conventional function that returns one fixed value. Measurement is probabilistic, simulators may represent results differently, and transpilation can replace the original gate sequence with an equivalent implementation. A useful test suite therefore checks several layers of behavior rather than relying on one expected bit string.
The first layer is structural: does the circuit contain the expected qubits, measurements, gates, and parameters? The second is mathematical: does an ideal statevector, probability distribution, or observable match the expected result within a defined tolerance? The third is operational: does the transpiled or hardware-executed circuit still meet its acceptance criteria under finite shots and noise?
Keep these layers separate. A statevector test is appropriate for an ideal simulator, but it does not prove that a noisy device will produce the same distribution. Conversely, a hardware test should not fail merely because sampling produces a slightly different count on each run. Your assertions should reflect the execution mode being tested.
For background on interpreting counts and probabilities, see the Quantum Measurement and Probability Distributions guide. If you are debugging an unexpected result, the Quantum Circuit Debugging guide provides a complementary workflow.
Checklist by scenario
1. Test circuit construction before execution
Start with inexpensive tests that do not call a simulator or device. Verify the number of qubits and classical bits, required measurements, parameter names, and important gate placement. These checks catch accidental changes early.
- Confirm the circuit has the expected width and measurement mapping.
- Check that required parameters are present and bound before execution.
- Assert that forbidden operations, such as an unintended measurement in the middle of a reusable subcircuit, are absent.
- Record resource expectations such as depth, two-qubit gate count, and total operations when they matter to performance.
Resource assertions should usually use an upper bound rather than exact equality. A transpiler may produce a different but valid decomposition after an SDK or backend change. The Quantum Circuit Resource Estimation guide can help define which metrics are worth tracking.
2. Assert ideal states and probabilities
For small circuits, compare the statevector or probability distribution with a hand-derived expectation. For example, applying an X gate to a zero-initialized qubit should produce a probability close to one for the state |1⟩. A Bell-state circuit should assign approximately equal probability to 00 and 11 and near-zero probability to 01 and 10.
Use numerical tolerances instead of direct floating-point equality. A practical starting point is an absolute tolerance around 1e-8 for deterministic, ideal statevector calculations and a looser tolerance when transformations or numerical approximations are involved. Treat these values as project settings, not universal laws; validate them against your simulator and circuit size.
Illustrative Python-style pseudocode looks like this:
probabilities = run_ideal_simulator(circuit)
assert abs(probabilities["00"] - 0.5) <= atol
assert abs(probabilities["11"] - 0.5) <= atol
assert probabilities.get("01", 0.0) <= atol
assert probabilities.get("10", 0.0) <= atolNormalize keys before comparing results. Qiskit and Cirq can differ in bit ordering, register representation, or result formatting. A test that compares raw dictionaries without an explicit convention can report a false failure.
3. Test finite-shot measurements with statistical tolerance
Shot-based tests should compare observed frequencies with an expected distribution, not with one exact count. If an outcome is expected with probability p over N shots, its observed frequency will vary. Define an allowed error using a project-specific statistical rule, such as a fixed frequency margin for simple tests or a confidence interval for more formal validation.
For every shot-based test, document the shot count, random seed where supported, expected distribution, and acceptance threshold. A low-shot smoke test can catch broken measurement wiring, while a higher-shot validation test provides a more meaningful distribution comparison. Do not silently increase the tolerance until a failing test passes; investigate whether the circuit, backend, or test assumption changed.
4. Compare observables for algorithmic tests
Many algorithms do not require the full statevector. Variational algorithms, chemistry experiments, and hybrid quantum-classical workflows often care about an expectation value such as the average of a Pauli operator or an objective function. Test that observable directly, with a tolerance appropriate to the estimator and shot noise.
For example, a VQE regression test might assert that the energy remains below a threshold or that it stays within a defined interval around a reference value. This is often more stable and more meaningful than asserting every amplitude. See the VQE tutorial and the hybrid quantum-classical workflow guide for contexts where observable-level testing is useful.
5. Check circuit equivalence deliberately
Two circuits can have different gate lists but implement the same unitary operation. When refactoring a reusable circuit, compare behavior rather than source text. For small circuits, simulate both versions and compare statevectors up to global phase. If the circuits include measurements, compare their probability distributions for representative inputs instead.
Equivalence tests should specify their scope. Are the circuits equivalent for one fixed input, for all computational-basis inputs, or as unitaries over the entire Hilbert space? A fixed-input comparison may miss errors that appear on another input. For a small component, test several basis states or random normalized states. For larger components, compare selected observables or use a trusted reference implementation.
6. Use seeded simulations for reproducible failures
Seeds can make simulator-based tests repeatable, but they do not make a noisy physical device deterministic. Set seeds for pseudorandom sampling, noise generation, and parameter initialization where the framework supports those controls. Store the seed in the failure output so a developer can reproduce the case locally.
Use deterministic seeds for unit tests and vary seeds in a separate robustness or property-based test. A test suite that uses only one seed may accidentally validate one favorable sample path.
7. Add property-based test cases
Property-based testing checks rules that should hold across many generated circuits or parameters. Useful quantum properties include:
- Applying a gate followed by its inverse returns the input state, within tolerance.
- Probabilities remain normalized and non-negative after numerical cleanup.
- Adding an idle qubit does not change the marginal distribution of the original subsystem.
- Equivalent parameterizations produce the same observable.
- A circuit and its inverse produce the expected identity behavior when composed.
Keep generated circuits small at first. Limit qubit count, depth, parameter range, and gate alphabet so failures remain understandable. Save the random seed and the minimized failing circuit as test artifacts.
Qiskit and Cirq testing patterns
| Testing need | Qiskit pattern | Cirq pattern |
|---|---|---|
| Build validation | Inspect circuit width, registers, instructions, parameters, and measurements. | Inspect qubit set, moments, operations, measurements, and symbols. |
| Ideal behavior | Run an appropriate statevector or shot-based simulator and normalize result keys. | Use a simulator for state or sampled results and make qubit order explicit. |
| Transpilation | Test the transpiled circuit separately from the source circuit; track resource limits. | Test transformed circuits separately and verify that the chosen transformer preserves the intended behavior. |
| Parameters | Bind parameters explicitly and test representative assignments. | Use a resolver for symbols and test both symbolic construction and resolved execution. |
| Hardware behavior | Use backend-specific calibration, coupling, measurement, and shot-aware acceptance criteria. | Account for sampler or device result formats, supported gates, and device noise. |
API names and result objects can change across releases. Treat the table as a testing pattern, then confirm the exact simulator, sampler, transpiler, and result APIs in the versions pinned by your project.
What to double-check
Before approving a circuit change, review the following test-plan template:
- Behavior: What should the circuit return—state amplitudes, probabilities, counts, an observable, or an optimization objective?
- Reference: Is the expected result derived analytically, produced by a trusted implementation, or recorded from a reviewed baseline?
- Representation: Are qubit order, classical-bit order, endianness, parameter names, and measurement mapping explicit?
- Tolerance: Is the threshold appropriate for ideal arithmetic, finite shots, noise, and the scale of the observable?
- Execution layer: Are source, transformed, simulator, and hardware tests distinct?
- Reproducibility: Are dependency versions, backend configuration, shots, seeds, and circuit serialization recorded?
- Failure evidence: Will a failure print the circuit, parameters, seed, counts, expected values, and actual values?
For hardware-aware checks, begin with a simulator baseline, then run a small calibration or smoke job, and finally evaluate the production-sized circuit if the experiment warrants it. Compare trends and acceptance ranges rather than expecting hardware counts to match an ideal simulator exactly. Noise, connectivity constraints, readout behavior, queue-time configuration, and transpilation can all affect results. The Quantum Hardware Job Submission Checklist covers operational checks that should accompany these tests.
Common mistakes
- Asserting exact shot counts: Sampling variation makes this brittle. Compare frequencies or observables with a declared tolerance.
- Comparing raw result keys: Different bit-order conventions can make equivalent results look different. Convert to one canonical format first.
- Testing only the original circuit: A correct source circuit can behave differently after transpilation. Test the transformed artifact when it is part of the workflow.
- Using one tolerance everywhere: Statevector, probability, observable, and hardware tests have different error sources.
- Confusing global phase with a functional difference: Statevectors that differ only by global phase can represent the same physical state.
- Relying on one random seed: A fixed seed improves diagnosis but does not measure robustness.
- Treating a simulator as hardware: An ideal simulator cannot validate device noise, readout errors, or connectivity constraints.
- Ignoring resource regressions: A mathematically correct refactor may add depth or two-qubit gates beyond an operational budget.
When to revisit
Review this testing plan whenever you add a gate family, change measurement layout, introduce parameterized circuits, alter transpiler settings, switch simulators, target a new device, or upgrade Qiskit, Cirq, a provider package, or a numerical dependency. A framework upgrade can change result objects, default transformations, supported operations, or numerical behavior even when your application code appears unchanged.
At each release, run the suite in this order: structural tests, ideal reference tests, circuit-equivalence tests, transformed-circuit tests, shot-based simulator tests, and hardware smoke tests. Compare both correctness and resources. Record any intentional baseline change in the test plan rather than simply replacing the old expected result.
As a maintenance habit, review tolerances and reference circuits at least twice a year and before major planning or release cycles. Keep a small set of canonical circuits—a single-qubit rotation, Bell state, inverse pair, parameterized circuit, and one representative algorithm circuit. These become early-warning tests when tools or workflows change. For algorithm-specific examples, the Grover’s Algorithm tutorial and quantum error mitigation benchmarking guide offer useful contexts for extending the checklist.
The practical goal is not to make every quantum test perfectly deterministic. It is to make expected variation explicit, preserve reproducible evidence, and detect meaningful changes before they reach a device or a production workflow.