Machine learning offers two things to quantum control. One is new ways to search: reinforcement-learning agents, neural-network controllers, Bayesian surrogates and tree-search methods. The other is new ways to adapt: control policies that react to measurement outcomes in real time, learned from simulation or directly from the experiment. Since the late 2010s these approaches have matched optimal control on standard tasks. They are especially valuable where classical optimal control struggles: feedback, model-free learning on hardware, and discrete or rugged control problems.

Reinforcement learning in a nutshell

In reinforcement learning (RL), an agent repeatedly chooses an action (the next piece of the control pulse, a gate, a measurement), observes the result, and receives a reward (a fidelity, a measurement outcome). It improves a policy to maximize the expected total reward. Two features make RL attractive here:

  • Model-free learning. The agent needs only rewards, so it can learn on the device itself, without a Hamiltonian;
  • Policies, not pulses. The agent learns a rule that maps observations to actions, so it can react to measurement outcomes, which is exactly what feedback requires.

What has been shown

  • State preparation. Deep RL finds driving protocols in non-integrable many-body systems with performance comparable to optimal-control methods. It also exposed a spin-glass-like transition in the space of protocols as a function of duration [1]; see Control landscapes. A comparative study found that deep Q-learning and policy-gradient methods outperform the other methods tested when the controls are discretized [2].
  • Gates. Agents trained with control noise in the simulation produce fast two-qubit gates that are robust to leakage and stochastic errors [3]. Deep Q-networks have been applied to single- and two-qubit gate control [4]. Machine-learning-based optimizers design high-fidelity single-shot three-qubit gates [5]. On a real superconducting processor, RL designed an error-robust universal gate set without a model of the Hamiltonian, controls or noise [6]. See Quantum gate synthesis.
  • Transport. Deep RL found control sequences for transporting a state through quantum dots that outperform the standard counterintuitive adiabatic sequence, including under detuning, dephasing and losses [7].
  • Feedback and error correction. A neural-network agent discovered complete quantum-error-correction strategies that adapt to measurement outcomes [8]. An agent running as a sub-microsecond-latency neural network on an FPGA has performed real-time feedback, initializing a superconducting qubit and trained only from measurements [9]. See Control for quantum error correction and Control electronics & software stacks.
  • Model-free control of oscillators. Circuit-based RL agents learn to prepare non-classical states of an oscillator coupled to an ancilla, using rewards built from experimentally available measurements [10]; see Bosonic & continuous-variable control.

Beyond reinforcement learning

  • Bayesian optimization builds a probabilistic surrogate of the landscape. It suits expensive, noisy experiments — for example preparing GHZ states in Rydberg arrays [11], optimizing with very few shots [12], or optimizing ultracold-atom experiments online [13]. See Closed-loop & in-situ optimization.
  • Deep exploration. An AlphaZero-style combination of neural networks and tree search explores rugged landscapes without good initial guesses [14].
  • Differentiable controllers. When a model is available, a neural-network controller can be trained by differentiating through the dynamics — model-based RL [15]. Feedback-GRAPE borrows RL ideas to optimize measurement-conditioned strategies while still taking gradients through the quantum dynamics [16]. See Automatic differentiation & differentiable programming.

When to use what

  • Known model, open-loop task. Gradient-based optimal control (GRAPE, GOAT) is usually far more sample-efficient than RL.
  • Feedback, adaptivity, discrete actions, or learning on hardware. RL and related methods come into their own.
  • Expensive, noisy experiments with few parameters. Bayesian optimization.

Tutorials introduce optimal control and RL side by side [17, 18], and a broad review covers machine learning across quantum technologies [19].