✨ Evaluate stochastic RL pass candidates

CHANGELOG.md

@@ -12,16 +12,28 @@ releases may include breaking changes.
 
 ### Added
 
+- ✨ Add opt-in per-pass timeouts for RL training and inference ([#789])
+  ([**@flowerthrower**])
+- ✨ Add Qiskit's `TrivialLayout`, `ElidePermutations`, `SabreSwap`,
+  `BasicSwap`, `LookaheadSwap`, `RemoveIdentityEquivalent`, and
+  `Optimize1qGatesSimpleCommutation` passes to the RL actions ([#785])
+  ([**@flowerthrower**])
+- ✨ Encode the RL qubit-count and depth observations as normalized
+  one-element `float32` arrays ([#784]) ([**@flowerthrower**])
 - ✨ Expand the RL observation with normalized OpenQASM operation frequencies
   and include measurements in the shared ML feature schema ([#758])
   ([**@flowerthrower**])
+- ✨ Evaluate configurable repeated candidates for stochastic Qiskit RL actions
+  and retain the highest-scoring result ([#757]) ([**@flowerthrower**])
 - 👷 Enable testing on Python 3.14 ([#488]) ([**@denialhaag**])
 - ✨ Add selectable `v2` and `v3` RL MDP strategies, make `v3` the default, and
   use the selected strategy in compilation traces and model artifact names
   ([#755]) ([**@flowerthrower**])
 
 ### Changed
 
+- 🐛 Make the `OptimizeCliffords` RL action collect standard Clifford gates
+  before optimizing them ([#785]) ([**@flowerthrower**])
 - 🔥 Drop support for Python 3.10 ([#773]) ([**@denialhaag**])
 - ♻️ Split RL actions package into `base` and `registry` modules ([#769])
   ([**@denialhaag**])
@@ -90,9 +102,13 @@ for previous changelogs._
 
 <!-- PR links -->
 
+[#789]: https://github.com/munich-quantum-toolkit/predictor/pull/789
 [#773]: https://github.com/munich-quantum-toolkit/predictor/pull/771
 [#769]: https://github.com/munich-quantum-toolkit/predictor/pull/769
+[#784]: https://github.com/munich-quantum-toolkit/predictor/pull/784
+[#785]: https://github.com/munich-quantum-toolkit/predictor/pull/785
 [#758]: https://github.com/munich-quantum-toolkit/predictor/pull/758
+[#757]: https://github.com/munich-quantum-toolkit/predictor/pull/757
 [#755]: https://github.com/munich-quantum-toolkit/predictor/pull/755
 [#731]: https://github.com/munich-quantum-toolkit/predictor/pull/731
 [#714]: https://github.com/munich-quantum-toolkit/predictor/pull/714

UPGRADING.md

@@ -6,6 +6,32 @@ of changes including minor and patch releases, please refer to the
 
 ## [Unreleased]
 
+### Expanded Qiskit action set
+
+The RL action space now includes the following Qiskit passes:
+
+- the `TrivialLayout` and `ElidePermutations` layout actions;
+- the `SabreSwap`, `BasicSwap`, and `LookaheadSwap` routing actions; and
+- the `RemoveIdentityEquivalent` and `Optimize1qGatesSimpleCommutation`
+  optimization actions.
+
+`ElidePermutations` establishes a trivial layout in the same action so its
+output permutation remains part of the canonical layout. `OptimizeCliffords` now
+collects standard Clifford gates before optimizing and decomposes the result for
+subsequent passes.
+
+Existing RL models must be retrained because the action-space size and the
+indices of later actions have changed. Code that persists or selects actions by
+numeric index must be updated.
+
+### Compact scalar RL observations
+
+The `num_qubits` and `depth` entries in RL observations are now one-element
+`float32` arrays in `[0, 1]` instead of discrete integers. The qubit count is
+linearly scaled by the target device's qubit count; the depth is `log1p`-scaled
+and capped at 999,999. Existing RL models must be retrained, and code that
+consumes `PredictorEnv` observations directly must handle the new array values.
+
 ### RL operation features
 
 The RL observation now includes normalized frequencies for supported OpenQASM
@@ -18,6 +44,14 @@ Starting with this release, MQT Predictor no longer supports Python 3.10. As a
 result, MQT Predictor is no longer tested under Python 3.10 and requires Python
 3.11 or later.
 
+### Repeated stochastic Qiskit actions
+
+The stochastic `QiskitSabreMapping` action now evaluates 20 candidates by
+default and retains the candidate with the best configured figure of merit. This
+can increase compilation time and change the selected circuit. Set
+`stochastic_action_trials=1` when constructing `PredictorEnv` to retain the
+previous single-attempt behavior.
+
 ### Atomic BQSKit compilation actions
 
 The composite actions `BQSKitO2`, `BQSKitSynthesis`, and `BQSKitMapping` are no

docs/setup.md

@@ -45,9 +45,14 @@ from mqt.bench.targets import get_device
 
 device = get_device("ibm_falcon_27")
 rl_pred = RL_Predictor(device=device, figure_of_merit="expected_fidelity")
-rl_pred.train_model(timesteps=100000)
+rl_pred.train_model(timesteps=100000, pass_timeout=600)
 ```
 
+`pass_timeout` is optional and limits each compilation pass during training. If
+it is omitted, pass execution is unbounded. Pass timeouts require POSIX signals
+and execution on the main thread; unsupported environments emit a warning and
+continue without a timeout.
+
 Currently, the following figures of merit are supported:
 
 ```{code-cell} ipython3
@@ -121,9 +126,16 @@ from mqt.predictor import qcompile
 from mqt.bench import get_benchmark, BenchmarkLevel
 
 uncompiled_qc = get_benchmark("ghz", level=BenchmarkLevel.ALG, circuit_size=5)
-compiled_qc, compilation_info, selected_device = qcompile(uncompiled_qc, figure_of_merit="expected_fidelity")
+compiled_qc, compilation_info, selected_device = qcompile(
+    uncompiled_qc,
+    figure_of_merit="expected_fidelity",
+    pass_timeout=60,
+)
 ```
 
+Inference has its own optional `pass_timeout`, so it can use a different limit
+than training.
+
 This returns:
 
 - the compiled quantum circuit,

src/mqt/predictor/qcompile.py

@@ -27,19 +27,29 @@ def qcompile(
     qc: QuantumCircuit,
     figure_of_merit: figure_of_merit = "expected_fidelity",
     tracer_output_path: str | Path | None = None,
+    pass_timeout: float | None = None,
 ) -> tuple[QuantumCircuit, list[str], str]:
     """Compiles a given quantum circuit to a device with the highest predicted figure of merit.
 
     Arguments:
         qc: The quantum circuit to be compiled.
         figure_of_merit: The figure of merit to be used for compilation. Defaults to "expected_fidelity".
         tracer_output_path: If provided, enables compiler tracing and exports the JSON log to this path/directory.
+        pass_timeout: Maximum duration in seconds for one compilation pass.
+            Defaults to None, which disables pass timeouts.
 
     Returns:
         A tuple containing the compiled quantum circuit, the compilation information, and the name of the device used for compilation.
+
+    Raises:
+        ValueError: If ``pass_timeout`` is not positive.
     """
     predicted_device = predict_device_for_figure_of_merit(qc, figure_of_merit=figure_of_merit)
     res = rl_compile(
-        qc, device=predicted_device, figure_of_merit=figure_of_merit, tracer_output_path=tracer_output_path
+        qc,
+        device=predicted_device,
+        figure_of_merit=figure_of_merit,
+        tracer_output_path=tracer_output_path,
+        pass_timeout=pass_timeout,
     )
     return *res, predicted_device

src/mqt/predictor/rl/actions/base.py

@@ -52,6 +52,7 @@ class Action:
         preserves_layout: Whether action preserves existing layout.
         preserves_routing: Whether action preserves existing routing.
         preserves_synthesis: Whether action preserves synthesis state.
+        stochastic: Whether repeated execution can yield different results.
     """
 
     name: str
@@ -61,6 +62,7 @@ class Action:
     preserves_layout: bool = False
     preserves_routing: bool = False
     preserves_synthesis: bool = False
+    stochastic: bool = False
 
 
 @dataclass

src/mqt/predictor/rl/actions/bqskit_actions.py

@@ -39,6 +39,8 @@
     IfThenElsePass,
     LEAPSynthesisPass,
     ManyQuditGatesPredicate,
+    MGDPass,
+    QSDPass,
     QSearchSynthesisPass,
     RestoreMeasurements,
     SetModelPass,
@@ -309,6 +311,18 @@ def bqskit_synthesis_actions() -> list[Action]:
                 IfThenElsePass(DiagonalPredicate(1e-9), WalshDiagonalSynthesisPass()),
             ),
         ),
+        DeferredDeviceAction(
+            "QSDPass",
+            CompilationOrigin.BQSKIT,
+            PassType.SYNTHESIS,
+            transpile_pass=lambda device: _bqskit_partitioned_synthesis_factory(device, QSDPass()),
+        ),
+        DeferredDeviceAction(
+            "MGDPass",
+            CompilationOrigin.BQSKIT,
+            PassType.SYNTHESIS,
+            transpile_pass=lambda device: _bqskit_partitioned_synthesis_factory(device, MGDPass()),
+        ),
         DeferredDeviceAction(
             "FullQSDPass",
             CompilationOrigin.BQSKIT,