Introduce QUARK framework integration

[armalikhani]

This PR introduces QUARK framework integration into the MQSS Benchmarking Framework as a benchmark pipeline implementation.

Note: This is an initial experimental version intended for testing and further refinement.

Changes

QUARK integration

• Add QUARK integration as a built-in benchmark pipeline plugin
• Add support for QUARK raw config files and inline configuration dictionaries
• Dynamically resolve and install required QUARK plugins from benchmark configuration
• Add an APPLICATION benchmark category
• Add a QUARK TSP benchmark example using external configuration
• Add an inline QUARK configuration example
• Add an install_plugins flag to control whether required QUARK plugins are installed
• Add a failfast flag to control benchmark failure handling
• Refactor and polish the built-in QUARK plugin implementation
• Update MBF example configurations for the new QUARK parameters
• Update tests for compatibility with the new functionality

QUARK result analysis and artifacts

• Add selected important metrics to QUARK result analysis:

  • successful_tree_runs
  • successful_pipeline_runs
  • average_runtime
  • average_result
  • additional_metrics

• Add the QUARK plotting artifact results.pdf to the MBF output

• Add a result field to the QUARK results JSON

• Include raw QUARK execution results in stored results and JSON output

Execution result model

• Refactor the existing ExecutionResult to CircuitExecutionResult across the framework for improved clarity and consistency
• Add a PipelineExecutionRecord type for pipeline engine execution results
• Define ExecutionResult as a union of CircuitExecutionResult and PipelineEngineExecutionResult
• Use the unified ExecutionResult type in PipelineResult.execution_results
• Update execution-result handling accordingly

CLI output

• Add brief per-execution CLI output containing:

  • Pipeline
  • Pipeline Status
  • Result
  • Runtime

• Keep detailed execution results available in the stored JSON output to avoid console clutter

Note: The essential extensions to the core MQSS Benchmarking Framework have already been merged into the default branch, including #52. The supplementary changes for the future entry-point-based, pip-installable plugin package and unified plugin system have also been merged in #54, and this branch has been updated accordingly.

This PR focuses on the QUARK integration and its associated result handling. In this version, QUARK is implemented as a built-in plugin. It can later be moved into a pip-installable plugin package.

Additional notes

Note: This version is intentionally limited to the initial QUARK integration. Additional tests, refinements, and improvements will be addressed in a separate PR or as part of the next implementation phase.

Note: Currently, specifying an adapter in the MBF configuration is mandatory. For this version, qiskit_simulator is included in the MBF configuration only to satisfy this requirement and prevent configuration errors. It is not actually used by the QUARK pipeline and effectively serves as a dummy configuration value. Future work will make the adapter field optional in the MBF configuration.

Note: This change modifies the core API. Existing users of previous versions may need minor adaptations to their implementations to remain compatible.

Issues resolved:
closes #64
closes #61
closes #60
closes #9

Future work:

• Further refine and expose additional QUARK pipeline metrics in benchmark results
• Consider adding support for passing environment objects, configuration, builtins and runtime context from the MBF (as orchestrator) to QUARK
• Improve adapter handling for adapter-free QUARK workflows
• Add support for analyzing and visualizing QUARK results within MBF
• Extend tests and add dedicated documentations for QUARK

Attachments:

Initial QUARK Integration into the MQSS Benchmarking Framework.pdf