diff --git a/_static/web-dashboard.png b/_static/web-dashboard.png new file mode 100644 index 0000000..c91bf03 Binary files /dev/null and b/_static/web-dashboard.png differ diff --git a/agent.rst b/agent.rst index 646424b..1f7d1c1 100644 --- a/agent.rst +++ b/agent.rst @@ -18,3 +18,31 @@ The QUANT-NET Agent that registers resources, interfaces with devices, and inter :prog: quantnet_agent :nested: full +Monitoring Events +----------------- + +The agent scheduler publishes ``MonitorEvent`` messages to the ``monitor`` +topic as tasks complete. Each event includes: + +- ``eventType``: ``"agentTaskResult"`` — identifies the message as a local + task result (distinct from a full experiment result). +- ``value.name``: the task name. +- ``value.exp_id``: the experiment/task ID, used by the controller to + correlate results when queried via ``getTasks``. +- ``value.result``: the task result payload (when present). + +Configuration +------------- + +The agent reads its configuration from an INI file. A path can be provided +explicitly with the ``-c``/``--config`` CLI flag; if the file does not exist +the agent exits immediately with an error. + +Without ``-c``, the config path depends on the environment: + + * If ``$QUANTNET_HOME`` is set: ``$QUANTNET_HOME/etc/agent.cfg`` + * Otherwise: ``/etc/quantnet/agent.cfg`` + +If no config file is found the agent continues with defaults and logs a +warning. Message broker log output is suppressed to warning level by default. + diff --git a/api.rst b/api.rst new file mode 100644 index 0000000..b97f0d6 --- /dev/null +++ b/api.rst @@ -0,0 +1,286 @@ +Northbound API +############## + +.. pypi-shield:: + :project: quantnet-api + :version: + + +.. github-shield:: + :last-commit: + :repository: qn-api + :branch: main + + +The northbound API server provides a RESTful interface for external applications to interact with the QUANT-NET control plane. It exposes endpoints for querying topology information, submitting experiments, and retrieving results. The API is designed to be intuitive and easy to use, enabling seamless integration with third-party tools and dashboards. + + +Running the API Server +---------------------- + +CLI +~~~ + +When installed as a package (``pip install quantnet-api``), the ``qn-api`` +console script is available: + +.. code-block:: console + + $ qn-api --help + usage: qn-api [-h] [--host HOST] [--port PORT] [--reload] + + Run the Quant-Net API + + options: + -h, --help show this help message and exit + --host HOST Host to bind (default: 0.0.0.0) + --port PORT Port to bind (default: 8081) + --reload Enable auto-reload on file changes (development only) + +Examples: + +.. code-block:: console + + # Start on the default host/port (0.0.0.0:8081) + $ qn-api + + # Bind to localhost on port 9000 + $ qn-api --host 127.0.0.1 --port 9000 + + # Development mode with auto-reload + $ qn-api --reload + +Alternatively, you can run the module directly with ``uvicorn``: + +.. code-block:: console + + $ uvicorn main:app --host 0.0.0.0 --port 8081 --reload + +Docker +~~~~~~ + +For containerised deployments, the +`qn-docker `_ repository provides a +``docker-compose.yml`` that orchestrates the API server alongside the +Controller, agents, message broker, and database in a single test stack. +Refer to the ``qn-docker`` README for setup instructions. + +Web Portal (Dashboard) +~~~~~~~~~~~~~~~~~~~~~~ + +The API server includes a web dashboard for interactive exploration of the network topology, agent calibration tasks, and experiment management. The dashboard is accessible at the root URL (``/``) when the server is running. + +.. image:: _static/web-dashboard.png + :width: 600 + :alt: Screenshot of the web dashboard + + +Configuration File +------------------ + +The API server reads the same INI-style configuration file used by the +:doc:`Controller `. The following paths are probed in order: + +* ``$QUANTNET_HOME/etc/quantnet.cfg`` +* ``$VIRTUAL_ENV/etc/quantnet.cfg`` +* ``/opt/quantnet/etc/quantnet.cfg`` + +If no file is found the server logs a warning and continues with built-in +defaults. + +Example (API-relevant sections only): :: + + [mq] + host = 127.0.0.1 + port = 1883 + mongo_host = 127.0.0.1 + mongo_port = 27017 + ws_host = 127.0.0.1 + ws_port = 8080 + rpc_server_topic = quantnet/rpc/server + rpc_client_topic = quantnet/rpc/client + + [schemas] + path = /opt/qn-plugins/schema + + [main] + request_types = ["spgRequest", "egpRequest", "bsmRequest"] + + +[mq] +==== + +.. confval:: api_mq_host + + :type: string + :default: ``127.0.0.1`` + + The MQTT broker host used for RPC communication with the Controller. + +.. confval:: api_mq_port + + :type: string + :default: ``1883`` + + The MQTT broker port. + +.. confval:: api_mongo_host + + :type: string + :default: ``127.0.0.1`` + + The MongoDB host used by the message bus layer. + +.. confval:: api_mongo_port + + :type: string + :default: ``27017`` + + The MongoDB port used by the message bus layer. + +.. confval:: api_ws_host + + :type: string + :default: ``127.0.0.1`` + + The WebSocket host exposed to the web dashboard for live updates. + +.. confval:: api_ws_port + + :type: string + :default: ``8080`` + + The WebSocket port exposed to the web dashboard. + +.. confval:: rpc_server_topic + + :type: string + + The MQTT topic the Controller listens on for RPC requests. + +.. confval:: rpc_client_topic + + :type: string + + The MQTT topic the API subscribes to for RPC responses. + +.. confval:: api_mq_path + + :type: string + + Optional filesystem path used by the message bus layer. + +[schemas] +========= + +.. confval:: api_schema_path + + :type: string + + Path to a directory of YAML schemas conforming to the :doc:`Message Bus ` spec. + When set, schemas are loaded at startup so the RPC client can validate + messages. + +[main] +====== + +.. confval:: request_types + + :type: list + :default: ``["spgRequest", "egpRequest", "bsmRequest"]`` + + The experiment request types surfaced in the web dashboard's request + submission form. + + +FastAPI Application +------------------- + +The application is defined in ``main.py`` and assembled from the components +described below. + +Router & Endpoints +~~~~~~~~~~~~~~~~~~ + +All REST endpoints are grouped under the ``/api`` prefix via a dedicated +``APIRouter``. The table below summarises the available routes: + +.. list-table:: + :header-rows: 1 + :widths: 10 30 60 + + * - Method + - Path + - Description + * - GET + - ``/api/topology`` + - Retrieve the network topology graph. Pass ``?full=true`` for complete + node definitions including channel data. + * - GET + - ``/api/node`` + - List all nodes in the network. + * - GET + - ``/api/node/{ID}`` + - Get information for a specific node by its identifier. + * - GET + - ``/api/experiments`` + - List all experiments. Supports ``?last=true``, ``?request=true``, and + ``?agent_id=`` query parameters. + * - GET + - ``/api/experiments/{expid}`` + - Get a specific experiment by ID. + * - GET + - ``/api/calibration`` + - List all calibrations. Supports ``?last=true``. + * - GET + - ``/api/calibration/{calid}`` + - Get a specific calibration by ID. + * - GET + - ``/api/tasks`` + - Get agent task results. Supports ``?agent_id=``. + * - POST + - ``/api/pingpong`` + - Send ping requests to a list of remote nodes. + Parameters: ``remotes`` (list), ``iterations`` (int, default 5). + * - POST + - ``/api/calibrate`` + - Run a calibration process. + Parameters: ``type``, ``src``, ``dst``, ``power``, ``light``. + * - POST + - ``/api/bsm`` + - Start a BSM experiment. + Parameters: ``src`` (list), ``rate`` (float), ``duration`` (float). + * - POST + - ``/api/spg`` + - Start a single-photon generation experiment. + Parameters: ``src`` (list), ``rate`` (float), ``duration`` (float). + * - POST + - ``/api/egp`` + - Start an entanglement generation experiment. + Parameters: ``src``, ``dst``, ``pairs``, ``bellState``, ``fidelity``. + +Interactive API documentation is auto-generated by FastAPI and available at +``/docs`` (Swagger UI) and ``/redoc`` (ReDoc) when the server is running. + +Error Handling +~~~~~~~~~~~~~~ + +Every endpoint delegates to a centralised ``_rpc_request`` helper that wraps +the underlying RPC call with uniform error handling: + +* **504 Gateway Timeout** — returned when the Controller does not respond + within the RPC timeout window (default 10 s). +* **502 Bad Gateway** — returned for any other communication failure with the + Controller. + +Both responses include a JSON body with ``error`` and ``detail`` fields. + + +OpenAPI Schema +~~~~~~~~~~~~~~ + +A custom OpenAPI schema is generated at startup with the title +**Quant-Net API**. The interactive documentation is served at: + +* ``/docs`` — Swagger UI +* ``/redoc`` — ReDoc diff --git a/comms.rst b/comms.rst index 8886c66..787f7de 100644 --- a/comms.rst +++ b/comms.rst @@ -168,3 +168,46 @@ We may then load and use the schema objects directly using ``quantnet_mq``: >>> req.serialize() '{"cmd": "myRequest", "agentId": "agent1", "payload": {"arg1": "Hello", "arg2": 99.99}, "agentID": "agent1", "command": "myRequest"}' >>> + + +EventType Enum +************** + +``quantnet_mq`` exports an ``EventType`` string enum that provides typed constants +for all monitor event types used across the control plane: + +.. code-block:: python + + from quantnet_mq import EventType + + EventType.AGENT_STATE # "agentState" + EventType.EXPERIMENT_RESULT # "experimentResult" + EventType.AGENT_HEARTBEAT # "agentHeartbeat" + EventType.AGENT_TASK_SCHEDULER_PHASE # "agentTaskSchedulerPhase" + EventType.AGENT_TASK_SCHEDULER_TASK # "agentTaskSchedulerTask" + EventType.AGENT_TASK_RESULT # "agentTaskResult" + +These constants correspond directly to the ``eventType`` field of +``MonitorEvent`` messages published to the ``monitor`` topic. +See :doc:`Monitoring ` for descriptions of each event type. + +Built-in Server RPC Schemas +**************************** + +qn-mq ships a set of built-in RPC message types used between agents and the +controller. Notable additions: + +**MonitorTask / MonitorTaskResponse** — enables clients to query recorded +agent task results from the controller's Monitor database via a ``getTasks`` +RPC call. The request may include an optional agent filter. +See :doc:`Monitoring ` for full details. + +Core Object Schemas +******************* + +The core schema bundled with qn-mq provides common object types reusable in +user-defined schemas. Recent additions: + +- ``Channel.length`` — a numeric property added to the ``Channel`` schema, + exposing quantum channel length. When the controller topology API is queried + in full mode, this field appears within each node's channel data. diff --git a/contributing.rst b/contributing.rst index 526e7c6..813a863 100644 --- a/contributing.rst +++ b/contributing.rst @@ -64,6 +64,6 @@ Before you submit a pull request, check that it meets these guidelines: 2. Code should adhere to PEP8 style guidelines. 3. If the pull request fixes a bug, the description should include a description of the bug and how the fix addresses it. -3. If the pull request adds functionality, the docs should be updated. Put +4. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring and add a README. diff --git a/history.rst b/history.rst index b93eca0..54063ae 100644 --- a/history.rst +++ b/history.rst @@ -2,6 +2,71 @@ Changelog ========= +1.1.0 (2026-04) +------------------ + +qn-api (new) +~~~~~~ +* Added northbound API server and web dashboard for experiment management and topology visualization. +* API server provides REST endpoints for querying topology, managing experiments, and retrieving results. +* :doc:`Northbound API ` documentation added to the docs. + +qn-mq +~~~~~ + +* Add ``EventType`` string enum with typed constants for all monitor event types + (``agentState``, ``experimentResult``, ``agentHeartbeat``, + ``agentTaskSchedulerPhase``, ``agentTaskSchedulerTask``, ``agentTaskResult``). +* Add ``agentTaskResult`` as a monitor event type in the message schema. +* Add RPC message types for querying agent task results from the controller + (``MonitorTask`` / ``MonitorTaskResponse``). +* Add a ``length`` property to the ``Channel`` schema, exposing quantum channel + length in topology data. +* Compatible with Python 3.10+. + +qn-server +~~~~~~~~~ + +* Add agent heartbeat validation: agents not heard from within 60 seconds are + excluded when matching nodes to an experiment path. +* Enhanced topology API: full mode returns complete node definitions including + channel data, plus aggregate node, qubit, and channel counts. +* Monitor plugin tracks heartbeat events to maintain each node's last-seen + timestamp in the database. +* Monitor plugin adds a ``getTasks`` RPC command for querying recorded agent + task results. +* Experiment protocol separates local task results from experiment-level + results; supports filtering queries by agent. +* Routing fixes: correctly handle mixed-direction graph topologies; recover + quantum links when graph conversion drops edges in multi-graph topologies. +* Configuration refactoring: centralized configuration management with explicit + config file selection via the ``-c``/``--config`` CLI flag; exits with an + error if the specified file does not exist. +* Config file search path updated — ``$QUANTNET_HOME/etc/quantnet.cfg`` + (when ``$QUANTNET_HOME`` is set) then ``/opt/quantnet/etc/quantnet.cfg`` + (always); a previously probed path has been removed. +* Database and message broker initialization decoupled from global + configuration. +* Calibration and experiment requests now follow separate execution paths, + each using its own protocol and schema when communicating with agents. +* Request parameters are now structured and typed before being forwarded + to agents. +* Allocation and result-retrieval errors now identify the specific agent + that failed. + +qn-agent +~~~~~~~~ + +* Scheduler monitor events now use the ``agentTaskResult`` event type + (previously ``experimentResult``) and include the experiment ID in the + event value. +* Configuration refactoring: centralized config management with auto-discovery; + ``-c``/``--config`` flag added, mirroring the controller pattern. Explicit + ``-c`` path exits with error if not found. +* Config file search path: ``$QUANTNET_HOME/etc/agent.cfg`` if + ``$QUANTNET_HOME`` is set, otherwise ``/etc/quantnet/agent.cfg``. +* Message broker log output suppressed to warning level by default. + 1.0.0 (2025-12-05) ------------------ diff --git a/index.rst b/index.rst index 0689bc7..4e77476 100644 --- a/index.rst +++ b/index.rst @@ -27,6 +27,7 @@ Welcome to the documentation pages for the QUANT-NET Control Plane! Controller Agent Message Bus + Northbound API .. Simulation .. toctree:: diff --git a/monitoring.rst b/monitoring.rst index e41d6a3..aa80a68 100644 --- a/monitoring.rst +++ b/monitoring.rst @@ -11,25 +11,74 @@ handlers for monitoring messages. A monitoring plugin must implement the abstract method ``handle_resource_update()`` to manage monitoring events received from a ``msgserver`` that listens to a chosen topic. -The built-in monitoring plugin can be used to handle monitoring events -received by the controller. It listens to the ``monitoring`` topic for -messages of type ``NodeState``. Upon receiving a message, the plugin: +The built-in monitoring plugin handles monitoring events received by the +controller. It subscribes to the ``monitor`` topic for ``MonitorEvent`` +messages. Upon receiving a message, the plugin: -1. Validates the message format based on the ``monitoring.yaml`` schema - defined in ``quantnet_mq/schema/messages`` -2. Stores the event object in the built-in database +1. Validates the message format against the monitoring message schema. +2. Dispatches on ``eventType``: + + - **agentHeartbeat** — updates the node's last-seen timestamp in the + database (not persisted to the Monitor DB). + - **agentState**, **agentTaskResult**, **experimentResult**, and all other + event types — persisted to the Monitor database. Protocol -------- -The :doc:`QuantNet MQ ` defines a built-in monitoring message +The :doc:`Message Bus ` defines a built-in monitoring message format in JSON, which includes the following fields: -- rid: ``str`` - Resource ID -- ts: ``str`` - UTC timestamp -- eventType: ``enum`` - Type of event -- value: ``str/int/float`` - Value of the monitored resource +- ``rid``: ``str`` — Resource ID (e.g. agent identifier) +- ``ts``: ``float`` — UTC Unix timestamp +- ``eventType``: ``str`` — Type of event (see ``EventType`` below) +- ``value``: ``str | int | float | object`` — Value of the monitored resource + +Event Types +----------- + +All valid event types are defined in the ``EventType`` enum exported by qn-mq +(see :doc:`Message Bus `): + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Event Type + - Description + * - ``agentState`` + - Agent state transition (e.g. Not Ready / Ready). Persisted to the + Monitor DB. + * - ``experimentResult`` + - Result of a completed experiment run. Persisted to the Monitor DB. + * - ``agentHeartbeat`` + - Periodic liveness ping from an agent. Updates the node's last-seen + timestamp; not stored in the Monitor DB. + * - ``agentTaskSchedulerPhase`` + - Phase update from the agent task scheduler. Persisted to the Monitor DB. + * - ``agentTaskSchedulerTask`` + - Individual task status from the agent task scheduler. Persisted to the + Monitor DB. + * - ``agentTaskResult`` + - Result of a completed agent local task, including the experiment ID and + result payload. Persisted to the Monitor DB and queryable via ``getTasks``. + +Agent Heartbeat Tracking +------------------------ + +The monitor plugin uses incoming ``agentHeartbeat`` events to maintain a +last-seen timestamp for each node in the database. This timestamp is used +by the controller to exclude stale agents when matching nodes to an +experiment path (see :doc:`Scheduling `). + +A node is considered stale if no heartbeat has been received within 60 seconds. + +getTasks RPC +------------ + +The monitor plugin exposes a ``getTasks`` RPC command that allows clients to +query recorded agent task results from the Monitor database. The request +accepts an optional agent filter to narrow results to a specific node. -QN agents and other resources connected to the message broker can send -monitoring messages to the ``monitoring`` topic. These messages will be -processed by the monitoring plugin in the controller. +Each entry in the response includes the task and experiment identifiers, +the result payload, status, timestamps, and the name of the originating task. diff --git a/requirements.txt b/requirements.txt index 96c55a7..70a8bb5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -13,3 +13,4 @@ pyyaml quantnet-server quantnet-agent quantnet-mq +quantnet-api diff --git a/routing.rst b/routing.rst index e2db1f1..168e201 100644 --- a/routing.rst +++ b/routing.rst @@ -51,7 +51,19 @@ The routing module generates the entanglement graph. Figure 2. Generated Engtanglement Graph -The results from the default shortest path algorithm and the one with entanglement constraints :: +The results from the default shortest path algorithm and the one with entanglement constraints :: Shortest Path hop by hop: ('UCB-Q', 'UCB-SWITCH', 'LBNL-SWITCH', 'LBNL-Q') - Entanglement Path hop by hop: ['UCB-Q', 'UCB-SWITCH', 'LBNL-SWITCH', 'LBNL-BSM', 'LBNL-SWITCH', 'LBNL-Q'] \ No newline at end of file + Entanglement Path hop by hop: ['UCB-Q', 'UCB-SWITCH', 'LBNL-SWITCH', 'LBNL-BSM', 'LBNL-SWITCH', 'LBNL-Q'] + +Implementation Notes +-------------------- + +**MultiGraph support** — the routing module correctly handles +topologies represented as either directed or undirected graphs, ensuring +neighbor traversal works regardless of how the topology was constructed. + +**Quantum link recovery** — when topology serialization causes quantum link +data to be lost during graph conversion, the routing module detects this and +falls back to the original topology source, ensuring path computation is not +degraded by serialization artifacts. \ No newline at end of file diff --git a/scheduling.rst b/scheduling.rst index b3e29dc..f9ff92b 100644 --- a/scheduling.rst +++ b/scheduling.rst @@ -107,3 +107,13 @@ With the task definitions, it performs the following steps: 4. If the root node’s state changes between *IN_SPEC* and *OUT_OF_SPEC*, the agent sends a monitoring message to the controller to report the status change. + +Controller-side request dispatch +--------------------------------- + +The controller distinguishes between experiment and calibration requests, +routing each through a dedicated execution path with its own protocol and +schema. Parameters are structured and typed before being forwarded to agents, +ensuring consistent handling regardless of request type. Errors during +allocation or result retrieval are reported with the identity of the +specific agent involved. diff --git a/server.rst b/server.rst index be20b45..6864be1 100644 --- a/server.rst +++ b/server.rst @@ -21,11 +21,12 @@ The QUANT-NET Controller provides the centralized controller functions for the c Configuration File ------------------ -The controller makes use of an INI configuration file. The search path for the controller configuration file includes: +The controller makes use of an INI configuration file. A path can be provided +explicitly with the ``-c``/``--config`` CLI flag; if the file does not exist +the controller exits immediately. Without ``-c``, the following paths are probed: - * ``$QUANTNET_HOME/etc/quantnet.cfg`` - * ``/opt/quantnet/etc/quantnet.cfg`` - * ``$VIRTUAL_ENV/etc/quantnet.cfg`` + * ``$QUANTNET_HOME/etc/quantnet.cfg`` (only when ``$QUANTNET_HOME`` is set) + * ``/opt/quantnet/etc/quantnet.cfg`` (always) Example: :: @@ -91,16 +92,30 @@ The configuration file may contain multiple sections as documented below. ==== .. confval:: host - + :type: string - The message queue broker host. + The message queue (MQTT) broker host. .. confval:: port :type: string - The message queue broker port. + The message queue (MQTT) broker port. + +.. confval:: mongo_host + + :type: string + :default: ``127.0.0.1`` + + The MongoDB host used by the message bus layer. + +.. confval:: mongo_port + + :type: string + :default: ``27017`` + + The MongoDB port used by the message bus layer. [experiment_definition] ======================= @@ -114,7 +129,7 @@ The configuration file may contain multiple sections as documented below. [plugins] ========= -.. confval:: path +.. confval:: plugins_path :type: string @@ -123,7 +138,7 @@ The configuration file may contain multiple sections as documented below. [schemas] ========= -.. confval:: path +.. confval:: schemas_path :type: string @@ -132,7 +147,7 @@ The configuration file may contain multiple sections as documented below. [scheduling] ============ -.. confval:: name +.. confval:: scheduling_name :type: string @@ -141,7 +156,7 @@ The configuration file may contain multiple sections as documented below. [routing] ========= -.. confval:: name +.. confval:: routing_name :type: string @@ -150,12 +165,24 @@ The configuration file may contain multiple sections as documented below. [monitoring] ============ -.. confval:: name +.. confval:: monitoring_name :type: string The name of a Monitoring module to make active in the Controller. +Topology API +------------ + +The controller's ``getInfo`` RPC with ``type="topology"`` supports an optional +``full`` parameter: + +- ``full=False`` (default) — returns a lightweight summary with each node's + type and qubit/channel counts. +- ``full=True`` — returns complete node definitions including full channel + data, plus aggregate counts of nodes, qubits, and channels across the + topology. + Logging File ------------ A Python logging module ``logger.conf`` may be included in the QUANT-NET configuration path. diff --git a/versions.yaml b/versions.yaml index ae9c868..e227b8b 100644 --- a/versions.yaml +++ b/versions.yaml @@ -19,3 +19,9 @@ - "quantnet-server==1.0.0" - "quantnet-agent==1.0.0" - "quantnet-mq==1.0.0" +"1.1.0": + tag: "v1.1.0" + packages: + - "quantnet-server==1.1.0" + - "quantnet-agent==1.1.0" + - "quantnet-mq==1.1.0"