docs: Add explanation of token usage for logins, API, DAGs etc.

docs/modules/airflow/pages/usage-guide/dag-run-information.adoc

@@ -0,0 +1,49 @@
+= Reading DAG run information in Airflow 3
+:description: Replace direct metadata database access in DAGs with the Airflow REST API.
+:airflow-upgrade-docs: https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading_to_airflow3.html
+
+In Airflow 3, task code can no longer access the Airflow metadata database directly, so queries such as `session.query(DagRun)` no longer work.
+See the {airflow-upgrade-docs}[Airflow 3 upgrade guide{external-link-icon}^] for background.
+
+DAGs that monitor other DAGs can use the REST API instead, authenticating as a technical user as described in xref:usage-guide/security.adoc#api-access[REST API access]:
+
+[source,python]
+----
+import os
+
+import pendulum
+import requests
+from airflow.sdk import dag, task
+
+AIRFLOW_URL = "http://<cluster>-webserver:8080"
+
+
+@dag(schedule=None, start_date=pendulum.datetime(2026, 1, 1, tz="UTC"), catchup=False)
+def monitor_dag_runs():
+    @task
+    def report(dag_id: str):
+        response = requests.post(
+            f"{AIRFLOW_URL}/auth/token",
+            json={
+                "username": os.environ["AIRFLOW_API_USERNAME"],
+                "password": os.environ["AIRFLOW_API_PASSWORD"],
+            },
+            timeout=30,
+        )
+        response.raise_for_status()
+        headers = {"Authorization": f"Bearer {response.json()['access_token']}"}
+
+        response = requests.get(
+            f"{AIRFLOW_URL}/api/v2/dags/{dag_id}/dagRuns", headers=headers, timeout=30
+        )
+        response.raise_for_status()
+        for run in response.json()["dag_runs"]:
+            print(run["dag_run_id"], run["state"], run["start_date"], run["end_date"])
+
+    report("<monitored_dag_id>")
+
+
+monitor_dag_runs()
+----
+
+The environment variables can be populated from a Secret holding the user's credentials with xref:usage-guide/overrides.adoc#_pod_overrides[Pod overrides].

docs/modules/airflow/pages/usage-guide/security.adoc

@@ -2,6 +2,7 @@
 :description: Secure Apache Airflow by configuring user authentication and authorization.
 :airflow-access-control-docs: https://airflow.apache.org/docs/apache-airflow-providers-fab/stable/auth-manager/access-control.html
 :keycloak: https://www.keycloak.org/
+:airflow-fab-token-docs: https://airflow.apache.org/docs/apache-airflow-providers-fab/stable/auth-manager/token.html
 
 Secure Apache Airflow by configuring user authentication and authorization.
 Airflow provides built-in user and role management, but can also connect to an LDAP server or an OIDC provider to manage users centrally instead.
@@ -140,6 +141,60 @@ A minimum client configuration in Keycloak for this example looks like this:
 
 Further information for specifying an AuthenticationClass for an OIDC provider can be found at the xref:concepts:authentication.adoc#_oidc[concepts page].
 
+=== [[api-access]]REST API access
+
+This section applies to Airflow 3.x.
+
+The authentication methods described above (built-in users, LDAP and OpenID Connect) govern how users log in to the web interface.
+The REST API (`/api/v2`) authenticates requests differently: it accepts only JSON Web Tokens (JWTs) issued by Airflow itself.
+A client obtains such a token by sending a username and password to the `/auth/token` endpoint of the webserver:
+
+[source,bash]
+----
+ACCESS_TOKEN=$(
+  curl -s -X POST http://<cluster>-webserver:8080/auth/token \
+    -H 'Content-Type: application/json' \
+    -d '{"username": "<user>", "password": "<password>"}' | jq -r '.access_token'
+)
+curl -s -H "Authorization: Bearer $ACCESS_TOKEN" \
+  http://<cluster>-webserver:8080/api/v2/dags/<dag_id>/dagRuns
+----
+
+The `/auth/token` endpoint checks the credentials against Airflow's internal user database (and against the LDAP server if LDAP authentication is configured).
+See the {airflow-fab-token-docs}[Airflow documentation on generating tokens{external-link-icon}^] for details.
+
+IMPORTANT: With OpenID Connect, Keycloak is used for the interactive login to the web interface only.
+Tokens issued by Keycloak (for example via the client credentials grant of a service account) are rejected by the REST API with `Invalid JWT token`.
+Users created through a Keycloak login have no usable password in Airflow, so their credentials are rejected by `/auth/token` with `Invalid credentials`.
+
+NOTE: Airflow allows tokens to be obtained for OAuth users by overriding the `create_token` method of the FAB auth manager, as described in the {airflow-fab-token-docs}[Airflow documentation on generating tokens{external-link-icon}^].
+The Stackable Data Platform does not provide or test such an override.
+It requires a custom auth manager in a custom image, and that code is then responsible for validating the token from the identity provider.
+
+IMPORTANT: The example in the Airflow documentation disables signature verification and (as also stated there) should not be used in production.
+
+==== Service-to-service access
+
+For API clients such as scripts, external services or DAGs, create a dedicated user in Airflow's internal user database.
+This works regardless of the authentication method configured for the web interface.
+Assign the least privileged role that covers the required API calls; for example, the `Viewer` role allows reading DAGs and DAG runs, but not modifying them:
+
+[source,bash]
+----
+kubectl exec <cluster>-webserver-<rolegroup>-0 -c airflow -- \
+  airflow users create \
+    --username <technical-user> \
+    --password <password> \
+    --role Viewer \
+    --firstname <first-name> \
+    --lastname <last-name> \
+    --email <email>
+----
+
+Store the credentials in a Kubernetes Secret and make them available to the client.
+Request a new token for each run rather than storing one, as tokens expire.
+See xref:usage-guide/dag-run-information.adoc[] for an example of a DAG that reads DAG run information via the REST API.
+
 == Authorization
 
 The Airflow Webserver delegates the {airflow-access-control-docs}[handling of user access control{external-link-icon}^] to the https://flask-appbuilder.readthedocs.io/en/latest/security.html[Flask AppBuilder{external-link-icon}^].

docs/modules/airflow/partials/nav.adoc

@@ -11,6 +11,7 @@
 ** xref:airflow:usage-guide/reverse-proxy.adoc[]
 ** xref:airflow:usage-guide/storage-resources.adoc[]
 ** xref:airflow:usage-guide/security.adoc[]
+** xref:airflow:usage-guide/dag-run-information.adoc[]
 ** xref:airflow:usage-guide/logging.adoc[]
 ** xref:airflow:usage-guide/monitoring.adoc[]
 ** xref:airflow:usage-guide/openlineage.adoc[]