From 3fd09ca712bb307f4acafbfe4493ddc746cfd6b8 Mon Sep 17 00:00:00 2001 From: Jason Barnwell Date: Fri, 31 Jul 2026 17:53:24 -0400 Subject: [PATCH 1/2] fix: enforce authorization for deferred-invoke endpoint Updated the deferred-invoke route to require a secret for non-loopback callers, enhancing security by preventing unauthorized access. Loopback requests are exempt from this requirement. Documentation and tests have been updated to reflect these changes, ensuring proper handling of authorization headers and client validation. --- .env.example | 4 +- CHANGELOG.md | 8 ++ docs/md/environment-configuration.md | 2 +- docs/md/serverless-mode.md | 2 +- jvspatial/api/deferred_invoke_route.py | 48 +++++++++--- .../test_deferred_invoke_fail_closed_audit.py | 74 ++++++++++++++++--- tests/serverless/test_deferred_invoke.py | 19 ++++- 7 files changed, 130 insertions(+), 27 deletions(-) diff --git a/.env.example b/.env.example index 4cf3883..4683ddf 100644 --- a/.env.example +++ b/.env.example @@ -394,7 +394,9 @@ JVSPATIAL_LOG_DB_PATH=./jvspatial_logs # JVSPATIAL_AWS_SQS_QUEUE_URL= # Optional: skip mounting POST …/_internal/deferred (e.g. alternate worker entry) # JVSPATIAL_DEFERRED_INVOKE_DISABLED=false -# Optional: require X-JVSPATIAL-Deferred-Authorize or Authorization: Bearer … +# Required for non-loopback callers of POST …/_internal/deferred (header +# X-JVSPATIAL-Deferred-Authorize or Authorization: Bearer …). LWA self-invoke +# from 127.0.0.1 is always allowed without this secret. # JVSPATIAL_DEFERRED_INVOKE_SECRET= # Work-claim lease TTL for persistent background work (seconds) # JVSPATIAL_WORK_CLAIM_STALE_SECONDS=600 diff --git a/CHANGELOG.md b/CHANGELOG.md index 311181d..273f1c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Deferred-invoke LWA self-invoke 401** (`jvspatial/api/deferred_invoke_route.py`). + Fail-closed when `JVSPATIAL_DEFERRED_INVOKE_SECRET` is unset rejected + Lambda Web Adapter pass-through from `127.0.0.1`, so serverless WhatsApp + (and other deferred tasks) never ran after scheduling. Loopback peers are + now always authorized; non-loopback callers still require the secret (or + get 401 when it is unset). Coverage in + `tests/api/test_deferred_invoke_fail_closed_audit.py`. + - **Partial-index repair log is INFO, not WARNING** (`jvspatial/db/sqlite.py`). Dropping a non-partial index so it can be recreated with `WHERE` is expected one-shot migration noise; log at info. Also satisfy ruff SIM110 in diff --git a/docs/md/environment-configuration.md b/docs/md/environment-configuration.md index dfc08e7..e3dd85b 100644 --- a/docs/md/environment-configuration.md +++ b/docs/md/environment-configuration.md @@ -81,7 +81,7 @@ See the [Caching Documentation](caching.md) for detailed information about cache |----------|------|---------|-------------| | `SERVERLESS_MODE` | boolean | auto | Force serverless-safe runtime behavior. When unset, auto-detects AWS Lambda (`AWS_LAMBDA_RUNTIME_API` / `AWS_LAMBDA_FUNCTION_NAME`) and other common serverless runtimes. | | `JVSPATIAL_DEFERRED_INVOKE_DISABLED` | boolean | `false` | When true, `register_deferred_invoke_route` does not mount `POST …/_internal/deferred`. | -| `JVSPATIAL_DEFERRED_INVOKE_SECRET` | string | _(empty)_ | When set, deferred-invoke HTTP requests must send this value via `X-JVSPATIAL-Deferred-Authorize` or `Authorization: Bearer …`. | +| `JVSPATIAL_DEFERRED_INVOKE_SECRET` | string | _(empty)_ | Required for non-loopback callers of `POST …/_internal/deferred` (`X-JVSPATIAL-Deferred-Authorize` or `Authorization: Bearer …`). Loopback (LWA self-invoke) is always allowed without this secret. | | `JVSPATIAL_WORK_CLAIM_STALE_SECONDS` | float | `600` | Default TTL for work-claim leases (`claim_record`). After this many seconds another worker can re-claim the document. | Use `is_serverless_mode()` from `jvspatial` or `jvspatial.runtime.serverless` to check at runtime. With no argument, `is_serverless_mode()` uses `get_current_server().config` when the server context is set (see serverless-mode docs): diff --git a/docs/md/serverless-mode.md b/docs/md/serverless-mode.md index 7d4dd67..6222991 100644 --- a/docs/md/serverless-mode.md +++ b/docs/md/serverless-mode.md @@ -118,7 +118,7 @@ Direct Lambda invocations (async invoke and EventBridge targets) deliver a JSON - **Canonical path**: `{JVSPATIAL_API_PREFIX}/_internal/deferred` (default **`/api/_internal/deferred`**), exposed by `register_deferred_invoke_route()` when core routes are registered (`AppBuilder.register_core_routes`). If you assemble a `FastAPI` app without that path, call `jvspatial.api.deferred_invoke_route.register_deferred_invoke_route(app)` yourself. - **Disable**: set `JVSPATIAL_DEFERRED_INVOKE_DISABLED=true` to skip registering the route (e.g. when another entrypoint handles deferred work). - **Dispatch**: the body must be a JSON object with a string **`task_type`**. jvspatial dispatches to handlers registered via **`register_deferred_invoke_handler(task_type, fn)`** or **`@deferred_invoke_handler("…")`** (also exported from top-level **`jvspatial`**). Unknown `task_type` yields HTTP 404. -- **Auth**: The deferred HTTP path is **always exempt from `AuthenticationMiddleware`** (JWT/API key): Lambda async invoke bodies cannot carry your app’s `Authorization` header. Optional **`JVSPATIAL_DEFERRED_INVOKE_SECRET`** is checked **inside** the deferred route only. If that secret is set, each request must send the same value in **`X-JVSPATIAL-Deferred-Authorize`** or **`Authorization: Bearer `**; otherwise the route returns 401. For same-function self-invoke, leave the secret unset unless you inject headers in infra. Still prefer private network / VPC boundaries for production. +- **Auth**: The deferred HTTP path is **always exempt from `AuthenticationMiddleware`** (JWT/API key): Lambda async invoke bodies cannot carry your app’s `Authorization` header. Inside the deferred route: **loopback peers** (`127.0.0.1` / `::1` / `localhost` — LWA pass-through self-invoke) are always allowed. **Non-loopback** callers (Function URL / API Gateway) require **`JVSPATIAL_DEFERRED_INVOKE_SECRET`** via **`X-JVSPATIAL-Deferred-Authorize`** or **`Authorization: Bearer `**; if the secret is unset, those callers get 401 (fail-closed). Still prefer private network / VPC boundaries for production. - **LWA environment (best-effort)**: when `is_serverless_mode()` is true, `detect_serverless_provider() == "aws"`, and LWA is detected (e.g. `AWS_LWA_PORT` or `AWS_LAMBDA_EXEC_WRAPPER` indicating the adapter), **`apply_aws_lwa_env_defaults()`** (`jvspatial.runtime.lwa`) runs from **`Server.__init__`** and uses `os.environ.setdefault` for **`AWS_LWA_PASS_THROUGH_PATH`** (same path rule as `{JVSPATIAL_API_PREFIX}/_internal/deferred`) and **`AWS_LWA_INVOKE_MODE=RESPONSE_STREAM`**. Set **`JVSPATIAL_LWA_ENV_DEFAULTS=true`** to force these defaults if detection misses; **`JVSPATIAL_LWA_ENV_DEFAULTS=false`** to disable. The LWA extension may still read env before Python starts, so **set them in Lambda / IaC** when you need guarantees. - **EventBridge default (best-effort)**: when `is_serverless_mode()` is true and `detect_serverless_provider() == "aws"`, **`apply_aws_eventbridge_env_default()`** (`jvspatial.runtime.lwa`) runs from **`Server.__init__`**. If **`JVSPATIAL_EVENTBRIDGE_SCHEDULER_ENABLED`** is absent, it sets **`true`** or **`false`** based on whether EventBridge prerequisites are satisfied (see `jvspatial.runtime.eventbridge_readiness`). Provide **`JVSPATIAL_EVENTBRIDGE_ROLE_ARN`** and either **`JVSPATIAL_EVENTBRIDGE_LAMBDA_ARN`** or **`AWS_LAMBDA_FUNCTION_NAME`** + **`AWS_ACCOUNT_ID`** (+ region) in IaC when you want scheduler-backed `run_at`. diff --git a/jvspatial/api/deferred_invoke_route.py b/jvspatial/api/deferred_invoke_route.py index 396525f..cb4a18b 100644 --- a/jvspatial/api/deferred_invoke_route.py +++ b/jvspatial/api/deferred_invoke_route.py @@ -4,7 +4,7 @@ import hmac import logging -from typing import Any, Dict +from typing import Any, Dict, Optional from fastapi import FastAPI, HTTPException, Request @@ -20,27 +20,52 @@ _DEFERRED_INVOKE_REGISTERED_ATTR = "_jvspatial_deferred_invoke_route_registered" +# LWA forwards non-HTTP (async self-invoke / EventBridge) payloads as POST from +# the local adapter process. Those requests cannot carry custom auth headers. +_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"}) + def _deferred_invoke_disabled() -> bool: return env("JVSPATIAL_DEFERRED_INVOKE_DISABLED", default=False, parse=parse_bool) +def _client_host(request: Request) -> Optional[str]: + client = request.client + if client is None: + return None + host = (client.host or "").strip().lower() + return host or None + + +def _is_loopback_client(request: Request) -> bool: + """True when the peer is LWA / local adapter (not API Gateway / public HTTP).""" + host = _client_host(request) + return host in _LOOPBACK_HOSTS + + def _deferred_invoke_secret_ok(request: Request) -> bool: """Authorize the internal deferred-invoke endpoint. - Fail-closed when ``JVSPATIAL_DEFERRED_INVOKE_SECRET`` is unset or - empty: the previous "no secret = allow everything" semantics were - a footgun — a misconfigured deployment exposed the internal - endpoint to any caller (audit §4.16 / SPEC §15.2). Disable the - route entirely via ``JVSPATIAL_DEFERRED_INVOKE_DISABLED=true`` if - you do not need it. + Lambda Web Adapter self-invoke POSTs from loopback without auth headers, so + loopback peers are always allowed. Non-loopback callers (Function URL / + API Gateway) fail closed when ``JVSPATIAL_DEFERRED_INVOKE_SECRET`` is unset + or empty (audit §4.16 / SPEC §15.2); when set, they must send the value in + ``X-JVSPATIAL-Deferred-Authorize`` or ``Authorization: Bearer …``. + + Disable the route entirely via ``JVSPATIAL_DEFERRED_INVOKE_DISABLED=true`` + if you do not need it. """ + if _is_loopback_client(request): + return True + secret = env("JVSPATIAL_DEFERRED_INVOKE_SECRET") or "" if not secret: logger.warning( "Deferred-invoke route rejected: " - "JVSPATIAL_DEFERRED_INVOKE_SECRET is unset. Either set a " - "secret or set JVSPATIAL_DEFERRED_INVOKE_DISABLED=true." + "JVSPATIAL_DEFERRED_INVOKE_SECRET is unset and peer is not " + "loopback (host=%r). Set a secret for public callers, or rely " + "on LWA self-invoke from 127.0.0.1.", + _client_host(request), ) return False hdr = (request.headers.get("X-JVSPATIAL-Deferred-Authorize") or "").strip() @@ -54,8 +79,9 @@ def _deferred_invoke_secret_ok(request: Request) -> bool: def register_deferred_invoke_route(app: FastAPI) -> None: """Mount the internal deferred-invoke endpoint. - When ``JVSPATIAL_DEFERRED_INVOKE_SECRET`` is set, requests must send the same - value in header ``X-JVSPATIAL-Deferred-Authorize`` or ``Authorization: Bearer …``. + Loopback callers (LWA pass-through) are always authorized. Non-loopback + callers require ``JVSPATIAL_DEFERRED_INVOKE_SECRET`` via header + ``X-JVSPATIAL-Deferred-Authorize`` or ``Authorization: Bearer …``. Set ``JVSPATIAL_DEFERRED_INVOKE_DISABLED=true`` to skip registering the route. """ diff --git a/tests/api/test_deferred_invoke_fail_closed_audit.py b/tests/api/test_deferred_invoke_fail_closed_audit.py index 24c3cb2..6d36d52 100644 --- a/tests/api/test_deferred_invoke_fail_closed_audit.py +++ b/tests/api/test_deferred_invoke_fail_closed_audit.py @@ -1,50 +1,102 @@ -"""Deferred-invoke fail-closed when secret unset (audit §4.16 / SPEC §15.2).""" +"""Deferred-invoke auth: fail-closed for public peers; loopback for LWA.""" + +from __future__ import annotations import os +from types import SimpleNamespace +from typing import Optional from unittest.mock import MagicMock, patch -from jvspatial.api.deferred_invoke_route import _deferred_invoke_secret_ok +from jvspatial.api.deferred_invoke_route import ( + _deferred_invoke_secret_ok, + _is_loopback_client, +) -def _fake_request(headers: dict) -> MagicMock: +def _fake_request(headers: dict, host: Optional[str] = "testclient") -> MagicMock: req = MagicMock() req.headers.get = lambda k, default=None: headers.get(k, default) req.headers.__getitem__ = lambda _self, k: headers[k] + if host is None: + req.client = None + else: + req.client = SimpleNamespace(host=host, port=50000) return req -def test_no_secret_set_denies_access(): +def test_no_secret_set_denies_non_loopback(): + with patch.dict(os.environ, {}, clear=False): + os.environ.pop("JVSPATIAL_DEFERRED_INVOKE_SECRET", None) + req = _fake_request({}, host="3.16.58.158") + assert _deferred_invoke_secret_ok(req) is False + + +def test_no_secret_set_allows_loopback(): + with patch.dict(os.environ, {}, clear=False): + os.environ.pop("JVSPATIAL_DEFERRED_INVOKE_SECRET", None) + for host in ("127.0.0.1", "::1", "localhost", "LOCALHOST"): + req = _fake_request({}, host=host) + assert _deferred_invoke_secret_ok(req) is True, host + + +def test_no_client_denies_when_secret_unset(): with patch.dict(os.environ, {}, clear=False): os.environ.pop("JVSPATIAL_DEFERRED_INVOKE_SECRET", None) - req = _fake_request({}) + req = _fake_request({}, host=None) assert _deferred_invoke_secret_ok(req) is False -def test_matching_header_allows(): +def test_loopback_allows_even_when_secret_set_without_header(): + """LWA self-invoke cannot attach custom headers; loopback must still work.""" with patch.dict( os.environ, {"JVSPATIAL_DEFERRED_INVOKE_SECRET": "shh"}, # pragma: allowlist secret clear=False, ): - req = _fake_request({"X-JVSPATIAL-Deferred-Authorize": "shh"}) + req = _fake_request({}, host="127.0.0.1") assert _deferred_invoke_secret_ok(req) is True -def test_matching_bearer_allows(): +def test_matching_header_allows_non_loopback(): with patch.dict( os.environ, {"JVSPATIAL_DEFERRED_INVOKE_SECRET": "shh"}, # pragma: allowlist secret clear=False, ): - req = _fake_request({"Authorization": "Bearer shh"}) + req = _fake_request( + {"X-JVSPATIAL-Deferred-Authorize": "shh"}, + host="3.16.58.158", + ) assert _deferred_invoke_secret_ok(req) is True -def test_mismatched_secret_denies(): +def test_matching_bearer_allows_non_loopback(): with patch.dict( os.environ, {"JVSPATIAL_DEFERRED_INVOKE_SECRET": "shh"}, # pragma: allowlist secret clear=False, ): - req = _fake_request({"X-JVSPATIAL-Deferred-Authorize": "wrong"}) + req = _fake_request( + {"Authorization": "Bearer shh"}, + host="3.16.58.158", + ) + assert _deferred_invoke_secret_ok(req) is True + + +def test_mismatched_secret_denies_non_loopback(): + with patch.dict( + os.environ, + {"JVSPATIAL_DEFERRED_INVOKE_SECRET": "shh"}, # pragma: allowlist secret + clear=False, + ): + req = _fake_request( + {"X-JVSPATIAL-Deferred-Authorize": "wrong"}, + host="3.16.58.158", + ) assert _deferred_invoke_secret_ok(req) is False + + +def test_is_loopback_client_helpers(): + assert _is_loopback_client(_fake_request({}, host="127.0.0.1")) is True + assert _is_loopback_client(_fake_request({}, host="testclient")) is False + assert _is_loopback_client(_fake_request({}, host=None)) is False diff --git a/tests/serverless/test_deferred_invoke.py b/tests/serverless/test_deferred_invoke.py index 14a3cdd..231cec8 100644 --- a/tests/serverless/test_deferred_invoke.py +++ b/tests/serverless/test_deferred_invoke.py @@ -79,8 +79,8 @@ async def test_dispatch_malformed_task_type(body: dict): def test_deferred_invoke_http_route(monkeypatch): - # Audit §4.16: route is now fail-closed when the secret is unset. - # Set a secret + send matching header for the happy-path test. + # TestClient peers as host "testclient" (not loopback), so a secret + + # matching header is required for the happy path — same as API Gateway. monkeypatch.setenv("JVSPATIAL_DEFERRED_INVOKE_SECRET", "test-secret-value") app = FastAPI() @@ -106,6 +106,21 @@ async def handler(event: dict) -> dict: assert r.json() == {"ok": True, "sender": "u1"} +def test_deferred_invoke_http_rejects_unset_secret_for_non_loopback(monkeypatch): + monkeypatch.delenv("JVSPATIAL_DEFERRED_INVOKE_SECRET", raising=False) + app = FastAPI() + + async def handler(event: dict) -> dict: + return {"ok": True} + + register_deferred_invoke_handler("app.task", handler) + register_deferred_invoke_route(app) + path = APIRoutes.deferred_invoke_full_path() + client = TestClient(app) + r = client.post(path, json={"task_type": "app.task"}) + assert r.status_code == 401 + + def test_deferred_invoke_http_unknown_returns_404(monkeypatch): monkeypatch.setenv("JVSPATIAL_DEFERRED_INVOKE_SECRET", "test-secret-value") app = FastAPI() From 3c2aad7074eaef0498f31f7bf4c8fcab6cd8ebbc Mon Sep 17 00:00:00 2001 From: Jason Barnwell Date: Fri, 31 Jul 2026 17:56:25 -0400 Subject: [PATCH 2/2] chore: update version to 0.0.15 and enhance CHANGELOG Bumped version to 0.0.15. Updated CHANGELOG to include a fix for the deferred-invoke LWA self-invoke 401 error, ensuring proper authorization for loopback and non-loopback requests. --- CHANGELOG.md | 20 ++++++++++++-------- jvspatial/version.py | 2 +- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 273f1c4..8337d38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,14 +24,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Deferred-invoke LWA self-invoke 401** (`jvspatial/api/deferred_invoke_route.py`). - Fail-closed when `JVSPATIAL_DEFERRED_INVOKE_SECRET` is unset rejected - Lambda Web Adapter pass-through from `127.0.0.1`, so serverless WhatsApp - (and other deferred tasks) never ran after scheduling. Loopback peers are - now always authorized; non-loopback callers still require the secret (or - get 401 when it is unset). Coverage in - `tests/api/test_deferred_invoke_fail_closed_audit.py`. - - **Partial-index repair log is INFO, not WARNING** (`jvspatial/db/sqlite.py`). Dropping a non-partial index so it can be recreated with `WHERE` is expected one-shot migration noise; log at info. Also satisfy ruff SIM110 in @@ -174,6 +166,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `(sort_value, id)` cursor — the default implementation tracks `id` only, so a non-`id` sort drops records that sort late but carry a lower `id`. +## [0.0.15] - 2026-07-31 + +### Fixed + +- **Deferred-invoke LWA self-invoke 401** (`jvspatial/api/deferred_invoke_route.py`). + Fail-closed when `JVSPATIAL_DEFERRED_INVOKE_SECRET` is unset rejected + Lambda Web Adapter pass-through from `127.0.0.1`, so serverless WhatsApp + (and other deferred tasks) never ran after scheduling. Loopback peers are + now always authorized; non-loopback callers still require the secret (or + get 401 when it is unset). Coverage in + `tests/api/test_deferred_invoke_fail_closed_audit.py`. + ## [0.0.11] - 2026-07-02 ### Fixed diff --git a/jvspatial/version.py b/jvspatial/version.py index 80864ec..c0150f6 100644 --- a/jvspatial/version.py +++ b/jvspatial/version.py @@ -9,4 +9,4 @@ # - MAJOR: Breaking changes # - MINOR: New features, backward compatible # - PATCH: Bug fixes, backward compatible -__version__ = "0.0.14" +__version__ = "0.0.15"