Support postgres as a Server database type - #35
Merged
Conversation
asyncpg pools and asyncio.Lock both bind to the loop they were created on, but _ensure_pool() memoized the pool and lock for the lifetime of the PostgresDB instance. A host that bootstraps inside one asyncio.run() and then serves from a second loop -- the common CLI-then-ASGI-server startup -- carried a pool bound to the first, now-closed loop into the server loop, and its first query failed with "cannot perform operation: another operation is in progress" / ConnectionDoesNotExistError. Track the loop the pool belongs to and drop both pool and lock when the running loop differs. The stale pool's connections are abandoned rather than closed, since close() would have to await on a loop that no longer runs; Postgres reaps them when the sockets drop. File-backed adapters never hit this, which is why it went unnoticed: it only bites the Server path on postgres.
DatabaseConfigurator.initialize_graph_context() dispatched db_type
through a hard-coded json/mongodb/sqlite/dynamodb chain and raised
"Unsupported database type: postgres" for anything else. PostgresDB and
create_database("postgres", ...) were already complete, and the
JVSPATIAL_POSTGRES_* keys were already allowlisted -- but Server() is the
only path most deployments use, so a documented backend was unreachable
from the API layer and from anything built on it.
Add the missing branch, and give ServerConfig.database the settings it
needs to describe the connection: postgres_dsn, postgres_min_pool_size,
postgres_max_pool_size, postgres_pooler_mode, mapped from env in
env_adapter alongside the mongodb and dynamodb keys. Connection settings
now reach the driver through the config object rather than only through
the driver's own env reads. Unset values are omitted so PostgresDB's
defaults still apply.
postgresql is accepted as an alias for postgres, matching the factory.
Benchmark comparisonThreshold: ±25% (informational, does not block merge)
|
This was referenced Aug 4, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Summary
What does this PR address?
Postgres is a documented jvspatial backend —
docs/md/postgres-guide.md,neon-deployment.md, andaurora-serverless-deployment.mdall describe deploying on it, and theJVSPATIAL_POSTGRES_*env keys are already on the allowlist — but it is unreachable fromServer(...). Two independent gaps sit betweendb_type="postgres"and a working process:DatabaseConfigurator.initialize_graph_context()rejects the type outright.PostgresDB's pool does not survive the loop handoff that a CLI-then-ASGI-server startup performs.This PR closes both, so the Server path reaches the backend that already exists underneath it.
Description
Bug Fixes:
Unsupported database type: postgresServer(db_type="postgres")raisesValueError: Unsupported database type: postgresat construction.initialize_graph_context()(jvspatial/api/components/database_configurator.py:177before this change) dispatches through a hard-codedjson/mongodb/sqlite/dynamodbif-chain and raises on anything else. The layer below is complete —create_database("postgres", ...)builds a fully workingPostgresDBtoday — so only the Server-facing branch was missing.postgresqlaccepted as an alias to matchdb/factory.py.Pool bound to a dead event loop
asyncio.run()and then serves from a second loop fails on its first server-loop query:_ensure_pool()memoizesself._pool— andself._pool_lockis built once in__init__. Bothasyncpg.Poolandasyncio.Lockbind to the loop they were created on, so both are unusable once that loop closes. File-backed adapters never notice, which is why this went unseen._ensure_pool()runs on a different loop, drop the pool and the lock and rebuild. The stale pool's connections are abandoned rather than closed —close()would have to await on a loop that no longer runs — and Postgres reaps them when the sockets drop.Feature Request:
ServerConfig.databasegainspostgres_dsn,postgres_min_pool_size,postgres_max_pool_size, andpostgres_pooler_mode, mapped from the existingJVSPATIAL_POSTGRES_*env keys inenv_adapter.pyalongside the mongodb and dynamodb keys.Motivation is the CONTRIBUTING invariant "Single source of truth for config — don't read environment variables ad-hoc from inside library code, go through the config object." Without these fields the configurator would have to hand
create_databasea baredb_typeand let the driver read env behind the config object's back, leaving Postgres the one backend whose connection settings a host cannot set programmatically. Unset values are omitted from the kwargs soPostgresDB's own defaults (including its env reads) still apply — no behavior change for existing directcreate_databaseusers.Changes Made
High-Level Summary:
jvspatial/api/components/database_configurator.py— add thepostgres/postgresqlbranch plus_resolve_postgres_kwargs().jvspatial/api/config_groups.py— fourpostgres_*fields onDatabaseConfig.jvspatial/env_adapter.py— mapJVSPATIAL_POSTGRES_*onto them (string and int keys).jvspatial/db/postgres.py—_pool_looptracking and_discard_pool_from_dead_loop();close()clears the loop handle.TestPoolLoopAffinityintests/db/test_postgres_unit.py, postgres cases intests/api/components/test_database_configurator.py, newtests/test_env_adapter_postgres.py.Server" and "Event loops" sections indocs/md/postgres-guide.md;CHANGELOG.mdunder[Unreleased].No breaking changes. Nothing existing changes shape; the only behavior change on a previously-working path is that a
PostgresDBreused across loops now works instead of raising.Checklist
asyncpgremains the existing[postgres]extra)Each new test was confirmed to fail against
mainbefore the corresponding fix landed.Pre-existing failures on my machine, unrelated to this PR. A full local run shows 8 failures. I re-ran every one against unmodified
mainand they fail identically there, so this branch introduces none of them:tests/storage/—detect_mime_type()returnsapplication/octet-stream(and onceapplication/SIMH-tape-data) instead ofimage/png. A libmagic/db difference on this host, not a code issue.tests/api/endpoints/test_walker_executor.py::TestWalkerExecutor::test_execute_direct_execution_walker— response-envelope mismatch ({'result': ...}vs{'success': True, 'data': {...}}). Order-dependent: it passes when run alone and fails under the fulltests/apirun — onmainas well as here. Might be worth a separate look; I left it alone since it's outside this change.Separately, and also on
main: the interpreter hangs at exit after the suite finishes (Py_FinalizeEx→wait_for_thread_shutdown), so a non-daemon thread isn't joining. Results are complete before it hangs, butpytest ... | tailnever flushes. Untouched by this PR; flagging in case it's news.Steps to Test
docker run -d --name jvspatial-pg \ -e POSTGRES_USER=jvspatial -e POSTGRES_PASSWORD=jvspatial -e POSTGRES_DB=jvdb \ -p 55432:5432 postgres:16-alpine pip install -e '.[dev,test,postgres]' pytest tests/db/test_postgres_unit.py tests/test_env_adapter_postgres.py \ tests/api/components/test_database_configurator.py -qUnit coverage needs no live database. For the loop-handoff path specifically, the shape that used to fail is:
Additional Context
Found while bringing up jvagent on Postgres — it is exactly the two-loop host described above (CLI bootstraps the application graph under
asyncio.run(), then hands off to uvicorn). Both gaps reproduce identically on 0.0.9, 0.0.12, and 0.0.15.End-to-end verification against this branch, with no patches on the jvagent side:
jvagent <app> bootstrappersists the full application graph to Postgres —node/edge/objecttables and their indexes created unattended./healthreports"database":"connected", lifecycle logs📊 Database: PostgresDB | 🌳 Root: n.Root.root.Full suite green locally,
pre-commit run --all-filesclean.Questions or Concerns
One judgment call worth a reviewer's eye: the stale pool's connections are abandoned, not closed, on a loop change. Closing them would require awaiting on the dead loop, which isn't possible from the new one. In the startup-handoff case this leaks at most one pool's worth of connections once per process, and Postgres reaps them when the sockets drop — but if you'd rather see an explicit
min_size=0recommendation for that window, or a warning log instead of debug, say the word.Happy to split this into two PRs (
db/fix andapi/support) if you'd prefer them reviewed separately — they're independent, though Postgres-on-Serverneeds both to actually work.