Skip to content

ci: run postgres integration tests#2637

Draft
jucor wants to merge 1 commit into
spr/edge/1530cd63from
spr/edge/62486a46
Draft

ci: run postgres integration tests#2637
jucor wants to merge 1 commit into
spr/edge/1530cd63from
spr/edge/62486a46

Conversation

@jucor

@jucor jucor commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Make the opt-in Postgres integration tests actually execute in CI, and share
the provisioning path.

  • Refactor tests/poller/test_integration_postgres.py to obtain its DB via the
    shared require_polis_postgres() helper (introduced in the T2 commit) instead
    of its own inline throwaway-docker fixture on fixed port 5435. It now applies
    000000 + 000006 and uses an ephemeral port (xdist-safe), or the CI service.
  • python-ci.yml step 6: export POLIS_TEST_POSTGRES_URL pointing at the compose
    postgres service (docker-compose.test.yml), whose image already bakes the
    polis migrations via docker-entrypoint-initdb.d — so the votes /
    votes_latest_unique schema + on_vote_insert_update_unique_table rule are
    present (equivalent to an explicit migration-apply step). Also copy
    delphi/scripts into the container so tests/test_generator_vote_copy.py can
    import the cold-start generator under test.

With POLIS_TEST_POSTGRES_URL set, require_polis_postgres verifies the schema and
yields the URL (no docker-in-docker needed); both integration tests then RUN
rather than self-skip.

Verified locally against a shared migrated postgres via POLIS_TEST_POSTGRES_URL:
tests/poller/test_integration_postgres.py ... PASSED
tests/test_generator_vote_copy.py ... PASSED (2 passed in 0.84s)

commit-id:62486a46


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

@jucor
jucor marked this pull request as draft July 18, 2026 13:28
@jucor
jucor changed the base branch from spr/edge/a68106bf to edge July 18, 2026 13:32
@jucor
jucor force-pushed the spr/edge/62486a46 branch 2 times, most recently from 94a5915 to 0e191f5 Compare July 18, 2026 13:35
@jucor
jucor changed the base branch from edge to spr/edge/be96f1da July 18, 2026 13:35
@jucor
jucor changed the base branch from spr/edge/be96f1da to spr/edge/90ba0c34 July 18, 2026 13:37
@jucor
jucor requested a review from Copilot July 21, 2026 08:38
@jucor
jucor force-pushed the spr/edge/62486a46 branch from 96b5b8c to a2f82fb Compare July 22, 2026 06:32
@jucor
jucor force-pushed the spr/edge/90ba0c34 branch from b95ecc2 to 5e0d16b Compare July 22, 2026 06:32
@jucor
jucor force-pushed the spr/edge/62486a46 branch from a2f82fb to 9e31f7b Compare July 22, 2026 06:33
@jucor
jucor force-pushed the spr/edge/90ba0c34 branch from 5e0d16b to 40782f6 Compare July 22, 2026 06:33
@jucor
jucor changed the base branch from spr/edge/90ba0c34 to edge July 22, 2026 06:50
@jucor
jucor force-pushed the spr/edge/62486a46 branch from 9e31f7b to 7b84501 Compare July 22, 2026 06:50
@jucor
jucor changed the base branch from edge to spr/edge/90ba0c34 July 22, 2026 06:50
@jucor
jucor force-pushed the spr/edge/90ba0c34 branch from d62b789 to f61f574 Compare July 22, 2026 06:52
@jucor
jucor force-pushed the spr/edge/62486a46 branch 2 times, most recently from b9fe52e to fecb18e Compare July 22, 2026 06:56
@jucor
jucor force-pushed the spr/edge/90ba0c34 branch from f61f574 to e68bb24 Compare July 22, 2026 06:56
@jucor
jucor changed the base branch from spr/edge/90ba0c34 to edge July 22, 2026 08:11
@jucor
jucor force-pushed the spr/edge/62486a46 branch from fecb18e to 090873b Compare July 22, 2026 08:11
@jucor
jucor changed the base branch from edge to spr/edge/90ba0c34 July 22, 2026 08:11
@jucor
jucor changed the base branch from spr/edge/90ba0c34 to edge July 22, 2026 09:58
@jucor
jucor force-pushed the spr/edge/62486a46 branch from 090873b to 03adcd1 Compare July 22, 2026 09:58
@jucor
jucor changed the base branch from edge to spr/edge/90ba0c34 July 22, 2026 09:58
@jucor
jucor force-pushed the spr/edge/90ba0c34 branch from ef07a71 to dc4a3ce Compare July 22, 2026 10:06
@jucor
jucor force-pushed the spr/edge/62486a46 branch from 03adcd1 to 97d7a8f Compare July 22, 2026 10:06
@jucor
jucor changed the base branch from spr/edge/90ba0c34 to edge July 22, 2026 11:20
@github-actions

Copy link
Copy Markdown

Delphi Coverage Report

File Stmts Miss Cover
init.py 2 0 100%
benchmarks/bench_pca.py 128 107 16%
benchmarks/bench_repness.py 81 65 20%
benchmarks/bench_update_votes.py 38 28 26%
benchmarks/benchmark_utils.py 34 24 29%
components/init.py 1 0 100%
components/config.py 165 133 19%
conversation/init.py 2 0 100%
conversation/conversation.py 1332 261 80%
conversation/manager.py 131 42 68%
database/init.py 1 0 100%
database/dynamodb.py 395 189 52%
database/postgres.py 310 104 66%
pca_kmeans_rep/init.py 5 0 100%
pca_kmeans_rep/clusters.py 257 21 92%
pca_kmeans_rep/corr.py 98 17 83%
pca_kmeans_rep/group_k_smoother.py 28 0 100%
pca_kmeans_rep/legacy_kmeans.py 154 5 97%
pca_kmeans_rep/pca.py 135 18 87%
pca_kmeans_rep/repness.py 224 5 98%
poller/init.py 4 0 100%
poller/math_writer.py 41 1 98%
poller/service.py 235 43 82%
poller/worker_pool.py 75 0 100%
regression/init.py 4 0 100%
regression/clojure_comparer.py 188 20 89%
regression/comparer.py 887 649 27%
regression/datasets.py 135 27 80%
regression/recorder.py 36 27 25%
regression/utils.py 130 85 35%
replay/init.py 2 0 100%
replay/certify.py 461 119 74%
replay/crosslang.py 128 3 98%
replay/driver.py 71 2 97%
replay/prodclone.py 185 1 99%
replay/real_data.py 49 3 94%
replay/schedule.py 126 2 98%
replay/stepcompare.py 77 1 99%
replay/store.py 120 11 91%
replay/types.py 67 8 88%
run_math_pipeline.py 261 114 56%
umap_narrative/500_generate_embedding_umap_cluster.py 210 109 48%
umap_narrative/501_calculate_comment_extremity.py 112 53 53%
umap_narrative/502_calculate_priorities.py 135 135 0%
umap_narrative/700_datamapplot_for_layer.py 502 502 0%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 432 432 0%
umap_narrative/801_narrative_report_batch.py 785 785 0%
umap_narrative/802_process_batch_results.py 268 268 0%
umap_narrative/803_check_batch_status.py 183 183 0%
umap_narrative/llm_factory_constructor/init.py 2 2 0%
umap_narrative/llm_factory_constructor/model_provider.py 192 192 0%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 270 270 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 160 9 94%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 3 0 100%
umap_narrative/polismath_commentgraph/utils/converter.py 283 237 16%
umap_narrative/polismath_commentgraph/utils/group_data.py 354 336 5%
umap_narrative/polismath_commentgraph/utils/storage.py 584 518 11%
umap_narrative/reset_conversation.py 159 50 69%
umap_narrative/run_pipeline.py 453 312 31%
utils/clj_hash.py 44 0 100%
utils/engine_mode.py 9 0 100%
utils/env_flags.py 13 0 100%
utils/general.py 62 40 35%
utils/serialization.py 9 5 44%
Total 13007 7550 42%

Make the opt-in Postgres integration tests actually execute in CI, and share
the provisioning path.

- Refactor tests/poller/test_integration_postgres.py to obtain its DB via the
  shared require_polis_postgres() helper (introduced in the T2 commit) instead
  of its own inline throwaway-docker fixture on fixed port 5435. It now applies
  000000 + 000006 and uses an ephemeral port (xdist-safe), or the CI service.
- python-ci.yml step 6: export POLIS_TEST_POSTGRES_URL pointing at the compose
  `postgres` service (docker-compose.test.yml), whose image already bakes the
  polis migrations via docker-entrypoint-initdb.d — so the votes /
  votes_latest_unique schema + on_vote_insert_update_unique_table rule are
  present (equivalent to an explicit migration-apply step). Also copy
  delphi/scripts into the container so tests/test_generator_vote_copy.py can
  import the cold-start generator under test.

With POLIS_TEST_POSTGRES_URL set, require_polis_postgres verifies the schema and
yields the URL (no docker-in-docker needed); both integration tests then RUN
rather than self-skip.

Verified locally against a shared migrated postgres via POLIS_TEST_POSTGRES_URL:
  tests/poller/test_integration_postgres.py ... PASSED
  tests/test_generator_vote_copy.py ... PASSED   (2 passed in 0.84s)

commit-id:62486a46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants