feat: the Weather COUNT runs while a staging maintains a copy - #88
Merged
Merged
Conversation
A staging keeps the source authoritative, and the controller now carries saved queries through a
staging, so the approved COUNT stays current on the staging's map. The starter's executor still
refused any map with a copy ("Execute the count demo before staging or after completed cutover"),
so a current COUNT could be neither exported nor run until the cutover. The controller's installed
acceptance of queries surviving operations found it: its export after each staging was refused by
this rule, not because the COUNT was out of date.
The executor now names the source on such a map too; a query written for the copy stays refused
until the cutover makes the copy the source. Relaxing the rule alone was not enough: the local run
opened its session with the source's adapter only, and a session refuses a map naming an engine it
has no adapter for. It now connects every engine the map names, as the application does during a
staging, counts on the source's adapter, and closes every adapter even when one cannot close.
Tests: a real staging through the starter's own operator on both engines, a second run writing
through both copies, and the COUNT for the staging's map counting every row (red before the fix
with the session's EngineError); the same test with a failing close; the metadata path on a map
loaded from a signed staging packet, and a query for the copy refused.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Weather starter's COUNT executor now runs while a staging maintains a copy. A staging keeps the
source authoritative, and the controller now carries saved queries through a staging, so the
approved COUNT stays current on the staging's map. The executor still refused any map with a copy
("Execute the count demo before staging or after completed cutover"), which left a current COUNT
that could be neither exported nor run until the cutover. The controller's installed acceptance of
queries surviving operations found this: its
export-count-queryafter each staging was refused bythis rule, not because the COUNT was out of date.
What changed
sde_demo.query_count._sourcenames the source on a map that also maintains a copy. A querywritten for the copy is still refused ("does not describe this project's current source and
map") until the cutover makes the copy the source.
run_count_queryopenedits session with the source's adapter only, and
sde.Sessionrefuses a map that names an engineit has no adapter for (
EngineError: the placement map refers to engines that were not supplied: ['clickhouse']). The executor now connects every engine the local map names, as the applicationdoes during a staging, counts on the source's adapter only, and closes every adapter even when one
cannot close (
ExitStack, one callback per adapter).docs/weather-starter.md: the rule, and the second connection during a staging.Tests
test_the_count_runs_on_the_source_of_a_real_staging[postgres|clickhouse](live): setup, a run,a real staging through the starter's own
operator stagewith a signed packet, a second runwriting through both copies, then the COUNT for the staging's map counts all 8 rows. The same test
makes one adapter's close fail and checks that the failure surfaces and both adapters were
closed. It was red before the fix with the
EngineErrorabove.test_a_count_reads_the_source_while_a_staging_maintains_a_copy[postgres|clickhouse]andtest_a_query_written_for_the_maintained_copy_is_refused: the metadata path on the staging's map,loaded from a signed staging packet. They were red on the old rule with its message.
make checklocally on this commit with both live engines: ruff and mypy clean, Python 2015passed and 10 skipped (the orderbook slice, which needs the engine's C++ library), TypeScript 953
passed.
Mutations on the committed tree, with both live engines: six killed - the old rule restored, the copy's
materialization accepted as the source, a session without the copy's engine, the count on another
adapter, adapters never closed, a cleanup that stops at the first failing close - and both controls
(two comments) survived.
The controller side, private, adds a test that
export-count-queryafter a completed stagingexports the carried revision bound to the staging's map. It was red on
mainwith the message theinstalled acceptance recorded.
🤖 Generated with Claude Code