docs: spec — BYO engine-adapter packages & deterministic python-node containers - #49
Closed
carolsimone wants to merge 1 commit into
Closed
docs: spec — BYO engine-adapter packages & deterministic python-node containers#49carolsimone wants to merge 1 commit into
carolsimone wants to merge 1 commit into
Conversation
Design doc (for review): publish the engine adapters to PyPI as continuo-<engine>-adapter, switch the engine images to install the versioned published libraries instead of building from source, and add a deterministic own-base domain template. Ordered as one v* tag (publish wheels, then build the image), generic across future engines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Design doc (for review) for publishing the engine adapters and enabling deterministic, bring-your-own python-node containers.
The gap
A python node can be containerized only one way today:
FROMa Continuo engine image (deterministic, but forces our base). Teams needing their own base — or wanting to reuse theWarehouseAdaptermethods elsewhere — have no deterministic path: the adapters aren't installable from a package index, and the images build them from source (repo head), which is neither available to users nor reproducible.What the spec commits to
continuo-<engine>-adapter(dist +continuo_<engine>_adaptermodule, together — no pip≠import trap), on the existingv*tag via a workspace-driven matrix +skip-existing: true. Generic across future engines (Snowflake, BigQuery) — publish-by-addition.--require-hashesfrom PyPI at release; a locally-built wheelhouse for CI/PR). Image ≡ what's on PyPI.needsthe wheel publish + a wait-until-installable retry. One PR bumps version + lock together. The only two-step risk is the--require-hasheshash, resolved by reproducible wheel builds (fallback: publish-then-pin).0.1.0projects (they predate the validation→runtime merge).Resolved open questions (spec §9)
0.2.0under the new name ·discover_adapter()fails loudly on >1 adapter · commit the lock · confirm reproducible wheels (fallback documented).Docs-only. Implementation follows as a separate bite-sized plan.
🤖 Generated with Claude Code