Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 7 additions & 30 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,33 +146,10 @@ jobs:
- run: mix compile --warnings-as-errors
- run: mix docs --warnings-as-errors

- name: Build and inspect unpacked Hex package
run: |
package_dir=$(mktemp -d)
trap 'rm -rf "$package_dir"' EXIT
mix hex.build --unpack --output "$package_dir"

for required in lib/replicant/session_identity.ex docs/adr/0007-actual-replication-session-identity.md .formatter.exs mix.exs README.md LICENSE NOTICE CHANGELOG.md usage-rules.md; do
test -e "$package_dir/$required" || {
echo "::error::Missing package path: $required"
exit 1
}
done

test ! -e "$package_dir/AGENTS.md" || {
echo "::error::Package contains the private contributor-agent contract"
exit 1
}

if grep -nH 'AGENTS\.md' "$package_dir/README.md" "$package_dir/usage-rules.md" "$package_dir/CONTRIBUTING.md" "$package_dir/notebooks/getting_started.livemd"; then
echo "::error::Published documentation references the excluded AGENTS.md contract"
exit 1
fi

if find "$package_dir" -type d \( -name test -o -name .forge -o -name _build -o -name superpowers \) -print -quit | grep -q .; then
echo "::error::Package contains test, lifecycle, Forge, or build residue"
exit 1
fi

grep -q 'handle_session_identity' "$package_dir/lib/replicant/sink.ex"
grep -q 'IDENTIFY_SYSTEM' "$package_dir/lib/replicant/query_builder.ex"
# Build throwaway bytes from the exact commit and run the complete package workflow without
# retaining a candidate: exact manifest, Hex validation, artifact-derived audit/compile/docs,
# fresh-consumer provenance + semantic smoke, immutable witness, strict collision checks, and
# credential-free exact-byte uploader dry-run. The real candidate is minted once, from merged
# main, after this review set and all matrix rows settle.
- name: Prove the complete package workflow without minting a candidate
run: BUILD_DATE=ci bash scripts/release/build_candidate.sh --check
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ priv/plts/
/.zcode/
/.agent/
/.aider/
/.kimosabe/

# Superpowers lifecycle artifacts (specs, plans, handoffs, reviews) — local only, never tracked
/docs/superpowers/
Expand Down
43 changes: 22 additions & 21 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.2.0] - 2026-08-19

### Added

- **Proven support for PostgreSQL 15, 16, 17, and 18, with version-gated capabilities.** The CI
Expand Down Expand Up @@ -50,6 +52,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
(`wal_status = 'lost'` is PG15's sole invalidation signal). Proven red-first at the unit level and
verified against live PostgreSQL 15/16/17/18.

- **An unknown checkpoint with an absent replication slot now halts fail-closed instead of
silently creating a fresh slot (data-integrity, fail-closed).** In sink-owned mode a
checkpoint read fault (`sink.checkpoint/0` raising or erroring) reads as `checkpoint_lsn 0`
with `checkpoint_state: :fault`. The §14.15 streaming fail-open (resume-from-0, the
idempotent sink dedups the re-stream) is safe only when the slot is **present** — a resume
clamps to the slot's server-side `confirmed_flush_lsn`, so nothing is skipped. With the slot
**absent** there is nothing to resume: the connect path previously treated the fault-as-0 as
a genuine empty first run and created a fresh `CREATE_REPLICATION_SLOT`, which begins
streaming at its own creation LSN and silently skips every transaction between the (unknown)
real checkpoint and now — an unrecoverable data gap. That path now halts fail-closed in the
`:data_gap` family with a distinct, value-free telemetry reason
(`[:replicant, :connection, :slot_invalidated]`, `reason: :checkpoint_unknown`) and never
emits `CREATE_REPLICATION_SLOT`, including in incremental-snapshot mode when its separate
progress token is empty. A genuinely **empty** checkpoint (`checkpoint_state: :empty` — a real
first activation / go-forward) still creates the slot as before. Covered by red-first
connect-decision unit tests across plain and incremental modes and a live PostgreSQL fault probe
(raising-checkpoint sink, absent slot → structural halt, zero slots created on the server).

### Security

- **Telemetry metadata and measurements are now validated by a closed key set AND a per-key
Expand Down Expand Up @@ -96,26 +116,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
field and is absent here) so the replay is a durable no-op, agreeing with `README.md` and
ADR-0001; the `handle_message/2` docstring states the same.

### Fixed

- **An unknown checkpoint with an absent replication slot now halts fail-closed instead of
silently creating a fresh slot (data-integrity, fail-closed).** In sink-owned mode a
checkpoint read fault (`sink.checkpoint/0` raising or erroring) reads as `checkpoint_lsn 0`
with `checkpoint_state: :fault`. The §14.15 streaming fail-open (resume-from-0, the
idempotent sink dedups the re-stream) is safe only when the slot is **present** — a resume
clamps to the slot's server-side `confirmed_flush_lsn`, so nothing is skipped. With the slot
**absent** there is nothing to resume: the connect path previously treated the fault-as-0 as
a genuine empty first run and created a fresh `CREATE_REPLICATION_SLOT`, which begins
streaming at its own creation LSN and silently skips every transaction between the (unknown)
real checkpoint and now — an unrecoverable data gap. That path now halts fail-closed in the
`:data_gap` family with a distinct, value-free telemetry reason
(`[:replicant, :connection, :slot_invalidated]`, `reason: :checkpoint_unknown`) and never
emits `CREATE_REPLICATION_SLOT`, including in incremental-snapshot mode when its separate
progress token is empty. A genuinely **empty** checkpoint (`checkpoint_state: :empty` — a real
first activation / go-forward) still creates the slot as before. Covered by red-first
connect-decision unit tests across plain and incremental modes and a live PostgreSQL fault probe
(raising-checkpoint sink, absent slot → structural halt, zero slots created on the server).

## [1.1.0] - 2026-08-13

### Fixed
Expand Down Expand Up @@ -558,7 +558,8 @@ against a real-PG16 crash-injection suite (loss = 0, effect-dup = 0).
**permanent** fail-closed halt (operator restart required), not auto-retry
(spec §6 / §14.18).

[Unreleased]: https://github.com/baselabs/replicant/compare/v1.1.0...HEAD
[Unreleased]: https://github.com/baselabs/replicant/compare/v1.2.0...HEAD
[1.2.0]: https://github.com/baselabs/replicant/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/baselabs/replicant/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/baselabs/replicant/compare/v0.3.1...v1.0.0
[0.3.1]: https://github.com/baselabs/replicant/compare/v0.3.0...v0.3.1
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ consumer sibling to [`arcadic`](https://github.com/baselabs/arcadic).
Multitenancy, classification, and Ash resources live one layer up, in the
[`ash_replicant`](https://hex.pm/packages/ash_replicant) sink adapter.

> **Status:** 1.1.0 is published on Hex (tagged `v1.1.0`). Replicant owns
> **Status:** 1.1.0 is the latest release published on Hex (tagged `v1.1.0`). 1.2.0
> is the prepared release candidate — it adds proven PostgreSQL 15–18 support, the typed
> `handle_slot_origin/2` callback, typed-shape telemetry validation, and a value-safety
> regression suite (see CHANGELOG `[1.2.0]`) — and is **not yet published or tagged**.
> Replicant owns
> the replication slot via `Postgrex.ReplicationConnection`, acks only after the
> sink durably commits (ack-after-checkpoint), halts fail-closed on slot
> invalidation, and is proven by a real-PG16 crash-injection suite
Expand Down
12 changes: 9 additions & 3 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Replicant — Feature Tracker

**Updated:** 2026-08-13 · **Release:** `v1.1.0` (tagged) · **Branch:** `main`
**Updated:** 2026-08-19 · **Latest published:** `v1.1.0` (tagged) · **Candidate:** `1.2.0` (built, unpublished) · **Branch:** `main`

> **⚠ Commit-SHA note.** Git history was rewritten after most of this file was
> written, so the historical commit SHAs cited in the slice rows below (e.g.
Expand All @@ -15,8 +15,14 @@
The initial sequencing plan below is complete, and both packages have continued
through later releases:

- **`replicant` 1.1.0 is published and tagged** `v1.1.0` (a post-1.0 hardening patch —
see CHANGELOG `[1.1.0]`; 1.0.0 shipped at `v1.0.0`).
- **`replicant` 1.1.0 is the latest published release**, tagged `v1.1.0` (a post-1.0
hardening patch — see CHANGELOG `[1.1.0]`; 1.0.0 shipped at `v1.0.0`).
- **`replicant` 1.2.0 is a built, verified release candidate — not yet published or
tagged.** It carries the R01–R05 fixes (fail-closed unknown-checkpoint halt, typed
telemetry shapes, logical-message value-safety, the `handle_slot_origin/2` callback,
and proven PostgreSQL 15–18 support); see CHANGELOG `[1.2.0]`. Publication and tagging
require separate explicit human authorization naming the exact version and artifact
digest.
- **`ash_replicant` 0.4.0 is published and tagged** `v0.4.0` at `d4e9457`.
Its current `main` at `197fca18594a4dbb20e57bc065ef78f30133ae5c` now consumes
Replicant `>= 1.0.0 and < 2.0.0-0`, locks 1.1.0, and independently tests exact
Expand Down
2 changes: 1 addition & 1 deletion mix.exs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
defmodule Replicant.MixProject do
use Mix.Project

@version "1.1.0"
@version "1.2.0"
@source_url "https://github.com/baselabs/replicant"

def project do
Expand Down
171 changes: 171 additions & 0 deletions scripts/release/build_candidate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
#!/usr/bin/env bash
# Build and prove package bytes from an exact commit. `--check` uses only throwaway bytes;
# the default mint mode is allowed only on clean main equal to live origin/main.
set -euo pipefail

repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "$repo_root"

mode="mint"
if [[ "${1:-}" == "--check" ]]; then
mode="check"
shift
fi
[[ $# -eq 0 ]] || { echo "usage: build_candidate.sh [--check]" >&2; exit 2; }

log() { echo "build_candidate: $*" >&2; }
die() { echo "::error::build_candidate: $*" >&2; exit 1; }

sha256_of() {
if command -v sha256sum >/dev/null 2>&1; then sha256sum "$1" | awk '{print $1}';
else shasum -a 256 "$1" | awk '{print $1}'; fi
}

[[ -z "$(git status --porcelain)" ]] || die "working tree is dirty"
commit="$(git rev-parse HEAD)"
version="$(grep -oE '@version "[^"]+"' mix.exs | head -1 | sed -E 's/@version "([^"]+)"/\1/')"
[[ -n "$version" ]] || die "could not read package version"

if [[ "$mode" == "mint" ]]; then
[[ "$(git branch --show-current)" == "main" ]] || die "candidate mint is allowed only on main"

if remote_main="$(git ls-remote --exit-code origin refs/heads/main 2>/dev/null)"; then
remote_main="$(printf '%s' "$remote_main" | awk '{print $1}')"
[[ "$remote_main" == "$commit" ]] || die "HEAD $commit does not equal live origin/main $remote_main"
else
die "could not prove live origin/main identity"
fi
fi

elixir -r "$repo_root/scripts/release/package_identity.exs" \
-e 'Replicant.PackageIdentity.verify_candidate!(hd(System.argv()))' -- "$version"

build_tree="$(mktemp -d "${TMPDIR:-/tmp}/replicant-package.XXXXXX")"
witness_ref=""
witness_oid=""
primary=""
by_digest=""
backup=""
receipt=""
artifacts_retained=0
receipt_retained=0
witness_owned=0
mint_complete=0
cleanup() {
status=$?
cleanup_failed=0
trap - EXIT

if [[ $witness_owned -eq 1 && ( "$mode" == "check" || $mint_complete -eq 0 ) ]]; then
if ! git update-ref -d "$witness_ref" "$witness_oid" >/dev/null 2>&1; then
echo "::error::build_candidate: package witness cleanup failed" >&2
cleanup_failed=1
fi
fi

if [[ "$mode" == "mint" && $mint_complete -eq 0 ]]; then
if [[ $receipt_retained -eq 1 ]]; then
chmod u+w "$receipt" >/dev/null 2>&1 || true
rm -f -- "$receipt"
fi

if [[ $artifacts_retained -eq 1 ]]; then
for target in "$primary" "$by_digest" "$backup"; do
chmod u+w "$target" >/dev/null 2>&1 || true
rm -f -- "$target"
done
fi
fi

rm -rf "$build_tree"
if [[ $status -eq 0 && $cleanup_failed -eq 1 ]]; then
status=1
fi
exit "$status"
}
trap cleanup EXIT

source_archive="$build_tree/source.tar"
git archive --format=tar --output="$source_archive" "$commit"
tar -xf "$source_archive" -C "$build_tree"
staged_tar="$build_tree/replicant-$version.tar"
( cd "$build_tree" && MIX_ENV=dev MIX_BUILD_PATH="$build_tree/_build" mix hex.build --output "$staged_tar" >/dev/null )
[[ -f "$staged_tar" ]] || die "mix hex.build produced no tarball"

digest="$(sha256_of "$staged_tar")"
size="$(wc -c < "$staged_tar" | tr -d ' ')"
lock_digest="$(sha256_of "$build_tree/mix.lock")"
verification="$build_tree/verification.txt"

EXPECTED_VERSION="$version" bash "$repo_root/scripts/release/verify_package.sh" "$staged_tar" >&2
bash "$repo_root/scripts/release/consume_candidate.sh" "$staged_tar" "$verification" >&2

if [[ "$mode" == "check" ]]; then
artifacts_dir="$build_tree/artifacts"
witness_ref="refs/attestations/checks/replicant/$version-$commit"
else
artifacts_dir="$repo_root/.kimosabe/artifacts"
witness_ref="refs/attestations/packages/replicant/$version"
fi

primary="$artifacts_dir/replicant-$version.tar"
by_digest="$artifacts_dir/by-digest/$digest.tar"
backup="$artifacts_dir/backups/replicant-$version.tar"
receipt="$artifacts_dir/replicant-$version-receipt.txt"

for target in "$primary" "$by_digest" "$backup" "$receipt"; do
[[ ! -e "$target" ]] || die "refusing to overwrite existing package evidence: $target"
done
Comment thread
coderabbitai[bot] marked this conversation as resolved.
if git show-ref --verify --quiet "$witness_ref"; then
die "package witness already exists: $witness_ref"
fi

receipt_tmp="$build_tree/receipt.txt"
{
echo "Replicant package candidate receipt"
echo "version: $version"
echo "source_commit: $commit"
echo "artifact: $primary"
echo "artifact_backup: $backup"
echo "artifact_by_digest: $by_digest"
echo "size_bytes: $size"
echo "sha256: $digest"
echo "source_mix_lock_sha256: $lock_digest"
echo "elixir: $(elixir --version 2>/dev/null | tail -1)"
echo "built_at: ${BUILD_DATE:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
echo "published: NO"
echo ""
cat "$verification"
} > "$receipt_tmp"

elixir -r "$repo_root/scripts/release/package_witness.exs" \
-e 'Replicant.PackageWitness.retain_copies!(hd(System.argv()), tl(System.argv()))' -- \
"$staged_tar" "$primary" "$by_digest" "$backup"
artifacts_retained=1

elixir -r "$repo_root/scripts/release/package_witness.exs" \
-e 'Replicant.PackageWitness.retain_copies!(hd(System.argv()), tl(System.argv()))' -- \
"$receipt_tmp" "$receipt"
receipt_retained=1

elixir -r "$repo_root/scripts/release/package_witness.exs" \
-e 'Replicant.PackageWitness.verify_copies!(tl(System.argv()), hd(System.argv()))' -- \
"$digest" "$primary" "$by_digest" "$backup"

witness_oid="$(elixir -r "$repo_root/scripts/release/package_witness.exs" \
-e 'IO.puts(Replicant.PackageWitness.create!(Enum.at(System.argv(), 0), Enum.at(System.argv(), 1), Enum.at(System.argv(), 2), Enum.at(System.argv(), 3)))' -- \
"$repo_root" "$witness_ref" "$commit" "$receipt"
)"
witness_owned=1

mix run --no-start "$repo_root/scripts/release/upload_candidate.exs" \
--artifact "$primary" --receipt "$receipt" --witness-ref "$witness_ref"

mint_complete=1

if [[ "$mode" == "check" ]]; then
log "CHECK PASS — throwaway package built, audited, documented, consumed, and uploader dry-run verified"
else
log "MINT PASS — immutable candidate retained and witnessed at $witness_ref"
fi
echo "$digest $primary"
Loading
Loading