Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,10 @@ under the file-access contract. It requires the operator to stop other writers.
Windows makes no power-loss claim. Trusted CLI restore writes a retained
original back through the same apply path and a new checkpoint. Trusted CLI
checkpoint deletion removes one resolved checkpoint and keeps its history.
Neither is power-loss qualified.
Neither is power-loss qualified. The ledger is compacted in place, by a trusted
CLI command or before an operation that would not fit, and that discards
deleted checkpoints' history ([ADR-0053](docs/adr/0053-compact-the-checkpoint-ledger-in-place.md)).
A [filesystem qualification plan](docs/filesystem-qualification.md) has not run.
Repository review and analysis are defined in
[docs/repository-checks.md](docs/repository-checks.md).
Do not describe planned features or security controls as implemented. Do not
Expand Down
43 changes: 43 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,49 @@ Changes are recorded here before a release version is assigned.

### Added

- Version 0.35.0 compacts the checkpoint ledger in place and raises its bounds
(Phase A5). See ADR-0053 and
[ledger compaction](docs/file-access.md#implemented-ledger-compaction-phase-a5).
Until now the ledger held at most 4,096 records and 8 MiB for an
installation's lifetime, so about 580 applies with deletion exhausted it.
It is on branch `codex/checkpoint-ledger-compaction`, stacked on
`codex/plan-baseline-refresh`; it is not merged. It adds no dependency.
- **Bounds:** the ledger may hold 65,536 records and 64 MiB, and the store's
aggregate quota grows from 64 MiB to 128 MiB. The 64-entry bound and the
64 KiB record bound are unchanged.
- **Format:** a compacted ledger is the original `created` record, one
`compacted` record with the replaced ledger's size, SHA-256, record count,
and revision floor, and one `checkpoint_carried` record per retained entry.
Replay rebuilds each carried entry through the ordinary transitions and
requires it to be resolved. `file checkpoints` reports `compacted_from`.
Readers older than 0.35.0 refuse both new record types.
- **Triggers:** the trusted `file compact-checkpoints --discard-history`
command, and apply, restore, and deletion when the ledger lacks room for
their records. Both refuse while an operation is unresolved or a deletion
is unfinished, and write nothing when nothing would be dropped.
- **Publication:** the store replays the candidate, writes and syncs
`ledger.compact`, reads it back, checks the live ledger is unchanged, and
replaces `ledger.jsonl` with one handle-relative rename: `renameat` on
Linux and a POSIX-semantics replace-if-exists `NtSetInformationFile`
rename on Windows. It flushes the directory and reopens and compares the
ledger. A failure after the candidate exists is uncertain, poisons the
writer, and is not retried. The new `PrivateDirectory::replace_file`
performs the rename and checks that the name then refers to the source.
Neither the rename nor the flush is power-loss qualified.
- **Replay:** an append validates only its own record, and operation IDs are
indexed, so a full replay is linear in its records.
- **Deleted history:** compaction discards every record of deleted entries.
Only the replaced ledger's digest, size, and record count remain.
- **Tests:** new core tests cover carried state, refusal of unresolved work,
each compaction fault, and compaction at the record cap before apply and
before deletion. New replay tests cover the record positions, carried
state reachability, and claim bounds. New CLI tests cover the command, and
new platform tests cover the native rename on both platforms. The replay
cap tests and the aggregate quota test use the new bounds.
- A [filesystem and power-loss qualification plan](docs/filesystem-qualification.md)
defines the release filesystem matrix and the block-level and physical
power-cut campaigns for the checkpoint store and trusted publication. None of
it has run, and it changes no support claim.
- A Testing build workflow (`.github/workflows/testing-build.yml`) runs on every
push to `main`. It calls the Rust and Frontend workflows, builds release
binaries of `omgcode` and `omgcode-desktop`, and packages them without an
Expand Down
10 changes: 5 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/omgcode-core", "crates/omgcode-cli", "crates/omgcode-platform", "crates/omgcode-app", "crates/omgcode-desktop"]

[workspace.package]
version = "0.34.0"
version = "0.35.0"
edition = "2024"
rust-version = "1.99.1"
license = "Apache-2.0"
Expand Down
18 changes: 16 additions & 2 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,13 @@ Implemented today:
`file delete-checkpoint --permanent` for one resolved checkpoint, on the same
filesystems as apply. Neither is power-loss qualified. See
[ADR-0025](docs/adr/0025-restore-and-delete-checkpoints-explicitly.md).
- In-place checkpoint ledger compaction (0.35.0, on branch
`codex/checkpoint-ledger-compaction`, not merged): 65,536 records and
64 MiB for the ledger, 128 MiB for the store, trusted
`file compact-checkpoints --discard-history`, and automatic compaction
before an apply, restore, or deletion that would not fit. Deleted entries'
history is discarded. Not power-loss qualified. See
[ADR-0053](docs/adr/0053-compact-the-checkpoint-ledger-in-place.md).

Not implemented:

Expand Down Expand Up @@ -765,8 +772,15 @@ at that review boundary:
keeps history and frees a retained slot. Inspection and acknowledgement
stay read-only. See
[ADR-0025](docs/adr/0025-restore-and-delete-checkpoints-explicitly.md).
Ledger compaction, release qualification, and power-loss evidence remain
open.
Ledger compaction was left to A5; release qualification and power-loss
evidence remain open.
- **A5 (branch `codex/checkpoint-ledger-compaction`, not merged):** raise
the ledger bounds to 65,536 records and 64 MiB and the store quota to
128 MiB, and compact the ledger in place by trusted command or before an
operation that would not fit. See
[ADR-0053](docs/adr/0053-compact-the-checkpoint-ledger-in-place.md).
The [filesystem qualification plan](docs/filesystem-qualification.md)
defines the release matrix and power-loss campaign; it has not run.

Contract decisions accepted in A1:

Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,9 @@ Windows NTFS. Stop independent writers first: a content check and rename cannot
guarantee no lost updates. Windows makes no power-loss claim. Trusted
`file restore` writes a retained original back through a new checkpoint after
an explicit current-content check. Trusted `file delete-checkpoint --permanent`
removes one resolved checkpoint's files and keeps its ledger history. Neither
removes one resolved checkpoint's files and keeps its ledger history until
trusted `file compact-checkpoints --discard-history`, or an apply, restore, or
deletion that would not fit, compacts the ledger. Neither
command is power-loss qualified. The model edit tool uses the same
checkpointed apply after an approved, writers-stopped decision; it cannot
create, delete, or rename files and does not detect a concurrent writer.
Expand Down
15 changes: 15 additions & 0 deletions crates/omgcode-cli/src/file_cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,16 @@ pub(crate) enum FileCommand {
#[arg(long, required = true, action = clap::ArgAction::SetTrue)]
permanent: bool,
},
/// Rewrite the checkpoint ledger in place, keeping only retained entries.
///
/// Deleted checkpoints' history is dropped; only the replaced ledger's
/// size, digest, and record count remain. No checkpoint entry or
/// workspace file is opened or changed.
CompactCheckpoints {
/// Acknowledge that deleted checkpoints' history is discarded.
#[arg(long, required = true, action = clap::ArgAction::SetTrue)]
discard_history: bool,
},
}

/// Restore output: the new operation's receipt and the checkpoint it read.
Expand Down Expand Up @@ -353,6 +363,11 @@ pub(crate) fn run(command: FileCommand, data_root: &Path) -> Result<(), CliError
.checkpoint_delete(operation_id)
.map_err(file_error)?,
),
FileCommand::CompactCheckpoints { discard_history: _ } => print_json(
&FileBroker::new(&mut control)
.checkpoint_compact()
.map_err(file_error)?,
),
}
}

Expand Down
67 changes: 67 additions & 0 deletions crates/omgcode-cli/tests/checkpoint_lifecycle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,70 @@ fn deleting_from_unused_storage_creates_nothing() {
ORIGINAL
);
}

#[cfg(any(target_os = "linux", windows))]
#[test]
fn compaction_drops_deleted_history_through_the_cli() {
let fixture = Fixture::new(ORIGINAL);
let deleted = fixture.apply_bytes(PROPOSED);
let retained = fixture.apply_bytes(ORIGINAL);
assert_success(&run_cli(
&fixture.data,
&["file", "delete-checkpoint", &deleted, "--permanent"],
));
let ledger = fixture.data.join("replacement-v1").join("ledger.jsonl");
let before = fs::read(&ledger).expect("ledger");

let output = run_cli(&fixture.data, &["file", "compact-checkpoints"]);
assert!(!output.status.success(), "missing acknowledgement accepted");
assert!(String::from_utf8_lossy(&output.stderr).contains("--discard-history"));
assert_eq!(fs::read(&ledger).expect("unchanged ledger"), before);

let output = run_cli(
&fixture.data,
&["file", "compact-checkpoints", "--discard-history"],
);
assert_success(&output);
let result = json_output(&output);
assert_eq!(result["compacted"], true);
assert_eq!(result["carried"], 1);
assert_eq!(result["dropped"], 1);
assert_eq!(result["previous_bytes"], before.len());
assert_eq!(result["records"], 3);

let output = run_cli(&fixture.data, &["file", "checkpoints"]);
assert_success(&output);
let view = json_output(&output);
let operations = view["operations"].as_array().expect("operations");
assert_eq!(operations.len(), 1);
assert_eq!(operations[0]["operation_id"], retained.as_str());
assert_eq!(operations[0]["retention"], "retained");
assert_eq!(
view["compacted_from"]["records"],
result["previous_records"]
);
assert_eq!(view["compacted_from"]["ledger"]["bytes"], before.len());

let output = run_cli(
&fixture.data,
&["file", "compact-checkpoints", "--discard-history"],
);
assert_success(&output);
assert_eq!(json_output(&output)["compacted"], false);
assert_eq!(
fs::read(fixture.workspace.join("note.txt")).expect("target untouched"),
ORIGINAL
);
}

#[test]
fn compacting_unused_storage_creates_nothing() {
let fixture = Fixture::new(ORIGINAL);
let output = run_cli(
&fixture.data,
&["file", "compact-checkpoints", "--discard-history"],
);
assert_error_code(&output, "checkpoint_unavailable");
assert!(!fixture.data.join("replacement-v1").exists());
assert!(!fixture.data.join("replacement-init-v1.jsonl").exists());
}
21 changes: 19 additions & 2 deletions crates/omgcode-core/src/files/checkpoint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ use super::{FileBroker, FileError, FileOperation, FileOutcome, FileTarget, valid
use crate::policy::{Operation, PolicyRequest, SessionBinding};
use crate::replacement::store::{self, Store};
use crate::replacement::{
CheckpointDeletion, CheckpointError, CheckpointInspection, CheckpointReceipt, RecordedArtifact,
CheckpointCompaction, CheckpointDeletion, CheckpointError, CheckpointInspection,
CheckpointReceipt, RecordedArtifact,
};
use crate::session::sha256_hex;

Expand All @@ -21,7 +22,9 @@ impl FileBroker<'_> {
/// not an apply token. No payload bytes appear in the returned receipt.
///
/// Each payload is at most 1 MiB. Storage retains at most 64 intents and
/// 64 MiB of managed file bytes, with capacity reserved before artifacts.
/// 128 MiB of managed file bytes, with capacity reserved before artifacts.
/// A ledger without room for the operation is compacted in place first,
/// which drops deleted entries' history (ADR-0053).
/// A write or sync error can leave uncertain private evidence. Reopen for
/// inspection; do not automatically retry or remove that evidence.
pub fn checkpoint_prepare(
Expand Down Expand Up @@ -98,6 +101,20 @@ impl FileBroker<'_> {
) -> Result<CheckpointDeletion, FileError> {
Ok(store::delete(self.control, operation_id)?)
}

/// Rewrite the checkpoint ledger in place, dropping deleted entries' history.
///
/// This is a trusted operator action, not a model tool route. It needs no
/// workspace binding and opens no workspace file or checkpoint entry.
/// Retained entries keep their recorded state as one carried record each;
/// only the replaced ledger's size, digest, and record count remain of the
/// rest. It is refused while an operation is unresolved or a deletion is
/// unfinished, and writes nothing when the ledger would not shrink. A
/// failure during the rewrite is uncertain; reopen to inspect, and do not
/// retry automatically.
pub fn checkpoint_compact(&mut self) -> Result<CheckpointCompaction, FileError> {
Ok(store::compact(self.control)?)
}
}

/// Require both permissions from the same live binding while control is locked.
Expand Down
Loading