Skip to content

ci: add Buildkite Tart pipeline #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
El-Fitz wants to merge 6 commits into
base: main
Choose a base branch
from
Open

ci: add Buildkite Tart pipeline #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
751ae86
ci: add Buildkite Tart pipeline
El-Fitz Jun 1, 2026
4e1f908
Merge remote-tracking branch 'origin/main' into chore/buildkite-tart-ci
El-Fitz Jun 1, 2026
7f483c9
test: apply rustfmt after main merge
El-Fitz Jun 1, 2026
7aae929
ci: accept downloaded site-data artifact paths
El-Fitz Jun 9, 2026
169755f
Merge remote-tracking branch 'origin/main' into chore/buildkite-tart-ci
El-Fitz Jun 9, 2026
c74118e
ci: keep Tart smoke lightweight
El-Fitz Jun 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .buildkite/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
env:
CARGO_TERM_COLOR: always

steps:
- label: ":linux: Linux ARM64"
key: linux-arm64
concurrency: 2
concurrency_group: "big-cabbage/tart-ci"
agents:
queue: "ci-linux-arm64"
plugins:
- github.com/Bande-a-Bonnot/tart-ci#v0.1.1:
image: "ci-linux-arm64-rust-bazel"
os: "linux"
always_pull: false
Comment on lines +12 to +15

@coderabbitai coderabbitai Bot Jun 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".buildkite/pipeline.yml"
echo "== File exists? =="; ls -la "$FILE"
echo "== Line numbered excerpt (10-20) =="; nl -ba "$FILE" | sed -n '1,120p'
echo "== Search for tart-ci plugin and image keys ==";
rg -n "tart-ci#v0\.1\.1|always_pull|image:" "$FILE"
echo "== Extract step blocks around first image ==";
nl -ba "$FILE" | sed -n '1,90p'
echo "== Extract step blocks around second image ==";
nl -ba "$FILE" | sed -n '40,90p'

Repository: Lightless-Labs/third-thoughts

Length of output: 262

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".buildkite/pipeline.yml"
echo "== Show first 120 lines with line numbers (awk) =="
awk '{printf "%5d:%s\n", NR, $0}' "$FILE" | sed -n '1,140p'

echo "== Search for tart-ci plugin and related keys =="
rg -n "tart-ci#v0\.1\.1|always_pull|image:" "$FILE" || true

echo "== Excerpt around lines 8-25 (as cited first block) =="
awk 'NR>=8 && NR<=25 {printf "%5d:%s\n", NR, $0}' "$FILE"

echo "== Excerpt around lines 45-65 (as cited second block) =="
awk 'NR>=45 && NR<=65 {printf "%5d:%s\n", NR, $0}' "$FILE"

Repository: Lightless-Labs/third-thoughts

Length of output: 5457

🌐 Web query:

tart-ci Bande-a-Bonnot always_pull false

💡 Result:

The configuration setting always_pull: false is a parameter used in the Tart Buildkite plugin to control how the plugin manages virtual machine images [1]. When always_pull is set to false, the plugin will not automatically execute a tart pull command to update the VM image from the registry before cloning the virtual machine [1]. By default, this value is true, meaning the plugin typically attempts to pull the latest version of the specified image before proceeding [1]. The "Bande-a-Bonnot" mentioned in your query is a GitHub organization [2]. There is no direct link between a specific "tart-ci" tool and this organization in authoritative documentation; it is likely you are referencing the cirruslabs/tart tool, which is a virtualization solution for macOS and Linux that integrates with CI/CD platforms like Buildkite [3][4][1]. In broader technical contexts, the term always_pull is also commonly found in other CI/CD integrations (such as Docker GitHub Actions) to determine whether the system should attempt to pull a newer version of an image from a registry [5][6]. When set to false in those contexts, it relies on the local version of the image if it is already present [5].

Citations:

Pin the Tart images to immutable refs.

With always_pull: false on github.com/Bande-a-Bonnot/tart-ci#v0.1.1, the plugin won’t pull/update the specified image before running, so the VM can use whatever version is already cached on each agent. Both steps use tag-based, non-immutable references (ci-linux-arm64-rust-bazel at line 13 and ghcr.io/cirruslabs/macos-sequoia-base:latest at line 51); switch both image: values to pinned digests (or immutable version tags) to keep CI deterministic.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.buildkite/pipeline.yml around lines 12 - 15, The pipeline uses tag-based,
mutable images (image: "ci-linux-arm64-rust-bazel" and image:
"ghcr.io/cirruslabs/macos-sequoia-base:latest") together with always_pull: false
which allows agents to use cached, potentially different images; update both
image: values to pinned immutable references (digest-based strings or fixed
version tags) for the steps that currently declare image:
"ci-linux-arm64-rust-bazel" and image:
"ghcr.io/cirruslabs/macos-sequoia-base:latest" so CI becomes deterministic while
leaving always_pull as appropriate.

headless: true
command: |
set -euo pipefail

if [[ -z "$${HOME:-}" || "$$HOME" == "/" ]]; then
export HOME=/home/admin
fi
export CARGO_HOME="$${CARGO_HOME:-$$HOME/.cargo}"
export RUSTUP_HOME="$${RUSTUP_HOME:-/opt/rustup}"
export PATH="/usr/local/bin:/opt/cargo/bin:$$CARGO_HOME/bin:$$PATH"

rustc --version
cargo --version
python3 --version

@gemini-code-assist gemini-code-assist Bot Jun 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Running python3 --version directly can cause the pipeline to fail if Python 3 is not installed or configured on the runner. Since this CI slice runs with --no-python, Python is not strictly required. It is safer to guard this check so that a missing Python installation does not block the entire pipeline.

      if command -v python3 >/dev/null 2>&1; then
        python3 --version
      fi


cd middens
cargo fmt --check
cargo test --locked --lib
cargo test --locked --test cucumber -- --input tests/features/smoke.feature
cargo build --release --locked

smoke_output="$$(mktemp -d)"
smoke_xdg="$$(mktemp -d)"
XDG_DATA_HOME="$$smoke_xdg" ./target/release/middens analyze tests/fixtures --split --no-python --output "$$smoke_output"
Comment on lines +37 to +39

@gemini-code-assist gemini-code-assist Bot Jun 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The temporary directories created via mktemp -d are not cleaned up after the smoke test completes. While the Buildkite/Tart VM is ephemeral, it is best practice to clean up temporary resources using a trap on EXIT to prevent potential disk space issues or resource leaks.

      smoke_output="$$(mktemp -d)"
      smoke_xdg="$$(mktemp -d)"
      trap 'rm -rf "$$smoke_output" "$$smoke_xdg"' EXIT
      XDG_DATA_HOME="$$smoke_xdg" ./target/release/middens analyze tests/fixtures --split --no-python --output "$$smoke_output"

test -f "$$smoke_output/interactive/markov.json"
test -f "$$smoke_output/subagent/markov.json"
test -f "$$smoke_output/autonomous/markov.json"

- label: ":mac: macOS Apple Silicon"
key: macos-apple-silicon
concurrency: 2
concurrency_group: "big-cabbage/tart-ci"
agents:
queue: "ci-macos-apple-silicon"
plugins:
- github.com/Bande-a-Bonnot/tart-ci#v0.1.1:
image: "ghcr.io/cirruslabs/macos-sequoia-base:latest"
os: "macos"
always_pull: false
headless: true
command: |
set -euo pipefail

if [[ -z "$${HOME:-}" ]]; then
export HOME=/Users/admin
fi
export CARGO_HOME="$${CARGO_HOME:-$$HOME/.cargo}"
export RUSTUP_HOME="$${RUSTUP_HOME:-$$HOME/.rustup}"
export PATH="$$CARGO_HOME/bin:/opt/homebrew/bin:/usr/local/bin:$$PATH"

if ! command -v cargo >/dev/null 2>&1; then
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| HOME="$$HOME" CARGO_HOME="$$CARGO_HOME" RUSTUP_HOME="$$RUSTUP_HOME" sh -s -- -y --profile minimal --no-modify-path
. "$$CARGO_HOME/env"
fi

if command -v rustup >/dev/null 2>&1; then
rustup component add rustfmt
fi

rustc --version
cargo --version
python3 --version

@gemini-code-assist gemini-code-assist Bot Jun 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Running python3 --version directly on macOS Sequoia base images can fail or trigger a prompt to install Xcode Command Line Tools if Python 3 is not fully configured. Since this CI slice runs with --no-python, Python is not strictly required. It is safer to guard this check.

      if command -v python3 >/dev/null 2>&1; then
        python3 --version
      fi


cd middens
cargo fmt --check
cargo test --locked --lib
cargo test --locked --test cucumber -- --input tests/features/smoke.feature
cargo build --release --locked

smoke_output="$$(mktemp -d)"
smoke_xdg="$$(mktemp -d)"
XDG_DATA_HOME="$$smoke_xdg" ./target/release/middens analyze tests/fixtures --split --no-python --output "$$smoke_output"
Comment on lines +86 to +88

@gemini-code-assist gemini-code-assist Bot Jun 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The temporary directories created via mktemp -d are not cleaned up after the smoke test completes. While the Buildkite/Tart VM is ephemeral, it is best practice to clean up temporary resources using a trap on EXIT to prevent potential disk space issues or resource leaks.

      smoke_output="$$(mktemp -d)"
      smoke_xdg="$$(mktemp -d)"
      trap 'rm -rf "$$smoke_output" "$$smoke_xdg"' EXIT
      XDG_DATA_HOME="$$smoke_xdg" ./target/release/middens analyze tests/fixtures --split --no-python --output "$$smoke_output"

test -f "$$smoke_output/interactive/markov.json"
test -f "$$smoke_output/subagent/markov.json"
test -f "$$smoke_output/autonomous/markov.json"
8 changes: 7 additions & 1 deletion docs/HANDOFF.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
# Session Handoff

**Last updated:** 2026-06-04 (public results trusted smoke/deploy green)
**Last updated:** 2026-06-09 (Buildkite/Tart CI PR rebased on public-results work)

Read this at the start of every session. Update before compaction or at natural milestones.

---

## Current session update — 2026-06-01

Started Buildkite/Tart CI migration on branch `chore/buildkite-tart-ci`. Added `.buildkite/pipeline.yml` with two non-credentialed PR jobs using `github.com/Bande-a-Bonnot/tart-ci#v0.1.1`: Linux ARM64 on local image `ci-linux-arm64-rust-bazel`, and macOS Apple Silicon on `ghcr.io/cirruslabs/macos-sequoia-base:latest` with Rust bootstrapped by rustup. The first slice intentionally covers lightweight Middens cross-platform checks: `cargo fmt --check`, `cargo test --locked --lib`, the Cucumber harness smoke feature, `cargo build --release --locked`, and `middens analyze tests/fixtures --split --no-python` with checks for all three stratum outputs. The full Cucumber/HF corpus batteries, scheduled workflows, release artifacts, x86_64 targets, Hugging Face registry/secrets, and large network downloads remain on GitHub Actions/deferred.

@coderabbitai coderabbitai Bot Jun 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Extract the cargo test commands from the Buildkite pipeline configuration

rg -A 2 'cargo test' .buildkite/pipeline.yml

Repository: Lightless-Labs/third-thoughts

Length of output: 380

CI test slice matches the handoff wording (lib + cucumber smoke)

.buildkite/pipeline.yml runs cargo test --locked --lib and then cargo test --locked --test cucumber -- --input tests/features/smoke.feature, matching the handoff doc’s "cargo test --locked --lib, the Cucumber harness smoke feature" description. If any other “PR objectives” text claims only cargo test --locked, align it to include the --lib and cucumber smoke invocations.

🧰 Tools
🪛 LanguageTool

[grammar] ~11-~11: Ensure spelling is correct
Context: ...bs/macos-sequoia-base:latest` with Rust bootstrapped by rustup. The first slice intentionally covers l...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/HANDOFF.md` at line 11, Update the PR objectives/summary text that
incorrectly states only `cargo test --locked` so it matches the actual CI slice
in `.buildkite/pipeline.yml`; specifically mention both `cargo test --locked
--lib` and the Cucumber smoke invocation `cargo test --locked --test cucumber --
--input tests/features/smoke.feature` (or equivalent phrasing) wherever the
objectives or handoff wording reference test commands so they accurately reflect
the lib + cucumber smoke checks.


Comment thread
coderabbitai[bot] marked this conversation as resolved.
Local validation before PR, after merging latest `origin/main`: `cd middens && cargo fmt --check`, `cargo test --locked --lib` (31 unit tests), Cucumber smoke feature, `cargo build --release --locked`, fixture split/no-python smoke, Buildkite YAML parse with Ruby, `bash -n` over both embedded commands after Buildkite dollar escaping, and `git diff --check` all passed. Full `cargo test --locked` was attempted after the latest main merge but exceeded the local harness timeout twice (>20 minutes), so it is intentionally left to the existing GitHub HF/full test workflow rather than the first Tart smoke slice. A small rustfmt-only fix was included for `middens/tests/steps/split.rs` because latest `main` had one formatting drift that would fail the new CI slice. `cargo clippy --all-targets --locked -- -D warnings` was tried before merging latest `main` but is not included in the first CI slice because current code has many pre-existing clippy warnings under the local/newer toolchain. Remote `buildkite-agent pipeline upload --dry-run` on `big-cabbage` was attempted but blocked by intermittent SSH publickey auth from this session; no host/config/image changes were made. Existing GitHub Actions `Build public results site` initially failed on PR #3; Azure-backed log/artifact downloads were unreachable from this workstation, but workflow inspection found a likely artifact path mismatch where downloaded artifacts may contain `site-data/corpora/...` instead of `.tmp/site-data/corpora/...`. The PR branch now accepts both path shapes in the collector; workflow YAML and a local `find` fixture validated.

## >>> Read this first <<<

**All 23 techniques are working. Zero errors, zero timeouts on 13,423 sessions (commit `867f57b`).**
Expand Down
8 changes: 7 additions & 1 deletion middens/tests/steps/split.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,13 @@ fn when_run_middens_analyze_with_split_on_the_mixed_corpus_using_technique(
run_analyze_with_args(
world,
true,
&["--techniques", technique.as_str(), "--timeout", "1800", "--force"],
&[
"--techniques",
technique.as_str(),
"--timeout",
"1800",
"--force",
],
);
}

Expand Down
Loading