Multi Loop Orchestration

🔄 Multi-Loop Orchestration

As autonomous agent workflows mature, a single repository will frequently run multiple concurrent or staggered loops. Multi-Loop Orchestration governs how multiple autonomous processes collaborate without corrupting shared state or generating merge conflicts.

---

1. Parallelism via Git Worktrees

The primary mechanism for concurrent agent execution is isolated Git Worktrees:

                  MAIN REPOSITORY (.git/)
                             │
     ┌───────────────────────┼───────────────────────┐
     ▼                       ▼                       ▼
WORKTREE 1: CI SWEEPER   WORKTREE 2: PR BABYSITTER   WORKTREE 3: TRIAGE
[Branch: agent/ci-fix]   [Branch: agent/rebase-42]   [Branch: main (read-only)]

Operational Rules for Worktrees

1. Isolated Working Directory: Each loop operates in a physically separate directory (.worktrees/<loop-name>/), preventing file write collisions.
2. Dedicated Branching: Every loop automatically branches off the latest origin/main commit (agent/<pattern>-<timestamp>).
3. Automated Teardown: Once a PR is opened or the task is finished, the worktree is cleanly removed:

   git worktree remove --force .worktrees/ci-sweeper
   git branch -D agent/ci-fix-temporary

---

2. Staggered Scheduling & Cadence

To prevent API rate-limiting, compute contention, and race conditions on pull requests, loops are scheduled with staggered cadences:

00:00 ──► DAILY TRIAGE (Scans backlog, prepares priorities)
   01:00 ──► DEPENDENCY SWEEPER (Audits dependencies)
      Every 15m ──► CI SWEEPER & PR BABYSITTER (Micro-maintenance)
         23:00 ──► CHANGELOG DRAFTER (Summarizes day's merges)

---

3. Pipeline Handoff Architecture

Loops can be composed into serialized multi-stage pipelines where one loop's output becomes the input context for the next:

┌─────────────────┐       ┌─────────────────┐       ┌─────────────────┐
│  ISSUE TRIAGE   │       │   CI SWEEPER    │       │  PR BABYSITTER  │
│ Categorizes &   ├──────►│ Reproduces bug  ├──────►│ Ensures clean   │
│ validates issue │       │ & writes fix    │       │ rebase & checks │
└─────────────────┘       └─────────────────┘       └─────────────────┘

1. Issue Triage Loop evaluates incoming issues, confirms reproduction, and applies the agent-ready label.
2. Implementation / Sweeper Loop detects the label, checks out a worktree, applies the minimal diff, and submits a PR.
3. PR Babysitter Loop monitors the resulting PR, verifies CI, and requests human review once green.

---

4. Conflict Resolution & State Locking

When two loops require access to the shared STATE.md:

• Atomic State Updates: Loops acquire a temporary file lock (.state.lock) before modifying STATE.md.
• Append-Only Logging: Historical events are appended to loop-run-log.md, avoiding merge conflicts.
• Drift Reconciliation: npx @cobusgreyling/loop-sync . is executed prior to any state commit to detect and resolve divergent claims.