[Epic] Inference-aware orchestration with cache locality, fair backpressure and semantic convergence

[wesleysimplicio]

Objective

Extend simplicio-loop so multi-task orchestration can exploit Runtime-managed local inference efficiently while preserving its optional-Runtime contract, evidence-gated completion, worktree isolation and bounded execution.

Why Loop benefits

Local inference has scarce stateful resources: model processes, slots, KV caches, RAM/VRAM and bounded queues. A scheduler that ignores slot locality can continually evict cache state, duplicate prompts, overload the backend and make parallel work slower. Loop should use inference metadata when available, but remain fully functional without Runtime.

Scope

1. Consume an optional versioned Runtime inference-capacity snapshot.
2. Schedule resumptions with session/slot affinity when beneficial.
3. Apply fair queueing, admission control and resource backpressure.
4. Prevent duplicate work and duplicate inference across equivalent active tasks.
5. Promote semantic progress from a prompt warning into a loop-level convergence signal.
6. Escalate, reroute, drain or stop based on evidence instead of iteration count alone.
7. Benchmark throughput, latency, cache reuse, fairness and solution quality under controlled workloads.

Constraints

• Runtime integration remains optional.
• No scheduler decision may bypass action gates, worktree isolation or evidence requirements.
• Cache locality must not starve older or lower-priority work.
• Unavailable capacity telemetry must be represented as unknown, never zero.
• A local model outage must degrade to configured alternatives or block honestly.
• No unbounded queue, retry storm or automatic worker multiplication.

Child issues

• [Scheduler] Consume Runtime inference capacity and preserve session-slot locality #675 — Runtime capacity adapter and session-slot locality
• [Scheduler] Add fair admission control, bounded backpressure and equivalent-work inference deduplication #676 — Fair admission, backpressure and inference deduplication
• [Convergence] Promote semantic progress signals into an evidence-based loop controller #677 — Evidence-based semantic convergence controller
• [Routing] Add evidence-driven local-to-remote model escalation and downgrade policy #678 — Evidence-driven local-to-remote routing policy
• [Benchmark] Measure inference-aware Loop throughput, fairness, cache reuse and solution quality #679 — Full-stack throughput, fairness, cache and quality benchmark

Epic acceptance criteria

• Loop behaves exactly as today when Runtime inference capabilities are absent.
• With Runtime available, resumable tasks preferentially reuse their healthy slot/cache.
• Queue saturation produces bounded backpressure rather than process explosion.
• Equivalent active work is detected before duplicate inference is issued.
• No-progress decisions reference concrete evidence and follow a bounded escalation ladder.
• Controlled benchmarks demonstrate whether locality improves or regresses end-to-end work.
• Fairness and starvation tests cover multi-session workloads.
• Local test evidence is sufficient; paid GitHub Actions are not required.

Source evaluated

• https://github.com/AtomicBot-ai/atomic-agent
• Runtime and Agent contracts are dependencies; Loop must not reimplement the inference backend.

[wesleysimplicio]

Simplicio-loop drain progress — 2026-07-23

• Live backlog re-queried: issues [Epic] Inference-aware orchestration with cache locality, fair backpressure and semantic convergence #674, [Scheduler] Consume Runtime inference capacity and preserve session-slot locality #675, [Scheduler] Add fair admission control, bounded backpressure and equivalent-work inference deduplication #676, [Convergence] Promote semantic progress signals into an evidence-based loop controller #677, [Routing] Add evidence-driven local-to-remote model escalation and downgrade policy #678, [Benchmark] Measure inference-aware Loop throughput, fairness, cache reuse and solution quality #679, [architecture] Turn non-critical execution blocks into technical-debt notifications #681, [P0][Mapper] Build task-specific handoffs and adaptive plans for every batch item #685, [P0][LLM] Pass mapper-selected evidence to runtime drivers and record actual use #686 and [P1][Planning] Use ContextGraph and Map Service views for conflict-aware fan-out #687 remain open.
• Wave 1 dispatched in six isolated agent lanes: [Scheduler] Consume Runtime inference capacity and preserve session-slot locality #675, [Scheduler] Add fair admission control, bounded backpressure and equivalent-work inference deduplication #676, [Convergence] Promote semantic progress signals into an evidence-based loop controller #677, [Routing] Add evidence-driven local-to-remote model escalation and downgrade policy #678, [architecture] Turn non-critical execution blocks into technical-debt notifications #681 and [P0][Mapper] Build task-specific handoffs and adaptive plans for every batch item #685.
• Each lane has frozen acceptance criteria, a backlog lease/fence, and must publish a PR to main with measured tests/evidence before closure.
• Next dependency-aware wave: [P0][LLM] Pass mapper-selected evidence to runtime drivers and record actual use #686, [P1][Planning] Use ContextGraph and Map Service views for conflict-aware fan-out #687, [Benchmark] Measure inference-aware Loop throughput, fairness, cache reuse and solution quality #679 and [Epic] Inference-aware orchestration with cache locality, fair backpressure and semantic convergence #674 integration; optional infrastructure gaps will be recorded explicitly, while authorization/source-drift/mandatory-verification failures remain hard blockers.
• Existing duplicate PRs feat(hub): add durable interactive Hub transport sessions (handshake/attach/replay + idempotent lifecycle) #688/feat(hub): add durable interactive Hub transport sessions (handshake/attach/replay + idempotent lifecycle) #689 remain open and merge-conflicted; they are being audited separately.

[wesleysimplicio]

Progress receipt: PR #713 merged as df9b8e3. Local evidence: 2 readiness-audit tests passed; the audit intentionally remains BLOCKED while live child issues/evidence are incomplete. No closure asserted.