Skip to content

Corrected RED semantics and fixed Days CUBIC trace conformance.#95

Merged
baochunli merged 11 commits into
mainfrom
codex/red-checker-spec
Apr 27, 2026
Merged

Corrected RED semantics and fixed Days CUBIC trace conformance.#95
baochunli merged 11 commits into
mainfrom
codex/red-checker-spec

Conversation

@baochunli

@baochunli baochunli commented Apr 27, 2026

Copy link
Copy Markdown
Collaborator

This PR corrected RED and RED_ECN checker semantics against the source-backed RED contract, added CUBIC flight-size backoff checking, and fixed Days CUBIC trace generation so raw Days-generated CUBIC traces now replay to ACCEPT under cubic_check.

Changes

  • Added a RED checker contract and validation notes covering the Floyd/Jacobson RED model, RFC 2309/RFC 3168 ECN behavior, the CSV witness contract, and intentional checker scope.

  • Corrected RED threshold classification, over-max handling, stateful between-threshold probability, per-queue count derivation, fixed-point probability arithmetic, and physical overflow precedence.

  • Enforced RED_ECN action and ECN transition rules so ECN-capable packets are marked to CE, Not-ECT congestion decisions drop, invalid ECN transitions reject, and physical overflow remains a hard drop.

  • Added exact-output RED and RED_ECN fixture runners with coverage fixtures for threshold regions, probability hit/miss, count growth, overflow, malformed fields, and ECN transition cases.

  • Extended the CUBIC trace contract with optional flight-size evidence and added checker fixtures that accept flight-size multiplicative decrease while continuing to reject cwnd-based backoff by more than one MSS.

  • Made Days CUBIC congestion and timeout reductions use sender flight size for post-signal backoff while preserving current-cwnd Wmax and fast-convergence bookkeeping.

  • Logged CUBIC flight size for congestion and timeout reactions, logged recovery-exit state transitions, and allowed bounded one-MSS tolerance only for float-derived byte snapshots in the Lean CUBIC checker.

  • Validated the CUBIC checker against ns-3.47 scratch traces: unpatched ns-3.47 rejected because it used cwnd-based backoff, and the patched trace accepted after using flight-size backoff.

  • Validated the corrected Days implementation by generating a fresh fattree CUBIC trace and checking the unmodified logs/fattree/cubic_events.csv directly with cubic_check; the trace returned ACCEPT.

Validation

  • cd lean && lake build aqm_check
  • bash lean/scripts/run-aqm-fixtures.sh
  • bash lean/scripts/run-aqm-coverage-fixtures.sh
  • bash lean/scripts/run-cubic-fixtures.sh
  • cargo test --features test -- --show-output
  • cargo test --features lean cubic -- --nocapture
  • cargo check --features lean
  • cd ../ns-3.47 && ./test.py -s tcp-abe-test
  • cd ../ns-3.47 && ./ns3 build
  • Ran ns-3.47 RED/ECN scratch experiments and checked generated CSVs with corrected aqm_check; the unpatched run rejected the full-queue mark_ecn row and the patched run accepted directly.
  • Ran ns-3.47 CUBIC scratch experiments and checked generated CSVs with corrected cubic_check; the unpatched run rejected cwnd-based backoff and the patched run accepted flight-size backoff.
  • Ran cargo run --features lean --bin days -- configs/benchmarks/topology/fattree_k4_tcp_f128_st.toml and checked logs/fattree/cubic_events.csv with lean/.lake/build/bin/cubic_check; the generated Days trace returned ACCEPT.

baochunli and others added 10 commits April 27, 2026 09:34
This commit added a source-backed RED checker contract that records the intended Floyd/Jacobson RED, RFC 2309, and RFC 3168 decision model for aqm_check. It also added a reusable AQM fixture runner plus baseline TailDrop accept/reject fixtures so later RED semantic fixes can be developed with red/green tests.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh

Linear: DT-390

Co-authored-by: Codex <codex@openai.com>
This commit changed RED threshold-region classification in aqm_check to use the logged RED EWMA average witness instead of instantaneous queue occupancy. It added fixtures for the previously inverted behavior: avg-over-max decisions with low instantaneous occupancy and avg-under-min decisions with high instantaneous occupancy.

The change intentionally preserves the existing random-witness and RED_ECN transition behavior for follow-up issues in the Linear chain.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh

Linear: DT-392

Co-authored-by: Codex <codex@openai.com>
This commit corrected aqm_check so RED decisions with avg at or above maxth signal congestion deterministically. The checker no longer requires or consults random witnesses in the over-max region, rejects over-max enqueue decisions, and accepts RED_ECN marking for ECN-capable packets without a random witness.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh

Linear: DT-394

Co-authored-by: Codex <codex@openai.com>
This commit added per-queue RED count state to aqm_check replay so between-threshold decisions use pa rather than the row-local pb value. The checker now derives count from canonical trace order, keeps counts isolated by scheduler and queue, and updates count after between-threshold misses or congestion signals.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh

Linear: DT-396

Co-authored-by: Codex <codex@openai.com>
This commit corrected RED probability arithmetic to operate in absolute queue units after converting ppb thresholds through capacity. It bounded pa to the ppb range, rejected malformed threshold configurations for RED probability decisions, and added hand-checkable packet-mode, byte-mode, malformed-threshold, and saturation fixtures.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh
- rg -n "Float|toPpb|ppbToFloat" lean/LeanGuard/Aqm lean/LeanGuard/AqmEventLog.lean

Linear: DT-398

Co-authored-by: Codex <codex@openai.com>
This commit added explicit ECN transition validation for AQM actions. RED_ECN congestion signaling now marks only ECN-capable packets to CE, drops Not-ECT packets, rejects unchanged mark_ecn rows, and rejects enqueue/drop rows that mutate ECN state.

Validation:
- bash lean/scripts/run-aqm-fixtures.sh

Linear: DT-400

Co-authored-by: Codex <codex@openai.com>
Moved the RED overflow check ahead of RED threshold and probability field parsing so physical capacity overflow is enforced as a hard drop before RED witnesses are required.

Added RED and RED_ECN fixtures covering overflow drops without RED fields and overflow enqueue rejections, preserving non-overflow behavior through the existing fixture suite.

Validation: bash lean/scripts/run-aqm-fixtures.sh.

Linear: DT-402.

Co-authored-by: Codex <codex@openai.com>
Replaced legacy RED coverage points with average-based region labels, stateful probability hit/miss labels, overflow coverage, and explicit RED_ECN signal branches.

Added coverage fixture tests for under-min, between hit and miss, over-max, overflow, and RED_ECN mark/drop branches, plus documentation for the final CSV witness and coverage contract.

Validation: bash lean/scripts/run-aqm-fixtures.sh; bash lean/scripts/run-aqm-coverage-fixtures.sh.

Linear: DT-404.

Co-authored-by: Codex <codex@openai.com>
Documented the Lean build, AQM semantic and coverage fixture suites, Rust tests, representative Days traces, and the ns-3.47 RED/ECN validation outcomes.

Recorded the copied ns-3 scratch source early malformed-threshold rejection and the retuned ns-3.47 run that isolates the full-queue ECN mark incompatibility.

Validation: cd lean && lake build aqm_check; bash lean/scripts/run-aqm-fixtures.sh; bash lean/scripts/run-aqm-coverage-fixtures.sh; cargo test --features test -- --show-output; cd /Users/bli/Playground/ns-3.47 && ./ns3 build.

Linear: DT-406.

Co-authored-by: Codex <codex@openai.com>
This commit extended the CUBIC LeanGuard trace schema with optional flight_size_bytes and used that value for congestion and timeout backoff when present, while preserving compatibility for existing traces without the column.

It added red/green CUBIC fixtures for the cwnd-vs-flight-size backoff case and a fixture runner for cubic_check.

Validation: bash lean/scripts/run-cubic-fixtures.sh; cargo check --features lean.

Co-authored-by: Codex <codex@openai.com>
@baochunli baochunli changed the title Corrected RED checker semantics. Corrected RED and CUBIC checker semantics. Apr 27, 2026
@baochunli baochunli changed the title Corrected RED and CUBIC checker semantics. Corrected RED semantics and added CUBIC flight-size checks. Apr 27, 2026
Updated Days CUBIC congestion and timeout reductions to use sender flight size for post-signal backoff while preserving current cwnd for Wmax and fast-convergence bookkeeping. Added flight-size trace plumbing, recovery-exit trace events, and bounded Lean byte snapshot tolerance for float-derived CUBIC fields so real generated traces replay cleanly without masking large cwnd-based backoff errors.

Validation:
- bash lean/scripts/run-cubic-fixtures.sh
- cargo test --features lean cubic -- --nocapture
- cargo check --features lean
- cargo run --features lean --bin days -- configs/benchmarks/topology/fattree_k4_tcp_f128_st.toml
- lean/.lake/build/bin/cubic_check logs/fattree/cubic_events.csv

Co-authored-by: Codex <codex@openai.com>
@baochunli baochunli changed the title Corrected RED semantics and added CUBIC flight-size checks. Corrected RED semantics and fixed Days CUBIC trace conformance. Apr 27, 2026
@baochunli baochunli merged commit a998820 into main Apr 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant