Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1546,6 +1546,80 @@ aster affected test --base=main --related --lane "$LANE"
[GitHub Actions](docs/ci/github-actions.md#fanning-lanes-out-as-a-job-matrix)
has a worked workflow.

### Replaying a plan instead of repeating the analysis

Under `--related` every job of a matrix repeats the source-level analysis the
plan job already did, which costs seconds to half a minute of a job's runner.
`--plan-out FILE` stores the plan, and `--plan-in FILE` runs one lane from it:

```console
aster affected test --base=main --related --matrix --plan-out plan.json
aster affected test --base=main --related --lane "$LANE" --plan-in plan.json
```

`--plan-out` works with `--matrix` and `--dry-run`. The file holds every lane
planned: the `--dry-run --json` plan, with the narrowed command of each
target, and what execution needs (selected projects, command overrides). It
also holds the key it is valid for: the commits `--base` and `--head` (default
`HEAD`) resolve to, the mode (`related`, `dependents`, `changed` or `all`), the
requested target, `--dependents`, `--only-affected-files`,
`--warnings-as-errors`, `--lang`, a digest of the changed files after
`affected.ignore` (uncommitted and untracked files included, and without
`--head` their content, so an edit after the plan was made is a different
plan), a digest of the
discovered projects and their targets, and the Aster version.

`--plan-in` replays a lane only when all of this holds, and otherwise logs one
line on stderr that says which check failed and runs the analysis as if the
flag were absent:

```text
[aster] --plan-in plan.json: base differs (plan has "4f2c...", this run has "9a1e..."); analysing instead
```

- the file exists, parses and has the supported schema;
- its key equals this invocation's key (any field above);
- exactly one `--lane` is given, the plan has it, and the lane runs the same
target as now;
- every project the lane names is still discovered;
- under `--dry-run`, `--json` is set (the text plan is built from the
analysis).

A replay prints the same `--json` plan a fresh analysis does, and runs the
same targets with the same commands. It does not carry anything the key does
not name: a plan is for one tree. Write it and replay it from checkouts of the
same commit with the same uncommitted state. A job that adds or edits files
before it runs `affected` changes the changed-file digest and falls back,
which is safe but pays for the analysis. That includes the plan file: keep it
outside the checkout (`$RUNNER_TEMP`) or under `.aster/`, which `affected`
never counts as a change.

`--plan-in` and `--matrix` cannot be combined, nor can `--plan-in` and
`--plan-out`.

### Where `affected` spends its time

A run of `affected` that is not `--quiet` ends its planning with one stderr
line naming the phases, so a long gap before the first target can be
attributed:

```text
[aster] affected phases: discover 1.1s, changes 0.6s, plan-in 0.0s, graph 0.0s, related 22.3s, select 0.2s, narrow 0.1s, quarantine 0.3s, total 24.6s
```

- `discover`: finding projects and targets;
- `changes`: reading the changed files from Git and applying `affected.ignore`;
- `plan-in`: reading and checking a stored plan (zero without `--plan-in`);
- `graph`: building and checking the project graph;
- `related`: the `--related` source analysis (zero without it, and under a
replay);
- `select`: lane selection and declared inputs, added up over the lanes;
- `narrow`: choosing narrowed commands, added up over the lanes;
- `quarantine`: loading the quarantine list, when results are recorded.

A replay marks the line `(replayed plan)`. `--verbose` adds the breakdown of
`related`.

### Running everything

`--all` treats every project as changed: each lane selects every project it
Expand Down
9 changes: 8 additions & 1 deletion docs/ci/github-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,5 +417,12 @@ jobs:
```

The plan and test jobs pass the same flags, so a row exists exactly when its
lane has something to run. A lane-specific setup step reads a `matrix` field
lane has something to run.

To skip the `--related` analysis in the test jobs, have the plan job write the
plan (`--matrix --plan-out plan.json`), upload it as an artifact, and pass the
downloaded file to each test job with `--plan-in plan.json`. A job whose
checkout, flags or Aster version differ from the plan job's logs why and
analyses as before. See
[Replaying a plan](../../README.md#replaying-a-plan-instead-of-repeating-the-analysis). A lane-specific setup step reads a `matrix` field
(`if: matrix.browser`) rather than comparing lane names.
19 changes: 19 additions & 0 deletions src/cli/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -536,6 +536,25 @@ pub enum Commands {
/// running `aster affected <target> --lane <lane>`. Implies --dry-run.
#[arg(long)]
matrix: bool,

/// Write the plan of every lane planned to FILE (needs --dry-run or --matrix)
///
/// The file holds each lane's dry-run plan, with the commands that
/// narrow its targets, and the key it is valid for: base and head
/// commits, mode, requested target, flags, changed files and Aster
/// version. Replay a lane from it with --plan-in.
#[arg(long, value_name = "FILE", conflicts_with = "plan_in")]
plan_out: Option<std::path::PathBuf>,

/// Run one lane from a plan written by --plan-out, skipping the analysis
///
/// Runs exactly what the analysis would select, using the stored
/// targets and narrowed commands. When the file is missing, cannot be
/// read, lacks the lane, or its key differs from this invocation, one
/// line on stderr says why and the analysis runs as usual. With
/// --dry-run, replays only under --json.
#[arg(long, value_name = "FILE", conflicts_with = "matrix")]
plan_in: Option<std::path::PathBuf>,
},

/// View logs from the last run
Expand Down
2 changes: 2 additions & 0 deletions src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ pub mod affected_files;
pub mod cache_plan;
pub mod commands;
pub mod output;
pub mod phases;
pub mod plan_replay;
pub mod run;
mod skills;
pub mod source_graph;
Expand Down
62 changes: 62 additions & 0 deletions src/cli/phases.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
//! Where an `aster affected` run spends its time before the first target
//! starts, as one stderr line.

use std::time::{Duration, Instant};

/// Named durations in the order the phases ran. A name marked twice adds up.
pub struct Phases {
last: Instant,
items: Vec<(&'static str, Duration)>,
}

impl Phases {
/// Start timing; `earlier` is time already spent (project discovery).
pub fn new(earlier: Duration) -> Phases {
Phases {
last: Instant::now(),
items: vec![("discover", earlier)],
}
}

/// Charge the time since the previous mark to `name`.
pub fn mark(&mut self, name: &'static str) {
let spent = self.last.elapsed();
self.last = Instant::now();
if let Some(entry) = self.items.iter_mut().find(|(n, _)| *n == name) {
entry.1 += spent;
} else {
self.items.push((name, spent));
}
}

/// `affected phases: discover 0.4s, changes 0.1s, ... total 24.3s`.
pub fn line(&self, replayed: bool) -> String {
let total: Duration = self.items.iter().map(|(_, spent)| *spent).sum();
let mut parts: Vec<String> = self
.items
.iter()
.map(|(name, spent)| format!("{name} {:.1}s", spent.as_secs_f64()))
.collect();
parts.push(format!("total {:.1}s", total.as_secs_f64()));
let how = if replayed { " (replayed plan)" } else { "" };
format!("[aster] affected phases{how}: {}", parts.join(", "))
}
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn phases_list_in_order_and_a_repeated_name_adds_up() {
let mut phases = Phases::new(Duration::from_millis(400));
phases.mark("changes");
phases.mark("narrow");
phases.mark("narrow");
let line = phases.line(false);
assert!(line.starts_with("[aster] affected phases: discover 0.4s, changes "));
assert_eq!(line.matches("narrow").count(), 1, "{line}");
assert!(line.contains(", total "), "{line}");
assert!(phases.line(true).contains("(replayed plan)"));
}
}
Loading
Loading