Skip to content

Wait for a window's panes concurrently to speed up loads#1056

Open
tony wants to merge 26 commits into
masterfrom
tmuxp-lag
Open

Wait for a window's panes concurrently to speed up loads#1056
tony wants to merge 26 commits into
masterfrom
tmuxp-lag

Conversation

@tony

@tony tony commented Jun 25, 2026

Copy link
Copy Markdown
Member

Summary

  • Fix slow workspace loads (Very slow performance opening layouts/workspaces #1053): the per-pane readiness wait added in fix(workspace/builder): Wait for shell prompt before layout and commands #1018 ran serially, so a window's load time grew with its pane count even though tmux warms every pane's shell concurrently.
  • Replace the serial per-pane wait with a single concurrent barrier (_wait_for_panes_ready) that polls all of a window's default-shell panes together under one shared timeout.
  • Preserve config-order semantics: each window is still built and finished before the next begins, so a later window can rely on a directory or side effect created by an earlier window's commands.
  • Preserve the Outputs % / percent sign after every command in some panes #365 zsh partial-line % fix: no pane is resized before its shell has drawn its prompt; select_layout runs once per window, after that window's barrier.
  • Update the load progress display and before_script handling to match the new build-event ordering (see Follow-on changes).
  • No config or YAML changes — same workspaces, faster loads.

Motivation

#1018 fixed the long-standing zsh % marker (#365) by waiting for each pane's shell to draw its prompt before applying its layout. That wait ran serially: pane N+1 wasn't split until pane N was ready. Because tmux starts a pane's shell the moment the pane is split, the shells in a window already warm up concurrently — only the observation of readiness was serial. With a heavy interactive rc file, a window's load cost became panes × shell-init, the regression reported in #1053.

Design decisions

Concurrent barrier per window. A window's panes are all split first, so their shells initialize together; a single barrier then waits for every default-shell pane in that window under one shared timeout. Readiness becomes one observation pass over all the panes instead of one blocking wait per pane.

Per-window, not whole-session. Windows are built one at a time: create panes → barrier → lay out → dispatch commands → finish → next window. A window's commands run before the next window is created, so a later window can depend on side effects of an earlier one (e.g. a start_directory whose directory an earlier command created). The readiness wait is collapsed within a window; window-to-window ordering is unchanged.

Split the newest pane; defer layout until after the barrier. A split resizes the pane it targets, and the build only ever splits the newest pane — created microseconds earlier and still sourcing its rc, which is safe to resize. The window's single select_layout is deferred until after its barrier, when every shell in the window is past its prompt. Why not keep a per-split select_layout? That call also resizes the older panes; if one is mid-prompt, the resize races zsh's prompt redraw and resurrects the % marker.

Adaptive space reclaim. Without a per-split layout, a window with many panes can run out of room for the next split. _split_pane_reclaiming_space handles this: on a no-space failure it first waits for the window's existing panes (so they are safe to resize), runs select_layout to redistribute, then retries. Reclaim never resizes a not-ready pane, and a genuinely-too-small window still raises as before (#800).

Internal and always-on. No new YAML keys or tunables; the barrier just makes the existing readiness behavior cheap.

Follow-on changes

Building a window's panes together (rather than one-ready-then-next) changes the grouping and ordering of build events, so two adjacent areas needed updates:

  • Progress display (src/tmuxp/cli/_progress.py): window_done events now carry window identity, and the default progress bar tracks created vs. finished windows so the bar and the window label stay in sync as panes are created.
  • before_script streaming (src/tmuxp/cli/load.py, src/tmuxp/util.py): before_script inherits the terminal's stdio by default so TTY-aware tools run interactively, the spinner starts only after before_script finishes, and its output is captured into the panel only when a nonzero --progress-lines is requested.
  • synchronize-panes restore (src/tmuxp/workspace/builder.py): the temporary synchronize-panes disable is isolated to pane-command dispatch and skips targets for panes that exited during startup.

Before / After

Readiness wait for a window with N default-shell panes:

Before:  N serial waits  (pane N+1 not split until pane N is ready)
After:   1 concurrent barrier over all N panes (shells already warming together)

Test plan

Builder / readiness:

  • test_wait_for_panes_ready_all_ready, test_wait_for_panes_ready_mixed — the barrier reports each pane ready and times out only non-prompt panes
  • test_wait_for_pane_ready_returns_true, test_wait_for_pane_ready_timeout — the one-pane wrapper
  • test_pane_readiness_waits_for_default_shell_panes — only default-shell panes enter the barrier (custom shell/window_shell skipped)
  • test_build_waits_for_each_window_before_dispatch — a window's panes are awaited together before its commands run
  • test_select_layout_called_once_per_window, test_layout_runs_after_readiness_barrier — the Outputs % / percent sign after every command in some panes #365 invariant: select_layout runs once per window, after that window's barrier
  • test_split_target_refreshes_without_readiness_wait — splitting no longer blocks on a per-pane prompt wait
  • test_build_dispatches_window_commands_before_later_start_directory — a later window can depend on an earlier window's command side effects
  • test_issue_800_default_size_many_windows — reclaim still re-raises when a window is genuinely too small (no space for new pane with main-horizontal and a few panes #800)
  • test_synchronize_panes_disabled_during_pane_commands, test_synchronize_panes_ignores_exited_targets — sync isolation and skipping exited targets
  • test_plugin_hook_orderon_window_create / after_window_finished order across windows

Progress / before_script:

  • test_builder_window_done_events_include_window_identity — completion events carry window identity
  • test_spinner_default_progress_tracks_started_window_in_phase_one, test_spinner_default_progress_tracks_completed_window_in_phase_two — bar and label stay in sync
  • test_load_workspace_pauses_spinner_for_before_script — spinner starts after before_script
  • test_load_workspace_handles_explicit_before_script_progress_linesbefore_script output captured into the panel only on explicit --progress-lines

Suite / quality:

  • uv run py.test — full suite green
  • uv run mypy — strict, clean
  • uv run ruff check . && uv run ruff format . — clean
  • uv run py.test --doctest-modules src/tmuxp/workspace/builder.py — doctests for the readiness helpers

@tony

tony commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.91549% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.19%. Comparing base (c706e5d) to head (1ea4b68).

Files with missing lines Patch % Lines
src/tmuxp/workspace/builder.py 88.06% 17 Missing and 4 partials ⚠️
src/tmuxp/util.py 47.05% 8 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1056      +/-   ##
==========================================
- Coverage   81.98%   81.19%   -0.80%     
==========================================
  Files          28       28              
  Lines        2548     2701     +153     
  Branches      485      512      +27     
==========================================
+ Hits         2089     2193     +104     
- Misses        328      374      +46     
- Partials      131      134       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony tony force-pushed the tmuxp-lag branch 2 times, most recently from cd2526a to ed3063a Compare June 27, 2026 10:45
@tony

tony commented Jun 27, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 1 issue:

  1. The build() docstring says plugin hooks "fire by phase" — that on_window_create runs for every window "before any after_window_finished" — but the build loop is per-window sequential (create → finish → next window). For a multi-window session, on_window_create for a later window runs after the earlier window's after_window_finished, so there is no global create-phase. The PR's own test_plugin_hook_order asserts this per-window order (on_window_create:one, after_window_finished:one, on_window_create:two, after_window_finished:two), which contradicts the docstring. This reads like a leftover from the earlier whole-session two-phase build before it was reverted to per-window ordering.

passed in on initialization to create a new Session object.
Plugin hooks fire by phase: ``on_window_create`` runs for every window
as it is created, before any ``after_window_finished``. Each window's
``after_window_finished`` still runs once that window has been laid out
and its pane commands dispatched.

Contradicted by the PR's own test:

),
expected_order=[
"before_workspace_builder",
"on_window_create:one",
"after_window_finished:one",
"on_window_create:two",
"after_window_finished:two",
],
),

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

tony added 16 commits June 27, 2026 16:16
why: PR #1018 waits for each pane's shell to draw its prompt before
the next pane is created, so workspace load time grew linearly with
pane count (issue #1053: 12 windows / 18 panes took ~25s). The shells
already initialize concurrently once tmux spawns them; only the
observation was serial.

what:
- Add _wait_for_panes_ready(), a concurrent readiness barrier that
  polls every pane under one shared timeout instead of one per pane.
- Restructure build() into phases: create all windows and panes, wait
  for every default-shell pane once, then lay out and send commands.
- Create panes by splitting the newest pane only and defer the layout
  until after the barrier, so no pane is resized mid-prompt — keeping
  the zsh partial-line '%' fix (issue #365) intact.
- Add _split_pane_reclaiming_space(): on a no-space split failure,
  wait for existing panes then select_layout and retry.
- Keep iter_create_panes() as a per-window back-compat wrapper.
- Reduce _wait_for_pane_ready() to a single-pane delegate.
- Update readiness/layout tests for the barrier; add tests for the
  shared barrier, whole-session wait, and layout-after-barrier order.
why: split-window resizes the pane it targets, so the branch's
create-all-panes phase could still resize a fresh default-shell pane
before its prompt had moved away from the origin.

what:
- Wait for the current default-shell split target before calling split
- Add an ordering regression for the pre-split readiness wait
- Update readiness barrier expectations for the extra one-pane wait
why: When a split runs out of space and the window has no layout, there
is nothing to redistribute, so waiting for panes and retrying just
repeats the same failure after a needless delay.

what:
- Re-raise the split failure immediately when layout is None.
- Only wait for panes and select_layout when a layout can reclaim space.
- Correct the docstring summary to match (reclaim is layout-gated).
why: The barrier comment narrated the removed per-pane wait, which the
commit history already records and a present-day reader cannot act on.

what:
- Keep the load-bearing rationale (shells warm up in parallel, so one
  shared wait covers them) and drop the "what used to be" clause.
why: The single shared timeout is intentional — shells start
concurrently — but its effect on a prompt slower than the timeout was
unstated.

what:
- Document that a slow prompt exceeding the timeout continues without
  blocking the rest of the workspace.
why: The phase-one comment claimed "nothing is resized", but the
space-reclaim fallback can call select_layout during creation.

what:
- State that layout and pane commands come after the barrier, dropping
  the absolute "nothing is resized" claim.
why: The _pane_has_drawn_prompt doctest showed _wait_for_pane_ready as
its primary call, not the predicate it documents.

what:
- Demote the readiness wait to setup (assigned to _) and show
  _pane_has_drawn_prompt as the demonstrated call.
why: The two-phase build fires on_window_create for every window before
any after_window_finished — observable plugin behavior that was only
implied by the build() structure.

what:
- Document the phase-based hook order in build()'s docstring.
- Add a parametrized test asserting the order for one- and two-window
  workspaces via an inline recording plugin.
why: The pre-split readiness wait restored a serial per-pane prompt
wait and made large workspaces slow again. A cheap pane refresh is
still needed because libtmux resolves the target pane's window during
split-window.

what:
- Remove the per-split prompt readiness wait and its ordering test
- Refresh the split target before splitting without waiting for a prompt
- Restore one shared workspace readiness barrier expectations
why: The two-phase build starts every window before any window is
finished. Progress rendering still treated the last started window as
current during completion, so the bar and window label drifted apart.

what:
- include window identity on window_done build events
- track the current started or completed window in BuildTree
- match named window completions during phase two
- cover phase-one and phase-two progress synchronization
why: The load spinner mixed created-window text with finished-window
bar state, which made progress appear stuck and caused single-frame
line shifts while panes were being created.

what:
- Emit named window completion events for non-interleaved builds
- Render the default progress bar with finished, created, and empty segments
- Show finished windows over created windows with stable pane progress text
- Document loading workspace phases and progress output
why: Panes can legitimately exit during startup commands. The temporary
synchronize-panes restore path should not fail loads when a recorded pane
is already gone.

what:
- Skip missing pane targets during sync option handling
- Keep non-missing window and pane sync restoration unchanged
- Add regression coverage for a pane that exits during command dispatch
why: before_script commands should run like real terminal commands.
Capturing their output through tmuxp pipes makes TTY-aware tools switch to
non-interactive output and can flood the load display. Workspaces with a
before_script should also avoid rendering a transient progress frame before
that script takes over the terminal.

what:
- Let run_before_script inherit stdio when no observer is supplied
- Delay spinner startup until after before_script completes
- Keep before_script output out of the spinner panel path
- Add regressions for inherited stdio and spinner startup ordering
why: Later windows may depend on commands from earlier windows, including
created directories used as start_directory values. Creating every window
before dispatching any commands broke that existing config-order behavior.

what:
- Build and finish each window before creating the next one
- Keep the pane readiness barrier inside the per-window boundary
- Cover start_directory dependencies and plugin hook order
why: Restoring native before_script streaming made progress-lines a no-op.
Explicit panel requests should still capture script output, while default
loads should leave TTY-aware scripts attached to the terminal.

what:
- Capture before_script output only for explicit nonzero progress-lines
- Keep default and zero-line modes on native script output
- Update CLI docs, env docs, changelog, and load tests
@tony tony changed the title Parallelize pane-readiness wait to fix slow workspace loads Wait for a window's panes concurrently to speed up loads Jun 27, 2026
tony added 3 commits June 27, 2026 16:41
why: Record the load-time and before_script behavior changes
shipping with the concurrent pane-readiness work, so the
upcoming release notes tell users what they gain and which
default changed.

what:
- Add "Faster workspace loads" under What's new
- Note before_script now runs attached to the terminal by
  default, with panel capture behind --progress-lines
why: The pane-readiness branch created every pane in a window before
any pane command ran. That regressed configs where an earlier pane
command prepares a later pane's split-time start_directory.

what:
- Add a same-window start_directory regression test
- Use sequential pane setup only when a later pane target directory is
  missing before split
- Keep the concurrent per-window setup path for ordinary windows
why: Follow-up fixes restored per-window ordering and added a
dependency fallback, so branch prose that described a global phase model
was stale.

what:
- Clarify build hook/docstring wording around per-window order
- Update loading docs and changelog to describe dependency-aware pane setup
- Trim stale phase wording from progress test docstrings
tony added 7 commits June 27, 2026 20:52
why: The token-lifecycle table listed the pre-session_created value of
{build_progress} as empty, but it is computed with no guard and reads
"0/0 win" from the first render.

what:
- Correct the pre-session_created cell for {build_progress} to "0/0 win"
why: windows_created and window_progress_created are usable in custom
--progress-format strings but were absent from the docs token table, and
windows_created was also missing from the token-lifecycle table.

what:
- Add {windows_created} and {window_progress_created} to docs/cli/load.md
- Add the {windows_created} row to the lifecycle table
why: The docstring claimed output is always buffered, but with no on_line
callback the script inherits the terminal's stdio and nothing is
buffered.

what:
- Describe both paths: line-forwarded via on_line, else terminal-inherited
why: The shipped 1.67.0 entry had been reworded to describe the new
opt-in behavior, mis-stating what 1.67.0 actually shipped, which
captured before_script output to the panel by default.

what:
- Restore the original "control how much ... appears in the panel" wording
why: The before_script default change (output now on the terminal
instead of the progress panel) is a user-visible behavior change; per
changelog conventions it belongs under Breaking changes with a migration
path, not What's new.

what:
- Add a Breaking changes section with the restore command
- Remove the duplicate What's new entry
why: The concurrent path splits panes without a per-pane readiness wait;
the rationale (only the freshest, pre-prompt pane is resized) and its
residual #365 edge case were undocumented.

what:
- Comment the first-split site: why no readiness wait is needed and the
  slow on_window_create plugin residual risk
why: The space-reclaim retry caught every LibTmuxException, so an
unrelated split failure triggered a pointless readiness wait, layout
redistribution, and re-split before propagating the same error.

what:
- Reclaim only when the error message is a no-space failure; re-raise
  others immediately
- Add a parametrized test: no-space reclaims, other errors propagate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant