Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 19 additions & 9 deletions shiny/shiny-py-optimize/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,20 @@ Order of preference for every fix:
4. **Relocate** work that blocks (`@reactive.extended_task`) — last resort; it
buys concurrency, not raw speed.

One fact drives most Python-specific advice: **Shiny runs reactive functions
serially on a single asyncio event loop — never concurrently, even when they
are `async def`, even across sessions in the same process.** One blocking call
in any reactive function freezes every session in the process. `async`/
`await` alone never fixes this; only `@reactive.extended_task` does.
One fact drives most Python-specific advice: **every session in a process
shares one asyncio event loop.** A *synchronous* blocking call in any reactive
function freezes every session in the process. What a slow `async def`
reactive blocks depends on the shiny version (check with
`python -c "import shiny; print(shiny.__version__)"`):

- **shiny < 1.9.0:** reactive functions run serially across the whole
process, so a slow `async` reactive also freezes every session.
- **shiny ≥ 1.9.0:** effects run concurrently, so a slow `async` reactive
holds only *its own* session (no input updates, no outputs until it
finishes); other sessions keep running.

Either way, `async`/`await` alone never keeps the *same* session responsive;
only `@reactive.extended_task` does.

## Workflow

Expand Down Expand Up @@ -68,7 +77,7 @@ of slow Shiny for Python apps.
| Identical expensive work repeated across sessions | Every user re-pays the same cost | Module-level cache (`functools.lru_cache`, `diskcache`) — there is no built-in `bindCache` equivalent → [caching.md](references/caching.md) |
| Sliders/text inputs feeding expensive chains directly | One drag = dozens of recomputations | `@reactive.event()` button gating; debounce helper → [reactive-graph.md](references/reactive-graph.md) |
| `time.sleep`, sync `requests`, heavy pandas/DuckDB calls inline in calcs/effects/renders | Blocks the event loop for **every user in the process** | `@reactive.extended_task` → [async-tasks.md](references/async-tasks.md) |
| `async def` reactives added expecting a speedup | Shiny runs reactive code serially even when async | Async only for async-only APIs; `ExtendedTask` for non-blocking → [async-tasks.md](references/async-tasks.md) |
| `async def` reactives added expecting a speedup | A slow async reactive still holds its session (and, on shiny < 1.9.0, every session) | Async only for async-only APIs; `ExtendedTask` for non-blocking → [async-tasks.md](references/async-tasks.md) |
| `@render.ui` used for routine value updates | DOM rebuild + input re-binding per update | `ui.update_*()` → [rendering-ui.md](references/rendering-ui.md) |
| `@render.table` on large data frames | pandas Styler → full HTML serialization | `@render.data_frame` with `render.DataGrid` → [rendering-ui.md](references/rendering-ui.md) |

Expand Down Expand Up @@ -111,7 +120,8 @@ replay with N workers → report). Tool selection and reading guides:
|---|---|---|
| First page load is slow (every session) | Per-session data loading (especially Express top-level); outputs computing at startup on empty inputs; heavy per-session setup | [data-loading.md](references/data-loading.md), [reactive-graph.md](references/reactive-graph.md) |
| One interaction recomputes far too much | Over-broad dependencies; duplicate computation; chatty inputs | [reactive-graph.md](references/reactive-graph.md), [caching.md](references/caching.md) |
| A specific action takes seconds and the app freezes — for **everyone** | Blocking operation on the shared event loop | [async-tasks.md](references/async-tasks.md) |
| A specific action takes seconds and the app freezes — for **everyone** | Sync blocking operation on the shared event loop (or any slow reactive, on shiny < 1.9.0) | [async-tasks.md](references/async-tasks.md) |
| A specific action takes seconds and the app freezes — only for **that user** | Slow `async` reactive holding its session (shiny ≥ 1.9.0) | [async-tasks.md](references/async-tasks.md) |
| Fast for one user, degrades with several | Blocking operations; repeated identical computation; per-user reloads; too few processes | [caching.md](references/caching.md), [async-tasks.md](references/async-tasks.md), [diagnosis.md](references/diagnosis.md) |
| Plots/tables slow to appear or update | Heavy rendering; `@render.table`; full redraws; un-aggregated Plotly | [rendering-ui.md](references/rendering-ui.md) |
| Downloads/uploads misbehave or recompute | Download handler recomputing per click; uploads parsed in renderers | [data-loading.md](references/data-loading.md) |
Expand Down Expand Up @@ -200,8 +210,8 @@ an artifact, say so in the report instead of asserting it.
(debouncing adds intentional latency; caches share results across
sessions).
- **`async` is not a performance fix.** `async def` reactives exist so you can
call async-only APIs. They do not run concurrently and do not make the app
responsive. Non-blocking means `@reactive.extended_task`.
call async-only APIs. They do not make the computation faster or keep the
session responsive. Non-blocking means `@reactive.extended_task`.
- **Caching before async.** Caching often yields 10–100×; async only adds
concurrency and carries serialization and debugging costs.
- **Never scale with multi-worker servers.** `uvicorn --workers >1`, the
Expand Down
59 changes: 44 additions & 15 deletions shiny/shiny-py-optimize/references/async-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,51 @@ the cost of serialization overhead and harder debugging.

## Why blocking hurts — and why `async def` doesn't help

Shiny for Python runs on a single asyncio event loop per process, and it
deliberately runs **reactive functions serially, never concurrently — even
when they are `async def`, even across sessions in the same process**. Two
`async` effects that each `await asyncio.sleep(1)` run one after the other;
one user's blocking call freezes every session in the process.
Shiny for Python runs every session in a process on a single asyncio event
loop. How `async def` reactives behave depends on the shiny version:

- **shiny < 1.9.0:** reactive functions run **serially, never concurrently —
even when they are `async def`, even across sessions**. Two `async` effects
that each `await asyncio.sleep(1)` run one after the other, and one user's
slow async effect freezes every session in the process.
- **shiny ≥ 1.9.0:** effects run concurrently. `async` effects interleave at
each `await`, and other sessions keep running while one awaits.
`priority` orders when effects start, not when they finish. A session
stays **busy** until all of its effects finish: it applies no input
changes and sends no outputs until then.

Consequences:

- `async def` on a calc/effect/renderer exists so you can call async-only
APIs (an async HTTP client, `session.send_custom_message`). **It does not
make the app faster or more responsive.** Replacing `time.sleep(5)` with
`await asyncio.sleep(5)` changes nothing for your users.
make the computation faster or keep the user's session responsive.**
Replacing `time.sleep(5)` with `await asyncio.sleep(5)` changes nothing on
shiny < 1.9.0; on shiny ≥ 1.9.0 it stops blocking *other* sessions, but
that user still waits 5 s with a frozen app.
- A *synchronous* call in any reactive function (`time.sleep`, `requests`,
a sync DuckDB/pandas query) blocks the event loop itself — nothing in the
process gets served until it returns. This is the single most destructive
pattern in a Shiny for Python app, and it's invisible in development with
one user.
process gets served until it returns, on any version. This is the single
most destructive pattern in a Shiny for Python app, and it's invisible in
development with one user.

### Background-task code on shiny ≥ 1.9.0

- `async with reactive.lock():` around a `.set()` followed by
`await reactive.flush()` is obsolete: `reactive.lock()` is deprecated and
does nothing, and `.set()` schedules its dependents itself. Drop the lock.
Keep the flush only if the code after it needs the dependents to have run,
and only outside an effect (e.g. an extended task's body). Inside an
effect, or a task an effect started that is still running, the flush
returns right away without waiting.
- To change state that a running effect still reads, wait until the
session's effects finish:
`session.run_once_when_idle(lambda: value.set(x))`. `fn` must be a
synchronous function.
- From another thread, never call `.set()` directly; use
`loop.call_soon_threadsafe(value.set, x)`.
- `ui.update_*()` calls and custom messages sent from a background task
reach the browser right away. Older versions held them until the next
input change, which looked like lag.

## The decision path

Expand Down Expand Up @@ -101,8 +129,8 @@ Semantics worth knowing cold:

## Offloading synchronous and CPU-bound work

`@reactive.extended_task` removes you from the reactive flush, but the body
still runs on the event loop. Match the offload to the work:
`@reactive.extended_task` lets the session's reactive cycle finish without
waiting for the task, but the body still runs on the event loop. Match the offload to the work:

| Work | Offload |
|---|---|
Expand Down Expand Up @@ -155,7 +183,7 @@ app.on_shutdown(pool.shutdown) # clean shutdown — don't skip
renderer.
4. **Don't mutate shared module-level objects** in the worker (they may be
pickled copies, or shared between sessions — either way it's wrong).
5. **Debugging is harder off the flush.** Errors re-raise in the consuming
5. **Debugging is harder outside the reactive cycle.** Errors re-raise in the consuming
output; py-spy and cProfile see less inside pools — profile the
synchronous version first, convert only the proven hotspot.

Expand All @@ -164,8 +192,9 @@ app.on_shutdown(pool.shutdown) # clean shutdown — don't skip
- `ui.input_task_button` shows a busy label and disables itself — feedback
instead of a frozen app.
- While a task runs the rest of the session stays live: verify with a
ticking output (`reactive.invalidate_later(1)` clock) in another session
or another output.
ticking output (`reactive.invalidate_later(1)` clock) in the **same**
session. (On shiny ≥ 1.9.0 a clock in *another* session keeps ticking even
during a slow async effect, so it doesn't prove the task works.)
- Busy indicators for *outputs* (rendering-ui.md) complement task buttons:
buttons for triggered work, spinners for render-time work.

Expand Down
5 changes: 3 additions & 2 deletions shiny/shiny-py-optimize/references/data-loading.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,9 @@ def server(input, output, session):
with a cheap check** (`@reactive.poll(lambda: SELECT MAX(updated_at)...)`)
instead of re-reading on a timer.
- Use async drivers (`asyncpg`, `aiosqlite`) only where you'd otherwise block
the loop — and remember async reactive code still runs serially
(async-tasks.md); extended tasks remain the responsiveness mechanism.
the loop — and remember a slow async reactive still holds its own session
(and every session on shiny < 1.9.0; async-tasks.md); extended tasks remain
the responsiveness mechanism.

## Downloads shouldn't recompute the dataset

Expand Down
10 changes: 6 additions & 4 deletions shiny/shiny-py-optimize/references/diagnosis.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ session_start
- **Widest repeated spans** are the bottleneck; one-time towers at session
start are per-session setup work (data-loading.md, Rule 1).
- `SHINY_OTEL_COLLECT` levels, least to most detail: `none`, `session`,
`reactive_update` (one span per flush — a good production level),
`reactive_update` (one span per flush on shiny < 1.9.0, one per session
cycle on shiny ≥ 1.9.0 — a good production level),
`reactivity` (a span per calc/effect/output — the development level),
`all` (currently same as `reactivity`).
- Production overhead control: lower the level, sample
Expand Down Expand Up @@ -145,9 +146,10 @@ async-tasks.md come before the infrastructure knobs below.

One Shiny for Python process = one asyncio event loop = one Python thread:

- **Reactive functions run serially, never concurrently — even async ones,
even across sessions.** One blocking call ties up every session on the
process. This is why code fixes come first (async-tasks.md).
- **One sync blocking call ties up every session on the process.** On
shiny < 1.9.0 a slow `async` reactive does too (reactive functions run
serially across sessions); on shiny ≥ 1.9.0 it holds only its own session.
This is why code fixes come first (async-tasks.md).
- **Sessions hold state in memory**, so a browser must keep talking to the
*same* process: sticky-session load balancing is mandatory whenever more
than one app process serves a URL. This rules out `uvicorn --workers >1`,
Expand Down
5 changes: 3 additions & 2 deletions shiny/shiny-py-optimize/references/reactive-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,9 @@ feature is tracked in py-shiny issues #564 and #1814). Options, best first:
```
Semantics that matter: debounce rate-limits the **invalidation signal**,
not execution — wrap only a cheap read (a direct input read) when the
*downstream* work is expensive. Timings are **minimums**: reactive code
runs serially, so a slow downstream computation can exceed the window.
*downstream* work is expensive. Timings are **minimums**: a slow
downstream computation keeps the session busy, and the timer can't fire
until it finishes.
Rely on the built-in client-side debouncing for ordinary sliders and text
inputs; add the helper only when profiling shows an invalidation storm.

Expand Down
10 changes: 7 additions & 3 deletions shiny/shiny-py-optimize/references/rendering-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ entirely.
virtualizes rows in the browser and is the right tool for anything sizable.
- Only return the columns the user needs (select before returning; wide
frames serialize slowly).
- shiny ≥ 1.9.0 serializes `@render.data_frame` 2–3.5× faster with less
memory (most for pandas datetime/timedelta, `Decimal`, and `pd.NA`
columns). Upgrading is a free win for table-heavy apps.
- Update without a full re-render: from an effect,
`await grid.update_data(new_df)` replaces the data while keeping
sort/filter state.
Expand All @@ -98,9 +101,10 @@ entirely.
operations feel faster.
- **Let the UI render first**: render the shell immediately and stream
results in, rather than blocking on data before first paint.
- One slow output holds up the flush that delivers *all* outputs of an
interaction — gate slow computes behind buttons or extended tasks so fast
outputs aren't held hostage.
- One slow output holds up the session cycle that delivers *all* outputs of an
interaction (on shiny < 1.9.0, the flush for every session) — gate slow
computes behind buttons or extended tasks so fast outputs aren't held
hostage.

## Quick reference

Expand Down
Loading