Skip to content

Add shiny-py-optimize skill for Shiny for Python performance - #93

Merged
gadenbuie merged 2 commits into
skill/shiny-optimizefrom
skill/shiny-py-optimize
Oct 14, 2026
Merged

gadenbuie merged 2 commits into
skill/shiny-optimizefrom
skill/shiny-py-optimize

Conversation

@gadenbuie

@gadenbuie gadenbuie commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Stacks on and pairs with #91

Summary

Adds shiny-py-optimize, a performance-optimization skill for Shiny for Python apps that mirrors the workflow of shiny-r-optimize (understand the complaint → scan for smells → measure → diagnose → fix cheapest-first → verify) while re-grounding every mechanic in Python:

  • Concurrency model: reactive functions run serially on a single asyncio event loop — even when async def, even across sessions — so one blocking call freezes every session. async/await is not a performance fix; only @reactive.extended_task is non-blocking (threads for sync I/O, process pools for CPU-bound work).
  • Caching: py-shiny has no bindCache equivalent (verified against the Shiny 1.8 API), so the skill teaches module-level functools.lru_cache/cachetools/diskcache patterns with hard rules (key completeness, read-only shared results, per-process caches).
  • Scoping: Express top-level code runs once per session — the most common silent trap; shared data belongs in imported modules.
  • Data: polars/parquet/DuckDB lazy loading with push-down, per-session DB connections with session.on_ended cleanup.
  • Reactivity: shared @reactive.calc, req(), @reactive.event() gating, a tested debounce helper (adapted from Joe Cheng's ratelimit.py gist; not built into py-shiny), and reactive.value.freeze() as the freezeReactiveValue() analogue.
  • Diagnosis: built-in OpenTelemetry tracing, py-spy, the new npm shinyloadtest rewrite for load testing, sticky-session scaling rules (never uvicorn --workers >1), and Connect/Connect Cloud capacity knobs.

Also updates shiny/README.md with the new skill section and research sources, and registers the skill in the shiny plugin's marketplace entry.

Verification

uv run count-skill-tokens.py shiny/shiny-py-optimize passes — all files within limits (warning thresholds are 5,000 tokens / 500 lines for SKILL.md and 100 tokens for the description):

Skill description: ~98 tokens

File Lines ~Tokens
SKILL.md 211 3,183
references/async-tasks.md 179 1,984
references/caching.md 147 1,686
references/data-loading.md 208 1,882
references/diagnosis.md 209 2,372
references/reactive-graph.md 300 2,792
references/rendering-ui.md 110 1,238

Key patterns were verified against a live py-shiny app (Shiny 1.8) with Playwright: the debounce helper (exactly one commit ~0.5s after the last keystroke, no intermediate commits), the ExtendedTask + input_task_button flow (busy state during run, result delivered, button reset), and input.freeze() as an update-loop breaker (never showed a mismatched country/city pair).

The shiny-r-optimize research-sources README commit was intentionally authored to match 7df2b22 on the base branch (same patch-id); the rebase onto skill/shiny-optimize skipped it as already applied, so this PR contains only the new skill.

@gadenbuie
gadenbuie marked this pull request as ready for review October 7, 2026 16:41
@gadenbuie
gadenbuie added this pull request to stack #94 October 7, 2026 16:41
@gadenbuie
gadenbuie force-pushed the skill/shiny-py-optimize branch from 1261cb8 to f25bdd8 Compare October 7, 2026 17:11
Mirror of shiny-r-optimize re-grounded in Python-specific mechanics:
the single asyncio event loop (reactive code runs serially even when
async; only @reactive.extended_task is non-blocking), manual module-
level caching (py-shiny has no bindCache equivalent), Core vs Express
scoping (Express top-level code runs once per session), and
polars/parquet/DuckDB lazy data loading.

- SKILL.md: same diagnostic workflow with a Python-ized smell table
- diagnosis.md: built-in OpenTelemetry tracing, py-spy, the npm
  shinyloadtest rewrite, sticky sessions, Connect capacity knobs
- reactive-graph.md: shared @reactive.calc, req/isolate/@reactive.event,
  reactive.value.freeze() loop-breaker, tested debounce helper
  (adapted from jcheng5's ratelimit.py gist; not built into py-shiny)
- caching.md: functools.lru_cache/cachetools/diskcache patterns and
  their hard rules
- async-tasks.md: ExtendedTask pattern, asyncio.to_thread for sync I/O,
  ProcessPoolExecutor for CPU-bound work
- data-loading.md: process-scope loading, lazy polars/DuckDB/ibis,
  per-session DB connections with session.on_ended cleanup
- rendering-ui.md: output suspension via tabs, @render.data_frame over
  @render.table, Plotly WebGL/aggregation, busy indicators

Key patterns were verified against a live py-shiny app (Shiny 1.8) with
Playwright: the debounce helper, the ExtendedTask + input_task_button
flow, and input freeze as an update-loop breaker. Registers the skill
in the shiny plugin's marketplace entry.
…thon

Mirrors eb1ef82 and f1bd14b from shiny-r-optimize, re-grounded in
py-shiny mechanics; R-specific changes (mirai-first async ordering,
vroom/readr, saveRDS verbatim) are skipped or adapted.

- SKILL.md: reactive ladder rung 4 — when updates compound, gate the
  batch behind ui.input_task_button() (drop-in for input_action_button).
  Report integrity paragraph now explains the outputs/measurements/
  folder layout (create if missing, one file per measurement).
- reactive-graph.md: debounce intro corrected — py-shiny's slider and
  text input bindings already carry a 250 ms client-side debounce rate
  policy (verified in py-shiny's client JS), so bursts matter only when
  a drag/edit outlasts the window or downstream compute exceeds the
  gaps; high-frequency client events are the clear-cut case. New
  subsection 'When smoothing isn't enough: gate behind a button':
  ui.input_task_button() disables itself and shows busy state until the
  server finishes the click-triggered reactivity, no server-side
  changes and no extended_task required (matches py docs).
- rendering-ui.md: gate-speculative-outputs notes the task-button swap;
  input_task_button bullet explains it pairs with extended_task which
  runs work off the event loop; quick-ref table row added.
- data-loading.md: precomputed artifacts that aren't data frames —
  serialize with pickle/joblib instead of a CSV round-trip (Python
  analogue of the saveRDS/readRDS note).
@gadenbuie
gadenbuie force-pushed the skill/shiny-py-optimize branch from f25bdd8 to ee937aa Compare October 14, 2026 20:26
@gadenbuie
gadenbuie merged commit 7cef2d7 into main Oct 14, 2026
1 check passed
@gadenbuie
gadenbuie deleted the skill/shiny-py-optimize branch October 14, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant