Skip to content

Async @render_widget outputs close each other's widgets after posit-dev/py-shiny#2549 #241

Closed
posit-dev/py-shiny
#2549

Description

@jat255

This problem does not happen with any released version of py-shiny. It starts when posit-dev/py-shiny#2549 merges. That PR changes py-shiny, not shinywidgets, but the fix needs a change here, so I am opening this issue to track it.

Summary

posit-dev/py-shiny#2549 runs each output in its own asyncio task, so async outputs in one session interleave at each await. Before, a flush ran outputs one after another, and each finished (including its awaits) before the next started.

shinywidgets keeps "the output that is rendering now" in one value on the session (WidgetRenderContext). __enter__ saves the old value and __exit__ restores it. This works only when renders are strictly nested. With interleaved outputs, a widget can be tied to the wrong output. These are the results:

  • Re-rendering one output closes the widget that another output shows. The other chart disappears from the page.
  • A widget can be tied to no output, so it is never closed and leaks.
  • After the renders finish, the session can still name an output as rendering. A widget built later outside a render function is then tied to that output.

Only async @render_widget functions that await before they build the widget are affected. Sync render functions cannot interleave.

shinywidgets-241-demo.mp4

Reproduce

  1. Install py-shiny from the PR branch: pip install "shiny @ git+https://github.com/posit-dev/py-shiny@schloerke/async-pr-2182-reimpl", and also shinywidgets and plotly.
  2. Run the app below, and wait for both charts.
  3. Click Re-render Forecast. The Sales chart disappears.
  4. Click Re-render Sales. The Sales chart comes back, but "Live widgets on the server" shows 3 instead of 2, because the first Forecast figure was never closed.

With the released py-shiny, both charts stay and the count stays at 2.

app.py
# Demo: two async @render_widget outputs in one session.
#
# Each output awaits before it builds its figure. "Sales" builds its figure while
# "Forecast" is still awaiting. shinywidgets records the output that is rendering
# in one slot on the session, so on the async stack the Sales figure is tied to the
# Forecast output.
#
# To see the problem: open the app, wait for both charts, then click
# "Re-render Forecast". On the async stack the Sales chart disappears. On main it
# stays. Then click "Re-render Sales": the "Live widgets" count goes to 3, because
# the first Forecast figure was never closed.
#
# Run with: shiny run app.py
import asyncio
from pathlib import Path

import plotly.graph_objects as go
import shiny
from shiny import App, reactive, render, ui
from shinywidgets import output_widget, render_widget
from shinywidgets._shinywidgets import WIDGET_INSTANCE_MAP

app_ui = ui.page_fillable(
    ui.h3("Async @render_widget outputs"),
    ui.p(
        "Shiny loaded from ",
        ui.code(str(Path(shiny.__file__).parent.parent)),
        f" (version {shiny.__version__})",
    ),
    ui.layout_columns(
        ui.input_action_button("rerender_sales", "Re-render Sales"),
        ui.input_action_button("rerender_forecast", "Re-render Forecast"),
        ui.output_text("live", inline=True),
        col_widths=(3, 3, 6),
    ),
    ui.layout_columns(
        ui.card(ui.card_header("Sales (awaits 0.5 s)"), output_widget("sales")),
        ui.card(ui.card_header("Forecast (awaits 1.0 s)"), output_widget("forecast")),
    ),
)


def server(input, output, session):
    @render_widget
    async def sales():
        n = input.rerender_sales()
        await asyncio.sleep(0.5)
        return go.FigureWidget(
            data=[go.Bar(x=list("ABCD"), y=[3, 5, 2, 6])],
            layout=dict(title=f"Sales, render {n + 1}"),
        )

    @render_widget
    async def forecast():
        n = input.rerender_forecast()
        await asyncio.sleep(1.0)
        return go.FigureWidget(
            data=[go.Scatter(x=[1, 2, 3, 4], y=[2, 4, 3, 5])],
            layout=dict(title=f"Forecast, render {n + 1}"),
        )

    @render.text
    def live():
        reactive.invalidate_later(0.5)
        # Counts every widget in the server process, so open only one tab.
        return f"Live widgets on the server: {len(WIDGET_INSTANCE_MAP)} (2 with one tab open)"


app = App(app_ui, server)

Cause

Sales awaits 0.5 s and Forecast awaits 1.0 s, and both start in the same flush:

Time Event Session's "rendering" value
0 Sales enters. It saves "none" and sets the value to Sales. Sales
0 Forecast enters. It saves "Sales" and sets the value to Forecast. Forecast
0.5 s Sales builds its figure. init_shiny_widget() reads Forecast, so the Sales figure closes when Forecast re-renders. Forecast
0.5 s Sales exits and restores "none". none
1.0 s Forecast builds its figure. init_shiny_widget() reads "none", so no output owns the figure. none
1.0 s Forecast exits and restores "Sales". Sales

Possible fix

Keep the current output id and render Context in a contextvars.ContextVar instead of in vars(session). Each asyncio task has its own copy of the context, so each output's task sees only its own value. Use ContextVar.set() in __enter__ and ContextVar.reset(token) in __exit__.

py-shiny has the same pattern for session.clientdata.output_*() with no id, and needs the same fix before posit-dev/py-shiny#2549 merges. If py-shiny exposes a task-safe "current output" as part of that fix, shinywidgets could read it instead of keeping its own value. Either way works.

Related: #221 added the render Context capture that this value stores.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions