Feature: Reporting Dashboard

[fderuiter]

Step 1: Scaffold the New Plugin Package

Create a new directory for the Streamlit plugin alongside your other packages. By isolating it, users who only want to write backend scripts won't be forced to install Streamlit.

Directory Structure:

imednet-python-sdk/
├── packages/
│   ├── core/                        
│   ├── plugins-workflows/           
│   ├── providers-airflow/           
│   └── plugins-streamlit/           <-- NEW DIRECTORY
│       ├── pyproject.toml
│       └── src/imednet_streamlit/
│           ├── __init__.py
│           ├── app.py               <-- The main Streamlit dashboard
│           └── components/          <-- Reusable UI widgets

Step 2: Define the Isolated Dependencies

Create the pyproject.toml inside packages/plugins-streamlit/. Notice how it explicitly depends on your core imednet package and optionally the imednet-workflows package if you want to reuse the Pandas logic you built there.

packages/plugins-streamlit/pyproject.toml

[project]
name = "imednet-streamlit"
version = "0.1.0"
description = "Interactive Streamlit reporting dashboards for iMednet EDC."
dependencies = [
    "imednet",             # Your core SDK
    "imednet-workflows",   # Optional: if you need your Pandas data extractors
    "streamlit>=1.30.0",
    "altair>=5.0.0"        # For charting
]

[build-system]
requires = ["hatchling"] # Or whichever build system your workspace uses
build-backend = "hatchling.build"

Step 3: Build the Dashboard Logic

Inside your new package, write the Streamlit application. It will import the strictly isolated core SDK to fetch data, ensuring all API calls go through your standardized retry policies and authentication layers.

packages/plugins-streamlit/src/imednet_streamlit/app.py

import streamlit as st
import pandas as pd
from imednet import ImednetSDK
from imednet.auth import APIKeyAuth

st.set_page_config(page_title="iMednet Reporting", layout="wide")

st.title("iMednet EDC Operations Dashboard")

# 1. Safely handle credentials via Streamlit's secrets management
api_key = st.sidebar.text_input("API Key", type="password")
study_key = st.sidebar.text_input("Study Key")

if api_key and study_key:
    # 2. Instantiate the Core SDK
    auth = APIKeyAuth(api_key=api_key, security_key="your-sec-key")
    sdk = ImednetSDK(auth=auth)
    
    st.success("Connected to iMednet API")

    # 3. Fetch Data using the SDK (Using caching to prevent API spam)
    @st.cache_data(ttl=600)
    def fetch_queries():
        # Using the core SDK to fetch the data
        queries = sdk.queries.list(study_key=study_key)
        # Convert to a format Streamlit/Pandas likes
        return pd.DataFrame([q.model_dump() for q in queries])

    with st.spinner("Fetching queries..."):
        df_queries = fetch_queries()
        
    # 4. Render UI
    st.subheader("Query Status Overview")
    st.dataframe(df_queries)
    
    # Example Altair Chart
    st.bar_chart(df_queries['annotationStatus'].value_counts())
else:
    st.warning("Please enter your credentials in the sidebar to begin.")

Step 4: Dynamically Attach it to the Core CLI

The best developer experience is allowing users to launch the dashboard directly from your CLI (e.g., imednet dashboard).

Update your core CLI to dynamically load a dashboard command, degrading gracefully if the user hasn't installed the imednet-streamlit package.

packages/core/src/imednet/cli/main.py

import typer
import subprocess
import sys

app = typer.Typer(help="iMednet Developer CLI")

# ... (Existing core commands) ...

@app.command("dashboard")
def run_dashboard():
    """
    Launch the interactive iMednet Streamlit dashboard.
    (Requires the 'imednet-streamlit' plugin)
    """
    try:
        # Check if the package is installed
        import imednet_streamlit
        import imednet_streamlit.app as app_module
        
        # Resolve the absolute path to the app.py file
        app_path = app_module.__file__
        
        typer.secho("Launching iMednet Dashboard...", fg=typer.colors.GREEN)
        
        # Subprocess call to launch the streamlit server
        subprocess.run([sys.executable, "-m", "streamlit", "run", app_path])
        
    except ImportError:
        typer.secho(
            "Dashboard plugin not found. Please install it by running:\n"
            "pip install imednet-streamlit", 
            fg=typer.colors.RED
        )
        raise typer.Exit(code=1)

Step 5: Update the Workspace Build Tooling

Finally, ensure your root orchestrator (like your Makefile) knows about the new package so it gets linted and tested automatically.

Makefile

.PHONY: test lint

# Added the new package to the list
PACKAGES = packages/core packages/plugins-workflows packages/providers-airflow packages/plugins-streamlit

lint:
	@for pkg in $(PACKAGES); do \
		echo "Linting $$pkg..."; \
		ruff check $$pkg/src $$pkg/tests; \
	done

Why this approach is bulletproof:

1. Zero Core Bloat: Users deploying your standard SDK in an AWS Lambda function won't be forced to download the massive streamlit and altair binaries.
2. Seamless UX: Users who do want the UI just run pip install imednet-streamlit and suddenly the imednet dashboard CLI command magically starts working.
3. API Consistency: The Streamlit app isn't making rogue requests.get() calls; it relies entirely on your hardened packages/core/src/imednet/sdk.py, meaning it automatically inherits all the retry logic and structural validation you've already built!

[fderuiter]

Feature: Implement imednet-streamlit Dashboard Plugin Workspace

Labels: enhancement, architecture, plugin, ui
Assignees: TBD
Epic: Monorepo Decoupling & Ecosystem Expansion

📖 Context & Background

Following our successful transition to a multi-package workspace (monorepo), we have strictly isolated our lightweight core SDK (packages/core) from opinionated workflows and heavy dependencies.

There is a growing need for interactive, read-only operational dashboards (e.g., Query tracking, Subject enrollment status) for Clinical Data Managers. We will implement this using Streamlit. To prevent bloating the core SDK for serverless/backend users, this UI must be built as a strictly isolated, optional plugin package (imednet-streamlit).

🎯 Objectives

1. Create a new standalone package packages/plugins-streamlit.
2. Build an interactive, multi-page Streamlit application that consumes the imednet core SDK.
3. Dynamically attach a dashboard command to the core CLI that gracefully degrades if the UI plugin is not installed.
4. Ensure zero UI/Charting dependency bleed into the core package.

---

🏗️ Architectural Specifications

1. Package scaffolding

• Location: packages/plugins-streamlit/
• Build System: Hatchling (aligned with workspace standards).
• Dependencies:
• imednet (Workspace reference)
• streamlit >= 1.30.0
• altair >= 5.0.0
• pandas >= 2.0.0 (Permitted here for DataFrame rendering)

2. Core SDK Integration Rules

• Strict SDK Usage: The Streamlit app MUST NOT use requests or httpx directly. All network calls must be routed through ImednetSDK(auth=...). This ensures the UI automatically benefits from our standardized retry policies, error handling, and logging.
• State Management: Credentials (API Keys, Study Keys) must be handled securely via st.session_state or st.secrets. Do not hardcode or log credentials.
• Caching: Heavy endpoints (e.g., /queries, /records) must be decorated with @st.cache_data(ttl=600) to prevent API rate-limiting on UI re-renders.

3. CLI Dynamic Integration

The core CLI (packages/core/src/imednet/cli/main.py) must be updated to include an imednet dashboard command.

• Lazy Loading: Use importlib or a try/except ImportError block to load the Streamlit module.
• Graceful Degradation: If imednet-streamlit is not installed in the user's environment, the command must catch the exception and print a helpful, colorized error message instructing them to run pip install imednet-streamlit.

---

📋 Implementation Tasks

• Task 1: Scaffold the packages/plugins-streamlit directory structure and create the pyproject.toml.
• Task 2: Develop the base app.py featuring a sidebar for Authentication (API Key, Study Key) and basic connection validation.
• Task 3: Implement at least one data view (e.g., Query Status Overview) utilizing sdk.queries.list() mapped to an Altair chart and Pandas DataFrame.
• Task 4: Update packages/core/src/imednet/cli/main.py with the dynamic dashboard command using subprocess.run(["streamlit", "run", app_path]).
• Task 5: Update the root Makefile to include packages/plugins-streamlit in the lint, format, and test loops.

---

✅ Acceptance Criteria

• Scenario 1: Plugin is NOT installed

• Given a user has only installed imednet (core).

• When they run imednet dashboard.

• Then the CLI exits with code 1 and displays: "Dashboard plugin not found. Please install it by running: pip install imednet-streamlit".

• Scenario 2: Plugin IS installed

• Given a user has installed imednet-streamlit.

• When they run imednet dashboard.

• Then a local Streamlit server boots up on port 8501 and opens in their default browser.

• Scenario 3: Dependency Integrity

• Given a fresh virtual environment.

• When I run pip install ./packages/core.

• Then pip freeze does NOT show streamlit, altair, or pandas.

---

🛑 Definition of Done (DoD)

• Code passes all ruff linting and formatting checks (make lint).
• Types are strictly defined and pass mypy.
• Dependencies are correctly isolated in their respective pyproject.toml files.
• A section has been added to docs/plugins.md detailing how to install and launch the dashboard.

---

Principal Dev Note: Treat this UI strictly as a consumer of the Core SDK. If you find yourself writing complex data parsing logic inside the Streamlit views, pause and evaluate if that logic belongs in packages/core/src/imednet/models/ or packages/plugins-workflows/ instead.