Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 15 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,8 @@ ref = (RasterPlot & key).fetch1('visualization')
print(ref.title) # "Spike Raster"
print(ref.description) # "Unit activity over time"

# Display in browser
# Serve it over HTTP and open it in a browser
ref.show()

# Or load the full FigpackView
view = ref.load()
```

### Jupyter Integration
Expand All @@ -86,7 +83,11 @@ ref # Shows title, description, and action hints

### Storage Structure

Visualizations are stored as Zarr folders under a **schema-addressed path chosen by the
A figpack figure is its bundle — viewer, data and extension manifest together — and the
whole folder is what gets stored. That is what lets a figure be served without `figpack`
installed, and what gives an extension view somewhere to keep its JavaScript.

Bundles are stored under a **schema-addressed path chosen by the
framework** (DataJoint's `build_object_path`): it mirrors the schema/table structure,
encodes primary keys as `attr=value` segments, and ends in a tokenized filename
(`{attribute}_{token}.zarr`), subject to the store's prefix/partitioning configuration —
Expand All @@ -96,6 +97,10 @@ for example:
{store_location}/demo_showcase/fluorescence_figpack/session_id=4/fig_NPhczfGY.zarr/
```

The stored folder holds `index.html`, `assets/`, `data.zarr/` and
`extension_manifest.json` — exactly what `FigpackView.save()` produced. Treat it as
opaque: a custom view may name its data folder differently or carry several.

The layout is browsable but framework-owned — do not hand-build or rely on exact paths;
the database column's metadata (`path`, `store`) is the source of truth.

Expand All @@ -114,18 +119,16 @@ Lazy reference returned when fetching `<figpack@>` attributes.
- `description` - Visualization description (no I/O)
- `path` - Storage path
- `store` - Store name
- `is_loaded` - Whether data has been cached

**Methods:**
- `load()` - Download and return the `FigpackView`
- `show(**kwargs)` - Download and display in browser
- `serve_under(base_dir)` - Materialize a servable viewer bundle; returns its relative URL
- `show(open_browser=True)` - Serve the figure over HTTP; returns its URL
- `serve_under(base_dir)` - Publish the stored bundle; returns its relative URL

### Serving a figure in a dashboard

`FigpackRef.serve_under(base_dir)` materializes a self-contained, servable viewer
bundle (figpack's viewer + the stored `data.zarr`) under `base_dir/<id>/` and returns
the relative URL `/<id>/index.html`. Dashboards (e.g. dash-datajoint-components'
`FigpackRef.serve_under(base_dir)` downloads the stored bundle into `base_dir/<id>/`
and returns the relative URL `/<id>/index.html`. Nothing is assembled and `figpack`
need not be installed in the serving process. Dashboards (e.g. dash-datajoint-components'
`PlotGrid`) serve `base_dir` over HTTP and embed the URL in an `<iframe>`; repeated
calls are idempotent and refresh the directory mtime for TTL-based cache cleaners.

Expand Down
4 changes: 2 additions & 2 deletions src/dj_figpack_codecs/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

This package provides a codec for storing figpack FigpackView objects
in DataJoint's schema-addressed object storage (OAS). Visualizations
are stored as Zarr folders and fetched lazily via FigpackRef.
are stored as self-contained bundles and fetched lazily via FigpackRef.

Usage::

Expand Down Expand Up @@ -34,7 +34,7 @@ def make(self, key):
# Fetch returns FigpackRef (lazy)
ref = Visualization.fetch1('figure')
print(ref.title) # No download
url = ref.serve_under("assets/serve") # Materialize a servable viewer bundle
url = ref.serve_under("assets/serve") # Publish the stored bundle, get its URL
"""

from .codec import FigpackCodec
Expand Down
40 changes: 17 additions & 23 deletions src/dj_figpack_codecs/codec.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""
FigpackCodec for storing figpack visualizations in DataJoint OAS.

This codec enables storing FigpackView objects as Zarr folders in
This codec enables storing FigpackView objects as self-contained bundles in
schema-addressed object storage, with lazy loading via FigpackRef.
"""

Expand All @@ -24,9 +24,9 @@

class FigpackCodec(SchemaCodec):
"""
Schema-addressed storage for figpack visualizations as Zarr folders.
Schema-addressed storage for figpack visualizations as self-contained bundles.

The ``<figpack@>`` codec stores FigpackView objects as Zarr folders under a
The ``<figpack@>`` codec stores FigpackView objects as figpack bundles under a
schema-addressed path chosen by the framework (mirrors schema/table, encodes
primary keys as ``attr=value``, tokenized ``{attribute}_{token}.zarr`` filename).
Visualizations are fetched lazily via ``FigpackRef``, which provides
Expand All @@ -35,11 +35,11 @@ class FigpackCodec(SchemaCodec):
Store only - requires ``@`` modifier.

Key Features:
- **Native format**: Stores as Zarr folder (figpack's native format)
- **Native format**: stores figpack's own bundle — viewer, data and
extension manifest together, so the object is what figpack produced
- **Lazy loading**: Metadata available without download
- **Dashboard serving**: ``ref.serve_under(base_dir)`` materializes a
self-contained viewer bundle (``ref.load()``/``ref.show()`` are not yet
implemented — see issue #3)
- **Serving**: ``ref.serve_under(base_dir)`` publishes the stored bundle and
``ref.show()`` serves it over HTTP; neither needs ``figpack`` installed
- **Jupyter integration**: Rich HTML display in notebooks
- **Schema-addressed**: Browsable paths that mirror database structure

Expand Down Expand Up @@ -76,7 +76,8 @@ def make(self, key):
url = ref.serve_under("assets/serve")

Storage Details:
- File format: Zarr folder (figpack native)
- File format: figpack bundle — index.html, assets/, data.zarr/ and the
extension manifest, exactly as ``FigpackView.save()`` emits them
- Path: schema-addressed, framework-chosen (e.g. ``{schema}/{table}/{pk_attr}={val}/{attribute}_{token}.zarr/``)
- Database column: JSON with ``{path, store, title, description}``

Expand All @@ -100,8 +101,7 @@ def validate(self, value: Any) -> None:
Raises
------
TypeError
If value is not a FigpackView instance, or is an extension-based
view (unsupported in data-only storage).
If value is not a FigpackView instance.
DataJointError
If the figpack package is not installed.
"""
Expand All @@ -119,14 +119,6 @@ def validate(self, value: Any) -> None:
if not isinstance(value, FigpackView):
raise TypeError(f"<figpack> requires figpack.FigpackView, got {type(value).__name__}")

from figpack.core.extension_view import ExtensionView

if isinstance(value, ExtensionView):
raise TypeError(
"<figpack> stores figure data only (data.zarr) and cannot yet preserve "
"extension JavaScript; extension-based views are not supported."
)

def encode(
self,
value: "FigpackView",
Expand All @@ -135,7 +127,7 @@ def encode(
store_name: str | None = None,
) -> dict:
"""
Save FigpackView as Zarr folder and upload to storage.
Save FigpackView as a bundle and upload the whole folder to storage.

Parameters
----------
Expand Down Expand Up @@ -184,13 +176,15 @@ def encode(
bundle_path = Path(tmpdir) / "bundle"

# figpack >= 0.3: save() requires keyword-only `title`. It emits a full
# viewer bundle (index.html + assets/ + data.zarr + extension manifest);
# we store ONLY data.zarr — the viewer is laid over it at render time by
# FigpackRef.serve_under(), so the store never duplicates viewer code.
# viewer bundle (index.html + assets/ + data.zarr + extension manifest)
# and the whole folder is the object: a figpack figure is not data with a
# viewer laid over it at render time, it is the bundle. Storing all of it
# is what lets the serving container render without figpack installed, and
# what gives an extension view somewhere to keep its JavaScript.
value.save(str(bundle_path), title=title, description=description)

backend = self._get_backend(store_name, config=config)
backend.put_folder(str(bundle_path / "data.zarr"), path)
backend.put_folder(str(bundle_path), path)

# Return metadata
return {
Expand Down
Loading