Skip to content

[feat]: wire Dreamverse creation studio backend and capabilities - #1854

Closed
aryan5v wants to merge 18 commits into
hao-ai-lab:ds8/dreamversev2-devfrom
aryan5v:aryan5v/dreamverse-backend-wiring-9f15
Closed

aryan5v wants to merge 18 commits into
hao-ai-lab:ds8/dreamversev2-devfrom
aryan5v:aryan5v/dreamverse-backend-wiring-9f15

Conversation

@aryan5v

@aryan5v aryan5v commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Why

Dreamverse creation studio sends model, mode, aspect ratio, resolution, duration, and reference images over the websocket, but the server did not parse, validate, or apply those fields end-to-end. This PR wires session init, echoes resolved creation_config, enforces per-model capabilities, and routes creation overrides into generation.

Scope

Backend (apps/dreamverse/dreamverse/):

  • session_creation_config.py — parse/validate model, generation_mode, aspect, resolution, duration
  • creation_capabilities.py + GET /creation-capabilities — caps for fast-ltx2, fast-ltx23, fast-h3
  • session/controller.py, gpu_pool.py, ltx2_generation.py — per-step overrides; fix LTX2 frame-size crash
  • Reference image validation; mock_server.py parity
  • Wire field: generation_mode (t2va / ref2va; fl2va rejected)

Frontend wiring (apps/dreamverse/web/):

  • creationPayload.ts, creationCapabilities.ts — init payload, capability fetch, client validation
  • Read-only in-session config pills until live WS updates exist

Note: This branch includes the creation studio UI uplift commits if they are not on main yet. If the UI PR lands first, GitHub will shrink this diff to backend-only.

Tradeoffs

Blast Radius

apps/dreamverse/ only. No fastvideo/ pipeline or kernel changes.

Verification

cd apps/dreamverse/web && npm run typecheck && npm test
# 80 passed, 27 skipped

PYTHONPATH=apps/dreamverse python3 -m pytest \
  apps/dreamverse/dreamverse/tests/test_creation_capabilities.py \
  apps/dreamverse/dreamverse/tests/test_session_creation_config.py
# 15 passed

Mock-server smoke: uvicorn dreamverse.mock_server:app + Next.js with BACKEND_PORT=8009.


Fixes #

Checklist

  • Unit tests for capabilities + session creation config
  • Frontend typecheck + unit tests pass
  • pre-commit run --all-files
  • GPU smoke on real Dreamverse deployment

cursoragent and others added 18 commits September 8, 2026 04:00
Add a lobby-first creation studio with model/mode/config pills, @ mentions,
reference upload preview, discovery feed, and nav rail. UI-only state for
now; existing session playback and ChatBar continuation flow are preserved.

Uses minimal Radix/shadcn primitives aligned with blocks.so patterns.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
- Darker Dreamina-style composer, masonry discovery feed, nav rail polish
- Mode-aware references: optional T2V, dual FL2AV frames, required Ref2AV
- Session ChatBar config pills synced from lobby settings at session start
- Restore preset quick-launch rail; extract shared ConfigPill component

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Remove quick-action cards and discovery feed. Hide FastH3 from the model
picker. Rename reference modes to Omni reference and first/last frame assets.
Add drag-and-drop on reference upload slots.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Replace the center AI Video title with the original animated hero tagline.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Connect lobby model/mode/aspect/resolution/duration and reference images
to session_init_v2 and project_init_v1. Backend parses creation config,
validates mode assets, resolves frame size and segment cap, and passes
per-step sampling overrides to the GPU worker. Frontend builds init
payloads from creation state and applies echoed creation_config.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Align the UI uplift with hao-ai-lab#1835 by mapping creation
studio mode IDs to canonical t2va/fl2va/ref2va wire values in
session_init_v2 and project_init_v1 payloads.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Use canonical t2va/fl2va/ref2va from hao-ai-lab#1834/hao-ai-lab#1835 instead of a parallel
creation_mode field. Wire model, resolution, duration, and image assets
through session_creation_config while leaving H3 mode routing to upstream.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Playwright was matching placeholder copy as visible text; the lobby
composer only exposes it via the textarea placeholder attribute.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Expose GET /creation-capabilities from backend and mirror it in the
frontend so only supported models, modes (t2va/ref2va), resolutions
(480p-1080p), and image references are selectable. Reject FL2VA, 4K,
and invalid assets client-side and in session_creation_config. Lock
in-session config pills read-only until live updates exist.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Derive lobby model_ids from MODEL_REGISTRY and expose per-model creation
capabilities so FastH3 appears in the composer alongside FastLTX models.
H3 keeps fixed 16:9 / 720p-class geometry while LTX models retain the
full aspect and resolution surface.

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
@mergify mergify Bot added the type: feat New feature or capability label Sep 15, 2026
@mergify

mergify Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Pre-commit checks failed

Hi @aryan5v, the pre-commit checks have failed. To fix them locally:

# Install pre-commit if you haven't already
uv pip install pre-commit
pre-commit install

# Run all checks and auto-fix what's possible
pre-commit run --all-files

Common fixes:

  • yapf: yapf -i <file> (formatting)
  • ruff: ruff check --fix <file> (linting)
  • codespell: codespell --write-changes <file> (spelling)

After fixing, commit and push the changes. The checks will re-run automatically.

For future commits, pre-commit will run automatically on changed files before each commit.

@Davids048
Davids048 changed the base branch from main to ds8/dreamversev2-dev September 15, 2026 23:45
@mergify

mergify Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

This PR has merge conflicts with the base branch. Please rebase:

git fetch origin main
git rebase origin/main
# Resolve any conflicts, then:
git push --force-with-lease

@mergify mergify Bot added the needs-rebase PR has merge conflicts label Sep 16, 2026
Davids048 added a commit that referenced this pull request Sep 16, 2026
Apply the backend-wiring changes beyond the UI uplift to
ds8/dreamversev2-dev for review and refactoring.

Source PR: #1854
Source range: 90d739a..8c5ee9b
The 28-file patch is identical to that source range.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
@Davids048

Copy link
Copy Markdown
Collaborator

@Davids048 Davids048 closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase PR has merge conflicts type: feat New feature or capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants