Vera: A Layered Diffusion Model for Content-Preserving Video Editing #18

Model/Pipeline/Scheduler description

Vera: A Layered Diffusion Model for Content-Preserving Video Editing (arXiv:2606.23610v1).

Vera proposes content-preserving video editing by generating an edit layer plus an alpha matte and compositing it over the source video, instead of regenerating every pixel. The two concrete technical contributions are: (1) a Mixture-of-Transformers (MoT) extension of a text-to-video DiT — separate DiTs per layer that interact only through joint self-attention — and (2) a purpose-built layered dataset with alpha mattes, diverse scenes/dynamics, and visual effects (486K frames). The paper reports gains in content preservation on its quantitative benchmark and a human-preference study versus open-source video-editing baselines.

What blocks a clean implementation in diffusers:

• No call site in the library. diffusers hosts video DiT pipelines (CogVideoX, HunyuanVideo, Mochi, LTX), but none are layered/edit pipelines and none expose a compositing alpha-matte path or a per-layer DiT. There is no existing module that naturally hosts the MoT edit layer.
• Training-bound contribution. Vera is defined by training a MoT on a custom 486K-frame layered dataset. diffusers is inference-oriented; there is no trainer or alpha-matte dataset path here, so the paper's core contribution cannot land as a code slice — only as new weights produced outside the repo.
• No checkpoint/loader path. There is no released Vera checkpoint in a diffusers-compatible format and no MoT model class to load one into. A convert_vera_to_diffusers.py script (the natural diffusers idiom) has nothing to convert and nothing to convert into.
• No eval path to consume. diffusers has unit tests, not a benchmark harness that ingests external video-editing benchmarks.

How to unblock this:

• Will Vera's weights (and alpha-matte layer outputs) be publicly released, and in what format? Without a checkpoint there is no inference path to land.
• If weights arrive, do we want the full VeraPipeline + MoT transformer model class + conversion script (the standard diffusers contribution shape), and is someone signed up to maintain it?
• Is there appetite for a smaller slice — e.g. just the MoT joint-self-attention transformer block as a reusable model primitive, decoupled from the editing pipeline — or is the layered compositing inseparable from the contribution?
• Should training/dataset construction stay out of scope (consistent with the rest of the library) and be handled upstream, with diffusers hosting only inference + conversion?

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler.)

No code repository or model URL surfaced in the paper, recommendation envelope, or arxiv abstract page. License: (none detected) (class: no-code-link, compat: 0.30). Worth confirming the paper has an open release before investing in adoption.

Provide useful links for the implementation

• Paper: Vera: A Layered Diffusion Model for Content-Preserving Video Editing (arXiv:2606.23610v1)
• Code / model weights: none surfaced — no GitHub repository, no released checkpoint in a diffusers-compatible format, and no authors' @Handles known.

Why the orchestrator opened an Issue instead of a PR

Pre-flight routed this paper to Issue before any coding agent ran. Vera's contribution — a Mixture-of-Transformers layered-editing pipeline trained on a custom 486K-frame alpha-matte dataset — has no natural call site in this inference-only diffusers library, no checkpoint/loader path, and no training or dataset infra to host it. No selection rationale named an implementable subset, and every plausible slice (the MoT architecture, a layered pipeline, a conversion script) depends on released weights or a trainer that aren't present. Routed to the team for discussion rather than landing a speculative freestanding module.

Reopen this Issue if you want Outrider to revisit this paper later. While it stays closed, the orchestrator will not re-recommend the same paper.

Drafted by Outrider — paper: arXiv:2606.23610v1.