Video Chat 3

VideoChat3-4B visual video understanding for ComfyUI. The node samples a video in chunks, checkpoints partial results, and synthesizes a final answer. Audio is not transcribed.

This is an independent ComfyUI integration for the upstream MCG-NJU/VideoChat3 project. Model weights and upstream source code are not included in this repository.

Installation

Clone this repository into ComfyUI/custom_nodes, install its Python dependencies with the same Python environment used by ComfyUI, and restart ComfyUI:

cd ComfyUI/custom_nodes
git clone https://github.com/endman100/ComfyUI-VideoChat3.git
cd ComfyUI-VideoChat3
python -m pip install -r requirements.txt

The node requires a CUDA-capable NVIDIA GPU for practical inference. The default model is approximately 8.3 GiB to download and needs additional GPU memory while running.

Node

• Video Chat 3: chunked video analysis with model source/path, pinned revision, download permission, precision (auto, bf16, fp16, fp32), and attention backend on the same node. The legacy class ID VideoChat3LongVideoUnderstand is preserved for workflow compatibility.

The defaults load the pinned MCG-NJU/VideoChat3-4B revision with auto precision.

Model paths

Local models are discovered under ComfyUI/models/videochat3. The node also accepts an absolute model folder or a path relative to any registered videochat3 root. A model folder must contain config.json and its normal Transformers files.

To use extra_model_paths.yaml:

videochat3_models:
  base_path: D:/AI
  videochat3: models/videochat3

The Hugging Face choice uses the normal Hugging Face cache. With allow_download disabled, loading is local-only; with it enabled, the pinned revision may be downloaded when the main node executes.

VideoChat3 loads pinned model-side Python code with trust_remote_code=True. Keep the default pinned revision unless you intentionally review and test another revision.

GPU memory

The model is scoped to one execution. The main node unloads existing ComfyUI models before loading VideoChat3 and releases the VideoChat3 model plus CUDA cache after success or failure.

Limitations

• The node analyzes sampled visual frames and may miss brief events between samples.
• It does not transcribe or reason over the audio track.
• Results are model-generated and should be checked before high-stakes use.

Acknowledgements

VideoChat3 and its model weights are provided by the VideoChat3 authors. Follow the upstream repository and model card for their applicable terms and citations.

License

No license has been granted for this wrapper repository yet. Public visibility alone does not grant permission to copy, modify, or redistribute the wrapper code. The upstream project, model weights, and model-side code have their own applicable terms.