Skip to content

feat(handler): add support for processing and uploading gif videos to S3#133

Open
peter064226 wants to merge 4 commits into
runpod-workers:mainfrom
peter064226:feat/support-video-output
Open

feat(handler): add support for processing and uploading gif videos to S3#133
peter064226 wants to merge 4 commits into
runpod-workers:mainfrom
peter064226:feat/support-video-output

Conversation

@peter064226
Copy link
Copy Markdown

Add logic to handle gif video outputs from ComfyUI nodes, including uploading to S3 bucket and cleaning up local files. The implementation follows the same pattern as image handling but with video-specific content type handling.

Motivation

​​User Demand​​: Support animated outputs (e.g., GIFs/MP4s) from video-generation nodes (e.g., AnimateDiff, GIF-compatible savers).
​​Scalability​​: Videos are larger than images and require dedicated S3 handling (e.g., multipart uploads for big files).
​​Maintenance​​: Unify media-handling logic (images + videos) to reduce tech debt.

Add logic to handle gif video outputs from ComfyUI nodes, including uploading to S3 bucket and cleaning up local files. The implementation follows the same pattern as image handling but with video-specific content type handling.
Split the output_data list into separate lists for images and videos to improve data organization and clarity. This allows for better handling of different media types in the final result.

Update the final result structure to include separate "images" and "videos" fields when present, and modify the completion log message to reflect both media types.
The error messages and status field were only referencing images when checking for empty output data. This change updates them to explicitly mention both images and videos to accurately reflect the possible output types and avoid confusion.
The handler function was not including an empty "videos" array in the final_result when no images or videos were processed, which could cause issues with clients expecting this field. This change ensures consistent response structure by initializing the field similar to how "images" is handled.
Copy link
Copy Markdown
Contributor

@TimPietruskyRunPod TimPietruskyRunPod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good direction — handling gifs (which ComfyUI uses for video/gif outputs, e.g. from ComfyUI-VideoHelperSuite or AnimateDiff) alongside images is a real gap in the current handler. Approving the concept, but several things need to change before merge:

Blocking

  1. Rebase required — the branch is behind main and conflicts. Several things have moved since you opened this (e.g. PR #197 added a process-liveness check, the response structure was clarified in #157's investigation).
  2. Variable rename is unnecessary and noisy — splitting output_data into output_data_images and output_data_videos adds churn for no functional benefit. The final response can just be:
    if output_data_images: final_result["images"] = output_data_images
    if output_data_videos: final_result["videos"] = output_data_videos
    …without renaming the existing images accumulator. Please keep output_data as the images list and introduce a new output_data_videos only.
  3. Hardcoded prefix=f"comfy/{job_id}" for video S3 uploads is inconsistent with how images are uploaded (which just uses rp_upload.upload_image(job_id, ...)). And it conflicts with #191 which is adding configurable BUCKET_PREFIX. Please align — either:
    • Use the same BUCKET_PREFIX env var (preferred), or
    • Use rp_upload.upload_image for symmetry.
  4. extra_args={"ContentType": f"video/{filename.split('.')[-1]}"} is dangerous: a .webp will become video/webp, .png will become video/png, etc. Either map known extensions (mp4, webm, gif, mov) to their MIME types explicitly, or skip the ContentType arg and let boto3 detect.
  5. No changeset — please add .changeset/feat-handle-video-outputs.md:
    ---
    "worker-comfyui": minor
    ---
    
    feat: detect `gifs` output entries from video/animation nodes (AnimateDiff, VideoHelperSuite, etc.) and return them as base64 or upload to S3 alongside images.

Non-blocking

  1. The local cleanup of os.remove(file_location) happens before checking that the S3 upload succeeded. If S3 upload throws, the original file is gone and we lose recoverability. Use try/finally like the image path does.

Happy to merge once the rebase and the items above are addressed. The feature itself is wanted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants