README

Generative Models by Stability AI

!sample1

 News


May 20, 2025
- We are releasing Stable Video 4D 2.0 (SV4D 2.0), an enhanced video-to-4D diffusion model for high-fidelity novel-view video synthesis and 4D asset generation. For research purposes:
    - SV4D 2.0 was trained to generate 48 frames (12 video frames x 4 camera views) at 576x576 resolution, given a 12-frame input video of the same size, ideally consisting of white-background images of a moving object.
    - Compared to our previous 4D model SV4D, SV4D 2.0 can generate videos with higher fidelity, sharper details during motion, and better spatio-temporal consistency. It also generalizes much better to real-world videos. Moreover, it does not rely on refernce multi-view of the first frame generated by SV3D, making it more robust to self-occlusions.
    - To generate longer novel-view videos, we autoregressively generate 12 frames at a time and use the previous generation as conditioning views for the remaining frames.
    - Please check our project page, arxiv paper and video summary for more details.

QUICKSTART :
- python scripts/sampling/simple_video_sample_4d2.py --input_path assets/sv4d_videos/camel.gif --output_folder outputs (after downloading sv4d2.safetensors from HuggingFace into checkpoints/)

To run SV4D 2.0 on a single input video of 21 frames:
- Download SV4D 2.0 model (sv4d2.safetensors) from here to checkpoints/: huggingface-cli download stabilityai/sv4d2.0 sv4d2.safetensors --local-dir checkpoints
- Run inference: python scripts/sampling/simple_video_sample_4d2.py --input_path 
    - input_path : The input video  can be
      - a single video file in gif or mp4 format, such as assets/sv4d_videos/camel.gif, or
      - a folder containing images of video frames in .jpg, .jpeg, or .png format, or
      - a file name pattern matching images of video frames.
    - num_steps : default is 50, can decrease to it to shorten sampling time.
    - elevations_deg : specified elevations (reletive to input view), default is 0.0 (same as input view).
    - Background removal : For input videos with plain background, (optionally) use rembg to remove background and crop video frames by setting --remove_bg=True. To obtain higher quality outputs on real-world input videos with noisy background, try segmenting the foreground object using Clipdrop or SAM2 before running SV4D.
    - Low VRAM environment : To run on GPUs with low VRAM, try setting --encoding_t=1 (of frames encoded at a time) and --decoding_t=1 (of frames decoded at a time) or lower video resolution like --img_size=512.

Notes:
- We also train a 8-view model that generates 5 frames x 8 views at a time (same as SV4D).
  - Download the model from huggingface: huggingface-cli download stabilityai/sv4d2.0 sv4d2_8views.safetensors --local-dir checkpoints
  - Run inference: python scripts/sampling/simple_video_sample_4d2.py --model_path checkpoints/sv4d2_8views.safetensors --input_path assets/sv4d_videos/chest.gif --output_folder o