Stability-AI /generative-models

Generative Models by Stability AI

★ 27,340⑂ 3,091👁 276⑃ 128● 355▤ 2
PythonMITCreated 2023-06-22
Download: 0.1.0: SD-XL 1.0 with permissive license0.1.0 ↓0.1.02023-07-27
+0
Daily
+14
Weekly
+37
Monthly
27,340
stars
3,091
forks
276
watchers
128
prs
355
issues
2
releases

README

Generative Models by Stability AI

!sample1

 News


May 20, 2025
- We are releasing Stable Video 4D 2.0 (SV4D 2.0), an enhanced video-to-4D diffusion model for high-fidelity novel-view video synthesis and 4D asset generation. For research purposes:
    - SV4D 2.0 was trained to generate 48 frames (12 video frames x 4 camera views) at 576x576 resolution, given a 12-frame input video of the same size, ideally consisting of white-background images of a moving object.
    - Compared to our previous 4D model SV4D, SV4D 2.0 can generate videos with higher fidelity, sharper details during motion, and better spatio-temporal consistency. It also generalizes much better to real-world videos. Moreover, it does not rely on refernce multi-view of the first frame generated by SV3D, making it more robust to self-occlusions.
    - To generate longer novel-view videos, we autoregressively generate 12 frames at a time and use the previous generation as conditioning views for the remaining frames.
    - Please check our project page, arxiv paper and video summary for more details.

QUICKSTART :
- python scripts/sampling/simple_video_sample_4d2.py --input_path assets/sv4d_videos/camel.gif --output_folder outputs (after downloading sv4d2.safetensors from HuggingFace into checkpoints/)

To run SV4D 2.0 on a single input video of 21 frames:
- Download SV4D 2.0 model (sv4d2.safetensors) from here to checkpoints/: huggingface-cli download stabilityai/sv4d2.0 sv4d2.safetensors --local-dir checkpoints
- Run inference: python scripts/sampling/simple_video_sample_4d2.py --input_path 
    - input_path : The input video  can be
      - a single video file in gif or mp4 format, such as assets/sv4d_videos/camel.gif, or
      - a folder containing images of video frames in .jpg, .jpeg, or .png format, or
      - a file name pattern matching images of video frames.
    - num_steps : default is 50, can decrease to it to shorten sampling time.
    - elevations_deg : specified elevations (reletive to input view), default is 0.0 (same as input view).
    - Background removal : For input videos with plain background, (optionally) use rembg to remove background and crop video frames by setting --remove_bg=True. To obtain higher quality outputs on real-world input videos with noisy background, try segmenting the foreground object using Clipdrop or SAM2 before running SV4D.
    - Low VRAM environment : To run on GPUs with low VRAM, try setting --encoding_t=1 (of frames encoded at a time) and --decoding_t=1 (of frames decoded at a time) or lower video resolution like --img_size=512.

Notes:
- We also train a 8-view model that generates 5 frames x 8 views at a time (same as SV4D).
  - Download the model from huggingface: huggingface-cli download stabilityai/sv4d2.0 sv4d2_8views.safetensors --local-dir checkpoints
  - Run inference: python scripts/sampling/simple_video_sample_4d2.py --model_path checkpoints/sv4d2_8views.safetensors --input_path assets/sv4d_videos/chest.gif --output_folder o

Top issues

Latest release

0.1.0: SD-XL 1.0 with permissive license 0.1.0 2023-07-27
Release
- Releasing two new open models with a permissive CreativeML Open RAIL++-M license (see Inference for file hashes):
    - SDXL-base-1.0: An improved version over SDXL-base-0.9.
    - SDXL-refiner-1.0: An improved version over SDXL-refiner-0.9.
- Sharing the technical report on SDXL is now available here

!001_with_eval

 Contributions
- 17: Setup packaging with hatch
- 59: Freeze requirements for .pt13, .pt2 environments, add black formatter & automatic build tests
- 57, 66: Add automatic inference tests with pytest, add codeowner
- 69: Improve inference code
- 68, 67, 70,

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON