chigozienri/animate-diff-sdxl ✓🔢❓📝 → 🖼️

▶️ 965 runs 📅 Nov 2023 ⚙️ Cog 0.8.6 🔗 GitHub 📄 Paper ⚖️ License
animatediff sdxl text-to-video

About

Animate Your Personalized Text-to-Image Diffusion Models (Long boot times!)

Example Output

Prompt:

"A panda standing on a surfboard in the ocean in sunset, 4k, high resolution. Realistic, Cinematic, high resolution"

Output

Performance Metrics

84.59s Prediction Time
84.62s Total Time
All Input Parameters
{
  "seed": 42,
  "steps": 25,
  "aspect": "16:9",
  "prompt": "A panda standing on a surfboard in the ocean in sunset, 4k, high resolution. Realistic, Cinematic, high resolution",
  "n_prompt": "",
  "checkpoint": "dynavision",
  "video_length": 16,
  "motion_module": "mm_sdxl_v10_beta",
  "guidance_scale": 8.5,
  "use_checkpoint": false
}
Input Parameters
mp4 Type: booleanDefault: true
Returns .mp4 if true or .gif if false
seed Type: integerDefault: null
Seed (0 = random, maximum: 2147483647)
steps Type: integerDefault: 25
Number of inference steps
aspect Default: 1:1
Aspect ratio
prompt Type: stringDefault: A panda standing on a surfboard in the ocean in sunset, 4k, high resolution. Realistic, Cinematic, high resolution
Input prompt
n_prompt Type: stringDefault:
Negative prompt
checkpoint Default: dynavision
Select a model checkpoint
video_length Type: integerDefault: 16
Video length
motion_module Default: mm_sdxl_v10_beta
Select a Motion Model (currently only one available)
guidance_scale Type: numberDefault: 8.5
guidance scale
use_checkpoint Type: booleanDefault: false
Output Schema

Output

Type: stringFormat: uri

Example Execution Logs
loaded temporal unet's pretrained weights from /AnimateDiff/models/StableDiffusion/unet ...
### missing keys: 420;
### unexpected keys: 0;
### Temporal Module Parameters: 236.7792 M
Loading motion module from /AnimateDiff/models/Motion_Module/mm_sdxl_v10_beta.ckpt...
Using seed: 42
sampling: A panda standing on a surfboard in the ocean in sunset, 4k, high resolution. Realistic, Cinematic, high resolution ...
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:02<01:08,  2.84s/it]
  8%|▊         | 2/25 [00:04<00:53,  2.34s/it]
 12%|█▏        | 3/25 [00:06<00:48,  2.18s/it]
 16%|█▌        | 4/25 [00:08<00:44,  2.11s/it]
 20%|██        | 5/25 [00:10<00:41,  2.07s/it]
 24%|██▍       | 6/25 [00:12<00:38,  2.05s/it]
 28%|██▊       | 7/25 [00:14<00:36,  2.03s/it]
 32%|███▏      | 8/25 [00:16<00:34,  2.02s/it]
 36%|███▌      | 9/25 [00:18<00:32,  2.02s/it]
 40%|████      | 10/25 [00:20<00:30,  2.01s/it]
 44%|████▍     | 11/25 [00:22<00:28,  2.01s/it]
 48%|████▊     | 12/25 [00:24<00:26,  2.01s/it]
 52%|█████▏    | 13/25 [00:26<00:24,  2.01s/it]
 56%|█████▌    | 14/25 [00:28<00:22,  2.00s/it]
 60%|██████    | 15/25 [00:30<00:20,  2.00s/it]
 64%|██████▍   | 16/25 [00:32<00:18,  2.00s/it]
 68%|██████▊   | 17/25 [00:34<00:16,  2.00s/it]
 72%|███████▏  | 18/25 [00:36<00:14,  2.01s/it]
 76%|███████▌  | 19/25 [00:38<00:12,  2.01s/it]
 80%|████████  | 20/25 [00:40<00:10,  2.01s/it]
 84%|████████▍ | 21/25 [00:42<00:08,  2.01s/it]
 88%|████████▊ | 22/25 [00:44<00:06,  2.01s/it]
 92%|█████████▏| 23/25 [00:46<00:04,  2.01s/it]
 96%|█████████▌| 24/25 [00:48<00:02,  2.01s/it]
100%|██████████| 25/25 [00:50<00:00,  2.01s/it]
100%|██████████| 25/25 [00:50<00:00,  2.04s/it]
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
Input #0, gif, from 'output.gif':
Duration: 00:00:02.08, start: 0.000000, bitrate: 15053 kb/s
Stream #0:0: Video: gif, bgra, 1344x768, 7.67 fps, 23.08 tbr, 100 tbn, 100 tbc
Stream mapping:
Stream #0:0 -> #0:0 (gif (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x5591449f0900] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x5591449f0900] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 0x5591449f0900] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=15 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=17 ip_ratio=1.40 pb_ratio=1.30 aq=0
Output #0, mp4, to '/tmp/tmpregdz633/out.mp4':
Metadata:
encoder         : Lavf58.76.100
Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 1344x768, q=2-31, 23.08 fps, 17728 tbn
Metadata:
encoder         : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    3 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A dup=2 drop=0 speed=N/A
frame=   48 fps=0.0 q=17.0 size=    1024kB time=00:00:00.99 bitrate=8418.9kbits/s dup=32 drop=0 speed=1.93x
[mp4 @ 0x5591449ef1c0] Starting second pass: moving the moov atom to the beginning of the file
frame=   48 fps=0.0 q=-1.0 Lsize=    1714kB time=00:00:01.94 bitrate=7204.0kbits/s dup=32 drop=0 speed=2.71x
video:1713kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.082493%
[libx264 @ 0x5591449f0900] frame I:2     Avg QP:14.00  size:154402
[libx264 @ 0x5591449f0900] frame P:13    Avg QP:17.00  size: 89540
[libx264 @ 0x5591449f0900] frame B:33    Avg QP:18.67  size:  8504
[libx264 @ 0x5591449f0900] consecutive B-frames:  8.3%  0.0%  0.0% 91.7%
[libx264 @ 0x5591449f0900] mb I  I16..4:  8.4% 31.6% 60.0%
[libx264 @ 0x5591449f0900] mb P  I16..4:  6.9% 24.7% 32.8%  P16..4: 11.1%  6.9%  3.2%  0.0%  0.0%    skip:14.4%
[libx264 @ 0x5591449f0900] mb B  I16..4:  0.5%  1.6%  2.7%  B16..8:  6.9%  1.3%  0.3%  direct: 1.1%  skip:85.7%  L0:44.9% L1:48.4% BI: 6.7%
[libx264 @ 0x5591449f0900] 8x8 transform intra:36.5% inter:69.5%
[libx264 @ 0x5591449f0900] coded y,uvDC,uvAC intra: 74.2% 90.2% 80.4% inter: 7.3% 11.5% 2.9%
[libx264 @ 0x5591449f0900] i16 v,h,dc,p: 29% 32%  9% 30%
[libx264 @ 0x5591449f0900] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  9% 51% 11%  3%  4%  3% 10%  2%  7%
[libx264 @ 0x5591449f0900] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 49%  9%  3%  4%  3%  9%  3%  6%
[libx264 @ 0x5591449f0900] i8c dc,h,v,p: 31% 50%  8% 11%
[libx264 @ 0x5591449f0900] Weighted P-Frames: Y:23.1% UV:23.1%
[libx264 @ 0x5591449f0900] ref P L0: 54.7%  9.2% 19.8% 13.8%  2.5%
[libx264 @ 0x5591449f0900] ref B L0: 81.3%  7.0% 11.6%
[libx264 @ 0x5591449f0900] ref B L1: 99.8%  0.2%
[libx264 @ 0x5591449f0900] kb/s:6745.89
saved to file
Version Details
Version ID
ac050d5ed78aab352dbb948a3dbb1d4bba95cacd1e6f458e30617ecc8fa021e9
Version Created
November 13, 2023
Run on Replicate →