lucataco/animate-diff ❓🔢📝 → 🖼️

▶️ 316.4K runs 📅 Jul 2023 ⚙️ Cog 0.8.6 🔗 GitHub 📄 Paper ⚖️ License
animatediff text-to-video

About

Animate Your Personalized Text-to-Image Diffusion Models

Example Output

Prompt:

"masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes"

Output

Performance Metrics

53.58s Prediction Time
95.61s Total Time
All Input Parameters
{
  "path": "toonyou_beta3.safetensors",
  "seed": 255224557,
  "steps": 25,
  "prompt": "masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes",
  "n_prompt": "badhandv4, easynegative, ng_deepnegative_v1_75t, verybadimagenegative_v1.3, bad-artist, bad_prompt_version2-neg, teeth",
  "motion_module": "mm_sd_v14",
  "guidance_scale": 7.5
}
Input Parameters
path Default: toonyou_beta3.safetensors
Select a Module
seed Type: integer
Seed (0 = random, maximum: 2147483647)
steps Type: integerDefault: 25Range: 1 - 100
Number of inference steps
prompt Type: stringDefault: masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes
Input prompt
n_prompt Type: stringDefault:
Negative prompt
motion_module Default: mm_sd_v14
Select a Motion Model
guidance_scale Type: numberDefault: 7.5Range: 1 - 10
guidance scale
Output Schema

Output

Type: stringFormat: uri

Example Execution Logs
loaded temporal unet's pretrained weights from /AnimateDiff/models/StableDiffusion/stable-diffusion-v1-5/unet ...
### missing keys: 560;
### unexpected keys: 0;
### Temporal Module Parameters: 417.1376 M
Using seed: 255224557
sampling: masterpiece, best quality, 1girl, solo, cherry blossoms, hanami, pink flower, white flower, spring season, wisteria, petals, flower, plum blossoms, outdoors, falling petals, white hair, black eyes ...
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:01<00:38,  1.59s/it]
  8%|▊         | 2/25 [00:03<00:36,  1.59s/it]
 12%|█▏        | 3/25 [00:04<00:34,  1.59s/it]
 16%|█▌        | 4/25 [00:06<00:33,  1.59s/it]
 20%|██        | 5/25 [00:07<00:31,  1.59s/it]
 24%|██▍       | 6/25 [00:09<00:30,  1.59s/it]
 28%|██▊       | 7/25 [00:11<00:28,  1.59s/it]
 32%|███▏      | 8/25 [00:12<00:27,  1.59s/it]
 36%|███▌      | 9/25 [00:14<00:25,  1.59s/it]
 40%|████      | 10/25 [00:15<00:23,  1.59s/it]
 44%|████▍     | 11/25 [00:17<00:22,  1.59s/it]
 48%|████▊     | 12/25 [00:19<00:20,  1.59s/it]
 52%|█████▏    | 13/25 [00:20<00:19,  1.59s/it]
 56%|█████▌    | 14/25 [00:22<00:17,  1.59s/it]
 60%|██████    | 15/25 [00:23<00:15,  1.59s/it]
 64%|██████▍   | 16/25 [00:25<00:14,  1.59s/it]
 68%|██████▊   | 17/25 [00:27<00:12,  1.59s/it]
 72%|███████▏  | 18/25 [00:28<00:11,  1.59s/it]
 76%|███████▌  | 19/25 [00:30<00:09,  1.59s/it]
 80%|████████  | 20/25 [00:31<00:07,  1.59s/it]
 84%|████████▍ | 21/25 [00:33<00:06,  1.59s/it]
 88%|████████▊ | 22/25 [00:35<00:04,  1.59s/it]
 92%|█████████▏| 23/25 [00:36<00:03,  1.59s/it]
 96%|█████████▌| 24/25 [00:38<00:01,  1.59s/it]
100%|██████████| 25/25 [00:39<00:00,  1.59s/it]
100%|██████████| 25/25 [00:39<00:00,  1.59s/it]
  0%|          | 0/16 [00:00<?, ?it/s]
 31%|███▏      | 5/16 [00:00<00:00, 46.80it/s]
 62%|██████▎   | 10/16 [00:00<00:00, 24.55it/s]
 88%|████████▊ | 14/16 [00:00<00:00, 21.66it/s]
100%|██████████| 16/16 [00:00<00:00, 22.73it/s]
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
Input #0, gif, from 'output.gif':
Duration: 00:00:02.08, start: 0.000000, bitrate: 6722 kb/s
Stream #0:0: Video: gif, bgra, 512x512, 7.67 fps, 23.08 tbr, 100 tbn, 100 tbc
Stream mapping:
Stream #0:0 -> #0:0 (gif (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x557ff6e9a080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x557ff6e9a080] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x557ff6e9a080] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=15 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=17 ip_ratio=1.40 pb_ratio=1.30 aq=0
Output #0, mp4, to '/tmp/tmp3xgjte17/out.mp4':
Metadata:
encoder         : Lavf58.76.100
Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, progressive), 512x512, q=2-31, 23.08 fps, 17728 tbn
Metadata:
encoder         : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    3 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A dup=2 drop=0 speed=N/A
[mp4 @ 0x557ff6e98e80] Starting second pass: moving the moov atom to the beginning of the file
frame=   48 fps=0.0 q=-1.0 Lsize=     696kB time=00:00:01.94 bitrate=2924.6kbits/s dup=32 drop=0 speed=7.18x
video:695kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.200772%
[libx264 @ 0x557ff6e9a080] frame I:1     Avg QP:14.00  size: 64566
[libx264 @ 0x557ff6e9a080] frame P:12    Avg QP:17.00  size: 44144
[libx264 @ 0x557ff6e9a080] frame B:35    Avg QP:18.66  size:  3323
[libx264 @ 0x557ff6e9a080] consecutive B-frames:  2.1%  0.0%  6.2% 91.7%
[libx264 @ 0x557ff6e9a080] mb I  I16..4:  6.3% 20.0% 73.6%
[libx264 @ 0x557ff6e9a080] mb P  I16..4:  5.6% 18.1% 36.8%  P16..4: 11.3% 16.5% 11.6%  0.0%  0.0%    skip: 0.1%
[libx264 @ 0x557ff6e9a080] mb B  I16..4:  0.3%  0.8%  2.3%  B16..8:  3.5%  1.9%  1.1%  direct: 1.0%  skip:89.1%  L0:43.5% L1:36.2% BI:20.2%
[libx264 @ 0x557ff6e9a080] 8x8 transform intra:27.9% inter:33.2%
[libx264 @ 0x557ff6e9a080] coded y,uvDC,uvAC intra: 74.8% 93.3% 84.6% inter: 13.4% 17.0% 11.7%
[libx264 @ 0x557ff6e9a080] i16 v,h,dc,p: 36% 28% 15% 21%
[libx264 @ 0x557ff6e9a080] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 12% 16%  9% 11% 11% 10%  8%  8%
[libx264 @ 0x557ff6e9a080] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 14% 15%  8% 11% 10%  9%  7%  6%
[libx264 @ 0x557ff6e9a080] i8c dc,h,v,p: 51% 17% 19% 13%
[libx264 @ 0x557ff6e9a080] Weighted P-Frames: Y:58.3% UV:58.3%
[libx264 @ 0x557ff6e9a080] ref P L0: 49.1% 18.4% 20.0%  7.2%  5.3%
[libx264 @ 0x557ff6e9a080] ref B L0: 74.2%  9.9% 15.9%
[libx264 @ 0x557ff6e9a080] ref B L1: 99.5%  0.5%
[libx264 @ 0x557ff6e9a080] kb/s:2733.88
saved to file
Version Details
Version ID
beecf59c4aee8d81bf04f0381033dfa10dc16e845b4ae00d281e2fa377e48a9f
Version Created
September 24, 2023
Run on Replicate →