lucataco/magic-animate-openpose 🔢🖼️ → 🖼️

▶️ 2.2K runs 📅 Dec 2023 ⚙️ Cog 0.8.6 🔗 GitHub 📄 Paper ⚖️ License
image-to-video motion-transfer openpose video-consistent-character-generation

About

MagicAnimate using an OpenPose input video

Example Output

Output

Performance Metrics

226.13s Prediction Time
480.48s Total Time
All Input Parameters 
{
  "image": "https://replicate.delivery/pbxt/K0kzPvawpylCI7kbDaglA1HI55TcHwIizaPpA0kvymsFOqM3/dalle8.jpeg",
  "video": "https://replicate.delivery/pbxt/K0kzQQLaQjwf8MYBJUVX9mpoL38pGSzzgygXvgQnEcSmZhft/hiphop-openpose.mp4",
  "guidance_scale": 7.5,
  "num_inference_steps": 25
}
Input Parameters
seed Type: integer
Random seed. Leave blank to randomize the seed
image (required) Type: string
Input image
video (required) Type: string
Input motion video
guidance_scale Type: numberDefault: 7.5Range: 1 - 50
Scale for classifier-free guidance
num_inference_steps Type: integerDefault: 25Range: 1 - 200
Number of denoising steps
Output Schema

Output

Type: string • Format: uri

Example Execution Logs 
Using seed: 2244563620
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
0it [00:00, ?it/s]
0it [00:00, ?it/s]
/src/magic-animate/magicanimate/pipelines/pipeline_animation.py:43: FutureWarning: Importing `DiffusionPipeline` or `ImagePipelineOutput` from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead.
from diffusers.pipeline_utils import DiffusionPipeline
loaded temporal unet's pretrained weights from /src/pretrained_models/stable-diffusion-v1-5/unet ...
### missing keys: 560;
### unexpected keys: 0;
### Temporal Module Parameters: 417.1376 M
It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8.
It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8.
It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8.
It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8.
/src/magic-animate/magicanimate/pipelines/pipeline_animation.py:103: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 0,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
is outdated. `steps_offset` should be set to 1 instead of 0. Please make sure to update the config accordingly as leaving `steps_offset` might led to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` file
deprecate("steps_offset!=1", "1.0.0", deprecation_message, standard_warn=False)
/src/magic-animate/magicanimate/pipelines/pipeline_animation.py:116: FutureWarning: The configuration file of this scheduler: DDIMScheduler {
"_class_name": "DDIMScheduler",
"_diffusers_version": "0.21.4",
"beta_end": 0.012,
"beta_schedule": "linear",
"beta_start": 0.00085,
"clip_sample": true,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "epsilon",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"set_alpha_to_one": true,
"steps_offset": 1,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null
}
has not set the configuration `clip_sample`. `clip_sample` should be set to False in the configuration file. Please make sure to update the config accordingly as not setting `clip_sample` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` file
deprecate("clip_sample not set", "1.0.0", deprecation_message, standard_warn=False)
current seed: 2244563620
  0%|          | 0/1 [00:00<?, ?it/s]/src/magic-animate/magicanimate/pipelines/pipeline_animation.py:624: FutureWarning: Accessing config attribute `in_channels` directly via 'UNet3DConditionModel' object attribute is deprecated. Please access 'in_channels' over 'UNet3DConditionModel's config object instead, e.g. 'unet.config.in_channels'.
num_channels_latents = self.unet.in_channels
  0%|          | 0/25 [00:00<?, ?it/s]
  4%|▍         | 1/25 [00:06<02:41,  6.74s/it]
  8%|▊         | 2/25 [00:13<02:33,  6.69s/it]
 12%|█▏        | 3/25 [00:20<02:26,  6.67s/it]
 16%|█▌        | 4/25 [00:26<02:20,  6.67s/it]
 20%|██        | 5/25 [00:33<02:13,  6.67s/it]
 24%|██▍       | 6/25 [00:40<02:06,  6.67s/it]
 28%|██▊       | 7/25 [00:46<02:00,  6.67s/it]
 32%|███▏      | 8/25 [00:53<01:53,  6.67s/it]
 36%|███▌      | 9/25 [01:00<01:46,  6.67s/it]
 40%|████      | 10/25 [01:06<01:40,  6.67s/it]
 44%|████▍     | 11/25 [01:13<01:33,  6.67s/it]
 48%|████▊     | 12/25 [01:20<01:26,  6.67s/it]
 52%|█████▏    | 13/25 [01:26<01:20,  6.67s/it]
 56%|█████▌    | 14/25 [01:33<01:13,  6.67s/it]
 60%|██████    | 15/25 [01:40<01:06,  6.67s/it]
 64%|██████▍   | 16/25 [01:46<01:00,  6.67s/it]
 68%|██████▊   | 17/25 [01:53<00:53,  6.67s/it]
 72%|███████▏  | 18/25 [02:00<00:46,  6.67s/it]
 76%|███████▌  | 19/25 [02:06<00:40,  6.67s/it]
 80%|████████  | 20/25 [02:13<00:33,  6.67s/it]
 84%|████████▍ | 21/25 [02:20<00:26,  6.68s/it]
 88%|████████▊ | 22/25 [02:26<00:20,  6.68s/it]
 92%|█████████▏| 23/25 [02:33<00:13,  6.68s/it]
 96%|█████████▌| 24/25 [02:40<00:06,  6.68s/it]
100%|██████████| 25/25 [02:46<00:00,  6.68s/it]
100%|██████████| 25/25 [02:46<00:00,  6.67s/it]
  0%|          | 0/160 [00:00<?, ?it/s]
  4%|▍         | 6/160 [00:00<00:03, 50.88it/s]
  8%|▊         | 12/160 [00:00<00:04, 33.01it/s]
 10%|█         | 16/160 [00:00<00:04, 30.35it/s]
 12%|█▎        | 20/160 [00:00<00:04, 28.94it/s]
 14%|█▍        | 23/160 [00:00<00:04, 28.27it/s]
 16%|█▋        | 26/160 [00:00<00:04, 27.76it/s]
 18%|█▊        | 29/160 [00:00<00:04, 27.40it/s]
 20%|██        | 32/160 [00:01<00:04, 27.15it/s]
 22%|██▏       | 35/160 [00:01<00:04, 26.97it/s]
 24%|██▍       | 38/160 [00:01<00:04, 26.84it/s]
 26%|██▌       | 41/160 [00:01<00:04, 26.74it/s]
 28%|██▊       | 44/160 [00:01<00:04, 26.68it/s]
 29%|██▉       | 47/160 [00:01<00:04, 26.64it/s]
 31%|███▏      | 50/160 [00:01<00:04, 26.60it/s]
 33%|███▎      | 53/160 [00:01<00:04, 26.59it/s]
 35%|███▌      | 56/160 [00:02<00:03, 26.56it/s]
 37%|███▋      | 59/160 [00:02<00:03, 26.55it/s]
 39%|███▉      | 62/160 [00:02<00:03, 26.54it/s]
 41%|████      | 65/160 [00:02<00:03, 26.54it/s]
 42%|████▎     | 68/160 [00:02<00:03, 26.55it/s]
 44%|████▍     | 71/160 [00:02<00:03, 26.55it/s]
 46%|████▋     | 74/160 [00:02<00:03, 26.54it/s]
 48%|████▊     | 77/160 [00:02<00:03, 26.53it/s]
 50%|█████     | 80/160 [00:02<00:03, 26.53it/s]
 52%|█████▏    | 83/160 [00:03<00:02, 26.53it/s]
 54%|█████▍    | 86/160 [00:03<00:02, 26.52it/s]
 56%|█████▌    | 89/160 [00:03<00:02, 26.51it/s]
 57%|█████▊    | 92/160 [00:03<00:02, 26.52it/s]
 59%|█████▉    | 95/160 [00:03<00:02, 26.53it/s]
 61%|██████▏   | 98/160 [00:03<00:02, 26.53it/s]
 63%|██████▎   | 101/160 [00:03<00:02, 26.53it/s]
 65%|██████▌   | 104/160 [00:03<00:02, 26.52it/s]
 67%|██████▋   | 107/160 [00:03<00:01, 26.53it/s]
 69%|██████▉   | 110/160 [00:04<00:01, 26.52it/s]
 71%|███████   | 113/160 [00:04<00:01, 26.53it/s]
 72%|███████▎  | 116/160 [00:04<00:01, 26.53it/s]
 74%|███████▍  | 119/160 [00:04<00:01, 26.52it/s]
 76%|███████▋  | 122/160 [00:04<00:01, 26.52it/s]
 78%|███████▊  | 125/160 [00:04<00:01, 26.53it/s]
 80%|████████  | 128/160 [00:04<00:01, 26.52it/s]
 82%|████████▏ | 131/160 [00:04<00:01, 26.52it/s]
 84%|████████▍ | 134/160 [00:04<00:00, 26.52it/s]
 86%|████████▌ | 137/160 [00:05<00:00, 26.51it/s]
 88%|████████▊ | 140/160 [00:05<00:00, 26.52it/s]
 89%|████████▉ | 143/160 [00:05<00:00, 26.53it/s]
 91%|█████████▏| 146/160 [00:05<00:00, 26.54it/s]
 93%|█████████▎| 149/160 [00:05<00:00, 26.54it/s]
 95%|█████████▌| 152/160 [00:05<00:00, 26.55it/s]
 97%|█████████▋| 155/160 [00:05<00:00, 26.55it/s]
 99%|█████████▉| 158/160 [00:05<00:00, 26.55it/s]
100%|██████████| 160/160 [00:05<00:00, 27.01it/s]
IMAGEIO FFMPEG_WRITER WARNING: input image is not divisible by macro_block_size=16, resizing from (1544, 516) to (1552, 528) to ensure video compatibility with most codecs and players. To prevent resizing, make your input image divisible by the macro_block_size or set the macro_block_size to 1 (risking incompatibility).
[swscaler @ 0x7239ac0] Warning: data is not aligned! This can lead to a speed loss
100%|██████████| 1/1 [03:21<00:00, 201.71s/it]
100%|██████████| 1/1 [03:21<00:00, 201.71s/it]
/src/magic-animate/samples/config-2023-12-08T05-30-28/videos/input_tmpqk3hj7h7hiphop-openpose.mp4
Version Details
Version ID
4e88cd2f48240dcfd59186b00b52b3763250486e0c65bb4d1ff4744506d222c3
Version Created
December 8, 2023
Run on Replicate →