fofr/video-morpher ❓🔢📝🖼️✓ → 🖼️
About
Generate a video that morphs between subjects, with an optional style
Example Output
Prompt:
"bright, vibrant, high contrast"
Output
Performance Metrics
181.96s
Prediction Time
181.96s
Total Time
All Input Parameters
{
"mode": "upscaled-and-interpolated",
"prompt": "bright, vibrant, high contrast",
"checkpoint": "3D",
"style_image": "https://replicate.delivery/pbxt/KnxLVfe6BRRu2zHc3gT99mnwaemKfR4JzaZWxMCLsZYSTKzp/2024-03-05--06-47-29-u-q1-fofr_tropical_purple_beksinski_aaad09f0-d194-4e40-b312-51054fc4ebbf.png",
"aspect_ratio": "3:4",
"style_strength": 0.5,
"use_controlnet": true,
"negative_prompt": "dark, gloomy",
"subject_image_1": "https://replicate.delivery/pbxt/KnxLX9qnA82YKCkScCJZf5VCX6hy5RuprGwLEAVjp3vu6Oh1/1.webp",
"subject_image_2": "https://replicate.delivery/pbxt/KnxLWqPg086DnRTUvxDM7gCBYi3W3coIbn3Q8jXnTJsfvZzt/2.webp",
"subject_image_3": "https://replicate.delivery/pbxt/KnxLWU6aHjc6kVW2QOF0xC4oen8zuPnlsK2o24GtOV2bJEn8/4.webp",
"subject_image_4": "https://replicate.delivery/pbxt/KnxLVg54ySFfg8s78YyfzK0Qgse8YdTbSmEZKpnkFkmSFlsf/4.webp"
}
Input Parameters
- mode
- Determines if you produce a quick experimental video or an upscaled interpolated one. (small ~20s, medium ~60s, upscaled ~2min, upscaled-and-interpolated ~4min)
- seed
- Set a seed for reproducibility. Random by default.
- prompt
- The prompt has a small effect, but most of the video is driven by the subject images
- checkpoint
- The checkpoint to use for the model
- style_image
- Apply the style from this image to the whole video
- aspect_ratio
- The aspect ratio of the video
- style_strength
- How strong the style is applied
- use_controlnet
- Use geometric circles to guide the generation
- negative_prompt
- What you do not want to see in the video
- subject_image_1 (required)
- The first subject of the video
- subject_image_2 (required)
- The second subject of the video
- subject_image_3 (required)
- The third subject of the video
- subject_image_4 (required)
- The fourth subject of the video
Output Schema
Output
Example Execution Logs
Random seed set to: 4049182242
Checking inputs
✅ /tmp/inputs/2.png
✅ /tmp/inputs/1.png
✅ /tmp/inputs/3.png
✅ /tmp/inputs/4.png
✅ /tmp/inputs/circles.mp4
✅ /tmp/inputs/style.png
====================================
Checking weights
Including weights for IPAdapter preset: PLUS (high strength)
✅ RealESRGAN_x4.pth
✅ film_net_fp32.pt
✅ ip-adapter-plus_sdxl_vit-h.safetensors
✅ CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
✅ AnimateLCM_sd15_t2v.ckpt
✅ control_v1p_sd15_qrcode_monster.safetensors
✅ vae-ft-mse-840000-ema-pruned.safetensors
✅ ip-adapter-plus_sd15.safetensors
✅ AnimateLCM_sd15_t2v_lora.safetensors
✅ rcnzCartoon3d_v20.safetensors
====================================
Running workflow
got prompt
Executing node 746, title: Load Video (Upload) 🎥🅥🅗🅢, class type: VHS_LoadVideo
Executing node 461, title: 🔧 Simple Math, class type: SimpleMath+
Executing node 454, title: RepeatImageBatch, class type: RepeatImageBatch
Executing node 458, title: Split Image Batch 🎥🅥🅗🅢, class type: VHS_SplitImages
Executing node 125, title: Apply Advanced ControlNet 🛂🅐🅒🅝, class type: ACN_AdvancedControlNetApply
Executing node 80, title: KSampler, class type: KSampler
[AnimateDiffEvo] - [0;32mINFO[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - [0;32mINFO[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
Requested to load ControlNet
Loading 1 new model
0%| | 0/11 [00:00<?, ?it/s]
9%|▉ | 1/11 [00:01<00:11, 1.11s/it]
18%|█▊ | 2/11 [00:02<00:09, 1.11s/it]
27%|██▋ | 3/11 [00:03<00:09, 1.18s/it]
36%|███▋ | 4/11 [00:04<00:08, 1.21s/it]
45%|████▌ | 5/11 [00:05<00:06, 1.14s/it]
55%|█████▍ | 6/11 [00:06<00:05, 1.11s/it]
64%|██████▎ | 7/11 [00:07<00:04, 1.08s/it]
73%|███████▎ | 8/11 [00:09<00:03, 1.13s/it]
82%|████████▏ | 9/11 [00:10<00:02, 1.12s/it]
91%|█████████ | 10/11 [00:11<00:01, 1.09s/it]
100%|██████████| 11/11 [00:12<00:00, 1.07s/it]
100%|██████████| 11/11 [00:12<00:00, 1.11s/it]
Executing node 84, title: VAE Decode, class type: VAEDecode
Executing node 53, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 203, title: Upscale Image By, class type: ImageScaleBy
Executing node 204, title: VAE Encode, class type: VAEEncode
Executing node 198, title: KSampler, class type: KSampler
[AnimateDiffEvo] - [0;32mINFO[0m - Sliding context window activated - latents passed in (96) greater than context_length 16.
[AnimateDiffEvo] - [0;32mINFO[0m - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
0%| | 0/11 [00:00<?, ?it/s]
9%|▉ | 1/11 [00:02<00:21, 2.20s/it]
18%|█▊ | 2/11 [00:04<00:19, 2.19s/it]
27%|██▋ | 3/11 [00:06<00:17, 2.19s/it]
36%|███▋ | 4/11 [00:08<00:15, 2.19s/it]
45%|████▌ | 5/11 [00:10<00:13, 2.19s/it]
55%|█████▍ | 6/11 [00:13<00:10, 2.19s/it]
64%|██████▎ | 7/11 [00:15<00:08, 2.19s/it]
73%|███████▎ | 8/11 [00:17<00:06, 2.19s/it]
82%|████████▏ | 9/11 [00:19<00:04, 2.18s/it]
91%|█████████ | 10/11 [00:21<00:02, 2.18s/it]
100%|██████████| 11/11 [00:24<00:00, 2.18s/it]
100%|██████████| 11/11 [00:24<00:00, 2.19s/it]
Requested to load AutoencoderKL
Loading 1 new model
Executing node 201, title: VAE Decode, class type: VAEDecode
Executing node 205, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 271, title: Upscale Image (using Model), class type: ImageUpscaleWithModel
Executing node 279, title: Upscale Image, class type: ImageScale
Executing node 272, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Executing node 770, title: FILM VFI, class type: FILM VFI
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Clearing cache...
Comfy-VFI: Done cache clearing
Comfy-VFI: Final clearing cache...
Comfy-VFI: Done cache clearing
Executing node 219, title: Video Combine 🎥🅥🅗🅢, class type: VHS_VideoCombine
Prompt executed in 176.75 seconds
outputs: {'53': {'gifs': [{'filename': 'preview_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '205': {'gifs': [{'filename': 'upscaled_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '272': {'gifs': [{'filename': 'upscaled_model_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}, '219': {'gifs': [{'filename': 'interpolated_00001.mp4', 'subfolder': '', 'type': 'output', 'format': 'video/h264-mp4'}]}}
====================================
preview_00001.mp4
upscaled_00001.mp4
upscaled_model_00001.mp4
interpolated_00001.mp4
Version Details
- Version ID
e70e975067d2b5dbe9e2d9022833d27230a1bdeb3f4af6fe6bb49a548a3039a7- Version Created
- April 24, 2024