buddhiraz/liveportraitv1 ✓🖼️🔢 → 🖼️

▶️ 96 runs 📅 Jul 2024 ⚙️ Cog 0.9.7

face-animation image-to-video

About

Example Output

Output

Performance Metrics

123.94s Prediction Time

269.18s Total Time

All Input Parameters

{
  "flag_pasteback": true,
  "input_image_path": "https://replicate.delivery/pbxt/LFzuXbwINYxNv26erUHNYNR85H3hROWAHbnR2nCsfhf4YHzA/Screenshot%20from%202024-07-08%2014-50-41.png",
  "input_video_path": "https://replicate.delivery/pbxt/LFzuXFbzMx12AJnrrpRNqXuwDXpTCEoLajXvo1LC3GSCFlHk/d14.mp4",
  "flag_do_crop_input": true,
  "flag_relative_input": true,
  "live_portrait_dsize": 512,
  "live_portrait_scale": 2.3,
  "video_frame_load_cap": 128,
  "live_portrait_lip_zero": true,
  "live_portrait_relative": true,
  "live_portrait_vx_ratio": 0,
  "live_portrait_vy_ratio": -0.12,
  "live_portrait_stitching": true,
  "video_select_every_n_frames": 1,
  "live_portrait_eye_retargeting": false,
  "live_portrait_lip_retargeting": false,
  "live_portrait_lip_retargeting_multiplier": 1,
  "live_portrait_eyes_retargeting_multiplier": 1
}

Input Parameters

flag_pasteback Type: booleanDefault: true: paste-back
input_image_path (required) Type: string: Portrait image with face
input_video_path (required) Type: string: A video to drive the animation
flag_do_crop_input Type: booleanDefault: true: We recommend checking the do crop option when facial areas occupy a relatively small portion of your image.
flag_relative_input Type: booleanDefault: true: relative motion
live_portrait_dsize Type: integerDefault: 512Range: 64 - 2048: Size of the output image
live_portrait_scale Type: numberDefault: 2.3Range: 1 - 4: Scaling factor for the face
video_frame_load_cap Type: integerDefault: 128: The maximum number of frames to load from the driving video. Set to 0 to use all frames.
live_portrait_lip_zero Type: booleanDefault: true: Enable lip zero
live_portrait_relative Type: booleanDefault: true: Use relative positioning
live_portrait_vx_ratio Type: numberDefault: 0Range: -1 - 1: Horizontal shift ratio
live_portrait_vy_ratio Type: numberDefault: -0.12Range: -1 - 1: Vertical shift ratio
live_portrait_stitching Type: booleanDefault: true: Enable stitching
video_select_every_n_frames Type: integerDefault: 1: Select every nth frame from the driving video. Set to 1 to use all frames.
live_portrait_eye_retargeting Type: booleanDefault: false: Enable eye retargeting
live_portrait_lip_retargeting Type: booleanDefault: false: Enable lip retargeting
live_portrait_lip_retargeting_multiplier Type: numberDefault: 1Range: 0.01 - 10: Multiplier for lip retargeting
live_portrait_eyes_retargeting_multiplier Type: numberDefault: 1Range: 0.01 - 10: Multiplier for eye retargeting

Output Schema

Output

Type: string • Format: uri

Example Execution Logs

[20:01:27] Load source image from                   live_portrait_pipeline.py:49
/tmp/tmpbuvf7ptkScreenshot from
2024-07-08 14-50-41.png
tensor([[[[0.7059, 0.9490, 0.9451,  ..., 0.0039, 0.0039, 0.0000],
[0.5098, 0.9059, 0.9529,  ..., 0.0078, 0.0039, 0.0000],
[0.5765, 0.8824, 0.9725,  ..., 0.0039, 0.0000, 0.0000],
...,
[0.8980, 0.8824, 0.7686,  ..., 0.8471, 0.8471, 0.8353],
[0.8902, 0.8392, 0.6667,  ..., 0.8431, 0.8471, 0.8353],
[0.8588, 0.7412, 0.5020,  ..., 0.8157, 0.8392, 0.8392]],
[[0.7333, 0.9725, 0.9725,  ..., 0.0392, 0.0471, 0.0471],
[0.5373, 0.9333, 0.9804,  ..., 0.0431, 0.0471, 0.0471],
[0.6039, 0.9098, 0.9961,  ..., 0.0471, 0.0471, 0.0471],
...,
[0.7020, 0.7176, 0.6392,  ..., 0.6745, 0.6863, 0.6863],
[0.6980, 0.6706, 0.5373,  ..., 0.6706, 0.6863, 0.6902],
[0.6902, 0.6078, 0.4118,  ..., 0.6431, 0.6745, 0.6902]],
[[0.7333, 0.9725, 0.9725,  ..., 0.0353, 0.0392, 0.0392],
[0.5373, 0.9333, 0.9765,  ..., 0.0392, 0.0392, 0.0392],
[0.6039, 0.9098, 0.9961,  ..., 0.0353, 0.0353, 0.0353],
...,
[0.5765, 0.5922, 0.5176,  ..., 0.4706, 0.4824, 0.4745],
[0.5725, 0.5490, 0.4235,  ..., 0.4706, 0.4824, 0.4784],
[0.5843, 0.5176, 0.3294,  ..., 0.4431, 0.4706, 0.4824]]]],
device='cuda:0')
[20:01:29] Load video: /tmp/tmpqda7wxbqd14.mp4      live_portrait_pipeline.py:99
The FPS of /tmp/tmpqda7wxbqd14.mp4 is:  live_portrait_pipeline.py:104
30
Load video file (mp4 mov avi etc...):   live_portrait_pipeline.py:106
/tmp/tmpqda7wxbqd14.mp4
[20:01:31] Start making motion template...         live_portrait_pipeline.py:110
Making motion templates... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:09
[20:01:48] Dump motion template to                 live_portrait_pipeline.py:127
/tmp/tmpqda7wxbqd14.pkl
Prepared pasteback mask done.           live_portrait_pipeline.py:139
🚀Animating... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:01:17
Concatenating result... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:03
[20:03:20] Video with audio generated successfully:                 video.py:197
/tmp/tmpbuvf7ptkScreenshot from 2024-07-08
14-50-41--tmpqda7wxbqd14_concat_with_audio.mp4
Replace /tmp/tmpbuvf7ptkScreenshot from live_portrait_pipeline.py:230
2024-07-08
14-50-41--tmpqda7wxbqd14_concat.mp4
with /tmp/tmpbuvf7ptkScreenshot from
2024-07-08
14-50-41--tmpqda7wxbqd14_concat_with_au
dio.mp4
[swscaler @ 0x568f240] Warning: data is not aligned! This can lead to a speed loss
[20:03:30] Video with audio generated successfully:                 video.py:197
/tmp/tmpbuvf7ptkScreenshot from 2024-07-08
14-50-41--tmpqda7wxbqd14_with_audio.mp4
Replace /tmp/tmpbuvf7ptkScreenshot from live_portrait_pipeline.py:244
2024-07-08 14-50-41--tmpqda7wxbqd14.mp4
with /tmp/tmpbuvf7ptkScreenshot from
2024-07-08
14-50-41--tmpqda7wxbqd14_with_audio.mp4
Animated template:                      live_portrait_pipeline.py:248
/tmp/tmpqda7wxbqd14.pkl, you can
specify `-d` argument with this
template path next time to avoid
cropping video, motion making and
protecting privacy.
Animated video:                         live_portrait_pipeline.py:249
/tmp/tmpbuvf7ptkScreenshot from
2024-07-08 14-50-41--tmpqda7wxbqd14.mp4
Animated video with concact:            live_portrait_pipeline.py:250
/tmp/tmpbuvf7ptkScreenshot from
2024-07-08
14-50-41--tmpqda7wxbqd14_concat.mp4

Version Details

Version ID: 8514ed7670d55e3e4811ee4003bb1741c15b583380a40e711833eadb988d2f44
Version Created: July 12, 2024

Run on Replicate →