lucataco/seine 🔢🖼️ → 🖼️

▶️ 13.8K runs 📅 Nov 2023 ⚙️ Cog 0.8.6 🔗 GitHub 📄 Paper ⚖️ License
image-to-video video-prediction

About

Image-to-video - SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Example Output

Output

Performance Metrics

68.22s Prediction Time
68.24s Total Time
All Input Parameters
{
  "image": "https://replicate.delivery/pbxt/JuhRUNPhsIsNIWrFdU8raXtP6rb5eSAUALNZKnkG4gWSeWvY/The_picture_shows_the_beauty_of_the_sea_.jpg",
  "width": 560,
  "height": 240,
  "run_time": 13,
  "cfg_scale": 8,
  "num_frames": 16,
  "num_sampling_steps": 250
}
Input Parameters
seed Type: integer
Random seed. Leave blank to randomize the seed
image (required) Type: string
Input image
width Type: integerDefault: 560
Width
height Type: integerDefault: 240
Height
run_time Type: integerDefault: 13
Run time
cfg_scale Type: numberDefault: 8Range: 1 - 50
Scale for classifier-free guidance
num_frames Type: integerDefault: 16
Number of frames
num_sampling_steps Type: integerDefault: 250
Number of sampling steps
Output Schema

Output

Type: stringFormat: uri

Example Execution Logs
Using seed: 28954
loading model
loading succeed
Warnning: using half percision for inferencing!
loading video from /tmp/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.jpg
loading the input image
  0%|          | 0/250 [00:00<?, ?it/s]
  0%|          | 1/250 [00:00<00:56,  4.38it/s]
  1%|          | 2/250 [00:00<00:48,  5.09it/s]
  1%|          | 3/250 [00:00<00:46,  5.36it/s]
  2%|▏         | 4/250 [00:00<00:44,  5.50it/s]
  2%|▏         | 5/250 [00:00<00:43,  5.58it/s]
  2%|▏         | 6/250 [00:01<00:43,  5.63it/s]
  3%|▎         | 7/250 [00:01<00:42,  5.66it/s]
  3%|▎         | 8/250 [00:01<00:42,  5.68it/s]
  4%|▎         | 9/250 [00:01<00:42,  5.70it/s]
  4%|▍         | 10/250 [00:01<00:42,  5.70it/s]
  4%|▍         | 11/250 [00:01<00:41,  5.71it/s]
  5%|▍         | 12/250 [00:02<00:41,  5.71it/s]
  5%|▌         | 13/250 [00:02<00:41,  5.72it/s]
  6%|▌         | 14/250 [00:02<00:41,  5.72it/s]
  6%|▌         | 15/250 [00:02<00:41,  5.72it/s]
  6%|▋         | 16/250 [00:02<00:40,  5.73it/s]
  7%|▋         | 17/250 [00:03<00:40,  5.73it/s]
  7%|▋         | 18/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 19/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 20/250 [00:03<00:40,  5.73it/s]
  8%|▊         | 21/250 [00:03<00:40,  5.72it/s]
  9%|▉         | 22/250 [00:03<00:39,  5.73it/s]
  9%|▉         | 23/250 [00:04<00:39,  5.73it/s]
 10%|▉         | 24/250 [00:04<00:39,  5.73it/s]
 10%|█         | 25/250 [00:04<00:39,  5.73it/s]
 10%|█         | 26/250 [00:04<00:39,  5.73it/s]
 11%|█         | 27/250 [00:04<00:38,  5.72it/s]
 11%|█         | 28/250 [00:04<00:38,  5.73it/s]
 12%|█▏        | 29/250 [00:05<00:38,  5.73it/s]
 12%|█▏        | 30/250 [00:05<00:38,  5.73it/s]
 12%|█▏        | 31/250 [00:05<00:38,  5.73it/s]
 13%|█▎        | 32/250 [00:05<00:38,  5.73it/s]
 13%|█▎        | 33/250 [00:05<00:37,  5.73it/s]
 14%|█▎        | 34/250 [00:05<00:37,  5.73it/s]
 14%|█▍        | 35/250 [00:06<00:37,  5.73it/s]
 14%|█▍        | 36/250 [00:06<00:37,  5.73it/s]
 15%|█▍        | 37/250 [00:06<00:37,  5.73it/s]
 15%|█▌        | 38/250 [00:06<00:37,  5.73it/s]
 16%|█▌        | 39/250 [00:06<00:36,  5.73it/s]
 16%|█▌        | 40/250 [00:07<00:36,  5.73it/s]
 16%|█▋        | 41/250 [00:07<00:36,  5.73it/s]
 17%|█▋        | 42/250 [00:07<00:36,  5.73it/s]
 17%|█▋        | 43/250 [00:07<00:36,  5.73it/s]
 18%|█▊        | 44/250 [00:07<00:35,  5.73it/s]
 18%|█▊        | 45/250 [00:07<00:35,  5.73it/s]
 18%|█▊        | 46/250 [00:08<00:35,  5.68it/s]
 19%|█▉        | 47/250 [00:08<00:35,  5.70it/s]
 19%|█▉        | 48/250 [00:08<00:35,  5.71it/s]
 20%|█▉        | 49/250 [00:08<00:35,  5.72it/s]
 20%|██        | 50/250 [00:08<00:34,  5.72it/s]
 20%|██        | 51/250 [00:08<00:34,  5.72it/s]
 21%|██        | 52/250 [00:09<00:34,  5.72it/s]
 21%|██        | 53/250 [00:09<00:34,  5.72it/s]
 22%|██▏       | 54/250 [00:09<00:34,  5.73it/s]
 22%|██▏       | 55/250 [00:09<00:34,  5.73it/s]
 22%|██▏       | 56/250 [00:09<00:33,  5.73it/s]
 23%|██▎       | 57/250 [00:10<00:33,  5.73it/s]
 23%|██▎       | 58/250 [00:10<00:33,  5.73it/s]
 24%|██▎       | 59/250 [00:10<00:33,  5.72it/s]
 24%|██▍       | 60/250 [00:10<00:33,  5.73it/s]
 24%|██▍       | 61/250 [00:10<00:33,  5.73it/s]
 25%|██▍       | 62/250 [00:10<00:32,  5.73it/s]
 25%|██▌       | 63/250 [00:11<00:32,  5.73it/s]
 26%|██▌       | 64/250 [00:11<00:32,  5.73it/s]
 26%|██▌       | 65/250 [00:11<00:32,  5.73it/s]
 26%|██▋       | 66/250 [00:11<00:32,  5.73it/s]
 27%|██▋       | 67/250 [00:11<00:31,  5.73it/s]
 27%|██▋       | 68/250 [00:11<00:31,  5.73it/s]
 28%|██▊       | 69/250 [00:12<00:31,  5.73it/s]
 28%|██▊       | 70/250 [00:12<00:31,  5.73it/s]
 28%|██▊       | 71/250 [00:12<00:31,  5.73it/s]
 29%|██▉       | 72/250 [00:12<00:31,  5.73it/s]
 29%|██▉       | 73/250 [00:12<00:30,  5.73it/s]
 30%|██▉       | 74/250 [00:12<00:30,  5.73it/s]
 30%|███       | 75/250 [00:13<00:30,  5.73it/s]
 30%|███       | 76/250 [00:13<00:30,  5.73it/s]
 31%|███       | 77/250 [00:13<00:30,  5.73it/s]
 31%|███       | 78/250 [00:13<00:30,  5.73it/s]
 32%|███▏      | 79/250 [00:13<00:29,  5.73it/s]
 32%|███▏      | 80/250 [00:14<00:29,  5.73it/s]
 32%|███▏      | 81/250 [00:14<00:29,  5.73it/s]
 33%|███▎      | 82/250 [00:14<00:29,  5.73it/s]
 33%|███▎      | 83/250 [00:14<00:29,  5.73it/s]
 34%|███▎      | 84/250 [00:14<00:28,  5.73it/s]
 34%|███▍      | 85/250 [00:14<00:28,  5.73it/s]
 34%|███▍      | 86/250 [00:15<00:28,  5.73it/s]
 35%|███▍      | 87/250 [00:15<00:29,  5.61it/s]
 35%|███▌      | 88/250 [00:15<00:28,  5.64it/s]
 36%|███▌      | 89/250 [00:15<00:28,  5.67it/s]
 36%|███▌      | 90/250 [00:15<00:28,  5.68it/s]
 36%|███▋      | 91/250 [00:15<00:27,  5.70it/s]
 37%|███▋      | 92/250 [00:16<00:27,  5.71it/s]
 37%|███▋      | 93/250 [00:16<00:27,  5.71it/s]
 38%|███▊      | 94/250 [00:16<00:27,  5.72it/s]
 38%|███▊      | 95/250 [00:16<00:27,  5.72it/s]
 38%|███▊      | 96/250 [00:16<00:26,  5.72it/s]
 39%|███▉      | 97/250 [00:17<00:26,  5.72it/s]
 39%|███▉      | 98/250 [00:17<00:26,  5.73it/s]
 40%|███▉      | 99/250 [00:17<00:26,  5.72it/s]
 40%|████      | 100/250 [00:17<00:26,  5.73it/s]
 40%|████      | 101/250 [00:17<00:26,  5.73it/s]
 41%|████      | 102/250 [00:17<00:25,  5.73it/s]
 41%|████      | 103/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 104/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 105/250 [00:18<00:25,  5.73it/s]
 42%|████▏     | 106/250 [00:18<00:25,  5.73it/s]
 43%|████▎     | 107/250 [00:18<00:24,  5.73it/s]
 43%|████▎     | 108/250 [00:18<00:24,  5.73it/s]
 44%|████▎     | 109/250 [00:19<00:24,  5.73it/s]
 44%|████▍     | 110/250 [00:19<00:24,  5.73it/s]
 44%|████▍     | 111/250 [00:19<00:24,  5.73it/s]
 45%|████▍     | 112/250 [00:19<00:24,  5.73it/s]
 45%|████▌     | 113/250 [00:19<00:23,  5.73it/s]
 46%|████▌     | 114/250 [00:19<00:23,  5.73it/s]
 46%|████▌     | 115/250 [00:20<00:23,  5.73it/s]
 46%|████▋     | 116/250 [00:20<00:23,  5.72it/s]
 47%|████▋     | 117/250 [00:20<00:23,  5.72it/s]
 47%|████▋     | 118/250 [00:20<00:23,  5.73it/s]
 48%|████▊     | 119/250 [00:20<00:22,  5.73it/s]
 48%|████▊     | 120/250 [00:21<00:22,  5.72it/s]
 48%|████▊     | 121/250 [00:21<00:22,  5.73it/s]
 49%|████▉     | 122/250 [00:21<00:22,  5.73it/s]
 49%|████▉     | 123/250 [00:21<00:22,  5.72it/s]
 50%|████▉     | 124/250 [00:21<00:22,  5.73it/s]
 50%|█████     | 125/250 [00:21<00:21,  5.73it/s]
 50%|█████     | 126/250 [00:22<00:21,  5.73it/s]
 51%|█████     | 127/250 [00:22<00:21,  5.73it/s]
 51%|█████     | 128/250 [00:22<00:21,  5.67it/s]
 52%|█████▏    | 129/250 [00:22<00:21,  5.69it/s]
 52%|█████▏    | 130/250 [00:22<00:21,  5.70it/s]
 52%|█████▏    | 131/250 [00:22<00:20,  5.71it/s]
 53%|█████▎    | 132/250 [00:23<00:20,  5.72it/s]
 53%|█████▎    | 133/250 [00:23<00:20,  5.72it/s]
 54%|█████▎    | 134/250 [00:23<00:20,  5.72it/s]
 54%|█████▍    | 135/250 [00:23<00:20,  5.72it/s]
 54%|█████▍    | 136/250 [00:23<00:19,  5.72it/s]
 55%|█████▍    | 137/250 [00:23<00:19,  5.73it/s]
 55%|█████▌    | 138/250 [00:24<00:19,  5.73it/s]
 56%|█████▌    | 139/250 [00:24<00:19,  5.73it/s]
 56%|█████▌    | 140/250 [00:24<00:19,  5.73it/s]
 56%|█████▋    | 141/250 [00:24<00:19,  5.73it/s]
 57%|█████▋    | 142/250 [00:24<00:18,  5.72it/s]
 57%|█████▋    | 143/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 144/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 145/250 [00:25<00:18,  5.72it/s]
 58%|█████▊    | 146/250 [00:25<00:18,  5.72it/s]
 59%|█████▉    | 147/250 [00:25<00:17,  5.72it/s]
 59%|█████▉    | 148/250 [00:25<00:17,  5.73it/s]
 60%|█████▉    | 149/250 [00:26<00:17,  5.73it/s]
 60%|██████    | 150/250 [00:26<00:17,  5.73it/s]
 60%|██████    | 151/250 [00:26<00:17,  5.72it/s]
 61%|██████    | 152/250 [00:26<00:17,  5.72it/s]
 61%|██████    | 153/250 [00:26<00:16,  5.73it/s]
 62%|██████▏   | 154/250 [00:26<00:16,  5.72it/s]
 62%|██████▏   | 155/250 [00:27<00:16,  5.72it/s]
 62%|██████▏   | 156/250 [00:27<00:16,  5.73it/s]
 63%|██████▎   | 157/250 [00:27<00:16,  5.73it/s]
 63%|██████▎   | 158/250 [00:27<00:16,  5.73it/s]
 64%|██████▎   | 159/250 [00:27<00:15,  5.73it/s]
 64%|██████▍   | 160/250 [00:28<00:15,  5.73it/s]
 64%|██████▍   | 161/250 [00:28<00:15,  5.73it/s]
 65%|██████▍   | 162/250 [00:28<00:15,  5.73it/s]
 65%|██████▌   | 163/250 [00:28<00:15,  5.73it/s]
 66%|██████▌   | 164/250 [00:28<00:15,  5.73it/s]
 66%|██████▌   | 165/250 [00:28<00:14,  5.73it/s]
 66%|██████▋   | 166/250 [00:29<00:14,  5.73it/s]
 67%|██████▋   | 167/250 [00:29<00:14,  5.72it/s]
 67%|██████▋   | 168/250 [00:29<00:14,  5.72it/s]
 68%|██████▊   | 169/250 [00:29<00:14,  5.60it/s]
 68%|██████▊   | 170/250 [00:29<00:14,  5.63it/s]
 68%|██████▊   | 171/250 [00:29<00:13,  5.66it/s]
 69%|██████▉   | 172/250 [00:30<00:13,  5.68it/s]
 69%|██████▉   | 173/250 [00:30<00:13,  5.69it/s]
 70%|██████▉   | 174/250 [00:30<00:13,  5.70it/s]
 70%|███████   | 175/250 [00:30<00:13,  5.70it/s]
 70%|███████   | 176/250 [00:30<00:12,  5.71it/s]
 71%|███████   | 177/250 [00:30<00:12,  5.71it/s]
 71%|███████   | 178/250 [00:31<00:12,  5.71it/s]
 72%|███████▏  | 179/250 [00:31<00:12,  5.71it/s]
 72%|███████▏  | 180/250 [00:31<00:12,  5.72it/s]
 72%|███████▏  | 181/250 [00:31<00:12,  5.72it/s]
 73%|███████▎  | 182/250 [00:31<00:11,  5.72it/s]
 73%|███████▎  | 183/250 [00:32<00:11,  5.73it/s]
 74%|███████▎  | 184/250 [00:32<00:11,  5.73it/s]
 74%|███████▍  | 185/250 [00:32<00:11,  5.73it/s]
 74%|███████▍  | 186/250 [00:32<00:11,  5.73it/s]
 75%|███████▍  | 187/250 [00:32<00:10,  5.73it/s]
 75%|███████▌  | 188/250 [00:32<00:10,  5.73it/s]
 76%|███████▌  | 189/250 [00:33<00:10,  5.73it/s]
 76%|███████▌  | 190/250 [00:33<00:10,  5.73it/s]
 76%|███████▋  | 191/250 [00:33<00:10,  5.73it/s]
 77%|███████▋  | 192/250 [00:33<00:10,  5.73it/s]
 77%|███████▋  | 193/250 [00:33<00:09,  5.73it/s]
 78%|███████▊  | 194/250 [00:33<00:09,  5.73it/s]
 78%|███████▊  | 195/250 [00:34<00:09,  5.73it/s]
 78%|███████▊  | 196/250 [00:34<00:09,  5.73it/s]
 79%|███████▉  | 197/250 [00:34<00:09,  5.73it/s]
 79%|███████▉  | 198/250 [00:34<00:09,  5.73it/s]
 80%|███████▉  | 199/250 [00:34<00:08,  5.72it/s]
 80%|████████  | 200/250 [00:35<00:08,  5.72it/s]
 80%|████████  | 201/250 [00:35<00:08,  5.72it/s]
 81%|████████  | 202/250 [00:35<00:08,  5.72it/s]
 81%|████████  | 203/250 [00:35<00:08,  5.72it/s]
 82%|████████▏ | 204/250 [00:35<00:08,  5.73it/s]
 82%|████████▏ | 205/250 [00:35<00:07,  5.72it/s]
 82%|████████▏ | 206/250 [00:36<00:07,  5.73it/s]
 83%|████████▎ | 207/250 [00:36<00:07,  5.73it/s]
 83%|████████▎ | 208/250 [00:36<00:07,  5.72it/s]
 84%|████████▎ | 209/250 [00:36<00:07,  5.72it/s]
 84%|████████▍ | 210/250 [00:36<00:07,  5.61it/s]
 84%|████████▍ | 211/250 [00:36<00:06,  5.65it/s]
 85%|████████▍ | 212/250 [00:37<00:06,  5.67it/s]
 85%|████████▌ | 213/250 [00:37<00:06,  5.69it/s]
 86%|████████▌ | 214/250 [00:37<00:06,  5.70it/s]
 86%|████████▌ | 215/250 [00:37<00:06,  5.70it/s]
 86%|████████▋ | 216/250 [00:37<00:05,  5.70it/s]
 87%|████████▋ | 217/250 [00:37<00:05,  5.71it/s]
 87%|████████▋ | 218/250 [00:38<00:05,  5.71it/s]
 88%|████████▊ | 219/250 [00:38<00:05,  5.71it/s]
 88%|████████▊ | 220/250 [00:38<00:05,  5.72it/s]
 88%|████████▊ | 221/250 [00:38<00:05,  5.72it/s]
 89%|████████▉ | 222/250 [00:38<00:04,  5.72it/s]
 89%|████████▉ | 223/250 [00:39<00:04,  5.72it/s]
 90%|████████▉ | 224/250 [00:39<00:04,  5.72it/s]
 90%|█████████ | 225/250 [00:39<00:04,  5.72it/s]
 90%|█████████ | 226/250 [00:39<00:04,  5.72it/s]
 91%|█████████ | 227/250 [00:39<00:04,  5.72it/s]
 91%|█████████ | 228/250 [00:39<00:03,  5.73it/s]
 92%|█████████▏| 229/250 [00:40<00:03,  5.72it/s]
 92%|█████████▏| 230/250 [00:40<00:03,  5.73it/s]
 92%|█████████▏| 231/250 [00:40<00:03,  5.73it/s]
 93%|█████████▎| 232/250 [00:40<00:03,  5.73it/s]
 93%|█████████▎| 233/250 [00:40<00:02,  5.73it/s]
 94%|█████████▎| 234/250 [00:40<00:02,  5.73it/s]
 94%|█████████▍| 235/250 [00:41<00:02,  5.73it/s]
 94%|█████████▍| 236/250 [00:41<00:02,  5.73it/s]
 95%|█████████▍| 237/250 [00:41<00:02,  5.73it/s]
 95%|█████████▌| 238/250 [00:41<00:02,  5.72it/s]
 96%|█████████▌| 239/250 [00:41<00:01,  5.72it/s]
 96%|█████████▌| 240/250 [00:42<00:01,  5.73it/s]
 96%|█████████▋| 241/250 [00:42<00:01,  5.72it/s]
 97%|█████████▋| 242/250 [00:42<00:01,  5.72it/s]
 97%|█████████▋| 243/250 [00:42<00:01,  5.72it/s]
 98%|█████████▊| 244/250 [00:42<00:01,  5.72it/s]
 98%|█████████▊| 245/250 [00:42<00:00,  5.72it/s]
 98%|█████████▊| 246/250 [00:43<00:00,  5.73it/s]
 99%|█████████▉| 247/250 [00:43<00:00,  5.72it/s]
 99%|█████████▉| 248/250 [00:43<00:00,  5.72it/s]
100%|█████████▉| 249/250 [00:43<00:00,  5.73it/s]
100%|██████████| 250/250 [00:43<00:00,  5.72it/s]
100%|██████████| 250/250 [00:43<00:00,  5.71it/s]
save in /src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4
/src/results/tmp8uw7ffncThe_picture_shows_the_beauty_of_the_sea_.mp4
Version Details
Version ID
a3a0a85f7f4dd9895ee1dd5bc8d4ffcad365c49cd94976b71603dd7fda408c11
Version Created
November 23, 2023
Run on Replicate →