aryamansital/instant_mesh 🔢🖼️✓ → 🖼️

▶️ 269.0K runs 📅 May 2024 ⚙️ Cog 0.9.7 🔗 GitHub 📄 Paper
image-to-3d image-to-video

About

InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view LRMs

Example Output

Output

Example outputExample output

Performance Metrics

29.49s Prediction Time
242.90s Total Time
All Input Parameters
{
  "seed": 42,
  "image_path": "https://replicate.delivery/pbxt/Kuf9PGp3vopyQmscqIszxJWBr1OYuzr1VWb67ijDJVBs6N7D/image.png",
  "export_video": true,
  "sample_steps": 75,
  "export_texmap": false,
  "remove_background": true
}
Input Parameters
seed Type: integerDefault: 42
image_path (required) Type: string
Input image
export_video Type: booleanDefault: true
sample_steps Type: integerDefault: 75
export_texmap Type: booleanDefault: false
remove_background Type: booleanDefault: true
Output Schema

Output

Type: arrayItems Type: stringItems Format: uri

Example Execution Logs
Downloading data from 'https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net.onnx' to file '/root/.u2net/u2net.onnx'.
  0%|                                               | 0.00/176M [00:00<?, ?B/s]
  1%|▏                                      | 975k/176M [00:00<00:18, 9.69MB/s]
  2%|▋                                     | 3.05M/176M [00:00<00:10, 16.2MB/s]
  4%|█▍                                    | 6.82M/176M [00:00<00:06, 25.9MB/s]
  7%|██▋                                   | 12.6M/176M [00:00<00:04, 38.5MB/s]
 12%|████▍                                 | 20.4M/176M [00:00<00:02, 52.5MB/s]
 16%|██████▏                               | 28.9M/176M [00:00<00:02, 63.8MB/s]
 21%|████████                              | 37.1M/176M [00:00<00:01, 69.7MB/s]
 26%|█████████▊                            | 45.3M/176M [00:00<00:01, 73.5MB/s]
 30%|███████████▌                          | 53.3M/176M [00:00<00:01, 75.7MB/s]
 35%|█████████████▎                        | 61.6M/176M [00:01<00:01, 77.7MB/s]
 40%|███████████████                       | 69.9M/176M [00:01<00:01, 79.5MB/s]
 44%|████████████████▉                     | 78.3M/176M [00:01<00:01, 80.9MB/s]
 49%|██████████████████▋                   | 86.5M/176M [00:01<00:01, 81.1MB/s]
 54%|████████████████████▍                 | 94.6M/176M [00:01<00:01, 81.3MB/s]
 58%|██████████████████████▊                | 103M/176M [00:01<00:00, 81.8MB/s]
 63%|████████████████████████▋              | 111M/176M [00:01<00:00, 82.2MB/s]
 68%|██████████████████████████▌            | 120M/176M [00:01<00:00, 82.8MB/s]
 73%|████████████████████████████▎          | 128M/176M [00:01<00:00, 82.8MB/s]
 77%|██████████████████████████████▏        | 136M/176M [00:01<00:00, 82.2MB/s]
 82%|████████████████████████████████       | 144M/176M [00:02<00:00, 81.8MB/s]
 87%|█████████████████████████████████▊     | 153M/176M [00:02<00:00, 81.6MB/s]
 91%|███████████████████████████████████▋   | 161M/176M [00:02<00:00, 80.8MB/s]
 96%|█████████████████████████████████████▍ | 169M/176M [00:02<00:00, 80.3MB/s]
0%|                                               | 0.00/176M [00:00<?, ?B/s]
100%|███████████████████████████████████████| 176M/176M [00:00<00:00, 1.32TB/s]
2024-05-13 17:30:40.700644309 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1121, index: 0, mask: {1, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.700823537 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1130, index: 9, mask: {10, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.700747598 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1126, index: 5, mask: {6, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.722960171 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1132, index: 11, mask: {12, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.725992506 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1136, index: 15, mask: {16, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.725981146 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1131, index: 10, mask: {11, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.729999340 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1138, index: 17, mask: {18, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.730008430 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1142, index: 21, mask: {22, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.700780808 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1128, index: 7, mask: {8, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.729992020 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1135, index: 14, mask: {15, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.714003865 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1127, index: 6, mask: {7, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.736119279 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1153, index: 32, mask: {33, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.729983850 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1137, index: 16, mask: {17, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.741155461 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1154, index: 33, mask: {34, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.714022925 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1129, index: 8, mask: {9, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.733990003 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1134, index: 13, mask: {14, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.745968675 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1155, index: 34, mask: {35, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.745993944 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1140, index: 19, mask: {20, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.749970379 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1141, index: 20, mask: {21, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.749980278 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1157, index: 36, mask: {37, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.756322725 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1158, index: 37, mask: {38, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.757976156 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1133, index: 12, mask: {13, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.749969699 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1156, index: 35, mask: {36, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.745970635 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1139, index: 18, mask: {19, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.745982295 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1143, index: 22, mask: {23, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.765969524 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1159, index: 38, mask: {39, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.769971617 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1144, index: 23, mask: {24, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.773970801 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1145, index: 24, mask: {25, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.777976385 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1146, index: 25, mask: {26, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.781971248 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1147, index: 26, mask: {27, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.781981438 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1160, index: 39, mask: {40, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.785973602 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1148, index: 27, mask: {28, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.785973862 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1161, index: 40, mask: {41, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.789973656 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1149, index: 28, mask: {29, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.789973426 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1162, index: 41, mask: {42, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.793983799 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1163, index: 42, mask: {43, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.794002809 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1152, index: 31, mask: {32, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.794035069 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1164, index: 43, mask: {44, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.793984779 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1150, index: 29, mask: {30, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.797976183 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1165, index: 44, mask: {45, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.801981787 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1151, index: 30, mask: {31, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.805971450 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1166, index: 45, mask: {46, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2024-05-13 17:30:40.809974415 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1167, index: 46, mask: {47, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
Seed set to 42
  0%|          | 0/75 [00:00<?, ?it/s]/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/diffusers/models/lora.py:96: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)
return F.conv2d(x, self.weight, self.bias, self.stride, self.padding, self.dilation, self.groups)
/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/torch/nn/modules/conv.py:456: UserWarning: Plan failed with a cudnnException: CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR: cudnnFinalize Descriptor Failed cudnn_status: CUDNN_STATUS_NOT_SUPPORTED (Triggered internally at ../aten/src/ATen/native/cudnn/Conv_v8.cpp:919.)
return F.conv2d(input, weight, bias, self.stride,
  1%|▏         | 1/75 [00:00<00:20,  3.61it/s]
  3%|▎         | 2/75 [00:00<00:13,  5.46it/s]
  4%|▍         | 3/75 [00:00<00:11,  6.52it/s]
  5%|▌         | 4/75 [00:00<00:09,  7.15it/s]
  7%|▋         | 5/75 [00:00<00:09,  7.56it/s]
  8%|▊         | 6/75 [00:00<00:08,  7.83it/s]
  9%|▉         | 7/75 [00:00<00:08,  8.01it/s]
 11%|█         | 8/75 [00:01<00:08,  8.14it/s]
 12%|█▏        | 9/75 [00:01<00:08,  8.22it/s]
 13%|█▎        | 10/75 [00:01<00:07,  8.27it/s]
 15%|█▍        | 11/75 [00:01<00:07,  8.32it/s]
 16%|█▌        | 12/75 [00:01<00:07,  8.35it/s]
 17%|█▋        | 13/75 [00:01<00:07,  8.38it/s]
 19%|█▊        | 14/75 [00:01<00:07,  8.40it/s]
 20%|██        | 15/75 [00:01<00:07,  8.40it/s]
 21%|██▏       | 16/75 [00:02<00:07,  8.40it/s]
 23%|██▎       | 17/75 [00:02<00:06,  8.41it/s]
 24%|██▍       | 18/75 [00:02<00:06,  8.43it/s]
 25%|██▌       | 19/75 [00:02<00:06,  8.45it/s]
 27%|██▋       | 20/75 [00:02<00:06,  8.45it/s]
 28%|██▊       | 21/75 [00:02<00:06,  8.45it/s]
 29%|██▉       | 22/75 [00:02<00:06,  8.45it/s]
 31%|███       | 23/75 [00:02<00:06,  8.44it/s]
 32%|███▏      | 24/75 [00:03<00:06,  8.45it/s]
 33%|███▎      | 25/75 [00:03<00:05,  8.43it/s]
 35%|███▍      | 26/75 [00:03<00:05,  8.32it/s]
 36%|███▌      | 27/75 [00:03<00:05,  8.34it/s]
 37%|███▋      | 28/75 [00:03<00:05,  8.38it/s]
 39%|███▊      | 29/75 [00:03<00:05,  8.40it/s]
 40%|████      | 30/75 [00:03<00:05,  8.42it/s]
 41%|████▏     | 31/75 [00:03<00:05,  8.44it/s]
 43%|████▎     | 32/75 [00:03<00:05,  8.45it/s]
 44%|████▍     | 33/75 [00:04<00:04,  8.45it/s]
 45%|████▌     | 34/75 [00:04<00:04,  8.45it/s]
 47%|████▋     | 35/75 [00:04<00:04,  8.46it/s]
 48%|████▊     | 36/75 [00:04<00:04,  8.46it/s]
 49%|████▉     | 37/75 [00:04<00:04,  8.45it/s]
 51%|█████     | 38/75 [00:04<00:04,  8.45it/s]
 52%|█████▏    | 39/75 [00:04<00:04,  8.45it/s]
 53%|█████▎    | 40/75 [00:04<00:04,  8.45it/s]
 55%|█████▍    | 41/75 [00:05<00:04,  8.45it/s]
 56%|█████▌    | 42/75 [00:05<00:03,  8.45it/s]
 57%|█████▋    | 43/75 [00:05<00:03,  8.45it/s]
 59%|█████▊    | 44/75 [00:05<00:03,  8.45it/s]
 60%|██████    | 45/75 [00:05<00:03,  8.44it/s]
 61%|██████▏   | 46/75 [00:05<00:03,  8.44it/s]
 63%|██████▎   | 47/75 [00:05<00:03,  8.45it/s]
 64%|██████▍   | 48/75 [00:05<00:03,  8.45it/s]
 65%|██████▌   | 49/75 [00:05<00:03,  8.45it/s]
 67%|██████▋   | 50/75 [00:06<00:02,  8.45it/s]
 68%|██████▊   | 51/75 [00:06<00:02,  8.45it/s]
 69%|██████▉   | 52/75 [00:06<00:02,  8.44it/s]
 71%|███████   | 53/75 [00:06<00:02,  8.45it/s]
 72%|███████▏  | 54/75 [00:06<00:02,  8.46it/s]
 73%|███████▎  | 55/75 [00:06<00:02,  8.45it/s]
 75%|███████▍  | 56/75 [00:06<00:02,  8.45it/s]
 76%|███████▌  | 57/75 [00:06<00:02,  8.45it/s]
 77%|███████▋  | 58/75 [00:07<00:02,  8.45it/s]
 79%|███████▊  | 59/75 [00:07<00:01,  8.45it/s]
 80%|████████  | 60/75 [00:07<00:01,  8.45it/s]
 81%|████████▏ | 61/75 [00:07<00:01,  8.44it/s]
 83%|████████▎ | 62/75 [00:07<00:01,  8.44it/s]
 84%|████████▍ | 63/75 [00:07<00:01,  8.44it/s]
 85%|████████▌ | 64/75 [00:07<00:01,  8.44it/s]
 87%|████████▋ | 65/75 [00:07<00:01,  8.43it/s]
 88%|████████▊ | 66/75 [00:07<00:01,  8.44it/s]
 89%|████████▉ | 67/75 [00:08<00:00,  8.44it/s]
 91%|█████████ | 68/75 [00:08<00:00,  8.44it/s]
 92%|█████████▏| 69/75 [00:08<00:00,  8.44it/s]
 93%|█████████▎| 70/75 [00:08<00:00,  8.44it/s]
 95%|█████████▍| 71/75 [00:08<00:00,  8.43it/s]
 96%|█████████▌| 72/75 [00:08<00:00,  8.43it/s]
 97%|█████████▋| 73/75 [00:08<00:00,  8.42it/s]
 99%|█████████▊| 74/75 [00:08<00:00,  8.43it/s]
100%|██████████| 75/75 [00:09<00:00,  8.42it/s]
100%|██████████| 75/75 [00:09<00:00,  8.29it/s]
/src/predict.py:41: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)
show_image = torch.from_numpy(show_image)     # (960, 640, 3)
/tmp/tmprdu7bxpx.obj
  0%|          | 0/6 [00:00<?, ?it/s]/content/InstantMesh/src/models/geometry/render/neural_render.py:51: UserWarning: Using torch.cross without specifying the dim arg is deprecated.
Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ../aten/src/ATen/native/Cross.cpp:62.)
face_normals = torch.cross(v1 - v0, v2 - v0)
 17%|█▋        | 1/6 [00:00<00:03,  1.27it/s]
 33%|███▎      | 2/6 [00:01<00:03,  1.32it/s]
 50%|█████     | 3/6 [00:02<00:02,  1.34it/s]
 67%|██████▋   | 4/6 [00:03<00:01,  1.34it/s]
 83%|████████▎ | 5/6 [00:03<00:00,  1.35it/s]
100%|██████████| 6/6 [00:04<00:00,  1.35it/s]
100%|██████████| 6/6 [00:04<00:00,  1.34it/s]
Video saved to /tmp/tmprdu7bxpx.mp4
Mesh saved to /tmp/tmprdu7bxpx.obj
Version Details
Version ID
e353a25cc764e0edb0aa9033df0bf4b82318dcda6d0a0cd9f2aace90566068ac
Version Created
May 13, 2024
Run on Replicate →