vt-vl-lab/3d-photo-inpainting ❓🖼️ → 🖼️
About
3D Photography using Context-aware Layered Depth Inpainting
Example Output
Output
Performance Metrics
539.99s
Prediction Time
540.12s
Total Time
All Input Parameters
{
"effect": "swing",
"image_path": "https://replicate.delivery/mgxm/f5b74651-0757-465f-9529-30ecbab6aa4b/moon.jpg"
}
Input Parameters
- effect (required)
- Video animation effect
- image_path (required)
- Input image
Output Schema
Output
Example Execution Logs
Saving input image to input/image.jpg...
running on device 0
Current Source ==> image
Running depth extraction at 1655928002.989555
BoostingMonocularDepth/inputs/*.jpg
BoostingMonocularDepth/outputs/*.png
device: cuda
Namespace(Final=True, R0=False, R20=False, colorize_results=False, data_dir='inputs/', depthNet=0, max_res=inf, net_receptive_field_size=None, output_dir='outputs', output_resolution=1, pix2pixsize=1024, savepatchs=0, savewholeest=0)
----------------- Options ---------------
Final: True [default: False]
R0: False
R20: False
aspect_ratio: 1.0
batch_size: 1
checkpoints_dir: ./pix2pix/checkpoints
colorize_results: False
crop_size: 672
data_dir: inputs/ [default: None]
dataroot: None
dataset_mode: depthmerge
depthNet: 0 [default: None]
direction: AtoB
display_winsize: 256
epoch: latest
eval: False
generatevideo: None
gpu_ids: 0
init_gain: 0.02
init_type: normal
input_nc: 2
isTrain: False [default: None]
load_iter: 0 [default: 0]
load_size: 672
max_dataset_size: 10000
max_res: inf
model: pix2pix4depth
n_layers_D: 3
name: void
ndf: 64
netD: basic
netG: unet_1024
net_receptive_field_size: None
ngf: 64
no_dropout: False
no_flip: False
norm: none
num_test: 50
num_threads: 4
output_dir: outputs [default: None]
output_nc: 1
output_resolution: None
phase: test
pix2pixsize: None
preprocess: resize_and_crop
savecrops: None
savewholeest: None
serial_batches: False
suffix:
verbose: False
----------------- End -------------------
initialize network with normal
loading the model from ./pix2pix/checkpoints/mergemodel/latest_net_G.pth
Loading weights: midas/model.pt
0%| | 0/1 [00:00<?, ?it/s]Using cache found in /root/.cache/torch/hub/facebookresearch_WSL-Images_main
start processing
processing image 0 : image
wholeImage being processed in : 2112
/root/.pyenv/versions/3.8.13/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
DEBUG| GPU THRESHOLD REACHED 2112 ---> 1568
Adjust factor is: 1.0
Selecting patchs ...
Target resolution: (4224, 4224, 3)
Dynamicly change merged-in resolution; scale: 0.4971590909090909
Resulted depthmap res will be : (2100, 2100)
patchs to process: 54
processing patch 0 | [ 0 286 1814 1814]
processing patch 1 | [ 143 286 1814 1814]
processing patch 2 | [ 286 286 1814 1814]
processing patch 3 | [ 0 430 1527 1527]
processing patch 4 | [ 0 573 1527 1527]
processing patch 5 | [ 143 573 1527 1527]
processing patch 6 | [ 286 573 1527 1527]
processing patch 7 | [ 430 573 1527 1527]
processing patch 8 | [ 573 430 1527 1527]
processing patch 9 | [ 573 573 1527 1527]
processing patch 10 | [ 0 573 1241 1241]
processing patch 11 | [ 0 716 1241 1241]
processing patch 12 | [ 0 859 1241 1241]
processing patch 13 | [ 143 859 1241 1241]
processing patch 14 | [ 286 859 1241 1241]
processing patch 15 | [ 430 859 1241 1241]
processing patch 16 | [ 573 859 1241 1241]
processing patch 17 | [ 716 859 1241 1241]
processing patch 18 | [ 859 573 1241 1241]
processing patch 19 | [ 859 716 1241 1241]
processing patch 20 | [ 859 859 1241 1241]
processing patch 21 | [ 0 716 955 955]
processing patch 22 | [ 0 859 955 955]
processing patch 23 | [ 0 1002 955 955]
processing patch 24 | [ 0 1145 955 955]
processing patch 25 | [ 430 1145 955 955]
processing patch 26 | [ 573 1145 955 955]
processing patch 27 | [ 716 1145 955 955]
processing patch 28 | [ 859 1145 955 955]
processing patch 29 | [1002 1145 955 955]
processing patch 30 | [1145 716 955 955]
processing patch 31 | [1145 859 955 955]
processing patch 32 | [1145 1002 955 955]
processing patch 33 | [1145 1145 955 955]
processing patch 34 | [ 0 859 668 668]
processing patch 35 | [ 0 1002 668 668]
processing patch 36 | [ 0 1145 668 668]
processing patch 37 | [ 0 1289 668 668]
processing patch 38 | [ 430 1432 668 668]
processing patch 39 | [ 573 1432 668 668]
processing patch 40 | [ 716 1432 668 668]
processing patch 41 | [ 859 1432 668 668]
processing patch 42 | [1002 1432 668 668]
processing patch 43 | [1145 1432 668 668]
processing patch 44 | [1289 1432 668 668]
processing patch 45 | [1432 859 668 668]
processing patch 46 | [1432 1002 668 668]
processing patch 47 | [1432 1145 668 668]
processing patch 48 | [1432 1289 668 668]
processing patch 49 | [1432 1432 668 668]
processing patch 50 | [ 0 1002 382 382]
processing patch 51 | [ 0 1145 382 382]
processing patch 52 | [ 0 1289 382 382]
processing patch 53 | [ 0 1432 382 382]
finished
Start Running 3D_Photo ...
Loading edge model at 1655928051.4778206
Loading depth model at 1655928051.7218115
Loading rgb model at 1655928052.55295
Writing depth ply (and basically doing everything) at 1655928053.30448
Writing mesh file mesh/image.ply ...
Making video at 1655928245.2453136
fov: 53.13010235415598
WARNING: could not determine DPI
Moviepy - Building video video/image_swing.mp4.
0%| | 0/1 [08:52<?, ?it/s]
Moviepy - Writing video video/image_swing.mp4
0%| | 0/1 [08:52<?, ?it/s]
t: 0%| | 0/241 [00:00<?, ?it/s, now=None][A
t: 1%| | 3/241 [00:00<00:09, 24.49it/s, now=None][A
t: 7%|▋ | 17/241 [00:00<00:02, 84.48it/s, now=None][A
t: 12%|█▏ | 30/241 [00:00<00:02, 103.85it/s, now=None][A
t: 18%|█▊ | 44/241 [00:00<00:01, 117.27it/s, now=None][A
t: 23%|██▎ | 56/241 [00:00<00:02, 84.58it/s, now=None] [A
t: 27%|██▋ | 66/241 [00:00<00:02, 75.16it/s, now=None][A
t: 32%|███▏ | 77/241 [00:00<00:01, 82.33it/s, now=None][A
t: 36%|███▌ | 87/241 [00:01<00:01, 85.23it/s, now=None][A
t: 40%|████ | 97/241 [00:01<00:01, 81.63it/s, now=None][A
t: 44%|████▍ | 106/241 [00:01<00:01, 81.88it/s, now=None][A
t: 48%|████▊ | 115/241 [00:01<00:01, 81.63it/s, now=None][A
t: 51%|█████▏ | 124/241 [00:01<00:01, 79.72it/s, now=None][A
t: 55%|█████▌ | 133/241 [00:01<00:01, 78.94it/s, now=None][A
t: 59%|█████▉ | 142/241 [00:01<00:01, 79.26it/s, now=None][A
t: 62%|██████▏ | 150/241 [00:01<00:01, 79.26it/s, now=None][A
t: 66%|██████▌ | 159/241 [00:01<00:01, 80.21it/s, now=None][A
t: 70%|██████▉ | 168/241 [00:02<00:00, 80.70it/s, now=None][A
t: 74%|███████▍ | 179/241 [00:02<00:00, 86.56it/s, now=None][A
t: 78%|███████▊ | 189/241 [00:02<00:00, 89.79it/s, now=None][A
t: 83%|████████▎ | 199/241 [00:02<00:00, 88.28it/s, now=None][A
t: 86%|████████▋ | 208/241 [00:02<00:00, 84.78it/s, now=None][A
t: 90%|█████████ | 217/241 [00:02<00:00, 79.44it/s, now=None][A
t: 94%|█████████▍| 226/241 [00:02<00:00, 79.45it/s, now=None][A
t: 98%|█████████▊| 235/241 [00:02<00:00, 79.48it/s, now=None][A
[A
Moviepy - Done !
0%| | 0/1 [08:55<?, ?it/s]
Moviepy - video ready video/image_swing.mp4
Done. Saving to output path: video/image_swing.mp4
0%| | 0/1 [08:55<?, ?it/s]
0%| | 0/1 [08:55<?, ?it/s]
Version Details
- Version ID
e598ca2ac7b080b1f90f2ba839aeab65970ebb8893edd2caa4e0dba37ea63602- Version Created
- June 22, 2022