pollinations/3d-photo-inpainting 🖼️ → 🖼️

▶️ 6.1K runs 📅 Feb 2023 ⚙️ Cog 0.6.1 🔗 GitHub ⚖️ License
3d-photography depth-inpainting image-to-video motion-parallax novel-view-synthesis rgb-d
About

3D Photography using Context-aware Layered Depth Inpainting
Example Output

Output

Performance Metrics

548.79s Prediction Time
636.04s Total Time
Input Parameters
image (required) Type: string: Image
Output Schema
Output
Type: array • Items Type: string • Items Format: uri
Example Execution Logs
total 52
-rw-r--r-- 1 root root 52789 Feb 10 22:09 image.jpg
running on device 0
0%|          | 0/1 [00:00<?, ?it/s]Current Source ==>  image
Running depth extraction at 1676066967.4100652
BoostingMonocularDepth/inputs/*.jpg
BoostingMonocularDepth/outputs/*.png
device: cuda
Namespace(Final=True, R0=False, R20=False, colorize_results=False, data_dir='inputs/', depthNet=0, max_res=inf, net_receptive_field_size=None, output_dir='outputs', output_resolution=1, pix2pixsize=1024, savepatchs=0, savewholeest=0)
----------------- Options ---------------
Final: True                          	[default: False]
R0: False
R20: False
aspect_ratio: 1.0
batch_size: 1
checkpoints_dir: ./pix2pix/checkpoints
colorize_results: False
crop_size: 672
data_dir: inputs/                       	[default: None]
dataroot: None
dataset_mode: depthmerge
depthNet: 0                             	[default: None]
direction: AtoB
display_winsize: 256
epoch: latest
eval: False
generatevideo: None
gpu_ids: 0
init_gain: 0.02
init_type: normal
input_nc: 2
isTrain: False                         	[default: None]
load_iter: 0                             	[default: 0]
load_size: 672
max_dataset_size: 10000
max_res: inf
model: pix2pix4depth
n_layers_D: 3
name: void
ndf: 64
netD: basic
netG: unet_1024
net_receptive_field_size: None
ngf: 64
no_dropout: False
no_flip: False
norm: none
num_test: 50
num_threads: 4
output_dir: outputs                       	[default: None]
output_nc: 1
output_resolution: None
phase: test
pix2pixsize: None
preprocess: resize_and_crop
savecrops: None
savewholeest: None
serial_batches: False
suffix:
verbose: False
----------------- End -------------------
initialize network with normal
loading the model from ./pix2pix/checkpoints/mergemodel/latest_net_G.pth
Loading weights:  midas/model.pt
Downloading: "https://github.com/facebookresearch/WSL-Images/zipball/main" to /root/.cache/torch/hub/main.zip
Downloading: "https://download.pytorch.org/models/ig_resnext101_32x8-c38310e5.pth" to /root/.cache/torch/hub/checkpoints/ig_resnext101_32x8-c38310e5.pth
  0%|          | 0.00/340M [00:00<?, ?B/s]
  2%|▏         | 6.51M/340M [00:00<00:05, 68.2MB/s]
  5%|▍         | 16.3M/340M [00:00<00:03, 88.3MB/s]
  8%|▊         | 25.9M/340M [00:00<00:03, 93.8MB/s]
 10%|█         | 35.1M/340M [00:00<00:03, 94.9MB/s]
 13%|█▎        | 44.2M/340M [00:00<00:03, 91.5MB/s]
 16%|█▌        | 53.2M/340M [00:00<00:03, 92.7MB/s]
 18%|█▊        | 62.8M/340M [00:00<00:03, 95.2MB/s]
 21%|██▏       | 72.3M/340M [00:00<00:02, 96.6MB/s]
 24%|██▍       | 82.0M/340M [00:00<00:02, 98.2MB/s]
 27%|██▋       | 91.8M/340M [00:01<00:02, 99.5MB/s]
 30%|██▉       | 101M/340M [00:01<00:02, 99.7MB/s] 
 33%|███▎      | 111M/340M [00:01<00:02, 101MB/s] 
 36%|███▌      | 121M/340M [00:01<00:02, 101MB/s]
 38%|███▊      | 130M/340M [00:01<00:02, 101MB/s]
 41%|████▏     | 140M/340M [00:01<00:02, 101MB/s]
 44%|████▍     | 150M/340M [00:01<00:01, 101MB/s]
 47%|████▋     | 160M/340M [00:01<00:01, 101MB/s]
 50%|████▉     | 169M/340M [00:01<00:01, 101MB/s]
 53%|█████▎    | 179M/340M [00:01<00:01, 101MB/s]
 56%|█████▌    | 189M/340M [00:02<00:01, 101MB/s]
 58%|█████▊    | 198M/340M [00:02<00:01, 99.5MB/s]
 61%|██████▏   | 208M/340M [00:02<00:01, 100MB/s] 
 64%|██████▍   | 218M/340M [00:02<00:01, 101MB/s]
 67%|██████▋   | 227M/340M [00:02<00:01, 101MB/s]
 70%|██████▉   | 237M/340M [00:02<00:01, 98.7MB/s]
 73%|███████▎  | 247M/340M [00:02<00:00, 99.6MB/s]
 75%|███████▌  | 256M/340M [00:02<00:00, 98.9MB/s]
 78%|███████▊  | 266M/340M [00:02<00:00, 98.8MB/s]
 81%|████████  | 275M/340M [00:02<00:00, 98.9MB/s]
 84%|████████▍ | 285M/340M [00:03<00:00, 99.8MB/s]
 87%|████████▋ | 294M/340M [00:03<00:00, 99.4MB/s]
 89%|████████▉ | 304M/340M [00:03<00:00, 94.2MB/s]
 92%|█████████▏| 314M/340M [00:03<00:00, 96.2MB/s]
 95%|█████████▌| 323M/340M [00:03<00:00, 95.8MB/s]
 98%|█████████▊| 332M/340M [00:03<00:00, 97.6MB/s]
100%|██████████| 340M/340M [00:03<00:00, 98.1MB/s]
start processing
processing image 0 : image
wholeImage being processed in : 1344
Adjust factor is: 1.0
Selecting patchs ...
Target resolution:  (2688, 2688, 3)
Dynamicly change merged-in resolution; scale: 0.19047619047619047
Resulted depthmap res will be : (512, 512)
patchs to process: 34
	 processing patch 0 | [  0   0 475 475]
	 processing patch 1 | [ 18  73 439 439]
	 processing patch 2 | [ 73  18 439 439]
	 processing patch 3 | [ 73  73 439 439]
	 processing patch 4 | [  0   0 366 366]
	 processing patch 5 | [  0  55 366 366]
	 processing patch 6 | [  0 110 366 366]
	 processing patch 7 | [ 55   0 366 366]
	 processing patch 8 | [ 18 183 329 329]
	 processing patch 9 | [ 73 183 329 329]
	 processing patch 10 | [128 183 329 329]
	 processing patch 11 | [183 128 329 329]
	 processing patch 12 | [183 183 329 329]
	 processing patch 13 | [  0   0 256 256]
	 processing patch 14 | [  0  55 256 256]
	 processing patch 15 | [  0 110 256 256]
	 processing patch 16 | [  0 165 256 256]
	 processing patch 17 | [  0 219 256 256]
processing patch 18 | [ 55   0 256 256]
	 processing patch 19 | [110   0 256 256]
	 processing patch 20 | [ 18 293 219 219]
	 processing patch 21 | [ 73 293 219 219]
processing patch 22 | [128 293 219 219]
	 processing patch 23 | [183 293 219 219]
	 processing patch 24 | [238 293 219 219]
	 processing patch 25 | [293 183 219 219]
	 processing patch 26 | [293 238 219 219]
	 processing patch 27 | [323 323 158 158]
	 processing patch 28 | [  0 110 146 146]
	 processing patch 29 | [  0 165 146 146]
	 processing patch 30 | [  0 219 146 146]
	 processing patch 31 | [  0 274 146 146]
processing patch 32 | [110   0 146 146]
	 processing patch 33 | [165   0 146 146]
finished
Start Running 3D_Photo ...
Loading edge model at 1676067019.1191473
Loading depth model at 1676067021.3657682
Loading rgb model at 1676067022.2461312
Writing depth ply (and basically doing everything) at 1676067023.0791702
writing
Writing mesh file mesh/image.ply ...
Making video at 1676067164.620918
fov: 53.13010235415598
Moviepy - Building video video/image_dolly-zoom-in.mp4.
  0%|          | 0/1 [04:41<?, ?it/s]
Moviepy - Writing video video/image_dolly-zoom-in.mp4
  0%|          | 0/1 [04:41<?, ?it/s]
t:   0%|          | 0/120 [00:00<?, ?it/s, now=None][A
t:   2%|▎         | 3/120 [00:00<00:04, 23.96it/s, now=None][A
t:  28%|██▊       | 33/120 [00:00<00:00, 170.09it/s, now=None][A
t:  43%|████▎     | 52/120 [00:00<00:00, 172.39it/s, now=None][A
t:  59%|█████▉    | 71/120 [00:00<00:00, 174.31it/s, now=None][A
t:  74%|███████▍  | 89/120 [00:00<00:00, 171.63it/s, now=None][A
t:  89%|████████▉ | 107/120 [00:00<00:00, 172.74it/s, now=None][A
                                                               [A
Moviepy - Done !
  0%|          | 0/1 [04:43<?, ?it/s]
Moviepy - video ready video/image_dolly-zoom-in.mp4
  0%|          | 0/1 [04:43<?, ?it/s]
Moviepy - Building video video/image_zoom-in.mp4.
0%|          | 0/1 [06:03<?, ?it/s]
Moviepy - Writing video video/image_zoom-in.mp4
  0%|          | 0/1 [06:03<?, ?it/s]
t:   0%|          | 0/120 [00:00<?, ?it/s, now=None][A
t:  22%|██▎       | 27/120 [00:00<00:00, 266.32it/s, now=None][A
t:  45%|████▌     | 54/120 [00:00<00:00, 179.66it/s, now=None][A
t:  62%|██████▏   | 74/120 [00:00<00:00, 178.36it/s, now=None][A
t:  78%|███████▊  | 93/120 [00:00<00:00, 179.47it/s, now=None][A
t:  93%|█████████▎| 112/120 [00:00<00:00, 161.27it/s, now=None][A
                                                               [A
Moviepy - Done !
  0%|          | 0/1 [06:04<?, ?it/s]
Moviepy - video ready video/image_zoom-in.mp4
  0%|          | 0/1 [06:04<?, ?it/s]
Moviepy - Building video video/image_circle.mp4.
0%|          | 0/1 [07:28<?, ?it/s]
Moviepy - Writing video video/image_circle.mp4
  0%|          | 0/1 [07:28<?, ?it/s]
t:   0%|          | 0/120 [00:00<?, ?it/s, now=None][A
t:  24%|██▍       | 29/120 [00:00<00:00, 285.58it/s, now=None][A
t:  48%|████▊     | 58/120 [00:00<00:00, 186.34it/s, now=None][A
t:  66%|██████▌   | 79/120 [00:00<00:00, 177.51it/s, now=None][A
t:  82%|████████▏ | 98/120 [00:00<00:00, 165.16it/s, now=None][A
t:  97%|█████████▋| 116/120 [00:00<00:00, 155.31it/s, now=None][A
                                                               [A
Moviepy - Done !
  0%|          | 0/1 [07:30<?, ?it/s]
Moviepy - video ready video/image_circle.mp4
  0%|          | 0/1 [07:30<?, ?it/s]
Moviepy - Building video video/image_swing.mp4.
0%|          | 0/1 [08:54<?, ?it/s]
Moviepy - Writing video video/image_swing.mp4
  0%|          | 0/1 [08:54<?, ?it/s]
t:   0%|          | 0/120 [00:00<?, ?it/s, now=None][A
t:  24%|██▍       | 29/120 [00:00<00:00, 283.42it/s, now=None][A
t:  48%|████▊     | 58/120 [00:00<00:00, 196.96it/s, now=None][A
t:  67%|██████▋   | 80/120 [00:00<00:00, 183.06it/s, now=None][A
t:  83%|████████▎ | 100/120 [00:00<00:00, 174.52it/s, now=None][A
t:  98%|█████████▊| 118/120 [00:00<00:00, 166.56it/s, now=None][A
                                                               [A
Moviepy - Done !
  0%|          | 0/1 [08:56<?, ?it/s]
Moviepy - video ready video/image_swing.mp4
  0%|          | 0/1 [08:56<?, ?it/s]
100%|██████████| 1/1 [08:56<00:00, 536.11s/it]
100%|██████████| 1/1 [08:56<00:00, 536.11s/it]
total 1160
-rw-r--r-- 1 root root 356648 Feb 10 22:16 image_circle.mp4
-rw-r--r-- 1 root root 244326 Feb 10 22:14 image_dolly-zoom-in.mp4
-rw-r--r-- 1 root root 321436 Feb 10 22:18 image_swing.mp4
-rw-r--r-- 1 root root 255991 Feb 10 22:15 image_zoom-in.mp4
Version Details
Version ID: 1fd4e970159fb9b7c2dfa39c8172c2d862f1f042c1eb1a7effca1b6e055bd4be
Version Created: February 11, 2023
Run on Replicate →