argildotai/sam2removevideobackground 📝🖼️ → 🖼️
About
This project uses the Segment Anything 2 (SAM2) model to remove backgrounds from videos.
Example Output
Output
Performance Metrics
23.60s
Prediction Time
181.42s
Total Time
All Input Parameters
{
"bg_color": "#00FF00",
"input_video": "https://replicate.delivery/pbxt/LSmApgOTlOeLHCvaa6dN4qCJFBK80vUwXPRcf1OAMANF7p8N/source3.mp4"
}
Input Parameters
- bg_color
- Background color (hex code)
- input_video (required)
- Input video file
Output Schema
Output
Example Execution Logs
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/tmpujkg2q_ksource3.mp4': Metadata: major_brand : mp42 minor_version : 1 compatible_brands: isommp41mp42 creation_time : 2024-08-16T03:18:21.000000Z Duration: 00:00:03.01, start: 0.000000, bitrate: 3784 kb/s Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 3616 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default) Metadata: creation_time : 2024-08-16T03:18:21.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : Lavc61.3.100 libx264 Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 129 kb/s (default) Metadata: creation_time : 2024-08-16T03:18:21.000000Z handler_name : Core Media Audio vendor_id : [0][0][0][0] Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native)) Press [q] to stop, [?] for help [swscaler @ 0x55cb6010bb40] deprecated pixel format used, make sure you did set range correctly Output #0, image2, to '/frames/%05d.jpg': Metadata: major_brand : mp42 minor_version : 1 compatible_brands: isommp41mp42 encoder : Lavf58.76.100 Stream #0:0(und): Video: mjpeg, yuvj420p(pc, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default) Metadata: creation_time : 2024-08-16T03:18:21.000000Z handler_name : Core Media Video vendor_id : [0][0][0][0] encoder : Lavc58.134.100 mjpeg Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A frame= 1 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 91 fps=0.0 q=2.0 size=N/A time=00:00:02.70 bitrate=N/A speed=5.04x frame= 91 fps=0.0 q=2.0 Lsize=N/A time=00:00:03.03 bitrate=N/A speed=5.56x video:15974kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown frame loading (JPEG): 0%| | 0/91 [00:00<?, ?it/s] frame loading (JPEG): 3%|▎ | 3/91 [00:00<00:03, 23.43it/s] frame loading (JPEG): 7%|▋ | 6/91 [00:00<00:03, 23.97it/s] frame loading (JPEG): 10%|▉ | 9/91 [00:00<00:03, 24.15it/s] frame loading (JPEG): 13%|█▎ | 12/91 [00:00<00:03, 24.07it/s] frame loading (JPEG): 16%|█▋ | 15/91 [00:00<00:03, 24.28it/s] frame loading (JPEG): 20%|█▉ | 18/91 [00:00<00:03, 24.23it/s] frame loading (JPEG): 23%|██▎ | 21/91 [00:00<00:02, 24.09it/s] frame loading (JPEG): 26%|██▋ | 24/91 [00:00<00:02, 24.14it/s] frame loading (JPEG): 30%|██▉ | 27/91 [00:01<00:02, 24.26it/s] frame loading (JPEG): 33%|███▎ | 30/91 [00:01<00:02, 24.36it/s] frame loading (JPEG): 36%|███▋ | 33/91 [00:01<00:02, 24.31it/s] frame loading (JPEG): 40%|███▉ | 36/91 [00:01<00:02, 23.79it/s] frame loading (JPEG): 43%|████▎ | 39/91 [00:01<00:02, 23.99it/s] frame loading (JPEG): 46%|████▌ | 42/91 [00:01<00:02, 23.96it/s] frame loading (JPEG): 49%|████▉ | 45/91 [00:01<00:01, 23.62it/s] frame loading (JPEG): 53%|█████▎ | 48/91 [00:02<00:01, 23.16it/s] frame loading (JPEG): 56%|█████▌ | 51/91 [00:02<00:01, 23.45it/s] frame loading (JPEG): 59%|█████▉ | 54/91 [00:02<00:01, 23.56it/s] frame loading (JPEG): 63%|██████▎ | 57/91 [00:02<00:01, 23.73it/s] frame loading (JPEG): 66%|██████▌ | 60/91 [00:02<00:01, 23.81it/s] frame loading (JPEG): 69%|██████▉ | 63/91 [00:02<00:01, 24.08it/s] frame loading (JPEG): 73%|███████▎ | 66/91 [00:02<00:01, 24.13it/s] frame loading (JPEG): 76%|███████▌ | 69/91 [00:02<00:00, 24.10it/s] frame loading (JPEG): 79%|███████▉ | 72/91 [00:03<00:00, 24.31it/s] frame loading (JPEG): 82%|████████▏ | 75/91 [00:03<00:00, 24.46it/s] frame loading (JPEG): 86%|████████▌ | 78/91 [00:03<00:00, 23.98it/s] frame loading (JPEG): 89%|████████▉ | 81/91 [00:03<00:00, 23.81it/s] frame loading (JPEG): 92%|█████████▏| 84/91 [00:03<00:00, 23.95it/s] frame loading (JPEG): 96%|█████████▌| 87/91 [00:03<00:00, 24.04it/s] frame loading (JPEG): 99%|█████████▉| 90/91 [00:03<00:00, 23.93it/s] frame loading (JPEG): 100%|██████████| 91/91 [00:03<00:00, 23.98it/s] INFO:root:Inference state initialized successfully /root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/sam2/sam2_video_predictor.py:873: UserWarning: cannot import name '_C' from 'sam2' (/root/.pyenv/versions/3.10.14/lib/python3.10/site-packages/sam2/__init__.py) Skipping the post-processing step due to the error above. You can still use SAM 2 and it's OK to ignore the error above, although some post-processing functionality may be limited (which doesn't affect the results in most cases; see https://github.com/facebookresearch/segment-anything-2/blob/main/INSTALL.md). pred_masks_gpu = fill_holes_in_mask_scores( INFO:root:New points added successfully propagate in video: 0%| | 0/91 [00:00<?, ?it/s] propagate in video: 3%|▎ | 3/91 [00:00<00:04, 18.53it/s] propagate in video: 5%|▌ | 5/91 [00:00<00:05, 15.87it/s] propagate in video: 8%|▊ | 7/91 [00:00<00:05, 14.71it/s] propagate in video: 10%|▉ | 9/91 [00:00<00:05, 14.08it/s] propagate in video: 12%|█▏ | 11/91 [00:00<00:05, 13.74it/s] propagate in video: 14%|█▍ | 13/91 [00:00<00:05, 13.57it/s] propagate in video: 16%|█▋ | 15/91 [00:01<00:05, 13.38it/s] propagate in video: 19%|█▊ | 17/91 [00:01<00:05, 13.31it/s] propagate in video: 21%|██ | 19/91 [00:01<00:05, 13.37it/s] propagate in video: 23%|██▎ | 21/91 [00:01<00:05, 13.28it/s] propagate in video: 25%|██▌ | 23/91 [00:01<00:05, 13.21it/s] propagate in video: 27%|██▋ | 25/91 [00:01<00:05, 13.13it/s] propagate in video: 30%|██▉ | 27/91 [00:01<00:04, 13.22it/s] propagate in video: 32%|███▏ | 29/91 [00:02<00:04, 13.21it/s] propagate in video: 34%|███▍ | 31/91 [00:02<00:04, 13.17it/s] propagate in video: 36%|███▋ | 33/91 [00:02<00:04, 13.20it/s] propagate in video: 38%|███▊ | 35/91 [00:02<00:04, 13.14it/s] propagate in video: 41%|████ | 37/91 [00:02<00:04, 13.22it/s] propagate in video: 43%|████▎ | 39/91 [00:02<00:03, 13.14it/s] propagate in video: 45%|████▌ | 41/91 [00:03<00:03, 13.22it/s] propagate in video: 47%|████▋ | 43/91 [00:03<00:03, 13.29it/s] propagate in video: 49%|████▉ | 45/91 [00:03<00:03, 13.35it/s] propagate in video: 52%|█████▏ | 47/91 [00:03<00:03, 13.37it/s] propagate in video: 54%|█████▍ | 49/91 [00:03<00:03, 13.39it/s] propagate in video: 56%|█████▌ | 51/91 [00:03<00:03, 13.27it/s] propagate in video: 58%|█████▊ | 53/91 [00:03<00:02, 13.21it/s] propagate in video: 60%|██████ | 55/91 [00:04<00:02, 13.17it/s] propagate in video: 63%|██████▎ | 57/91 [00:04<00:02, 13.15it/s] propagate in video: 65%|██████▍ | 59/91 [00:04<00:02, 13.09it/s] propagate in video: 67%|██████▋ | 61/91 [00:04<00:02, 13.04it/s] propagate in video: 69%|██████▉ | 63/91 [00:04<00:02, 13.14it/s] propagate in video: 71%|███████▏ | 65/91 [00:04<00:01, 13.20it/s] propagate in video: 74%|███████▎ | 67/91 [00:05<00:01, 13.28it/s] propagate in video: 76%|███████▌ | 69/91 [00:05<00:01, 13.28it/s] propagate in video: 78%|███████▊ | 71/91 [00:05<00:01, 13.28it/s] propagate in video: 80%|████████ | 73/91 [00:05<00:01, 13.20it/s] propagate in video: 82%|████████▏ | 75/91 [00:05<00:01, 13.18it/s] propagate in video: 85%|████████▍ | 77/91 [00:05<00:01, 13.25it/s] propagate in video: 87%|████████▋ | 79/91 [00:05<00:00, 13.24it/s] propagate in video: 89%|████████▉ | 81/91 [00:06<00:00, 13.24it/s] propagate in video: 91%|█████████ | 83/91 [00:06<00:00, 13.31it/s] propagate in video: 93%|█████████▎| 85/91 [00:06<00:00, 13.32it/s] propagate in video: 96%|█████████▌| 87/91 [00:06<00:00, 13.34it/s] propagate in video: 98%|█████████▊| 89/91 [00:06<00:00, 13.24it/s] propagate in video: 100%|██████████| 91/91 [00:06<00:00, 13.27it/s] propagate in video: 100%|██████████| 91/91 [00:06<00:00, 13.36it/s] INFO:root:Video segments propagated successfully ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 Input #0, image2, from '/output_frames/%05d.jpg': Duration: 00:00:03.03, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 30 tbn, 30 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264)) Press [q] to stop, [?] for help [swscaler @ 0x5599641b4b40] deprecated pixel format used, make sure you did set range correctly [libx264 @ 0x559963cff8c0] using SAR=1/1 [libx264 @ 0x559963cff8c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x559963cff8c0] profile High, level 4.0, 4:2:0, 8-bit [libx264 @ 0x559963cff8c0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to '/output.mp4': Metadata: encoder : Lavf58.76.100 Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 15360 tbn Metadata: encoder : Lavc58.134.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A frame= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x frame= 21 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x frame= 47 fps= 38 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= 0x frame= 89 fps= 51 q=29.0 size= 0kB time=00:00:01.20 bitrate= 0.3kbits/s speed=0.681x frame= 91 fps= 43 q=-1.0 Lsize= 541kB time=00:00:02.93 bitrate=1511.9kbits/s speed=1.38x video:539kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.357171% [libx264 @ 0x559963cff8c0] frame I:1 Avg QP:22.83 size: 25457 [libx264 @ 0x559963cff8c0] frame P:23 Avg QP:26.34 size: 11953 [libx264 @ 0x559963cff8c0] frame B:67 Avg QP:28.22 size: 3751 [libx264 @ 0x559963cff8c0] consecutive B-frames: 1.1% 2.2% 0.0% 96.7% [libx264 @ 0x559963cff8c0] mb I I16..4: 23.7% 72.9% 3.4% [libx264 @ 0x559963cff8c0] mb P I16..4: 1.2% 4.4% 0.2% P16..4: 12.7% 3.7% 1.7% 0.0% 0.0% skip:76.2% [libx264 @ 0x559963cff8c0] mb B I16..4: 0.1% 0.3% 0.0% B16..8: 11.7% 1.5% 0.2% direct: 0.4% skip:85.8% L0:46.3% L1:49.5% BI: 4.2% [libx264 @ 0x559963cff8c0] 8x8 transform intra:73.5% inter:86.2% [libx264 @ 0x559963cff8c0] coded y,uvDC,uvAC intra: 33.8% 51.0% 7.2% inter: 3.2% 6.2% 1.2% [libx264 @ 0x559963cff8c0] i16 v,h,dc,p: 44% 19% 7% 29% [libx264 @ 0x559963cff8c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 15% 40% 3% 3% 3% 3% 3% 3% [libx264 @ 0x559963cff8c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 19% 24% 4% 4% 4% 5% 4% 2% [libx264 @ 0x559963cff8c0] i8c dc,h,v,p: 63% 18% 17% 3% [libx264 @ 0x559963cff8c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x559963cff8c0] ref P L0: 59.6% 17.9% 16.2% 6.3% [libx264 @ 0x559963cff8c0] ref B L0: 92.1% 6.5% 1.4% [libx264 @ 0x559963cff8c0] ref B L1: 97.1% 2.9% [libx264 @ 0x559963cff8c0] kb/s:1455.05 INFO:root:Processed 91 frames INFO:root:Background removed video saved as /output.mp4
Version Details
- Version ID
86104887984d5f6ac2d8a12a3d206543e3fb27de3d2b85c27793ed6ce2730dea- Version Created
- September 3, 2024