nelsonjchen/op-replay-clipper 📝🔢❓✓ → 🖼️

▶️ 6.0K runs 📅 Sep 2023 ⚙️ Cog 0.17.2 🔗 GitHub ⚖️ License

autonomous-driving dashcam openpilot telemetry-visualization video-editing video-generation video-rendering

About

GPU accelerated replay renderer / video data clipper for comma.ai connect's openpilot route data. SEE README.

Example Output

Output

Performance Metrics

35.42s Prediction Time

256.38s Total Time

All Input Parameters

{
  "notes": "",
  "route": "https://connect.comma.ai/5beb9b58bd12b691/0000010a--a51155e496/90/105",
  "fileSize": 9,
  "fileFormat": "auto",
  "renderType": "ui-alt",
  "smearAmount": 3,
  "uiAltVariant": "device",
  "anonymizationProfile": "none",
  "passengerRedactionStyle": "blur"
}

Input Parameters

notes Type: stringDefault:: Optional notes for your own reference. Does not affect output.
route Type: stringDefault: https://connect.comma.ai/a2a0ccea32023010/1690488131496/1690488151496: One full comma connect clip URL (e.g. https://connect.comma.ai/18277b1abce7bbe4/00000029--e1c8705a52/132/144). Public Access must be enabled or a valid JWT Token must be provided. All required files for the selected render type in Comma Connect must be uploaded from device. Please see the Quick Usage section of the README on GitHub at https://github.com/nelsonjchen/op-replay-clipper#quick-usage for instructions on generating an appropriate comma connect URL.
fileSize Type: integerDefault: 9Range: 5 - 200: Target output size in MB. Actual size may vary.
jwtToken Type: stringDefault:: Optional JWT Token from https://jwt.comma.ai for routes without Public Access. Do not share this token: it is valid for 90 days and cannot be revoked early. Public Access is usually the safer option.
fileFormat Default: auto: Output codec. Auto is recommended: it uses HEVC for 360 renders and H.264 for the others. HEVC usually gives better quality for the size, but some browsers and devices do not play it well.
renderType Default: ui: UI renders with the comma openpilot UI. Forward, Wide, and Driver process the raw, segmented, and low-compatibility HEVC video files into a portable HEVC or H264 MP4 file, are fast transcodes, and are great for quick previews. 360 requires viewing/uploading the video file in VLC or YouTube to pan around in a sphere or post-processing with software such as Insta360 Studio or similar software for reframing. 360-ui is an expensive 360 render with an openpilot HUD/path overlay on the wide view. Forward Upon Wide roughly overlays Forward video on Wide video for increased detail in Forward video. 360 Forward Upon Wide is 360 with Forward Upon Wide as the forward video and scales up to render at 8K for reframing with Insta360 Studio or similar software.
smearAmount Type: integerDefault: 3Range: 3 - 40: (UI only) Seconds of hidden preroll before the visible clip start. Increase this if important UI state is missing at the beginning of the clip.
includeAudio Type: booleanDefault: false: Opt in to copying AAC audio from qcamera into the final MP4. The prediction fails early if qcamera audio is unavailable.
uiAltVariant: (ui-alt only) Alternate composition. `device` keeps a single main camera view with telemetry, while the stacked modes require wide video.
anonymizationProfile Default: none: Seat anonymization strategy for driver-backed renders such as driver, driver-debug, 360, 360-ui, and 360_forward_upon_wide. Available values are: none, driver unchanged/passenger hidden, driver unchanged/passenger face swap, driver face swap/passenger unchanged, driver face swap/passenger hidden, driver face swap/passenger face swap.
passengerRedactionStyle Default: blur: How to hide the passenger when the chosen anonymization profile uses passenger hidden mode.

Output Schema

Output

Type: string • Format: uri

Example Execution Logs

NOTES:


Route: 5beb9b58bd12b691|0000010a--a51155e496
Matched Route: 5beb9b58bd12b691|0000010a--a51155e496
Start Seconds: 90
Start Time: 90
Length Seconds: 15
Route or segment: 5beb9b58bd12b691|0000010a--a51155e496
Downloading route 5beb9b58bd12b691|0000010a--a51155e496
Downloading file list from https://api.commadotai.com/v1/route/5beb9b58bd12b691%7C0000010a--a51155e496/files
/root/.pyenv/versions/3.12.11/lib/python3.12/site-packages/parfive/downloader.py:258: UserWarning: This download has been started in a thread which is not the main thread. You will not be able to interrupt the download.
  warnings.warn(

Files Downloaded:   0%|          | 0/3 [00:00<?, ?file/s]


ecamera.hevc:   0%|          | 0.00/37.5M [00:00<?, ?B/s][A[A

fcamera.hevc:   0%|          | 0.00/37.4M [00:00<?, ?B/s][A



rlog.zst:   0%|          | 0.00/11.4M [00:00<?, ?B/s][A[A[A



rlog.zst:   0%|          | 16.6k/11.4M [00:00<01:56, 97.0kB/s][A[A[A

fcamera.hevc:   0%|          | 32.9k/37.4M [00:00<03:26, 181kB/s][A


ecamera.hevc:   0%|          | 16.6k/37.5M [00:00<07:06, 87.9kB/s][A[A

fcamera.hevc:   2%|▏         | 652k/37.4M [00:00<00:13, 2.82MB/s][A


ecamera.hevc:   2%|▏         | 817k/37.5M [00:00<00:10, 3.44MB/s] [A[A



rlog.zst:   6%|▌         | 636k/11.4M [00:00<00:03, 2.79MB/s] [A[A[A

fcamera.hevc:   6%|▌         | 2.16M/37.4M [00:00<00:04, 7.26MB/s][A


ecamera.hevc:   7%|▋         | 2.49M/37.5M [00:00<00:04, 8.22MB/s][A[A



rlog.zst:  16%|█▌        | 1.80M/11.4M [00:00<00:01, 5.74MB/s][A[A[A

fcamera.hevc:  11%|█         | 3.99M/37.4M [00:00<00:03, 11.1MB/s][A


ecamera.hevc:  13%|█▎        | 5.04M/37.5M [00:00<00:02, 12.9MB/s][A[A



rlog.zst:  36%|███▌      | 4.08M/11.4M [00:00<00:00, 10.4MB/s][A[A[A


ecamera.hevc:  20%|█▉        | 7.48M/37.5M [00:00<00:01, 16.5MB/s][A[A

fcamera.hevc:  17%|█▋        | 6.50M/37.4M [00:00<00:02, 14.2MB/s][A



rlog.zst:  61%|██████    | 6.87M/11.4M [00:00<00:00, 15.8MB/s][A[A[A

fcamera.hevc:  25%|██▍       | 9.20M/37.4M [00:00<00:01, 18.0MB/s][A



rlog.zst:  81%|████████  | 9.19M/11.4M [00:00<00:00, 18.0MB/s][A[A[A


ecamera.hevc:  27%|██▋       | 10.3M/37.5M [00:00<00:01, 18.0MB/s][A[A

fcamera.hevc:  34%|███▍      | 12.8M/37.4M [00:00<00:01, 22.0MB/s][A



rlog.zst:  99%|█████████▊| 11.2M/11.4M [00:00<00:00, 17.6MB/s][A[A[A



                                                              [A[A[A


ecamera.hevc:  38%|███▊      | 14.2M/37.5M [00:00<00:01, 23.1MB/s][A[A
Files Downloaded:  33%|███▎      | 1/3 [00:01<00:02,  1.12s/file]

fcamera.hevc:  45%|████▍     | 16.7M/37.4M [00:00<00:00, 25.9MB/s][A


ecamera.hevc:  48%|████▊     | 18.2M/37.5M [00:00<00:00, 26.9MB/s][A[A

fcamera.hevc:  57%|█████▋    | 21.5M/37.4M [00:01<00:00, 32.0MB/s][A


ecamera.hevc:  62%|██████▏   | 23.1M/37.5M [00:01<00:00, 33.2MB/s][A[A

fcamera.hevc:  66%|██████▌   | 24.8M/37.4M [00:01<00:00, 32.2MB/s][A


ecamera.hevc:  71%|███████   | 26.6M/37.5M [00:01<00:00, 33.0MB/s][A[A

fcamera.hevc:  78%|███████▊  | 29.2M/37.4M [00:01<00:00, 33.7MB/s][A


ecamera.hevc:  80%|███████▉  | 30.0M/37.5M [00:01<00:00, 33.2MB/s][A[A

fcamera.hevc:  88%|████████▊ | 33.0M/37.4M [00:01<00:00, 35.0MB/s][A


ecamera.hevc:  91%|█████████ | 34.0M/37.5M [00:01<00:00, 35.2MB/s][A[A


                                                                  [A[A
Files Downloaded:  67%|██████▋   | 2/3 [00:01<00:00,  1.26file/s]

fcamera.hevc:  98%|█████████▊| 36.5M/37.4M [00:01<00:00, 34.5MB/s][A

                                                                  [A
Files Downloaded: 100%|██████████| 3/3 [00:01<00:00,  1.69file/s]
Applied openpilot runtime patches: OpenpilotPatchReport(framereader_compat=True, ui_recording=True, ui_null_egl=True, augmented_road_fill=True, model_renderer_lead_position=True)
UI recording encoder: h264_nvenc (nvidia)
UI units: IsMetric missing in rlog.zst; defaulting to imperial
+ /home/batman/openpilot/.venv/bin/python /src/renderers/big_ui_engine.py 5beb9b58bd12b691/0000010a--a51155e496 --openpilot-dir /home/batman/openpilot -s 86 -e 105 -o /src/shared/cog-clip.mp4 -f 8 --big --layout-mode alt --ui-alt-variant device -d /src/shared/openpilot_data_dir
2026-05-01 21:56:44,541 INFO	Clipping 5beb9b58bd12b691|0000010a--a51155e496, 86s-105s (19s) with exact frame replay
2026-05-01 21:56:45,098 INFO	Loading log segment 1
GLFW forced null connect

Rendering:   0%|          | 0/380 [00:00<?, ?frame/s]
Rendering:   0%|          | 1/380 [00:00<05:11,  1.22frame/s]
Rendering:   1%|          | 3/380 [00:02<04:07,  1.52frame/s]FPS dropped below 20: 11

Rendering:   3%|▎         | 13/380 [00:02<00:41,  8.85frame/s]
Rendering:   6%|▌         | 22/380 [00:02<00:23, 15.25frame/s]
Rendering:   7%|▋         | 27/380 [00:03<00:45,  7.79frame/s]
Rendering:   9%|▉         | 34/380 [00:03<00:30, 11.52frame/s]
Rendering:  11%|█         | 41/380 [00:03<00:21, 16.01frame/s]Render progress: 43/380 frames, avg 7.77 fps, recent 7.77 fps, route 88.10s

Rendering:  12%|█▏        | 46/380 [00:05<00:43,  7.62frame/s]
Rendering:  14%|█▎        | 52/380 [00:05<00:31, 10.42frame/s]
Rendering:  15%|█▌        | 58/380 [00:05<00:23, 13.80frame/s]
Rendering:  17%|█▋        | 63/380 [00:07<00:46,  6.75frame/s]FPS dropped below 20: 6

Rendering:  18%|█▊        | 68/380 [00:07<00:35,  8.83frame/s]
Rendering:  20%|█▉        | 75/380 [00:07<00:23, 12.71frame/s]
Rendering:  21%|██▏       | 81/380 [00:07<00:17, 16.68frame/s]
Rendering:  23%|██▎       | 86/380 [00:08<00:26, 10.92frame/s]
Rendering:  24%|██▍       | 93/380 [00:08<00:18, 15.31frame/s]
Rendering:  26%|██▋       | 100/380 [00:09<00:13, 20.36frame/s]
Rendering:  28%|██▊       | 105/380 [00:09<00:22, 12.43frame/s]
Rendering:  29%|██▉       | 112/380 [00:09<00:15, 17.07frame/s]
Rendering:  31%|███▏      | 119/380 [00:10<00:11, 22.54frame/s]Render progress: 123/380 frames, avg 11.24 fps, recent 14.79 fps, route 92.10s

Rendering:  33%|███▎      | 125/380 [00:10<00:18, 13.59frame/s]
Rendering:  35%|███▍      | 132/380 [00:11<00:13, 18.13frame/s]
Rendering:  37%|███▋      | 139/380 [00:11<00:10, 23.56frame/s]
Rendering:  38%|███▊      | 145/380 [00:12<00:16, 13.90frame/s]
Rendering:  40%|███▉      | 151/380 [00:12<00:13, 17.56frame/s]
Rendering:  41%|████▏     | 157/380 [00:12<00:10, 21.79frame/s]
Rendering:  43%|████▎     | 163/380 [00:13<00:16, 13.42frame/s]FPS dropped below 20: 10

Rendering:  44%|████▍     | 169/380 [00:13<00:12, 17.28frame/s]
Rendering:  46%|████▋     | 176/380 [00:13<00:09, 22.63frame/s]
Rendering:  48%|████▊     | 183/380 [00:14<00:12, 15.25frame/s]
Rendering:  50%|████▉     | 189/380 [00:14<00:09, 19.19frame/s]
Rendering:  51%|█████▏    | 195/380 [00:14<00:07, 23.80frame/s]
Rendering:  53%|█████▎    | 202/380 [00:14<00:05, 30.13frame/s]
Rendering:  55%|█████▍    | 208/380 [00:15<00:11, 15.05frame/s]
Rendering:  57%|█████▋    | 215/380 [00:15<00:08, 20.08frame/s]
Rendering:  58%|█████▊    | 222/380 [00:15<00:06, 25.76frame/s]Render progress: 223/380 frames, avg 13.60 fps, recent 18.35 fps, route 97.10s

Rendering:  60%|██████    | 228/380 [00:16<00:10, 14.52frame/s]
Rendering:  62%|██████▏   | 235/380 [00:16<00:07, 19.36frame/s]
Rendering:  64%|██████▍   | 243/380 [00:17<00:10, 13.28frame/s]
Rendering:  66%|██████▌   | 250/380 [00:17<00:07, 17.41frame/s]
Rendering:  67%|██████▋   | 255/380 [00:17<00:06, 20.50frame/s]
Rendering:  69%|██████▉   | 262/380 [00:17<00:04, 26.39frame/s]FPS dropped below 20: 10

Rendering:  71%|███████   | 268/380 [00:18<00:07, 14.71frame/s]
Rendering:  72%|███████▏  | 275/380 [00:18<00:05, 19.54frame/s]
Rendering:  74%|███████▍  | 282/380 [00:18<00:03, 25.00frame/s]
Rendering:  76%|███████▌  | 288/380 [00:19<00:06, 13.64frame/s]
Rendering:  78%|███████▊  | 295/380 [00:20<00:04, 18.26frame/s]
Rendering:  79%|███████▉  | 302/380 [00:20<00:03, 23.58frame/s]
Rendering:  81%|████████  | 308/380 [00:21<00:05, 13.75frame/s]
Rendering:  83%|████████▎ | 315/380 [00:21<00:03, 18.34frame/s]
Rendering:  85%|████████▌ | 323/380 [00:21<00:04, 13.80frame/s]Render progress: 323/380 frames, avg 14.69 fps, recent 17.87 fps, route 102.10s

Rendering:  87%|████████▋ | 329/380 [00:22<00:02, 17.21frame/s]
Rendering:  88%|████████▊ | 335/380 [00:22<00:02, 21.19frame/s]
Rendering:  90%|█████████ | 342/380 [00:22<00:01, 26.80frame/s]
Rendering:  92%|█████████▏| 348/380 [00:23<00:02, 14.08frame/s]
Rendering:  93%|█████████▎| 354/380 [00:23<00:01, 17.78frame/s]
Rendering:  95%|█████████▍| 360/380 [00:23<00:00, 22.26frame/s]FPS dropped below 20: 14

Rendering:  96%|█████████▌| 365/380 [00:24<00:01, 12.61frame/s]
Rendering:  98%|█████████▊| 372/380 [00:24<00:00, 17.41frame/s]
Rendering: 100%|█████████▉| 379/380 [00:24<00:00, 23.00frame/s]Render progress: 380/380 frames, avg 15.44 fps, recent 21.76 fps, route 104.95s

Rendering: 100%|██████████| 380/380 [00:24<00:00, 15.44frame/s]
2026-05-01 21:57:12,869 INFO	Clip saved to: /src/shared/cog-clip.mp4
Render stats: frames=380, render_seconds=24.62, avg_fps=15.44
2026-05-01 21:57:12,869 INFO	Generated 19s in 28.3s (0.7x realtime) | import=0.00s, logs=1.28s, setup=1.90s, render=24.62s (1x), ffmpeg=0.53s
+ ffmpeg -y -ss 3 -i /src/shared/cog-clip.mp4 -c copy -movflags +faststart /src/shared/cog-clip.tmp.mp4
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/src/shared/cog-clip.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.76.100
  Duration: 00:00:18.00, start: 0.000000, bitrate: 3489 kb/s
  Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 2160x1080 [SAR 1:1 DAR 2:1], 3487 kb/s, 20 fps, 20 tbr, 10240 tbn, 40 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
Output #0, mp4, to '/src/shared/cog-clip.tmp.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.76.100
  Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 2160x1080 [SAR 1:1 DAR 2:1], q=2-31, 3487 kb/s, 20 fps, 20 tbr, 10240 tbn, 10240 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-1.0 size=       0kB time=-00:00:00.14 bitrate=N/A speed=N/A    
[mp4 @ 0x5c07495fdc80] Starting second pass: moving the moov atom to the beginning of the file
frame=  300 fps=0.0 q=-1.0 Lsize=    6343kB time=00:00:14.80 bitrate=3511.1kbits/s speed=1.6e+03x    
video:6340kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.055052%

Version Details

Version ID: 6f63c0b2f561bd46c25068446e94688e6dd1ec6f573f8ceaad1d9d1fd5b62364
Version Created: May 16, 2026

Run on Replicate →