nelsonjchen/op-replay-clipper 📝🔢❓ → 🖼️

▶️ 5.7K runs 📅 Sep 2023 ⚙️ Cog 0.17.2 🔗 GitHub ⚖️ License

autonomous-driving dashcam openpilot telemetry-visualization video-editing video-generation video-rendering

About

GPU accelerated replay renderer / video data clipper for comma.ai connect's openpilot route data. SEE README.

Example Output

Output

Performance Metrics

39.82s Prediction Time

181.38s Total Time

All Input Parameters

{
  "notes": "prod example refresh 2026-03-27 ui-alt",
  "route": "https://connect.comma.ai/5beb9b58bd12b691/0000010a--a51155e496/90/105",
  "fileSize": 9,
  "jwtToken": "",
  "fileFormat": "auto",
  "renderType": "ui-alt",
  "smearAmount": 3
}

Input Parameters

notes Type: stringDefault:: Optional notes for your own reference. Does not affect output.
route Type: stringDefault: https://connect.comma.ai/a2a0ccea32023010/1690488131496/1690488151496: One full comma connect clip URL (e.g. https://connect.comma.ai/18277b1abce7bbe4/00000029--e1c8705a52/132/144). Public Access must be enabled or a valid JWT Token must be provided. All required files for the selected render type in Comma Connect must be uploaded from device. Please see the Quick Usage section of the README on GitHub at https://github.com/nelsonjchen/op-replay-clipper#quick-usage for instructions on generating an appropriate comma connect URL.
fileSize Type: integerDefault: 9Range: 5 - 200: Target output size in MB. Actual size may vary.
jwtToken Type: stringDefault:: Optional JWT Token from https://jwt.comma.ai for routes without Public Access. Do not share this token: it is valid for 90 days and cannot be revoked early. Public Access is usually the safer option.
fileFormat Default: auto: Output codec. Auto is recommended: it uses HEVC for 360 renders and H.264 for the others. HEVC usually gives better quality for the size, but some browsers and devices do not play it well.
renderType Default: ui: UI renders with the comma openpilot UI. Forward, Wide, and Driver process the raw, segmented, and low-compatibility HEVC video files into a portable HEVC or H264 MP4 file, are fast transcodes, and are great for quick previews. 360 requires viewing/uploading the video file in VLC or YouTube to pan around in a sphere or post-processing with software such as Insta360 Studio or similar software for reframing. Forward Upon Wide roughly overlays Forward video on Wide video for increased detail in Forward video. 360 Forward Upon Wide is 360 with Forward Upon Wide as the forward video and scales up to render at 8K for reframing with Insta360 Studio or similar software.
smearAmount Type: integerDefault: 3Range: 3 - 40: (UI only) Seconds of hidden preroll before the visible clip start. Increase this if important UI state is missing at the beginning of the clip.
uiAltVariant: (ui-alt only) Alternate composition. `device` keeps a single main camera view with telemetry, while the stacked modes require wide video.
anonymizationProfile Default: none: Seat anonymization strategy for driver-backed renders such as driver, driver-debug, 360, and 360_forward_upon_wide. Recommended values are: none, driver unchanged/passenger hidden, driver unchanged/passenger face swap, driver face swap/passenger hidden, driver face swap/passenger face swap.
passengerRedactionStyle Default: blur: How to hide the passenger when the chosen anonymization profile uses passenger hidden mode.

Output Schema

Output

Type: string • Format: uri

Example Execution Logs

NOTES:
prod example refresh 2026-03-27 ui-alt
Route: 5beb9b58bd12b691|0000010a--a51155e496
Matched Route: 5beb9b58bd12b691|0000010a--a51155e496
Start Seconds: 90
Start Time: 90
Length Seconds: 15
Route or segment: 5beb9b58bd12b691|0000010a--a51155e496
Downloading route 5beb9b58bd12b691|0000010a--a51155e496
Downloading file list from https://api.commadotai.com/v1/route/5beb9b58bd12b691%7C0000010a--a51155e496/files
Downloading route segment info from https://api.comma.ai/v1/devices/5beb9b58bd12b691/routes_segments?end=1774663506253&start=0
Route 5beb9b58bd12b691|0000010a--a51155e496 starts at 1772040690000 and ends at 1772040750000
View the route at https://connect.comma.ai/5beb9b58bd12b691/1772040690000/1772040750000
Files Downloaded:   0%|          | 0/3 [00:00<?, ?file/s]
rlog.zst:   0%|          | 0.00/11.4M [00:00<?, ?B/s][A[A[A
ecamera.hevc:   0%|          | 0.00/37.5M [00:00<?, ?B/s][A[A
fcamera.hevc:   0%|          | 0.00/37.4M [00:00<?, ?B/s][A
rlog.zst:   0%|          | 1.02k/11.4M [00:00<2:37:54, 1.20kB/s][A[A[A
fcamera.hevc:   0%|          | 16.6k/37.4M [00:00<34:24, 18.1kB/s][A
ecamera.hevc:   0%|          | 1.02k/37.5M [00:00<9:20:11, 1.12kB/s][A[A
rlog.zst:   2%|▏         | 261k/11.4M [00:00<00:30, 368kB/s]    [A[A[A
ecamera.hevc:   1%|          | 407k/37.5M [00:01<01:07, 547kB/s]    [A[A
fcamera.hevc:   2%|▏         | 718k/37.4M [00:01<00:38, 948kB/s]  [A
rlog.zst:   9%|▉         | 1.03M/11.4M [00:01<00:06, 1.58MB/s][A[A[A
ecamera.hevc:   4%|▍         | 1.65M/37.5M [00:01<00:14, 2.46MB/s][A[A
fcamera.hevc:   6%|▋         | 2.37M/37.4M [00:01<00:10, 3.41MB/s][A
rlog.zst:  23%|██▎       | 2.57M/11.4M [00:01<00:02, 4.22MB/s][A[A[A
ecamera.hevc:  10%|█         | 3.88M/37.5M [00:01<00:05, 5.98MB/s][A[A
fcamera.hevc:  14%|█▍        | 5.37M/37.4M [00:01<00:03, 8.02MB/s][A
rlog.zst:  42%|████▏     | 4.74M/11.4M [00:01<00:00, 7.97MB/s][A[A[A
ecamera.hevc:  19%|█▉        | 7.05M/37.5M [00:01<00:02, 11.3MB/s][A[A
rlog.zst:  55%|█████▌    | 6.25M/11.4M [00:01<00:00, 9.40MB/s][A[A[A
fcamera.hevc:  21%|██        | 7.87M/37.4M [00:01<00:02, 10.8MB/s][A
ecamera.hevc:  26%|██▌       | 9.72M/37.5M [00:01<00:01, 14.3MB/s][A[A
rlog.zst:  78%|███████▊  | 8.90M/11.4M [00:01<00:00, 12.4MB/s][A[A[A
fcamera.hevc:  30%|███       | 11.3M/37.4M [00:01<00:01, 14.6MB/s][A
ecamera.hevc:  31%|███▏      | 11.8M/37.5M [00:01<00:01, 15.7MB/s][A[A
rlog.zst:  93%|█████████▎| 10.6M/11.4M [00:01<00:00, 13.2MB/s][A[A[A
ecamera.hevc:  40%|███▉      | 14.9M/37.5M [00:01<00:01, 19.7MB/s][A[A
fcamera.hevc:  41%|████      | 15.3M/37.4M [00:01<00:01, 18.5MB/s][A
                                                              [A[A[A
Files Downloaded:  33%|███▎      | 1/3 [00:01<00:03,  1.94s/file]
ecamera.hevc:  49%|████▉     | 18.3M/37.5M [00:01<00:00, 22.8MB/s][A[A
fcamera.hevc:  51%|█████▏    | 19.2M/37.4M [00:01<00:00, 22.5MB/s][A
ecamera.hevc:  59%|█████▉    | 22.3M/37.5M [00:01<00:00, 26.8MB/s][A[A
fcamera.hevc:  62%|██████▏   | 23.1M/37.4M [00:01<00:00, 26.2MB/s][A
ecamera.hevc:  71%|███████▏  | 26.8M/37.5M [00:01<00:00, 32.0MB/s][A[A
fcamera.hevc:  74%|███████▍  | 27.8M/37.4M [00:01<00:00, 31.4MB/s][A
ecamera.hevc:  83%|████████▎ | 31.2M/37.5M [00:02<00:00, 35.1MB/s][A[A
fcamera.hevc:  87%|████████▋ | 32.6M/37.4M [00:02<00:00, 35.0MB/s][A
ecamera.hevc:  98%|█████████▊| 36.7M/37.5M [00:02<00:00, 40.7MB/s][A[A
                                                                  [A
Files Downloaded:  67%|██████▋   | 2/3 [00:02<00:01,  1.07s/file]
                                                                  [A[A
Files Downloaded: 100%|██████████| 3/3 [00:02<00:00,  1.25file/s]
Applied openpilot runtime patches: OpenpilotPatchReport(framereader_compat=True, ui_recording=True, ui_null_egl=True, augmented_road_fill=True, model_renderer_lead_position=True)
UI recording encoder: h264_nvenc (nvidia)
UI units: IsMetric missing in rlog.zst; defaulting to imperial
kj/filesystem-disk-unix.c++:1734: warning: PWD environment variable doesn't match current directory; pwd = /src
+ /home/batman/openpilot/.venv/bin/python /src/renderers/big_ui_engine.py 5beb9b58bd12b691/0000010a--a51155e496 --openpilot-dir /home/batman/openpilot -s 86 -e 105 -o /src/shared/cog-clip.mp4 -f 8 --big --layout-mode alt -d /src/shared/openpilot_data_dir
2026-03-28 02:05:14,559 INFO	Clipping 5beb9b58bd12b691|0000010a--a51155e496, 86s-105s (19s) with exact frame replay
2026-03-28 02:05:14,991 INFO	Loading log segment 1
GLFW forced null connect
Rendering:   0%|          | 0/380 [00:00<?, ?frame/s]
Rendering:   0%|          | 1/380 [00:00<02:44,  2.30frame/s]
Rendering:   1%|          | 3/380 [00:01<03:20,  1.88frame/s]
Rendering:   2%|▏         | 9/380 [00:01<00:57,  6.48frame/s]FPS dropped below 20: 13
Rendering:   5%|▍         | 18/380 [00:01<00:23, 15.11frame/s]
Rendering:   6%|▌         | 23/380 [00:03<00:43,  8.16frame/s]
Rendering:   7%|▋         | 27/380 [00:03<00:34, 10.14frame/s]
Rendering:   8%|▊         | 31/380 [00:03<00:27, 12.64frame/s]
Rendering:   9%|▉         | 35/380 [00:03<00:22, 15.35frame/s]
Rendering:  10%|█         | 39/380 [00:03<00:18, 18.21frame/s]
Rendering:  11%|█▏        | 43/380 [00:04<00:32, 10.48frame/s]
Rendering:  12%|█▏        | 46/380 [00:04<00:26, 12.39frame/s]
Rendering:  13%|█▎        | 50/380 [00:04<00:21, 15.01frame/s]
Rendering:  14%|█▍        | 53/380 [00:04<00:19, 17.14frame/s]
Rendering:  15%|█▍        | 56/380 [00:04<00:16, 19.16frame/s]
Rendering:  16%|█▌        | 59/380 [00:04<00:15, 20.65frame/s]Render progress: 61/380 frames, avg 12.18 fps, recent 12.18 fps, route 89.00s
Rendering:  16%|█▋        | 62/380 [00:05<00:14, 21.38frame/s]
Rendering:  17%|█▋        | 65/380 [00:05<00:28, 10.92frame/s]
Rendering:  18%|█▊        | 69/380 [00:05<00:22, 13.89frame/s]
Rendering:  19%|█▉        | 73/380 [00:05<00:18, 16.91frame/s]
Rendering:  20%|██        | 77/380 [00:06<00:15, 19.75frame/s]
Rendering:  21%|██▏       | 81/380 [00:06<00:13, 21.71frame/s]
Rendering:  22%|██▏       | 84/380 [00:06<00:23, 12.40frame/s]FPS dropped below 20: 15
Rendering:  23%|██▎       | 87/380 [00:06<00:20, 14.42frame/s]
Rendering:  24%|██▎       | 90/380 [00:06<00:17, 16.70frame/s]
Rendering:  25%|██▍       | 94/380 [00:07<00:14, 20.29frame/s]
Rendering:  26%|██▌       | 97/380 [00:07<00:12, 22.02frame/s]
Rendering:  27%|██▋       | 101/380 [00:07<00:11, 24.60frame/s]
Rendering:  27%|██▋       | 104/380 [00:07<00:22, 12.38frame/s]
Rendering:  28%|██▊       | 108/380 [00:07<00:16, 16.03frame/s]
Rendering:  29%|██▉       | 112/380 [00:08<00:14, 18.74frame/s]
Rendering:  31%|███       | 116/380 [00:08<00:12, 21.92frame/s]
Rendering:  32%|███▏      | 120/380 [00:08<00:10, 24.73frame/s]
Rendering:  33%|███▎      | 124/380 [00:08<00:19, 13.29frame/s]
Rendering:  34%|███▎      | 128/380 [00:09<00:15, 16.12frame/s]
Rendering:  34%|███▍      | 131/380 [00:09<00:13, 18.14frame/s]
Rendering:  36%|███▌      | 135/380 [00:09<00:11, 21.35frame/s]
Rendering:  36%|███▋      | 138/380 [00:09<00:10, 22.83frame/s]
Rendering:  37%|███▋      | 141/380 [00:09<00:09, 24.25frame/s]Render progress: 143/380 frames, avg 14.13 fps, recent 16.04 fps, route 93.10s
Rendering:  38%|███▊      | 144/380 [00:10<00:20, 11.38frame/s]
Rendering:  39%|███▊      | 147/380 [00:10<00:17, 13.67frame/s]
Rendering:  40%|███▉      | 151/380 [00:10<00:13, 17.42frame/s]
Rendering:  41%|████      | 155/380 [00:10<00:10, 20.82frame/s]
Rendering:  42%|████▏     | 158/380 [00:10<00:09, 22.27frame/s]
Rendering:  42%|████▏     | 161/380 [00:10<00:09, 23.23frame/s]
Rendering:  43%|████▎     | 164/380 [00:11<00:17, 12.28frame/s]
Rendering:  44%|████▍     | 168/380 [00:11<00:13, 15.56frame/s]
Rendering:  45%|████▌     | 171/380 [00:11<00:11, 17.86frame/s]
Rendering:  46%|████▌     | 174/380 [00:11<00:10, 19.76frame/s]
Rendering:  47%|████▋     | 178/380 [00:11<00:08, 23.01frame/s]
Rendering:  48%|████▊     | 182/380 [00:11<00:07, 25.67frame/s]FPS dropped below 20: 16
Rendering:  49%|████▊     | 185/380 [00:12<00:15, 12.80frame/s]
Rendering:  50%|████▉     | 189/380 [00:12<00:11, 16.19frame/s]
Rendering:  51%|█████     | 193/380 [00:12<00:09, 19.44frame/s]
Rendering:  52%|█████▏    | 197/380 [00:12<00:08, 22.39frame/s]
Rendering:  53%|█████▎    | 201/380 [00:12<00:07, 24.14frame/s]
Rendering:  54%|█████▎    | 204/380 [00:13<00:14, 12.54frame/s]
Rendering:  54%|█████▍    | 207/380 [00:13<00:11, 14.76frame/s]
Rendering:  55%|█████▌    | 210/380 [00:13<00:10, 17.00frame/s]
Rendering:  56%|█████▋    | 214/380 [00:13<00:08, 20.13frame/s]
Rendering:  57%|█████▋    | 217/380 [00:13<00:07, 22.01frame/s]
Rendering:  58%|█████▊    | 221/380 [00:14<00:06, 23.44frame/s]
Rendering:  59%|█████▉    | 224/380 [00:14<00:12, 12.35frame/s]
Rendering:  60%|█████▉    | 227/380 [00:14<00:10, 14.65frame/s]
Rendering:  61%|██████    | 231/380 [00:14<00:08, 18.08frame/s]
Rendering:  62%|██████▏   | 235/380 [00:14<00:06, 21.01frame/s]
Rendering:  63%|██████▎   | 239/380 [00:15<00:05, 23.89frame/s]Render progress: 240/380 frames, avg 15.84 fps, recent 19.28 fps, route 97.95s
Rendering:  64%|██████▍   | 243/380 [00:15<00:10, 12.91frame/s]
Rendering:  65%|██████▌   | 247/380 [00:15<00:08, 15.94frame/s]
Rendering:  66%|██████▌   | 251/380 [00:15<00:06, 19.26frame/s]
Rendering:  67%|██████▋   | 255/380 [00:16<00:05, 22.44frame/s]
Rendering:  68%|██████▊   | 259/380 [00:16<00:04, 25.79frame/s]
Rendering:  69%|██████▉   | 263/380 [00:16<00:09, 11.96frame/s]
Rendering:  70%|███████   | 267/380 [00:17<00:07, 15.02frame/s]
Rendering:  71%|███████▏  | 271/380 [00:17<00:05, 18.28frame/s]
Rendering:  72%|███████▏  | 275/380 [00:17<00:04, 21.75frame/s]
Rendering:  73%|███████▎  | 279/380 [00:17<00:04, 25.14frame/s]
Rendering:  74%|███████▍  | 283/380 [00:18<00:08, 10.95frame/s]FPS dropped below 20: 14
Rendering:  76%|███████▌  | 287/380 [00:18<00:06, 13.69frame/s]
Rendering:  77%|███████▋  | 291/380 [00:18<00:05, 17.03frame/s]
Rendering:  78%|███████▊  | 295/380 [00:18<00:04, 20.52frame/s]
Rendering:  79%|███████▊  | 299/380 [00:18<00:03, 23.88frame/s]
Rendering:  80%|███████▉  | 303/380 [00:19<00:06, 11.47frame/s]
Rendering:  81%|████████  | 307/380 [00:19<00:05, 14.28frame/s]
Rendering:  82%|████████▏ | 311/380 [00:19<00:03, 17.62frame/s]
Rendering:  83%|████████▎ | 315/380 [00:19<00:03, 21.03frame/s]
Rendering:  84%|████████▍ | 319/380 [00:19<00:02, 23.73frame/s]
Rendering:  85%|████████▌ | 323/380 [00:20<00:04, 11.88frame/s]Render progress: 323/380 frames, avg 15.71 fps, recent 15.34 fps, route 102.10s
Rendering:  86%|████████▌ | 327/380 [00:20<00:03, 14.75frame/s]
Rendering:  87%|████████▋ | 331/380 [00:20<00:02, 17.70frame/s]
Rendering:  88%|████████▊ | 334/380 [00:20<00:02, 18.08frame/s]
Rendering:  89%|████████▉ | 338/380 [00:21<00:01, 21.39frame/s]
Rendering:  90%|█████████ | 342/380 [00:21<00:01, 24.58frame/s]
Rendering:  91%|█████████ | 346/380 [00:21<00:02, 12.64frame/s]
Rendering:  92%|█████████▏| 350/380 [00:21<00:01, 15.82frame/s]
Rendering:  93%|█████████▎| 354/380 [00:22<00:01, 19.27frame/s]
Rendering:  94%|█████████▍| 358/380 [00:22<00:00, 22.64frame/s]
Rendering:  95%|█████████▌| 362/380 [00:22<00:00, 25.56frame/s]
Rendering:  96%|█████████▋| 366/380 [00:22<00:01, 12.29frame/s]
Rendering:  97%|█████████▋| 370/380 [00:23<00:00, 15.46frame/s]
Rendering:  98%|█████████▊| 374/380 [00:23<00:00, 18.81frame/s]
Rendering:  99%|█████████▉| 378/380 [00:23<00:00, 22.09frame/s]Render progress: 380/380 frames, avg 16.26 fps, recent 20.36 fps, route 104.95s
Rendering: 100%|██████████| 380/380 [00:23<00:00, 16.26frame/s]
2026-03-28 02:05:41,816 INFO	Clip saved to: /src/shared/cog-clip.mp4
Render stats: frames=380, render_seconds=23.38, avg_fps=16.25
2026-03-28 02:05:41,817 INFO	Generated 19s in 27.3s (0.7x realtime) | import=0.00s, logs=1.17s, setup=2.04s, render=23.38s (1x), ffmpeg=0.66s
+ ffmpeg -y -ss 3 -i /src/shared/cog-clip.mp4 -c copy -movflags +faststart /src/shared/cog-clip.tmp.mp4
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/src/shared/cog-clip.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
encoder         : Lavf58.76.100
Duration: 00:00:18.00, start: 0.000000, bitrate: 3494 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 2160x2430 [SAR 1:1 DAR 8:9], 3493 kb/s, 20 fps, 20 tbr, 10240 tbn, 40 tbc (default)
Metadata:
handler_name    : VideoHandler
vendor_id       : [0][0][0][0]
Output #0, mp4, to '/src/shared/cog-clip.tmp.mp4':
Metadata:
major_brand     : isom
minor_version   : 512
compatible_brands: isomiso2avc1mp41
encoder         : Lavf58.76.100
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 2160x2430 [SAR 1:1 DAR 8:9], q=2-31, 3493 kb/s, 20 fps, 20 tbr, 10240 tbn, 10240 tbc (default)
Metadata:
handler_name    : VideoHandler
vendor_id       : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=-1.0 size=       0kB time=-00:00:03.14 bitrate=N/A speed=N/A
[mp4 @ 0x5c21f7354640] Starting second pass: moving the moov atom to the beginning of the file
frame=  360 fps=0.0 q=-1.0 Lsize=    7679kB time=00:00:14.80 bitrate=4250.3kbits/s speed=1.5e+03x
video:7675kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.048936%

Version Details

Version ID: 798dea4cb2795a324b5387a0b63430a36f06d6893c8a9847308dc4570712e3fa
Version Created: April 15, 2026

Run on Replicate →