Skip to main content

​
Camera calibration

Operator workflow for chessboard targets and dimos cameracalibrate (ROS-style CameraInfo YAML). The square size you pass to the CLI must match the board you actually print and measure. dimos cameracalibrate uses --cols and --rows as the number of inner corner points along each axis, the same convention as cv2.findChessboardCorners patternSize=(cols, rows). A practical default is 8 by 6 inner corners on A4: enough intersections for a stable solve without making each square too small for a typical desk webcam.
  1. Generate a checkerboard. OpenCV documents pattern creation in Create calibration pattern. To draw your own board, download and run OpenCV’s gen_pattern.py from that OpenCV version (see the tutorial for dependencies). In that script, --columns and --rows count checker squares along each axis. For 8 by 6 inner corners, use 9 columns and 7 rows of squares (inner corners are one less than square count in each direction):
python gen_pattern.py -o chessboard_a4.svg -T checkerboard --columns 9 --rows 7 --square_size 25 -u mm -a A4
Tune --square_size so the pattern fits with margins; convert SVG to PDF in your viewer if needed.
  1. Print at nominal scale. Turn off “fit to page” or other scaling that would change the printed square size relative to the file.
  2. Measure one printed square with calipers. Use the edge length of a single black or white square on the printed sheet, not the value from the generator unless you verified the print. Convert to meters for --square-size-m (for example 24.85 mm becomes 0.02485).

​
Capture practice

  • Aim for 15-25 frames with the board fully in view and inner corners detected in each; keep a few spares so you can drop outliers.
  • Cover the full image over the set: include poses where the board reaches toward the frame edges and corners, not only the center.
  • Vary tilt and camera-to-board distance between frames so the solver sees diverse rigid poses.
  • Lock exposure and use fixed white balance when the camera or capture app allows it, so brightness does not drift across the sequence.
  • Avoid motion blur: mount or brace the camera, use enough light, and only save frames when the preview is sharp (same for stills saved to a folder).
Example after you have calibration images in ./capture: 
uv run dimos cameracalibrate --source folder --images ./capture --cols 8 --rows 6 --square-size-m 0.02485 --out ./camera_info.yaml ./camera_info.preview.png
Wrong --square-size-m skews metric geometry even when reprojection error looks good.

​
Run dimos cameracalibrate

Run from the repo or any directory where uv run dimos resolves (same pattern as other dimos CLIs). Required flags are always --source, --cols, --rows, and --square-size-m. Folder mode also requires --images. Webcam (interactive). Open a live preview on device --device-index. When inner corners are detected, the board is drawn on the preview; press Space to accept the current frame. The CLI collects --target-count accepted frames (default 20) then solves. Press q to quit early (that aborts unless enough frames were already accepted). The detector first tries --cols and --rows as inner-corner counts, then also accepts the common square-count form (for example a 12 by 8 square board is detected as 11 by 7 inner corners). 
uv run dimos cameracalibrate --source webcam --device-index 0 --cols 8 --rows 6 --square-size-m 0.02485 --out ./camera_info.yaml ./camera_info.preview.png
Folder (stills). Load every *.png, *.jpg, and *.jpeg in the directory, sorted by filename. Each image should show the full board with detectable inner corners. 
uv run dimos cameracalibrate --source folder --images ./capture/ --cols 8 --rows 6 --square-size-m 0.02485 --out ./camera_info.yaml ./camera_info.preview.png
Topic (live pubsub stream). Subscribe to a camera that is already publishing over the dimos pubsub bus — for example a robot blueprint that’s currently running — instead of opening a local webcam device. The same interactive UX applies: press Space to accept a frame, q to quit early, stop after --target-count (default 20). The publisher must emit sensor_msgs.Image; the calibration script normalizes to BGR via Image.to_opencv(). 
uv run dimos cameracalibrate --source topic --topic "jpeg_lcm:/color_image" --cols 8 --rows 6 --square-size-m 0.02485 --out ./camera_info.yaml ./camera_info.preview.png
--topic is a single URI <proto>:<channel> from the pubsub registry (dimos.protocol.pubsub.registry). Pick the proto that matches how the publisher transports the image:
  • lcm:/color_image — standard typed LCM stream.
  • jpeg_lcm:/color_image — Go2 smart blueprint and other JPEG-LCM publishers.
  • pshm:color_image — Go2 basic blueprint (pickled shared memory).
  • shm / jpeg_shm / plcm are also accepted for niche cases.
The optional #<msg_type> URI suffix forwards a fully-qualified message name to the registry (e.g. lcm:/color_image#sensor_msgs.Image); when omitted, the calibration CLI passes sensor_msgs.Image for typed protos. Pickled / self-describing protos (plcm, pshm, …) ignore the type. If the topic stays silent, --topic-timeout-sec (default 60) aborts the run instead of hanging the terminal. Increase it if your publisher is slow to start.

​
Distortion model: pinhole vs fisheye

--distortion-model selects the lens model the solver uses. Default is plumb_bob; pass --distortion-model fisheye for genuine wide-angle / fisheye lenses (e.g. the Go2 front camera).
  • plumb_bob — cv2.calibrateCamera with the 5-coefficient radial-tangential model. Right for near-pinhole lenses (typical webcams, narrow-FOV USB cameras). YAML emits distortion_model: plumb_bob.
  • fisheye — cv2.fisheye.calibrate with the 4-coefficient Kannala-Brandt model. YAML emits distortion_model: equidistant (the ROS-canonical name). Use this whenever the lens has noticeable barrel distortion or HFOV beyond roughly 100°.
How to tell you picked the wrong model: solver “succeeds” but the recovered K and D are nonsense. Plumb-bob fit to a fisheye lens typically produces inflated focal lengths and k coefficients far outside the usual [-0.5, 0.5] range (you’ll see numbers like k1 ≈ -1.6, k2 ≈ 4.7). Fisheye fit to a near-pinhole lens just over-parametrises and behaves similarly. When in doubt, look at the printed focal length vs the lens’s nominal FOV — fx ≈ width / (2 · tan(HFOV/2)) is a useful sanity check. 
uv run dimos cameracalibrate --source topic --topic "lcm:/color_image" \
  --distortion-model fisheye \
  --cols 8 --rows 6 --square-size-m 0.02485 --out ./camera_info.yaml
Downstream consumers that undistort or project points must branch on distortion_model; raw K works for either model, but anything that touches D (e.g. cv2.undistort vs cv2.fisheye.undistortImage) needs to pick the matching OpenCV function. Output files are explicit. Pass --out ./camera_info.yaml to write the ROS CameraInfo YAML. Pass a preview PNG path immediately after it to write a corner-overlay preview, for example --out ./camera_info.yaml ./camera_info.preview.png. If you omit both output paths, the command still runs calibration and prints RMS, but does not write YAML or PNG files. A preview PNG path without --out is rejected. Optional flags (shared across sources): --target-count (webcam/topic; default 20), --camera-name (default webcam), --no-display (no OpenCV window; for headless or automation), --debug (write detailed capture logs to the system temp directory). On success the process prints the calibration RMS, the detected pattern, and any output paths you requested. Example: 
RMS: 0.342187 px (20 frame(s) used)
Detected pattern: (8, 6) (requested inner corners)
Wrote camera info YAML to camera_info.yaml
Wrote preview overlay PNG to camera_info.preview.png
Your RMS and frame count depend on the capture. Paths echo only the files you explicitly requested.

​
Verify the YAML

You can sanity-check an existing CameraInfo YAML without rerunning calibration.
  1. Open the file and confirm image_width and image_height match the resolution your camera actually delivers in the stack (same width and height as your calibration images or as your Webcam width and height settings). Wrong dimensions mean intrinsics are being applied to the wrong raster shape.
  2. Confirm distortion_model is plumb_bob. dimos load_camera_info reads this key (not camera_model); other values are only valid if every consumer in your pipeline agrees on the same model.
  3. frame_id: the YAML written by dimos cameracalibrate does not embed frame_id. When you build a CameraInfo from the file (for example load_camera_info(path, frame_id=...)), the frame_id you pass must match the Image.header.frame_id used for that camera in your graph. The stock Webcam publishes color images with frame id camera_optical (or {frame_id_prefix}/camera_optical if frame_id_prefix is set). The no-robot desk blueprint (ticket T5 in the feature backlog) publishes TF so that optical frame is consistent with that naming; keep your loaded CameraInfo header aligned with whatever your desk stack actually publishes for the optical camera.
For the shape of the ROS-style fields only (matrix blocks, row and column counts, key names), see the checked-in example dimos/hardware/sensors/camera/zed/single_webcam.yaml. Treat it as a schema reference, not as calibration numbers for your camera.