Monado contract with application (openXR api) and devices:
| Input | What it contains | Static or streamed |
|---|---|---|
| Camera frames | Pixel data, format, timestamp, sequence number, per camera (including eye cameras) | Streamed |
| IMU samples | Acceleration and angular velocity, with timestamps | Streamed |
| Camera intrinsics | Focal lengths, principal point, lens distortion model and coefficients, resolution | Static (calibration file) |
| Camera extrinsics | Each camera's fixed transform relative to the IMU or headset origin; camera-to-IMU time offset | Static (calibration file) |
| IMU calibration | Noise densities, bias random walk, optionally scale and axis misalignment | Static (calibration file) |
| Display description | Per-eye resolution, refresh rate, field of view, distortion model | Static (calibration file) |
| Head-to-eye transforms | Fixed transforms from the tracking origin to each eye's view, plus IPD | Static, or per-user |
| Display timing | Vblank and present timestamps | Streamed |
| Eye-tracking calibration | Per-user mapping from eye images to gaze | Static, per user |
| Data | Interface | Device nodes / location | Notes |
|---|---|---|---|
| Camera frames | V4L2 streaming I/O | /dev/videoN |
Buffers by mmap, or exported as dma-buf for GPU import. Each buffer carries a timestamp and sequence number. |
| Camera pipeline setup | Media controller + V4L2 sub-devices | /dev/mediaN, /dev/v4l-subdevN |
Links and per-pad formats. Likely done once by a setup script rather than by the runtime. |
| Camera controls (exposure, gain, frame rate) | V4L2 controls on sensor sub-devices | /dev/v4l-subdevN |
For cameras behind the FPGA, the control path is open. It may pass through the FPGA driver instead. |
| Per-frame hardware metadata (e.g., FPGA timestamps) | V4L2 metadata capture node, or data embedded in the image | /dev/videoN (metadata type) |
Depends on the hardware lead's design. Needs to be specified. |
| IMU samples | IIO buffered capture | /dev/iio:deviceN for samples; /sys/bus/iio/devices/… for scale, rate, channel enables, timestamp clock |
If the IMU ends up behind the FPGA, this becomes whatever that link exposes. |
| Display output and timing | DRM/KMS: atomic commits, page-flip and vblank events | /dev/dri/cardN |
Usually reached through Vulkan's display extensions (VK_KHR_display, VK_EXT_acquire_drm_display) rather than raw ioctls. |
| GPU compute and rendering | Vulkan (Turnip) on the DRM render node | /dev/dri/renderD128 |
Imports camera dma-bufs for the GPU SLAM front end. |
| DSP/NPU offload (tentative) | FastRPC via the Hexagon runtime libraries | /dev/fastrpc-* |
Depends on the toolchain question. |
| Eye-tracking IR illuminators | Unknown: GPIO, LED class, or FPGA | — | Only if not fully handled by the FPGA strobe. |
| Buttons, proximity sensor (if present) | evdev | /dev/input/eventN |
Optional. |
| Calibration and display description | Files | Monado config directory (JSON) | Covers intrinsics, extrinsics, IMU noise, distortion, and head-to-eye transforms. |
| Time base | clock_gettime(CLOCK_MONOTONIC) |
— | Every timestamp above must be in, or convertible to, this clock. |