---
config:
theme: redux
layout: dagre
---
flowchart TD
subgraph INPUT["Input"]
HAND_TRACKING_CAMERA["Hand Tracking Camera"]
SLAM_CAMERA["SLAM Camera (Self location)"]
EYE_TRACKING_CAMERA["Eye tracking Camera"]
IMU["Inertia MU"]
end
subgraph OUTPUT["Output"]
MIPI_DISPLAY["MIPI DSI Displays"]
SOUND_CARD["Sound Card"]
end
subgraph HARDWARE["Hardware"]
CM5["RPi CM5"]
PMIC["PMIC (Power management)"]
INPUT
OUTPUT
end
subgraph XR_SYSTEM["XR System"]
drivers["Low-Level Drivers"]
drm["Linux DRM"]
OPEN_XR_RUNTIME["OPEN_XR_RUNTIME"]
apps["XR applications"]
end
subgraph DRIVERS["OpenXR Drivers"]
display_drivers["Display Drivers"]
hand_tracking_drivers["Hand Tracking Drivers"]
slam_drivers["SLAM Drivers"]
end
subgraph OPEN_XR_RUNTIME["Custom OpenXR Runtime"]
DRIVERS
Compositor["Compositor"]
OpenXR_ST["OpenXR State Tracker"]
InteractionEngine["Interaction Engine (optional)"]
end
subgraph LAUNCHER["Custom Launcher"]
Launcherapp["Launcher Application"]
WebXR["WebXR Runtime (chromium)"]
WebXRapps["WebXR Applications"]
end
CM5 -- output MIPI DSI --> MIPI_DISPLAY
CM5 -- I2S (optional) --> SOUND_CARD
PMIC <== I2C ==> CM5
IMU -- SPI --> CM5
HAND_TRACKING_CAMERA -- USB --> CM5
SLAM_CAMERA -- USB --> CM5
EYE_TRACKING_CAMERA -- optional input --> CM5
drivers --> HARDWARE
drm --> drivers
apps -- OpenXR --> OPEN_XR_RUNTIME
display_drivers --> drm
hand_tracking_drivers --> drivers
slam_drivers --> drivers
Compositor --> DRIVERS
OpenXR_ST --> DRIVERS
InteractionEngine --> DRIVERS
WebXR -- OpenXR --> OPEN_XR_RUNTIME
WebXRapps --> WebXR
Launcherapp -- OpenXR --> OPEN_XR_RUNTIME
Launcherapp --> WebXRapps
HAND_TRACKING_CAMERA@{ shape: rect}
SLAM_CAMERA@{ shape: rect}
Subproject: Standalone Launcher
Subproject: V35ZSM Panel Driver
Project Northstar is an open-source mixed-reality headset originally developed by Leap Motion. As an open-source project, it remains relatively impressive with a great field-of-view, high resolution, and a straightforward setup.
Currently, the Northstar headset requires linking to external hardware (e.g. a PC) to be functional. Though this had been a common requirement in older headsets, newer headsets like the Meta Quest 3 have dropped this setup and have become standalone. That is, they no longer require any external hardware other than the headset itself.
So, at UWRL, we’ve decided to take it a step further and ask: How can we make Project Northstar standalone? Though seemingly a straightforward task, this has become an involved project requiring effort in both hardware design and software development.
Overall, completion will involve (from lowest to highest abstraction level):
3.5 inch screen that has been used on the northstar for a long time, also used on the valve index. resolution is 1440x1600 at 90hz, 8 bit color and it’s a video mode display. It’ll need sync packets and video packet types.
VS035ZSM display datasheet
Specification of the physical interface used for MIPI, the CM5 should implement this specification given that its compliant with the MIPI alliance specifications.
MIPI D-PHY specification
MIPI Alliance Specification for D-PHY Version 2.5 (MIPI Alliance)[2019][—] (Z-Library).pdf
Very useful doc to see how IO has been broken out.
CM5 IO breakout board datasheet