Event-camera + LiDAR sensor fusion for sub-100ms obstacle detection

A sensor-fusion system for mobile robots operating in mixed warehouse environments, comprising a high-speed event camera [src-1], a solid-state LiDAR depth unit [src-2], and a purpose-built alignment kernel that reconciles the two fundamentally dissimilar data streams into a coherent obstacle field at an effective rate of 240Hz.

The novelty does not reside in any one sensor. Event cameras and solid-state LiDAR are each commercially mature. It resides in the alignment kernel — a hardware-software boundary that timestamps pixel events against discrete LiDAR frames to within 8ms [src-3]  [founder intake Q7], and an accompanying reactive controller that consumes the fused field directly, without a frame-based intermediate representation.

The practical result is end-to-end sensor-photon-to-avoidance-actuation latency below 100ms — 42ms at the median and 78ms at the 99th percentile  [uploaded benchmark report, p.14] — driving the avoidance controller at an effective 240Hz [src-4] (roughly 10× the usable frame rate of a conventional frame-based vision stack) with field-tuned suppression of fluorescent-flicker event-rate overload [src-5], and fast enough to bring an autonomous mobile robot to a safe stop within the stopping distance required by OSHA 1910.178  [regulatory note, attorney-supplied] for mixed human/robot aisles.

The event camera produces asynchronous pixel events — tuples of (x, y, polarity, timestamp) — at sub-millisecond latency, with a native rate that scales with scene motion rather than a fixed frame clock. Simultaneously, the solid-state LiDAR emits a conventional depth frame at 10Hz  [architecture doc §3.2]. The alignment kernel, running on a dedicated FPGA fabric, maintains a rolling 12ms buffer of events keyed on hardware-clock timestamps  [internal whitepaper §2] and projects each event into the nearest LiDAR frame’s voxel grid using a learned epipolar correspondence.

Once aligned, each incoming event carries a depth estimate within ±4cm at 5m range  [uploaded benchmark report, p.14]. This augmented event stream is consumed directly by a reactive avoidance controller — a purely reactive module, no map — that evaluates a commanded trajectory against the evolving obstacle field every 4.2ms on average  [internal whitepaper §3]. When the event camera saturates (luminance transients, direct sunlight at loading-dock doors), the controller falls back to LiDAR-only at 10Hz with reduced commanded speed, gated by a hysteresis threshold to prevent oscillation  [founder intake Q15].

The prevailing trajectory in robotics perception over the past five years has been to increase frame-based camera rates — 120fps, then 240fps global-shutter CMOS  [industry survey, founder reference] — to close the latency gap. The present invention instead abandons the frame as the fundamental unit of vision, treating the event stream as primary and the LiDAR frame as a secondary correction signal. A practitioner of ordinary skill would not have been motivated to invert the relationship: event cameras are widely perceived as supplementary sensors, useful for high-speed specialty applications (ball tracking, spark inspection) rather than as the backbone of warehouse perception.

Further, the alignment kernel operates on heterogeneous timebases that were previously considered incompatible at production scale. The closest analogues in the literature ([ref-3], [ref-4]) synchronize event streams against conventional CMOS video, which shares a global shutter; alignment against a rolling-acquired LiDAR depth frame is materially harder, and the ±4cm accuracy we achieve  [uploaded benchmark report, p.14] is not anticipated by any single reference.

• Saturation-fallback switchover thresholds (claim element 5) appear only in the CES demo brief; no ELN bench log covers the fallback path.