scan_detection_fusion

A reusable ROS 2 library that fuses 2D camera object detections with a planar LiDAR scan to produce range-resolved, semantically labeled obstacles and class-aware footprint polygons for navigation.

A monocular detector knows what an object is and roughly which direction it lies in, but not how far away it is. A 2D LiDAR knows precise distance at every bearing, but nothing about what it is seeing. This package marries the two: it matches each detection’s bearing to the LiDAR returns at that bearing, estimates a robust range, projects the result into the map frame, and publishes a labeled object together with a reconstructed ground footprint.

The fusion algorithm is implemented as a plain, ROS-free Python class (LidarCameraFuser) wrapped by a thin ROS 2 node, so the math can be reused and unit-tested independently of the ROS graph.

Status

This is an open-source ROS 2 package targeting the Jazzy distribution. It is standards-based: detections arrive as vision_msgs/Detection2DArray, camera geometry is read from sensor_msgs/CameraInfo, and sensor placement is taken from the TF tree, so it is not tied to any particular detector or robot.

What it produces

For each detection the node publishes a fused object carrying its class label, estimated range, bearing, map-frame position, and the angular span it subtended. On top of the baseline fusion, two contributions sharpen the output:

Semantic zone routing. LiDAR sampling is directed to the vertical region of the bounding box where a given class actually intersects the scan plane, rather than sampling the whole box, so the range estimate reflects the physical object and not the background behind it.

Class-aware footprint reconstruction. Each object is given a full W × D ground polygon (published as geometry_msgs/PolygonStamped for a Nav2 obstacle layer) from a per-class size table. When the observed LiDAR arc implies a wider object than the table assumes, the width is refined from the measured angular span using

W_obs = 2 · d̂ · tan( (θ_right − θ_left) / 2 )

where d̂ is the estimated range and (θ_right − θ_left) is the detection’s angular span. The polygon is oriented so its width spans perpendicular to the line of sight and its depth along it.

Installation

From the ROS index (after release)

Once released into the Jazzy distribution this package will be installable with:

sudo apt install ros-jazzy-scan-detection-fusion

From source (current)

cd ~/ros2_ws/src
git clone https://github.com/HexboxRC/scan_detection_fusion.git
cd ~/ros2_ws
rosdep install --from-paths src --ignore-src -r -y
colcon build --packages-select scan_detection_fusion
source install/setup.bash

Running the node

ros2 run scan_detection_fusion fuser_node

The following must already be publishing on the graph:

Every topic name is a parameter, so the node remaps onto an existing graph without code changes:

ros2 run scan_detection_fusion fuser_node --ros-args \
  -p topic_scan:=/lidar/scan -p topic_detections:=/yolo/detections

Interface

Subscribed

File truncated at 100 lines see the full file