Analog Devices 3DToF Image Stitching

Overview

The ADI 3DToF Image Stitching is a ROS (Robot Operating System) package for stitching depth images from multiple Time-Of-Flight sensors like ADI’s ADTF3175D ToF sensor. This node subscribes to captured Depth and IR images from multiple ADI 3DToF ADTF31xx nodes, stitches them to create a single expanded field of view and publishes the stitched Depth, IR and PointCloud as ROS topics. The node publishes Stitched Depth and IR Images at 2048x512 (16 bits per image) resolution @ 10FPS in realtime mode on AAEON BOXER-8250AI, while stitching inputs from 4 different EVAL-ADTF3175D and giving an expanded FOV of 278 Degrees. Along with the Stitched Depth and IR frames, the Stitched Point Cloud is also published at 10FPS.

Background

• Supported Time-of-flight boards: ADTF3175D
• Supported ROS and OS distro: Humble (Ubuntu 22.04 and Ubuntu 20.04(Source Build))
• Supported platform: armV8 64-bit (arm64) and Intel Core x86_64(amd64) processors(Core i3, Core i5 and Core i7)

Hardware

For the tested setup with GPU inference support, the following are used:

• 4 x EVAL-ADTF3175D
• 1 x AAEON BOXER-8250AI
• 1 x External 12V power supply
• 4 x Usb type-c to type-A cables - with 5gbps data speed support

Minimum requirements for a test setup on host laptop/computer CPU:

• 2 x EVAL-ADTF3175D
• Host laptop with intel i5 or higher cpu running Ubuntu-20.04LTS or WSL2 with Ubuntu-20.04
• 2 x USB type-c to type-A cables - with 5gbps data speed support
• USB power hub

:memo: _Note: Refer the User Guide to ensure the eval module has adequate power supply during operation.

The image below shows the connection diagram of the setup (with labels):

The image below shows the actual setup used (for reference):

Hardware setup

Follow the below mentioned steps to get the hardware setup ready:

1. Setup the ToF devices with adi_3dtof_adtf31xx_sensor node following the steps mentioned in the repository.
2. Ensure all devices are running at 10fps by following these steps.
3. Position the cameras properly as per the dimentions shown in the below Diagram.

:memo: Notes:

• Please contact the Maintainers to get the CAD design for the baseplate setup shown above.

1. Finally, connect the devices to the host(Linux PC or AAEON BOXER-8250AI) using usb cables as shown below.

Software

Software Dependencies

Assumptions before building this package:

• Linux System or WSL2(Only Simulation Mode supported) running Ubuntu 22.04LTS
• ROS2 Humble: If not installed, follow these steps.
• Setup colcon workspace (with workspace folder named as “ros2_ws”).
• System Date/Time is updated: Make sure the Date/Time is set properly before compiling and running the application. Connecting to a WiFi network would make sure the Date/Time is set properly.
• NTP server is setup for device Synchronization. If not, please refer this Link to setup NTP server on Host.

Software Requirements for Running on AAEON BOXER-8250AI :

• Nvidia Jetpack OS 5.0.2 .
• CUDA 11.4. It comes preinstalled with Jetpack OS. If not, follow these steps.

Clone

1. Clone the repo and checkout the correct release branch/ tag into ros2 workspace directory

    $ cd ~/ros2_ws/src
    $ git clone https://github.com/analogdevicesinc/adi_3dtof_image_stitching.git -b v2.0.0
    

Build

Do proper exports first:

$ source /opt/ros/<ROS version>/setup.bash

Where:

• “ROS version” is the user’s actual ROS version

Then:

For setup with GPU inference support using CUDA:

$ cd ~/ros2_ws/
$ colcon build --cmake-target clean
$ colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release -DENABLE_GPU=True --event-handlers console_direct+
$ source install/setup.bash

For setup with CPU inference support using openMP:

```bash $ cd ~/ros2_ws/

File truncated at 100 lines see the full file