Processing ROS event camera data in python

This repository holds ROS/ROS2 tools for processing event_camera_msgs under ROS and ROS2 with python. These messages are produced by the metavision_driver and the libcaer_driver. For decoding, the event_camera_codecs package is used internally.

With this repository you can quickly load events from a ROS/ROS2 bag into your python code. The decoder will return a structured numpy array of the same format that the Metavision SDK uses:

dtype={'names':['x','y','p','t'], 'formats':['<u2','<u2','i1','<i4'], 'offsets':[0,2,4,8], 'itemsize':12})]

To access e.g. the timestamps (in microseconds) you would use foo['t'], where foo is the numpy array returned by the decoder. See sample code below.

Supported platforms

Continuous integration is tested under Ubuntu with the following ROS2 distros:

No package is released for ROS1, but integration is tested under Ubuntu 20.04 and ROS Noetic.

How to install from packages

Under ROS2 you can install the package via

sudo apt-get install ros-${ROS_DISTRO}-event-camera-py

How to build from source

Set the following shell variables:

repo=event_camera_py
url=https://github.com/ros-event-camera/${repo}.git

and follow the instructions here

Decoding event array messages

Here is a sample decoder for ROS2. It uses the BagReader helper class that you can find in the src folder.

from bag_reader_ros2 import BagReader
from event_camera_py import Decoder

topic = '/event_camera/events'
bag = BagReader('foo', topic)
decoder = Decoder()

while bag.has_next():
    topic, msg, t_rec = bag.read_next()
    decoder.decode(msg)
    cd_events = decoder.get_cd_events()
    print(cd_events)
    trig_events = decoder.get_ext_trig_events()
    print(trig_events)

The following sample code shows how to decode event array messages under ROS1.

import rosbag
from event_camera_py import Decoder

topic = '/event_camera/events'
bag = rosbag.Bag('foo.bag')
decoder = Decoder()

for topic, msg, t in bag.read_messages(topics=topic):
    decoder.decode_bytes(msg.encoding, msg.width, msg.height,
	                     msg.time_base, msg.events)
    cd_events = decoder.get_cd_events()
    print(cd_events)
    trig_events = decoder.get_ext_trig_events()
    print(trig_events)

The returned event arrays are structured numpy ndarrays that are compatible with Prophesee’s Metavision SDK.

About timestamps

A message in a recorded rosbag has three sources of time information:

1. The recording timestamp. This is when the message was written into the bag by the rosbag recorder. It is the least precise of all time stamps and therefore usually not used.

2. The message time stamp in the header (header.stamp). This is the time when the ROS driver host received the first event packet from the SDK for that ROS message. Remember that a ROS message can contain multiple SDK packets, but the header.stamp refers to the first SDK packet received.

3. The sensor time encoded in the packets. This time stamp depends on the encoding.

   • For ‘evt3’ (metavision) encoding the raw packet needs to be decoded to obtain the sensor time. The encoded sensor time has two quirks: it

File truncated at 100 lines see the full file

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).