ROS2 image transport for ffmpeg/libav

This ROS2 image transport plugin supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265 or HEVC, using Nvidia or other hardware acceleration when available.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is encoding and the one that is decoding (viewing).

Parameters

Publisher (camera driver)

Here is a list of the available encoding parameters:

• encoding: the libav (ffmpeg) encoder being used. The default is libx264, which is on-CPU unaccelerated encoding. Depending on your hardware, your encoding options may include the hardware accelerated h264_nvenc or h264_vaapi. You can list all available encoders with ffmpeg --codecs. In the h264 row, look for (encoders).
• preset: default is empty (“”). Valid values can be for instance slow, ll (low latency) etc. To find out what presets are available, run e.g. ffmpeg -hide_banner -f lavfi -i nullsrc -c:v libx264 -preset help -f mp4 - 2>&1
• profile: For instance baseline, main. See the ffmpeg website.
• tune: See the ffmpeg website. The default is empty(“”).
• gop_size: The number of frames between keyframes. Default: 10. The larger this number the more latency you will have, but also the more efficient the compression becomes.
• bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is 8242880.
• crf: Constant Rate Factor, affects the image quality. Value range is [0, 51]; 0 is lossless, 23 is default, 51 is worst quality.
• delay: Not sure what it does, but doesn’t help with delay. Default is empty (“”).
• pixel_format: Forces a different pixel format for internal conversions. Experimental, don’t use.
• qmax: Max quantization rate. Defaults to 10. See ffmpeg documentation. The larger this number, the worse the image looks, and the more efficient the encoding.
• measure_performance: For performance debugging (developers only). Defaults to false.
• performance_interval: How many frames to wait between logging performance data.

The parameters are under the ffmpeg variable block. If you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'ffmpeg_image_transport.encoding': 'hevc_nvenc',
                     'ffmpeg_image_transport.profile': 'main',
                     'ffmpeg_image_transport.preset': 'll',
                     'ffmpeg_image_transport.gop_size': 15}]

See the example launch file for a V4L USB camera

Subscriber (viewer)

File truncated at 100 lines see the full file

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).