ROS2 image transport for ffmpeg/libav

This ROS2 image transport plugin supports encoding/decoding with the FFMpeg library, for example encoding h264 and h265/hevc, using Nvidia or other hardware acceleration when available.

The publisher plugin of the transport produces ffmpeg image transport messages. These are raw, encoded packets that are then transmitted and decoded by the subscriber plugin of the transport. The transport library contains both the publisher(encoder) and subscriber(decoder) plugin and therefore must be installed on both sides to be useful.

To extract e.g. frames or an mp4 file from a recorded bag, have a look at the ffmpeg_image_transport_tools repository.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

Installation

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport

From source

Set the following shell variables:

repo=ffmpeg_image_transport
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here

Make sure to source your workspace’s install/setup.bash afterwards. If all goes well you should see the transport show up:

ros2 run image_transport list_transports

should give output (among other transport plugins):

"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frames into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frames from ffmpeg compressed packets

Remember to install the plugin on both hosts, the one that is publishing and the one that is subscribing (viewing).

Parameters

Below is a short description of the ROS parameters exposed by the Publisher and Subscriber plugins. The parameters refer to encoder and decoder variables described in more detail in the ffmpeg_encoder_decoder repository.

Publisher (camera driver)

Here is a list of the available encoding parameters:

• encoder: the libav (ffmpeg) encoder being used. The default is libx264, which is on-CPU unaccelerated encoding. Depending on your hardware, your encoding options may include the hardware accelerated h264_nvenc or h264_vaapi. You can list all available encoders with ffmpeg --codecs. In the h264 row, look for (encoders).
• encoder_av_options: default is empty (“”). Comma-separeted list of valid libav options in the form of (key:value), e.g.:'preset:ll,profile:main,crf:0'. See the ffmpeg documentation for more.
• gop_size: The number of frames between keyframes. Default: 10. The larger this number the more latency you will have, but also the more efficient the compression becomes.
• bit_rate: The max bit rate [in bits/s] that the encoding will target. Default is 8242880.
• pixel_format: Forces a different pixel for encoding. This pixel format corresponds to the av_source_pixel_format in the ffmpeg_encoder_decoder documentation.
• qmax: Max quantization rate. Defaults to 10. See ffmpeg documentation. The larger this number, the worse the image looks, and the more efficient the encoding.
• encoder_measure_performance: For performance debugging (developers only). Defaults to false.

The parameters are under the ffmpeg variable block. If you launch your publisher node (camera driver), you can give it a parameter list on the way like so:

        parameters=[{'image_raw.ffmpeg.encoder': 'hevc_nvenc',
                     'image_raw.ffmpeg.encoder_av_options': 'preset:ll,profile:main,crf:0'}]

See the example launch file for a V4L USB camera (usb_camera.launch.py). If the above parameter settings don’t work, use the ros2 param dump <your_node_name> command to find out what the proper parameter path is.

File truncated at 100 lines see the full file

Changelog for package ffmpeg_image_transport

3.0.0 (2025-08-08)

• support falling back to alternative decoders, rename parameters, add tests etc
  • adopt the new ffmpeg_encoder_api which allows for probing of decoders. The decoder parameter can now contain a list of comma-separated decoders which will be tried in order.
  • adds gtests to the repo
  • reformat to the black python formatter
  • add arguments to some example launch files
  • allow arbitrary AVOptions setting via av_option
  • remove parameters "preset", "tune", "delay", and "crf" (must now be set via "av_options")
  • remove some default values (like bit_rate) to force user to set them explicitly
  • change the parameter names: remove the leading ".", so now the parameters are named specify "camera.image_raw.." as opposed to ".camera.image_raw..."
  • adapt to new ffmpeg_encoder_decoder API
  • change name of parameter from encoding->encoder
  • provide encoding when initializing encoder (new encoder/decoder API)
  • handle new API for image transport 6.3.0
  • deal with Humble bug: topic is passed in without being prefixed by the namespace, but then namespace is removed!
  • package splitting functions into utilities file
• stop building on foxy but still support humble
• Replaced deprecated code
• Contributors: Alejandro Hernandez Cordero, Bernd Pfrommer

2.0.3 (2025-05-26)

• avoid ament_target_dependencies
• Contributors: Bernd Pfrommer

2.0.2 (2025-03-30)

• fix bug: segfault when publish function pointer changes
• updated broken badge, fixed typo in readme
• Contributors: Bernd Pfrommer

2.0.1 (2025-03-17)

• fix bug that prevents compilation in humble
• Contributors: Bernd Pfrommer

2.0.0 (2025-03-15)

• use ffmpeg_encoder_decoder
• align parameter handling with compressed image transport
• point to new instructions
• Contributors: Bernd Pfrommer

1.0.2 (2025-03-07)

• updated badges and fixed deprecation warnings (#36)

  • updated badges and fixed deprecation warnings
  • only use AV_FRAME_FLAG_KEY when available

• Configurable CRF (#34)

  • Added CRF support

  * fixed gop parameter in the README examples ---------Co-authored-by: Alexey Shtern <<alexey.shtern@xtend.me>>

• README: Add usage instructions for Jazzy The syntax mentioned in the README no longer works there.

• fix typo in link

• added documentation for enabling NVMPI on the jetson

• Contributors: Alexey Shtern, Bernd Pfrommer, Danil Tolkachev, Michal Sojka

1.0.1 (2024-04-17)

• use appropriate header for cv_bridge
• added documentation and improved cmake exporting
• move encoder->decoder map to decoder for public use
• added frame delay control
• changed install directory so other pkgs can ament_target_depend on this library
• Contributors: Bernd Pfrommer, Toby Buckley

1.0.0 (2024-01-11)

• initial release of ROS2 package
• Contributors: Akshay Srinivasan, Bernd Pfrommer, Francesco Ferroni, Michal Sojka, akssri