Tools for the ffmpeg_image_transport

This repository hosts code for handling data streams produced by the ffmpeg_image_transport.

Supported systems

Continuous integration is tested under Ubuntu with the following ROS2 distros:

How to install

From packages

sudo apt-get install ros-${ROS_DISTRO}-ffmpeg-image-transport-tools

From source

Set the following shell variables:

repo=ffmpeg_image_transport_tools
url=https://github.com/ros-misc-utilities/${repo}.git

and follow the instructions here.

About encoders, decoders, and pixel formats

The tools in this package often have many parameters related to encoders, decoders, and pixel formats. It is important to understand the terminology, and how single-channel images (like mono8 and Bayer images) are handled. Please consult the ffmpeg_encoder_decoder repo for more information.

Programs for processing rosbags

bag_to_file

The bag_to_file tool concatenates the FFMPEGPacket message content from a rosbag into a file, and runs the ffmpeg tool to embed the stream into a an mp4 container.

bag_to_file -i input_bag -t topic -r rate [-o out_file] [-T timestamp_file] [-s start_time (sec since epoch)] [-e end_time (sec since epoch)]

The rate determines the fps used by ffmpeg when producing the output. The video is not transcoded, so what you get is the original stream, just in an mp4 container. To get correct playing speed the rate must match the rate at which the stream was originally recorded.

For example the following line produces a file video.mp4 and timestamps.txt from a rosbag:

ros2 run ffmpeg_image_transport_tools bag_to_file -t /cam1/image_raw/ffmpeg -r 40 -i ./my_rosbag/ -e 1710085164.466

The timestamp.txt file facilitates correlating ROS time stamps with frame numbers. It has the following entries:

# packet no, pts, header_stamp  recording_stamp
0 0 1710085154473057750 1710085156001866724
1 1 1710085155950467594 1710085156024209913

A H264 packet typically corresponds to a frame so the packet number should conincide with the frame number.

bag_to_frames

The bag_to_frames program decodes the ffmpeg-generated packets from a rosbag into frames:

bag_to_frames -i input_bag -t topic [options]

options:
 -o out_dir         name of the output directory (defaults to "frames")
 -d decoder         name of the libav decoder (hevc_cuvid, libx264 etc)
 -O output_format   ros encoding ('bgr8', 'mono', ...) to convert to before writing image.
 -f file_type       frame file type ('png', 'jpeg'). Defaults to jpeg.
 -T timestamp_file  name of time stamp file.
 -s start_time      time in sec since epoch.
 -e end_time        time in sec since epoch.

If not specified, the decoder will be automatically (not necessarily correctly) picked. Not all decoders for a codec can handle all encoded image formats. For example if the codec is hevc and a bayer image (bayer_rggb8) has been encoded with image format nv12, then the h265 will throw the following error:

[INFO] [1753428586.070137592] [bag_to_frames]: using decoder: hevc
[WARN] [1753428586.406598438] [Decoder]: hardware frame transfer failed for pixel format yuv420p

Before the image is written a final conversion to the output_format is performed. The default is bgr8. The format string must follow ROS convention rather than the libav convention, i.e. bgr8 rather than bgr24.

compress_bag

Use ros2 run ffmpeg_image_transport_tools compress_bag to encode a video stored as Image messages into FFMPEGPacket format. Usage is as follows:

``` compress_bag -i in_bag -o out_bag -t topic [-t topic … ] [options] options: -q enable quality check -I write debug images -m max_num_frames_to_keep (for matching encoded/decoded packets. defaults to 100) -s start_time [in sec since epoch] -e end_time [in sec since epoch] ——- encoder options ——- -E encoding options: encoder: (defaults to libx264) cv_bridge_target_format: (defaults to rgb8)

File truncated at 100 lines see the full file

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).