GStreamer ROS 2 Babel Fish

This project provides GStreamer elements for bridging ROS 2 image topics with GStreamer pipelines. It allows for high-performance streaming of image data between ROS 2 and GStreamer, supporting both raw and compressed image formats.

For a detailed understanding of the system’s design and component interactions, please refer to the Architecture Overview.

Features

• rbfimagesrc: GStreamer source element that subscribes to ROS 2 image topics.
  • Supports sensor_msgs/msg/Image (raw) and sensor_msgs/msg/CompressedImage.
  • Automatically handles format conversion.
  • Configurable QoS settings.
• rbfimagesink: GStreamer sink element that publishes to ROS 2 image topics.
  • Supports publishing sensor_msgs/msg/Image (raw).
  • Supports publishing sensor_msgs/msg/CompressedImage (e.g., JPEG, PNG) if input caps match.
  • Zero-copy publishing where possible (using shared memory / loaned messages if supported).

Installation

This package is a ROS 2 package. You can build it using colcon.

1. Create a ROS 2 workspace (if you haven’t already):

    mkdir -p ~/ros2_ws/src
    cd ~/ros2_ws/src
    

1. Clone the repository:

    git clone https://github.com/StefanFabian/gstreamer_ros_babel_fish
    

1. Install dependencies using rosdep:

    cd ~/ros2_ws
    rosdep update
    rosdep install --from-paths src --ignore-src -r -y
    

1. Build the package:

    colcon build --packages-select gstreamer_ros_babel_fish
    

1. Source the workspace:

    source install/setup.bash
    

Usage

After sourcing the workspace, the GStreamer elements rbfimagesrc and rbfimagesink should be available. You can verify this with:

gst-inspect-1.0 rbfimagesrc
gst-inspect-1.0 rbfimagesink

Example Pipelines

1. Display a ROS Image Topic

Subscribe to a ROS topic /camera/image_raw and display it using autovideosink.

gst-launch-1.0 rbfimagesrc topic=/camera/image_raw ! videoconvert ! autovideosink

2. Stream Video to ROS

Generate a test video pattern and publish it to /test_video.

gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! rbfimagesink topic=/test_video sync=true

3. Publish Compressed Images

Encode video as JPEG and publish to /compressed_video/compressed.

gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! jpegenc ! rbfimagesink topic=/compressed_video sync=true

[!NOTE] When receiving compressed data (image/jpeg, image/png), rbfimagesink automatically publishes to <topic>/compressed.

4. Roundtrip (ROS -> GStreamer -> ROS)

Subscribe to /input_topic, process it (e.g., flip), and publish to /output_topic.

gst-launch-1.0 rbfimagesrc topic=/input_topic ! videoflip method=clockwise ! rbfimagesink topic=/output_topic

5. Using with Compressed Input

Subscribe to a compressed topic /compressed_video/compressed (JPEG), decode it, and display.

File truncated at 100 lines see the full file