rosbag2 repository

Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version humble
Last Updated 2024-02-23
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

License GitHub Action Status

Repository for implementing rosbag2 as described in its corresponding design article.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2*

Note that the above command installs all packages related to rosbag2. This also includes the plugin for reading ROS1 bag files, which brings a hard dependency on the ros1_bridge with it and therefore ROS1 packages. If you want to install only the ROS2 related packages for rosbag, please use the following command:

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2-transport

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

[Note]: if you are only building rosbag2 on top of a Debian Installation of ROS2, please git clone the branch following your current ROS2 distribution.

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Simulation time

In ROS 2, "simulation time" refers to publishing a clock value on the /clock topic, instead of using the system clock to tell time. By passing --use-sim-time argument to ros2 bag record, we turn on this option for the recording node. Messages written to the bag will use the latest received value of /clock for the timestamp of the recorded message.

Note: Until the first /clock message is received, the recorder will not write any messages. Before that message is received, the time is 0, which leads to a significant time jump once simulation time begins, making the bag essentially unplayable if messages are written first with time 0 and then time N from /clock.

Splitting recorded bag files

rosbag2 offers the capability to split bag files when they reach a maximum size or after a specified duration. By default rosbag2 will record all data into a single bag file, but this can be changed using the CLI options.

Splitting by size: ros2 bag record -a -b 100000 will split the bag files when they become greater than 100 kilobytes. Note: the batch size's units are in bytes and must be greater than 86016. This option defaults to 0, which means data is written to a single file.

Splitting by time: ros2 bag record -a -d 9000 will split the bag files after a duration of 9000 seconds. This option defaults to 0, which means data is written to a single file.

If both splitting by size and duration are enabled, the bag will split at whichever threshold is reached first.

Recording with compression

By default rosbag2 does not record with compression enabled. However, compression can be specified using the following CLI options.

For example, ros2 bag record -a --compression-mode file --compression-format zstd will record all topics and compress each file using the zstd compressor.

Currently, the only compression-format available is zstd. Both the mode and format options default to none. To use a compression format, a compression mode must be specified, where the currently supported modes are compress by file or compress by message.

It is recommended to use this feature with the splitting options.

Recording with a storage configuration

Storage configuration can be specified in a YAML file passed through the --storage-config-file option. This can be used to optimize performance for specific use-cases.

For the default storage plugin (sqlite3), the file has a following syntax:

read:
  pragmas: <list of pragma settings for read-only>
write:
  pragmas: <list of pragma settings for read/write>

By default, SQLite settings are significantly optimized for performance. This might have consequences of bag data being corrupted after an application or system-level crash. This consideration only applies to current bagfile in case bag splitting is on (through --max-bag-* parameters). If increased crash-caused corruption resistance is necessary, use resilient option for --storage-preset-profile setting.

Settings are fully exposed to the user and should be applied with understanding. Please refer to documentation of pragmas.

An example configuration file could look like this:

write:
  pragmas: ["journal_mode = MEMORY", "synchronous = OFF", "schema.cache_size = 1000", "schema.page_size = 4096"]


Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Controlling playback via services

The Rosbag2 player provides the following services for remote control, which can be called via ros2 service commandline or from your nodes,

  • ~/burst [rosbag2_interfaces/srv/Burst]
    • Can only be used while player is paused, publishes num_messages in order as fast as possible, moving forward the play head.
  • ~/get_rate [rosbag2_interfaces/srv/GetRate]
    • Return the current playback rate.
  • ~/is_paused [rosbag2_interfaces/srv/IsPaused]
    • Return whether playback is paused.
  • ~/pause [rosbag2_interfaces/srv/Pause]
    • Pause playback. Has no effect if already paused.
  • ~/play [rosbag2_interfaces/srv/Play]
    • Play from a starting offset timestamp, either until the end, an ending timestamp or for a set duration. Only works when stopped (not paused).
  • ~/play_next [rosbag2_interfaces/srv/PlayNext]
    • Play a single next message from the bag. Only works while paused.
  • ~/resume [rosbag2_interfaces/srv/Resume]
    • Resume playback if paused.
  • ~/seek [rosbag2_interfaces/srv/Seek]
    • Change the play head to the specified timestamp. Can be forward or backward in time, the next played message is the next immediately after the seeked timestamp.
  • ~/set_rate [rosbag2_interfaces/srv/SetRate]
    • Sets the rate of playback, for example 2.0 will play messages twice as fast.
  • ~/toggle_paused [rosbag2_interfaces/srv/TogglePaused]
    • Pause if playing, resume if paused.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Converting bags

Rosbag2 provides a tool ros2 bag convert (or, rosbag2_transport::bag_rewrite in the C++ API). This allows the user to take one or more input bags, and write them out to one or more output bags with new settings. This flexible feature enables the following features: * Merge (multiple input bags, one output bag) * Split top-level bags (one input bag, multiple output bags) * Split internal files (by time or size - one input bag with fewer internal files, one output bag with more, smaller, internal files) * Compress/Decompress (output bag(s) with different compression settings than the input(s)) * Serialization format conversion * ... and more!

Here is an example command:

ros2 bag convert --input /path/to/bag1 --input /path/to/bag2 storage_id --output-options output_options.yaml

The --input argument may be specified any number of times, and takes 1 or 2 values. The first value is the URI of the input bag. If a second value is supplied, it specifies the storage implementation of the bag. If no storage implementation is specified, rosbag2 will try to determine it automatically from the bag.

The --output-options argument must point to the URI of a YAML file specifying the full recording configuration for each bag to output (StorageOptions + RecordOptions). This file must contain a top-level key output_bags, which contains a list of these objects.

The only required value in the output bags is uri and storage_id. All other values are options (however, if no topic selection is specified, this output bag will be empty!).

This example notes all fields that can have an effect, with a comment on the required ones.

output_bags
- uri: /output/bag1  # required
  storage_id: sqlite3  # required
  max_bagfile_size: 0
  max_bagfile_duration: 0
  storage_preset_profile: ""
  storage_config_uri: ""
  all: false
  topics: []
  rmw_serialization_format: ""  # defaults to using the format of the input topic
  regex: ""
  exclude: ""
  compression_mode: ""
  compression_format: ""
  compression_queue_size: 1
  compression_threads: 0
  include_hidden_topics: false
  include_unpublished_topics: false

Example merge:

$ ros2 bag convert -i bag1 -i bag2 -o out.yaml

# out.yaml
output_bags:
- uri: merged_bag
  storage_id: sqlite3
  all: true

Example split:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: split1
  storage_id: sqlite3
  topics: [/topic1, /topic2]
- uri: split2
  storage_id: sqlite3
  topics: [/topic3]

Example compress:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: compressed
  storage_id: sqlite3
  all: true
  compression_mode: file
  compression_format: zstd

Overriding QoS Profiles

When starting a recording or playback workflow, you can pass a YAML file that contains QoS profile settings for a specific topic. The YAML schema for the profile overrides is a dictionary of topic names with key/value pairs for each QoS policy. Below is an example profile set to the default ROS2 QoS settings.

/topic_name:
  history: keep_last
  depth: 10
  reliability: reliable
  durability: volatile
  deadline:
    # unspecified/infinity
    sec: 0
    nsec: 0
  lifespan:
    # unspecified/infinity
    sec: 0
    nsec: 0
  liveliness: system_default
  liveliness_lease_duration:
    # unspecified/infinity
    sec: 0
    nsec: 0
  avoid_ros_namespace_conventions: false

You can then use the override by specifying the --qos-profile-overrides-path argument in the CLI:

# Record
ros2 bag record --qos-profile-overrides-path override.yaml -a -o my_bag
# Playback
ros2 bag play --qos-profile-overrides-path override.yaml my_bag

See the official QoS override tutorial and "About QoS Settings" for more detail.

Using in launch

We can invoke the command line tool from a ROS launch script as an executable (not a node action). For example, to launch the command to record all topics you can use the following launch script:

<launch>
  <executable cmd="ros2 bag record -a" output="screen" />
</launch>

Here's the equivalent Python launch script:

import launch


def generate_launch_description():
    return launch.LaunchDescription([
        launch.actions.ExecuteProcess(
            cmd=['ros2', 'bag', 'record', '-a'],
            output='screen'
        )
    ])

Use the ros2 launch command line tool to launch either of the above launch scripts. For example, if we named the above XML launch script, record_all.launch.xml:

$ ros2 launch record_all.launch.xml

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).


Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version iron
Last Updated 2024-02-08
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

License GitHub Action Status

Repository for implementing rosbag2 as described in its corresponding design article.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2*

Note that the above command installs all packages related to rosbag2. This also includes the plugin for reading ROS1 bag files, which brings a hard dependency on the ros1_bridge with it and therefore ROS1 packages. If you want to install only the ROS2 related packages for rosbag, please use the following command:

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2-transport

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

[Note]: if you are only building rosbag2 on top of a Debian Installation of ROS2, please git clone the branch following your current ROS2 distribution.

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Simulation time

In ROS 2, "simulation time" refers to publishing a clock value on the /clock topic, instead of using the system clock to tell time. By passing --use-sim-time argument to ros2 bag record, we turn on this option for the recording node. Messages written to the bag will use the latest received value of /clock for the timestamp of the recorded message.

Note: Until the first /clock message is received, the recorder will not write any messages. Before that message is received, the time is 0, which leads to a significant time jump once simulation time begins, making the bag essentially unplayable if messages are written first with time 0 and then time N from /clock.

Splitting recorded bag files

rosbag2 offers the capability to split bag files when they reach a maximum size or after a specified duration. By default rosbag2 will record all data into a single bag file, but this can be changed using the CLI options.

Splitting by size: ros2 bag record -a -b 100000 will split the bag files when they become greater than 100 kilobytes. Note: the batch size's units are in bytes and must be greater than 86016. This option defaults to 0, which means data is written to a single file.

Splitting by time: ros2 bag record -a -d 9000 will split the bag files after a duration of 9000 seconds. This option defaults to 0, which means data is written to a single file.

If both splitting by size and duration are enabled, the bag will split at whichever threshold is reached first.

Recording with compression

By default rosbag2 does not record with compression enabled. However, compression can be specified using the following CLI options.

For example, ros2 bag record -a --compression-mode file --compression-format zstd will record all topics and compress each file using the zstd compressor.

Currently, the only compression-format available is zstd. Both the mode and format options default to none. To use a compression format, a compression mode must be specified, where the currently supported modes are compress by file or compress by message.

It is recommended to use this feature with the splitting options.

Recording with a storage configuration

Storage configuration can be specified in a YAML file passed through the --storage-config-file option. This can be used to optimize performance for specific use cases.

See storage plugin documentation for more detail: * mcap * sqlite3

Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Converting bags

Rosbag2 provides a tool ros2 bag convert (or, rosbag2_transport::bag_rewrite in the C++ API). This allows the user to take one or more input bags, and write them out to one or more output bags with new settings. This flexible feature enables the following features: * Merge (multiple input bags, one output bag) * Split top-level bags (one input bag, multiple output bags) * Split internal files (by time or size - one input bag with fewer internal files, one output bag with more, smaller, internal files) * Compress/Decompress (output bag(s) with different compression settings than the input(s)) * Serialization format conversion * ... and more!

Here is an example command:

ros2 bag convert --input /path/to/bag1 --input /path/to/bag2 storage_id --output-options output_options.yaml

The --input argument may be specified any number of times, and takes 1 or 2 values. The first value is the URI of the input bag. If a second value is supplied, it specifies the storage implementation of the bag. If no storage implementation is specified, rosbag2 will try to determine it automatically from the bag.

The --output-options argument must point to the URI of a YAML file specifying the full recording configuration for each bag to output (StorageOptions + RecordOptions). This file must contain a top-level key output_bags, which contains a list of these objects.

The only required value in the output bags is uri and storage_id. All other values are options (however, if no topic selection is specified, this output bag will be empty!).

This example notes all fields that can have an effect, with a comment on the required ones.

output_bags:
- uri: /output/bag1  # required
  storage_id: ""  # will use the default storage plugin, if unspecified
  max_bagfile_size: 0
  max_bagfile_duration: 0
  storage_preset_profile: ""
  storage_config_uri: ""
  all: false
  topics: []
  rmw_serialization_format: ""  # defaults to using the format of the input topic
  regex: ""
  exclude: ""
  compression_mode: ""
  compression_format: ""
  compression_queue_size: 1
  compression_threads: 0
  include_hidden_topics: false
  include_unpublished_topics: false

Example merge:

$ ros2 bag convert -i bag1 -i bag2 -o out.yaml

# out.yaml
output_bags:
- uri: merged_bag
  all: true

Example split:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: split1
  topics: [/topic1, /topic2]
- uri: split2
  topics: [/topic3]

Example compress:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: compressed
  all: true
  compression_mode: file
  compression_format: zstd

Overriding QoS Profiles

When starting a recording or playback workflow, you can pass a YAML file that contains QoS profile settings for a specific topic. The YAML schema for the profile overrides is a dictionary of topic names with key/value pairs for each QoS policy. Below is an example profile set to the default ROS2 QoS settings.

/topic_name:
  history: keep_last
  depth: 10
  reliability: reliable
  durability: volatile
  deadline:
    # unspecified/infinity
    sec: 0
    nsec: 0
  lifespan:
    # unspecified/infinity
    sec: 0
    nsec: 0
  liveliness: system_default
  liveliness_lease_duration:
    # unspecified/infinity
    sec: 0
    nsec: 0
  avoid_ros_namespace_conventions: false

You can then use the override by specifying the --qos-profile-overrides-path argument in the CLI:

# Record
ros2 bag record --qos-profile-overrides-path override.yaml -a -o my_bag
# Playback
ros2 bag play --qos-profile-overrides-path override.yaml my_bag

See the official QoS override tutorial and "About QoS Settings" for more detail.

Using in launch

We can invoke the command line tool from a ROS launch script as an executable (not a node action). For example, to launch the command to record all topics you can use the following launch script:

<launch>
  <executable cmd="ros2 bag record -a" output="screen" />
</launch>

Here's the equivalent Python launch script:

import launch


def generate_launch_description():
    return launch.LaunchDescription([
        launch.actions.ExecuteProcess(
            cmd=['ros2', 'bag', 'record', '-a'],
            output='screen'
        )
    ])

Use the ros2 launch command line tool to launch either of the above launch scripts. For example, if we named the above XML launch script, record_all.launch.xml:

$ ros2 launch record_all.launch.xml

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field Storage id:. Rosbag2 was designed to support multiple storage formats to adapt to individual use cases. This repository provides two storage plugins, mcap and sqlite3. The default is mcap, which is provided to code by rosbag2_storage::get_default_storage_id() and defined in default_storage_id.cpp

If not specified otherwise, rosbag2 will write data using the default plugin.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument --storage:

$ ros2 bag record --storage <storage_id>

Bag reading commands can detect the storage plugin automatically, but if for any reason you want to force a specific plugin to read a bag, you can use the --storage option on any ros2 bag verb.

To write your own Rosbag2 storage implementation, refer to this document describing that process

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).


Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version rolling
Last Updated 2024-03-28
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

License GitHub Action Status

Repository for implementing rosbag2 as described in its corresponding design article.

Installation

Debian packages

rosbag2 packages are available via debian packages, and will already be included with any -ros-base installation (which is included within -desktop)

$ export ROS_DISTRO=humble
$ sudo apt-get install ros-$ROS_DISTRO-rosbag2

Other binaries

You can follow the instructions at http://docs.ros.org/en/humble/Installation/Alternatives/Ubuntu-Install-Binary.html (or for your chosen distro), and Rosbag2 will be included in that installation.

Build from source

To build from source, follow the instructions in DEVELOPING.md

Using rosbag2

rosbag2 is part of the ROS 2 command line interface as ros2 bag. These verbs are available for ros2 bag:

  • ros2 bag burst
  • ros2 bag convert
  • ros2 bag info
  • ros2 bag list
  • ros2 bag play
  • ros2 bag record
  • ros2 bag reindex

For up-to-date information on the available options for each, use ros2 bag <verb> --help.

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Simulation time

In ROS 2, "simulation time" refers to publishing a clock value on the /clock topic, instead of using the system clock to tell time. By passing --use-sim-time argument to ros2 bag record, we turn on this option for the recording node. Messages written to the bag will use the latest received value of /clock for the timestamp of the recorded message.

Note: Until the first /clock message is received, the recorder will not write any messages. Before that message is received, the time is 0, which leads to a significant time jump once simulation time begins, making the bag essentially unplayable if messages are written first with time 0 and then time N from /clock.

Splitting files during recording

rosbag2 offers the capability to split bag files when they reach a maximum size or after a specified duration. By default rosbag2 will record all data into a single bag file, but this can be changed using the CLI options.

Splitting by size: ros2 bag record -a -b 100000 will split the bag files when they become greater than 100 kilobytes. Note: the batch size's units are in bytes and must be greater than 86016. This option defaults to 0, which means data is written to a single file.

Splitting by time: ros2 bag record -a -d 9000 will split the bag files after a duration of 9000 seconds. This option defaults to 0, which means data is written to a single file.

If both splitting by size and duration are enabled, the bag will split at whichever threshold is reached first.

Recording with compression

By default rosbag2 does not record with compression enabled. However, compression can be specified using the following CLI options.

For example, ros2 bag record -a --compression-mode file --compression-format zstd will record all topics and compress each file using the zstd compressor.

Currently, the only compression-format available is zstd. Both the mode and format options default to none. To use a compression format, a compression mode must be specified, where the currently supported modes are compress by file or compress by message.

It is recommended to use this feature with the splitting options.

Note: Some storage plugins may have their own compression methods, which are separate from the rosbag2 compression specified by the CLI options --compression-mode and --compression-format. Notably, the MCAP file format offered by the rosbag2_storage_mcap storage plugin supports compression in a way that produces files that are still indexable (whereas using the rosbag2 compression will not). To utilize storage plugin specific compression or other options, see Recording with a storage configuration.

Recording with a storage configuration

Storage configuration can be specified in a YAML file passed through the --storage-config-file option. This can be used to optimize performance for specific use cases.

See storage plugin documentation for more detail: * mcap * sqlite3

Controlling recordings via services

The rosbag2 recorder provides the following services for remote control, which can be called via ros2 service commandline, or from your nodes:

  • ~/is_paused [rosbag2_interfaces/srv/IsPaused]
    • Returns whether recording is currently paused.
  • ~/pause [rosbag2_interfaces/srv/Pause]
    • Pauses recording. All messages that have already arrived will be written, but all messages that arrive after pausing will be discarded. Has no effect if already paused. Takes no arguments.
  • ~/resume [rosbag2_interfaces/srv/Resume]
    • Resume recording if paused. Has no effect if not paused. Takes no arguments.
  • ~/split_bagfile [rosbag2_interfaces/srv/SplitBagfile]
    • Triggers a split to a new file, even if none of the configured split criteria (such as --max-bag-size or --max-bag-duration) have been met yet
  • ~/snapshot [rosbag2_interfaces/srv/Snapshot]
    • enabled if --snapshot-mode is specified. Takes no arguments, triggers a snapshot.

Snapshot mode

The Recorder provides a "snapshot mode", enabled via --snapshot-mode or StorageOptions.snapshot_mode, which does not write messages to disk as they come in, but instead keeps an in-memory circular buffer of size --max-cache-size. This entire buffer can be dumped to disk on request, saving data only in specified circumstances such as a detected error condition or point of interest, capturing the "last N bytes" of incoming data, therefore making sure that you can trigger snapshot after the fact of the event.

The snapshot is taken by calling the ~/snapshot service on the recorder, described previously.

Replaying data

When you have a recorded bag, you can use Rosbag2 to play it back:

$ ros2 bag play <bag>

The bag argument can be a directory containing metadata.yaml and one or more storage files, or to a single storage file such as .mcap or .db3. The Player will automatically detect which storage implementation to use for playing.

Controlling playback via services

The Rosbag2 player provides the following services for remote control, which can be called via ros2 service commandline or from your nodes,

  • ~/burst [rosbag2_interfaces/srv/Burst]
    • Can only be used while player is paused, publishes num_messages in order as fast as possible, moving forward the play head.
  • ~/get_rate [rosbag2_interfaces/srv/GetRate]
    • Return the current playback rate.
  • ~/is_paused [rosbag2_interfaces/srv/IsPaused]
    • Return whether playback is paused.
  • ~/pause [rosbag2_interfaces/srv/Pause]
    • Pause playback. Has no effect if already paused.
  • ~/play [rosbag2_interfaces/srv/Play]
    • Play from a starting offset timestamp, either until the end, an ending timestamp or for a set duration. Only works when stopped (not paused).
  • ~/play_next [rosbag2_interfaces/srv/PlayNext]
    • Play a single next message from the bag. Only works while paused.
  • ~/resume [rosbag2_interfaces/srv/Resume]
    • Resume playback if paused.
  • ~/seek [rosbag2_interfaces/srv/Seek]
    • Change the play head to the specified timestamp. Can be forward or backward in time, the next played message is the next immediately after the seeked timestamp.
  • ~/set_rate [rosbag2_interfaces/srv/SetRate]
    • Sets the rate of playback, for example 2.0 will play messages twice as fast.
  • ~/stop [rosbag2_interfaces/srv/Stop]
    • Stop the player, putting the play head in "undefined position" outside the bag. Must call play before other operations can be done.
  • ~/toggle_paused [rosbag2_interfaces/srv/TogglePaused]
    • Pause if playing, resume if paused.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Converting bags

Rosbag2 provides a tool ros2 bag convert (or, rosbag2_transport::bag_rewrite in the C++ API). This allows the user to take one or more input bags, and write them out to one or more output bags with new settings. This flexible feature enables the following features: * Merge (multiple input bags, one output bag) * Split top-level bags (one input bag, multiple output bags) * Split internal files (by time or size - one input bag with fewer internal files, one output bag with more, smaller, internal files) * Compress/Decompress (output bag(s) with different compression settings than the input(s)) * Serialization format conversion * ... and more!

Here is an example command:

ros2 bag convert --input /path/to/bag1 --input /path/to/bag2 storage_id --output-options output_options.yaml

The --input argument may be specified any number of times, and takes 1 or 2 values. The first value is the URI of the input bag. If a second value is supplied, it specifies the storage implementation of the bag. If no storage implementation is specified, rosbag2 will try to determine it automatically from the bag.

The --output-options argument must point to the URI of a YAML file specifying the full recording configuration for each bag to output (StorageOptions + RecordOptions). This file must contain a top-level key output_bags, which contains a list of these objects.

The only required value in the output bags is uri and storage_id. All other values are options (however, if no topic selection is specified, this output bag will be empty!).

This example notes all fields that can have an effect, with a comment on the required ones.

output_bags:
- uri: /output/bag1  # required
  storage_id: ""  # will use the default storage plugin, if unspecified
  max_bagfile_size: 0
  max_bagfile_duration: 0
  storage_preset_profile: ""
  storage_config_uri: ""
  all: false
  topics: []
  rmw_serialization_format: ""  # defaults to using the format of the input topic
  regex: ""
  exclude: ""
  compression_mode: ""
  compression_format: ""
  compression_queue_size: 1
  compression_threads: 0
  include_hidden_topics: false
  include_unpublished_topics: false

Example merge:

$ ros2 bag convert -i bag1 -i bag2 -o out.yaml

# out.yaml
output_bags:
- uri: merged_bag
  all: true

Example split:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: split1
  topics: [/topic1, /topic2]
- uri: split2
  topics: [/topic3]

Example compress:

$ ros2 bag convert -i bag1 -o out.yaml

# out.yaml
output_bags:
- uri: compressed
  all: true
  compression_mode: file
  compression_format: zstd

Overriding QoS Profiles

When starting a recording or playback, you can pass a YAML file that contains QoS profile settings for a specific topic. The YAML schema for the profile overrides is a dictionary of topic names with key/value pairs for each QoS policy. Below is an example profile set to the default ROS2 QoS settings.

/topic_name:
  history: keep_last
  depth: 10
  reliability: reliable
  durability: volatile
  deadline:
    # unspecified/infinity
    sec: 0
    nsec: 0
  lifespan:
    # unspecified/infinity
    sec: 0
    nsec: 0
  liveliness: system_default
  liveliness_lease_duration:
    # unspecified/infinity
    sec: 0
    nsec: 0
  avoid_ros_namespace_conventions: false

You can then use the override by specifying the --qos-profile-overrides-path argument in the CLI:

# Record
ros2 bag record --qos-profile-overrides-path override.yaml -a -o my_bag
# Playback
ros2 bag play --qos-profile-overrides-path override.yaml my_bag

See the official QoS override tutorial and "About QoS Settings" for more detail.

Using in launch

We can invoke the command line tool from a ROS launch script as an executable (not a node action). For example, to launch the command to record all topics you can use the following launch script:

<launch>
  <executable cmd="ros2 bag record -a" output="screen" />
</launch>

Here's the equivalent Python launch script:

import launch


def generate_launch_description():
    return launch.LaunchDescription([
        launch.actions.ExecuteProcess(
            cmd=['ros2', 'bag', 'record', '-a'],
            output='screen'
        )
    ])

Use the ros2 launch command line tool to launch either of the above launch scripts. For example, if we named the above XML launch script, record_all.launch.xml:

$ ros2 launch record_all.launch.xml

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field Storage id:. Rosbag2 was designed to support multiple storage formats to adapt to individual use cases. This repository provides two storage plugins, mcap and sqlite3. The default is mcap, which is provided to code by rosbag2_storage::get_default_storage_id() and defined in default_storage_id.cpp

If not specified otherwise, rosbag2 will write data using the default plugin.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument --storage:

$ ros2 bag record --storage <storage_id>

Bag reading commands can detect the storage plugin automatically, but if for any reason you want to force a specific plugin to read a bag, you can use the --storage option on any ros2 bag verb.

To write your own Rosbag2 storage implementation, refer to Storage Plugin Development

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).


Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version crystal
Last Updated 2019-04-08
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

THIS IS WORK IN PROGRESS AND SHOULD BE USED WITH CARE

Repository for implementing rosbag2 as described in its corresponding design article

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag* ros-$CHOOSE_ROS_DISTRO-rosbag2*

For other platforms than Linux, rosbag2 has to be built from source as it's currently not part of the latest (ros2.repos file)[https://github.com/ros2/ros2/blob/master/ros2.repos].

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database. Secondly, rosbag2 provides support for legacy ROS 1 bag data. TODO(karsten1987) Add link when rosindex ran.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version eloquent
Last Updated 2020-11-04
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

Repository for implementing rosbag2 as described in its corresponding design article.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag* ros-$CHOOSE_ROS_DISTRO-rosbag2*

For other platforms than Linux, rosbag2 has to be built from source as it's currently not part of the latest ros2.repos file.

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version dashing
Last Updated 2020-10-29
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

Repository for implementing rosbag2 as described in its corresponding design article.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag* ros-$CHOOSE_ROS_DISTRO-rosbag2*

For other platforms than Linux, rosbag2 has to be built from source as it's currently not part of the latest ros2.repos file.

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version galactic
Last Updated 2022-09-29
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

License GitHub Action Status

Repository for implementing rosbag2 as described in its corresponding design article.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2*

Note that the above command installs all packages related to rosbag2. This also includes the plugin for reading ROS1 bag files, which brings a hard dependency on the ros1_bridge with it and therefore ROS1 packages. If you want to install only the ROS2 related packages for rosbag, please use the following command:

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2-transport

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

[Note]: if you are only building rosbag2 on top of a Debian Installation of ROS2, please git clone the branch following your current ROS2 distribution.

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Splitting recorded bag files

rosbag2 offers the capability to split bag files when they reach a maximum size or after a specified duration. By default rosbag2 will record all data into a single bag file, but this can be changed using the CLI options.

Splitting by size: ros2 bag record -a -b 100000 will split the bag files when they become greater than 100 kilobytes. Note: the batch size's units are in bytes and must be greater than 86016. This option defaults to 0, which means data is written to a single file.

Splitting by time: ros2 bag record -a -d 9000 will split the bag files after a duration of 9000 seconds. This option defaults to 0, which means data is written to a single file.

If both splitting by size and duration are enabled, the bag will split at whichever threshold is reached first.

Recording with compression

By default rosbag2 does not record with compression enabled. However, compression can be specified using the following CLI options.

For example, ros2 bag record -a --compression-mode file --compression-format zstd will record all topics and compress each file using the zstd compressor.

Currently, the only compression-format available is zstd. Both the mode and format options default to none. To use a compression format, a compression mode must be specified, where the currently supported modes are compress by file or compress by message.

It is recommended to use this feature with the splitting options.

Recording with a storage configuration

Storage configuration can be specified in a YAML file passed through the --storage-config-file option. This can be used to optimize performance for specific use-cases.

For the default storage plugin (sqlite3), the file has a following syntax:

read:
  pragmas: <list of pragma settings for read-only>
write:
  pragmas: <list of pragma settings for read/write>

By default, SQLite settings are significantly optimized for performance. This might have consequences of bag data being corrupted after an application or system-level crash. This consideration only applies to current bagfile in case bag splitting is on (through --max-bag-* parameters). If increased crash-caused corruption resistance is necessary, use resilient option for --storage-preset-profile setting.

Settings are fully exposed to the user and should be applied with understanding. Please refer to documentation of pragmas.

An example configuration file could look like this:

write:
  pragmas: ["journal_mode = MEMORY", "synchronous = OFF", "schema.cache_size = 1000", "schema.page_size = 4096"]


Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Overriding QoS Profiles

When starting a recording or playback workflow, you can pass a YAML file that contains QoS profile settings for a specific topic. The YAML schema for the profile overrides is a dictionary of topic names with key/value pairs for each QoS policy. Below is an example profile set to the default ROS2 QoS settings.

/topic_name:
  history: keep_last
  depth: 10
  reliability: reliable
  durability: volatile
  deadline:
    # unspecified/infinity
    sec: 0
    nsec: 0
  lifespan:
    # unspecified/infinity
    sec: 0
    nsec: 0
  liveliness: system_default
  liveliness_lease_duration:
    # unspecified/infinity
    sec: 0
    nsec: 0
  avoid_ros_namespace_conventions: false

You can then use the override by specifying the --qos-profile-overrides-path argument in the CLI:

# Record
ros2 bag record --qos-profile-overrides-path override.yaml -a -o my_bag
# Playback
ros2 bag play --qos-profile-overrides-path override.yaml my_bag

See the official QoS override tutorial and "About QoS Settings" for more detail.

Using in launch

We can invoke the command line tool from a ROS launch script as an executable (not a node action). For example, to launch the command to record all topics you can use the following launch script:

<launch>
  <executable cmd="ros2 bag record -a" output="screen" />
</launch>

Here's the equivalent Python launch script:

import launch


def generate_launch_description():
    return launch.LaunchDescription([
        launch.actions.ExecuteProcess(
            cmd=['ros2', 'bag', 'record', '-a'],
            output='screen'
        )
    ])

Use the ros2 launch command line tool to launch either of the above launch scripts. For example, if we named the above XML launch script, record_all.launch.xml:

$ ros2 launch record_all.launch.xml

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.

Contributors must sign-off each commit by adding a Signed-off-by: ... line to commit messages to certify that they have the right to submit the code they are contributing to the project according to the Developer Certificate of Origin (DCO).


Repository Summary

Checkout URI https://github.com/ros2/rosbag2.git
VCS Type git
VCS Version foxy
Last Updated 2023-05-31
Dev Status DEVELOPED
CI status No Continuous Integration
Released RELEASED
Tags No category tags.
Contributing Help Wanted (0)
Good First Issues (0)
Pull Requests to Review (0)

README

rosbag2

License GitHub Action Status

Repository for implementing rosbag2 as described in its corresponding design article.

[Note] This branch contains the official binary release of rosbag2 for ROS 2 Foxy. The code here is what you get from the APT repositories.

There is an alternate supported branch foxy-future that contains a version much closer to Galactic, including extensive performance improvements and some features, with modifications to build against the ROS 2 Foxy core. Not all of the improvements could be backported into the binary release without breaking API/ABI, so users who want these improvements can build that branch from source into their application - please see the README on the foxy-future branch for further information.

Installation instructions

Debian packages

rosbag2 packages are available via debian packages and thus can be installed via

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2*

Note that the above command installs all packages related to rosbag2. This also includes the plugin for reading ROS1 bag files, which brings a hard dependency on the ros1_bridge with it and therefore ROS1 packages. If you want to install only the ROS2 related packages for rosbag, please use the following command:

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2-transport

Build from source

It is recommended to create a new overlay workspace on top of your current ROS 2 installation.

$ mkdir -p ~/rosbag_ws/src
$ cd ~/rosbag_ws/src

Clone this repository into the source folder:

$ git clone https://github.com/ros2/rosbag2.git

[Note]: if you are only building rosbag2 on top of a Debian Installation of ROS2, please git clone the branch following your current ROS2 distribution.

Then build all the packages with this command:

$ colcon build [--merge-install]

The --merge-install flag is optional and installs all packages into one folder rather than isolated folders for each package.

Executing tests

The tests can be run using the following commands:

$ colcon test [--merge-install]
$ colcon test-result --verbose

The first command executes the test and the second command displays the errors (if any).

Using rosbag2

rosbag2 is part of the ROS 2 command line interfaces. This repo introduces a new verb called bag and thus serves as the entry point of using rosbag2. As of the time of writing, there are three commands available for ros2 bag:

  • record
  • play
  • info

Recording data

In order to record all topics currently available in the system:

$ ros2 bag record -a

The command above will record all available topics and discovers new topics as they appear while recording. This auto-discovery of new topics can be disabled by given the command line argument --no-discovery.

To record a set of predefined topics, one can specify them on the command line explicitly.

$ ros2 bag record <topic1> <topic2> … <topicN>

The specified topics don't necessarily have to be present at start time. The discovery function will automatically recognize if one of the specified topics appeared. In the same fashion, this auto discovery can be disabled with --no-discovery.

If not further specified, ros2 bag record will create a new folder named to the current time stamp and stores all data within this folder. A user defined name can be given with -o, --output.

Replaying data

After recording data, the next logical step is to replay this data:

$ ros2 bag play <bag_file>

The bag file is by default set to the folder name where the data was previously recorded in.

Analyzing data

The recorded data can be analyzed by displaying some meta information about it:

$ ros2 bag info <bag_file>

You should see something along these lines:

Files:             demo_strings.db3
Bag size:          44.5 KiB
Storage id:        sqlite3
Duration:          8.501s
Start:             Nov 28 2018 18:02:18.600 (1543456938.600)
End                Nov 28 2018 18:02:27.102 (1543456947.102)
Messages:          27
Topic information: Topic: /chatter | Type: std_msgs/String | Count: 9 | Serialization Format: cdr
                   Topic: /my_chatter | Type: std_msgs/String | Count: 18 | Serialization Format: cdr

Using in launch

We can invoke the command line tool from a ROS launch script as an executable (not a node action). For example, to launch the command to record all topics you can use the following launch script:

<launch>
  <executable cmd="ros2 bag record -a" output="screen" />
</launch>

Here's the equivalent Python launch script:

import launch


def generate_launch_description():
    return launch.LaunchDescription([
        launch.actions.ExecuteProcess(
            cmd=['ros2', 'bag', 'record', '-a'],
            output='screen'
        )
    ])

Use the ros2 launch command line tool to launch either of the above launch scripts. For example, if we named the above XML launch script, record_all.launch.xml:

$ ros2 launch record_all.launch.xml

Using the API

rosbag2 offers a complete API for manipulating bag files. Code samples demonstrating how to do so can be found in the rosbag2_samples directory.

Storage format plugin architecture

Looking at the output of the ros2 bag info command, we can see a field called storage id:. rosbag2 specifically was designed to support multiple storage formats. This allows a flexible adaptation of various storage formats depending on individual use cases. As of now, this repository comes with two storage plugins. The first plugin, sqlite3 is chosen by default. If not specified otherwise, rosbag2 will store and replay all recorded data in an SQLite3 database.

In order to use a specified (non-default) storage format plugin, rosbag2 has a command line argument for it:

$ ros2 bag <record> | <play> | <info> -s <sqlite3> | <rosbag2_v2> | <custom_plugin>

Have a look at each of the individual plugins for further information.

Serialization format plugin architecture

Looking further at the output of ros2 bag info, we can see another field attached to each topic called Serialization Format. By design, ROS 2 is middleware agnostic and thus can leverage multiple communication frameworks. The default middleware for ROS 2 is DDS which has cdr as its default binary serialization format. However, other middleware implementation might have different formats. If not specified, ros2 bag record -a will record all data in the middleware specific format. This however also means that such a bag file can't easily be replayed with another middleware format.

rosbag2 implements a serialization format plugin architecture which allows the user the specify a certain serialization format. When specified, rosbag2 looks for a suitable converter to transform the native middleware protocol to the target format. This also allows to record data in a native format to optimize for speed, but to convert or transform the recorded data into a middleware agnostic serialization format.

By default, rosbag2 can convert from and to CDR as it's the default serialization format for ROS 2.

CONTRIBUTING

Any contribution that you make to this repository will be under the Apache 2 License, as dictated by that license:

5. Submission of Contributions. Unless You explicitly state otherwise,
   any Contribution intentionally submitted for inclusion in the Work
   by You to the Licensor shall be under the terms and conditions of
   this License, without any additional terms or conditions.
   Notwithstanding the above, nothing herein shall supersede or modify
   the terms of any separate license agreement you may have executed
   with Licensor regarding such Contributions.