|Tags||No category tags.|
|License||Apache License 2.0|
- Ralph Lange
The examples_rclcpp_cbg_executor package provides a demo and test bench for the Callback-group-level Executor concept. This concept was developed in 2018 and has been integrated in ROS 2 mainline in 2020, i.e., is available from ROS 2 Galactic on. It does not add a new Executor but leverages callback groups for refining the Executor API to callback-group-level granularity.
This allows a single node to have callbacks with different real-time requirements assigned to different Executor instances – within one process. Thus, an Executor instance can be dedicated to one or few specific callback groups and the Executor’s thread (or threads) can be prioritized according to the real-time requirements of these groups. For example, all critical callbacks may be handled by an Executor instance based on an thread running at the highest scheduler priority.
Introduction to demo
The demo comprises a Ping Node and a Pong Node which exchange messages on two communication paths simultaneously. There is a high priority path formed by the topics high_ping and high_pong and a low priority path formed by low_ping and low_pong, respectively.
The Ping Node sends ping messages on both paths simultaneously at a configurable rate. The Pong Node takes these ping messages and replies each of them. Before sending a reply, it burns a configurable number of CPU cycles (thereby varying the processor load) to simulate some message processing.
All callbacks of the Ping Node (i.e., for the timer for sending ping messages and for the two subscription on high_pong and low_pong) are handled in one callback group and thus Executor instance. However, the two callbacks of the Pong Node that process the incoming ping messages and answer with a pong message are assigned to two different callback groups. In the main function, these two groups are distributed to two Executor instances and threads. Both threads are pinned to the same CPU (No. 0) and thus share its processing power, but with different scheduler priorities following the names high and low.
Running the demo
The Ping Node and Pong Node may be either started in one process or in two processes. Please note that on Linux the demo requires sudo privileges to be able to change the thread priorities using
Running the two nodes in one process:
sudo bash source /opt/ros/[ROS_DISTRO]/setup.bash ros2 run examples_rclcpp_cbg_executor ping_pong
Example of a typical output - note the zero pongs received on the low prio path:
[INFO] [..] [pong_node]: Running experiment from now on for 10s ... [INFO] [..] [ping_node]: Both paths: Sent out 982 of configured 1000 pings, i.e. 98%. [INFO] [..] [ping_node]: High prio path: Received 980 pongs, i.e. for 99% of the pings. [INFO] [..] [ping_node]: High prio path: Average RTT is 17.2ms. [INFO] [..] [ping_node]: Low prio path: Received 0 pongs, i.e. for 0% of the pings. [INFO] [..] [pong_node]: High priority executor thread ran for 9995ms. [INFO] [..] [pong_node]: Low priority executor thread ran for 0ms.
Running the two nodes in separate processes:
sudo bash source /opt/ros/[ROS_DISTRO]/setup.bash ros2 run examples_rclcpp_cbg_executor ping
sudo bash source /opt/ros/[ROS_DISTRO]/setup.bash ros2 run examples_rclcpp_cbg_executor pong
The two processes should be started simultaneously as the experiment runtime is just 10 seconds.
There are three parameters to configure the experiment:
ping_period- period (double value in seconds) for sending out pings on the topics high_ping and low_ping simultaneously in the Ping Node.
high_busyloop- duration (double value in seconds) for burning CPU cycles on receiving a message from high_ping in the Pong Node.
low_busyloop- duration (double value in seconds) for burning CPU cycles on receiving a message from low_ping in the Pong Node.
The default values are 0.01 seconds for all three parameters.
Example for changing the values on the command line:
ros2 run examples_rclcpp_cbg_executor ping_pong --ros-args -p ping_period:=0.033 -p high_busyloop:=0.025
With these values, about (0.033s - 0.025s) / 0.010s = 80% of the ping messages on the low prio path should be processed and answered by a pong message:
... [INFO] [..] [ping_node]: Both paths: Sent out 303 of configured 303 pings, i.e. 100%. [INFO] [..] [ping_node]: High prio path: Received 302 pongs, i.e. for 99% of the pings. [INFO] [..] [ping_node]: High prio path: Average RTT is 25.2ms. [INFO] [..] [ping_node]: Low prio path: Received 231 pongs, i.e. for 76% of the pings. [INFO] [..] [ping_node]: Low prio path: Average RTT is 196.1ms. ...
The Ping Node and the Pong Node are implemented in two classes
PingNode (see ping_node.hpp) and
PongNode (see pong_node.hpp), respectively. In addition to the mentioned timer and subscriptions, the PingNode class provides a function
print_statistics() to print statistics on the number of sent and received messages on each path and the average round trip times. To burn the specified number of CPU cycles, the PongNode class contains a function
burn_cpu_cycles(duration) to simulate a given processing time before replying with a pong.
The Ping and Pong nodes, the two executors, etc. are composed and configured in the
main(..) function of main.cpp. This function also starts and ends the experiment for a duration of 10 seconds and prints out the throughput and round trip time (RTT) statistics.
The demo also runs on Windows, where the two threads are prioritized as above normal and below normal, respectively, which does not require elevated privileges. When running the demo on Linux without sudo privileges, a warning is shown but the execution is not stopped.
On macOS the core pinning failed silently in our experiments. Please see the function
configure_native_thread(..) in utilities.hpp for details.
Changelog for package examples_rclcpp_cbg_executor
- Fix clang warnings about type mismatches. (#309)
- Contributors: Chris Lalancette
- Support for cbg_executor package on QNX (#305)
- Contributors: joshua-qnx
- Demo for callback-group-level executor concept. (#302)
- Contributors: Ralph Lange