System metrics collector

Supported targets

The system metrics collector package supports the following:

• ROS Distro
  • ROS 2: eloquent
• OS: Ubuntu Bionic

Description

This package aims to integrate lightweight aggregation tools in order to collect, measure, and publish aggregate metrics.

The moving average tools provide constant time sample aggregation to produce average, min, max, standard deviation, and sample count. The collector provides a thread-safe mechanism to perform measurements. The PeriodicMeasurementNode is a ROS2 Lifecycle Node which utilizes the aggregation tools and provides an abstraction for ROS2: specifically a configurable timer (default 1 second) is used to perform measurements, which is published (default 1 minute - but configurable) to a topic (default /system_metrics).

The lifecycle state transitions are automatically invoked such that the Node is in an activated state after creation. The goal is to start metric collection automatically after launching. However, the lifecycle actions (activate, deactivate, shutdown) can be manually invoked. Please see this example.

Parameters

There are two parameters defined:

• measurement_period:

Parameter name: measurement_period
Type: integer
  Description: The period in milliseconds between each measurement
  Constraints:
    Read only: true
    Min value: 1
    Max value: 9223372036854775807
    Step: 1

• publish_period:

Parameter name: publish_period
  Type: integer
  Description: The period in milliseconds between each published MetricsMessage. This must be less than
  the measurement_period.
  Constraints:
    Read only: true
    Min value: 1
    Max value: 9223372036854775807
    Step: 1

Usage:

We provide multiple example entry points to use this package in the examples directory, which demonstrate how to run the metric collection nodes and instrument existing nodes in order to measure their performance.

ROS2 Launch

The talker_listener_example launch file demonstrates how to measure the CPU and memory of a ROS2 process. Specifically, this example instruments the demo_nodes_cpp talker and listener nodes and launches them, in separate processes, with the system CPU and memory measurement nodes. This example can be run using ros2launch.

ros2 launch system_metrics_collector talker_listener_example.launch.py

The system_cpu_and_memory_configuration_example can be launched with ros2launch, which demonstrates how to configure the metric collection nodes.

ros2 launch system_metrics_collector system_cpu_and_memory.launch.py

Manual Execution

You can run the example_main executable which will manually start the system CPU, system memory, process CPU and process memory measurement nodes. This also sets all the nodes' verbosity levels to DEBUG, which will print collection and status information to the console. Using ros2run

ros2 run system_metrics_collector example_main

Change publish_period or measurement_period using --ros-args:

ros2 run system_metrics_collector example_main --ros-args -p measurement_period:=100 -publish_period 1000

This node will generate 4 nodes. Using ros2node

ros2 node list
/linuxCpuCollector
/linuxMemoryCollector
/linuxProcessCpuCollector
/linuxProcessMemoryCollector

Interaction via the ROS2 CLI

This section describes how to interact with this package using the ROS2 Command Line Interface (CLI). Example output is provided where relevant.

Subscribe to /system_metrics to visualize all the statistics.

Using ros2topic

ros2 topic echo /system_metrics
measurement_source_name: linuxMemoryCollector
metrics_source: system_memory_percent_used
unit: percent
window_start:
  sec: 1579638873
  nanosec: 125927653
window_stop:
  sec: 1579638933
  nanosec: 125649059
statistics:
- data_type: 1
  data: 51.665252685546875
- data_type: 3
  data: 51.906898498535156
- data_type: 2
  data: 51.5028190612793
- data_type: 5
  data: 60.0
- data_type: 4
  data: 0.09770704805850983
---
measurement_source_name: linuxCpuCollector
metrics_source: system_cpu_percent_used
unit: percent
window_start:
  sec: 1579638873
  nanosec: 125928189
window_stop:
  sec: 1579638933
  nanosec: 125649074
statistics:
- data_type: 1
  data: 11.075708389282227
- data_type: 3
  data: 22.03821563720703
- data_type: 2
  data: 3.9845757484436035
- data_type: 5
  data: 60.0
- data_type: 4
  data: 5.842845916748047
---

Review all the available parameters

Using ros2param

ros2 param list
/linuxCpuCollector:
  measurement_period
  publish_period
  use_sim_time
/linuxMemoryCollector:
  measurement_period
  publish_period
  use_sim_time
/linuxProcessCpuCollector:
  measurement_period
  publish_period
  use_sim_time
/linuxProcessMemoryCollector:
  measurement_period
  publish_period
  use_sim_time

Inspect and change lifecycle state

Using ros2lifecycle

List lifecycle nodes:

ros2 lifecycle nodes
/linuxCpuCollector
/linuxMemoryCollector

Get the state of a specific node:

ros2 lifecycle get /linuxCpuCollector
active [3]

Activate the node (start measurement collection and data publishing):

ros2 lifecycle set /linuxCpuCollector activate
Transitioning successful

Deactivate the node (stop measurement collection and data publishing):

ros2 lifecycle set /linuxCpuCollector deactivate
Transitioning successful

Changelog for package system_metrics_collector

0.1.1 (2020-09-02)

• Fix test_subscriber_topic_statistics being referenced out of scope causing colcon build with flag -DBUILD_TESTING=0 to fail. (#175) (#176)
• Contributors: Jaison Titus

0.1.0 (2020-08-13)

• Bump package version to 0.1.0 (#170)
• Allow topic statistics to subscribe to multiple topics (#143)
• Add topic statistics executable, launch file and e2e test (#125)
• Move metrics_statistics_msgs to rcl_interfaces (#135)
• Add DummyMessage talker node and launch file for it (#119)
• Fix test subscriber failures (#126)
• Move message generation function to libstatistics_collector (#122)
• Replace IMU messages with new dummy messages (#121)
• Add SubscriberTopicStatisticsNode and tests (#112)
• Create libstatistics_collector package (#117)
• Add statistic name and unit interfaces (#115)
• Use rcutils_time_point_value_t instead of rclcpp::Time (#113)
• Remove deprecated launch params (#109)
• Add message age collector class (#106)
• Refactor Linux process CPU measurement tests (#107)
• Dabonnie/fix cpu measurement test (#102)
• Move common test utilities (#101)
• Refactor Linux memory measurement tests (#100)
• Cleanup package.xml dependencies (#97)
• Add received message period measurement (#94)
• Fix relative #include paths (#95)
• Add test for collector state (#93)
• Move stateless method to free function (#92)
• Add e2e test workflow (#88)
• Fix uncrustify linting (#89)
• Add e2e test (#86)
• Add unit information to MetricsMessage (#85)
• Clarify that the main executable is an example (#84)
• Dabonnie/readme update (#82)
• Add example launch file (#81)
• Add configurable launch parameters (#80)
• Update README.md (#79)
• Dabonnie/add documentation (#77)
• Add unit test for composeability of nodes (#76)
• Make LinuxProcessCpuMeasurementNode and LinuxProcessMemoryMeasurementNode composeable (#67)
• Added some doc (#75)
• Refactor PeriodicMeasurementNode to inherit from LifecycleNode (#72)
• Use rclcpp::NodeOptions during node construction (#73)
• Add input argument checking to PeriodicMeasurementNode (#69)
• Add process CPU usage measurement node (#59)
• Fix linting issues (#54)
• Add launch file to launch system CPU and Memory collector nodes (#63)
• Replace test method with implementation (#62)
• Add process CPU usage measurement dependencies (#57)
• Follow variable naming conventions of ROS2 style guide (#56)
• Follow function naming conventions of ROS2 style guide (#53)
• Style guide constants (#51)
• Revert \"Add new package to view system metric data (#46)\" (#52)
• Revert \"add process cpu measurement node and tests (#40)\" (#48)
• add process cpu measurement node and tests (#40)
• Add new package to view system metric data (#46)
• refactor cpu measurement utilities and tests (#45)
• Fix function erroneously declared as static (#37)
• Remove clear_measurements_on_publish (#31)
• move private constants to .cpp file (#39)
• Install system metrics collector (#34)
• Process memory measurement (#28)
• Use uint64_t for /proc/stat cpu parsed type (#29)
• Add publishing of MetricMessages to CPU/Memory measurement nodes (#16)
• Move publish message creation to the super class (#26)
• Enable code coverage in CI builds (#7)
• Fix parsing overflow issue (#24) (#25)
• Add MetricsMessagePublisher interface and implementation (#22)
• Add utilities file (#23)
• Remove DEFAULT_PUBLISH_WINDOW (#19)
• fix bug in computeCpuActivePercentage() (#18)
• Add namespaces (#15)
• Add node to measure linux system memory (#12)
• Measure linux cpu active percentage (#11)
• Add periodic API (#6)
• Collector (#4)
• Add Moving Average Statistics Calculators (#3)
• Add system_metrics_collector package (#2)
• Contributors: Alejandro Hern