reductstore_agent

reductstore_agent is a ROS 2 node that records selected topics into ReductStore, a high-performance storage and streaming solution. ReductStore is an ELT-based system for robotics and industrial IoT data acquisition. It ingests and streams time-series data of any size—images, sensor readings, logs, files, MCAP, ROS bags—and stores it with time indexing and labels for ultra-fast retrieval and management.

This agent is fully configurable via YAML and designed to solve storage, bandwidth, and workflow limitations commonly found in field robotics. It streams data to ReductStore in near real-time with optional compression, splitting, dynamic labeling, and per-pipeline controls.

System Requirements

To use this agent, you must have a running instance of ReductStore. You can start a local instance using Docker, install it via Snap or from binaries. Refer to the official guide for setup instructions: ReductStore Getting Started Guide

This agent is tested with:

• ROS 2: Jazzy and Rolling
• OS: Ubuntu 24.04 (Noble)
• Python: 3.12

Motivation

• Continuous recording: Prevent oversized rosbag files by splitting recordings by time, size, or topic groups.
• Bandwidth constraints: Filter and compress data before optionally replicating to a central server or the cloud.
• Manual workflows: Replace manual drive swaps, custom scripts, and bag handling with automated data management.
• Lack of filtering: Apply dynamic labels (e.g., mission ID) to tag, search, and retrieve specific data segments.
• Ubuntu Core: Distributed as a Snap and suitable for deployment on Ubuntu Core–based robotic systems.

Documentation

📘 ROS 2 ReductStore Agent Documentation
👉 https://www.reduct.store/docs/integrations/ros2-agent

📚 ReductStore Platform Documentation
👉 https://www.reduct.store/docs

The website is the single source of truth for:

• YAML configuration options
• Pipeline behavior
• Storage quotas
• Compression & downsampling
• MCAP / CDR formats
• Performance tuning

Quickstart and Installation

1. Start a ReductStore Instance

The agent requires a running ReductStore instance.

For local testing, you can start ReductStore using Docker:

docker run -p 8383:8383 reductstore/reductstore:latest

2. Install the ROS 2 Agent

Choose one of the following installation methods:

a) Snap Package (Recommended)

sudo snap install reductstore-agent --edge

Optionally you can also install a rosbag_replayer node with the –replayer flag

sudo snap install reductstore-agent --edge --replayer

b) Build from Source

# Create workspace and clone repository
mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src
git clone https://github.com/reductstore/reductstore_agent.git
cd ..

# Install dependencies
rosdep install --from-paths src --ignore-src -r -y

# Set up virtual environment
python3 -m venv .venv --system-site-packages
source .venv/bin/activate

# Install Python dependencies
pip install -U reduct-py mcap mcap-ros2-support

# Build the package
python -m colcon build --symlink-install

3. Run the Recorder

# For both Snap and source installs:
ros2 run reductstore_agent recorder --ros-args --params-file ./config/params.yml
# rosbag_replayer
ros2 run reductstore_agent rosbag_replayer

Note: The recorder requires a parameters file to start. Both the Snap and source distributions include a default configuration suitable for local testing.

File truncated at 100 lines see the full file

Changelog for package reductstore_agent

0.2.0 (2025-07-09)

• Merge pull request #23 from reductstore/9-add-support-for-include_regex-and-exclude_regex Add support for include/exclude topics using regex
• Merge pull request #24 from reductstore/12-add-support-for-static-labels-from-pipeline-config Add support for static labels from pipeline config
• Merge pull request #22 from reductstore/8-add-reductstore-bucket-settings-to-config Add support for bucket settings

0.1.0 (2025-06-09)

• Merge pull request #20 from reductstore/16-open-pr-to-rosrosdistro-to-publish-ros-package Update name to reductstore_agent
• Merge pull request #19 from reductstore/7-enable-pylint-in-github-actions Add code linting with flake8, pep257, and copyright checks
• Merge pull request #6 from reductstore/3-minimum-recording-implementation Implement minimum recorder agent for ROS 2 MCAP files
• Merge pull request #10 from reductstore/10-add-downsampling-option Implement downsampling option per pipeline
• Merge pull request #11 from reductstore/11-support-raw-output Implement raw output format per pipeline
• Merge pull request #38 from reductstore/38-add-downsampling-integration-test Add integration test for downsampling
• Merge pull request #14 from reductstore/14-support-dynamic-labels Add dynamic labels feature
• Merge pull request #42 from reductstore/35-configuration-bucket-auto-sync Add remote configuration auto-sync feature
• Merge pull request #47 from reductstore/43-reorganize-documentation Reorganize documentation
• Merge pull request #49 from reductstore/45-ros-node-for-producing-data Add ROS node for producing data