```python
# Device.msg
/DeviceAttributesEntry[] attributes
```
</td>
</tr>
</table>
</div>
##### Any types
Dynamically typed (i.e. `google.protobuf.Any`) Protobuf messages are mapped to ROS 2 messages as specified by _any expansions_.
An any expansion is a type set $T$ that fully characterizes what to expect of a `google.protobuf.Any` message field.
These are indexed after Protobuf message name and field name. Cardinality $|T|$ always satisfies $|T| > 0$.
Given an applicable any expansion is found:
- if $|T| == 1$ and `allow_any_casts` is enabled, the corresponding equivalent ROS 2 message type for that sole Protobuf
message type, as dictated by message type mapping rules, will be used (as if that statically typed Protobuf message
type had been found in place of `google.protobuf.Any`);
- otherwise, a dynamically typed (i.e. `proto2ros/Any`) ROS 2 message is used as the equivalent ROS 2 message type,
which will bear the equivalent ROS 2 type set (with $|T| > 0$) as dictated by message type mapping rules.
Else, a `proto2ros/AnyProto` ROS 2 message type is used as the equivalent ROS 2 message type, bearing the unmodified,
serialized Protobuf message.
For example, given the following configuration overlay:
```yaml
any_expansions:
third_party.data.Storage.params: third_party.data.StorageParams
third_party.data.StorageParams.implementation_specific: [third_party.data.S3Params, third_party.data.PGParams]
allow_any_casts: true
```
`google.protobuf.Any` for the `params` field in the `third_party.data.Storage` Protobuf message would map to the ROS 2
equivalent, as per message type mapping rules, of the `third_party.data.StorageParams` Protobuf message, whereas
`google.protobuf.Any` for the `implementation_specific` field in the `third_party.data.StorageParams` Protobuf message
would map to `proto2ros/Any`. Conversion APIs, however, can use the information provided by these any expansions to
perform the necessary casting in runtime.
#### Recursive types
Protobuf supports recursive message definitions but ROS 2 does not. To workaround this limitation, a message dependency graph
reflecting the composition relationships between known messages is built and analyzed for cycles. Once a cycle has been identified,
it is broken by the weakest link (i.e. the minimal change set) using `proto2ros/Any`, functionally type erasing one or more fields.
#### Field mapping
##### Optional fields
Optional fields in Protobuf messages, and fields with [explicit presence](https://protobuf.dev/programming-guides/field_presence)
tracking in general, are conventionally implemented using a bit mask field in ROS 2 messages. As ROS 2 messages lack the notion
of optional fields entirely, an unsigned integer `has_field` field explicitly conveys which message fields bear meaningful
information. For each optional field `f`, an unsigned integer constant `F_FIELD_SET` bit mask is defined. Bitwise binary
operations can then be used to explicitly indicate and check for field presence. A sample equivalence is shown below.
| Protobuf .proto definition |
ROS 2 .msg definition |
|
```proto
// some.proto
message Option {
optional string value = 1;
}
```
|
```python
// Option.msg
uint8 VALUE_FIELD_SET=1
string value
uint8 has_field 255
```
|
Note that, to match ROS 2 message semantics, the bit mask is fully set by default. That is, all fields are assumed to be present by default.
**Implementation note**: bit masks can be 8, 16, 32, or 64 bit long, depending on the number of optional fields. Protobuf messages
with more than 64 optional fields are therefore not supported.
##### Repeated fields
Repeated fields in Protobuf messages are mapped to array fields in ROS 2 messages. This applies to all field types except to `bytes`
fields. This exception is necessary as scalar `bytes` fields are already mapped to array fields in ROS 2. In this case, scalar type
mapping rules are overridden and repeated `bytes` fields are mapped to array fields of `proto2ros/Bytes` ROS 2 message type. A sample
equivalence is shown below.
| Protobuf .proto definition |
ROS 2 .msg definition |
|
```proto
// some.proto
message Payload {
repeated int32 keys = 1;
repeated bytes blobs = 2;
bytes checksum = 3;
}
```
|
```python
# Payload.msg
int32[] keys
proto2ros/Bytes[] blobs
uint8[] checksum
```
|
##### One-of fields
As ROS 2 messages lack the notion of one-of fields entirely, a ROS 2 message is generated for each one-of construct in a Protobuf message,
bearing all one-of fields, as well as an integer `which` field. This ROS 2 message is functionally equivalent to a [tagged union](https://en.wikipedia.org/wiki/Tagged_union).
For each field `f` in the one-of construct `o`, an integer constant `O_F_SET` tag is defined. Assigning the `which` field to a given tag thus
conveys presence of the corresponding field. In place for each one-of construct, a message field of the corresponding type is defined. A sample
equivalence is shown below.
| Protobuf .proto definition |
ROS 2 .msg definition |
|
```proto
// some.proto
message Timestamp {
oneof value {
uint64 seconds_since_epoch = 1;
string datestring = 2;
}
}
```
|
```python
# Timestamp.msg
/TimestampOneOfValue value
```
</td>
</tr>
|
|
```python
# TimestampSecondsSinceEpoch.msg
uint64 seconds_since_epoch
```
|
|
|
```python
# TimestampDatestring.msg
string datestring
```
|
|
```python
# TimestampOneOfValue.msg
int8 VALUE_NOT_SET=0
int8 VALUE_SECOND_SINCE_EPOCH_SET=1
int8 VALUE_DATESTRING_SET=2
/TimestampSecondsSinceEpoch seconds_since_epoch
/TimestampDatestring datestring
int8 value_choice # deprecated
int8 which
```
</td>
</tr>
</table>
</div>
**Implementation note**: 8 bit tags are used for one-of constructs. Protobuf messages with more than 256 one-of fields are therefore not supported.
##### Deprecated fields
Deprecated fields are kept, unless `drop_deprecated` is enabled. If kept, these fields are annotated with a comment
in the corresponding ROS 2 message definition. A sample equivalence is shown below.
| Protobuf .proto definition |
ROS 2 .msg definition (drop_deprecated disabled) |
ROS 2 .msg definition (drop_deprecated enabled) |
|
```proto
// some.proto
message Duration {
int64 seconds = 1;
int64 nanosec = 2 [deprecated = true];
int64 nanoseconds = 3;
}
```
|
```python
# Duration.msg
int64 seconds
int64 nanosec # deprecated
int64 nanoseconds
```
|
```python
# Duration.msg
int64 seconds
int64 nanoseconds
```
|
##### Reserved fields
Reserved fields are ignored.
| Protobuf .proto definition |
ROS 2 .msg definition |
|
```proto
// some.proto
message Goal {
string location = 1;
reserved "time_budget";
}
```
|
```python
# Goal.msg
string location
```
|
### Code generation
To simplify conversion from Protobuf messages to equivalent ROS 2 messages and back, `proto2ros` generates conversion code,
nicely wrapped around `convert(from, to)` function overloads (i.e. type dispatched). Note, however, that conversion code is
only generated for message equivalences that `proto2ros` itself generated in full. For ad-hoc equivalences, as specified using
message mappings, the user must implement the corresponding overloads. For auxiliary messages underpinning enums, map types,
one-of fields, and the like, no overloads are generated at all (as there is no Protobuf message to convert to/from).
#### Python APIs
Conversion APIs in Python are exposed on a package basis, as `{ros_package_name}.conversions.convert` overloads.
For each pair of equivalent `ROSMessageT` and `ProtoMessageT` types, `proto2ros` generates the following overloads:
- `convert(ros_msg: ROSMessageT, proto_msg: ProtoMessageT) -> None` for ROS 2 message to Protobuf message conversion
- `convert(proto_msg: ProtoMessageT, ros_msg: ROSMessageT) -> None` for Protobuf message to ROS 2 message conversion
While convenient, [the mechanisms](https://pypi.org/project/multipledispatch) that enable these overloads do not play
along with static analyzers such as `mypy`. To workaround this limitation, each overload is also made available, fully
type annotated, under a unique name. This name is derived from argument type names as follows:
- `convert_{ros_package_name}_{ros_message_name}_message_to_{proto_package_name}_{proto_message_name}_proto` for ROS 2 message
to Protobuf message conversion
- `convert_{proto_package_name}_{proto_message_name}_proto_to_{ros_package_name}_{ros_message_name}_message` for Protobuf message
to ROS 2 message conversion
All message names above are [snake-cased](https://en.wikipedia.org/wiki/Snake_case). Note that user-defined overloads for ad-hoc
equivalences must follow the same pattern.
**Implementation note**: all explicit and implicit `_pb2` (i.e. Protobuf) Python imports must be available at generation time. This
requirement allows `proto2ros` to cope with an omission in Protobuf descriptor sets: these do not specify the mapping between fully
qualified Protobuf message names and their Python counterparts. To workaround this limitation, known `_pb2` modules are traversed
to reconstruct this mapping.
#### C++ APIs
Conversion APIs in C++ are exposed on a package basis as `{ros_package_name}::conversions::convert` overloads, available from
`{ros_package_name}/conversions.hpp` headers. For each pair of equivalent `ROSMessageT` and `ProtoMessageT` types, `proto2ros`
generates the following overloads:
- `void {ros_package_name}::conversions::Convert(const ROSMessageT& ros_msg, ProtoMessageT* proto_msg)` for ROS 2 message
to Protobuf message conversion
- `void {ros_package_name}::conversions::Convert(const ProtoMessageT& proto_msg, ROSMessageT* ros_msg)` for Protobuf message
to ROS 2 message conversion
Note that user-defined overloads for ad-hoc equivalences must follow the same pattern.
## Configuration
Both message and code generation are configured by a number of settings, listed below.
| Name | Description | Default value |
|---|---|---|
| `drop_deprecated` | Whether to drop deprecated fields on conversion or not. If not dropped, deprecated fields are annotated with a comment. | `False` |
| `passthrough_unknown` | Whether to forward Protobuf messages for which no equivalent ROS message is known as a serialized binary blob in a `proto2ros/AnyProto` field or not. | `True` |
| `message_mapping` | A mapping from fully qualified Protobuf message names to fully qualified ROS message names. This mapping comes first during composite type translation. | `{google.protobuf.Any: proto2ros/AnyProto, google.protobuf.Timestamp: builtin_interfaces/Time, google.protobuf.Duration: builtin_interfaces/Duration, google.protobuf.DoubleValue: std_msgs/Float64, google.protobuf.FloatValue: std_msgs/Float32, google.protobuf.Int64Value: std_msgs/Int64, google.protobuf.UInt64Value: std_msgs/UInt64, google.protobuf.Int32Value: std_msgs/Int32, google.protobuf.UInt32Value: std_msgs/UInt32, google.protobuf.BoolValue: std_msgs/Bool, google.protobuf.StringValue: std_msgs/String, google.protobuf.BytesValue: proto2ros/Bytes, google.protobuf.ListValue: proto2ros/List, google.protobuf.Value: proto2ros/Value, google.protobuf.Struct: proto2ros/Struct}` |
| `package_mapping` | A mapping from Protobuf package names to ROS package names, to tell where a ROS equivalent for a Protobuf construct will be found. Note that no checks for package existence are performed. This mapping comes second during composite type translation (i.e. when direct message mapping fails). | `{}` |
| `any_expansions` | A mapping from fully qualified Protobuf field names (i.e. a fully qualified Protobuf message name followed by a dot "." followed by the field name) of `google.protobuf.Any` type to Protobuf message type sets that these fields are expected to pack. A single Protobuf message type may also be specified in lieu of a single element set. All Protobuf message types must be fully qualified. | `{}` |
| `allow_any_casts` | When a single Protobuf message type is specified in an any expansion, allowing any casts means to allow using the equivalent ROS message type instead of a dynamically typed, `proto2ros/Any` field. For further reference on any expansions, see `Any types` section below. | `True` |
| `known_message_specifications` | A mapping from ROS message names to known message specifications. Necessary to cascade message generation for interdependent packages. | `{}` |
| `cpp_headers` | Set of C++ headers to be included (as ``#include <{header}>``) in generated C++ conversion headers. Typically, Protobuf and ROS message C++ headers. | `[]` |
| `inline_cpp_namespaces` | Set of C++ namespaces bearing conversion overloads, for which unqualified lookup (``using {namespace}::Convert``) is necessary in generated C++ conversion sources. | `[]` |
| `python_imports` | Set of Python modules to be imported (as `import `) in generated conversion modules. Typically, Protobuf and ROS message Python modules. | `[std_msgs.msg, proto2ros.msg, builtin_interfaces.msg, google.protobuf.any_pb2, google.protobuf.duration_pb2, google.protobuf.struct_pb2, google.protobuf.timestamp_pb2, google.protobuf.wrappers_pb2]` |
| `inline_python_imports` | Set of Python modules to be imported into module scope (as `from import *`) in generated conversion modules. Typically, conversion Python modules. | `[proto2ros.conversions.basic]` |
| `skip_implicit_imports` | Whether to skip importing Python modules for known Protobuf and ROS packages in generated conversion modules or not. These known modules are those derived from `.proto` source file names and the one homonymous to the ROS 2 package that hosts the generated interfaces. | `False` |
These defaults can be replaced entirely via configuration file or overridden one by one via _configuration overlays_. Configuration overlays are configuration files that update the baseline configuration, default or user-defined. Scalar values are replaced, lists are extended, dictionaries are updated (i.e. shallow merged).
## Use cases
### Dual Protobuf / ROS 2 package
A package may provide both Protobuf and ROS 2 messages, all generated from Protobuf definitions.
```cmake
cmake_minimum_required(VERSION 3.12)
project(proto2ros_tests)
find_package(ament_cmake REQUIRED)
find_package(builtin_interfaces REQUIRED)
find_package(rosidl_default_generators REQUIRED)
find_package(proto2ros REQUIRED)
find_package(rclcpp REQUIRED)
find_package(Protobuf REQUIRED)
# Generate Python code for some.proto
protobuf_generate(
LANGUAGE python
OUT_VAR proto_py_sources
PROTOS some.proto
IMPORT_DIRS proto
)
# Generate C++ code for some.proto
protobuf_generate(
LANGUAGE cpp
OUT_VAR proto_cpp_sources
PROTOS some.proto
IMPORT_DIRS proto
)
# Build generated C++ code
add_library(${PROJECT_NAME}_proto SHARED ${proto_cpp_sources})
target_include_directories(${PROJECT_NAME}_proto PUBLIC
"$<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>"
"$<INSTALL_INTERFACE:include/${PROJECT_NAME}>"
)
target_link_libraries(${PROJECT_NAME}_proto protobuf::libprotobuf)
# Add dependable target for generated Protobuf code
add_custom_target(
${PROJECT_NAME}_proto_gen ALL
DEPENDS ${proto_py_sources} ${proto_cpp_sources}
)
# Generate equivalent ROS 2 messages and conversion sources
proto2ros_generate(
${PROJECT_NAME}_messages_gen
PROTOS proto/some.proto
INTERFACES_OUT_VAR ros_messages
PYTHON_OUT_VAR ros_py_sources
CPP_OUT_VAR cpp_sources
INCLUDE_OUT_VAR cpp_include_dir
APPEND_PYTHONPATH "${CMAKE_CURRENT_BINARY_DIR}"
DEPENDS ${PROJECT_NAME}_proto_gen
)
# Generate ROS 2 message code.
rosidl_generate_interfaces(
${PROJECT_NAME} ${ros_messages}
DEPENDENCIES builtin_interfaces proto2ros
)
add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_messages_gen)
# Build C++ conversion library
add_library(${PROJECT_NAME}_conversions SHARED ${cpp_sources} src/manual_conversions.cpp)
target_include_directories(${PROJECT_NAME}_conversions PUBLIC
"$<BUILD_INTERFACE:${cpp_include_dir}>"
"$<INSTALL_INTERFACE:include/${PROJECT_NAME}>"
)
rosidl_get_typesupport_target(${PROJECT_NAME}_cpp_msgs ${PROJECT_NAME} "rosidl_typesupport_cpp")
target_link_libraries(${PROJECT_NAME}_conversions
${${PROJECT_NAME}_cpp_msgs} ${PROJECT_NAME}_proto protobuf::libprotobuf)
ament_target_dependencies(${PROJECT_NAME}_conversions builtin_interfaces proto2ros rclcpp)
# Install generated Python _pb2 and conversion code to the
# Python package that is implicitly defined and installed by the
# rosidl pipeline
rosidl_generated_python_package_add(
${PROJECT_NAME}_additional_modules
MODULES ${proto_py_sources} ${py_sources}
PACKAGES ${PROJECT_NAME}
DESTINATION ${PROJECT_NAME}
)
# Install generated C++ .pb.h and conversion headers
set(cpp_headers ${cpp_sources} ${proto_cpp_sources})
list(FILTER cpp_headers INCLUDE REGEX ".*\.hpp$")
install(
FILES ${cpp_headers}
DESTINATION include/${PROJECT_NAME}/${PROJECT_NAME}/
)
# Install C++ Protobuf messages and conversion libraries
install(
TARGETS
${PROJECT_NAME}_proto
${PROJECT_NAME}_conversions
EXPORT ${PROJECT_NAME}
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin
)
ament_export_dependencies(builtin_interfaces proto2ros rclcpp)
ament_export_targets(${PROJECT_NAME})
ament_package()
```
[`proto2ros_tests`](https://github.com/bdaiinstitute/proto2ros/tree/main/proto2ros_tests) is a good example of this.
### ROS 2 vendored Protobuf messages
Protobuf messages may already be provided by some third-party package, in which case, it is only the equivalent ROS 2 messages that are relevant.
For a third-party package and `.proto` files that are hosted on public repositories, the [`FetchContent`](https://cmake.org/cmake/help/latest/module/FetchContent.html)
module and the `proto2ros_vendor_package` CMake macro fully address this use case:
```cmake
cmake_minimum_required(VERSION 3.8)
project(vendored_third_party)
find_package(ament_cmake REQUIRED)
find_package(proto2ros REQUIRED)
# Fetch third party package sources (incl. .proto files)
include(FetchContent)
FetchContent_Declare(
third_party
GIT_REPOSITORY ...
GIT_TAG ..._
)
FetchContent_Populate(third_party)
# Collect third party .proto files
set(${PROJECT_NAME}_PROTO_DIR "${third_party_SOURCE_DIR}/protos")
file(GLOB ${PROJECT_NAME}_PROTOS "${${PROJECT_NAME}_PROTO_DIR}/*.proto")
# Generate ROS 2 messages and code (wraps rosidl)
proto2ros_vendor_package(${PROJECT_NAME}
PROTOS ${${PROJECT_NAME}_PROTOS}
IMPORT_DIRS ${${PROJECT_NAME}_PROTO_DIR}
)
ament_package()
```
[`bosdyn_msgs`](https://github.com/bdaiinstitute/bosdyn_msgs) is a good example of this.
|
|
|
ROS Index