This is the active ROS2 branch of this repository. If your are looking for the old ROS1 version, checkout the noetic branch.

Mesh Tools

We introduce a set of tools to make 3D environment mesh representations accessible and manageable in ROS. We provide RViz tools to visualize and annotate huge meshes in combination with generated textures and different cost layers, which are based on the geometric analyses of the environment, or which represent different sensor readings, e.g. RGB image or even hyper-spectral image textures.

Introduction

Over the last years, the Knowledge Based Systems Group has developed an extensive set of tools to automatically generate triangle meshes from 3D laser scans. These tools are implemented in the freely available Las Vegas Surface Reconstruction Toolkit (LVR) and have been successfully applied in different robotic contexts.

With terrestrial and mobile laser scanning it is possible to create 3D point clouds of large environments in short time. However, the amount of collected data is too large to be processed on a mobile robot. To make this data available on mobile robots, we suggest to compute 3D triangle meshes from point cloud data as corresponding map representations. Such maps have several advantages over raw point cloud data and voxel representations. They overcome the discretization of voxel maps and deliver topologically connected surface representations, which ease segmentation, annotation and enable intuitive visualization.

However, to make the maps accessible in ROS, corresponding message definitions and tools for storing, handling and editing such maps are required.

Citation

Please reference the following papers when using mesh_tools in your scientific work.

@inproceedings{putz2019ecmr,
  title={{Tools for visualizing, annotating and storing triangle meshes in ROS and RViz}},
  author={P{\"u}tz, Sebastian and Wiemann, Thomas and Hertzberg, Joachim},
  booktitle={2019 European Conference on Mobile Robots (ECMR)},
  pages={1--6},
  year={2019},
  organization={IEEE}
}

@article{putz2021ras,
  title = {{The Mesh Tools Package – Introducing Annotated 3D Triangle Maps in ROS}},
  journal = {Robotics and Autonomous Systems},
  volume = {138},
  pages = {103688},
  year = {2021},
  doi = {https://doi.org/10.1016/j.robot.2020.103688},
  author = {P{\"u}tz, Sebastian and Wiemann, Thomas and Hertzberg, Joachim},
}

Message Definitions

We divided the mesh structure in its geometry, textures and cost layers. These different components are then associated using a UUID. This structure enables passing the geometry at first to RViz and other nodes. Theses nodes may analyze the geometry, e.g the roughness or height difference and compute corresponding mesh cost layers which are published as individual message with a corresponding mesh UUID. Thereby other nodes can associate these layers with a certain mesh object. In RViz a certain layer can then be selected by the user and visualized coloring the corresponding costs or displaying the textures. In the following gives an overview on most important mesh messages:

• MeshGeometry(Stamped): Geometric structure of a triangle mesh using a buffer vertices, normals and faces
• MeshVertexColors(Stamped): Vertex color information of a corresponding mesh
• MeshVertexCosts(Stamped): Vertex cost information of a corresponding mesh coloring the mesh using a rainbow color scheme.
• ClusterLabel: Grouping a set of triangles / faces of a corresponding mesh using the face indices, optional cluster label
• Texture: Texture using an Image and an ID
• Material: Color and optional texture ID
• VertexTexCoords: Texture coordinate
• MeshMaterials(Stamped): Combining the materials with texture coordinates and clusters of a corresponding mesh
• Feature: A feature at a specific location and its feature descriptor
• MeshFeatures: List of features for a corresponding mesh

RViz Plugins

We developed several RViz plugins to display 3D meshes together with different information layers (RGB textures, semantic labels, trafficability, etc.). It is possible to render the received meshes in different modes (lighting, materials and wireframe). The supported modes are selectable based on availability in the tree view. For path planning, we also implemented an interactive tool to set navigation goal poses on the mesh surface. To generate semantic maps, we implemented an interactive tool that can be used to select connected clusters of triangles and assign semantic labels to them. Some screenshots and videos demonstrate these plugins (see Supplementary Material).

RViz Mesh Plugin

The TexturedMesh displays a mesh with optional a fixed color, textures, vertex colors or vertex costs. Additionally, it can display the wire-frame as well as the vertex normals of the mesh. The MeshGoal Tool provides the possibility to select a geometry_msgs/PoseStamped on the surface of the mesh.

RViz Map Plugin

The Mesh Display displays labeled faces. The ClusterLabel Display displays labeled clusters. The ClusterLabel Tool enables the labeling of certain faces using different selection and de-selection methods as shown in a video (see Supplementary Material. The ClusterLabel Panel allows to manage your clusters and label names as well as the corresponding colors.

Videos

File truncated at 100 lines see the full file