This section tries to give guidance about how to improve the software quality of ROS2 packages. The guide uses a pattern language based approach to improve the readers experience (“read little, understand fast, understand much, apply easily”).
What this sections is about:
- ROS2 core, application and ecosystem packages.
- ROS2 core client libraries C++ and Python (right now: mainly C++)
- Design and implementation considerations to improve quality attributes like “Reliability”, “Security”, “Maintainability”, “Determinism”, etc. which relate to non-functional requirements (right now: mainly “Reliability”).
What this section is not about:
- Design and implementation considerations which go beyond a single ROS2 package and a single ROS2 node (means no integration considerations w.r.t. ROS2 graphs, etc.).
- Organizational considerations to improve software quality (an organizations structure and processes, etc.).
- Infrastructural considerations which go beyond a single repository (overall continuous integration infrastructure, etc.)
Relation to other sections:
- The Design Guide summarizes design patterns for ROS2 packages. As quality is highly impacted by design it is a good idea to have a look into there before.
- The Developer Guide explains what to consider when contributing to ROS2 packages w.r.t. to contribution workflow (organizational), coding conventions, documentation considerations, etc. All these consideration may have an impact on single or several quality attributes.
- Static code analysis
- Static code analysis using a single tool
- Static code analysis using multiple tools complementary
- Static code analysis as part of the ament package build
- Dynamic code analysis
- ROS2 library test
- ROS2 node unit test
- (generic use cases of
- (generic use cases of
Static code analysis as part of the ament package build¶
- You have developed your C++ production code.
- You have created a ROS2 package with build support with
- Library level static code analysis is not run as part of the package build procedure.
- Library level static code analysis needs to be executed manually.
- Risk of forgetting to execute library level static code analysis before building a new package version.
- Use the integration capabilities of
amentto execute static code analysis as part of the package build procedure.
- Insert into the packages
... if(BUILD_TESTING) find_package(ament_lint_auto REQUIRED) ament_lint_auto_find_test_dependencies() ... endif() ...
- Insert the
ament_linttest dependencies into the packages
... <package format="2"> ... <test_depend>ament_lint_auto</test_depend> <test_depend>ament_lint_common</test_depend> ... </package>
- The static code analysis tools supported by
amentare run as part of the package build.
- Static code analysis tools not supported by
amentneed to be executed separately.
Dynamic analysis (data races & deadlocks)¶
- You are developing/debugging your multithreaded C++ production code.
- You use pthreads or C++11 threading + llvm libc++ (in case of ThreadSanatizer).
- You do not use Libc/libstdc++ static linking (in case of ThreadSanatizer).
- You do not build non-position-independent executables (in case of ThreadSanatizer).
- Data races and deadlocks can lead to critical bugs.
- Data races and deadlocks cannot be detected using static analysis (reason: limitation of static analysis).
- Data races and deadlocks must not show up during development debugging / testing (reason: usually not all possible control paths through production code exercised).
- Use a dynamic analysis tool which focuses on finding data races and deadlocks (here clang ThreadSanatizer).
- Compile and link the production code with clang using the option
-fsanitize=thread(this instruments the production code).
- In case different production code shall be executed during anaylsis consider conditional compilation e.g. ThreadSanatizers _has_feature(thread_sanitizer).
- In case some code shall not be instrumented consider ThreadSanatizers _/*attribute*/_((no_sanitize(“thread”))).
- In case some files shall not be instrumented consider file or function level exclusion ThreadSanatizers blacklisting, more specific: ThreadSanatizers Sanitizer Special Case List or with ThreadSanatizers no_sanitize(“thread”) and use the option
- Higher chance to find data races and deadlocks in production code before deploying it.
- Analysis result may lack reliability, tool in beta phase stage (in case of ThreadSanatizer).
- Overhead due to production code instrumentation (maintenance of separate branches for instrumented/not instrumented production code, etc.).
- Instrumented code needs more memory per thread (in case of ThreadSanatizer).
- Instrumented code maps a lot virtual address space (in case of ThreadSanatizer).
Code coverage analysis¶
You have written tests for the library level production code of a ROS2 package (usually refered to as “unit tests”).
You do not know how much of the production code is exercised during the execution of the tests.
Select and use a code coverage analysis tool to determine the code coverage.
- Is it possible to integrate the tool with your source code editor?
- If not web service based: Is it possible to integrate the tool with your continuous integration infrastructure?
- What type(s) of coverage measurements (e.g. statement coverage) does the tool support?
- You know how much of your production code was exercised during the execution of the unit tests.
- You have a more or less immediate feedback about the code coverage (editor integration / web service front end).
- You do not know anything about the quality of your tests. (The only way to figure that out is some kind of review).