Automated testing

The automated tests in scenario_test_tools provide a basic framework for mocked, scriptable and introspectable ROS services and action servers. These are operated from the same process so they can offer a correct, integral mock for various aspects of the robot. This allows to guide the high-level behavior into edge-cases and verify they are handled as intended.

The core idea is that a tests specifies a sequence of services/actions that the behavior should call. A test can inspect the request/goal and reply with e.g. success or failure and then check if the failure is handled correctly etc.

Framework

The core of the framework is the ScriptableBase, on which ScriptableActionserver and ScriptableServiceServer are based.

ScriptableBase allows to

• introspect past goals via received_goals. This is simply an list of all received goals.
  • This can be combined with the remember_goals context manager, which makes goals be forgotten before entering the context and after exiting again. Useful when you have multiple tests in sequence.
• set the next result to a to-be-received goal with the reply method. There are several similar methods:
  • reply_conditionally: which takes a callable that returns True or False based on the goal and sends back the corresponding result
  • await_goal + reply_directly. await_goal blocks and returns a goal once it’s received, so the goal can be inspected. Based on that, specify the reply with reply_directly.
• Set default results via default_result without bothering with reply* most of the time, e.g for actions that are expected to always success on the real robot, eg. for doing text-to-speech. If you want to override temporarily, use this syntax:

  with scriptable_thing.custom_reply():
      scriptable_thing.reply(SomeActionResult('non-default'))
  

Examples

The scripts directory contains several scripts:

• example_test.py and dummy_behavior.py work together. The dummy behavior will wait for commands to have a robot charge itself and plug itself in. The test sends these commands and checks that the behavior handles this command correctly.
  • Run the dummy_behavior with rosrun scenario_test_tools dummy_behavior.py
  • Then start the test with rosrun scenario_test_tools example_test.py --start_scenario=test_all
• move_base_success.py uses the default_result feature to have a move_base implementation that always succeeds after a given time
• example_move_base is a more elaborate version of the above, handling each goal a differently to show other features of the framework. First goal received succeeds, 2nd is aborted and 3rd is ignored. Further goals are not expected and not handled at all.

Changelog for package scenario_test_tools

0.6.35 (2024-04-30)

0.6.34 (2024-02-18)

0.6.33 (2023-11-06)

0.6.32 (2023-04-29)

0.6.31 (2023-01-04)

0.6.30 (2022-11-17)

0.6.29 (2022-07-29)

0.6.28 (2022-03-15)

0.6.27 (2022-01-12)

0.6.26 (2021-11-26)

0.6.25 (2021-08-02)

0.6.24 (2021-07-02)

0.6.23 (2021-07-01)

0.6.22 (2021-05-10)

0.6.21 (2021-04-06)

0.6.20 (2021-01-25)

0.6.19 (2020-12-02)

0.6.18 (2020-10-21)

0.6.17 (2020-10-17)

• Merge pull request #284 from fmessmer/test_noetic test noetic
• use setuptools instead of distutils
• Bump CMake version to avoid CMP0048 warning
• Merge pull request #283 from LoyVanBeek/feature/await_connections Allow scripts to await for ScriptableActionServers to be connected
• Allow scripts to await for ScriptableActionSeervers to be connected
• Contributors: Felix Messmer, Loy van Beek, fmessmer

0.6.16 (2020-03-18)

• Merge pull request #277 from LoyVanBeek/feature/document_move_base Document move base
• Sort CMakelist
• Document other scripts
• Give 'always_succeeding_move_base' a shorter name
• Allow to both abort a goal but also send a result with that abort signal
• fixup! Add another example for move_base
• Document example_move_base further
• Add another example for move_base
• make publishing transform configurable
• get params from private ns
• get timeout param
• use node name
• Catch exceptions when replying to show a better error message
• Log how long a eply will wait for a goal
• Set default timeout to None, to match the documentation
• Put move_base a private namespace
• Use the default_reply_delay in other reply-methods as well by default
• Make names for reply_delay and 'think' more consistent
• Make result_delay an optional parameter
• Add more docs for ScriptableActionServer
• Add example move base script
• Merge pull request #274 from LoyVanBeek/feature/scenario_test_tools Add new package for scenario test tools
• Fix formatting issue, passing too many args to str.format
• Make executeables installable
• Add license header to relevant files
• Add new package for scenario test tools
• Contributors: Felix Messmer, Loy van Beek, hyb

0.6.15 (2019-11-07)

0.6.14 (2019-08-07)

0.6.13 (2019-07-29)

0.6.12 (2019-06-07)

0.6.11 (2019-04-05)

0.6.10 (2019-03-14)

0.6.9 (2018-07-21 15:27)

0.6.8 (2018-07-21 14:47)

0.6.7 (2018-01-07)

0.6.6 (2017-07-17)

0.6.5 (2016-10-10)

0.6.4 (2016-04-01)

0.6.3 (2015-08-25)

0.6.2 (2015-06-17)

0.6.1 (2014-12-15)

0.6.0 (2014-09-18)

0.5.2 (2014-08-28)

0.5.1 (2014-03-27)