|
robosoft_openai package from robosoft_openai reporobosoft_openai |
|
|
Package Summary
| Tags | No category tags. |
| Version | 0.0.0 |
| License | Apache-2.0 |
| Build type | AMENT_CMAKE |
| Use | RECOMMENDED |
Repository Summary
| Checkout URI | https://github.com/robosoft-ai/robosoft_openai.git |
| VCS Type | git |
| VCS Version | humble |
| Last Updated | 2024-07-16 |
| Dev Status | DEVELOPED |
| CI status | No Continuous Integration |
| Released | UNRELEASED |
| Tags | No category tags. |
| Contributing |
Help Wanted (0)
Good First Issues (0) Pull Requests to Review (0) |
Package Description
Additional Links
Maintainers
- andy
Authors
robosoft_openai
OpenAI server node for ROS2 Applications
What is this useful for?
If you don't have an image processing pipeline or a GPU on your robot, calls to OpenAI can replace much of that functionality. As of July 2024 we generally recommend yes/no questions, otherwise you will need to parse the response yourself. This package includes a simple filter which parses for yes/no. For example:
- Is it safe for the robot to drive forward? Please respond in one word, yes or no.
- Is the door open? Please respond in one word, yes or no.
- Is there a customer at that table? Please respond in one word, yes or no.
- Did the robot successfully grasp the cube? Please respond in one word, yes or no.
- Does the control panel look identical to the last time I checked? Please respond in one word, yes or no.
Some of these examples are useful for reinforcement learning, i.e. for determining if a reward was earned.
Setup
Requires an OpenAI key as described here: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
Set it as an environment variable: export OPENAI_API_KEY="..."
Dependencies
sudo apt install -y libcurl4-openssl-dev
sudo apt install -y nlohmann-json3-dev
sudo apt install -y libb64-dev
Run
Here's a quick example:
ros2 run robosoft_openai openai_server
Send it a prompt. Here's an example that returns a bool from a yes/no question. Note that the image field of the service request may be left empty...
ros2 service call /openai_server ai_prompt_msgs/srv/StringImagePrompt prompt:\ "Are you a pirate? Please respond with a one-word answer, yes or no"
Here's an example that returns a full string...
ros2 service call /openai_server ai_prompt_msgs/srv/StringImagePrompt prompt:\ "Are you a pirate?"
Here's an example client which sends an image of a wooden table and prompts whether it is indeed a wooden table:
ros2 run robosoft_openai example_client
Citation
If you use this work, please cite it like so:
- Zelenak, A., Lock, J., & Aldrich, B. (2024) An OpenAI Server for ROS2. Github. https://github.com/robosoft-ai/robosoft_openai
Wiki Tutorials
Package Dependencies
| Deps | Name | |
|---|---|---|
| 1 | ament_cmake | |
| 1 | ament_lint_auto | |
| 1 | ament_lint_common | |
| 1 | ai_prompt_msgs | |
| 1 | cv_bridge | |
| 1 | rclcpp | |
| 2 | sensor_msgs |
System Dependencies
Dependant Packages
Launch files
Messages
Services
Plugins
Recent questions tagged robosoft_openai at Robotics Stack Exchange
|
|
robosoft_openai package from robosoft_openai reporobosoft_openai |
|
|
Package Summary
| Tags | No category tags. |
| Version | 0.0.0 |
| License | Apache-2.0 |
| Build type | AMENT_CMAKE |
| Use | RECOMMENDED |
Repository Summary
| Checkout URI | https://github.com/robosoft-ai/robosoft_openai.git |
| VCS Type | git |
| VCS Version | jazzy |
| Last Updated | 2024-07-29 |
| Dev Status | DEVELOPED |
| CI status | No Continuous Integration |
| Released | UNRELEASED |
| Tags | No category tags. |
| Contributing |
Help Wanted (0)
Good First Issues (0) Pull Requests to Review (0) |
Package Description
Additional Links
Maintainers
- andy
Authors
robosoft_openai
OpenAI server node for ROS2 Applications
What is this useful for?
If you don't have an image processing pipeline or a GPU on your robot, calls to OpenAI can replace much of that functionality. As of July 2024 we generally recommend yes/no questions, otherwise you will need to parse the response yourself. This package includes a simple filter which parses for yes/no. For example:
- Is it safe for the robot to drive forward? Please respond in one word, yes or no.
- Is the door open? Please respond in one word, yes or no.
- Is there a customer at that table? Please respond in one word, yes or no.
- Did the robot successfully grasp the cube? Please respond in one word, yes or no.
- Does the control panel look identical to the last time I checked? Please respond in one word, yes or no.
Some of these examples are useful for reinforcement learning, i.e. for determining if a reward was earned.
Setup
Requires an OpenAI key as described here: https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key
Set it as an environment variable: export OPENAI_API_KEY="..."
Dependencies
sudo apt install -y libcurl4-openssl-dev
sudo apt install -y nlohmann-json3-dev
sudo apt install -y libb64-dev
Run
Here's a quick example:
ros2 run robosoft_openai openai_server
Send it a prompt. Here's an example that returns a bool from a yes/no question. Note that the image field of the service request may be left empty...
ros2 service call /openai_server ai_prompt_msgs/srv/StringImagePrompt prompt:\ "Are you a pirate? Please respond with a one-word answer, yes or no"
Here's an example that returns a full string...
ros2 service call /openai_server ai_prompt_msgs/srv/StringImagePrompt prompt:\ "Are you a pirate?"
Here's an example client which sends an image of a wooden table and prompts whether it is indeed a wooden table:
ros2 run robosoft_openai example_client
Citation
If you use this work, please cite it like so:
- Zelenak, A., Lock, J., & Aldrich, B. (2024) An OpenAI Server for ROS2. Github. https://github.com/robosoft-ai/robosoft_openai
Wiki Tutorials
Package Dependencies
| Deps | Name | |
|---|---|---|
| 1 | ament_cmake | |
| 1 | ament_lint_auto | |
| 1 | ament_lint_common | |
| 1 | ament_index_cpp | |
| 1 | ai_prompt_msgs | |
| 1 | cv_bridge | |
| 1 | rclcpp | |
| 2 | sensor_msgs |