Part of the [EMOS](https://github.com/automatika-robotics/emos) ecosystem [](https://opensource.org/licenses/MIT) [](https://www.python.org/downloads/) [](https://docs.ros.org/en/humble/index.html) [](https://discord.gg/B9ZU6qjzND) **The production-grade framework for deploying Physical AI** [**EMOS Documentation**](https://emos.automatikarobotics.com) | [**Developer Docs**](https://automatika-robotics.github.io/embodied-agents/) | [**Discord**](https://discord.gg/B9ZU6qjzND)

What is EmbodiedAgents?

EmbodiedAgents is the intelligence layer of the EMOS (Embodied Operating System) ecosystem. It enables you to create interactive, physical agents that don’t just chat, but understand, move, manipulate, and adapt to their environment.

For full documentation, tutorials, and recipes, visit emos.automatikarobotics.com.

Key Features

• Production Ready – Robust orchestration layer built on native ROS 2. Deploy Physical AI that is simple, scalable, and reliable.

• Self-Referential Logic – Agents that are self-aware. Start, stop, or reconfigure components based on internal or external events. Switch between cloud and local ML on the fly.

• Run Fully Offline – Built-in local models for LLM, VLM, STT, and TTS. No server required. Optimized for edge devices and NVIDIA Jetson.

• Spatio-Temporal Memory – Hierarchical spatio-temporal memory and semantic routing. Build arbitrarily complex graphs for agentic information flow.

Quick Start

Create a VLM-powered agent that can answer questions about what it sees:

from agents.clients.ollama import OllamaClient
from agents.components import VLM
from agents.models import OllamaModel
from agents.ros import Topic, Launcher

text0 = Topic(name="text0", msg_type="String")
image0 = Topic(name="image_raw", msg_type="Image")
text1 = Topic(name="text1", msg_type="String")

qwen_vl = OllamaModel(name="qwen_vl", checkpoint="qwen2.5vl:latest")
qwen_client = OllamaClient(qwen_vl)

vlm = VLM(
    inputs=[text0, image0],
    outputs=[text1],
    model_client=qwen_client,
    trigger=text0,
    component_name="vqa"
)

launcher = Launcher()
launcher.add_pkg(components=[vlm])
launcher.bringup()

Run Fully Offline

Every AI component can run with a built-in local model – no server, no cloud, no heavy frameworks. Just set enable_local_model=True:

```python from agents.components import LLM from agents.config import LLMConfig from agents.ros import Topic, Launcher

config = LLMConfig( enable_local_model=True, device_local_model=”cpu”, # or “cuda” ncpu_local_model=4, )

llm = LLM( inputs=[Topic(name=”user_query”, msg_type=”String”)], outputs=[Topic(name=”response”, msg_type=”String”)], config=config, trigger=Topic(name=”user_query”, msg_type=”String”), component_name=”local_brain”, )

File truncated at 100 lines see the full file

Changelog for package automatika_embodied_agents

0.7.0 (2026-04-11)

• (feature) Adds an optional output topic in cortex for capturing outputs when an action is not needed
• (docs) Updates docs for component actions setup
• (chore) Bumps up required sugarcoat version
• (fix) Fixes model class type check in roboml client
• (feature) Adds Action goal final status to Cortex feedback lines
• (fix) Fixes converter for detections message to handle empty detections
• (feature) Adds tracking action to vision component
• (fix) Fixes publishing to multiple output topics in model component
• (feature) Adds a safe restart context manager to base component
• (fix) Fixes sender for tracking based on new roboml api
• (chore) Adds CI workflow to run tests
• (feature) Adds a describe action to the vlm component
• (fix) Standardizes param name in POI publishing
• (fix) Increases default max token limits
• (fix) Fixes monitoring ongoing actions and feedback status update for step decision
• (fix) Fixes return value when sending action goal to component
• (feature) Adds async execution of action clients and monitoring using the main action loop
• (feature) Adds helper methods to monitor ongoing action clients and cancel their goals
• (fix) Adds fixes for python3.8 compatibility
• (refactor) Updates examples for new roboml api
• (feature) Enables registering component additional ROS entrypoints as system tools
• (feature) Updates model definitions and default checkpoints based on new version of RoboML
• (fix) Removes unnecessary config validator for stt
• (fix) Fixes empty buffer inference call in stt
• (refactor) Simplifies text to speech playback pipeline for less jitter
• (feature) Adds internal events setup to cortex launch
• (fix) Fixes say action for abrupt stopping of play on device thread
• (feature) Adds multistep planning loop
  • Separates planning and execution tools
  • inpection is used only in planning loop
  • execution handles failures when missed by llm
• (chore) Locks methods not implemented in VLA component
• (fix) Fixes tool calling for string args in llm component
• (fix) Fixes local model download cache paths
• (feature) Adds component config params inspection to _inspect_component
• (feature) Registers tools in Cortex for sending Goals to any available component Action Server
• (refactor) Adds a meaninful action name to the VLA component
• (feature) Adds running component actions as service call and captures their output for subsequent steps
• (feature) Adds tool registration from component tools and adds inspect component tool to cortex
• (feature) Adds tool descriptions to available component actions
• (feature) Adds planning and execution phases to cortex
• (feature) Extends the base Launcher to use the Cortex component as a monitor
• (feature) Adds Cortex, the master planning and execution node
• (refactor) Moves strip think token to utilities
• (feature) Adds feedback to VisionLanguageAction feedback msg
• Contributors: ahr, mkabtoul

0.6.0 (2026-03-21)

• (docs) Adds advanced guides
• (docs) Adds developer docs
• (refactor) Adds sherpa-onnx for stt and tts local models
• (refactor) Adds ggml based local llm and vlm models with llama cpp
• (feature) Adds config option to strip think tokens
• (fix) Fixes execution provider setup when executor is set to be cpu
• (refactor) Standardizes local model handling in vision model
• (docs) Updates fallback example and adds new local model fallback recipe
• (feature) Adds fallback_to_local as a model component action for switching to local model on the fly
• (chore) Adds markers for running tests that require model serving
• (chore) Adds comprehensive test coverage for components
• (fix) Fixes bug in model components
• (fix) Fixes init of local LLM in semantic router for multiprocessing setup
• (feature) Adds local models for STT and TTS
• (feature) Enables local llm in semantic router component
• (feature) Adds moondream as local vlm for VLM component
• (feature) Adds local llm model for LLM component
• (fix) Adds extra check in rgbd callback
• (refactor) Adds warning for timed components when calling take_picture or record_video
• (docs) Updates vlm based planning recipe
• Contributors: ahr, mkabtoul

0.5.1 (2026-02-16)

• (feature) Adds a record_video action to the Vision component
• Takes specific input topic name and duration to record video
• (feature) Adds a take_picture action to the vision component
• (feature) Adds arbitrary action execution to router
• (feature) Adds handling of topic and action lists in semantic router
• (refactor) Makes action methods in Vision component use

File truncated at 100 lines see the full file