Part of the [EMOS](https://github.com/automatika-robotics/emos) ecosystem [](https://opensource.org/licenses/MIT) [](https://www.python.org/downloads/) [](https://docs.ros.org/en/humble/index.html) [](https://discord.gg/B9ZU6qjzND) **The production-grade framework for deploying Physical AI** [**EMOS Documentation**](https://emos.automatikarobotics.com) | [**Developer Docs**](https://automatika-robotics.github.io/embodied-agents/) | [**Discord**](https://discord.gg/B9ZU6qjzND)

What is EmbodiedAgents?

EmbodiedAgents is the intelligence layer of the EMOS (Embodied Operating System) ecosystem. It enables you to create interactive, physical agents that don’t just chat, but understand, move, manipulate, and adapt to their environment.

For full documentation, tutorials, and recipes, visit emos.automatikarobotics.com.

Key Features

• Production Ready – Robust orchestration layer built on native ROS 2. Deploy Physical AI that is simple, scalable, and reliable.

• Self-Referential Logic – Agents that are self-aware. Start, stop, or reconfigure components based on internal or external events. Switch between cloud and local ML on the fly.

• Run Fully Offline – Built-in local models for LLM, VLM, STT, and TTS. No server required. Optimized for edge devices and NVIDIA Jetson.

• Spatio-Temporal Memory – Hierarchical spatio-temporal memory and semantic routing. Build arbitrarily complex graphs for agentic information flow.

Quick Start

Create a VLM-powered agent that can answer questions about what it sees:

from agents.clients.ollama import OllamaClient
from agents.components import VLM
from agents.models import OllamaModel
from agents.ros import Topic, Launcher

text0 = Topic(name="text0", msg_type="String")
image0 = Topic(name="image_raw", msg_type="Image")
text1 = Topic(name="text1", msg_type="String")

qwen_vl = OllamaModel(name="qwen_vl", checkpoint="qwen2.5vl:latest")
qwen_client = OllamaClient(qwen_vl)

vlm = VLM(
    inputs=[text0, image0],
    outputs=[text1],
    model_client=qwen_client,
    trigger=text0,
    component_name="vqa"
)

launcher = Launcher()
launcher.add_pkg(components=[vlm])
launcher.bringup()

Run Fully Offline

Every AI component can run with a built-in local model – no server, no cloud, no heavy frameworks. Just set enable_local_model=True:

```python from agents.components import LLM from agents.config import LLMConfig from agents.ros import Topic, Launcher

config = LLMConfig( enable_local_model=True, device_local_model=”cpu”, # or “cuda” ncpu_local_model=4, )

llm = LLM( inputs=[Topic(name=”user_query”, msg_type=”String”)], outputs=[Topic(name=”response”, msg_type=”String”)], config=config, trigger=Topic(name=”user_query”, msg_type=”String”), component_name=”local_brain”, )

File truncated at 100 lines see the full file

Changelog for package automatika_embodied_agents

0.6.0 (2026-03-21)

• (docs) Adds advanced guides
• (docs) Adds developer docs
• (refactor) Adds sherpa-onnx for stt and tts local models
• (refactor) Adds ggml based local llm and vlm models with llama cpp
• (feature) Adds config option to strip think tokens
• (fix) Fixes execution provider setup when executor is set to be cpu
• (refactor) Standardizes local model handling in vision model
• (docs) Updates fallback example and adds new local model fallback recipe
• (feature) Adds fallback_to_local as a model component action for switching to local model on the fly
• (chore) Adds markers for running tests that require model serving
• (chore) Adds comprehensive test coverage for components
• (fix) Fixes bug in model components
• (fix) Fixes init of local LLM in semantic router for multiprocessing setup
• (feature) Adds local models for STT and TTS
• (feature) Enables local llm in semantic router component
• (feature) Adds moondream as local vlm for VLM component
• (feature) Adds local llm model for LLM component
• (fix) Adds extra check in rgbd callback
• (refactor) Adds warning for timed components when calling take_picture or record_video
• (docs) Updates vlm based planning recipe
• Contributors: ahr, mkabtoul

0.5.1 (2026-02-16)

• (feature) Adds a record_video action to the Vision component
• Takes specific input topic name and duration to record video
• (feature) Adds a take_picture action to the vision component
• (feature) Adds arbitrary action execution to router
• (feature) Adds handling of topic and action lists in semantic router
• (refactor) Makes action methods in Vision component use extra_callback to avoid starvation
• (refactor) Increases default max new tokens param for llm/vlm configs
• (refactor) Makes changes for new api of events/actions in sugar
• (fix) Fixes semantic router handling of action routes based on new api
• (fix) Fixes warmup for model components
• (chore) Adds think option for ollama models
• (chore) Adds proper docstrings to exposed methods and classes
• (docs) Updates model definitions in examples
• (docs) Adds new prompt for llms.txt and new tutorial intros
• (docs) Fix incorrect import in docs
• (docs) Updates installation instructions
• Contributors: ahr, mkabtoul

0.5.0 (2026-01-19)

• (docs) Adds supported types table and component runtype in basics
• (docs) Adds VLA tutorial to docs and refactors them
• (docs) Adds tutorials and examples for using events/fallbacks
• (refactor) Enables health broadcast by default
• (feautre) Adds feature for passing additional model clients to components Component can switch model clients based on events
• (feature) Adds external events as component triggers for certain components that can be run in a timed way
• (feature) Adds stt and tts endpoints and generic model wrappers for the generic http client (openai compatible)
• (feature) Enables support for tool calls in generic http client
• (feature) Adds llms.txt in docs
• (docs) Adds explanation of both routing modes to semantic router docs
• (feature) Adds LLM based agentic routing to semantic router
• (feature) Adds timeout for event based termination if the event doesnt happen within max_timesteps
• (feature) Create observation spec form mapping dicts if dataset info.json is not provided
• (feature) Adds publisher for joint state msg
• (feature) Adds parameter to regulate observation sending rate
• (fix) Adds various fixes for running in multiprocessing (vla)
• (feature) Adds dynamically adjusting loop rate to action server (vla)
• (feature) Adds termination logic to the VLA action server
• (feature) Adds capping of actions based on joint limits before publishing
• (feature) Adds action sending in VLA for multiple datatypes
• (feature) Adds a setter for passing an external aggregation function
• (feature) Adds mapping between LeRobot dataset keys and robot urdf and topics
  • Adds warning and errors for mapped keys, topics
  • Adds creation of input for VLA
  • Adds action receiving and aggregation
• (feature) Adds utility for reading joint limits from URDF files/urls
  • Adds mapping utility function in internal joint state class
• (fix) Adds destruction of stream timer if its created for async websocket clients
• (docs) Adds documentaion for lerobot client
• (feature) Adds an async grpc client for lerobot
• (docs) Removes onnxruntime as a mandatory dependancy, its only required for local models
• (feature) Adds LeRobotPolicy wrapper in models Adds utility for extracting features and actions from info.json
• (feature) Implements replacing input topics to handle trigger

File truncated at 100 lines see the full file