Menlo Robot SDK

Python client for the Menlo robot platform — robots, sessions, room keys, and live robot control

Overview

Work in Progress

menlo-robot-sdk is under active development (currently 0.2.x). APIs may evolve between releases — 0.2.0 changed invoke() to return the terminal ActionResult instead of a bare action id.

menlo-robot-sdk is the Python client for the Menlo robot platform. Use it from agents, scripts, and backend services to create robots, manage sessions, create room keys for the browser viewer, and drive robots live.

Guided notebook

The guided Colab notebook walks the complete flow end to end: install from PyPI, paste an API key, create a robot, get a room key, and drive it.

Open minted viewer URLs in Google Chrome and keep exactly one viewer tab open per robot.

Installation

Install from PyPI with uv or pip.

Configuration

An sk_live_… API key from the platform + the RCS URL.

Robots API

CRUD robots, sessions, participant tokens.

Drive a robot

connect() → discover skills → invoke() → ActionResult.

Installation

uv add "menlo-robot-sdk[livekit]"      # uv
pip install "menlo-robot-sdk[livekit]"  # pip

The base package (no extra) covers the control plane — robots, sessions, room keys. The [livekit] extra adds live robot control (connect() runtime methods, get_vision).

Verify:

python -c "import menlo_robot_sdk; print(menlo_robot_sdk.__version__)"

Architecture

Your code → menlo-robot-sdk ── HTTP ──→ api.menlo.ai/rcs   (control plane: robots, sessions, tokens)
                     │
                     └────── LiveKit SFU (runtime.* RPC) ──→ robot runtime   (drive, state, vision)

The control plane authenticates your API key and manages records. Robot control happens through the live session — the SDK joins and calls the in-room runtime worker (for SimpleSim, the browser viewer tab is the runtime).

Configuration

API key (recommended)

Create an sk_live_… key in the console (Settings → API keys), then:

Setting	Value
Console	https://platform.menlo.ai
RCS URL	`https://api.menlo.ai/rcs`

export MENLO_API_KEY=sk_live_...
export MENLO_RCS_URL=https://api.menlo.ai/rcs

from menlo_robot_sdk import AsyncClient

client = AsyncClient()                          # reads MENLO_RCS_URL + MENLO_API_KEY
client = AsyncClient(rcs_url=..., api_key=...)  # or pass explicitly

The platform edge verifies the key and mints the signed identity — the SDK never handles identity tokens in this mode. Keep keys in env vars or a secrets manager; never commit them. (repr() of SDK objects redacts credentials.)

Environment variables

Variable	Description
`MENLO_API_KEY`	`sk_live_…` key from the platform (API-key mode)
`MENLO_RCS_URL`	Platform RCS URL (default: `https://api.menlo.ai/rcs`)
`MENLO_ROBOT_SDK_TIMEOUT`	Per-request timeout in seconds (default `30`)
`MENLO_ROBOT_SDK_MAX_RETRIES`	Max retries on transport errors (default `2`)
`MENLO_ROBOT_SDK_DEBUG`	Set to `1` to log httpx request/response events (never logs headers or bodies)
`MENLO_ROBOT_VIEWER_URL`	Viewer base URL for room-key links (default: `https://sim.menlo.ai`)

Constructing `Client` / `AsyncClient`

No kwargs → loads everything from env (MENLO_RCS_URL + MENLO_API_KEY).

Explicit kwargs → pass rcs_url and api_key.

timeout / max_retries kwargs override the env defaults.

Quickstart

Install the package, create an API key in the platform, then:

Create a robot and session

import asyncio

from menlo_robot_sdk import AsyncClient, connect

async def main() -> None:
    async with AsyncClient() as client:   # MENLO_RCS_URL + MENLO_API_KEY from env
        created = await client.robots.create(name="My bot", model="asimov-v0")
        robot_id = created.robot.id
        print(created.pin_code)  # show-once PIN — save securely

        session = await connect(
            client,
            robot_id,
            worker_names=[],                  # SimpleSim: the browser viewer is the runtime
            rcw_identity_prefix="simplesim",
            join_livekit=True,                # needs the [livekit] extra
        )

Get a room key and open the viewer

        from menlo_robot_sdk.experimental import generate_room_key

        key = await generate_room_key(client, robot_id)
        print(f"https://sim.menlo.ai/?key={key}")   # open in Chrome — that tab IS the robot

The room key is a short-lived (4h) viewer credential scoped to this one room — share it like a meeting link, don't post it publicly. It contains no API key.

Drive it

        skills = await session.discover_skills()  # wait for the viewer to join first
        result = await session.invoke(
            "go_to",
            {"target": {"kind": "entity", "entity_id": "pad_B"}},
            timeout_s=300,
        )
        print(result.status, result.error)        # "done" | "failed"

        jpeg = await session.get_vision("pov")    # what the robot sees, as JPEG bytes

        await session.disconnect()
        await client.robots.delete(robot_id)

asyncio.run(main())

For a runnable, cell-by-cell version of this flow, open the Colab notebook.

Drive a robot

connect() wraps session creation and the LiveKit join, returning a MenloSession:

from menlo_robot_sdk import AsyncClient, ConnectCallbacks, connect

async with AsyncClient() as client:
    session = await connect(
        client,
        "rb_myrobot",
        worker_names=[],                   # or ["sim-worker"] to dispatch a server-side worker
        rcw_identity_prefix="simplesim",   # how to find the browser runtime in the room
        join_livekit=True,
        callbacks=ConnectCallbacks(on_action_result=lambda e: print(e.phase, e.action_id)),
    )

    skills = await session.discover_skills()
    status = await session.state.get("robot_status")

    result = await session.invoke("set_velocity", {"wz": 0.8, "duration_s": 2.0})
    # invoke() blocks until the action resolves and returns the terminal ActionResult:
    #   result.status          "done" | "failed"
    #   result.error           code/message when failed (e.g. NAVIGATION_STUCK, CANCELLED)
    #   result.meta.action_id  the action id

    await session.cancel_action("cancel-active")   # cancel the active action
    await session.disconnect()                     # closes the room + deletes the session

Behavior	Status
Session create + `disconnect()` (rolls back cleanly on partial failures)	Works
`join_livekit=True` + `ConnectCallbacks` lifecycle hooks	Works with the `[livekit]` extra
`invoke()` → terminal `ActionResult`; `on_action_result` fires with the same event	Works
`MenloSession.state.get` / `state.list`, `discover_skills`, `cancel_action`, `get_vision`	Work via the in-room runtime over the SFU
`environment_id` / `scene_id`, `get_logs`	Blocked

Long actions: invoke()'s RPC waits for the action to finish — pass timeout_s= above the expected duration (or set a session default with connect(runtime_call_timeout_s=...)). A timeout raises the standard TimeoutError; the action may still be running on the robot.

Camera capture (`get_vision`)

On-demand JPEG bytes from the robot runtime — not a continuous video stream. Requires join_livekit=True and a runtime in the room (for SimpleSim: an open Chrome viewer tab).

pov = await session.get_vision("pov")       # default 1280×720, quality 0.7
top = await session.get_vision("topdown", width=640, height=480)
assert pov[:3] == b"\xff\xd8\xff"

Byte-stream transport (no RPC size cap)

The runtime.cameras_capture RPC only triggers the capture; the runtime returns the JPEG over a LiveKit byte stream (topic menlo.camera_capture), so full-resolution frames work without the 15 KiB RPC payload cap. Capture renders on the runtime's main thread — prefer smaller sizes for high-frequency polling.

Robots API

Both Client and AsyncClient expose the same methods on client.robots:

Method	HTTP	Notes
`create(name=, model=, ...)`	`POST /v1/robots`	Returns `pin_code` once — store it securely
`list(limit=, after_id=)`	`GET /v1/robots`	One page; use `next_id` for manual pagination
`iter(limit=)`	`GET /v1/robots`	Async/sync iterator over all robots
`get(robot_id)`	`GET /v1/robots/{id}`
`update(robot_id, ...)`	`PATCH /v1/robots/{id}`	Partial update — only kwargs you pass are sent
`delete(robot_id)`	`DELETE /v1/robots/{id}`	Returns `None` (`204`)
`create_session(robot_id, worker_names=)`	`POST /v1/robots/{id}/session`	Returns session URL, participant token, and room metadata
`delete_session(robot_id)`	`DELETE /v1/robots/{id}/session`	Returns `None` (`204`)
`create_participant_token(robot_id, participant_identity=, ...)`	`POST /v1/robots/{id}/session/participant-token`	Extra room credential (what room keys are built from); TTL capped at 6h server-side

Worker dispatch

create_session(robot_id, worker_names=[...]) is the dispatch contract: the platform dispatches agents per name.

SimpleSim (browser runtime): worker_names=[] — no dispatch; the viewer tab fills the runtime slot by presence.

Omitted entirely: the platform applies its configured default dispatch.

Deprecated

The legacy workers=WorkerConfig(...) parameter (and workers.for_rcs_stack()) no longer drives dispatch and emits a DeprecationWarning — use worker_names.

Room keys (viewer links)

A room key packs a short-lived viewer credential + the session URL into one compact string the SimpleSim viewer understands — as a ?key= link or pasted into the viewer's connect gate.

from menlo_robot_sdk.experimental import generate_room_key, generate_room_key_url

key = await generate_room_key(client, robot_id)                   # the compact key
url = await generate_room_key_url(                                 # ready-to-open link
    client, robot_id, viewer_base_url="https://sim.menlo.ai",      # or MENLO_ROBOT_VIEWER_URL
)

Default TTL 4 hours (server cap 6h). Scoped to the one robot room; contains no API key.

It is a publish-capable room credential (the viewer acts as the robot runtime) — treat it like a meeting link, not a public URL.

The platform automatically includes the viewer connection URL in the room key.

Sync variants exist (generate_room_key_sync, …), plus the legacy long-URL helper get_session_viewer_url.

Viewer caveats: Google Chrome only; one tab per robot; don't reload the tab (the credential bakes a fixed identity — create a fresh key instead).

Errors and retries

All SDK-defined exceptions derive from MenloSDKError:

Exception	When	Useful fields
`MenloAPIError`	HTTP ≥ 400 from the platform	`.status_code`, `.code`, `.message`, `.request_id`
`RcwRuntimeError`	A `runtime.*` call failed at the robot runtime	`.code`, `.runtime_message`
`MenloNotAvailableError`	Feature blocked on upstream work	`.feature`, `.issue`

Timeouts are intentionally the stdlib TimeoutError (RPC replies, wait_until_ready, capture streams) — not under the umbrella. Platform-edge error bodies ({"error": …}) and FastAPI validation errors ({"detail": …}) are normalized into MenloAPIError, so an expired key reads http.401: unauthorized instead of an unknown error.

Retries (default 2) apply to transport failures and 429 on safe methods. Never retried (non-idempotent): POST /v1/robots, POST …/session, POST …/session/command, POST …/session/participant-token. Backoff is exponential with jitter, capped at 30s.

OpenAPI

When RCS is running, OpenAPI is available at {RCS}/openapi.json. The SDK ships a committed snapshot for typed models and contract checks during development.

Asimov API — commands and telemetry after you join a robot session

Menlo Robot SDK

Installation

Configuration

Robots API

Drive a robot

On this page