Menlo

Drive from Python

Give the Python SDK a way in, install it on your computer, then stand, balance, walk, record and play sound from short scripts.

menlo-sdk drives Asimov 1 from Python on your computer: the same Damp, Stand and Walk sequence as the Cockpit, plus the camera, the microphone and the speaker. This page takes a freshly built robot from its gantry hook to walking under a script, and back. Everything on it comes from the example scripts in the SDK repository, which the Python SDK section explains line by line.

Nothing in the SDK is an emergency stop

Keep Asimov Manager open at the E-Stop whenever the robot stands, and the battery unit within reach to cut power (Stopping the Robot). Every command a script sends passes through the robot's safety layer, and the Cockpit and a paired gamepad outrank it: close the Cockpit and unpair the gamepad before you run a script.

Before You Start

  • The robot is assembled, powered and signed in to Asimov Manager, and you have driven it once from the Cockpit (Stand Up and Walk).
  • The robot hangs from its gantry hook with both feet on the floor and 2 m of clear floor ahead. The keyboard step also walks it backward and sideways: for that, keep clear floor all around it.
  • Your computer is on the same network as the robot and runs Python 3.12 or newer.

Choose How the SDK Connects

The SDK reaches the robot in one of three connection modes. Pick one now; the rest of the page tells you which step each one needs.

Connection modeCommands and state travelCamera and audioNeeds
udpover the robot's networknoUDP control turned on and state sent to your computer
hybridover the robot's networkyesthe same, plus an SDK credential
livekitthrough the robot's LiveKit roomyesan SDK credential with the Control role

Choose udp to drive only, hybrid to drive and use the camera and microphone from the same script, and livekit when your computer is not on the robot's network. Details are in Connection Modes.

Get an SDK Credential

The hybrid and livekit connection modes join the robot's LiveKit room with an SDK credential from Asimov Manager. Skip this step for udp.

  1. In Asimov Manager, open Developer in the sidebar. Under How a script connects, note the Manager URL: it is the address the SDK uses to reach the robot. You do not need the LiveKit URL or the Room. The SDK asks the robot for both each time it connects, and when the LiveKit URL reads localhost, the SDK uses the robot's address instead.
  2. Under Issue a credential, enter a Name for the computer that will use it, such as my-laptop. Issue one per computer, so you can revoke one without cutting off the others.
  3. Under What it may do, choose Control to drive the robot from your computer. Observe watches the camera and listens to the microphone only; the robot drops its commands.
  4. Select Issue credential, then copy the credential with the copy button next to it. It is shown only this once: the robot keeps the credential's ID, not the credential, so it cannot show it again. If you lose it, revoke it and issue another.
  5. The credential appears under Issued credentials with its role and ID. To take it back, select Revoke next to it; the computer holding it can no longer connect, and a session that is already running keeps working until its LiveKit token expires, at most 12 hours.

Turn On UDP Control

The udp and hybrid connection modes carry commands over the robot's network and the robot sends its state to one computer. Skip this step for livekit.

  1. In Asimov Manager, open Advanced Settings in the sidebar and find the Asimov Edge section.
  2. Set udp-control to on.
  3. Set udp-state-host to your computer's address on the robot's network. Leave udp-state-port at its default, 8851.
  4. Select Save & Restart Asimov Edge and wait for the robot to come back.

Install the SDK and Save the Robot

  1. On your computer, install the package:
    pip install menlo-sdk
  2. Save the robot:
    menlo setup
    The wizard asks for a name, the connection mode you chose, and what that mode needs: the robot's address for udp, the Manager URL from the Developer page and the credential for livekit, both for hybrid. It checks each field against the robot before it saves, so a wrong address or a revoked credential shows up here, not in a script.
  3. Check that the SDK sees the robot:
    menlo status
    The panel shows the robot mode, whether the state stream is fresh, the battery and any fault, and a READY or NOT READY badge with the first reason. It sends nothing.
  4. Get the example scripts:
    git clone https://github.com/menloresearch/menlo-sdk.git
    cd menlo-sdk/examples

Check the Robot from a Script

check.py connects, prints what stands in the way of a stand, a walk and a trajectory, and sends nothing. Run it before anything that moves the robot:

examples/check.pyView on GitHub ↗
# The SDK counts the 0.5 s a robot in STAND must be upright to arm from its own samples.
time.sleep(0.6)
print(f"robot mode {robot.get_state().mode.name}, armed {robot.armed}")
for action in ("stand", "move", "trajectory"):
    print(robot.preflight(action))  # "ready to move", or "not ready to move:" and why
python check.py

Every script that moves the robot runs the same check first and exits with the problems listed when the robot is not ready; the codes are in Check Before You Move.

Walk It Around

The scripts move the robot through the same three robot modes as the Cockpit's postures: DAMP, STAND and MOVE. STAND holds a pose without a balance loop, so the robot stays on its gantry hook until it is in MOVE; see Standing and Balancing.

  1. With the robot in DAMP on its gantry hook, stand it up:
    python stand.py
    The actuators take the standing pose and the script returns once the robot is armed: STAND held upright for 0.5 s.
examples/stand.pyView on GitHub ↗
try:
    # Checks the robot first. Returns once it reports STAND and has been upright for
    # 0.5 s (armed): the firmware accepts MOVE only after that.
    robot.stand()
except (NotReadyError, WaitTimeoutError) as exc:
    print(exc)  # what is wrong and what fixes it
    sys.exit(1)
print(f"robot mode {robot.get_state().mode.name}, armed {robot.armed}")
  1. Balance it:
    python balance.py
    The robot enters MOVE at zero velocity and the walking policy balances it in place.
examples/balance.pyView on GitHub ↗
try:
    # Checks the robot first when it is in STAND. Returns once it reports MOVE.
    robot.balance()
except (NotReadyError, WaitTimeoutError) as exc:
    print(exc)  # what is wrong and what fixes it
    sys.exit(1)
print(f"robot mode {robot.get_state().mode.name}, balancing in place")
  1. Slacken the gantry gradually until the robot carries its own weight.
  2. Drive it from the keyboard. The keyboard walks the robot backward, sideways and round as well as forward, so keep clear floor all around it:
    python keyboard.py
    KeyEffect
    w / swalk forward / back
    a / dstrafe left / right
    q / eturn left / right
    spacebalance in place
    tstand (from DAMP only)
    bdamp, after asking
    xquit with zero velocity
    Each key press holds its velocity for about 0.3 s, so releasing a key stops the robot. Press space somewhere level, with the robot not mid-stride, before you go on. walk.py does the same without the keyboard: 0.3 m/s forward for 3 s, then balance in place.
examples/keyboard.pyView on GitHub ↗
while True:
    show(robot, last)
    key = next_key(TICK_S)
    if key is None:
        continue
    if key in QUIT:
        break
    if key in MOVES:
        last = move(robot, key)
    elif key == " ":
        last = balance(robot)
    elif key == "t":
        last = stand(robot)
    elif key == "b":
        last = confirm_and_damp(robot, next_key)
  1. Support the robot: take up the gantry's slack so it holds the robot again.
  2. Damp it:
    python damp.py
    The script shows the robot mode and asks Is the robot supported? Damp now? [y/N]. Answer y only with the robot supported: every actuator stops holding its position and a free-standing robot falls.
examples/damp.pyView on GitHub ↗
# damp() is never refused, so there is no check here, only the question.
if not YES:
    mode = robot.get_state().mode.name
    question = f"Robot mode {mode}. Is the robot supported? Damp now? [y/N] "
    if input(question).strip().lower() not in ("y", "yes"):
        print("nothing sent")
        sys.exit(0)
try:
    robot.damp()  # returns once the robot reports DAMP
except WaitTimeoutError as exc:  # sent, but DAMP was not reported: the message says what next
    print(exc)
    sys.exit(1)
print(f"robot mode {robot.get_state().mode.name}")

The same four steps run from the terminal without a script: menlo stand, menlo balance, menlo walk --vx 0.3 --duration 3 and menlo damp; each one asks before it sends. See Command Line.

Record Video

The camera, the microphone and the speaker come through the robot's LiveKit room, so this step and the next two need the hybrid or livekit connection mode. The scripts send no motion command and run whatever robot mode the robot is in.

camera_and_audio.py saves a photo, then a 3 s clip as one JPEG per frame in clip/ with its sound in clip.wav, then plays a tone on the speaker:

pip install pillow
python camera_and_audio.py
examples/camera_and_audio.pyView on GitHub ↗
if robot.has("camera"):
    photo = robot.camera.photo()  # the next fresh frame, rgb8
    Path(PHOTO).write_bytes(photo.to_jpeg())
    print(f"saved {PHOTO}, {photo.width}x{photo.height}")
    clip = robot.camera.capture_clip(CLIP_S, audio=robot.has("microphone"))
    frames = clip.save_frames(CLIP_DIR)
    print(f"saved {len(frames)} frames to {CLIP_DIR}/ at {clip.fps:.1f} fps")
    if clip.audio:
        print("saved", clip.save_wav(CLIP_WAV))
else:
    print("this connection carries no camera; use hybrid or livekit")

if robot.has("speaker"):
    # 16-bit mono PCM: a sine wave at TONE_HZ for TONE_S.
    tone = b"".join(
        struct.pack("<h", int(8000 * math.sin(2 * math.pi * TONE_HZ * i / SAMPLE_RATE_HZ)))
        for i in range(int(SAMPLE_RATE_HZ * TONE_S))
    )
    robot.speaker.play_pcm(tone, sample_rate_hz=SAMPLE_RATE_HZ)
    print(f"played {TONE_S:.0f} s at {TONE_HZ:.0f} Hz")

Camera and Audio covers frames, clips and MP4 output.

Record the Microphone

record_audio.py records 3 s from the robot's microphone into microphone.wav:

python record_audio.py
examples/record_audio.pyView on GitHub ↗
if not robot.has("microphone"):
    print("this connection carries no microphone; use hybrid or livekit")
    sys.exit(1)
# Every chunk in order, as 16-bit PCM. chunks() raises WaitTimeoutError when the
# microphone says nothing for its timeout.
chunks: list[AudioChunk] = []
recorded_s = 0.0
for chunk in robot.microphone.chunks(timeout=5.0):
    chunks.append(chunk)
    recorded_s += chunk.duration_s
    if recorded_s >= SECONDS:
        break
with wave.open(PATH, "wb") as wav:
    wav.setnchannels(chunks[0].channels)
    wav.setsampwidth(2)  # 16-bit samples
    wav.setframerate(chunks[0].sample_rate_hz)
    wav.writeframes(b"".join(c.data for c in chunks))
print(f"saved {PATH}, {recorded_s:.1f} s at {chunks[0].sample_rate_hz} Hz")

Play a Sound on the Speaker

play_audio.py plays a 16-bit PCM WAV file, hello.wav, through the robot's speaker in 1 s chunks. It reads hello.wav from the directory you run it from; with no file there, it plays a one-second 440 Hz tone. Over hybrid and livekit, the credential needs the Control role for the speaker:

python play_audio.py
examples/play_audio.pyView on GitHub ↗
if not robot.has("speaker"):
    print("this connection carries no speaker; use hybrid or livekit")
    sys.exit(1)
if os.path.exists(PATH):
    with wave.open(PATH, "rb") as wav:
        if wav.getsampwidth() != 2:
            print(f"{PATH} is not 16-bit PCM; convert it first")
            sys.exit(1)
        rate, channels = wav.getframerate(), wav.getnchannels()
        pcm = wav.readframes(wav.getnframes())
    name = PATH
else:  # 16-bit mono PCM: one second of a 440 Hz sine wave
    rate, channels, name = 16_000, 1, f"a 440 Hz tone (no {PATH} here)"
    pcm = b"".join(
        struct.pack("<h", int(8000 * math.sin(2 * math.pi * 440 * i / rate)))
        for i in range(rate)
    )
# Sent in short pieces, in order: play_pcm() blocks until the room has taken each one,
# so the file plays at its own pace.
step = int(rate * CHUNK_S) * 2 * channels
for start in range(0, len(pcm), step):
    robot.speaker.play_pcm(pcm[start : start + step], sample_rate_hz=rate, channels=channels)
print(f"played {name}, {len(pcm) / (2 * channels * rate):.1f} s at {rate} Hz")
  • Python SDK: every page of the SDK guide
  • Quickstart: the stand, balance and walk scripts, line by line
  • Safety: what the SDK checks, what it does not, and the watchdogs
  • Stand Up and Walk: the same sequence from the Cockpit
  • Asimov Manager: where credentials live

How is this guide?

On this page