Work / 01

M.I.R.A.: Modular Interactive Robotic Agent

A software-first embodied assistant built with Python and PySide6: an animated face driven by a renderer-independent embodiment model, a modular cognition layer with safe local actions, and an ambient notch interface in development. It is a desktop prototype, not a hardware robot yet, designed as the foundation for a future robotic body.

My contribution: personal project; design, implementation and tests are my own. The published repository holds the main branch; the notch, voice activation and camera work are on development branches that are not yet published.

1 Overview

M.I.R.A. explores the interaction and behavior layer of an assistant before it has a physical body. A face with expressive eyes shows what the assistant is doing (idle, listening, thinking, speaking) and how an action went, while a cognition layer turns typed requests into intents, validated local actions and short replies.

The embodiment is modelled independently of the renderer: motions, gaze and expressions are computed as immutable frames that any surface can draw. The main window and the notch already share one embodiment this way, and the same separation is what a future robot head would plug into.

Fig. 1 Main window: idle blinks and gaze drift, a typed greeting and a time request handled by the rule-based engine, with the face moving through thinking and speaking states and the action status shown below the conversation. Replies are in Italian. Rendered from the running application on Qt's offscreen platform; 12 s, no sound.

2 Notch interface

The notch is an ambient surface: a small frameless overlay at the top of the screen that carries the same eyes as the main window. It stays compact (200 × 32 px) and expands (400 × 112 px) with an animated transition on hover, on system notices (devices, audio, connectivity, running tasks), and while listening, when it shows the live microphone waveform. Right-click opens controls drawn inside the notch, and a double-click opens the main window.

Its geometry, eye layout and behavior are configurable through a versioned settings file and a live tuning window, and an eye-bounds check keeps every eye pose inside the notch outline.

Fig. 2 Notch: compact state, expansion on pointer hover, blink, and collapse. No notice is active, so the expanded notch shows only the eyes. Rendered offscreen over a neutral background; on the desktop it sits at the top edge of the screen. 5 s.

Development branch. The notch, system notices and voice activation are implemented and tested on a development branch and are not yet merged into the main branch on GitHub. The notch targets Linux on X11.

3 Architecture

Requests flow from the user interface to the cognition layer and back as state changes. Inference runs off the UI thread; its side effects are committed afterwards in a single serialized step, and results that arrive after a newer request are discarded. Only one component commits the visible activity state, and import-layering and state-authority rules are checked automatically.

Chat panel or notchTyped input; the surfaces present one shared embodiment
BrainAsynchronous request handling, off the UI thread
Intent engineRule-based, or a local LLM with an allowlisted schema and fallback to the rules
Action policy and executorValidated contracts and an allowlist of local actions
Response builder, session memoryReply text and in-session context
Activity authorityThe single writer of the visible state, through the event bus
EmbodimentRenderer-independent frames drawn by the main window and the notch

Fig. 3 Path from user input to the rendered state, as implemented.

4 Key features

**Table 1** Features and where they are implemented
Feature	Description	Where
Expressive eyes	Blink, idle wander, cursor-following gaze, thinking and speaking motion, and expressions (neutral, happy, confused and others). On the development branch these are computed by a Qt-free micro-motion and gaze model	GitHub main
Semantic cues	Success, warning and error are shown through the eyes, and a running task through a narrowed, breathing "focused" look	Development
Intent engine	Rule-based parsing for deterministic commands (Italian), with an optional local LLM path through Ollama, confidence gating and fallback to the rules	GitHub main
Safe local actions	A registry, execution policy and executor run a small set of desktop actions behind explicit contracts; actions that need a confirmation are blocked until a confirmation flow exists	GitHub main
Session memory	Recent interaction context is kept within a session; nothing persists across sessions	GitHub main
Compact main window	Face and conversation, with a hidden debug drawer for expression tuning and diagnostics	GitHub main
Notch interface	Ambient compact and expanded states, notices, in-notch controls, one runtime per user shared by both surfaces	Development
Voice activation	Keyboard or optional wake-word activation, voice-activity detection and utterance segmentation; speech-to-text is not implemented yet	Development
Camera capture	Opt-in capture foundation on a background thread; frames are not analyzed yet. Face-presence detection exists on a separate experimental branch	Development

5 Validation and scope

The main branch on GitHub runs its test suite headless on GitHub Actions (Qt offscreen platform), together with the layering and state-authority checks. On the development branch, 71 test files cover action contracts and policy, the asynchronous brain contract, rule and LLM intent handling, the response builder, embodiment and micro-motion, eye rendering, notch geometry, transitions and hover, notice producers, and the audio and listening pipeline. Camera and microphone are replaced by test doubles, so no test opens a device.

Scope. M.I.R.A. is not a hardware robot yet. Speech-to-text, spoken replies, persistent memory and a perception-driven gaze are planned, not implemented.

6 Source code

[1] M.I.R.A. repository (GitHub)