The Lobster Leaves the ERP
Several months ago I published Rexclaw Companions for Odoo — AI companions embedded in Odoo that wreak havoc (the good kind) all over your ERP. It turns out a surprising number of people wanted the lobster without the ERP. So I ported it to a standalone desktop app, and along the way added the features I'd been itching to build: group calls, VR support, combo gestures, and a shareable avatar pack system. This post is about the port, the new toys, and the odd engineering problems in between.
The Port
The first decision was the tech stack, and the guiding principle was laziness of the productive kind: pick things that map one-to-one onto what already existed, so the engine code ports instead of being rewritten.
- Odoo's Python controllers and backend logic became FastAPI + uvicorn. A controller route is a controller route; the ORM's job went to plain SQLite — this is a single-user app, and shipping a database server with a desktop companion app would be peak over-engineering.
- The frontend moved from Owl to React + Vite. Odoo's component model is close enough to React that most of the JavaScript — the realtime voice client, the three.js avatar renderer, the tool dispatcher — ported almost line-for-line. The big files still track their Odoo counterparts closely enough that fixes port between them with a diff.
- In the Odoo module, three.js and @pixiv/three-vrm were loaded from a CDN at runtime. The standalone bundles them properly through npm/Vite — the app works fully offline (well, apart from the talking part), and nothing breaks when a CDN has a bad day.
Shipping Python in a Desktop App
This was the problem I expected to hate. Python famously does not want to be a desktop app: PyInstaller-style freezing is fragile, and "just install Python first" is where user interest goes to die.
The trick that made Electron feasible: Windows ships an official embeddable Python package — a ~15 MB self-contained interpreter distributed as a zip, no installer, no registry, no system state. The build script downloads it, installs the server dependencies into it, and electron-builder ships the whole thing as a resource. At launch, the Electron shell simply spawns python -m uvicorn server.main:app as a sidecar process and points its window at localhost. The frontend is the same Vite bundle the browser version serves; user data lands in %APPDATA%\Rexclaw.
No freezing, no import-hook archaeology — the actual Python running on the user's machine is the same interpreter layout I develop against. CI builds the zip on every release and butler pushes it to itch.io automatically, so itch-app users get delta patches of a few MB instead of a 300 MB re-download.
Group Calls and the Turn Director
The flagship new feature. Back in the Odoo version I made an early decision to implement the Grok realtime voice API from scratch rather than lean on an SDK — and that decision quietly paid for this entire feature. Since the client owns the WebSocket layer, a group call is "just" several connections at once: the companion you're speaking to holds the live audio leg, while every other participant joins as a deaf leg that receives the conversation as speaker-labelled text relays. No audio cross-feeding, no feedback loops, no double transcription bills.
The interesting problem isn't the plumbing — it's who speaks next. Every finished utterance, from you or from a companion, gets routed to a director agent whose entire job is to read the recent transcript and answer one question: who has the floor? The answer has to arrive in well under a second, or the conversation develops the awkward pauses of a bad video call — so the director runs on Grok's fast non-reasoning model with a compact set of priority rules distilled from actual turn-taking research (a named companion always wins; a companion addressing the user yields; the current floor holder continues). I did try a reasoning model here out of curiosity: correct answers, four seconds late, which in conversation terms is worse than wrong.
Combo Gestures
Group calls made the companions talk to each other; combo gestures make them move together. A combo stages a second character beside the performer and plays synchronized animation clips on both — the app ships with a synchronized backflip, which I can report is exactly as entertaining in a roleplay gym session as it sounds.
VR
Because the renderer is three.js in a browser, VR came cheap: WebXR gives headset support to the same codebase that draws the desktop view. It runs in fully virtual scenes or passthrough mixed reality, where you place companions around your actual room.
Then it stopped being cheap, because I fell down a hole: ragdoll physics via Jolt Physics compiled to WASM (lazy-loaded, since not everyone needs to knock their companion over), hand-tracking touch detection with body-region-aware reactions, and controller haptics. Yes — you can pat your companion on the head and she reacts happily. This is the feature people remember.
Avatar Packs
To make the whole thing feel less like my app and more like a platform: an avatar is now just a folder — a VRM model, an avatar.json manifest, and optionally outfits, custom gestures (solo or combo), and backgrounds. Drop a pack folder into the data directory and it appears in the app; zip it and it's shareable with anyone. The UI can duplicate a bundled avatar into an editable copy, so customizing starts from something that already works.
The Speed Round
- Memory recall was Postgres full-text + trigram search in Odoo; the standalone reimplements the trigram fuzzy matching in ~15 lines of pure Python — same typo-tolerance, zero database extensions.
- Workout gestures (push-ups, pike walks, squats) loop until told to stop, which combined with group calls enables the AI-personal-trainer scenario in the trailer below.
- One gh release create fans out to GitHub, a Docker image on GHCR, and the itch.io channel. Releases went from a chore to a automated.
Check It Out
Windows build is live on itch.io — free, open source, bring your own xAI key:
Source, Docker image, and the full README live on GitHub (https://github.com/Codemarchant/rexclaw) . And if you missed it, the original Odoo version has its own dev diary https://codemarchant.com/blog/dev-diaries-1/rexclaw-companions-i-put-a-half-lobster-in-my-erp-8
