Kumohime · Public beta
How we turned KumoKodo's cyberpunk-anime mascot into a productized, multi-tenant platform for live talking-AI avatars — a self-hosted Claude brain, a cloned ElevenLabs voice, browser-mic listening, page-awareness, and a three.js/VRM (or inline-SVG) rig that lip-syncs on the spot — all embeddable on any brand's site with a single snippet.
A talking avatar is three separate problems — a brain that decides what to say, a voice that says it, and animation that moves and lip-syncs to arbitrary speech in real time. The animation is the part everyone underestimates: her art is a flat, baked illustration, and a live avatar needs a rig — a puppet built to move — plus all the parts hidden in the original image (mouth visemes, the eye behind the open eye, the face behind the hair).
Veo, Sora, and Kling are clip generators — prompt in, a fixed video out. A chatbot says something different every reply, instantly; you can't render a fresh clip per message, and they don't lip-sync to specific words. Perfect for marketing loops, wrong for a live agent.
Simli, Tavus, HeyGen, and D-ID are tuned for photoreal humans — anime looks wrong and often runs as per-response video. The stylized / mascot / character-IP corner of the live-avatar market was left wide open. That's the wedge Kumohime takes.
A multi-tenant platform where every avatar is a rendered character that listens, thinks, speaks, and lip-syncs live — never a canned clip — grounded in its own brand's knowledge.
The browser's Web Speech API hears the visitor on-device (audio never uploaded), a self-hosted Claude brain writes the reply, a cloned ElevenLabs voice speaks it, and the character lip-syncs to that audio on the spot via a viseme timeline built from ElevenLabs character timestamps.
A 3D path (three.js + GLTFLoader + meshopt rendering a VRM character delivered as GLB, with a Mixamo → VRM bone-map retarget baked to animation clips) and a dependency-free 2D inline-SVG path — both drop in behind one vendor-agnostic AvatarHandle contract (speak / stream / emote / stop).
Each avatar answers strictly from its brand's own crawled knowledge base, remembers the conversation for follow-ups, and is page-aware — it can answer about whatever the visitor is currently looking at, used per-reply only and never logged.
Every avatar resolves per slug, so adding a customer is data and art — not a fork. Brands license the ready-made Kumohime character or commission a custom avatar from their own mascot, logo, or illustration, with AI generating the rig inputs (visemes, expressions, turnarounds).
Rather than build a RAG pipeline with embeddings and a vector store, Kumohime carries the entire ~70K-token site crawl in a one-hour-cached Claude system-prompt prefix. The whole brand knowledge base rides in context on every turn — no retrieval step, no relevance misses, no stale index. Anthropic prompt caching makes it cheap: a cached turn runs about $0.007 versus ~$0.07 uncached, a roughly 10× saving, and a Vercel cron keeps the function and the cache warm. Measured end to end: ~2.05s average response and ~2.0s time-to-first-speech.
Because the character is itself AI-generated, AI can produce the rig inputs — her visemes, eye states, brow expressions, and turnarounds — with the same character-consistency trick. That automates most of the tedious "redraw the hidden bits" labor that makes hand-drawn mascots expensive to rig.
On-device browser mic (Web Speech API) — audio never leaves the visitor's machine.
Self-hosted Claude brain, answering only from the brand's own crawled knowledge base.
A per-avatar cloned ElevenLabs voice, streamed for a fast time-to-first-speech.
Blink, gaze, smile, and gestures driven live off the speech audio — never a fixed clip.
Corner widget, full-page host, or kiosk — a single snippet drops an avatar onto any site.
Conversation memory in Neon for follow-ups, with 30-day retention and a privacy policy.
Meet Kumohime on kumohime.ai, or contact us to commission a custom avatar — your character, cloned voice, and knowledge base, embedded on your site.