Hearing
A local streaming recogniser: the words appear while you are still talking, and a short silence ends your turn.
NewPreview · v0.1.4
Aura
Aura gives an AI character emotions, memory and curiosity. When to speak, what to say and what to do are her own decisions — not a reply to a command.
ScrollIllustration — the numbers are examples
The heartbeat
When nobody is talking, she does not stop. The core beats on a steady rhythm.
Most assistants wait for a message. Aura keeps running: a heartbeat notices what happened, checks how she feels, and decides whether to speak — or to do nothing at all. The code describes; the model decides. Not one number about her feelings ever reaches it. Only words.
Architecture
EntrancesDiscord, her own Hub, voice and a web panel are the doors in.
Event busOne internal event bus carries everything, in and out.
AuraCoreThe core is only about who she is and how she thinks.
Activity interfaceGames speak one small WebSocket protocol.
ModulesEvery extra is a module, in a repository of its own.
Hover or tap any block.
python scripts/modules.py fetch.A conversation
From the moment you speak to the moment she answers.
Hear
Voice activity detection notices you speaking; Whisper turns the sound into text.
Notice
She works out whether the line was meant for her, and whether to join in.
Recall
Memory offers what it has about this.
Think
A language model writes the reply — local Ollama, or any OpenAI-compatible service.
Feel
Her mood right now shapes tone, pace and pitch.
Check
Before the words leave, the output filter reads them once more.
Speak
Speech starts while the reply is still being written.
Voice
The first sentence is on its way while the rest is still being written. And you can cut in whenever you like.
How was your day?
Illustration
A local streaming recogniser: the words appear while you are still talking, and a short silence ends your turn.
Kokoro on her own machine by default; a cloud voice or the system voice if you prefer. Nothing falls back to the cloud unless you allow it.
Her mood changes speed, pitch and loudness. A pronunciation list keeps names right.
Give her a song: the vocals are separated and turned into her voice. It is prepared ahead — she does not sing live.
Turn-taking
Every line is sorted before she answers. Joining in is a choice, and staying quiet is always allowed.
Pick a line to see how she sorts it.
Rules sort the clear cases; a quick model call settles the rest. Today a little chance still helps her decide whether to join in — the plan is less chance, more judgment.
Memory
What just happened stays short-term.
At night she goes through it, the way sleep does.
Who she is changes only after a waiting period and enough weight — strangers alone can never change it. Every change is versioned and can be rolled back.
Interfaces
Games, apps, voices and other AIs reach her through a few doors, in layers. The game door speaks the Neuro SDK protocol, so a game written for it connects as it is.
In a few quiet moments she imagines a sentence or two — always marked as a daydream, never told as fact.
She notices slang she doesn't know, tries it, adopts it after a few right uses — and lets it fade when nobody says it any more.
Who won the match last night?
Let me google that.
…found it. Want the score?
The answer comes back in the same turn — at most two searches, with a limit per hour.
A day
An imagined day — but every moment in it is something Aura does today. Keep scrolling: the hours go by.
Her first hours start with a note to herself: yesterday in short, what is due today, and last night's dream.
the dream's morning noteShe is told who came in, and may say hello in her own words. Never twice an hour to the same person.
new in v0.2.0Each voice is heard on its own and by name. A line meant for someone else is left alone.
turn-takingShe starts a game of Wordle by herself. The moment somebody speaks, the person comes first.
her own initiativeA craft fails for want of a plank. She goes for the plank instead of wandering off — and only claims what the inventory shows.
gamesA daydream in two sentences: a snack she wants, a cardboard box with a grudge. Marked as a daydream, so it is never told as fact.
imaginationTiredness has built up since morning. Her voice slows a little, her answers get shorter.
feelingsOnce it is quiet she goes over the day: keeps what mattered, lets the sting of a bad moment fade, and dreams a short, strange scene — shown to her in the morning as only a dream.
memoryPersona
Identity, traits, how she talks, where her limits are — in one plain text file. Her name is a setting.
{name} →
No name is written in the file: {name} is filled in from settings, so renaming her is one setting.
Emotion
Something happens, a feeling rises, then drifts back to where she usually is. The model never sees a number: only a few lines about how she feels.
Face calm
Pace ×1.06Pitch ×1.11
The rises, resting points and half-lives are the ones in the code, still being tuned. A repeated event counts for less; tiredness only builds while she is awake, and only sleep takes it away. Time runs ten times faster here.
irritated — “*hmpf*” · uneasy · down and quiet · exhausted · giddy
furious · terrified · hostile to the people she talks to
However high a feeling goes, the words stop there: “a bit irritated — not furious”, “a little scared — not terrified”. What she then does is the model's choice; the code only describes.
In VTube Studio — checked on a real one — she has 36 poses: smile, squint, wink, think, nod and more. What she says can ask for one, and so can you (“wink at me”). Her mood moves the rest, and lip sync follows her voice.
Games
Games plug in through a small WebSocket interface, compatible with the Neuro game SDK. What gets wired in is what she can see and what she can do.
Through a Fabric client mod she plays survival as a real player: mine, craft, build, fight.
Reads the game through tosu and comments on it. Plays herself only if you allow it.
Drives her avatar over OSC — walk, turn, emote, write in the chat box — and reads back how far she really moved.
Text channels, voice channels and slash commands.
Twitch viewers talk to her through the same entrance as Discord.
Other MCP servers' tools become her actions; other AIs can talk to her, even drive her.
She can start a game of Wordle by herself and play it to the end. If someone speaks, the person comes first.
Interfaces
Games, apps, voices and other AIs reach her through a few doors, in layers. The game door speaks the Neuro SDK protocol, so a game written for it connects as it is.
Who is connecting?
Safety
Safety runs both ways: she should not say what she must not, and she should not be treated badly.
Every line she says passes a word filter first — her replies, each spoken sentence, what she says on her own. Type a line, or pick one.
She saysThat was a great round, well played!
Inside the bounds: nothing to do.
It reads words, not meaning, so the last example is stopped too. A second check, off by default, has another model read the meaning of the whole line (hate, harassment, self-harm, personal information …) — before it is said and before it goes into her memory. Neither filters what people say to her; for that there is the rest button.
Two people press the rest button.
Two different reviewers within five minutes, by default. One alone is not enough.
Every door closes for 20 minutes.
Nothing said to her in that time is remembered. She keeps thinking, quietly.
She is asked, privately, what to forget.
The lines said to her in the last half hour. Her choice; by default nothing goes.
Then she comes back.
Ending it early takes ten reviewers' approval.
Deployment
Models, voice and games can live on different machines. Same code.
Everything together. The simplest start.
One runs Aura and the voice; the other runs only the game and a small adapter.
A server runs the models, each AI gets its own instance, and your own computer streams and develops.
Privacy and control
She says she is an AI when she joins a call. Anyone can ask her to forget them — and she decides.
Quality
The first release passed more than four thousand automated tests — and the running app was driven from outside, with a real local model, before it shipped.
In active development. The first release, v0.1.0, is out, and the v0.1.4 preview is the newest: tried on one machine, not yet with real people in a real call.
The core, voice from ear to mouth, memory and the nightly dream, the rest button, the Hub, the VTube Studio and VRChat drivers, three game modules, and deployment from one machine to many.
Cut in on her and she finishes the sentence, then stops — and keeps what she meant to say next. “Forget me” is her own choice. When the model fails, she says so. Less dice, more judgment. A face that is not a stare, and a hello for people who join her call. Coding agents such as Claude Code and Codex can be connected.
Discord voice with real people, VRChat, osu! and Minecraft in the real games, runs of many hours, and her faces on other avatars.
From the plan in the repository. Nothing here comes with a date.
Today a few things are decided by dice — whether to greet, to join in, to use a name. The plan is to let her decide, guided by written rules, with hard limits in code only where a mistake would hurt.