NewPreview · v0.1.4

Aura

She decides.

Aura gives an AI character emotions, memory and curiosity. When to speak, what to say and what to do are her own decisions — not a reply to a command.

Scroll
What is on her mindher choice
Just nowSomeone says: so tired today
  • Comfort them58%
  • Ask why27%
  • Tell a joke9%
  • Stay quiet6%
Just nowTen minutes of silence
  • Stay quiet52%
  • Hum a tune21%
  • Recall yesterday17%
  • Start a talk10%
Just nowShe just finished speaking
  • Follow up44%
  • Wait a little30%
  • Change topic16%
  • Say thanks10%

Illustration — the numbers are examples

The heartbeat

Always running.

When nobody is talking, she does not stop. The core beats on a steady rhythm.

  1. What happenedShe notices what just happened.
  2. How she feelsShe checks how she feels.
  3. Speak or notShe decides whether to speak.
  4. Do somethingShe does something — or nothing.

Under the hood

Main loop
perceive → think → feel → decide → act
When it is quiet
The autonomous engine takes over: think of something, start a talk, or stay silent.
After she speaks
The follow-up engine may add one more question on the same topic.
Energy
She tires. Sleep restores her, and at night she tidies her memory.
On her own
She speaks up when nobody asked — less often the longer nobody answers — and starts nothing when nobody could hear it. She can even start a game of Wordle by herself.
A person first
A spoken question is answered even while she is in the middle of her own game.

Most assistants wait for a message. Aura keeps running: a heartbeat notices what happened, checks how she feels, and decides whether to speak — or to do nothing at all. The code describes; the model decides. Not one number about her feelings ever reaches it. Only words.

Architecture

A core, and everything around it.

EntrancesDiscord, her own Hub, voice and a web panel are the doors in.

Event busOne internal event bus carries everything, in and out.

AuraCoreThe core is only about who she is and how she thinks.

Activity interfaceGames speak one small WebSocket protocol.

ModulesEvery extra is a module, in a repository of its own.

Hover or tap any block.

Modules
Activity interfaceWebSocket
AuraCore
Event bus
Entrances

Under the hood

Modules
Fetched with one command: python scripts/modules.py fetch.
The rule
A module uses only the core's public interface, never its internals.
Left out
A module that is not installed does not affect the rest.
Ways in
Discord, the Hub, a browser voice page, a console, Twitch chat and MCP. A Discord voice gateway in C# comes as source.

A conversation

One sentence. Seven steps.

From the moment you speak to the moment she answers.

  1. 1Hear
  2. 2Notice
  3. 3Recall
  4. 4Think
  5. 5Feel
  6. 6Check
  7. 7Speak

Hear

Voice activity detection notices you speaking; Whisper turns the sound into text.

Notice

She works out whether the line was meant for her, and whether to join in.

Recall

Memory offers what it has about this.

Think

A language model writes the reply — local Ollama, or any OpenAI-compatible service.

Feel

Her mood right now shapes tone, pace and pitch.

Check

Before the words leave, the output filter reads them once more.

Speak

Speech starts while the reply is still being written.

Under the hood

Streaming
The model writes while the voice already speaks — no waiting for the whole sentence.
Two voice paths
Classic: recognise → language model → synthesise. Direct: for models that take audio. If the direct path fails, the classic one takes over and the audio is not lost.

Voice

She speaks while she thinks.

The first sentence is on its way while the rest is still being written. And you can cut in whenever you like.

You

How was your day?

Aura

Writing
Speaking

Illustration

Hearing

A local streaming recogniser: the words appear while you are still talking, and a short silence ends your turn.

Voices

Kokoro on her own machine by default; a cloud voice or the system voice if you prefer. Nothing falls back to the cloud unless you allow it.

Delivery

Her mood changes speed, pitch and loudness. A pronunciation list keeps names right.

Singing

Give her a song: the vocals are separated and turned into her voice. It is prepared ahead — she does not sing live.

Under the hood

Cutting in
Talk over her, or say “stop”, and she finishes the sentence she is in — at most six seconds, 2.5 after “stop” — then falls silent. What she meant to say next is kept and shown to her once, so she can pick it up or let it go. An “mm-hm” does not count.
Your tone
Hearing the emotion in your voice (emotion2vec). Off by default.
Native hearing
Models that take audio can listen directly and pick up tone. Off by default.
GPU voice
An optional voice server for a machine with a graphics card.
A slow model
If the model takes a while, she says something like “Hmm, hang on a second.” instead of going silent.
Half sentences
If you pause on “and”, “my” or “um”, she waits for the rest instead of answering half of it.
No help-desk talk
Lines like “Hope that helps!” or “Is anyone still there?” are never voiced.

Turn-taking

She knows when it's her turn.

Every line is sorted before she answers. Joining in is a choice, and staying quiet is always allowed.

Pick a line
To her
To someone else
About her
To everyone
Can't tell

Pick a line to see how she sorts it.

Rules sort the clear cases; a quick model call settles the rest. Today a little chance still helps her decide whether to join in — the plan is less chance, more judgment.

Under the hood

The floor
While anyone is talking, and for three seconds after, she starts nothing on her own.
Misheard names
Speech recognition often spells a name wrong, so look-alike spellings count too.
Text chat
A quieter guest: at most three unasked messages an hour.
Silence
She can always answer with nothing at all, and it leaves no trace.
Hello
She is told who joins and leaves her call, and may greet a newcomer in her own words — never twice an hour to the same person.
Names
She uses your name now and then, and always if you ask her to.

Memory

She remembers. At night, she sorts it out.

What just happened stays short-term.

At night she goes through it, the way sleep does.

Just now
Today
Long-term

From words to seasons

  1. Word for word
  2. Episodes
  3. Months
  4. Quarters

Memories fade. Recalling them makes them stronger.

The day she found iron

The Self is protected.

Who she is changes only after a waiting period and enough weight — strangers alone can never change it. Every change is versioned and can be rolled back.

Under the hood

Layers
Working memory keeps the last lines word for word; emotional moments go to long-term memory. Four pools: the day, the month, things to ask later, keepsakes.
The dream
Once a night, in the quiet hours: she reviews the day, files things to ask later, writes a short dream and leaves herself a morning note. Anyone speaking wakes her.
Honest memory
When nothing comes back, she says she doesn't remember instead of inventing.
Word-for-word archive
MemPalace keeps a verbatim copy she can never read; only the owner searches it.
Forget me
Anyone can ask her to forget them. She is told, answers in her own words and decides about her long-term memory of the person; the short term stays. What is heard and kept is covered by a privacy note and an audit log.

Interfaces

Many doors. One protocol at heart.

Games, apps, voices and other AIs reach her through a few doors, in layers. The game door speaks the Neuro SDK protocol, so a game written for it connects as it is.

Daydreams

daydream

In a few quiet moments she imagines a sentence or two — always marked as a daydream, never told as fact.

New words

  1. Notice
  2. Try
  3. Adopt
  4. Fade

She notices slang she doesn't know, tries it, adopts it after a few right uses — and lets it fade when nobody says it any more.

Looking it up

Who won the match last night?

Let me google that.

…found it. Want the score?

The answer comes back in the same turn — at most two searches, with a limit per hour.

Under the hood

Two dialects
The official Neuro SDK format and Aura's older flat one, on the same port; 54 conformance tests.
The same checks
Every action — from a game, a tool or another AI — is checked against its schema, a list of dangerous actions and a rate limit. A game with a body (Minecraft) cannot be driven from outside.
Keys
Everything listens on this machine only by default. From another machine: one key for the game door, another for HTTP and MCP; TLS optional.

A day

A day with her.

An imagined day — but every moment in it is something Aura does today. Keep scrolling: the hours go by.

Good morning.

Her first hours start with a note to herself: yesterday in short, what is due today, and last night's dream.

the dream's morning note

Somebody joins the call.

She is told who came in, and may say hello in her own words. Never twice an hour to the same person.

new in v0.2.0

Two people talk at once.

Each voice is heard on its own and by name. A line meant for someone else is left alone.

turn-taking

Nobody is around.

She starts a game of Wordle by herself. The moment somebody speaks, the person comes first.

her own initiative

Minecraft in the afternoon.

A craft fails for want of a plank. She goes for the plank instead of wandering off — and only claims what the inventory shows.

games

Her mind wanders.

A daydream in two sentences: a snack she wants, a cardboard box with a grudge. Marked as a daydream, so it is never told as fact.

imagination

Getting tired.

Tiredness has built up since morning. Her voice slows a little, her answers get shorter.

feelings

The dream.

Once it is quiet she goes over the day: keeps what mattered, lets the sting of a bad moment fade, and dreams a short, strange scene — shown to her in the morning as only a dream.

memory

Persona

One file makes her who she is.

Identity, traits, how she talks, where her limits are — in one plain text file. Her name is a setting.

prompts/persona.md

{name} →

  1. ## Identity
  2. ## Core traits
  3. ## How you talk
  4. ## With people
  5. ## In games
  6. ## Boundaries
  7. ## Examples

No name is written in the file: {name} is filled in from settings, so renaming her is one setting.

Under the hood

A check
A probe script checks that a new model keeps her format, does not leak the prompt and stays in character.
Several characters
Each with its own persona, voice, mood and memory; one speaks at a time. Built and tested, off by default.
Two together
Two instances can talk to each other over the network.
No emoji
There is not a single emoji in her prompts.
Tastes of her own
The persona gives her likes and dislikes of her own — and a faint wish to be a person.

Emotion

Ten feelings. Told in words.

Something happens, a feeling rises, then drifts back to where she usually is. The model never sees a number: only a few lines about how she feels.

Face calm

Pace ×1.06Pitch ×1.11

  • Joy
  • Sadness
  • Anger
  • Fear
  • Surprise
  • Curiosity
  • Boredom
  • Tiredness
  • Excitement
  • Affection

What the model reads

  • You're interested. Want to know more.

Make something happen

The rises, resting points and half-lives are the ones in the code, still being tuned. A repeated event counts for less; tiredness only builds while she is awake, and only sleep takes it away. Time runs ten times faster here.

Inside the bounds

irritated — “*hmpf*” · uneasy · down and quiet · exhausted · giddy

Never described

furious · terrified · hostile to the people she talks to

However high a feeling goes, the words stop there: “a bit irritated — not furious”, “a little scared — not terrified”. What she then does is the model's choice; the code only describes.

In VTube Studio — checked on a real one — she has 36 poses: smile, squint, wink, think, nod and more. What she says can ask for one, and so can you (“wink at me”). Her mood moves the rest, and lip sync follows her voice.

Under the hood

Voice
Mood changes the pace and pitch of her speech.
Face
An expression bus drives Live2D (VTube Studio) and VRChat; a motion engine moves the model thirty times a second.
State
Tiredness and sleep change how she feels too.
Video
She watches a clip or a link when asked; models that take video see the motion.

Games

She plays, too.

Games plug in through a small WebSocket interface, compatible with the Neuro game SDK. What gets wired in is what she can see and what she can do.

Under the hood

Protocol
Any game that speaks the Neuro SDK protocol can plug in (see the next chapter). Press + on a card for more.
osu! autopilot
Off by default. Automation on the official servers can get an account banned — use it offline or on a private server.
Minecraft brain
A fast body model picks actions; routine work runs on rules; a success counts only when the inventory shows it; a notebook remembers spawn, deaths, ores and chests.
Game knowledge
Items, blocks and recipes offline, plus a 21-page guide. A failed craft brings up the right recipe.

Interfaces

Many doors. One protocol at heart.

Games, apps, voices and other AIs reach her through a few doors, in layers. The game door speaks the Neuro SDK protocol, so a game written for it connects as it is.

Who is connecting?

01Inside the programDLC — web panel, stream chat, MCP
02Game protocolws :9700 — Neuro SDK + Aura's dialect
03Voice channelws :9700/game/‹name›/voice
04HTTP:8081 — chat, watch, captions
05MCP:9701 — both ways

      

Under the hood

Two dialects
The official Neuro SDK format and Aura's older flat one, on the same port; 54 conformance tests.
The same checks
Every action — from a game, a tool or another AI — is checked against its schema, a list of dangerous actions and a rate limit. A game with a body (Minecraft) cannot be driven from outside.
Keys
Everything listens on this machine only by default. From another machine: one key for the game door, another for HTTP and MCP; TLS optional.
Removable
Each module lives in a repository of its own and is cloned in; without it Aura still starts. The MCP folder can be deleted whole.

Safety

Built to stay safe. And to stop.

Safety runs both ways: she should not say what she must not, and she should not be treated badly.

What she must not say

Every line she says passes a word filter first — her replies, each spoken sentence, what she says on her own. Type a line, or pick one.

Gets through

She saysThat was a great round, well played!

Inside the bounds: nothing to do.

  • Inside: anything else — teasing, a mild “hmph”, an honest no.
  • Outside: “go die”, “nobody wants you”, “you should disappear” and a few more. Softened: “idiot”, “stupid”, “shut up”, “useless” …

It reads words, not meaning, so the last example is stopped too. A second check, off by default, has another model read the meaning of the whole line (hate, harassment, self-harm, personal information …) — before it is said and before it goes into her memory. Neither filters what people say to her; for that there is the rest button.

When people go too far

Two people press the rest button.

Two different reviewers within five minutes, by default. One alone is not enough.

Every door closes for 20 minutes.

Nothing said to her in that time is remembered. She keeps thinking, quietly.

She is asked, privately, what to forget.

The lines said to her in the last half hour. Her choice; by default nothing goes.

Then she comes back.

Ending it early takes ten reviewers' approval.

Under the hood

Output filter
A word filter that stops only extreme personal attacks and discrimination, plus an optional semantic check where another model reads the meaning of the whole line.
Three kinds of quiet
Mute is the operator's gag. Sleep is hers — she wakes when spoken to. The dream is memory tidying.
This machine only
Interfaces listen on this machine by default; other machines need a key, and TLS is optional.
Next
Planned: the rest button in her own hands.

Deployment

From one laptop to a machine room.

Models, voice and games can live on different machines. Same code.

One machine

Everything together. The simplest start.

Two machines

One runs Aura and the voice; the other runs only the game and a small adapter.

The studio

A server runs the models, each AI gets its own instance, and your own computer streams and develops.

Under the hood

Hardware
Chosen automatically: NVIDIA, then Apple, then AMD, then CPU — with Tensor Cores and the Neural Engine where they apply.
Across machines
The voice can run on its own machine with a graphics card; across machines it needs a token and HTTPS.
Measured
For the first release, with a local 12B model: about 2.5 s an answer on a busy machine; with the GPU voice, the first sound in under two seconds. How it was measured: QUALITY.md.
Memory
One language model at a time; everything else is small and runs inside Aura — about 10 GB in all, measured on a 64 GB machine.

Privacy and control

You stay in control.

She says she is an AI when she joins a call. Anyone can ask her to forget them — and she decides.

Terminal consoleillustration

Under the hood

Forget me
By voice or by text. The reply is fixed text the model cannot refuse; it erases the person everywhere and says what it could not reach.
What is kept
Logs count words, not what was said. Backups go after 30 days. An audit log records privacy events — never content. No voiceprints.
Settings panel
Every setting on one page, with what it does, its default, and whether it needs a restart. Secrets are never shown.
Locked by default
Private endpoints refuse everyone until an API key is set.
Discord commands
Joining, leaving, muting and sleep commands work only for the owner, named operators and people who can manage the server.

Quality

Tested like it matters.

The first release passed more than four thousand automated tests — and the running app was driven from outside, with a real local model, before it shipped.

0automated tests
0of 23 checks in the smoke test
0of 35 conversations answered
0turns in the five-person room test

Still growing.

In active development. The first release, v0.1.0, is out, and the v0.1.4 preview is the newest: tried on one machine, not yet with real people in a real call.

In v0.1.0

The core, voice from ear to mouth, memory and the nightly dream, the rest button, the Hub, the VTube Studio and VRChat drivers, three game modules, and deployment from one machine to many.

New in the v0.1.4 preview

Cut in on her and she finishes the sentence, then stops — and keeps what she meant to say next. “Forget me” is her own choice. When the model fails, she says so. Less dice, more judgment. A face that is not a stare, and a hello for people who join her call. Coding agents such as Claude Code and Codex can be connected.

Not yet seen for real

Discord voice with real people, VRChat, osu! and Minecraft in the real games, runs of many hours, and her faces on other avatars.

What comes next

From the plan in the repository. Nothing here comes with a date.

Less chance, more judgment.

Today a few things are decided by dice — whether to greet, to join in, to use a name. The plan is to let her decide, guided by written rules, with hard limits in code only where a mistake would hurt.

Next

An answer in 300–500 ms.

  1. Measure where the time goes on real prompts: reading the prompt, writing the answer, what the server can reuse.
  2. Start thinking on the first words heard, and confirm when the sentence ends.
  3. A faster model for the first sentence only.
  4. An end of turn that learns: answer sooner when a sentence is clearly over, wait when it is not.
Planned

Hearing that works for everybody — and a voice with more feeling.

  1. A second listening pass with a model that is already loaded.
  2. The names in the call and the game's words as hotwords.
  3. A test set of many voices and accents — no tuning for one person.
  4. Later: a streaming voice with more expression.
Planned

Her face on more avatars.

  1. Poses together with each model's own expressions.
  2. A helper that wires blush and body lean into a model — only with consent, and a backup first.
  3. Her faces in time with her real voice in a call.
Planned

Messier calls, and a hello that fits the room.

  1. Crosstalk, garbled names, long calls.
  2. Coming into a busy call: a hello to everyone. Only a few people: their names.
  3. Something she really remembers about a person — never invented.
Planned

More say for the people around her.

  1. “What do you know about me?” — see what is held before deciding.
  2. Opt out of mood guessing, one person at a time.
  3. “Don't remember this channel.”
  4. A lock-out after wrong keys, and a rate limit on private endpoints.
Planned

Seen for real, and easier to install.

  1. Discord voice in a real call, the real games, runs of several hours, Windows and Linux.
  2. A bundle with the voice models, and installers.
  3. A native version in C#, beside Python — the same game protocol, voice channel, modules, MCP and settings.
  4. Python stays the main version as long as it costs little in speed. So far it does.
Further out

Not started yet.

  1. A mixed-reality branch: the room, where people are in it, and an avatar she controls.
  2. The Minecraft mod in a repository of its own.

What she sees

    What she can do

      How it works

        Where it stands