Measured

Tests and results

What has been measured, on what hardware, and what is still open. The numbers are from the 0.1.4 preview unless stated otherwise; the weak spots are listed with the good ones.

The suite

pytest, whole project
4867 passed, 3 skipped, 0 failed
real-app smoke
23 / 23
command-line mode
28 / 28
from the unpacked package
23 / 23
what it covers
memory, attention, the voice pipeline, games, the panel, privacy, the scripted conversations

With the real model (gemma 12B, one run)

who-is-who
11 / 11
attention
17 / 17
callbacks
6 / 6
pirate (role kept)
4 / 4
replies of four words or fewer
1 / 8 — weak
preferences recalled
3 / 7 — weak
"forget me" wrote the tag
4 / 4 for real requests, 0 / 4 for "just kidding"
a cut-off note
dropped or continued sensibly, both tried

Answers: 35 conversations (2026-10-17)

answered
35 / 35
words per answer
median 24, p90 31, max 47
seconds per answer
median 1.3, p90 1.9; a web lookup 10-12 s
assistant phrases
0.06 per answer
stage directions spoken aloud
0
emoji
0
repeats of an earlier answer
0
five people at once
all answered, 1.7-7.7 s (one model, so one after the other)
ten lines in one second
no error; put together and answered once
empty input
HTTP 400, as it should be
6,000 characters, control characters, her own tags typed by a person
all answered, nothing executed

These move between runs: the same battery the day before read median 25-27 words and 1.4-1.5 s. Read them as ranges, not as a score.

Not verified

Discord voice with real people (DAVE)
not tried
her joining a voice channel
not tried
the sound board
not tried
VRChat, osu!, Minecraft in the real games
not tried
runs of many hours
not tried
the recogniser on other voices
not tried
a second Live2D model
not tried
a real Claude Code or Codex session using her
not tried

Run it yourself

python3 -m pytest
the whole suite; no model needed
scripts/smoke.py
a sandbox app, started with AURA_NO_PLAYBACK=1
scripts/quality_battery.py --quick --json out.json
the 35 conversations, about a third; needs a running Aura and her key
scripts/people_and_bits.py
who-is-who, attention, callbacks and pirate