Personality pack (`resources/commands/personality/`):
- 5 voice commands × ~7-29 phrases each across RU and EN.
- personality.greet: 4 time-of-day buckets (morning/midday/evening/night),
pulls one of ~7 lines per bucket per language.
- personality.thanks / .compliment / .how_are_you / .tony_quote.
- how_are_you embeds live memory size + active profile via jarvis.health()
and jarvis.memory.all() for a "feels alive" effect.
- All use jarvis.cmd.ok helpers, no inline PowerShell SAPI.
- Built by sub-agent. Verified: 6 rust command tests + 60 python tests.
Idle banter (`crates/jarvis-core/src/idle_banter.rs`):
- Background thread chimes in periodically without being asked. Gated by
JARVIS_IDLE_BANTER env (default OFF — intrusion is opt-in).
- Quiet hours 23:00–07:00, skipped under "sleep" profile, paused during
active interactions via `pause()`.
- 30+ static offline lines split into RU/EN × morning/evening/generic
buckets — no network required.
- Lua API jarvis.banter.{fire, pause, resume, enabled}.
- New voice pack `banter/` exposes "скажи что-нибудь интересное",
"помолчи", "можешь говорить".
- 6 unit tests covering pool selection, quiet hours, interval clamp,
pause/resume, opt-in default.
Conversation continuity (`crates/jarvis-core/src/llm/history.rs`):
- New `ConversationHistory::with_persistence(path)` builder. Every
push/clear/pop atomically writes to `<APP_CONFIG_DIR>/llm_history.json`
so daemon restart picks up the thread.
- System prompt is intentionally NOT persisted — comes from current init
call so prompt edits take effect immediately on restart.
- `llm::init_history` wires the path in automatically.
- 4 new tests: round-trip, clear wipes file, corrupt file tolerated,
len/is_empty helpers.
Offline-first math (`resources/commands/math/math.lua`):
- Was: always-LLM, hard fail without GROQ_TOKEN, inline PowerShell SAPI.
- Now: shunting-yard parser handles 95% of voice queries in <50ms — no
network, no token. Russian operator words ("плюс", "умножить на",
"в степени", "квадрат", ...) normalised to symbols first. Patterns
for "корень из X" and "X процентов от Y". Falls back to LLM only on
parse failure (word problems / equations / unit conversions).
- Drops inline PowerShell — speaks via jarvis.cmd.ok.
- 10-case shunting-yard kernel test added (basic ops, precedence,
parens, unary minus, div-by-zero, garbage rejected).
DuckDuckGo Instant Answer (`resources/commands/ddg_answer/`):
- New pack — short factual Q&A without API key. Trigger phrases
"что такое", "кто такой", "расскажи про", "what is", etc.
- Reads AbstractText → Answer → Definition → RelatedTopics[0] in order
from DDG's free JSON API. Opens the search page only if nothing
useful comes back.
- Sandbox full (needs http + system.open).
Tests: 128 → 139 (+11). Release build green.
User asked the code stays easy to fix/extend as the feature surface grows.
This commit refactors what was duplicated, centralises what was scattered,
adds tests where there were none, and writes the architecture doc that
contributors will need.
Deduplication
- tts::play_wav extracted to tts/mod.rs as pub(crate). Was identical in
piper.rs and silero.rs (~25 lines × 2). Both backends now call super::play_wav.
Centralised env config
- new crates/jarvis-core/src/runtime_config.rs — single doc-file for every
JARVIS_* / GROQ_* / OLLAMA_* env var. Includes:
- ENV_* constants with doc comments
- get(name) / get_bool(name, default) / get_parse(name, default) helpers
- feature-flag wrappers: llm_tts_enabled(), llm_router_enabled(),
llm_router_threshold()
- log_effective_config() prints active values on startup
- migrated llm_fallback (JARVIS_LLM_TTS), llm_router (JARVIS_LLM_ROUTER,
JARVIS_LLM_ROUTER_THRESHOLD) to use the new helpers. Pattern set for
future migrations.
Lua boilerplate killer
- new crates/jarvis-core/src/lua/api/cmd.rs exposing:
jarvis.cmd.ok(msg?) — play_ok + speak + {chain=false}
jarvis.cmd.chain_ok(msg?) — same but chain=true
jarvis.cmd.error(msg?) — play_error + speak + {chain=false}
jarvis.cmd.not_found(msg?) — play_not_found + speak + {chain=false}
jarvis.cmd.silent_ok / silent_error
- refactored 5 packs (daily_briefing/off, memory_pack/list, pomodoro/stop,
habit_nudge/stop_all, scheduler/clear) — each lost 3-4 lines of repetitive
play_*/speak/return boilerplate. Pattern for future packs documented in
ARCHITECTURE.md.
Tests (was 32, now 49)
- long_term_memory: 8 new tests for normalize_key, search_in (rank, limit,
empty), build_context_from (empty + populated), serde round-trips for
MemoryRecord and Store. Extracted pure logic (search_in, build_context_from)
into pub(crate) functions to enable testing without global state.
- profiles: 6 new tests for Profile::allows_command (empty/whitelist/blacklist/
deny-wins-over-allow), serde round-trip with all fields + minimal-fields
tolerance via #[serde(default)].
- runtime_config: 2 tests for get_bool / get_parse defaults.
- All 49 tests pass.
New mood/energy log pack
- resources/commands/mood_log/ with 2 commands:
mood.record "запиши настроение 7" / "сегодня мне грустно" → stores
timestamped entry via jarvis.memory.remember
mood.recap "как прошла неделя" → LLM summarises last 30 entries
- Showcase: composes memory + llm + cmd helpers in <30 lines per script.
ARCHITECTURE.md (new, 250 lines)
- Crate layout, data flow diagram (mic→action 10 steps), per-module
responsibility table, configuration layers, TTS pipeline diagram,
Lua sandbox details with API quick-ref, background-services overview,
"how to add a pack/feature/TTS backend" recipes, test coverage map,
build instructions with MSVC env, git workflow with Forgejo NO_PROXY trick.
- Aimed at someone who just cloned the repo and needs to fix a bug fast.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 49/49 jarvis-core unit tests pass.
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
Architecture — close the trigger-strip duplication:
lua/api/text.rs (new):
+ jarvis.text.strip_trigger(phrase, triggers) — case-insensitive
"longest trigger wins" stripping, returns trimmed remainder. The
11 packs that previously inlined a `for _, t in ipairs(triggers) do
string.find ... s == 1 then sub` loop can drop to one call.
Demonstrated in spelling/, window_switch/, summarize/.
+ jarvis.text.contains_any(phrase, needles) — boolean check for any
of N substrings, case-insensitive. Useful for keyword routing.
5 new portable Lua packs (sandbox=full). 36 → 41 total. cargo test
commands::tests still 3/3.
brightness/ — brightness_up / down / max / min. WMI WmiMonitorBrightness
(read CurrentBrightness) + WmiMonitorBrightnessMethods.WmiSetBrightness
(write). ±20% step for up/down; max=100, min=10. Desktop monitors that
don'\''t expose the WMI namespace get a friendly "не поддерживается"
toast instead of a silent failure.
window_switch/ — switch_to_window. Strip trigger ("переключись на" /
"switch to" / etc.), apply the same alias map as process_kill
(хром→chrome, телега→telegram, ...), then PowerShell finds processes
by name OR window-title substring sorted by StartTime desc and uses
(New-Object -ComObject WScript.Shell).AppActivate($pid) on the top
match. Speaks the window title that got focus.
spelling/ — spell_out. Strips "произнеси по буквам" / "spell out" /
etc., iterates `word:gmatch(".")`, joins letters with ". " so SAPI
gives each one a natural micro-pause. Uses jarvis.text.strip_trigger
+ jarvis.speak — 24 lines total.
network/ — open_wifi_settings (ms-settings:network-wifi) +
open_bluetooth_settings (ms-settings:bluetooth) + my_ip. my_ip pulls
the first non-loopback, non-APIPA IPv4 with Get-NetIPAddress and
fetches WAN IP from api.ipify.org (5s timeout); speaks "Локальный X.
Внешний Y".
summarize/ — summarize_selection. Synthesises Ctrl+C through user32!
keybd_event (so the focused window copies its selection), waits
220 ms for the clipboard to populate, reads via
jarvis.system.clipboard.get(), bails if < 20 chars, truncates at
8000, sends to Groq @ temp 0.3 with a "3-5 sentences, keep names"
prompt, drops the summary back into the clipboard and speaks it.
Lets you select ANY text in ANY app (browser, PDF, IDE) and say
"суммируй" to get a spoken TL;DR.
The new packs explicitly use the new jarvis.text + jarvis.llm +
jarvis.speak API surface — no inline PS-SAPI boilerplate, no inline
Groq plumbing. Code is now intent-first.
Architecture — DRY at the Lua API surface:
lua/api/tts.rs (new): jarvis.speak(text, opts?). opts.lang = ISO
two-letter code (default "ru"); auto-picks first installed voice
matching the culture. opts.async = true|false (default true). The
9 packs that previously inlined a 60-character PS-SAPI string for
every spoken reply can now reduce to one line. Refactored fun/,
dice/, clipboard_read/ as proof — fun/ask.lua went from 75 lines
of HTTP+SAPI boilerplate to ~28 lines of intent code.
lua/api/llm.rs (new): jarvis.llm(messages, opts?). messages is a
list of {role, content} tables, opts.max_tokens / .temperature /
.top_p. Returns the assistant string or nil. Gated by sandbox
allows_http() (so Standard+ packs only). Internally uses the
existing jarvis_core::llm::LlmClient::complete_with, no duplicated
HTTP plumbing.
llm/client.rs: split complete() into complete_with(messages,
max_tokens, temperature, top_p) for the new caller; old complete()
is now a thin wrapper at temp=0.7 top_p=1.0 for backward compat.
GUI — /commands page rewrite:
Replaces the "Раздел в разработке" placeholder. Calls Tauri command
get_commands_list (already existed in tauri_commands/commands.rs,
was wired but unused). Renders a search-filterable card grid:
- cmd id (monospace)
- type badge with colour by kind (lua=blue, ahk=red, cli=grey,
voice=green, terminate=red, stop_chaining=violet)
- sandbox badge
- description line (if non-empty)
- all phrases for currentLanguage, fallback to en, fallback to
first available; each phrase in a small inline pill
Toolbar: text input with magnifier icon + reload button (↻).
Counter line shows visible/total + "Скажи Джарвис + любую фразу"
hint. Counts down as you type the filter.
New packs (3, total 33 → 36):
games/ — launch_game / list_games. Reads or creates
%USERPROFILE%\Documents\jarvis-games.json (the sample preloads dota,
cs2, elden ring, witcher 3, factorio + minecraft launcher path +
fortnite epic URI). Trigger-strip + fuzzy name/alias match, then
launches via steam://rungameid/N (Steam), epic:// (Epic Games), or
start "" "path" (anything else). list_games speaks the configured
library.
mouse/ — left/right/middle/double click + scroll up/down. Single
dispatch.lua + _mouse_helper.ps1. PS P/Invokes user32!mouse_event
with the right flag pair (LEFTDOWN+LEFTUP / etc.), 30ms gap, doubles
do two LEFT cycles with a 50ms gap, wheel uses ±360 ticks.
random_choice/ — "выбери из X или Y или Z". Strips the trigger,
splits on " или " / " or " / ", " / " либо " / " чи ", math.random
picks one, speaks "Я выбираю N".
cargo test commands::tests still 3/3. 36 packs verified via
jarvis-gui startup log.
Bossiara13