Architecture — DRY at the Lua API surface:
lua/api/tts.rs (new): jarvis.speak(text, opts?). opts.lang = ISO
two-letter code (default "ru"); auto-picks first installed voice
matching the culture. opts.async = true|false (default true). The
9 packs that previously inlined a 60-character PS-SAPI string for
every spoken reply can now reduce to one line. Refactored fun/,
dice/, clipboard_read/ as proof — fun/ask.lua went from 75 lines
of HTTP+SAPI boilerplate to ~28 lines of intent code.
lua/api/llm.rs (new): jarvis.llm(messages, opts?). messages is a
list of {role, content} tables, opts.max_tokens / .temperature /
.top_p. Returns the assistant string or nil. Gated by sandbox
allows_http() (so Standard+ packs only). Internally uses the
existing jarvis_core::llm::LlmClient::complete_with, no duplicated
HTTP plumbing.
llm/client.rs: split complete() into complete_with(messages,
max_tokens, temperature, top_p) for the new caller; old complete()
is now a thin wrapper at temp=0.7 top_p=1.0 for backward compat.
GUI — /commands page rewrite:
Replaces the "Раздел в разработке" placeholder. Calls Tauri command
get_commands_list (already existed in tauri_commands/commands.rs,
was wired but unused). Renders a search-filterable card grid:
- cmd id (monospace)
- type badge with colour by kind (lua=blue, ahk=red, cli=grey,
voice=green, terminate=red, stop_chaining=violet)
- sandbox badge
- description line (if non-empty)
- all phrases for currentLanguage, fallback to en, fallback to
first available; each phrase in a small inline pill
Toolbar: text input with magnifier icon + reload button (↻).
Counter line shows visible/total + "Скажи Джарвис + любую фразу"
hint. Counts down as you type the filter.
New packs (3, total 33 → 36):
games/ — launch_game / list_games. Reads or creates
%USERPROFILE%\Documents\jarvis-games.json (the sample preloads dota,
cs2, elden ring, witcher 3, factorio + minecraft launcher path +
fortnite epic URI). Trigger-strip + fuzzy name/alias match, then
launches via steam://rungameid/N (Steam), epic:// (Epic Games), or
start "" "path" (anything else). list_games speaks the configured
library.
mouse/ — left/right/middle/double click + scroll up/down. Single
dispatch.lua + _mouse_helper.ps1. PS P/Invokes user32!mouse_event
with the right flag pair (LEFTDOWN+LEFTUP / etc.), 30ms gap, doubles
do two LEFT cycles with a 50ms gap, wheel uses ±360 ticks.
random_choice/ — "выбери из X или Y или Z". Strips the trigger,
splits on " или " / " or " / ", " / " либо " / " чи ", math.random
picks one, speaks "Я выбираю N".
cargo test commands::tests still 3/3. 36 packs verified via
jarvis-gui startup log.
LLM auto-fallback (jarvis-app/src/app.rs + jarvis-core/src/config.rs):
- When neither intent classifier nor levenshtein finds a command match,
route the utterance straight to the LLM instead of just playing the
"not found" sound. Triggers ("скажи X", "answer Y") still work and
take precedence — they short-circuit before command lookup.
- Two new config knobs:
LLM_AUTO_FALLBACK = true — master toggle
LLM_AUTO_FALLBACK_MIN_CHARS = 4 — suppress for very short utterances
so background noise doesn't burn
Groq quota
- Requires GROQ_TOKEN; if absent, behaviour is unchanged (play_not_found).
codegen/ command pack:
- Phrases: "напиши код X", "сгенерируй скрипт Y", "write code Z", etc.
- Strips the trigger from the recognized phrase, sends what remains to
Groq with a strict system prompt ("return ONLY code, no fences, no
commentary") at temperature 0.2.
- Parses the JSON content, unescapes \n / \" / \\ / \t, strips any
remaining ```lang ... ``` fences, drops the result into the clipboard
via jarvis.system.clipboard.set.
- Notifies a 120-char preview + plays ok-sound. Works for any language
the model handles (Python by default if unspecified).
- GROQ_TOKEN / GROQ_MODEL / GROQ_BASE_URL read from env at call time —
same envvars the voice-loop fallback already uses.
ocr/ command pack:
- Phrases: "прочитай экран", "что на экране", "read screen", etc.
- Captures the primary screen via System.Windows.Forms + System.Drawing
to a temp PNG, then shells to tesseract.exe (-l rus+eng for ru/ua,
-l eng for en).
- Resolves Tesseract by PATH first, then C:\Program Files\Tesseract-OCR
and the x86 install dir; if none works the user gets a friendly
"winget install UB-Mannheim.TesseractOCR" hint in a notification.
- Recognized text goes to clipboard + a 200-char preview notification.
All three features are portable (env-var resolution, no hardcoded user
paths). Command-pack total is now 11. cargo test -p jarvis-core --lib
commands::tests passes 3/3.
cargo test on jarvis-core was failing with `LNK1181 cannot open input
file "libvosk.lib"`. The link-search path is declared in jarvis-app's
build.rs (and jarvis-cli's), but the test binary for jarvis-core has
no such config and the workspace .cargo/config.toml didn't cover it.
Adds a minimal build.rs that emits `cargo:rustc-link-search=native=...`
pointing at lib/windows/amd64 (resolved relative to CARGO_MANIFEST_DIR,
no hardcoded absolute paths so the project still builds for anyone
cloning under a different drive/path).
Verified: cargo test -p jarvis-core --lib commands::tests now passes
3/3 (every_command_toml_parses, lua_command_scripts_exist,
every_command_has_phrases).
Three unit tests catch the kind of bug that broke weather/set_city (a
phrases array instead of lang→array map silently dropped the whole pack):
- every_command_toml_parses: every resources/commands/*/command.toml round-
trips through toml::from_str::<JCommandsList>. Reports all failures at
once instead of failing on the first.
- lua_command_scripts_exist: for every type="lua" command, the named (or
default script.lua) script file exists in the pack dir.
- every_command_has_phrases: structural commands (terminate/stop_chaining)
excluded, every other command has ≥1 phrase across all languages.
Tests use env!("CARGO_MANIFEST_DIR") -> ../.. -> resources/commands so they
work from any cwd.
NB: cargo test does not run on this machine right now — aws-lc-sys needs
cl.exe (MSVC Build Tools missing). The release binary was built earlier
with MSVC available; toolchain install is a follow-up. The tests are still
valid for CI / a re-armed dev box.
new module crates/jarvis-app/src/llm_fallback.rs holds an optional
LlmClient + ConversationHistory built once at startup. if GROQ_TOKEN is
unset the module logs a warning and stays disabled — voice commands keep
working as before.
both the wake-word voice path (recognize_command in app.rs) and the
gui-side text command path (process_text_command) now check whether the
recognized phrase starts with one of the configured trigger phrases
(ru: 'скажи' / 'ответь' / 'произнеси'). when it does, the remainder of
the phrase is sent to Groq and the reply is published as a new
IpcEvent::LlmReply { text } so the gui can speak it.
on api error the trailing user turn is popped from history, the russian
fallback line is sent over ipc and a 'error' voice cue plays. the loop
itself never panics.
add LLM_DEFAULT_ENABLED / LLM_DEFAULT_MAX_HISTORY / LLM_DEFAULT_MAX_TOKENS
defaults, the russian J.A.R.V.I.S. system prompt, a fallback error line,
and per-language helpers get_llm_trigger_phrases() and get_llm_system_prompt()
so the voice loop can opt into Groq with a 'скажи …' style prefix.
split llm.rs into a module with separate client and history submodules.
ConversationHistory holds an optional system prompt plus a FIFO of user/
assistant turns capped at max_turns; oldest turns evict on overflow,
system prompt is preserved across truncation and clear().
New jarvis-core::llm module providing a blocking client for
OpenAI-compatible chat completions endpoints (Groq by default).
- LlmClient::new / from_env (GROQ_TOKEN, GROQ_BASE_URL, GROQ_MODEL)
- complete(messages, max_tokens) -> String
- thiserror-based LlmError / ConfigError
- gated behind a new llm feature, included in default jarvis_app
Not yet wired into the wake-word/intent loop; that lands in v0.3.
