GUI front-end does not subscribe to IpcEvent::LlmReply, so the LLM
auto-fallback was effectively silent — user got the OK chime but no
spoken answer. Backlog item resolved.
llm_fallback::handle now also invokes speak_via_sapi() after firing
IpcEvent::LlmReply (both for successful replies and the fallback
error message). The text goes through System.Speech.Synthesis.
SpeechSynthesizer, with a preference pass that picks the first
installed ru-RU voice if present (Microsoft Irina / Pavel etc.) so
Russian replies sound right out of the box, and falls back to the
default English voice otherwise.
PowerShell is invoked fire-and-forget (no Wait, stdout/stderr null)
so the voice loop never blocks. A long answer keeps speaking while
jarvis returns to wake-word listening; the mic may pick up some of
the speech which is the same compromise voices::play_* already makes.
A cleaner version would pause pv_recorder for the duration of the
utterance — that goes in the backlog as part of the eventual
unified IpcEvent::Speak refactor.
Toggle: set JARVIS_LLM_TTS=false to disable (e.g. on a server where
the GUI is the speaker). Non-Windows builds get a no-op stub.
LLM auto-fallback (jarvis-app/src/app.rs + jarvis-core/src/config.rs):
- When neither intent classifier nor levenshtein finds a command match,
route the utterance straight to the LLM instead of just playing the
"not found" sound. Triggers ("скажи X", "answer Y") still work and
take precedence — they short-circuit before command lookup.
- Two new config knobs:
LLM_AUTO_FALLBACK = true — master toggle
LLM_AUTO_FALLBACK_MIN_CHARS = 4 — suppress for very short utterances
so background noise doesn't burn
Groq quota
- Requires GROQ_TOKEN; if absent, behaviour is unchanged (play_not_found).
codegen/ command pack:
- Phrases: "напиши код X", "сгенерируй скрипт Y", "write code Z", etc.
- Strips the trigger from the recognized phrase, sends what remains to
Groq with a strict system prompt ("return ONLY code, no fences, no
commentary") at temperature 0.2.
- Parses the JSON content, unescapes \n / \" / \\ / \t, strips any
remaining ```lang ... ``` fences, drops the result into the clipboard
via jarvis.system.clipboard.set.
- Notifies a 120-char preview + plays ok-sound. Works for any language
the model handles (Python by default if unspecified).
- GROQ_TOKEN / GROQ_MODEL / GROQ_BASE_URL read from env at call time —
same envvars the voice-loop fallback already uses.
ocr/ command pack:
- Phrases: "прочитай экран", "что на экране", "read screen", etc.
- Captures the primary screen via System.Windows.Forms + System.Drawing
to a temp PNG, then shells to tesseract.exe (-l rus+eng for ru/ua,
-l eng for en).
- Resolves Tesseract by PATH first, then C:\Program Files\Tesseract-OCR
and the x86 install dir; if none works the user gets a friendly
"winget install UB-Mannheim.TesseractOCR" hint in a notification.
- Recognized text goes to clipboard + a 200-char preview notification.
All three features are portable (env-var resolution, no hardcoded user
paths). Command-pack total is now 11. cargo test -p jarvis-core --lib
commands::tests passes 3/3.
new module crates/jarvis-app/src/llm_fallback.rs holds an optional
LlmClient + ConversationHistory built once at startup. if GROQ_TOKEN is
unset the module logs a warning and stays disabled — voice commands keep
working as before.
both the wake-word voice path (recognize_command in app.rs) and the
gui-side text command path (process_text_command) now check whether the
recognized phrase starts with one of the configured trigger phrases
(ru: 'скажи' / 'ответь' / 'произнеси'). when it does, the remainder of
the phrase is sent to Groq and the reply is published as a new
IpcEvent::LlmReply { text } so the gui can speak it.
on api error the trailing user turn is popped from history, the russian
fallback line is sent over ipc and a 'error' voice cue plays. the loop
itself never panics.
