2026-04-23 02:16:02 +03:00
|
|
|
use once_cell::sync::OnceCell;
|
|
|
|
|
use parking_lot::Mutex;
|
|
|
|
|
|
|
|
|
|
use jarvis_core::config;
|
|
|
|
|
use jarvis_core::i18n;
|
|
|
|
|
use jarvis_core::ipc::{self, IpcEvent};
|
|
|
|
|
use jarvis_core::llm::{ChatMessage, ConversationHistory, LlmClient};
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
use jarvis_core::long_term_memory;
|
|
|
|
|
use jarvis_core::profiles;
|
|
|
|
|
use jarvis_core::tts::{self, SpeakOpts};
|
2026-04-23 02:16:02 +03:00
|
|
|
use jarvis_core::voices;
|
|
|
|
|
|
|
|
|
|
struct State {
|
|
|
|
|
client: LlmClient,
|
|
|
|
|
history: Mutex<ConversationHistory>,
|
|
|
|
|
max_tokens: u32,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static STATE: OnceCell<Option<State>> = OnceCell::new();
|
|
|
|
|
|
|
|
|
|
pub fn init() {
|
|
|
|
|
let _ = STATE.set(build_state());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn build_state() -> Option<State> {
|
|
|
|
|
if !config::LLM_DEFAULT_ENABLED {
|
|
|
|
|
info!("LLM fallback disabled by config.");
|
|
|
|
|
return None;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let client = match LlmClient::from_env() {
|
|
|
|
|
Ok(c) => c,
|
|
|
|
|
Err(e) => {
|
feat: Ollama LLM backend + 5 utility packs (daily briefing, pomodoro, currency convert, stocks, habits, translate clipboard)
Local-LLM support so J.A.R.V.I.S. works offline without GROQ_TOKEN.
LLM multi-backend (crates/jarvis-core/src/llm/client.rs)
- LlmBackend enum {Groq, Ollama}. Both use OpenAI-compatible /v1/chat/completions
(Ollama exposes one natively at :11434/v1).
- LlmClient::from_env() now dispatches on JARVIS_LLM=groq|ollama. Auto-detect:
Groq if GROQ_TOKEN set, else Ollama.
- New helpers: LlmClient::groq() (was: from_env), LlmClient::ollama().
- Reqwest client now has a 60s timeout (was: unbounded — Groq freezes could hang).
- api_key skipped when empty (Ollama doesn't need auth).
- Defaults: OLLAMA_BASE_URL=http://localhost:11434/v1, OLLAMA_MODEL=qwen2.5:3b.
Override via OLLAMA_BASE_URL / OLLAMA_MODEL env.
- llm_fallback + llm_router log the active backend on init.
- Tests: 32/32 pass. Renamed `from_env_fails` → `groq_fails_when_token_missing`,
added `ollama_works_without_token`.
Daily briefing pack (resources/commands/daily_briefing/, 3 commands)
- setup.lua: "настрой утренний брифинг на 9:00" → adds a daily scheduled task
that runs now.lua via the scheduler's Lua-action support.
- now.lua: greeting + time + active profile + scheduled task count + memory recap
+ LLM motivational nudge. Voice-triggered ("утренний брифинг") works the same.
- off.lua: removes the scheduled task.
Pomodoro pack (resources/commands/pomodoro/, 2 commands)
- start.lua / stop.lua + tick.lua (re-scheduled by itself). State stored in
jarvis.state — phase alternates work (25 min) / break (5 min) until stop.
Currency conversion (resources/commands/currency/convert.lua)
- "сколько будет 1000 долларов в рублях" — fetches CBR daily rates from
cbr-xml-daily.ru, normalises by Nominal, prints both directions.
- Recognises USD/EUR/CNY/GBP/JPY/RUB by substring. Pluralises russian units.
Stocks pack (resources/commands/stocks/, 1 command)
- "сколько Сбер" → MOEX ISS /iss/engines/stock/markets/shares/securities/<TKR>.
- Mapping of 22 popular Russian tickers to common names (Сбер→SBER, Газпром→GAZP,
Лукойл→LKOH, Яндекс→YDEX, Т-банк→T, etc).
- Picks TQBR (main board) row, reports LAST + LASTTOPREVPRICE%.
Habit nudges (resources/commands/habit_nudge/, 4 commands)
- "напоминай мне пить воду" → every 2 hours
- "напоминай мне размяться" → every 50 minutes
- "напоминай отдыхать глазам" → every 20 minutes (20-20-20 rule)
- "отключи все привычки" → stops all three by id.
Translate clipboard (resources/commands/translate/clipboard.lua)
- "переведи буфер на английский" → grabs clipboard, sends to LLM, speaks first
sentence, replaces clipboard with full translation, fires notification.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green. 32/32 tests.
To use Ollama:
1. Install + run Ollama (https://ollama.com).
2. `ollama pull qwen2.5:3b` (or any chat model).
3. Optional: `set JARVIS_LLM=ollama` (auto-picked if GROQ_TOKEN unset).
2026-05-15 15:51:24 +03:00
|
|
|
warn!("LLM fallback disabled: {}. Set GROQ_TOKEN or run Ollama to enable.", e);
|
2026-04-23 02:16:02 +03:00
|
|
|
return None;
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
let lang = i18n::get_language();
|
|
|
|
|
let prompt = config::get_llm_system_prompt(&lang);
|
|
|
|
|
let history = Mutex::new(ConversationHistory::new(prompt, config::LLM_DEFAULT_MAX_HISTORY));
|
|
|
|
|
|
feat: Ollama LLM backend + 5 utility packs (daily briefing, pomodoro, currency convert, stocks, habits, translate clipboard)
Local-LLM support so J.A.R.V.I.S. works offline without GROQ_TOKEN.
LLM multi-backend (crates/jarvis-core/src/llm/client.rs)
- LlmBackend enum {Groq, Ollama}. Both use OpenAI-compatible /v1/chat/completions
(Ollama exposes one natively at :11434/v1).
- LlmClient::from_env() now dispatches on JARVIS_LLM=groq|ollama. Auto-detect:
Groq if GROQ_TOKEN set, else Ollama.
- New helpers: LlmClient::groq() (was: from_env), LlmClient::ollama().
- Reqwest client now has a 60s timeout (was: unbounded — Groq freezes could hang).
- api_key skipped when empty (Ollama doesn't need auth).
- Defaults: OLLAMA_BASE_URL=http://localhost:11434/v1, OLLAMA_MODEL=qwen2.5:3b.
Override via OLLAMA_BASE_URL / OLLAMA_MODEL env.
- llm_fallback + llm_router log the active backend on init.
- Tests: 32/32 pass. Renamed `from_env_fails` → `groq_fails_when_token_missing`,
added `ollama_works_without_token`.
Daily briefing pack (resources/commands/daily_briefing/, 3 commands)
- setup.lua: "настрой утренний брифинг на 9:00" → adds a daily scheduled task
that runs now.lua via the scheduler's Lua-action support.
- now.lua: greeting + time + active profile + scheduled task count + memory recap
+ LLM motivational nudge. Voice-triggered ("утренний брифинг") works the same.
- off.lua: removes the scheduled task.
Pomodoro pack (resources/commands/pomodoro/, 2 commands)
- start.lua / stop.lua + tick.lua (re-scheduled by itself). State stored in
jarvis.state — phase alternates work (25 min) / break (5 min) until stop.
Currency conversion (resources/commands/currency/convert.lua)
- "сколько будет 1000 долларов в рублях" — fetches CBR daily rates from
cbr-xml-daily.ru, normalises by Nominal, prints both directions.
- Recognises USD/EUR/CNY/GBP/JPY/RUB by substring. Pluralises russian units.
Stocks pack (resources/commands/stocks/, 1 command)
- "сколько Сбер" → MOEX ISS /iss/engines/stock/markets/shares/securities/<TKR>.
- Mapping of 22 popular Russian tickers to common names (Сбер→SBER, Газпром→GAZP,
Лукойл→LKOH, Яндекс→YDEX, Т-банк→T, etc).
- Picks TQBR (main board) row, reports LAST + LASTTOPREVPRICE%.
Habit nudges (resources/commands/habit_nudge/, 4 commands)
- "напоминай мне пить воду" → every 2 hours
- "напоминай мне размяться" → every 50 minutes
- "напоминай отдыхать глазам" → every 20 minutes (20-20-20 rule)
- "отключи все привычки" → stops all three by id.
Translate clipboard (resources/commands/translate/clipboard.lua)
- "переведи буфер на английский" → grabs clipboard, sends to LLM, speaks first
sentence, replaces clipboard with full translation, fires notification.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green. 32/32 tests.
To use Ollama:
1. Install + run Ollama (https://ollama.com).
2. `ollama pull qwen2.5:3b` (or any chat model).
3. Optional: `set JARVIS_LLM=ollama` (auto-picked if GROQ_TOKEN unset).
2026-05-15 15:51:24 +03:00
|
|
|
info!("LLM fallback enabled (backend: {}, model: {}).", client.backend().name(), client.model());
|
2026-04-23 02:16:02 +03:00
|
|
|
Some(State {
|
|
|
|
|
client,
|
|
|
|
|
history,
|
|
|
|
|
max_tokens: config::LLM_DEFAULT_MAX_TOKENS,
|
|
|
|
|
})
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn is_enabled() -> bool {
|
|
|
|
|
STATE.get().and_then(|s| s.as_ref()).is_some()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn extract_prompt(text: &str) -> Option<String> {
|
|
|
|
|
let lang = i18n::get_language();
|
|
|
|
|
let triggers = config::get_llm_trigger_phrases(&lang);
|
|
|
|
|
let lowered = text.to_lowercase();
|
|
|
|
|
|
|
|
|
|
for trig in triggers {
|
|
|
|
|
if let Some(rest) = strip_trigger(&lowered, trig) {
|
|
|
|
|
let trimmed = rest.trim();
|
|
|
|
|
if !trimmed.is_empty() {
|
|
|
|
|
return Some(trimmed.to_string());
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
None
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn strip_trigger(text: &str, trigger: &str) -> Option<String> {
|
|
|
|
|
let t = text.trim_start();
|
|
|
|
|
if let Some(after) = t.strip_prefix(trigger) {
|
|
|
|
|
let next = after.chars().next();
|
|
|
|
|
if next.map_or(true, |c| !c.is_alphanumeric()) {
|
|
|
|
|
return Some(after.to_string());
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
None
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn handle(prompt: &str) {
|
|
|
|
|
let state = match STATE.get().and_then(|s| s.as_ref()) {
|
|
|
|
|
Some(s) => s,
|
|
|
|
|
None => {
|
|
|
|
|
warn!("LLM fallback called while disabled — ignoring.");
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
info!("LLM prompt: {}", prompt);
|
|
|
|
|
|
|
|
|
|
let snapshot: Vec<ChatMessage> = {
|
|
|
|
|
let mut h = state.history.lock();
|
|
|
|
|
h.push_user(prompt);
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
let mut snap = h.snapshot();
|
|
|
|
|
|
|
|
|
|
// Inject (a) profile personality (b) relevant long-term memory as a fresh
|
|
|
|
|
// system message right after the base prompt. Both are optional.
|
|
|
|
|
let profile = profiles::active();
|
|
|
|
|
let mut overlay = String::new();
|
|
|
|
|
if !profile.llm_personality.is_empty() {
|
|
|
|
|
overlay.push_str(&format!("Активный профиль: {} {}\nХарактер для ответа: {}\n",
|
|
|
|
|
profile.icon, profile.name, profile.llm_personality));
|
|
|
|
|
}
|
|
|
|
|
let mem_ctx = long_term_memory::build_context(prompt, 5);
|
|
|
|
|
if !mem_ctx.is_empty() {
|
|
|
|
|
overlay.push_str(&mem_ctx);
|
|
|
|
|
}
|
|
|
|
|
if !overlay.is_empty() {
|
|
|
|
|
// Insert after the base system prompt (index 0 typically), before user msgs.
|
|
|
|
|
let insert_at = if !snap.is_empty() && snap[0].role == "system" { 1 } else { 0 };
|
|
|
|
|
snap.insert(insert_at, ChatMessage::system(overlay));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
snap
|
2026-04-23 02:16:02 +03:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
match state.client.complete(&snapshot, state.max_tokens) {
|
|
|
|
|
Ok(reply) => {
|
|
|
|
|
let reply = reply.trim().to_string();
|
|
|
|
|
info!("LLM reply: {}", reply);
|
|
|
|
|
state.history.lock().push_assistant(reply.clone());
|
2026-05-15 01:51:02 +03:00
|
|
|
ipc::send(IpcEvent::LlmReply { text: reply.clone() });
|
2026-04-23 02:16:02 +03:00
|
|
|
voices::play_ok();
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
speak_reply(&reply);
|
2026-04-23 02:16:02 +03:00
|
|
|
}
|
|
|
|
|
Err(e) => {
|
|
|
|
|
error!("LLM request failed: {}", e);
|
|
|
|
|
state.history.lock().pop_last_user();
|
2026-05-15 01:51:02 +03:00
|
|
|
let err_text = config::LLM_FALLBACK_ERROR_RU.to_string();
|
|
|
|
|
ipc::send(IpcEvent::LlmReply { text: err_text.clone() });
|
2026-04-23 02:16:02 +03:00
|
|
|
voices::play_error();
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
speak_reply(&err_text);
|
2026-04-23 02:16:02 +03:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-05-15 01:51:02 +03:00
|
|
|
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
fn speak_reply(text: &str) {
|
refactor: maintainability pass — dedup, central env config, cmd helpers, tests, ARCHITECTURE.md
User asked the code stays easy to fix/extend as the feature surface grows.
This commit refactors what was duplicated, centralises what was scattered,
adds tests where there were none, and writes the architecture doc that
contributors will need.
Deduplication
- tts::play_wav extracted to tts/mod.rs as pub(crate). Was identical in
piper.rs and silero.rs (~25 lines × 2). Both backends now call super::play_wav.
Centralised env config
- new crates/jarvis-core/src/runtime_config.rs — single doc-file for every
JARVIS_* / GROQ_* / OLLAMA_* env var. Includes:
- ENV_* constants with doc comments
- get(name) / get_bool(name, default) / get_parse(name, default) helpers
- feature-flag wrappers: llm_tts_enabled(), llm_router_enabled(),
llm_router_threshold()
- log_effective_config() prints active values on startup
- migrated llm_fallback (JARVIS_LLM_TTS), llm_router (JARVIS_LLM_ROUTER,
JARVIS_LLM_ROUTER_THRESHOLD) to use the new helpers. Pattern set for
future migrations.
Lua boilerplate killer
- new crates/jarvis-core/src/lua/api/cmd.rs exposing:
jarvis.cmd.ok(msg?) — play_ok + speak + {chain=false}
jarvis.cmd.chain_ok(msg?) — same but chain=true
jarvis.cmd.error(msg?) — play_error + speak + {chain=false}
jarvis.cmd.not_found(msg?) — play_not_found + speak + {chain=false}
jarvis.cmd.silent_ok / silent_error
- refactored 5 packs (daily_briefing/off, memory_pack/list, pomodoro/stop,
habit_nudge/stop_all, scheduler/clear) — each lost 3-4 lines of repetitive
play_*/speak/return boilerplate. Pattern for future packs documented in
ARCHITECTURE.md.
Tests (was 32, now 49)
- long_term_memory: 8 new tests for normalize_key, search_in (rank, limit,
empty), build_context_from (empty + populated), serde round-trips for
MemoryRecord and Store. Extracted pure logic (search_in, build_context_from)
into pub(crate) functions to enable testing without global state.
- profiles: 6 new tests for Profile::allows_command (empty/whitelist/blacklist/
deny-wins-over-allow), serde round-trip with all fields + minimal-fields
tolerance via #[serde(default)].
- runtime_config: 2 tests for get_bool / get_parse defaults.
- All 49 tests pass.
New mood/energy log pack
- resources/commands/mood_log/ with 2 commands:
mood.record "запиши настроение 7" / "сегодня мне грустно" → stores
timestamped entry via jarvis.memory.remember
mood.recap "как прошла неделя" → LLM summarises last 30 entries
- Showcase: composes memory + llm + cmd helpers in <30 lines per script.
ARCHITECTURE.md (new, 250 lines)
- Crate layout, data flow diagram (mic→action 10 steps), per-module
responsibility table, configuration layers, TTS pipeline diagram,
Lua sandbox details with API quick-ref, background-services overview,
"how to add a pack/feature/TTS backend" recipes, test coverage map,
build instructions with MSVC env, git workflow with Forgejo NO_PROXY trick.
- Aimed at someone who just cloned the repo and needs to fix a bug fast.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 49/49 jarvis-core unit tests pass.
2026-05-15 16:06:18 +03:00
|
|
|
if !jarvis_core::runtime_config::llm_tts_enabled() {
|
2026-05-15 01:51:02 +03:00
|
|
|
return;
|
|
|
|
|
}
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
tts::speak(text, &SpeakOpts::lang("ru"));
|
2026-05-15 01:51:02 +03:00
|
|
|
}
|