2026-04-23 02:16:02 +03:00
|
|
|
use once_cell::sync::OnceCell;
|
|
|
|
|
|
|
|
|
|
use jarvis_core::config;
|
|
|
|
|
use jarvis_core::i18n;
|
|
|
|
|
use jarvis_core::ipc::{self, IpcEvent};
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
use jarvis_core::llm::{self, ChatMessage};
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
use jarvis_core::long_term_memory;
|
|
|
|
|
use jarvis_core::profiles;
|
|
|
|
|
use jarvis_core::tts::{self, SpeakOpts};
|
2026-04-23 02:16:02 +03:00
|
|
|
use jarvis_core::voices;
|
|
|
|
|
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
// State now only holds the max_tokens config — both the LLM client AND the
|
|
|
|
|
// conversation history live in `jarvis_core::llm::*` globals. This lets voice
|
|
|
|
|
// commands ("сбрось контекст", "повтори последнее") and Lua scripts share the
|
|
|
|
|
// same buffer instead of each module owning its own.
|
2026-04-23 02:16:02 +03:00
|
|
|
struct State {
|
|
|
|
|
max_tokens: u32,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static STATE: OnceCell<Option<State>> = OnceCell::new();
|
|
|
|
|
|
|
|
|
|
pub fn init() {
|
|
|
|
|
let _ = STATE.set(build_state());
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn build_state() -> Option<State> {
|
|
|
|
|
if !config::LLM_DEFAULT_ENABLED {
|
|
|
|
|
info!("LLM fallback disabled by config.");
|
|
|
|
|
return None;
|
|
|
|
|
}
|
|
|
|
|
|
feat: hot-swap LLM backend + media keys + codebase Q&A + scheduler cancel-by-text + TTS pre-warm
Single biggest user-facing change: you can now switch local↔cloud LLM by voice
without restarting. Plus four new packs and a perf nudge.
Shared LLM global (crates/jarvis-core/src/llm/mod.rs)
- GLOBAL: Lazy<RwLock<Option<Arc<LlmClient>>>>. Single source of truth.
- init_global() / current() / swap_to(LlmBackend) / current_backend_name() / parse_backend(str).
- llm_fallback + llm_router migrated: they no longer own a separate LlmClient,
they read from the global on each request. Hot-swap is now LIVE — change
backend, next chat call uses the new one.
- parse_backend accepts both english and russian aliases:
"groq"/"cloud"/"облако"/"клауд" → Groq
"ollama"/"local"/"локал"/"локальный" → Ollama
Voice + Lua LLM switcher (resources/commands/llm_switch/, 3 commands)
- "переключись на локальный" / "перейди на оллама" → swap to Ollama
- "переключись на облако" / "используй грок" → swap to Groq
- "какой у тебя мозг" / "облако или локально" → status query
Underlying Lua API (registered globally):
jarvis.llm({messages}, opts) -- as before, now uses global client
jarvis.llm_status() -> "groq" | "ollama" | "none"
jarvis.llm_switch(name) -> name on success, nil on failure
Media keys pack (resources/commands/media_keys/, 4 commands)
- "пауза" / "плей" → VK_MEDIA_PLAY_PAUSE
- "следующий трек" → VK_MEDIA_NEXT_TRACK
- "предыдущий трек" → VK_MEDIA_PREV_TRACK
- "стоп музыка" → VK_MEDIA_STOP
- Helper _media.ps1 uses user32.keybd_event (P/Invoke through Add-Type).
- Works with anything that listens to global media keys: Spotify, Yandex Music,
YouTube (focused tab), Foobar2000, Winamp, etc. No OAuth, no API keys.
Codebase Q&A pack (resources/commands/codebase_qa/, 3 commands)
- "укажи проект <path>" → stores path in jarvis.memory under codebase.root
- "какой проект сейчас" → speaks current path
- "что делает функция X" / "найди в коде Y" / "объясни код"
→ walks the folder (depth 3, 30 files cap, 4KB per file, 50KB total),
filters by source extensions (rs/py/ts/lua/go/...), feeds digest to LLM
with a "senior reviewer" system prompt, speaks 3-5 sentence answer.
Scheduler cancel-by-text (crates/jarvis-core/src/scheduler.rs)
- new pub fn find_by_text(query, limit) -> Vec<ScheduledTask>
- new pub fn remove_by_text(query) -> usize (count removed)
- new pure helper find_by_text_in(tasks, query, limit) for tests
- Lua: jarvis.scheduler.{find_by_text, remove_by_text}
- Voice: "отмени напоминание про воду" / "удали задачу про разминку"
- 3 new unit tests (52 total in jarvis-core).
TTS pre-warm (crates/jarvis-app/src/main.rs)
- Call jarvis_core::tts::backend() once on startup so the first speak()
doesn't pay Piper binary discovery + voice loading cost. Cuts first-speak
latency by ~150-300ms depending on backend.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:15:59 +03:00
|
|
|
// Initialise the shared global client (idempotent — fine if init was already called).
|
|
|
|
|
if let Err(e) = llm::init_global() {
|
|
|
|
|
warn!("LLM fallback disabled: {}. Set GROQ_TOKEN or run Ollama to enable.", e);
|
|
|
|
|
return None;
|
|
|
|
|
}
|
2026-04-23 02:16:02 +03:00
|
|
|
|
|
|
|
|
let lang = i18n::get_language();
|
|
|
|
|
let prompt = config::get_llm_system_prompt(&lang);
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
llm::init_history(prompt, config::LLM_DEFAULT_MAX_HISTORY);
|
2026-04-23 02:16:02 +03:00
|
|
|
|
feat: hot-swap LLM backend + media keys + codebase Q&A + scheduler cancel-by-text + TTS pre-warm
Single biggest user-facing change: you can now switch local↔cloud LLM by voice
without restarting. Plus four new packs and a perf nudge.
Shared LLM global (crates/jarvis-core/src/llm/mod.rs)
- GLOBAL: Lazy<RwLock<Option<Arc<LlmClient>>>>. Single source of truth.
- init_global() / current() / swap_to(LlmBackend) / current_backend_name() / parse_backend(str).
- llm_fallback + llm_router migrated: they no longer own a separate LlmClient,
they read from the global on each request. Hot-swap is now LIVE — change
backend, next chat call uses the new one.
- parse_backend accepts both english and russian aliases:
"groq"/"cloud"/"облако"/"клауд" → Groq
"ollama"/"local"/"локал"/"локальный" → Ollama
Voice + Lua LLM switcher (resources/commands/llm_switch/, 3 commands)
- "переключись на локальный" / "перейди на оллама" → swap to Ollama
- "переключись на облако" / "используй грок" → swap to Groq
- "какой у тебя мозг" / "облако или локально" → status query
Underlying Lua API (registered globally):
jarvis.llm({messages}, opts) -- as before, now uses global client
jarvis.llm_status() -> "groq" | "ollama" | "none"
jarvis.llm_switch(name) -> name on success, nil on failure
Media keys pack (resources/commands/media_keys/, 4 commands)
- "пауза" / "плей" → VK_MEDIA_PLAY_PAUSE
- "следующий трек" → VK_MEDIA_NEXT_TRACK
- "предыдущий трек" → VK_MEDIA_PREV_TRACK
- "стоп музыка" → VK_MEDIA_STOP
- Helper _media.ps1 uses user32.keybd_event (P/Invoke through Add-Type).
- Works with anything that listens to global media keys: Spotify, Yandex Music,
YouTube (focused tab), Foobar2000, Winamp, etc. No OAuth, no API keys.
Codebase Q&A pack (resources/commands/codebase_qa/, 3 commands)
- "укажи проект <path>" → stores path in jarvis.memory under codebase.root
- "какой проект сейчас" → speaks current path
- "что делает функция X" / "найди в коде Y" / "объясни код"
→ walks the folder (depth 3, 30 files cap, 4KB per file, 50KB total),
filters by source extensions (rs/py/ts/lua/go/...), feeds digest to LLM
with a "senior reviewer" system prompt, speaks 3-5 sentence answer.
Scheduler cancel-by-text (crates/jarvis-core/src/scheduler.rs)
- new pub fn find_by_text(query, limit) -> Vec<ScheduledTask>
- new pub fn remove_by_text(query) -> usize (count removed)
- new pure helper find_by_text_in(tasks, query, limit) for tests
- Lua: jarvis.scheduler.{find_by_text, remove_by_text}
- Voice: "отмени напоминание про воду" / "удали задачу про разминку"
- 3 new unit tests (52 total in jarvis-core).
TTS pre-warm (crates/jarvis-app/src/main.rs)
- Call jarvis_core::tts::backend() once on startup so the first speak()
doesn't pay Piper binary discovery + voice loading cost. Cuts first-speak
latency by ~150-300ms depending on backend.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:15:59 +03:00
|
|
|
info!("LLM fallback enabled (backend: {}).", llm::current_backend_name());
|
2026-04-23 02:16:02 +03:00
|
|
|
Some(State {
|
|
|
|
|
max_tokens: config::LLM_DEFAULT_MAX_TOKENS,
|
|
|
|
|
})
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn is_enabled() -> bool {
|
|
|
|
|
STATE.get().and_then(|s| s.as_ref()).is_some()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn extract_prompt(text: &str) -> Option<String> {
|
|
|
|
|
let lang = i18n::get_language();
|
|
|
|
|
let triggers = config::get_llm_trigger_phrases(&lang);
|
|
|
|
|
let lowered = text.to_lowercase();
|
|
|
|
|
|
|
|
|
|
for trig in triggers {
|
|
|
|
|
if let Some(rest) = strip_trigger(&lowered, trig) {
|
|
|
|
|
let trimmed = rest.trim();
|
|
|
|
|
if !trimmed.is_empty() {
|
|
|
|
|
return Some(trimmed.to_string());
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
None
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn strip_trigger(text: &str, trigger: &str) -> Option<String> {
|
|
|
|
|
let t = text.trim_start();
|
|
|
|
|
if let Some(after) = t.strip_prefix(trigger) {
|
|
|
|
|
let next = after.chars().next();
|
|
|
|
|
if next.map_or(true, |c| !c.is_alphanumeric()) {
|
|
|
|
|
return Some(after.to_string());
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
None
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn handle(prompt: &str) {
|
|
|
|
|
let state = match STATE.get().and_then(|s| s.as_ref()) {
|
|
|
|
|
Some(s) => s,
|
|
|
|
|
None => {
|
|
|
|
|
warn!("LLM fallback called while disabled — ignoring.");
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
info!("LLM prompt: {}", prompt);
|
|
|
|
|
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
llm::history_push_user(prompt);
|
|
|
|
|
let mut snapshot: Vec<ChatMessage> = llm::history_snapshot();
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
// Inject (a) profile personality (b) relevant long-term memory as a fresh
|
|
|
|
|
// system message right after the base prompt. Both are optional.
|
|
|
|
|
let profile = profiles::active();
|
|
|
|
|
let mut overlay = String::new();
|
|
|
|
|
if !profile.llm_personality.is_empty() {
|
|
|
|
|
overlay.push_str(&format!("Активный профиль: {} {}\nХарактер для ответа: {}\n",
|
|
|
|
|
profile.icon, profile.name, profile.llm_personality));
|
|
|
|
|
}
|
|
|
|
|
let mem_ctx = long_term_memory::build_context(prompt, 5);
|
|
|
|
|
if !mem_ctx.is_empty() {
|
|
|
|
|
overlay.push_str(&mem_ctx);
|
|
|
|
|
}
|
|
|
|
|
if !overlay.is_empty() {
|
|
|
|
|
// Insert after the base system prompt (index 0 typically), before user msgs.
|
|
|
|
|
let insert_at = if !snapshot.is_empty() && snapshot[0].role == "system" { 1 } else { 0 };
|
|
|
|
|
snapshot.insert(insert_at, ChatMessage::system(overlay));
|
|
|
|
|
}
|
2026-04-23 02:16:02 +03:00
|
|
|
|
feat: hot-swap LLM backend + media keys + codebase Q&A + scheduler cancel-by-text + TTS pre-warm
Single biggest user-facing change: you can now switch local↔cloud LLM by voice
without restarting. Plus four new packs and a perf nudge.
Shared LLM global (crates/jarvis-core/src/llm/mod.rs)
- GLOBAL: Lazy<RwLock<Option<Arc<LlmClient>>>>. Single source of truth.
- init_global() / current() / swap_to(LlmBackend) / current_backend_name() / parse_backend(str).
- llm_fallback + llm_router migrated: they no longer own a separate LlmClient,
they read from the global on each request. Hot-swap is now LIVE — change
backend, next chat call uses the new one.
- parse_backend accepts both english and russian aliases:
"groq"/"cloud"/"облако"/"клауд" → Groq
"ollama"/"local"/"локал"/"локальный" → Ollama
Voice + Lua LLM switcher (resources/commands/llm_switch/, 3 commands)
- "переключись на локальный" / "перейди на оллама" → swap to Ollama
- "переключись на облако" / "используй грок" → swap to Groq
- "какой у тебя мозг" / "облако или локально" → status query
Underlying Lua API (registered globally):
jarvis.llm({messages}, opts) -- as before, now uses global client
jarvis.llm_status() -> "groq" | "ollama" | "none"
jarvis.llm_switch(name) -> name on success, nil on failure
Media keys pack (resources/commands/media_keys/, 4 commands)
- "пауза" / "плей" → VK_MEDIA_PLAY_PAUSE
- "следующий трек" → VK_MEDIA_NEXT_TRACK
- "предыдущий трек" → VK_MEDIA_PREV_TRACK
- "стоп музыка" → VK_MEDIA_STOP
- Helper _media.ps1 uses user32.keybd_event (P/Invoke through Add-Type).
- Works with anything that listens to global media keys: Spotify, Yandex Music,
YouTube (focused tab), Foobar2000, Winamp, etc. No OAuth, no API keys.
Codebase Q&A pack (resources/commands/codebase_qa/, 3 commands)
- "укажи проект <path>" → stores path in jarvis.memory under codebase.root
- "какой проект сейчас" → speaks current path
- "что делает функция X" / "найди в коде Y" / "объясни код"
→ walks the folder (depth 3, 30 files cap, 4KB per file, 50KB total),
filters by source extensions (rs/py/ts/lua/go/...), feeds digest to LLM
with a "senior reviewer" system prompt, speaks 3-5 sentence answer.
Scheduler cancel-by-text (crates/jarvis-core/src/scheduler.rs)
- new pub fn find_by_text(query, limit) -> Vec<ScheduledTask>
- new pub fn remove_by_text(query) -> usize (count removed)
- new pure helper find_by_text_in(tasks, query, limit) for tests
- Lua: jarvis.scheduler.{find_by_text, remove_by_text}
- Voice: "отмени напоминание про воду" / "удали задачу про разминку"
- 3 new unit tests (52 total in jarvis-core).
TTS pre-warm (crates/jarvis-app/src/main.rs)
- Call jarvis_core::tts::backend() once on startup so the first speak()
doesn't pay Piper binary discovery + voice loading cost. Cuts first-speak
latency by ~150-300ms depending on backend.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:15:59 +03:00
|
|
|
let client = match llm::current() {
|
|
|
|
|
Some(c) => c,
|
|
|
|
|
None => {
|
|
|
|
|
warn!("LLM call but global client missing — ignoring.");
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
match client.complete(&snapshot, state.max_tokens) {
|
2026-04-23 02:16:02 +03:00
|
|
|
Ok(reply) => {
|
|
|
|
|
let reply = reply.trim().to_string();
|
|
|
|
|
info!("LLM reply: {}", reply);
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
llm::history_push_assistant(reply.clone());
|
2026-05-15 01:51:02 +03:00
|
|
|
ipc::send(IpcEvent::LlmReply { text: reply.clone() });
|
2026-04-23 02:16:02 +03:00
|
|
|
voices::play_ok();
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
speak_reply(&reply);
|
2026-04-23 02:16:02 +03:00
|
|
|
}
|
|
|
|
|
Err(e) => {
|
|
|
|
|
error!("LLM request failed: {}", e);
|
feat: persist LLM/TTS backend choice + reset/repeat context + quick-search + diagnostics
Closes the UX hole I created in 5c72450: voice-swap to Ollama used to vanish
after restart. Plus P0.3 (long-standing roadmap item) and two new packs.
Persistent backend choice (crates/jarvis-core/src/db/structs.rs)
- Settings struct gains llm_backend + tts_backend fields (both String,
"" / "auto" = follow env/auto-detect).
- set("llm_backend", "groq"|"ollama"|"auto") validates input.
- llm::init_global() reads DB first, then JARVIS_LLM env, then auto-detect.
- llm::swap_to() now persists the choice via db::save_settings.
- Voice swap "переключись на локальный" now survives restart.
Shared conversation history (P0.3, crates/jarvis-core/src/llm/mod.rs)
- HISTORY: Lazy<RwLock<Option<ConversationHistory>>> singleton.
- Helpers: init_history, history_push_user, history_push_assistant,
history_snapshot, history_clear, history_pop_last_user, history_last_assistant.
- llm_fallback migrated off its own Mutex<History> — now reads/writes shared.
- ConversationHistory gains last_assistant() method.
New Lua APIs
- jarvis.llm_reset() → clear conversation turns (keeps system prompt).
- jarvis.llm_last_reply() → string or nil (last assistant message text).
- jarvis.health() → debug table {tts_backend, llm_backend, llm_model,
active_profile, memory_facts, scheduled_tasks,
language, voice, microphone, vosk_model,
noise_suppression}. No secrets included.
New voice packs
- resources/commands/llm_context/ (P0.3)
* "сбрось контекст" / "забудь разговор" → llm.reset
* "повтори последнее" / "повтори ответ" → llm.repeat (uses last_assistant)
- resources/commands/quick_search/ (imba P1 item)
* "найди в гугле <X>" / "загугли <X>"
* Uses DuckDuckGo Instant Answer API (api.duckduckgo.com, no key required).
Pulls AbstractText or RelatedTopics into LLM prompt; falls back to pure
LLM knowledge if DDG returns nothing useful. Speaks 2-4 sentence answer.
- resources/commands/diagnostics/
* "диагностика" / "доложи о себе" / "статус"
* Reads jarvis.health() and speaks a one-line summary. Useful when
debugging — user can read out their current state for a bug report.
Build: cargo build --release -p jarvis-app -p jarvis-gui green.
Tests: 52/52 jarvis-core unit tests pass.
2026-05-15 16:25:28 +03:00
|
|
|
llm::history_pop_last_user();
|
2026-05-15 01:51:02 +03:00
|
|
|
let err_text = config::LLM_FALLBACK_ERROR_RU.to_string();
|
|
|
|
|
ipc::send(IpcEvent::LlmReply { text: err_text.clone() });
|
2026-04-23 02:16:02 +03:00
|
|
|
voices::play_error();
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
speak_reply(&err_text);
|
2026-04-23 02:16:02 +03:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-05-15 01:51:02 +03:00
|
|
|
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
fn speak_reply(text: &str) {
|
refactor: maintainability pass — dedup, central env config, cmd helpers, tests, ARCHITECTURE.md
User asked the code stays easy to fix/extend as the feature surface grows.
This commit refactors what was duplicated, centralises what was scattered,
adds tests where there were none, and writes the architecture doc that
contributors will need.
Deduplication
- tts::play_wav extracted to tts/mod.rs as pub(crate). Was identical in
piper.rs and silero.rs (~25 lines × 2). Both backends now call super::play_wav.
Centralised env config
- new crates/jarvis-core/src/runtime_config.rs — single doc-file for every
JARVIS_* / GROQ_* / OLLAMA_* env var. Includes:
- ENV_* constants with doc comments
- get(name) / get_bool(name, default) / get_parse(name, default) helpers
- feature-flag wrappers: llm_tts_enabled(), llm_router_enabled(),
llm_router_threshold()
- log_effective_config() prints active values on startup
- migrated llm_fallback (JARVIS_LLM_TTS), llm_router (JARVIS_LLM_ROUTER,
JARVIS_LLM_ROUTER_THRESHOLD) to use the new helpers. Pattern set for
future migrations.
Lua boilerplate killer
- new crates/jarvis-core/src/lua/api/cmd.rs exposing:
jarvis.cmd.ok(msg?) — play_ok + speak + {chain=false}
jarvis.cmd.chain_ok(msg?) — same but chain=true
jarvis.cmd.error(msg?) — play_error + speak + {chain=false}
jarvis.cmd.not_found(msg?) — play_not_found + speak + {chain=false}
jarvis.cmd.silent_ok / silent_error
- refactored 5 packs (daily_briefing/off, memory_pack/list, pomodoro/stop,
habit_nudge/stop_all, scheduler/clear) — each lost 3-4 lines of repetitive
play_*/speak/return boilerplate. Pattern for future packs documented in
ARCHITECTURE.md.
Tests (was 32, now 49)
- long_term_memory: 8 new tests for normalize_key, search_in (rank, limit,
empty), build_context_from (empty + populated), serde round-trips for
MemoryRecord and Store. Extracted pure logic (search_in, build_context_from)
into pub(crate) functions to enable testing without global state.
- profiles: 6 new tests for Profile::allows_command (empty/whitelist/blacklist/
deny-wins-over-allow), serde round-trip with all fields + minimal-fields
tolerance via #[serde(default)].
- runtime_config: 2 tests for get_bool / get_parse defaults.
- All 49 tests pass.
New mood/energy log pack
- resources/commands/mood_log/ with 2 commands:
mood.record "запиши настроение 7" / "сегодня мне грустно" → stores
timestamped entry via jarvis.memory.remember
mood.recap "как прошла неделя" → LLM summarises last 30 entries
- Showcase: composes memory + llm + cmd helpers in <30 lines per script.
ARCHITECTURE.md (new, 250 lines)
- Crate layout, data flow diagram (mic→action 10 steps), per-module
responsibility table, configuration layers, TTS pipeline diagram,
Lua sandbox details with API quick-ref, background-services overview,
"how to add a pack/feature/TTS backend" recipes, test coverage map,
build instructions with MSVC env, git workflow with Forgejo NO_PROXY trick.
- Aimed at someone who just cloned the repo and needs to fix a bug fast.
Build: cargo build --release -p jarvis-app -p jarvis-gui both green.
Tests: 49/49 jarvis-core unit tests pass.
2026-05-15 16:06:18 +03:00
|
|
|
if !jarvis_core::runtime_config::llm_tts_enabled() {
|
2026-05-15 01:51:02 +03:00
|
|
|
return;
|
|
|
|
|
}
|
feat: TTS backend abstraction + 4 'imba' features + Lua packs
Big push driven by user feedback ("делай имбу") and web research on what
voice assistants need to be the ideal:
TTS backend abstraction (P0.1)
- new module crates/jarvis-core/src/tts/{mod,sapi,piper,silero}.rs
- TtsBackend trait with SapiBackend (current PowerShell), PiperBackend
(rhasspy/piper, neural quality), SileroBackend (python subprocess)
- JARVIS_TTS env var picks (sapi|piper|silero). Auto-detect Piper if
binary + voice present in tools/piper/. Falls back to SAPI on missing.
- SpeakOpts {lang, detached, raw} replaces ad-hoc args. text_utils
sanitiser applied unless raw=true.
- llm_fallback + lua/api/tts both routed through tts::backend().
- tools/piper/install.ps1 downloads piper.exe + ru_RU-irina-medium.onnx
from rhasspy releases + huggingface. Smoke-test included.
- tools/silero/silero_tts.py helper (PyTorch); rust spawns it as subprocess.
IMBA-1 Agentic LLM router
- crates/jarvis-app/src/llm_router.rs
- When fuzzy/intent matcher fails, LLM picks the closest command from the
full registry. Returns JSON {command_id, confidence, reason}.
- Threshold-gated re-dispatch via substitute phrase. JARVIS_LLM_ROUTER=1
enables; JARVIS_LLM_ROUTER_THRESHOLD overrides 0.55 default.
- Inserted in app.rs::execute_command between "no match" and existing
llm_fallback chat fallback.
IMBA-2 Long-term memory
- crates/jarvis-core/src/long_term_memory.rs — JSON store at
APP_CONFIG_DIR/long_term_memory.json. Atomic write-through.
- remember/recall/search/forget/all/build_context API.
- Lua bindings: jarvis.memory.* (5 functions).
- llm_fallback auto-injects relevant facts (substring search of prompt)
into system message before LLM call.
- Pack resources/commands/memory_pack/ with 4 commands: remember, recall,
forget, list.
IMBA-3 Profile switching (work/game/sleep/driving/default)
- crates/jarvis-core/src/profiles.rs — JSON profiles at APP_CONFIG_DIR/profiles/
Auto-seeds 5 defaults on first run with personality + allow/deny lists +
greetings + emoji icons.
- active_profile.txt persists choice across restart.
- Lua bindings: jarvis.profile.{active,set,list,allows,active_name}.
- llm_fallback prepends profile personality to system prompt.
- Pack resources/commands/profile_switch/ with 6 voice triggers.
IMBA-4 Multimodal screenshot + vision LLM
- crates/jarvis-core/src/lua/api/vision.rs — gated on HTTP sandbox.
- jarvis.vision.screenshot() captures via PowerShell System.Drawing.
- jarvis.vision.describe(prompt?) sends base64 PNG to Groq vision model
(default llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- Pack resources/commands/vision/ with 2 commands: describe + read_error.
P0.2 Continuous conversation grace window
- config::CONVERSATION_GRACE_MS = 30_000.
- app.rs: after command result, if grace_ms > 0 keep listening WITHOUT
re-wake for the grace duration. Existing CMS_WAIT_DELAY back-dated so
the existing timeout fires at start + grace_ms.
Tests: 24/24 jarvis-core unit tests pass (including 5 text_utils).
Build: cargo build --release -p jarvis-app and -p jarvis-gui both succeed
on Windows MSVC (VS 2026 Enterprise vcvars64).
Notes for setup:
- Piper voice install: pwsh tools/piper/install.ps1 (downloads ~90 MB).
- GROQ_TOKEN needed for IMBA-1 (router) and IMBA-4 (vision).
- All features are opt-in via env vars or auto-detect; existing SAPI +
fuzzy match path remains the default.
2026-05-15 15:32:44 +03:00
|
|
|
tts::speak(text, &SpeakOpts::lang("ru"));
|
2026-05-15 01:51:02 +03:00
|
|
|
}
|