feat: LLM auto-fallback + codegen pack + OCR pack
LLM auto-fallback (jarvis-app/src/app.rs + jarvis-core/src/config.rs):
- When neither intent classifier nor levenshtein finds a command match,
route the utterance straight to the LLM instead of just playing the
"not found" sound. Triggers ("скажи X", "answer Y") still work and
take precedence — they short-circuit before command lookup.
- Two new config knobs:
LLM_AUTO_FALLBACK = true — master toggle
LLM_AUTO_FALLBACK_MIN_CHARS = 4 — suppress for very short utterances
so background noise doesn't burn
Groq quota
- Requires GROQ_TOKEN; if absent, behaviour is unchanged (play_not_found).
codegen/ command pack:
- Phrases: "напиши код X", "сгенерируй скрипт Y", "write code Z", etc.
- Strips the trigger from the recognized phrase, sends what remains to
Groq with a strict system prompt ("return ONLY code, no fences, no
commentary") at temperature 0.2.
- Parses the JSON content, unescapes \n / \" / \\ / \t, strips any
remaining ```lang ... ``` fences, drops the result into the clipboard
via jarvis.system.clipboard.set.
- Notifies a 120-char preview + plays ok-sound. Works for any language
the model handles (Python by default if unspecified).
- GROQ_TOKEN / GROQ_MODEL / GROQ_BASE_URL read from env at call time —
same envvars the voice-loop fallback already uses.
ocr/ command pack:
- Phrases: "прочитай экран", "что на экране", "read screen", etc.
- Captures the primary screen via System.Windows.Forms + System.Drawing
to a temp PNG, then shells to tesseract.exe (-l rus+eng for ru/ua,
-l eng for en).
- Resolves Tesseract by PATH first, then C:\Program Files\Tesseract-OCR
and the x86 install dir; if none works the user gets a friendly
"winget install UB-Mannheim.TesseractOCR" hint in a notification.
- Recognized text goes to clipboard + a 200-char preview notification.
All three features are portable (env-var resolution, no hardcoded user
paths). Command-pack total is now 11. cargo test -p jarvis-core --lib
commands::tests passes 3/3.
This commit is contained in:
parent
429113175a
commit
69a38b0d89
6 changed files with 255 additions and 5 deletions
|
|
@ -459,12 +459,21 @@ fn execute_command(text: &str, rt: &tokio::runtime::Runtime) -> bool {
|
|||
}
|
||||
} else {
|
||||
info!("No command found for: {}", text);
|
||||
voices::play_not_found();
|
||||
ipc::send(IpcEvent::Error {
|
||||
message: format!("Command not found: {}", text)
|
||||
});
|
||||
|
||||
if jarvis_core::config::LLM_AUTO_FALLBACK
|
||||
&& crate::llm_fallback::is_enabled()
|
||||
&& text.chars().count() >= jarvis_core::config::LLM_AUTO_FALLBACK_MIN_CHARS
|
||||
{
|
||||
info!("Auto-routing to LLM (no command match): {}", text);
|
||||
crate::llm_fallback::handle(text);
|
||||
} else {
|
||||
voices::play_not_found();
|
||||
ipc::send(IpcEvent::Error {
|
||||
message: format!("Command not found: {}", text)
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
ipc::send(IpcEvent::Idle);
|
||||
false // no chain on error or not found
|
||||
}
|
||||
|
|
|
|||
|
|
@ -197,6 +197,13 @@ pub const CMS_WAIT_DELAY: std::time::Duration = std::time::Duration::from_secs(1
|
|||
pub const LLM_DEFAULT_ENABLED: bool = true;
|
||||
pub const LLM_DEFAULT_MAX_HISTORY: usize = 8;
|
||||
pub const LLM_DEFAULT_MAX_TOKENS: u32 = 256;
|
||||
|
||||
// Auto-route unrecognized commands to the LLM (no "скажи" trigger required).
|
||||
// Triggers still work and take precedence — this only kicks in when neither
|
||||
// the intent classifier nor the levenshtein fallback found a command match.
|
||||
pub const LLM_AUTO_FALLBACK: bool = true;
|
||||
// Don't auto-fallback for very short utterances — too noisy.
|
||||
pub const LLM_AUTO_FALLBACK_MIN_CHARS: usize = 4;
|
||||
pub const LLM_SYSTEM_PROMPT_RU: &str = "Ты — J.A.R.V.I.S. (Just A Rather Very Intelligent System), \
|
||||
ИИ-ассистент Тони Старка из киновселенной Marvel (до событий Age of Ultron — ты НЕ Vision и НЕ FRIDAY). \
|
||||
Ведёшь себя как британский дворецкий: вежливо, иронично, с лёгким сарказмом, обращаешься к пользователю «сэр». \
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue