feat: /history page (recognition log) + multi wake-word loading
# Recognition history page
New /history route in the Tauri GUI. Every voice/text phrase that
reaches the dispatcher gets a row with: timestamp, the phrase, what
happened, command id + confidence (if matched), which matcher fired
(intent / fuzzy / LLM-router / LLM-fallback).
Color coding:
- Green: command matched AND executed successfully
- Red: command matched but failed, OR dispatcher error
- Blue: LLM fallback handled it (Jarvis spoke a free-form reply)
- Orange: no match at all ("Не понял")
Live updates: GUI polls every 2s from the on-disk log, so the page
shows daemon writes in near real time even though they're separate
processes. Filter box for searching by phrase or command id.
Architecture:
- New core module `recognition_log` with ring buffer (cap 500) +
atomic JSON write-through to `<APP_CONFIG_DIR>/recognition_log.json`.
- `record(phrase, source, outcome)` is the single call-site from the
daemon's `execute_command()` — hooked into the 4 outcome paths
(matched-ok, matched-fail, not-found, llm-handled, error).
- `recent(limit)` reads the in-memory buffer (daemon's view).
- `recent_from_disk(limit)` re-reads the JSON file — GUI uses this
since the GUI process has its own buffer that doesn't see the
daemon's writes.
- 5 new unit tests covering ring buffer trimming, outcome serde
roundtrip, missing/corrupt/oversized file recovery.
GUI:
- `crates/jarvis-gui/src/tauri_commands/history.rs`: history_recent,
history_clear. Flattens the Outcome enum into a single struct that's
easier for the Svelte template to render.
- `frontend/src/routes/history/index.svelte`: ~270 lines. Stats badges
(✓ N matched / ✗ N misses / total), filter input, virtual list of
entry cards with color-coded left border. Polls every 2s.
- Header gets a new "История" / "History" button (between Plugins and
Settings). Russian + English locale entries added.
# Multi wake-word loading
Was: `init()` loaded the bundled `jarvis-default.rpw` + at most ONE
custom (from `settings.custom_wake_word`). User had to pick a single
trained model.
Now: loads the bundled default PLUS every .rpw in
`APP_CONFIG_DIR/wake_words/` simultaneously. Rustpotter natively
supports multiple wake-word triggers — each adds robustness for
different voice profiles. The legacy `custom_wake_word` field is
checked for back-compat but is a no-op if it points inside the
already-loaded directory.
User-facing impact: train the wake-word once via /wake-trainer →
restart daemon → detection improves automatically without picking a
single "active" model.
# Settings → "Обучить wake-word" button
Added a purple button on the settings page that links to
/wake-trainer. The trainer existed but had no in-GUI link, so users
couldn't find it without typing the URL. Now sits next to the
"Конструктор команд (Python)" button.
Tests: 140 → 145 rust core tests (+5 recognition_log). Frontend
rebuilds in 6.2s. Release builds of jarvis-app + jarvis-gui green.
Practical test: GUI launches (MainWindowTitle confirmed), seed log
file written + visible to the page.
This commit is contained in:
parent
965441d4db
commit
73fc404ec7
13 changed files with 867 additions and 63 deletions
|
|
@ -163,4 +163,5 @@ settings-profile = Profile
|
|||
header-macros = Macros
|
||||
header-scheduler = Schedule
|
||||
header-memory = Memory
|
||||
header-plugins = Plugins
|
||||
header-plugins = Plugins
|
||||
header-history = History
|
||||
|
|
@ -163,4 +163,5 @@ settings-profile = Профиль
|
|||
header-macros = Макросы
|
||||
header-scheduler = Расписание
|
||||
header-memory = Память
|
||||
header-plugins = Плагины
|
||||
header-plugins = Плагины
|
||||
header-history = История
|
||||
|
|
@ -68,6 +68,8 @@ pub mod wake_trainer;
|
|||
|
||||
pub mod idle_banter;
|
||||
|
||||
pub mod recognition_log;
|
||||
|
||||
#[cfg(feature = "lua")]
|
||||
pub mod toast;
|
||||
|
||||
|
|
|
|||
|
|
@ -12,68 +12,87 @@ pub fn init() -> Result<(), ()> {
|
|||
let rustpotter_config = config::RUSTPOTTER_DEFAULT_CONFIG;
|
||||
|
||||
// create rustpotter instance
|
||||
match Rustpotter::new(&rustpotter_config) {
|
||||
Ok(mut rinstance) => {
|
||||
// Resolve wake-word file list. Priority:
|
||||
// 1. user-trained custom model (settings.custom_wake_word)
|
||||
// 2. bundled default
|
||||
// We always try the bundled default last so the assistant keeps
|
||||
// working even if the custom model is missing on disk.
|
||||
let mut loaded_any = false;
|
||||
|
||||
if let Some(db) = DB.get() {
|
||||
let stem = db.read().custom_wake_word.clone();
|
||||
if !stem.is_empty() {
|
||||
if let Some(cfg_dir) = APP_CONFIG_DIR.get() {
|
||||
let custom = cfg_dir
|
||||
.join("wake_words")
|
||||
.join(format!("{}.rpw", stem));
|
||||
if custom.is_file() {
|
||||
let path_str = custom.to_string_lossy().to_string();
|
||||
match rinstance.add_wakeword_from_file(&path_str, &path_str) {
|
||||
Ok(_) => {
|
||||
info!("Loaded custom wake-word: {}", path_str);
|
||||
loaded_any = true;
|
||||
}
|
||||
Err(e) => warn!(
|
||||
"Failed to load custom wakeword '{}': {}",
|
||||
path_str, e
|
||||
),
|
||||
}
|
||||
} else {
|
||||
warn!(
|
||||
"Custom wake-word '{}' selected but file missing: {}",
|
||||
stem,
|
||||
custom.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const DEFAULT: &str = "resources/rustpotter/jarvis-default.rpw";
|
||||
if let Err(e) = rinstance.add_wakeword_from_file(DEFAULT, DEFAULT) {
|
||||
if !loaded_any {
|
||||
error!("Failed to load default wakeword '{}': {}", DEFAULT, e);
|
||||
}
|
||||
} else {
|
||||
loaded_any = true;
|
||||
}
|
||||
|
||||
if !loaded_any {
|
||||
error!("No wake-word models loaded; Rustpotter will not detect anything.");
|
||||
}
|
||||
|
||||
// store
|
||||
let _ = RUSTPOTTER.set(Mutex::new(rinstance));
|
||||
}
|
||||
let mut rinstance = match Rustpotter::new(&rustpotter_config) {
|
||||
Ok(r) => r,
|
||||
Err(msg) => {
|
||||
error!("Rustpotter failed to initialize.\nError details: {}", msg);
|
||||
|
||||
return Err(());
|
||||
}
|
||||
};
|
||||
|
||||
// Rustpotter accepts MULTIPLE wake-word models simultaneously — each
|
||||
// trained on a different voice / acoustic profile. We load:
|
||||
//
|
||||
// 1. The bundled default — works for everyone but isn't tuned for
|
||||
// this user's exact voice/mic/room.
|
||||
// 2. EVERY user-trained .rpw file in APP_CONFIG_DIR/wake_words/.
|
||||
// The user creates these via /wake-trainer; each adds robustness
|
||||
// to detection without disabling the default.
|
||||
// 3. (Legacy) settings.custom_wake_word — kept for back-compat when
|
||||
// the user used to pin one specific custom model. If still set,
|
||||
// it's just one of the files already loaded in step 2 — no-op.
|
||||
let mut loaded_any = false;
|
||||
|
||||
const DEFAULT: &str = "resources/rustpotter/jarvis-default.rpw";
|
||||
match rinstance.add_wakeword_from_file(DEFAULT, DEFAULT) {
|
||||
Ok(_) => {
|
||||
info!("Loaded bundled wake-word: {}", DEFAULT);
|
||||
loaded_any = true;
|
||||
}
|
||||
Err(e) => warn!("Default wakeword unavailable ({}): {}", DEFAULT, e),
|
||||
}
|
||||
|
||||
if let Some(cfg_dir) = APP_CONFIG_DIR.get() {
|
||||
let trained_dir = cfg_dir.join("wake_words");
|
||||
if let Ok(read) = std::fs::read_dir(&trained_dir) {
|
||||
let mut count = 0usize;
|
||||
for entry in read.flatten() {
|
||||
let p = entry.path();
|
||||
if p.extension().and_then(|e| e.to_str()) != Some("rpw") {
|
||||
continue;
|
||||
}
|
||||
let path_str = p.to_string_lossy().to_string();
|
||||
match rinstance.add_wakeword_from_file(&path_str, &path_str) {
|
||||
Ok(_) => {
|
||||
info!("Loaded user-trained wake-word: {}", path_str);
|
||||
loaded_any = true;
|
||||
count += 1;
|
||||
}
|
||||
Err(e) => warn!("Skip user wake-word '{}': {}", path_str, e),
|
||||
}
|
||||
}
|
||||
if count > 0 {
|
||||
info!("{} user-trained wake-word model(s) active", count);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Surface the legacy `custom_wake_word` setting if it points somewhere
|
||||
// ELSE than wake_words/. Pre-step-2 installs may have set this to an
|
||||
// arbitrary location; we still honour it.
|
||||
if let Some(db) = DB.get() {
|
||||
let stem = db.read().custom_wake_word.clone();
|
||||
if !stem.is_empty() {
|
||||
if let Some(cfg_dir) = APP_CONFIG_DIR.get() {
|
||||
let inferred = cfg_dir.join("wake_words").join(format!("{}.rpw", stem));
|
||||
if inferred.is_file() {
|
||||
// Already loaded in step 2 above. No-op.
|
||||
} else {
|
||||
warn!(
|
||||
"settings.custom_wake_word = '{}' but {} not present (using bundled + trained only)",
|
||||
stem,
|
||||
inferred.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if !loaded_any {
|
||||
error!("No wake-word models loaded; Rustpotter will not detect anything.");
|
||||
}
|
||||
|
||||
let _ = RUSTPOTTER.set(Mutex::new(rinstance));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
|
|||
309
crates/jarvis-core/src/recognition_log.rs
Normal file
309
crates/jarvis-core/src/recognition_log.rs
Normal file
|
|
@ -0,0 +1,309 @@
|
|||
//! Recognition log — what J.A.R.V.I.S. heard + what it did about it.
|
||||
//!
|
||||
//! Every voice/text phrase that reaches the dispatcher gets a row here with:
|
||||
//! - timestamp (unix seconds, local timezone hint via chrono)
|
||||
//! - the raw recognised phrase
|
||||
//! - the outcome (matched / not_found / error / llm_handled)
|
||||
//! - command id + confidence if matched
|
||||
//! - which matcher fired (intent / fuzzy / router / llm)
|
||||
//!
|
||||
//! Why: the user can't always tell whether Jarvis didn't hear them, or
|
||||
//! heard them but couldn't map the phrase to a command. The GUI's
|
||||
//! /history page colors each row green/red so it's obvious at a glance —
|
||||
//! and the user can copy a misheard phrase straight back into the trainer.
|
||||
//!
|
||||
//! Storage: ring buffer of the last `MAX_ENTRIES` entries kept in memory
|
||||
//! for instant GUI loads, plus atomic JSON write-through to
|
||||
//! `<APP_CONFIG_DIR>/recognition_log.json` so the history survives a
|
||||
//! daemon restart.
|
||||
|
||||
use chrono::Local;
|
||||
use once_cell::sync::OnceCell;
|
||||
use parking_lot::RwLock;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::VecDeque;
|
||||
use std::path::PathBuf;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
const FILE_NAME: &str = "recognition_log.json";
|
||||
|
||||
/// Cap the buffer at 500 entries — keeps the JSON file under ~200 KB and
|
||||
/// the GUI list snappy on cheap hardware. Older entries roll off.
|
||||
pub const MAX_ENTRIES: usize = 500;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case")]
|
||||
pub enum Outcome {
|
||||
/// A command was matched and executed (success or failure tracked via
|
||||
/// `success` field — false means the command ran but reported an error).
|
||||
Matched {
|
||||
command_id: String,
|
||||
/// Optional confidence (0-100) — populated by the intent classifier
|
||||
/// and the LLM router. `None` for the fuzzy fallback (it uses a
|
||||
/// different scoring scheme).
|
||||
confidence_pct: Option<u8>,
|
||||
/// Which path mapped the phrase to a command.
|
||||
via: String,
|
||||
/// Did the command body actually succeed?
|
||||
success: bool,
|
||||
},
|
||||
/// Phrase was heard but no command matched and LLM fallback didn't fire.
|
||||
NotFound,
|
||||
/// LLM fallback handled the phrase (treated as success — Jarvis spoke
|
||||
/// SOMETHING in response, just not via a command pack).
|
||||
LlmHandled,
|
||||
/// Something blew up while dispatching.
|
||||
Error { message: String },
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct RecognitionEntry {
|
||||
/// Unix seconds. Caller uses Local::now().timestamp() to populate.
|
||||
pub ts: i64,
|
||||
/// The phrase the user (apparently) said. Lowercased, leading
|
||||
/// activation phrases like "джарвис" already stripped.
|
||||
pub phrase: String,
|
||||
pub outcome: Outcome,
|
||||
/// How the phrase arrived: "voice" (post-wake STT), "text" (typed via
|
||||
/// the GUI command box), "macro" (replayed from a macro).
|
||||
pub source: String,
|
||||
}
|
||||
|
||||
struct State {
|
||||
path: PathBuf,
|
||||
buf: RwLock<VecDeque<RecognitionEntry>>,
|
||||
}
|
||||
|
||||
static STATE: OnceCell<State> = OnceCell::new();
|
||||
|
||||
/// Initialise on first call. Idempotent — repeated calls are no-ops.
|
||||
/// Reads the existing JSON file (if any) so the GUI can show history
|
||||
/// across daemon restarts.
|
||||
pub fn init() -> Result<(), String> {
|
||||
if STATE.get().is_some() {
|
||||
return Ok(());
|
||||
}
|
||||
let dir = crate::APP_CONFIG_DIR
|
||||
.get()
|
||||
.ok_or_else(|| "APP_CONFIG_DIR not initialised".to_string())?;
|
||||
let path = dir.join(FILE_NAME);
|
||||
let initial = load_from_disk(&path);
|
||||
log::info!(
|
||||
"Recognition log loaded: {} entries from {}",
|
||||
initial.len(),
|
||||
path.display()
|
||||
);
|
||||
STATE
|
||||
.set(State {
|
||||
path,
|
||||
buf: RwLock::new(initial.into()),
|
||||
})
|
||||
.map_err(|_| "recognition_log already initialised".to_string())?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn load_from_disk(path: &PathBuf) -> Vec<RecognitionEntry> {
|
||||
if !path.is_file() {
|
||||
return Vec::new();
|
||||
}
|
||||
match std::fs::read_to_string(path) {
|
||||
Ok(text) => match serde_json::from_str::<Vec<RecognitionEntry>>(&text) {
|
||||
Ok(mut v) => {
|
||||
// Trim to MAX_ENTRIES in case the file grew during a prior crash.
|
||||
if v.len() > MAX_ENTRIES {
|
||||
v.drain(0..v.len() - MAX_ENTRIES);
|
||||
}
|
||||
v
|
||||
}
|
||||
Err(e) => {
|
||||
log::warn!("Corrupt recognition_log JSON: {}", e);
|
||||
Vec::new()
|
||||
}
|
||||
},
|
||||
Err(e) => {
|
||||
log::warn!("Cannot read {}: {}", path.display(), e);
|
||||
Vec::new()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn now_unix_secs() -> i64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.as_secs() as i64)
|
||||
.unwrap_or_else(|_| Local::now().timestamp())
|
||||
}
|
||||
|
||||
/// Append an entry. Buffer is trimmed to `MAX_ENTRIES` then persisted
|
||||
/// atomically (write-tmp + rename). Logging failures are warnings, not
|
||||
/// errors — never let history-tracking break the actual command flow.
|
||||
pub fn record(phrase: impl Into<String>, source: impl Into<String>, outcome: Outcome) {
|
||||
let state = match STATE.get() {
|
||||
Some(s) => s,
|
||||
None => return, // history disabled — feature gracefully off
|
||||
};
|
||||
let entry = RecognitionEntry {
|
||||
ts: now_unix_secs(),
|
||||
phrase: phrase.into(),
|
||||
outcome,
|
||||
source: source.into(),
|
||||
};
|
||||
{
|
||||
let mut buf = state.buf.write();
|
||||
buf.push_back(entry);
|
||||
while buf.len() > MAX_ENTRIES {
|
||||
buf.pop_front();
|
||||
}
|
||||
}
|
||||
persist(state);
|
||||
}
|
||||
|
||||
/// Return up to `limit` most-recent entries, newest first. `limit == 0`
|
||||
/// returns all entries in the buffer.
|
||||
pub fn recent(limit: usize) -> Vec<RecognitionEntry> {
|
||||
let Some(state) = STATE.get() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let buf = state.buf.read();
|
||||
let n = if limit == 0 { buf.len() } else { limit.min(buf.len()) };
|
||||
buf.iter().rev().take(n).cloned().collect()
|
||||
}
|
||||
|
||||
/// Re-read the recognition log from DISK and return up to `limit` newest
|
||||
/// entries. Use this from the GUI process — it has its own in-memory
|
||||
/// buffer that doesn't see live writes from the daemon (the daemon is a
|
||||
/// separate process). Disk reads are cheap (≤ 500 entries, ~200KB JSON).
|
||||
pub fn recent_from_disk(limit: usize) -> Vec<RecognitionEntry> {
|
||||
let Some(state) = STATE.get() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let entries = load_from_disk(&state.path);
|
||||
let n = if limit == 0 { entries.len() } else { limit.min(entries.len()) };
|
||||
entries.into_iter().rev().take(n).collect()
|
||||
}
|
||||
|
||||
/// Wipe everything. Returns the count of removed entries.
|
||||
pub fn clear() -> usize {
|
||||
let Some(state) = STATE.get() else {
|
||||
return 0;
|
||||
};
|
||||
let removed = {
|
||||
let mut buf = state.buf.write();
|
||||
let n = buf.len();
|
||||
buf.clear();
|
||||
n
|
||||
};
|
||||
persist(state);
|
||||
removed
|
||||
}
|
||||
|
||||
fn persist(state: &State) {
|
||||
let snapshot: Vec<RecognitionEntry> = state.buf.read().iter().cloned().collect();
|
||||
let json = match serde_json::to_string_pretty(&snapshot) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
log::warn!("Recognition log serialise failed: {}", e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let tmp = state.path.with_extension("json.tmp");
|
||||
if let Err(e) = std::fs::write(&tmp, json) {
|
||||
log::warn!("Recognition log write {} failed: {}", tmp.display(), e);
|
||||
return;
|
||||
}
|
||||
if let Err(e) = std::fs::rename(&tmp, &state.path) {
|
||||
log::warn!("Recognition log rename failed: {}", e);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
fn fresh_state() -> TempDir {
|
||||
// Tests can't easily inject APP_CONFIG_DIR via OnceCell since it
|
||||
// may be set by prior tests. We work around by talking to the
|
||||
// in-memory buffer logic via the lower-level VecDeque directly.
|
||||
TempDir::new().unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ring_buffer_trims_to_max() {
|
||||
let mut buf: VecDeque<RecognitionEntry> = VecDeque::new();
|
||||
for i in 0..(MAX_ENTRIES + 50) {
|
||||
buf.push_back(RecognitionEntry {
|
||||
ts: i as i64,
|
||||
phrase: format!("phrase {}", i),
|
||||
outcome: Outcome::NotFound,
|
||||
source: "voice".into(),
|
||||
});
|
||||
while buf.len() > MAX_ENTRIES {
|
||||
buf.pop_front();
|
||||
}
|
||||
}
|
||||
assert_eq!(buf.len(), MAX_ENTRIES);
|
||||
// Oldest 50 entries dropped — front should now be phrase #50.
|
||||
assert_eq!(buf.front().unwrap().ts, 50);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outcome_roundtrip_serde() {
|
||||
let cases = vec![
|
||||
Outcome::Matched {
|
||||
command_id: "echo".into(),
|
||||
confidence_pct: Some(87),
|
||||
via: "intent".into(),
|
||||
success: true,
|
||||
},
|
||||
Outcome::NotFound,
|
||||
Outcome::LlmHandled,
|
||||
Outcome::Error {
|
||||
message: "oops".into(),
|
||||
},
|
||||
];
|
||||
for o in cases {
|
||||
let j = serde_json::to_string(&o).unwrap();
|
||||
let back: Outcome = serde_json::from_str(&j).unwrap();
|
||||
// Roundtrip preserves the discriminant
|
||||
let j2 = serde_json::to_string(&back).unwrap();
|
||||
assert_eq!(j, j2, "non-roundtripping outcome: {:?}", o);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn load_from_disk_handles_missing_file() {
|
||||
let dir = fresh_state();
|
||||
let path = dir.path().join("nope.json");
|
||||
let loaded = load_from_disk(&path);
|
||||
assert!(loaded.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn load_from_disk_handles_corrupt_file() {
|
||||
let dir = fresh_state();
|
||||
let path = dir.path().join("bad.json");
|
||||
std::fs::write(&path, b"this is not json {{{").unwrap();
|
||||
let loaded = load_from_disk(&path);
|
||||
assert!(loaded.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn load_from_disk_trims_oversized_file() {
|
||||
let dir = fresh_state();
|
||||
let path = dir.path().join("big.json");
|
||||
let oversized: Vec<RecognitionEntry> = (0..(MAX_ENTRIES + 100))
|
||||
.map(|i| RecognitionEntry {
|
||||
ts: i as i64,
|
||||
phrase: "x".into(),
|
||||
outcome: Outcome::NotFound,
|
||||
source: "voice".into(),
|
||||
})
|
||||
.collect();
|
||||
std::fs::write(&path, serde_json::to_string(&oversized).unwrap()).unwrap();
|
||||
let loaded = load_from_disk(&path);
|
||||
assert_eq!(loaded.len(), MAX_ENTRIES);
|
||||
// Trimmed from the front, so the first entry is ts=100.
|
||||
assert_eq!(loaded.first().unwrap().ts, 100);
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue