J.A.R.V.I.S-rust/ARCHITECTURE.md

# J.A.R.V.I.S. — Architecture

This is the code map for the Rust fork. If you're here to fix a bug or add a
feature, start by reading the relevant section, then jump to the file.

The Python fork at `C:\Jarvis\python` follows the same conceptual flow but is
a single-process Python daemon — much smaller surface area. This doc covers
the Rust side.

## Crate layout

```
crates/
  jarvis-core   library: STT, wake-word, TTS, intent, commands, IPC, LLM,
                Lua sandbox, scheduler, memory, profiles. ZERO runtime, only
                building blocks. Other crates wire it together.
  jarvis-app    daemon binary. Holds the microphone, runs the listening loop,
                dispatches commands, starts the scheduler thread. Tray icon.
  jarvis-gui    Tauri 2 + Svelte frontend. Shows command list, settings, mic
                state. Communicates with jarvis-app via IPC websocket.
  jarvis-cli    debug CLI (classify / execute / list). Currently has a known
                runtime panic on startup; use jarvis-app instead.
```

```
resources/commands/<pack_name>/
  command.toml   — id, phrases per language, script path, sandbox level, timeout
  *.lua          — script(s); one per command id when multiple ids share a pack
```

The `tools/` directory hosts optional native helpers (Piper TTS binary, Silero
helper script). Both are downloaded on demand via `tools/piper/install.ps1` etc.

## Data flow (mic → action)

```
[1] pv_recorder ──► raw 16 kHz mono PCM
[2] webrtc-vad ───► voice activity windows
[3] Vosk (wake-word + STT) ──► utf-8 transcript
[4] intent classifier (MiniLM) ──► (intent_id, confidence)?
[5] levenshtein fallback ──► (pack, command)?
[6] llm_router (IMBA-1) ──► canonical phrase substitution?
[7] llm_fallback chat ──► spoken reply
[8] command execution ──► Lua sandbox / Python subprocess / WinAPI call
[9] TTS (sapi | piper | silero) ──► speech out
[10] voices::play_*  ──► reaction sounds (ok, error, not_found)
```

Each step lives in a separate module — replace any one and the rest still
works.

| Step | Module                                            |
|------|---------------------------------------------------|
| 1    | `jarvis-core::recorder`                           |
| 2    | `jarvis-core::audio_processing::vad::webrtc`      |
| 3    | `jarvis-core::stt` + `jarvis-core::listener`      |
| 4    | `jarvis-core::intent`                             |
| 5    | `jarvis-core::commands::fetch_command`            |
| 6    | `jarvis-app::llm_router`                          |
| 7    | `jarvis-app::llm_fallback`                        |
| 8    | `jarvis-core::commands::execute_command`          |
| 9    | `jarvis-core::tts`                                |
| 10   | `jarvis-core::voices`                             |

The listening loop driving all of this is `jarvis-app/src/app.rs::start()`.

## Configuration

Two layers:

1. **`db::structs::Settings`** — persistent user preferences (voice, language,
   wake-word engine, noise-suppression backend). Lives in a sqlite file under
   `<APP_CONFIG_DIR>/`. Edited via the GUI Settings page.

2. **`runtime_config`** — environment-variable knobs read at startup. See
   `crates/jarvis-core/src/runtime_config.rs` for the full list with doc
   comments. Examples:
     - `JARVIS_TTS=sapi|piper|silero`
     - `JARVIS_LLM=groq|ollama`
     - `JARVIS_LLM_ROUTER=1` (default on), `JARVIS_LLM_ROUTER_THRESHOLD=0.55`
     - `GROQ_TOKEN`, `GROQ_MODEL`, `OLLAMA_BASE_URL`, `OLLAMA_MODEL`

   `runtime_config::log_effective_config()` is called once on `main()` so the
   active values appear in the startup log.

`dev.env` next to the exe is auto-loaded by `dotenvy`. Walks up 5 parents — so
running from `crates/jarvis-app/` during development still works.

## TTS pipeline

```
text ─► text_utils::sanitize_for_speech ─► TtsBackend::speak(text, opts)
                                                │
                              ┌─────────────────┼─────────────────┐
                              ▼                 ▼                 ▼
                         SapiBackend      PiperBackend      SileroBackend
                         (PowerShell)     (subprocess)      (python helper)
                              │                 │                 │
                              ▼                 ▼                 ▼
                          SAPI direct      WAV → play_wav    WAV → play_wav
                                              (shared)         (shared)
```

`tts::backend()` is initialised lazily on first call; subsequent calls reuse
the same `Arc<dyn TtsBackend>`. Backend is picked by `JARVIS_TTS` env var
with auto-detect fallback: if `tools/piper/piper.exe` exists, use Piper.

`play_wav` is the shared synchronous WAV player (PowerShell SoundPlayer on
Windows, no-op stub elsewhere). Used by both Piper and Silero — kept in
`tts/mod.rs` as `pub(crate)` to deduplicate.

## Lua sandbox

`mlua` 0.11 with three sandbox levels:

| Level     | Allowed                                    |
|-----------|--------------------------------------------|
| Minimal   | log, sleep, speak, audio, context, cmd     |
| Standard  | + http, llm, state, fs (within pack dir)   |
| Full      | + system.exec, clipboard.set, abs paths    |

Default is `standard`. Set per command in `command.toml` via `sandbox = "full"`.

Sandbox is enforced by:
- `LuaEngine::new(level)` controls which `StdLib` flags are loaded
- `lua/engine.rs::register_api` gates HTTP/state/fs registration on the level
- `globals.set("io", Nil)` removes the `io` stdlib unless `Full`
- `os.execute / os.exit / os.remove / os.rename / os.setlocale` removed
  even in `Full` mode (security defence)

### Available APIs for command scripts

See `crates/jarvis-core/src/lua/api/*.rs`. Quick reference:

```lua
jarvis.speak(text, opts?)        -- TTS; opts.lang/.async/.raw
jarvis.cmd.ok(msg?)              -- boilerplate-killer: play_ok + speak + {chain=false}
jarvis.cmd.error(msg?)           -- play_error + speak + {chain=false}
jarvis.cmd.not_found(msg?)       -- play_not_found + speak + {chain=false}
jarvis.cmd.chain_ok(msg?)        -- like ok() but chain=true

jarvis.audio.play_ok / play_error / play_not_found / play_reply / play_goodbye / play_thanks
jarvis.context.{phrase, command_id, command_path, language, time, slots}

jarvis.text.strip_trigger(phrase, {triggers...})
jarvis.text.contains_any(phrase, {needles...})

jarvis.llm({messages}, {opts})   -- Groq or Ollama, selected by JARVIS_LLM
jarvis.http.{get, post, post_json, json}

jarvis.memory.{remember, recall, search, forget, all}
jarvis.profile.{active, active_name, set, list, allows}
jarvis.scheduler.{add, list, count, remove, clear}
jarvis.vision.{screenshot, describe}   -- HTTP sandbox required

jarvis.fs.{read, write, append, exists, is_file, is_dir, list, mkdir, remove}
jarvis.state.{get, set}                                -- pack-scoped k/v
jarvis.system.{open, exec, notify, env, platform, clipboard.{get, set}}
jarvis.settings.{get, set}                             -- db settings table
```

### Recommended pack structure

```lua
-- Single-id pack:
local phrase = (jarvis.context.phrase or ""):lower()
local body = jarvis.text.strip_trigger(phrase, { "trigger1", "trigger2" })
body = body:gsub("^[%s,:%.]+", ""):gsub("%s+$", "")

if body == "" then
    return jarvis.cmd.error("Что именно?")
end

-- ... do work ...

return jarvis.cmd.ok("Готово.")
```

For multi-id packs (one Lua script shared between several `[[commands]]`),
dispatch on `jarvis.context.command_id`. The `profile_switch/switch.lua` is a
good example — maps `profile.work` → `work`, `profile.game` → `game`, etc.

## Background services

Two long-running threads beyond the listening loop:

1. **`scheduler` tick thread** (`crates/jarvis-core/src/scheduler.rs`).
   Wakes every 30 seconds, fires due tasks (Speak via TTS or Lua scripts).
   Persists changes to `<APP_CONFIG_DIR>/schedule.json`.

2. **`ipc::start_server`** (`crates/jarvis-core/src/ipc/`). Tokio
   websocket server on a configurable port. The GUI connects to this to
   send commands and receive events (`SpeechRecognized`, `LlmReply`,
   `CommandExecuted`, etc.).

## How to add a new command pack

1. Create `resources/commands/<your_pack>/`.
2. Add `command.toml`:
   ```toml
   [[commands]]
   id = "your_pack.thing"
   type = "lua"
   script = "thing.lua"
   sandbox = "standard"
   timeout = 5000

   [commands.phrases]
   ru = ["сделай штуку", ...]
   en = ["do the thing", ...]
   ```
3. Write `thing.lua`. Start from the snippet under "Recommended pack structure".
4. Add a `commands::tests` assertion in `crates/jarvis-core/src/commands/tests.rs`
   so the parser regression test covers your TOML (the existing
   `every_command_toml_parses` already loads any new dir automatically).
5. Run `cargo test -p jarvis-core --lib commands::tests`.
6. Rebuild jarvis-app: it'll pick up the new pack on next start.

For Python parity, mirror the YAML entry + handler in `C:\Jarvis\python\`.

## How to add a new Rust core feature

1. Add module under `crates/jarvis-core/src/<feature>.rs`. Pure data layer +
   thin global wrapper if needed (see `long_term_memory.rs` as template).
2. Register in `crates/jarvis-core/src/lib.rs` (`pub mod <feature>;`).
3. Expose Lua API at `crates/jarvis-core/src/lua/api/<feature>.rs` if scripts
   need it. Register in `lua/api.rs` and call from `lua/engine.rs`.
4. Wire init from `crates/jarvis-app/src/main.rs`.
5. Add a `runtime_config::ENV_*` constant if there's an env-driven knob.
6. Add unit tests in a `#[cfg(test)] mod tests {}` block in the new file.
7. Document the public surface in this file (data flow / API section).

## How to add a new TTS backend

Implement `TtsBackend` (see `crates/jarvis-core/src/tts/mod.rs`). Use the
`super::play_wav` helper if your engine emits WAV files. Wire selection in
`init_backend()` near the top of the same file. Document the env var in
`runtime_config.rs`.

## Testing

```
cargo test -p jarvis-core --lib          # all jarvis-core unit tests
cargo test -p jarvis-core --lib scheduler:: # filter by module
```

Test coverage map:

| Module               | Tests                                        |
|----------------------|----------------------------------------------|
| `text_utils`         | sanitize_for_speech (5 cases)                |
| `long_term_memory`   | normalize_key, search_in, build_context, serde (8) |
| `profiles`           | allows_command logic, serde (6)              |
| `scheduler`          | parse_*, next_fire (7)                       |
| `runtime_config`     | get_bool / get_parse fallbacks (2)           |
| `llm::client`        | response parsing, env handling, helpers (4)  |
| `llm::history`       | conversation history truncation (5)          |
| `commands::tests`    | every command.toml parses + has phrases (3)  |
| `lua::tests`         | sandbox levels, fs escape, timeout (4)       |
| `audio_processing::vad::listen_window` | listening window logic (4)        |

Run-time integration tests are manual:
- mic available? → `jarvis-app` logs `recorder::init`
- wake word fires? → log shows `WakeWordDetected`
- TTS speaks? → check Piper/SAPI logs
- Scheduler fires? → wait 30s after `scheduler::add` with `in 30 seconds`

## Build

```powershell
# One-time: set up MSVC env (PowerShell 7 / pwsh):
$vc = "C:\Program Files\Microsoft Visual Studio\18\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
cmd /c "`"$vc`" >NUL && set" | % { if ($_ -match '^([^=]+)=(.*)$') { [Environment]::SetEnvironmentVariable($matches[1], $matches[2], 'Process') } }

# Then:
cargo build --release -p jarvis-app -p jarvis-gui
```

The `jarvis-gui` `build.rs` auto-invokes `npm run build` and emits
`cargo:rerun-if-changed=frontend/src` so changes to the Svelte frontend are
picked up by a plain `cargo build`. No need to run `cargo tauri build`
manually unless you want a packaged installer.

## Git workflow

- `master` is protected.
- `feature/pc-tools` is the integration branch where everything is currently
  shipping.
- Self-hosted git on Forgejo at `http://192.168.0.10:3000/bossiara13/J.A.R.V.I.S-rust`.
  Push command: `NO_PROXY=192.168.0.10,localhost,127.0.0.1` + `git -c http.proxy= push forgejo <branch>`
  (the local V2Ray proxy at 10809 will otherwise block LAN traffic).
- GitHub origin at `https://github.com/DmitryBykov-ISPO/J.A.R.V.I.S-rust` —
  manually pushed for public visibility.

Commit messages: descriptive, no `Co-Authored-By: Claude` tag (project rule).