Mirrors the new rust packs from the same iteration. Total commands in
commands.yaml is now 49 (was 33). yaml.safe_load + ast.parse both pass.
Window management (8 commands) — uses the existing `keys` action with
pyautogui hotkey strings: winleft+d (show_desktop), winleft+up/down
(maximize/minimize), winleft+left/right (snap left/right), alt+f4
(close_window), winleft+shift+m (restore_all), winleft+tab (task_view).
No new action type needed.
Clipboard read aloud — new clipboard_read action. Shells to
PowerShell Get-Clipboard, caps preview at 400 chars, speaks via the
existing tts.va_speak (which already handles recorder.stop/start).
Notes — new notes_add + notes_open actions. notes_add strips the
trigger phrase, appends "[YYYY-MM-DD HH:MM] body\n" to
%USERPROFILE%\Documents\jarvis-notes.txt, creates the dir if needed.
notes_open shells "cmd /c start" so .txt opens in the user-default
editor.
Process killer — new process_kill action. Strips the trigger ("убей
процесс"/"закрой программу"/etc.), looks up the remainder in the same
alias map as the rust version (хром→chrome, телега→telegram, спотифай
→spotify, едж/эдж/edge→msedge, дискорд→discord, стим→steam, обс→obs64,
проводник/блокнот/калькулятор/диспетчер задач, vs code→code, etc.),
appends .exe if needed, runs taskkill /F /IM, speaks success or
"не нашёл процесс N".
Web search — new websearch action with a `service` param. Bases:
google → https://www.google.com/search?q=
youtube → https://www.youtube.com/results?search_query=
wiki → https://ru.wikipedia.org/wiki/Special:Search?search=
yandex → https://yandex.ru/search/?text=
Strips trigger, urllib.parse.quote-s the rest, opens via webbrowser.open
(respects the user'\''s default browser; config.BROWSER_PATHS only kicks in
for the older url-action commands).
Portability: USERPROFILE used for notes path, no other hardcoded paths.
Brings the python fork in line with the new feature/pc-tools work that
landed in rust. Same UX, same trigger phrases, same env vars.
config.py:
+ LLM_AUTO_FALLBACK = True
+ LLM_AUTO_FALLBACK_MIN_CHARS = 4
Mirrors the rust config knobs. The old "скажи X" trigger still works
(VA_TBR strips it before intent matching, so what remains hits this
fallback anyway) — but now ANY unrecognized query goes to Groq, not
just "скажи" ones.
main.py:
va_respond: replaced the fuzz-on-"скажи" gate with a unified auto-
fallback that fires whenever no command matches, GROQ_TOKEN is set,
and the utterance is at least LLM_AUTO_FALLBACK_MIN_CHARS long.
+ _speak(text): recorder.stop() → tts.va_speak → start, used by the
new action handlers so the mic doesn't pick up the assistant.
+ _set_clipboard(text): subprocess to PowerShell Set-Clipboard, no
new python deps.
+ _strip_trigger(voice, triggers): shared helper for codegen/file_search
where we need the text AFTER the trigger phrase.
New action handlers (registered in run_action dispatch):
+ groq_codegen — extracts query, calls Groq with strict "code only"
system prompt at temp 0.2, strips ```lang ... ``` fences, puts the
result in the clipboard, speaks "Код в буфере, сэр. Строк: N".
+ ocr_screen — PowerShell System.Drawing screenshot to a temp PNG,
shells to tesseract.exe (-l rus+eng), speaks a 120-char preview,
clipboard. Looks for tesseract on PATH first then standard install
dirs; gives a clear "Tesseract не установлен" if missing.
+ sysinfo — CIM-based battery/cpu/ram/disk via PS one-liners, time
via python datetime (gets us localized weekday for free). Topics:
time / battery / cpu / ram / disk / all.
+ file_search — strips trigger, recursive Get-ChildItem in Desktop/
Documents/Downloads (depth 3, top 5), opens explorer /select on
the first hit, speaks count and basename.
+ audio_devices_panel — opens mmsys.cpl. Proper COM-based device
switching (like the rust IPolicyConfig version) is deferred — TODO
in backlog.
commands.yaml:
Total 33 commands (was 19). New:
+ open_notepad / open_calculator / open_explorer / open_terminal
/ open_settings / open_task_manager / lock_screen / screenshot
— all via shell action (no AHK exes needed).
+ volume_up / volume_down — five chained keys=volumeup/volumedown
presses each, like rust's _volume_helper.ps1 with Times=5.
+ media_stop — keys=stop.
+ sysinfo_time / sysinfo_battery / sysinfo_cpu / sysinfo_ram /
sysinfo_disk / sysinfo_all — sysinfo action.
+ file_search, codegen, read_screen, switch_audio_device.
Replaced (were AHK exes that Priler ships and we don't have):
~ music → keys playpause
~ music_off → keys playpause (toggle)
~ music_next → keys nexttrack
~ music_prev → keys prevtrack
Removed (Priler hardware/setup specific, can't sensibly port):
- gaming_mode_on / gaming_mode_off (display config switch)
- music_save (depends on the specific player AHK was driving)
- switch_to_headphones / switch_to_dynamics (replaced with the
generic switch_audio_device that opens mmsys.cpl)
Kept as-is (worked already, no AHK dependency):
open_browser / open_youtube / open_google (url action)
sound_off / sound_on (pycaw via system.volume_mute/unmute)
thanks / stupid / off
Smoke-tested: yaml.safe_load → 33 commands, ast.parse main.py → OK.
Portability: USERPROFILE for screenshot path, $env:SystemDrive for disk,
PATH lookup for tesseract before falling back to standard install dirs.
No hardcoded user-specific paths.
Adds optional 'browser' field to url-action; runtime resolves it via
config.BROWSER_PATHS (subprocess.Popen with the chosen exe). When
omitted, falls back to webbrowser.open (system default). Constructor
GUI gains a browser dropdown on URL action and on multi-step URL.
Existing open_browser/youtube/google updated to use yandex by default
since user prefers Yandex Browser.
Replaces fuzzywuzzy intent matching with ONNX MiniLM-L6-v2 sentence
embeddings (cosine similarity, threshold 0.45). Now triggers on
semantically equivalent phrases not literally in commands.yaml.
Adds optional noisereduce preprocessing before Vosk (off by default,
DENOISE_ENABLED config flag). Inference latency: 2.5ms median CPU.
Single window (900x720, dark background) loading web/index.html with a
Python bridge exposing list_existing_commands, list_exes,
list_sound_names, preview_sound, slugify, validate_draft, save_command,
plus llm_suggest hook for M3. Sound preview spawned on a daemon thread
so the GUI does not block.
ruamel.yaml is needed by tools/command_builder/yaml_writer to round-trip
commands.yaml without losing comments. pyautogui drives the new keys
action. pywebview is added now too so M2's GUI bootstrap can land
without another deps bump.
shell uses subprocess (with shell=True so user can write a normal
command line); url uses webbrowser.open; keys uses pyautogui.hotkey
for combinations or pyautogui.typewrite for raw text. pyautogui is
imported lazily so the bot still starts on machines without a GUI.
Uses ruamel.yaml so existing comments / formatting in commands.yaml
survive a save. Each write creates commands.yaml.bak first, then
atomically renames a .tmp file into place. Tests cover round-trip for
all action types plus comment-preservation on append.
Adds webrtcvad-based smart command-end detection (fixed 10s window
replaced by VAD with configurable silence/min/max thresholds) and
migrates execute_cmd to a YAML-driven plugin system with typed
actions (exe, play_sound, system, multi, sleep). 17 commands
migrated; all original behavior preserved.
All 17 voice commands migrated from a flat phrase list to the
phrases/action shape. Behaviour preserved 1:1 with the previous
elif chain — including the 200ms post-Popen pause for music_*
controls and the 500ms pause before the 'ready' chime on the
audio-device switch commands.
Replaces the 100-line elif chain with a small dispatcher. Each entry in
commands.yaml now carries both 'phrases' (existing trigger list) and
'action' describing how to run it. Action types: exe, play_sound,
system (volume_mute/unmute/exit), sleep, multi.
The data migration lives in the next commit; the dispatcher in this
commit assumes the new schema.
Replace the fixed 10s post-wake-word listen window with a VAD-driven
loop. Once at least 500ms of speech have been seen, 1.2s of silence
closes the window. Hard cap at 15s as a safety net.
Tunable in config.py via VAD_AGGRESSIVENESS, COMMAND_END_SILENCE_MS,
COMMAND_MIN_SPEECH_MS, COMMAND_MIN_LISTEN_MS, COMMAND_MAX_LISTEN_MS.
- Names the assistant as J.A.R.V.I.S. (acronym + MCU pre-Age-of-Ultron)
- British butler tone, addresses user as «сэр»
- Brief replies (1-3 sentences)
- No prudish censorship
wake-word is now handled by vosk, so the picovoice access key is no longer
needed. config.py already had PICOVOICE_TOKEN removed in the previous commit.
uses a second KaldiRecognizer with a JSON grammar of WAKE_WORDS + [unk] for
the wake phase, then hands off to the full-vocab recognizer for the 10s
command window. drops the pvporcupine dependency entirely.
Real secrets now live in untracked dev.env (gitignored). dev.env.example
holds placeholders and is the canonical template — copy it to dev.env on
fresh checkout and fill in real keys.
