Compare commits

..

32 commits
v0.2.0 ... main

Author SHA1 Message Date
Bossiara13
53678fbdfa Release v0.4.1 — browser choice for url-action 2026-04-27 20:06:58 +03:00
Bossiara13
b8fc7b4eb7 chore: bump VA_VER to 0.4.1 2026-04-27 20:06:52 +03:00
Bossiara13
d0eac79275 Merge feature/browser-choice-in-url into dev 2026-04-27 20:06:51 +03:00
Bossiara13
a4171ec6b1 feat(url): optional browser field (yandex/chrome/firefox/edge)
Adds optional 'browser' field to url-action; runtime resolves it via
config.BROWSER_PATHS (subprocess.Popen with the chosen exe). When
omitted, falls back to webbrowser.open (system default). Constructor
GUI gains a browser dropdown on URL action and on multi-step URL.
Existing open_browser/youtube/google updated to use yandex by default
since user prefers Yandex Browser.
2026-04-27 20:06:26 +03:00
Bossiara13
47927fcf41 Release v0.4.0 — semantic intent (ONNX MiniLM) + optional denoise 2026-04-23 11:09:54 +03:00
Bossiara13
d22a35c9c7 chore: bump VA_VER to 0.4.0 2026-04-23 11:09:48 +03:00
Bossiara13
59aece53da Merge feature/onnx-intent-and-denoise into dev
Replaces fuzzywuzzy intent matching with ONNX MiniLM-L6-v2 sentence
embeddings (cosine similarity, threshold 0.45). Now triggers on
semantically equivalent phrases not literally in commands.yaml.
Adds optional noisereduce preprocessing before Vosk (off by default,
DENOISE_ENABLED config flag). Inference latency: 2.5ms median CPU.
2026-04-23 11:09:46 +03:00
Bossiara13
8655b4fc69 docs: README — semantic intent + denoise 2026-04-23 10:55:07 +03:00
Bossiara13
cc3fa60c28 chore: pin onnxruntime / tokenizers / noisereduce in requirements 2026-04-23 10:54:38 +03:00
Bossiara13
13370da5b7 feat: optional noisereduce preprocessing before vosk 2026-04-23 10:54:20 +03:00
Bossiara13
6a710c55b8 feat: onnx intent classifier replacing fuzzy match 2026-04-23 10:53:18 +03:00
Bossiara13
9cb55f4730 chore: vendor MiniLM-L6-v2 onnx model from Qdrant 2026-04-23 10:51:53 +03:00
Bossiara13
31a2dd1a6c chore: enable git lfs for .onnx files 2026-04-23 10:51:29 +03:00
Bossiara13
e7d4304a62 Release v0.3.0 — TTS effects + GUI command constructor 2026-04-23 02:17:09 +03:00
Bossiara13
1eaaf9eddc Merge feature/command-builder into dev
GUI command constructor (pywebview, Russian, dark theme matching
Rust GUI palette). 3-step stepper + LLM-suggest tab. Writes to
commands.yaml with ruamel-preserved comments and .bak backup.
Three new action types: shell / url / keys (pyautogui-based).
2026-04-23 02:17:02 +03:00
Bossiara13
e86906801a Merge feature/ai-style-postproc into dev
Optional bandpass + short reverb on Silero TTS output. Off by default
(TTS_EFFECTS_ENABLED in config.py). Preview script in utils/.
2026-04-23 02:17:02 +03:00
Bossiara13
fbfe3a7ff6 chore: bump VA_VER to 0.3.0 2026-04-23 02:13:49 +03:00
Bossiara13
b535d025f3 docs: command builder README + user-facing README section + run_builder.bat 2026-04-23 02:13:47 +03:00
Bossiara13
521b124f05 feat(ui): keyboard nav (Enter advances step) + yaml-health warning on launch 2026-04-23 02:13:43 +03:00
Bossiara13
0b1604e018 feat(tools): llm-suggest via groq with JSON response format 2026-04-23 02:11:45 +03:00
Bossiara13
e4d4bf7a08 feat(ui): steps 1-3 (trigger / action / params) with live YAML preview 2026-04-23 02:11:01 +03:00
Bossiara13
232490d9fa chore: vendor js-yaml.min.js 2026-04-23 02:10:57 +03:00
Bossiara13
9814181d1a feat(tools): pywebview bootstrap + python bridge API
Single window (900x720, dark background) loading web/index.html with a
Python bridge exposing list_existing_commands, list_exes,
list_sound_names, preview_sound, slugify, validate_draft, save_command,
plus llm_suggest hook for M3. Sound preview spawned on a daemon thread
so the GUI does not block.
2026-04-22 23:21:20 +03:00
Bossiara13
80d9eedead chore: add ruamel.yaml + pyautogui deps
ruamel.yaml is needed by tools/command_builder/yaml_writer to round-trip
commands.yaml without losing comments. pyautogui drives the new keys
action. pywebview is added now too so M2's GUI bootstrap can land
without another deps bump.
2026-04-22 23:17:44 +03:00
Bossiara13
7b8cd31cf3 feat: add shell / url / keys action types to runtime dispatcher
shell uses subprocess (with shell=True so user can write a normal
command line); url uses webbrowser.open; keys uses pyautogui.hotkey
for combinations or pyautogui.typewrite for raw text. pyautogui is
imported lazily so the bot still starts on machines without a GUI.
2026-04-22 23:17:40 +03:00
Bossiara13
0f6b794db9 feat(tools): yaml writer with ruamel (preserves comments, atomic + backup)
Uses ruamel.yaml so existing comments / formatting in commands.yaml
survive a save. Each write creates commands.yaml.bak first, then
atomically renames a .tmp file into place. Tests cover round-trip for
all action types plus comment-preservation on append.
2026-04-22 23:17:24 +03:00
Bossiara13
68e73b3c53 feat(tools): scaffold command builder schema
Dataclass-based schema mirroring commands.yaml plus new action types
(shell, url, keys). Round-trip dict<->dataclass with validation.
Russian-aware slugify for auto-generating command IDs.
2026-04-22 23:17:17 +03:00
Bossiara13
f9e1e2468f chore: gitignore voice_ref/ and sound/_*.wav experiments 2026-04-22 23:10:54 +03:00
Bossiara13
e5903d8076 docs: README — TTS effects section 2026-04-22 20:00:04 +03:00
Bossiara13
34f5cd4f5a chore: add utils/preview_effects.py for A/B comparison 2026-04-22 19:59:45 +03:00
Bossiara13
64ad7b963e feat: wire effects into va_speak behind config.TTS_EFFECTS_ENABLED 2026-04-22 19:58:42 +03:00
Bossiara13
bd6bc294ad feat: add tts effects module (bandpass + reverb + optional pitch) 2026-04-22 19:58:20 +03:00
32 changed files with 64134 additions and 26 deletions

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
*.onnx filter=lfs diff=lfs merge=lfs -text

3
.gitignore vendored
View file

@ -191,3 +191,6 @@ dev.env
# Misc artifacts
.git-filter-redactions.txt
# Experiments / voice cloning artifacts (not part of repo)
voice_ref/
sound/_*.wav

View file

@ -43,7 +43,9 @@ python main.py
- `MICROPHONE_INDEX = -1` — индекс микрофона (`-1` означает устройство по умолчанию). Если микрофонов несколько и берётся не тот, поменяйте число.
- `WAKE_WORDS = ('jarvis', 'джарвис')` — варианты транскрипции wake-word, которые принимает Vosk. Если ваш голос/акцент стабильно расшифровывается как, например, `жарвис` — добавьте этот вариант сюда (всё в нижнем регистре).
- `GROQ_MODEL` — имя модели Groq (по умолчанию `llama-3.3-70b-versatile`).
- `commands.yaml` — набор голосовых команд и их вариантов для fuzzy-матчинга.
- `INTENT_SIMILARITY_THRESHOLD = 0.45` — минимальная косинусная близость для срабатывания команды (см. ниже).
- `DENOISE_ENABLED = False` — шумоподавление перед Vosk (см. ниже).
- `commands.yaml` — набор голосовых команд и их вариантов; матчится семантически по эмбеддингам.
- `custom-commands/` — скомпилированные AHK-скрипты под конкретный сетап автора оригинала (переключение мониторов, управление Яндекс.Музыкой и т.п.). На вашей машине большая часть из них работать не будет, но ассистент запустится в любом случае.
## Использование
@ -63,6 +65,47 @@ python main.py
- `COMMAND_MIN_LISTEN_MS` — минимальное время записи, чтобы не закрыться мгновенно (1000 мс).
- `COMMAND_MAX_LISTEN_MS` — жёсткий потолок (15000 мс).
## Семантический матчинг команд
Раньше распознанная фраза сравнивалась с фразами из `commands.yaml` через `fuzzywuzzy.fuzz.ratio` — это срабатывало только на формальное совпадение букв. Теперь используется sentence-transformer `all-MiniLM-L6-v2` (ONNX, CPU) — фраза кодируется в 384-мерный вектор, считается косинусная близость с фразами всех команд, побеждает самая близкая.
Это означает, что фраза `«запусти браузер»` сматчится с командой `open_browser`, даже если в YAML записано только `«открой браузер»`у них близкий смысл.
- Модель лежит в `models/all-MiniLM-L6-v2/` (87 МБ, хранится через Git LFS — `git lfs pull` после клонирования).
- Порог `INTENT_SIMILARITY_THRESHOLD = 0.45` (косинус, диапазон [-1; 1], для близких смыслов обычно 0.7+). Ниже порога — «команда не распознана», сработает `not_found.wav` (или ветка `«скажи …»` для LLM).
- Все фразы из `commands.yaml` эмбеддятся один раз при старте; на каждой команде эмбеддится только распознанная пользователем фраза (~1-5 мс на CPU).
## Шумоподавление перед STT
Опциональный пре-процессинг аудио командного буфера через `noisereduce` (спектральное гейтирование). Помогает в шумных помещениях (кулер, кондиционер, фоновая музыка). К wake-word фазе не применяется — там важна отзывчивость.
В `config.py`:
- `DENOISE_ENABLED = False` — главный тумблер.
- `DENOISE_PROP = 0.85` — доля подавления (0..1).
- `DENOISE_STATIONARY = False``False` для меняющегося шума (рекомендуется), `True` для стабильного фона.
## Эффекты под AI-ассистента
Silero звучит естественно, но иногда хочется лёгкой «AI-окраски» в духе J.A.R.V.I.S. — узкую полосу, еле заметную реверберацию, при желании небольшой сдвиг высоты. Это тонкий постпроцессинг, голос и sample rate не меняются.
По умолчанию эффекты **выключены**. Включаются в `config.py`:
- `TTS_EFFECTS_ENABLED` — главный тумблер (`False` по умолчанию).
- `TTS_BANDPASS_LOW_HZ` / `TTS_BANDPASS_HIGH_HZ` — полоса пропускания (по умолчанию 2007000 Гц). Сужение полосы даёт ощущение «голоса по радио».
- `TTS_REVERB_WET` (0..1) и `TTS_REVERB_DECAY_MS` — доля мокрого сигнала и длительность хвоста короткой реверберации (по умолчанию 0.20 и 100 мс).
- `TTS_PITCH_SEMITONES` — сдвиг высоты в полутонах (по умолчанию 0, разумный диапазон ±2).
Фильтры реализованы на `scipy.signal` (IIR, sosfiltfilt — нулевая фазовая задержка), реверберация — несколько затухающих задержанных копий, pitch — ресэмплинг по линейной интерполяции. Всё in-process, без временных файлов и лишних зависимостей.
Быстрое A/B сравнение перед тем как переключать тумблер:
```bash
.venv\Scripts\python.exe utils\preview_effects.py
```
Скрипт сгенерирует `sound/_preview_silero_raw.wav` и `sound/_preview_silero_fx.wav` — откройте оба в плеере и решите, нравится ли эффект. Параметры fx-версии берутся из текущего `config.py` (независимо от `TTS_EFFECTS_ENABLED`).
## Схема commands.yaml
Каждая команда — это пара `phrases` + `action`:
@ -94,6 +137,27 @@ sound_off:
- {type: system, op: volume_mute}
```
## Конструктор команд
GUI для добавления команд в `commands.yaml` без ручной правки YAML.
Запуск:
```
python -m tools.command_builder
```
Или из корня репо `run_builder.bat` (под Windows). Открывается окно pywebview с пошаговым мастером: фразы → тип действия → параметры → предпросмотр YAML и сохранение. На шаге «Действие» доступна вкладка «Подсказать (LLM)» — модель Groq предложит готовый блок по описанию на русском.
После сохранения команды Jarvis нужно перезапустить, чтобы она подхватилась.
Требования:
- Windows 10/11 с установленным WebView2 Runtime (обычно уже стоит вместе с Edge — иначе [скачать у Microsoft](https://developer.microsoft.com/microsoft-edge/webview2/)).
- Зависимости из `requirements.txt` (включая `pywebview`, `ruamel.yaml`, `pyautogui`).
Подробности для разработчиков — [tools/command_builder/README.md](tools/command_builder/README.md). TODO: добавить скриншот окна.
## Лицензия
CC BY-NC-SA 4.0 — см. [LICENSE.txt](LICENSE.txt). Авторство оригинала — Abraham Tugalov / Priler. Изменения в этом форке — Bossiara13 (Dmitry Bykov), 2026.

View file

@ -2,23 +2,23 @@ open_browser:
phrases:
- открой браузер
- запусти браузер
- открой гугл хром
- гугл хром
action: {type: exe, file: "Run browser.exe"}
- открой яндекс
- яндекс браузер
action: {type: url, href: "about:blank", browser: "yandex"}
open_youtube:
phrases:
- открой ютуб
- ютуб
- запусти ютуб
action: {type: exe, file: "Run youtube.exe"}
action: {type: url, href: "https://www.youtube.com", browser: "yandex"}
open_google:
phrases:
- открой гугл
- гугл
- запусти гугл
action: {type: exe, file: "Run google.exe"}
action: {type: url, href: "https://www.google.com", browser: "yandex"}
music:
phrases:

View file

@ -6,7 +6,7 @@ load_dotenv("dev.env")
# Конфигурация
VA_NAME = 'Jarvis'
VA_VER = "0.1.0"
VA_VER = "0.4.1"
VA_ALIAS = ('джарвис',)
VA_TBR = ('скажи', 'покажи', 'ответь', 'произнеси', 'расскажи', 'сколько', 'слушай')
@ -19,8 +19,12 @@ WAKE_WORDS = ('jarvis', 'джарвис')
# -1 это стандартное записывающее устройство
MICROPHONE_INDEX = -1
# Путь к браузеру Google Chrome
CHROME_PATH = 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe %s'
BROWSER_PATHS = {
'yandex': 'C:/Program Files/Yandex/YandexBrowser/Application/browser.exe',
'chrome': 'C:/Program Files/Google/Chrome/Application/chrome.exe',
'firefox': 'C:/Program Files/Mozilla Firefox/firefox.exe',
'edge': 'C:/Program Files (x86)/Microsoft/Edge/Application/msedge.exe',
}
# Токен Groq
GROQ_TOKEN = os.getenv('GROQ_TOKEN')
@ -34,3 +38,23 @@ COMMAND_END_SILENCE_MS = 1200
COMMAND_MIN_SPEECH_MS = 500
COMMAND_MIN_LISTEN_MS = 1000
COMMAND_MAX_LISTEN_MS = 15000
TTS_EFFECTS_ENABLED = False
TTS_BANDPASS_LOW_HZ = 200
TTS_BANDPASS_HIGH_HZ = 7000
TTS_REVERB_WET = 0.20
TTS_REVERB_DECAY_MS = 100
TTS_PITCH_SEMITONES = 0
# Семантический матчинг команд через эмбеддинги MiniLM-L6-v2 (ONNX, CPU).
# Порог — косинусная близость [0..1]. На вход поступает уже отфильтрованная
# фраза (без алиасов и vа_tbr); 0.45 эмпирически отделяет валидные команды
# от шума, оставляя запас на разговорные перефразировки.
INTENT_SIMILARITY_THRESHOLD = 0.45
# Шумоподавление аудио перед Vosk на этапе захвата команды (после wake-word).
# Спектральное гейтирование через noisereduce; даёт небольшую латентность,
# поэтому выключено по умолчанию и применяется только к буферу команды.
DENOISE_ENABLED = False
DENOISE_PROP = 0.85
DENOISE_STATIONARY = False

71
effects.py Normal file
View file

@ -0,0 +1,71 @@
import numpy as np
from scipy.signal import butter, sosfiltfilt
def _ensure_float32_mono(audio):
arr = np.asarray(audio, dtype=np.float32)
if arr.ndim > 1:
arr = arr.mean(axis=1).astype(np.float32)
return arr
def apply_bandpass(audio, sr, low_hz=200, high_hz=7000, order=4):
arr = _ensure_float32_mono(audio)
nyq = 0.5 * sr
low = max(low_hz, 1) / nyq
high = min(high_hz, int(nyq) - 1) / nyq
if not (0 < low < high < 1):
return arr
sos = butter(order, [low, high], btype='band', output='sos')
return sosfiltfilt(sos, arr).astype(np.float32)
def apply_reverb(audio, sr, wet=0.20, decay_ms=100, taps=4):
arr = _ensure_float32_mono(audio)
if wet <= 0 or decay_ms <= 0:
return arr
dry = max(0.0, 1.0 - wet)
out = dry * arr
gap = max(1, int(sr * (decay_ms / 1000.0) / taps))
gain = wet
for i in range(1, taps + 1):
delay = gap * i
gain *= 0.6
if delay >= len(arr):
break
padded = np.zeros_like(arr)
padded[delay:] = arr[:len(arr) - delay]
out += gain * padded
peak = float(np.max(np.abs(out))) if out.size else 0.0
if peak > 0.99:
out *= 0.99 / peak
return out.astype(np.float32)
def apply_pitch(audio, sr, semitones=0):
arr = _ensure_float32_mono(audio)
if semitones == 0:
return arr
factor = 2.0 ** (semitones / 12.0)
n_out = int(round(len(arr) / factor))
if n_out <= 1:
return arr
src_idx = np.linspace(0, len(arr) - 1, num=n_out).astype(np.float32)
lo = np.floor(src_idx).astype(np.int64)
hi = np.clip(lo + 1, 0, len(arr) - 1)
frac = (src_idx - lo).astype(np.float32)
resampled = (1.0 - frac) * arr[lo] + frac * arr[hi]
return resampled.astype(np.float32)
def process(audio, sr, bandpass=None, reverb=None, pitch_semitones=0):
out = _ensure_float32_mono(audio)
if bandpass is not None:
low, high = bandpass
out = apply_bandpass(out, sr, low, high)
if reverb is not None:
wet, decay_ms = reverb
out = apply_reverb(out, sr, wet=wet, decay_ms=decay_ms)
if pitch_semitones:
out = apply_pitch(out, sr, semitones=pitch_semitones)
return out

79
intent.py Normal file
View file

@ -0,0 +1,79 @@
import os
import numpy as np
import onnxruntime as ort
from tokenizers import Tokenizer
import config
MODEL_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "models", "all-MiniLM-L6-v2")
MAX_TOKENS = 128
def _mean_pool(last_hidden: np.ndarray, attention_mask: np.ndarray) -> np.ndarray:
mask = attention_mask.astype(np.float32)[..., None]
summed = (last_hidden * mask).sum(axis=1)
counts = np.clip(mask.sum(axis=1), a_min=1e-9, a_max=None)
return summed / counts
def _l2_normalize(x: np.ndarray) -> np.ndarray:
norm = np.linalg.norm(x, axis=-1, keepdims=True)
return x / np.clip(norm, a_min=1e-12, a_max=None)
class IntentClassifier:
def __init__(self, model_dir: str = MODEL_DIR):
tok_path = os.path.join(model_dir, "tokenizer.json")
onnx_path = os.path.join(model_dir, "model.onnx")
self._tokenizer = Tokenizer.from_file(tok_path)
self._tokenizer.enable_truncation(max_length=MAX_TOKENS)
self._tokenizer.enable_padding(pad_id=0, pad_token="[PAD]")
so = ort.SessionOptions()
so.intra_op_num_threads = max(1, (os.cpu_count() or 2) // 2)
so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
self._session = ort.InferenceSession(onnx_path, sess_options=so, providers=["CPUExecutionProvider"])
self._input_names = {inp.name for inp in self._session.get_inputs()}
self._cache: dict[str, np.ndarray] = {}
def embed(self, texts: list[str]) -> np.ndarray:
if not texts:
return np.zeros((0, 384), dtype=np.float32)
encs = self._tokenizer.encode_batch(texts)
ids = np.array([e.ids for e in encs], dtype=np.int64)
mask = np.array([e.attention_mask for e in encs], dtype=np.int64)
feeds = {"input_ids": ids, "attention_mask": mask}
if "token_type_ids" in self._input_names:
feeds["token_type_ids"] = np.zeros_like(ids)
outputs = self._session.run(None, feeds)
last_hidden = outputs[0]
pooled = _mean_pool(last_hidden, mask)
return _l2_normalize(pooled).astype(np.float32)
def prime(self, candidates: dict[str, list[str]]) -> None:
self._cache.clear()
for cmd_id, phrases in candidates.items():
if not phrases:
continue
self._cache[cmd_id] = self.embed(list(phrases))
def match(self, utterance: str, candidates: dict[str, list[str]]) -> dict:
utterance = (utterance or "").strip()
if not utterance:
return {"cmd": "", "score": 0.0}
for cmd_id, phrases in candidates.items():
if cmd_id not in self._cache and phrases:
self._cache[cmd_id] = self.embed(list(phrases))
query = self.embed([utterance])[0]
best_cmd = ""
best_score = -1.0
for cmd_id, mat in self._cache.items():
sims = mat @ query
top = float(sims.max()) if sims.size else -1.0
if top > best_score:
best_score = top
best_cmd = cmd_id
threshold = getattr(config, "INTENT_SIMILARITY_THRESHOLD", 0.45)
if best_score < threshold:
return {"cmd": "", "score": max(0.0, best_score)}
return {"cmd": best_cmd, "score": max(0.0, best_score)}

68
main.py
View file

@ -7,8 +7,10 @@ import struct
import subprocess
import sys
import time
import webbrowser
from ctypes import POINTER, cast
import numpy as np
import openai
from openai import OpenAI
import simpleaudio as sa
@ -26,6 +28,7 @@ from rich import print
import config
import tts
from intent import IntentClassifier
# some consts
CDIR = os.getcwd()
@ -59,6 +62,28 @@ VAD_FRAME_MS = 30
VAD_FRAME_BYTES = int(samplerate * VAD_FRAME_MS / 1000) * 2
vad = webrtcvad.Vad(config.VAD_AGGRESSIVENESS)
intent_classifier = IntentClassifier()
intent_classifier.prime({c: v['phrases'] for c, v in VA_CMD_LIST.items()})
if config.DENOISE_ENABLED:
import noisereduce as nr
else:
nr = None
def denoise_pcm(raw: bytes) -> bytes:
if not raw or nr is None:
return raw
samples = np.frombuffer(raw, dtype=np.int16).astype(np.float32) / 32768.0
reduced = nr.reduce_noise(
y=samples,
sr=samplerate,
prop_decrease=config.DENOISE_PROP,
stationary=config.DENOISE_STATIONARY,
)
clipped = np.clip(reduced * 32768.0, -32768, 32767).astype(np.int16)
return clipped.tobytes()
def gpt_answer():
global message_log
@ -139,9 +164,10 @@ def va_respond(voice: str):
print(cmd)
min_percent = int(round(config.INTENT_SIMILARITY_THRESHOLD * 100))
if len(cmd['cmd'].strip()) <= 0:
return False
elif cmd['percent'] < 70 or cmd['cmd'] not in VA_CMD_LIST.keys():
elif cmd['percent'] < min_percent or cmd['cmd'] not in VA_CMD_LIST.keys():
# play("not_found")
# tts.va_speak("Что?")
if fuzz.ratio(voice.join(voice.split()[:1]).strip(), "скажи") > 75:
@ -178,15 +204,9 @@ def filter_cmd(raw_voice: str):
def recognize_cmd(cmd: str):
rc = {'cmd': '', 'percent': 0}
for c, v in VA_CMD_LIST.items():
for x in v['phrases']:
vrt = fuzz.ratio(cmd, x)
if vrt > rc['percent']:
rc['cmd'] = c
rc['percent'] = vrt
return rc
candidates = {c: v['phrases'] for c, v in VA_CMD_LIST.items()}
res = intent_classifier.match(cmd, candidates)
return {'cmd': res['cmd'], 'percent': int(round(res['score'] * 100))}
def _set_mute(state: int):
@ -226,6 +246,26 @@ def run_action(action):
_shutdown()
elif t == 'sleep':
time.sleep(action['ms'] / 1000)
elif t == 'shell':
if action.get('wait'):
subprocess.check_call(action['cmd'], shell=True)
else:
subprocess.Popen(action['cmd'], shell=True)
elif t == 'url':
browser = action.get('browser')
if browser:
exe = config.BROWSER_PATHS.get(browser)
if exe and os.path.isfile(exe):
subprocess.Popen([exe, action['href']])
return
webbrowser.open(action['href'])
elif t == 'keys':
import pyautogui
if action.get('sequence'):
keys = [k.strip() for k in action['sequence'].split('+') if k.strip()]
pyautogui.hotkey(*keys)
elif action.get('text'):
pyautogui.typewrite(action['text'], interval=0.02)
elif t == 'multi':
for step in action['steps']:
run_action(step)
@ -280,6 +320,7 @@ while True:
silence_ms = 0
vad_buf = b""
finalize = False
cmd_buf = b"" if config.DENOISE_ENABLED else None
while True:
elapsed_ms = (time.time() - listen_started) * 1000
@ -288,7 +329,10 @@ while True:
pcm = recorder.read()
sp = struct.pack("h" * len(pcm), *pcm)
kaldi_rec.AcceptWaveform(sp)
if cmd_buf is None:
kaldi_rec.AcceptWaveform(sp)
else:
cmd_buf += sp
vad_buf += sp
while len(vad_buf) >= VAD_FRAME_BYTES:
@ -307,6 +351,8 @@ while True:
break
if finalize or speech_ms > 0:
if cmd_buf is not None:
kaldi_rec.AcceptWaveform(denoise_pcm(cmd_buf))
text = json.loads(kaldi_rec.FinalResult()).get("text", "")
if text:
va_respond(text)

View file

@ -0,0 +1,35 @@
---
license: apache-2.0
pipeline_tag: sentence-similarity
---
ONNX port of [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) for text classification and similarity searches.
### Usage
Here's an example of performing inference using the model with [FastEmbed](https://github.com/qdrant/fastembed).
```py
from fastembed import TextEmbedding
documents = [
"You should stay, study and sprint.",
"History can only prepare us to be surprised yet again.",
]
model = TextEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2")
embeddings = list(model.embed(documents))
# [
# array([
# 0.00611658, 0.00068912, -0.0203846, ..., -0.01751488, -0.01174267,
# 0.01463472
# ],
# dtype=float32),
# array([
# 0.00173448, -0.00329958, 0.01557874, ..., -0.01473586, 0.0281806,
# -0.00448205
# ],
# dtype=float32)
# ]
```

View file

@ -0,0 +1,25 @@
{
"_name_or_path": "sentence-transformers/all-MiniLM-L6-v2",
"architectures": [
"BertModel"
],
"attention_probs_dropout_prob": 0.1,
"classifier_dropout": null,
"gradient_checkpointing": false,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 384,
"initializer_range": 0.02,
"intermediate_size": 1536,
"layer_norm_eps": 1e-12,
"max_position_embeddings": 512,
"model_type": "bert",
"num_attention_heads": 12,
"num_hidden_layers": 6,
"pad_token_id": 0,
"position_embedding_type": "absolute",
"transformers_version": "4.36.2",
"type_vocab_size": 2,
"use_cache": true,
"vocab_size": 30522
}

BIN
models/all-MiniLM-L6-v2/model.onnx (Stored with Git LFS) Normal file

Binary file not shown.

View file

@ -0,0 +1,37 @@
{
"cls_token": {
"content": "[CLS]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"mask_token": {
"content": "[MASK]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "[PAD]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"sep_token": {
"content": "[SEP]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "[UNK]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,64 @@
{
"added_tokens_decoder": {
"0": {
"content": "[PAD]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"100": {
"content": "[UNK]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"101": {
"content": "[CLS]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"102": {
"content": "[SEP]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"103": {
"content": "[MASK]",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"clean_up_tokenization_spaces": true,
"cls_token": "[CLS]",
"do_basic_tokenize": true,
"do_lower_case": true,
"mask_token": "[MASK]",
"max_length": 128,
"model_max_length": 512,
"never_split": null,
"pad_to_multiple_of": null,
"pad_token": "[PAD]",
"pad_token_type_id": 0,
"padding_side": "right",
"sep_token": "[SEP]",
"stride": 0,
"strip_accents": null,
"tokenize_chinese_chars": true,
"tokenizer_class": "BertTokenizer",
"truncation_side": "right",
"truncation_strategy": "longest_first",
"unk_token": "[UNK]"
}

File diff suppressed because it is too large Load diff

View file

@ -26,3 +26,15 @@ pycaw
sounddevice
PyYAML
python-dotenv
# Semantic intent classifier (MiniLM-L6-v2 ONNX)
onnxruntime>=1.17
tokenizers>=0.15
# Noise suppression before STT (optional, off by default)
noisereduce>=3.0
# GUI command constructor (tools/command_builder)
ruamel.yaml
pywebview
pyautogui

6
run_builder.bat Normal file
View file

@ -0,0 +1,6 @@
@echo off
if exist .venv\Scripts\python.exe (
.venv\Scripts\python.exe -m tools.command_builder
) else (
python -m tools.command_builder
)

View file

@ -0,0 +1,68 @@
# Конструктор команд (developer notes)
GUI-надстройка для редактирования `commands.yaml` без ручной правки YAML. Запускается отдельным процессом, не зависит от рантайма Jarvis.
## Запуск
```
python -m tools.command_builder
```
или из корня репо `run_builder.bat`.
## Стек
- `pywebview` (бэкенд EdgeWebView2 на Windows) — оборачивает локальный HTML.
- Frontend — vanilla JS + CSS, без сборки. Vendored `js-yaml` только для предпросмотра YAML на стороне UI.
- `ruamel.yaml` — чтение/запись `commands.yaml` с сохранением комментариев и форматирования.
- `pyautogui` — рантайм для action-типа `keys`.
## Структура
```
tools/command_builder/
__main__.py точка входа python -m
app.py pywebview bootstrap + Python bridge (js_api)
schema.py dataclass-схема action-блоков и валидация
yaml_writer.py atomic write + .bak бэкап перед каждой записью
llm_suggest.py Groq/OpenAI SDK 1.x, response_format=json_object
web/ frontend
tests/ pytest для schema + writer
```
## Bridge API (методы класса `Bridge` в `app.py`)
Все методы возвращают сериализуемые структуры. Ошибки идут как `{"ok": false, "error": "..."}`.
| метод | назначение |
|-------|-----------|
| `list_existing_commands()` | список ключей верхнего уровня в `commands.yaml` |
| `list_exes()` | basename'ы `*.exe` из `custom-commands/` |
| `list_sound_names()` | разрешённые имена для `play_sound` (из `schema.VALID_SOUND_NAMES`) |
| `preview_sound(name)` | проигрывает звук в фоновом потоке через `simpleaudio` |
| `slugify(phrase)` | транслит для генерации `cmd_id` |
| `validate_draft(json_str)` | прогон через `command_from_dict` |
| `save_command(json_str)` | append/upsert + бэкап (`overwrite: true` чтобы перезаписать) |
| `llm_suggest(prompt)` | Groq → JSON → провалидированный action-блок |
## Расширение схемы (новый тип action)
1. В `schema.py`: добавить dataclass с полем `type` и методом `to_dict()`. Включить в Union `Action`. Расширить `action_from_dict()`.
2. В `tests/test_schema.py`: добавить roundtrip + negative-case тесты.
3. В `web/app.js`:
- карточка в `#action-grid` (`index.html`);
- ветка в `renderActionForm()`;
- ветка в `updateAction()`;
- ветка в `renderInlineEditor()` для использования внутри multi.steps.
4. В `main.py``run_action()`: обработчик нового `type`.
5. В `llm_suggest.py``_ACTION_SCHEMA_DOC`: описание для LLM.
## Бэкапы
`yaml_writer.append_command` / `upsert_command` перед записью копируют `commands.yaml` в `commands.yaml.bak` (через `shutil.copy2`). При запуске `app.py` вызывается `_check_yaml_health()` — если основной файл не парсится, в stderr печатается путь к `.bak` для ручного восстановления.
## Тесты
```
.venv\Scripts\python.exe -m pytest tools/command_builder/tests/
```

View file

View file

@ -0,0 +1,4 @@
from tools.command_builder.app import main
if __name__ == "__main__":
main()

View file

@ -0,0 +1,158 @@
from __future__ import annotations
import json
import os
import sys
import threading
from pathlib import Path
import webview
REPO_ROOT = Path(__file__).resolve().parents[2]
WEB_DIR = Path(__file__).resolve().parent / "web"
COMMANDS_YAML = REPO_ROOT / "commands.yaml"
CUSTOM_COMMANDS_DIR = REPO_ROOT / "custom-commands"
SOUND_DIR = REPO_ROOT / "sound"
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
from tools.command_builder.schema import (
CommandEntry,
SchemaError,
VALID_SOUND_NAMES,
action_from_dict,
command_from_dict,
slugify,
)
from tools.command_builder import yaml_writer
class Bridge:
def list_existing_commands(self):
try:
return yaml_writer.list_command_ids(COMMANDS_YAML)
except Exception as ex:
return {"error": str(ex)}
def list_exes(self):
if not CUSTOM_COMMANDS_DIR.exists():
return []
names = []
for p in sorted(CUSTOM_COMMANDS_DIR.iterdir()):
if p.suffix.lower() == ".exe":
names.append(p.stem)
return names
def list_sound_names(self):
return list(VALID_SOUND_NAMES)
def preview_sound(self, name):
if name not in VALID_SOUND_NAMES:
return {"ok": False, "error": "unknown sound"}
candidates = []
if name in ("ok", "greet"):
for i in (1, 2, 3):
candidates.append(SOUND_DIR / f"{name}{i}.wav")
candidates.append(SOUND_DIR / f"{name}.wav")
target = next((c for c in candidates if c.exists()), None)
if target is None:
return {"ok": False, "error": f"file not found for {name}"}
def _play():
try:
import simpleaudio as sa
wave_obj = sa.WaveObject.from_wave_file(str(target))
play_obj = wave_obj.play()
play_obj.wait_done()
except Exception:
pass
threading.Thread(target=_play, daemon=True).start()
return {"ok": True}
def slugify(self, phrase):
return slugify(phrase or "")
def validate_draft(self, draft_json):
try:
data = json.loads(draft_json) if isinstance(draft_json, str) else draft_json
cmd_id = data.get("cmd_id") or ""
if not cmd_id.strip():
return {"ok": False, "error": "Не указан идентификатор команды"}
entry = command_from_dict(cmd_id, {
"phrases": data.get("phrases") or [],
"action": data.get("action") or {},
**({"confirm_sound": data["confirm_sound"]} if data.get("confirm_sound") else {}),
})
return {"ok": True, "id": entry.cmd_id}
except SchemaError as ex:
return {"ok": False, "error": str(ex)}
except Exception as ex:
return {"ok": False, "error": f"Внутренняя ошибка: {ex}"}
def save_command(self, draft_json):
try:
data = json.loads(draft_json) if isinstance(draft_json, str) else draft_json
cmd_id = (data.get("cmd_id") or "").strip()
if not cmd_id:
return {"ok": False, "error": "Не указан идентификатор команды"}
entry = command_from_dict(cmd_id, {
"phrases": data.get("phrases") or [],
"action": data.get("action") or {},
**({"confirm_sound": data["confirm_sound"]} if data.get("confirm_sound") else {}),
})
existing = set(yaml_writer.list_command_ids(COMMANDS_YAML))
if entry.cmd_id in existing and not data.get("overwrite"):
return {"ok": False, "error": f"Команда {entry.cmd_id} уже существует"}
if data.get("overwrite"):
yaml_writer.upsert_command(entry, path=COMMANDS_YAML)
else:
yaml_writer.append_command(entry, path=COMMANDS_YAML)
return {"ok": True, "id": entry.cmd_id}
except SchemaError as ex:
return {"ok": False, "error": str(ex)}
except Exception as ex:
return {"ok": False, "error": f"Не удалось сохранить: {ex}"}
def llm_suggest(self, prompt):
try:
from tools.command_builder import llm_suggest as ls
return ls.suggest(prompt or "")
except Exception as ex:
return {"ok": False, "error": f"LLM недоступен: {ex}"}
def _check_yaml_health():
if not COMMANDS_YAML.exists():
return
try:
yaml_writer.load_yaml(COMMANDS_YAML)
except Exception as ex:
bak = COMMANDS_YAML.with_suffix(COMMANDS_YAML.suffix + ".bak")
msg = f"[command_builder] Не удалось разобрать {COMMANDS_YAML.name}: {ex}"
if bak.exists():
msg += f"\n[command_builder] Резервная копия доступна: {bak} — можно восстановить вручную."
print(msg, file=sys.stderr)
def main():
_check_yaml_health()
bridge = Bridge()
index_path = WEB_DIR / "index.html"
if not index_path.exists():
raise SystemExit(f"web/index.html not found at {index_path}")
webview.create_window(
title="J.A.R.V.I.S — конструктор команд",
url=str(index_path),
js_api=bridge,
width=900,
height=720,
min_size=(720, 560),
background_color="#0a1214",
)
webview.start()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,153 @@
from __future__ import annotations
import json
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
import config
from tools.command_builder.schema import (
SchemaError,
VALID_SOUND_NAMES,
VALID_SYSTEM_OPS,
action_from_dict,
slugify,
)
_ACTION_SCHEMA_DOC = """
Допустимые типы действий (action.type):
- exe запуск .exe из custom-commands/
{"type":"exe","file":"<basename.exe>","wait":<bool optional>,"delay_ms":<int optional>}
- play_sound встроенный звук
{"type":"play_sound","name":"<one of: %SOUNDS%>"}
- system системная операция
{"type":"system","op":"<one of: %SYS_OPS%>"}
- sleep пауза в миллисекундах (только внутри multi)
{"type":"sleep","ms":<int>}
- shell PowerShell/cmd строка
{"type":"shell","cmd":"<string>","wait":<bool optional>}
- url открыть URL в системном браузере
{"type":"url","href":"<https://...>"}
- keys нажатие клавиш ИЛИ ввод текста (только одно из двух)
{"type":"keys","sequence":"ctrl+shift+m"}
{"type":"keys","text":"печатаемая строка"}
- multi последовательность шагов
{"type":"multi","steps":[<action>, ...]}
""".strip()
def _build_system_prompt(exe_names: list[str]) -> str:
schema = (
_ACTION_SCHEMA_DOC
.replace("%SOUNDS%", ", ".join(VALID_SOUND_NAMES))
.replace("%SYS_OPS%", ", ".join(VALID_SYSTEM_OPS))
)
if exe_names:
exe_lines = "\n".join(f" - {n}.exe" for n in exe_names)
else:
exe_lines = " (папка custom-commands/ пуста)"
return (
"Ты помогаешь собрать команду для голосового ассистента Jarvis. "
"Пользователь описывает желаемое поведение по-русски. "
"Верни ОДИН JSON-объект ровно с такими полями:\n"
"{\n"
' "cmd_id": "<латиница, snake_case, опционально — иначе будет сгенерирован>",\n'
' "phrases": ["<голосовая фраза 1>", "..."],\n'
' "action": <один action из схемы ниже>,\n'
' "confirm_sound": "<опционально, имя звука для подтверждения>"\n'
"}\n\n"
f"{schema}\n\n"
"Доступные .exe файлы (используй ровно эти имена в action.file):\n"
f"{exe_lines}\n\n"
"Правила:\n"
"- Только валидный JSON, без пояснений и markdown.\n"
"- phrases — 1-3 коротких русских фразы, без слова-активатора «джарвис».\n"
"- Если нужно несколько шагов — используй multi.\n"
"- sleep валиден только внутри multi.steps."
)
def _get_client():
from openai import OpenAI
if not config.GROQ_TOKEN:
raise RuntimeError("GROQ_TOKEN не задан в dev.env")
return OpenAI(api_key=config.GROQ_TOKEN, base_url=config.GROQ_BASE_URL)
def _list_exes() -> list[str]:
custom = REPO_ROOT / "custom-commands"
if not custom.exists():
return []
return sorted(p.stem for p in custom.iterdir() if p.suffix.lower() == ".exe")
def suggest(prompt: str) -> dict:
user_prompt = (prompt or "").strip()
if not user_prompt:
return {"ok": False, "error": "Пустой запрос"}
try:
client = _get_client()
except Exception as ex:
return {"ok": False, "error": str(ex)}
system_prompt = _build_system_prompt(_list_exes())
try:
response = client.chat.completions.create(
model=config.GROQ_MODEL,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
],
response_format={"type": "json_object"},
temperature=0.2,
max_tokens=800,
)
except Exception as ex:
return {"ok": False, "error": f"Ошибка запроса к Groq: {ex}"}
raw = response.choices[0].message.content or ""
try:
parsed = json.loads(raw)
except json.JSONDecodeError as ex:
return {"ok": False, "error": f"Модель вернула не-JSON: {ex}", "raw": raw}
phrases = parsed.get("phrases")
if not isinstance(phrases, list) or not phrases:
return {"ok": False, "error": "В ответе нет phrases", "raw": parsed}
phrases = [p for p in phrases if isinstance(p, str) and p.strip()]
if not phrases:
return {"ok": False, "error": "phrases пуст после фильтрации", "raw": parsed}
action_raw = parsed.get("action")
if not isinstance(action_raw, dict):
return {"ok": False, "error": "В ответе нет action", "raw": parsed}
try:
action_obj = action_from_dict(action_raw)
except SchemaError as ex:
return {"ok": False, "error": f"Action не прошёл валидацию: {ex}", "raw": parsed}
cmd_id = parsed.get("cmd_id")
if not isinstance(cmd_id, str) or not cmd_id.strip():
cmd_id = slugify(phrases[0])
confirm = parsed.get("confirm_sound")
if confirm is not None and confirm not in VALID_SOUND_NAMES:
confirm = None
suggestion = {
"cmd_id": cmd_id,
"phrases": phrases,
"action": action_obj.to_dict(),
}
if confirm:
suggestion["confirm_sound"] = confirm
return {"ok": True, "suggestion": suggestion}

View file

@ -0,0 +1,247 @@
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, List, Optional, Union
class SchemaError(ValueError):
pass
@dataclass
class ExeAction:
file: str
wait: Optional[bool] = None
delay_ms: Optional[int] = None
type: str = "exe"
def to_dict(self) -> dict:
out: dict = {"type": "exe", "file": self.file}
if self.wait is not None:
out["wait"] = bool(self.wait)
if self.delay_ms is not None:
out["delay_ms"] = int(self.delay_ms)
return out
@dataclass
class PlaySoundAction:
name: str
type: str = "play_sound"
def to_dict(self) -> dict:
return {"type": "play_sound", "name": self.name}
@dataclass
class SystemAction:
op: str
type: str = "system"
def to_dict(self) -> dict:
return {"type": "system", "op": self.op}
@dataclass
class SleepAction:
ms: int
type: str = "sleep"
def to_dict(self) -> dict:
return {"type": "sleep", "ms": int(self.ms)}
@dataclass
class ShellAction:
cmd: str
wait: Optional[bool] = None
type: str = "shell"
def to_dict(self) -> dict:
out: dict = {"type": "shell", "cmd": self.cmd}
if self.wait is not None:
out["wait"] = bool(self.wait)
return out
KNOWN_BROWSERS = ("yandex", "chrome", "firefox", "edge")
@dataclass
class UrlAction:
href: str
browser: Optional[str] = None
type: str = "url"
def to_dict(self) -> dict:
out: dict = {"type": "url", "href": self.href}
if self.browser:
out["browser"] = self.browser
return out
@dataclass
class KeysAction:
sequence: Optional[str] = None
text: Optional[str] = None
type: str = "keys"
def to_dict(self) -> dict:
out: dict = {"type": "keys"}
if self.sequence is not None:
out["sequence"] = self.sequence
if self.text is not None:
out["text"] = self.text
return out
Action = Union[
"ExeAction",
"PlaySoundAction",
"SystemAction",
"SleepAction",
"ShellAction",
"UrlAction",
"KeysAction",
"MultiAction",
]
@dataclass
class MultiAction:
steps: List[Action] = field(default_factory=list)
type: str = "multi"
def to_dict(self) -> dict:
return {"type": "multi", "steps": [s.to_dict() for s in self.steps]}
VALID_SYSTEM_OPS = ("volume_mute", "volume_unmute", "exit")
VALID_SOUND_NAMES = (
"ok", "ready", "thanks", "not_found", "greet", "run", "stupid", "off",
)
def action_from_dict(data: Any) -> Action:
if not isinstance(data, dict):
raise SchemaError("action must be a mapping")
t = data.get("type")
if not t:
raise SchemaError("action is missing 'type'")
if t == "exe":
if not data.get("file"):
raise SchemaError("exe action requires 'file'")
return ExeAction(
file=data["file"],
wait=data.get("wait"),
delay_ms=data.get("delay_ms"),
)
if t == "play_sound":
name = data.get("name")
if not name:
raise SchemaError("play_sound action requires 'name'")
if name not in VALID_SOUND_NAMES:
raise SchemaError(f"unknown sound name: {name}")
return PlaySoundAction(name=name)
if t == "system":
op = data.get("op")
if op not in VALID_SYSTEM_OPS:
raise SchemaError(f"unknown system op: {op}")
return SystemAction(op=op)
if t == "sleep":
ms = data.get("ms")
if not isinstance(ms, int) or ms < 0:
raise SchemaError("sleep action requires non-negative integer 'ms'")
return SleepAction(ms=ms)
if t == "shell":
cmd = data.get("cmd")
if not cmd or not isinstance(cmd, str):
raise SchemaError("shell action requires non-empty 'cmd' string")
return ShellAction(cmd=cmd, wait=data.get("wait"))
if t == "url":
href = data.get("href")
if not href or not isinstance(href, str):
raise SchemaError("url action requires non-empty 'href' string")
browser = data.get("browser")
if browser is not None and browser not in KNOWN_BROWSERS:
raise SchemaError(f"url.browser must be one of {KNOWN_BROWSERS} or omitted")
return UrlAction(href=href, browser=browser)
if t == "keys":
sequence = data.get("sequence")
text = data.get("text")
if not sequence and not text:
raise SchemaError("keys action requires 'sequence' or 'text'")
if sequence and text:
raise SchemaError("keys action cannot have both 'sequence' and 'text'")
return KeysAction(sequence=sequence, text=text)
if t == "multi":
steps_raw = data.get("steps")
if not isinstance(steps_raw, list) or not steps_raw:
raise SchemaError("multi action requires non-empty 'steps' list")
steps = [action_from_dict(s) for s in steps_raw]
return MultiAction(steps=steps)
raise SchemaError(f"unknown action type: {t}")
@dataclass
class CommandEntry:
cmd_id: str
phrases: List[str]
action: Action
confirm_sound: Optional[str] = None
def to_dict(self) -> dict:
body: dict = {
"phrases": list(self.phrases),
"action": self.action.to_dict(),
}
if self.confirm_sound is not None:
body["confirm_sound"] = self.confirm_sound
return body
def command_from_dict(cmd_id: str, data: Any) -> CommandEntry:
if not isinstance(data, dict):
raise SchemaError(f"command {cmd_id!r} body must be a mapping")
phrases = data.get("phrases")
if not isinstance(phrases, list) or not phrases:
raise SchemaError(f"command {cmd_id!r} requires non-empty 'phrases' list")
if not all(isinstance(p, str) and p.strip() for p in phrases):
raise SchemaError(f"command {cmd_id!r} phrases must be non-empty strings")
action_raw = data.get("action")
if action_raw is None:
raise SchemaError(f"command {cmd_id!r} requires 'action'")
action = action_from_dict(action_raw)
confirm = data.get("confirm_sound")
if confirm is not None and confirm not in VALID_SOUND_NAMES:
raise SchemaError(f"command {cmd_id!r} has unknown confirm_sound: {confirm}")
return CommandEntry(cmd_id=cmd_id, phrases=phrases, action=action, confirm_sound=confirm)
_SLUG_TRANSLIT = {
"а": "a", "б": "b", "в": "v", "г": "g", "д": "d", "е": "e", "ё": "e",
"ж": "zh", "з": "z", "и": "i", "й": "y", "к": "k", "л": "l", "м": "m",
"н": "n", "о": "o", "п": "p", "р": "r", "с": "s", "т": "t", "у": "u",
"ф": "f", "х": "h", "ц": "ts", "ч": "ch", "ш": "sh", "щ": "sch",
"ъ": "", "ы": "y", "ь": "", "э": "e", "ю": "yu", "я": "ya",
}
def slugify(phrase: str) -> str:
lowered = phrase.lower().strip()
out_chars: list = []
for ch in lowered:
if ch in _SLUG_TRANSLIT:
out_chars.append(_SLUG_TRANSLIT[ch])
elif ch.isalnum() and ord(ch) < 128:
out_chars.append(ch)
elif ch in (" ", "_", "-"):
out_chars.append("_")
slug = "".join(out_chars)
while "__" in slug:
slug = slug.replace("__", "_")
slug = slug.strip("_")
return slug or "cmd"

View file

View file

@ -0,0 +1,263 @@
from __future__ import annotations
import io
from pathlib import Path
import pytest
from ruamel.yaml import YAML
from tools.command_builder.schema import (
CommandEntry,
ExeAction,
KeysAction,
MultiAction,
PlaySoundAction,
SchemaError,
ShellAction,
SleepAction,
SystemAction,
UrlAction,
action_from_dict,
command_from_dict,
slugify,
)
from tools.command_builder.yaml_writer import (
append_command,
load_yaml,
upsert_command,
)
def _roundtrip(action_obj):
d = action_obj.to_dict()
y = YAML()
buf = io.StringIO()
y.dump(d, buf)
buf.seek(0)
loaded = y.load(buf)
return action_from_dict(dict(loaded))
def test_exe_roundtrip():
a = ExeAction(file="Run browser.exe", wait=True, delay_ms=200)
b = _roundtrip(a)
assert b.file == a.file and b.wait is True and b.delay_ms == 200
def test_exe_minimal():
a = ExeAction(file="x.exe")
b = _roundtrip(a)
assert b.file == "x.exe" and b.wait is None and b.delay_ms is None
def test_play_sound_roundtrip():
a = PlaySoundAction(name="ok")
b = _roundtrip(a)
assert b.name == "ok"
def test_play_sound_rejects_unknown():
with pytest.raises(SchemaError):
action_from_dict({"type": "play_sound", "name": "nope"})
def test_system_roundtrip():
for op in ("volume_mute", "volume_unmute", "exit"):
a = SystemAction(op=op)
b = _roundtrip(a)
assert b.op == op
def test_system_rejects_unknown_op():
with pytest.raises(SchemaError):
action_from_dict({"type": "system", "op": "reboot"})
def test_sleep_roundtrip():
a = SleepAction(ms=1500)
b = _roundtrip(a)
assert b.ms == 1500
def test_sleep_rejects_negative():
with pytest.raises(SchemaError):
action_from_dict({"type": "sleep", "ms": -1})
def test_shell_roundtrip():
a = ShellAction(cmd="echo hello", wait=True)
b = _roundtrip(a)
assert b.cmd == "echo hello" and b.wait is True
def test_shell_minimal():
a = ShellAction(cmd="echo x")
b = _roundtrip(a)
assert b.wait is None
def test_shell_requires_cmd():
with pytest.raises(SchemaError):
action_from_dict({"type": "shell"})
def test_url_roundtrip():
a = UrlAction(href="https://example.com")
b = _roundtrip(a)
assert b.href == "https://example.com"
def test_url_requires_href():
with pytest.raises(SchemaError):
action_from_dict({"type": "url"})
def test_keys_sequence_roundtrip():
a = KeysAction(sequence="ctrl+shift+m")
b = _roundtrip(a)
assert b.sequence == "ctrl+shift+m" and b.text is None
def test_keys_text_roundtrip():
a = KeysAction(text="hello world")
b = _roundtrip(a)
assert b.text == "hello world" and b.sequence is None
def test_keys_requires_one_of():
with pytest.raises(SchemaError):
action_from_dict({"type": "keys"})
def test_keys_rejects_both():
with pytest.raises(SchemaError):
action_from_dict({"type": "keys", "sequence": "ctrl+c", "text": "x"})
def test_multi_roundtrip():
a = MultiAction(steps=[
PlaySoundAction(name="ok"),
ExeAction(file="x.exe", wait=True),
SleepAction(ms=500),
UrlAction(href="https://example.com"),
KeysAction(sequence="ctrl+s"),
ShellAction(cmd="dir"),
SystemAction(op="volume_mute"),
])
b = _roundtrip(a)
assert isinstance(b, MultiAction)
assert len(b.steps) == 7
assert isinstance(b.steps[0], PlaySoundAction)
assert isinstance(b.steps[4], KeysAction)
assert b.steps[4].sequence == "ctrl+s"
def test_unknown_type_rejected():
with pytest.raises(SchemaError):
action_from_dict({"type": "wat"})
def test_command_entry_roundtrip():
entry = CommandEntry(
cmd_id="open_site",
phrases=["открой сайт", "открой пример"],
action=UrlAction(href="https://example.com"),
)
d = entry.to_dict()
restored = command_from_dict("open_site", d)
assert restored.cmd_id == "open_site"
assert restored.phrases == ["открой сайт", "открой пример"]
assert isinstance(restored.action, UrlAction)
assert restored.action.href == "https://example.com"
def test_command_requires_phrases():
with pytest.raises(SchemaError):
command_from_dict("x", {"phrases": [], "action": {"type": "system", "op": "exit"}})
def test_slugify_russian():
assert slugify("Открой Браузер") == "otkroy_brauzer"
assert slugify("hello world") == "hello_world"
assert slugify(" ") == "cmd"
def _write_fixture(path: Path) -> None:
path.write_text(
"existing_cmd:\n"
" phrases:\n"
" - пример\n"
" action: {type: play_sound, name: ok}\n"
"\n"
"another:\n"
" phrases:\n"
" - другой\n"
" action: {type: system, op: exit}\n",
encoding="utf-8",
)
def test_append_writes_and_creates_backup(tmp_path):
yaml_path = tmp_path / "commands.yaml"
_write_fixture(yaml_path)
original = yaml_path.read_text(encoding="utf-8")
entry = CommandEntry(
cmd_id="open_docs",
phrases=["открой доки", "документация"],
action=UrlAction(href="https://docs.example.com"),
)
append_command(entry, path=yaml_path)
bak = yaml_path.with_suffix(yaml_path.suffix + ".bak")
assert bak.exists()
assert bak.read_text(encoding="utf-8") == original
data = load_yaml(yaml_path)
assert "open_docs" in data
assert "existing_cmd" in data
assert data["open_docs"]["action"]["href"] == "https://docs.example.com"
def test_append_rejects_duplicate_id(tmp_path):
yaml_path = tmp_path / "commands.yaml"
_write_fixture(yaml_path)
entry = CommandEntry(
cmd_id="existing_cmd",
phrases=["дубликат"],
action=SystemAction(op="exit"),
)
with pytest.raises(SchemaError):
append_command(entry, path=yaml_path)
def test_upsert_overwrites(tmp_path):
yaml_path = tmp_path / "commands.yaml"
_write_fixture(yaml_path)
entry = CommandEntry(
cmd_id="existing_cmd",
phrases=["обновлено"],
action=SystemAction(op="volume_mute"),
)
upsert_command(entry, path=yaml_path)
data = load_yaml(yaml_path)
assert data["existing_cmd"]["action"]["op"] == "volume_mute"
def test_ruamel_preserves_inline_comments(tmp_path):
yaml_path = tmp_path / "commands.yaml"
yaml_path.write_text(
"# header comment about commands\n"
"open_browser:\n"
" phrases:\n"
" - открой браузер\n"
" action: {type: exe, file: \"Run browser.exe\"} # inline note\n",
encoding="utf-8",
)
entry = CommandEntry(
cmd_id="open_new",
phrases=["новое"],
action=UrlAction(href="https://new.example.com"),
)
append_command(entry, path=yaml_path)
text = yaml_path.read_text(encoding="utf-8")
assert "# header comment about commands" in text
assert "# inline note" in text
assert "open_new" in text

View file

@ -0,0 +1,866 @@
(function () {
const state = {
step: 1,
phrases: [""],
cmdId: "",
cmdIdAuto: true,
existingIds: [],
actionType: null,
action: null,
confirmSound: null,
};
const els = {
steps: document.querySelectorAll(".step"),
panels: document.querySelectorAll(".panel"),
phrase0: document.getElementById("phrase-0"),
extra: document.getElementById("extra-phrases"),
addPhrase: document.getElementById("add-phrase"),
cmdId: document.getElementById("cmd-id"),
cmdIdHint: document.getElementById("cmd-id-hint"),
actionGrid: document.getElementById("action-grid"),
paramsForm: document.getElementById("params-form"),
yamlPreview: document.getElementById("yaml-preview"),
saveBtn: document.getElementById("save-btn"),
cancelBtn: document.getElementById("cancel-btn"),
saveStatus: document.getElementById("save-status"),
toast: document.getElementById("toast"),
toStep3: document.getElementById("to-step-3"),
tabs: document.querySelectorAll(".tab"),
tabContents: document.querySelectorAll("[data-tab-content]"),
llmPrompt: document.getElementById("llm-prompt"),
llmGenerate: document.getElementById("llm-generate"),
llmStatus: document.getElementById("llm-status"),
llmSuggestions: document.getElementById("llm-suggestions"),
};
function api() {
return (window.pywebview && window.pywebview.api) || null;
}
async function callApi(method, ...args) {
const a = api();
if (!a || typeof a[method] !== "function") {
return { ok: false, error: "bridge not ready: " + method };
}
try {
return await a[method](...args);
} catch (ex) {
return { ok: false, error: String(ex) };
}
}
function setStep(n) {
state.step = n;
els.panels.forEach((p) => {
p.hidden = parseInt(p.dataset.panel, 10) !== n;
});
els.steps.forEach((s) => {
s.classList.toggle("active", parseInt(s.dataset.step, 10) === n);
});
if (n === 4) {
renderPreview();
}
}
function showToast(msg, isError) {
els.toast.textContent = msg;
els.toast.classList.toggle("error", !!isError);
els.toast.hidden = false;
clearTimeout(showToast._t);
showToast._t = setTimeout(() => { els.toast.hidden = true; }, 2800);
}
function refreshExtraPhrases() {
els.extra.innerHTML = "";
for (let i = 1; i < state.phrases.length; i++) {
const row = document.createElement("div");
row.className = "extra-row";
const inp = document.createElement("input");
inp.type = "text";
inp.value = state.phrases[i];
inp.placeholder = "синоним";
inp.addEventListener("input", () => {
state.phrases[i] = inp.value;
});
const rm = document.createElement("button");
rm.type = "button";
rm.textContent = "удалить";
rm.addEventListener("click", () => {
state.phrases.splice(i, 1);
refreshExtraPhrases();
});
row.appendChild(inp);
row.appendChild(rm);
els.extra.appendChild(row);
}
}
function transliterateSlug(s) {
const map = {
"а":"a","б":"b","в":"v","г":"g","д":"d","е":"e","ё":"e","ж":"zh",
"з":"z","и":"i","й":"y","к":"k","л":"l","м":"m","н":"n","о":"o",
"п":"p","р":"r","с":"s","т":"t","у":"u","ф":"f","х":"h","ц":"ts",
"ч":"ch","ш":"sh","щ":"sch","ъ":"","ы":"y","ь":"","э":"e","ю":"yu","я":"ya",
};
let out = "";
const lower = (s || "").toLowerCase().trim();
for (const ch of lower) {
if (map[ch] !== undefined) out += map[ch];
else if (/[a-z0-9]/i.test(ch)) out += ch;
else if (ch === " " || ch === "_" || ch === "-") out += "_";
}
out = out.replace(/_+/g, "_").replace(/^_|_$/g, "");
return out || "cmd";
}
function checkCmdIdCollision() {
if (!state.cmdId) {
els.cmdIdHint.className = "row-hint";
els.cmdIdHint.textContent = "";
return;
}
if (state.existingIds.includes(state.cmdId)) {
els.cmdIdHint.className = "row-hint error";
els.cmdIdHint.textContent = `Идентификатор «${state.cmdId}» уже используется`;
} else {
els.cmdIdHint.className = "row-hint ok";
els.cmdIdHint.textContent = "Свободен";
}
}
function syncCmdId() {
if (state.cmdIdAuto) {
state.cmdId = transliterateSlug(state.phrases[0] || "");
els.cmdId.value = state.cmdId;
}
checkCmdIdCollision();
}
function renderActionForm() {
const t = state.actionType;
const f = els.paramsForm;
f.innerHTML = "";
if (!t) {
f.innerHTML = `<p class="hint">Сначала выберите тип на шаге 2.</p>`;
return;
}
if (t === "exe") {
const wrap = document.createElement("div");
wrap.innerHTML = `
<div class="field">
<label>Файл из custom-commands/</label>
<select id="exe-file"><option value=""> выбрать </option></select>
</div>
<div class="params-row">
<div class="field">
<label>Задержка после запуска (мс)</label>
<input type="number" id="exe-delay" min="0" step="50" placeholder="0">
</div>
</div>
<label class="checkbox"><input type="checkbox" id="exe-wait"> ждать завершения</label>
<div class="row-hint" id="exe-hint"></div>
`;
f.appendChild(wrap);
const sel = wrap.querySelector("#exe-file");
callApi("list_exes").then((list) => {
if (Array.isArray(list)) {
for (const name of list) {
const o = document.createElement("option");
o.value = `${name}.exe`;
o.textContent = name;
sel.appendChild(o);
}
if (state.action && state.action.file) sel.value = state.action.file;
}
});
sel.addEventListener("change", () => updateAction());
wrap.querySelector("#exe-delay").addEventListener("input", () => updateAction());
wrap.querySelector("#exe-wait").addEventListener("change", () => updateAction());
if (state.action) {
wrap.querySelector("#exe-delay").value = state.action.delay_ms ?? "";
wrap.querySelector("#exe-wait").checked = !!state.action.wait;
}
return;
}
if (t === "url") {
f.innerHTML = `
<div class="field">
<label>URL</label>
<input type="text" id="url-href" placeholder="https://example.com">
</div>
<div class="field">
<label>Браузер</label>
<select id="url-browser">
<option value="">по умолчанию (системный)</option>
<option value="yandex">Yandex Browser</option>
<option value="chrome">Google Chrome</option>
<option value="firefox">Mozilla Firefox</option>
<option value="edge">Microsoft Edge</option>
</select>
</div>
`;
const inp = f.querySelector("#url-href");
const sel = f.querySelector("#url-browser");
if (state.action) {
inp.value = state.action.href || "";
sel.value = state.action.browser || "";
}
inp.addEventListener("input", () => updateAction());
sel.addEventListener("change", () => updateAction());
return;
}
if (t === "keys") {
f.innerHTML = `
<div class="field">
<label>Режим</label>
<select id="keys-mode">
<option value="sequence">Сочетание клавиш</option>
<option value="text">Печатать текст</option>
</select>
</div>
<div class="field" id="keys-seq-field">
<label>Сочетание</label>
<input type="text" id="keys-sequence" placeholder="ctrl+shift+m">
<div class="row-hint">Разделяйте плюсом. Пример: ctrl+alt+delete</div>
</div>
<div class="field" id="keys-text-field" hidden>
<label>Текст</label>
<input type="text" id="keys-text" placeholder="Привет, мир">
</div>
`;
const mode = f.querySelector("#keys-mode");
const seqField = f.querySelector("#keys-seq-field");
const textField = f.querySelector("#keys-text-field");
const seq = f.querySelector("#keys-sequence");
const text = f.querySelector("#keys-text");
if (state.action && state.action.text) {
mode.value = "text";
text.value = state.action.text;
seqField.hidden = true;
textField.hidden = false;
} else if (state.action && state.action.sequence) {
seq.value = state.action.sequence;
}
const sync = () => {
if (mode.value === "sequence") {
seqField.hidden = false;
textField.hidden = true;
} else {
seqField.hidden = true;
textField.hidden = false;
}
updateAction();
};
mode.addEventListener("change", sync);
seq.addEventListener("input", () => updateAction());
text.addEventListener("input", () => updateAction());
return;
}
if (t === "play_sound") {
f.innerHTML = `
<div class="params-row">
<div class="field">
<label>Имя звука</label>
<select id="snd-name"></select>
</div>
<button type="button" class="ghost" id="snd-preview">прослушать</button>
</div>
`;
const sel = f.querySelector("#snd-name");
callApi("list_sound_names").then((list) => {
if (Array.isArray(list)) {
for (const name of list) {
const o = document.createElement("option");
o.value = name;
o.textContent = name;
sel.appendChild(o);
}
if (state.action && state.action.name) sel.value = state.action.name;
updateAction();
}
});
sel.addEventListener("change", () => updateAction());
f.querySelector("#snd-preview").addEventListener("click", () => {
callApi("preview_sound", sel.value);
});
return;
}
if (t === "system") {
f.innerHTML = `
<div class="field">
<label>Операция</label>
<select id="sys-op">
<option value="volume_mute">Выключить звук</option>
<option value="volume_unmute">Включить звук</option>
<option value="exit">Выгрузить Jarvis</option>
</select>
</div>
`;
const sel = f.querySelector("#sys-op");
if (state.action && state.action.op) sel.value = state.action.op;
sel.addEventListener("change", () => updateAction());
return;
}
if (t === "shell") {
f.innerHTML = `
<div class="field">
<label>Команда (PowerShell)</label>
<textarea id="shell-cmd" rows="3" placeholder="Get-Process | Select-Object -First 5"></textarea>
</div>
<label class="checkbox"><input type="checkbox" id="shell-wait"> ждать завершения</label>
`;
const cmd = f.querySelector("#shell-cmd");
const wait = f.querySelector("#shell-wait");
if (state.action) {
cmd.value = state.action.cmd || "";
wait.checked = !!state.action.wait;
}
cmd.addEventListener("input", () => updateAction());
wait.addEventListener("change", () => updateAction());
return;
}
if (t === "sleep") {
f.innerHTML = `
<div class="field">
<label>Длительность (мс)</label>
<input type="number" id="sleep-ms" min="0" step="50" value="500">
</div>
<div class="row-hint">Обычно используется внутри последовательности.</div>
`;
const inp = f.querySelector("#sleep-ms");
if (state.action && state.action.ms != null) inp.value = state.action.ms;
inp.addEventListener("input", () => updateAction());
return;
}
if (t === "multi") {
renderMultiForm();
return;
}
}
function renderMultiForm() {
const f = els.paramsForm;
f.innerHTML = `
<p class="hint">Шаги выполняются по порядку.</p>
<div class="steps-list" id="steps-list"></div>
<button type="button" class="ghost" id="add-step">+ шаг</button>
`;
const list = f.querySelector("#steps-list");
if (!state.action || state.action.type !== "multi") {
state.action = { type: "multi", steps: [] };
}
const renderList = () => {
list.innerHTML = "";
state.action.steps.forEach((step, idx) => {
const card = document.createElement("div");
card.className = "step-card";
card.innerHTML = `
<div class="step-head">
<span class="step-label">Шаг ${idx + 1}</span>
<div>
<button type="button" class="icon-btn" data-up></button>
<button type="button" class="icon-btn" data-down></button>
<button type="button" class="icon-btn" data-rm>удалить</button>
</div>
</div>
`;
const body = document.createElement("div");
renderInlineEditor(body, step, (next) => {
state.action.steps[idx] = next;
updateAction(true);
});
card.appendChild(body);
card.querySelector("[data-up]").addEventListener("click", () => {
if (idx > 0) {
const tmp = state.action.steps[idx - 1];
state.action.steps[idx - 1] = state.action.steps[idx];
state.action.steps[idx] = tmp;
renderList();
updateAction(true);
}
});
card.querySelector("[data-down]").addEventListener("click", () => {
if (idx < state.action.steps.length - 1) {
const tmp = state.action.steps[idx + 1];
state.action.steps[idx + 1] = state.action.steps[idx];
state.action.steps[idx] = tmp;
renderList();
updateAction(true);
}
});
card.querySelector("[data-rm]").addEventListener("click", () => {
state.action.steps.splice(idx, 1);
renderList();
updateAction(true);
});
list.appendChild(card);
});
};
renderList();
f.querySelector("#add-step").addEventListener("click", () => {
state.action.steps.push({ type: "play_sound", name: "ok" });
renderList();
updateAction(true);
});
updateAction(true);
}
function renderInlineEditor(host, step, onChange) {
const typeSel = document.createElement("select");
const types = [
["play_sound", "Звук"],
["exe", ".exe"],
["url", "URL"],
["keys", "Клавиши"],
["shell", "Shell"],
["system", "Система"],
["sleep", "Пауза"],
];
for (const [v, label] of types) {
const o = document.createElement("option");
o.value = v;
o.textContent = label;
if (v === step.type) o.selected = true;
typeSel.appendChild(o);
}
host.appendChild(wrapField("Тип", typeSel));
const body = document.createElement("div");
host.appendChild(body);
const renderBody = () => {
body.innerHTML = "";
const t = typeSel.value;
if (t === "play_sound") {
const sel = document.createElement("select");
callApi("list_sound_names").then((list) => {
if (Array.isArray(list)) {
for (const name of list) {
const o = document.createElement("option");
o.value = name;
o.textContent = name;
if (step.name === name) o.selected = true;
sel.appendChild(o);
}
}
});
sel.addEventListener("change", () => onChange({ type: "play_sound", name: sel.value }));
body.appendChild(wrapField("Звук", sel));
} else if (t === "exe") {
const sel = document.createElement("select");
callApi("list_exes").then((list) => {
if (Array.isArray(list)) {
for (const name of list) {
const o = document.createElement("option");
o.value = `${name}.exe`;
o.textContent = name;
if (step.file === `${name}.exe`) o.selected = true;
sel.appendChild(o);
}
}
});
const wait = document.createElement("input");
wait.type = "checkbox";
wait.checked = !!step.wait;
const waitLabel = document.createElement("label");
waitLabel.className = "checkbox";
waitLabel.appendChild(wait);
waitLabel.appendChild(document.createTextNode("ждать"));
const delay = document.createElement("input");
delay.type = "number";
delay.value = step.delay_ms ?? "";
delay.placeholder = "delay_ms";
const apply = () => {
const next = { type: "exe", file: sel.value };
if (wait.checked) next.wait = true;
if (delay.value) next.delay_ms = parseInt(delay.value, 10);
onChange(next);
};
sel.addEventListener("change", apply);
wait.addEventListener("change", apply);
delay.addEventListener("input", apply);
body.appendChild(wrapField("Файл", sel));
body.appendChild(wrapField("Задержка (мс)", delay));
body.appendChild(waitLabel);
} else if (t === "url") {
const inp = document.createElement("input");
inp.type = "text";
inp.value = step.href || "";
inp.placeholder = "https://...";
const sel = document.createElement("select");
sel.innerHTML = `
<option value="">по умолчанию (системный)</option>
<option value="yandex">Yandex Browser</option>
<option value="chrome">Google Chrome</option>
<option value="firefox">Mozilla Firefox</option>
<option value="edge">Microsoft Edge</option>`;
sel.value = step.browser || "";
const apply = () => {
const next = { type: "url", href: inp.value };
if (sel.value) next.browser = sel.value;
onChange(next);
};
inp.addEventListener("input", apply);
sel.addEventListener("change", apply);
body.appendChild(wrapField("URL", inp));
body.appendChild(wrapField("Браузер", sel));
} else if (t === "keys") {
const inp = document.createElement("input");
inp.type = "text";
inp.value = step.sequence || "";
inp.placeholder = "ctrl+shift+m";
inp.addEventListener("input", () => onChange({ type: "keys", sequence: inp.value }));
body.appendChild(wrapField("Сочетание", inp));
} else if (t === "shell") {
const inp = document.createElement("input");
inp.type = "text";
inp.value = step.cmd || "";
const wait = document.createElement("input");
wait.type = "checkbox";
wait.checked = !!step.wait;
const waitLabel = document.createElement("label");
waitLabel.className = "checkbox";
waitLabel.appendChild(wait);
waitLabel.appendChild(document.createTextNode("ждать"));
const apply = () => {
const next = { type: "shell", cmd: inp.value };
if (wait.checked) next.wait = true;
onChange(next);
};
inp.addEventListener("input", apply);
wait.addEventListener("change", apply);
body.appendChild(wrapField("Команда", inp));
body.appendChild(waitLabel);
} else if (t === "system") {
const sel = document.createElement("select");
for (const op of ["volume_mute", "volume_unmute", "exit"]) {
const o = document.createElement("option");
o.value = op;
o.textContent = op;
if (step.op === op) o.selected = true;
sel.appendChild(o);
}
sel.addEventListener("change", () => onChange({ type: "system", op: sel.value }));
body.appendChild(wrapField("Операция", sel));
} else if (t === "sleep") {
const inp = document.createElement("input");
inp.type = "number";
inp.value = step.ms ?? 500;
inp.addEventListener("input", () => onChange({ type: "sleep", ms: parseInt(inp.value, 10) || 0 }));
body.appendChild(wrapField("Миллисекунды", inp));
}
};
typeSel.addEventListener("change", () => {
const t = typeSel.value;
const fresh = ({
play_sound: { type: "play_sound", name: "ok" },
exe: { type: "exe", file: "" },
url: { type: "url", href: "" },
keys: { type: "keys", sequence: "" },
shell: { type: "shell", cmd: "" },
system: { type: "system", op: "volume_mute" },
sleep: { type: "sleep", ms: 500 },
})[t];
onChange(fresh);
step = fresh;
renderBody();
});
renderBody();
}
function wrapField(label, control) {
const w = document.createElement("div");
w.className = "field";
const l = document.createElement("label");
l.textContent = label;
w.appendChild(l);
w.appendChild(control);
return w;
}
function updateAction(skipRender) {
const t = state.actionType;
if (!t) return;
if (t === "exe") {
const file = document.getElementById("exe-file");
const delay = document.getElementById("exe-delay");
const wait = document.getElementById("exe-wait");
const a = { type: "exe", file: file ? file.value : "" };
if (wait && wait.checked) a.wait = true;
if (delay && delay.value) a.delay_ms = parseInt(delay.value, 10);
state.action = a;
} else if (t === "url") {
const a = { type: "url", href: (document.getElementById("url-href")||{}).value || "" };
const b = (document.getElementById("url-browser")||{}).value;
if (b) a.browser = b;
state.action = a;
} else if (t === "keys") {
const mode = (document.getElementById("keys-mode")||{}).value;
if (mode === "text") {
state.action = { type: "keys", text: (document.getElementById("keys-text")||{}).value || "" };
} else {
state.action = { type: "keys", sequence: (document.getElementById("keys-sequence")||{}).value || "" };
}
} else if (t === "play_sound") {
state.action = { type: "play_sound", name: (document.getElementById("snd-name")||{}).value || "ok" };
} else if (t === "system") {
state.action = { type: "system", op: (document.getElementById("sys-op")||{}).value || "volume_mute" };
} else if (t === "shell") {
const a = { type: "shell", cmd: (document.getElementById("shell-cmd")||{}).value || "" };
if ((document.getElementById("shell-wait")||{}).checked) a.wait = true;
state.action = a;
} else if (t === "sleep") {
state.action = { type: "sleep", ms: parseInt((document.getElementById("sleep-ms")||{}).value || "0", 10) };
}
}
function buildDraft() {
return {
cmd_id: state.cmdId,
phrases: state.phrases.map((p) => p.trim()).filter(Boolean),
action: state.action,
confirm_sound: state.confirmSound || undefined,
};
}
function renderPreview() {
const draft = buildDraft();
if (!draft.cmd_id || !draft.phrases.length || !draft.action) {
els.yamlPreview.textContent = "(черновик неполный)";
return;
}
const body = { phrases: draft.phrases, action: draft.action };
if (draft.confirm_sound) body.confirm_sound = draft.confirm_sound;
const obj = {};
obj[draft.cmd_id] = body;
try {
els.yamlPreview.textContent = jsyaml.dump(obj, { lineWidth: 4096, noRefs: true });
} catch (ex) {
els.yamlPreview.textContent = "Ошибка YAML: " + ex.message;
}
}
async function refreshExistingIds() {
const r = await callApi("list_existing_commands");
if (Array.isArray(r)) {
state.existingIds = r;
checkCmdIdCollision();
}
}
function bind() {
els.phrase0.addEventListener("input", () => {
state.phrases[0] = els.phrase0.value;
syncCmdId();
});
els.addPhrase.addEventListener("click", () => {
state.phrases.push("");
refreshExtraPhrases();
});
els.cmdId.addEventListener("input", () => {
state.cmdIdAuto = false;
state.cmdId = els.cmdId.value.trim();
checkCmdIdCollision();
});
els.actionGrid.addEventListener("click", (ev) => {
const card = ev.target.closest(".card");
if (!card) return;
const t = card.dataset.action;
els.actionGrid.querySelectorAll(".card").forEach((c) => c.classList.toggle("selected", c === card));
if (state.actionType !== t) {
state.actionType = t;
state.action = null;
}
});
els.toStep3.addEventListener("click", () => {
if (!state.actionType) {
showToast("Выберите тип действия", true);
return;
}
renderActionForm();
setStep(3);
});
document.querySelectorAll("[data-go]").forEach((b) => {
if (b.id === "to-step-3") return;
b.addEventListener("click", () => setStep(parseInt(b.dataset.go, 10)));
});
els.steps.forEach((s) => {
s.addEventListener("click", () => {
const target = parseInt(s.dataset.step, 10);
if (target === 3) renderActionForm();
setStep(target);
});
});
els.tabs.forEach((t) => {
t.addEventListener("click", () => {
els.tabs.forEach((x) => x.classList.toggle("active", x === t));
els.tabContents.forEach((c) => {
c.hidden = c.dataset.tabContent !== t.dataset.tab;
});
});
});
els.llmGenerate.addEventListener("click", async () => {
els.llmStatus.className = "row-hint";
els.llmStatus.textContent = "Запрос к модели...";
els.llmSuggestions.innerHTML = "";
const r = await callApi("llm_suggest", els.llmPrompt.value || "");
if (!r || !r.ok) {
els.llmStatus.className = "row-hint error";
els.llmStatus.textContent = (r && r.error) || "Не удалось получить ответ";
return;
}
els.llmStatus.className = "row-hint ok";
els.llmStatus.textContent = "Готово. Примите блок или отредактируйте вручную.";
renderLlmSuggestion(r.suggestion);
});
els.saveBtn.addEventListener("click", async () => {
const draft = buildDraft();
const v = await callApi("validate_draft", JSON.stringify(draft));
if (!v.ok) {
els.saveStatus.className = "row-hint error";
els.saveStatus.textContent = v.error;
return;
}
const overwrite = state.existingIds.includes(draft.cmd_id);
if (overwrite) {
if (!confirm(`Команда «${draft.cmd_id}» уже существует. Перезаписать?`)) return;
draft.overwrite = true;
}
const r = await callApi("save_command", JSON.stringify(draft));
if (!r.ok) {
els.saveStatus.className = "row-hint error";
els.saveStatus.textContent = r.error;
return;
}
els.saveStatus.className = "row-hint ok";
els.saveStatus.textContent = `Сохранено: ${r.id}`;
showToast(`Команда «${r.id}» сохранена`);
refreshExistingIds();
});
els.cancelBtn.addEventListener("click", () => {
if (!confirm("Сбросить черновик?")) return;
resetDraft();
});
document.addEventListener("keydown", (ev) => {
if (ev.key === "Escape") {
if (state.step > 1) setStep(state.step - 1);
return;
}
if (ev.key === "Enter" && !ev.shiftKey && !ev.ctrlKey && !ev.altKey) {
const tag = (ev.target && ev.target.tagName) || "";
if (tag === "TEXTAREA") return;
if (tag === "BUTTON") return;
if (state.step < 4) {
if (state.step === 2 && state.actionType) renderActionForm();
setStep(state.step + 1);
ev.preventDefault();
}
}
});
}
function renderLlmSuggestion(suggestion) {
const host = els.llmSuggestions;
host.innerHTML = "";
if (!suggestion) return;
const block = document.createElement("div");
block.className = "suggested-block";
const pre = document.createElement("pre");
try {
pre.textContent = jsyaml.dump(suggestion, { lineWidth: 4096, noRefs: true });
} catch (ex) {
pre.textContent = JSON.stringify(suggestion, null, 2);
}
block.appendChild(pre);
const accept = document.createElement("button");
accept.type = "button";
accept.className = "primary";
accept.textContent = "Принять";
accept.addEventListener("click", () => {
if (suggestion.phrases) state.phrases = suggestion.phrases.slice();
if (suggestion.cmd_id) {
state.cmdId = suggestion.cmd_id;
state.cmdIdAuto = false;
els.cmdId.value = state.cmdId;
} else {
syncCmdId();
}
els.phrase0.value = state.phrases[0] || "";
refreshExtraPhrases();
if (suggestion.action) {
state.actionType = suggestion.action.type;
state.action = suggestion.action;
els.actionGrid.querySelectorAll(".card").forEach((c) => {
c.classList.toggle("selected", c.dataset.action === state.actionType);
});
els.tabs.forEach((x) => x.classList.toggle("active", x.dataset.tab === "manual"));
els.tabContents.forEach((c) => { c.hidden = c.dataset.tabContent !== "manual"; });
renderActionForm();
}
setStep(3);
});
const discard = document.createElement("button");
discard.type = "button";
discard.className = "ghost";
discard.textContent = "Отклонить";
discard.addEventListener("click", () => { host.innerHTML = ""; });
const actions = document.createElement("div");
actions.className = "actions";
actions.appendChild(discard);
actions.appendChild(accept);
block.appendChild(actions);
host.appendChild(block);
}
function resetDraft() {
state.phrases = [""];
state.cmdId = "";
state.cmdIdAuto = true;
state.actionType = null;
state.action = null;
state.confirmSound = null;
els.phrase0.value = "";
els.cmdId.value = "";
els.cmdIdHint.textContent = "";
refreshExtraPhrases();
els.actionGrid.querySelectorAll(".card").forEach((c) => c.classList.remove("selected"));
els.paramsForm.innerHTML = "";
els.yamlPreview.textContent = "";
els.saveStatus.textContent = "";
setStep(1);
}
function init() {
bind();
refreshExtraPhrases();
setStep(1);
const tryReady = () => {
if (api()) {
refreshExistingIds();
} else {
setTimeout(tryReady, 80);
}
};
tryReady();
}
document.addEventListener("DOMContentLoaded", init);
})();

View file

@ -0,0 +1,106 @@
<!doctype html>
<html lang="ru">
<head>
<meta charset="utf-8">
<title>J.A.R.V.I.S — конструктор команд</title>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<div class="app">
<header class="topbar">
<h1>Конструктор команд</h1>
<div class="steps">
<button class="step" data-step="1" type="button"><span>1</span> Фразы</button>
<button class="step" data-step="2" type="button"><span>2</span> Действие</button>
<button class="step" data-step="3" type="button"><span>3</span> Параметры</button>
<button class="step" data-step="4" type="button"><span>4</span> Сохранить</button>
</div>
</header>
<main class="main">
<section class="panel" data-panel="1">
<h2>Голосовые фразы</h2>
<p class="hint">Минимум одна фраза. Идентификатор подставится автоматически из первой фразы.</p>
<div class="field">
<label for="phrase-0">Основная фраза</label>
<input type="text" id="phrase-0" data-phrase-input placeholder="например, открой документацию">
</div>
<div id="extra-phrases"></div>
<button type="button" class="ghost" id="add-phrase">+ синоним</button>
<div class="field">
<label for="cmd-id">Идентификатор команды</label>
<input type="text" id="cmd-id" placeholder="auto">
<div class="row-hint" id="cmd-id-hint"></div>
</div>
<div class="actions">
<button type="button" class="primary" data-go="2">Далее</button>
</div>
</section>
<section class="panel" data-panel="2" hidden>
<div class="tabs">
<button type="button" class="tab active" data-tab="manual">Вручную</button>
<button type="button" class="tab" data-tab="llm">Подсказать (LLM)</button>
</div>
<div class="tab-content" data-tab-content="manual">
<h2>Тип действия</h2>
<div class="grid" id="action-grid">
<button type="button" class="card" data-action="exe"><span class="card-title">Выполнить .exe</span><span class="card-sub">из custom-commands/</span></button>
<button type="button" class="card" data-action="url"><span class="card-title">Открыть сайт</span><span class="card-sub">через системный браузер</span></button>
<button type="button" class="card" data-action="keys"><span class="card-title">Нажать клавиши</span><span class="card-sub">или напечатать текст</span></button>
<button type="button" class="card" data-action="play_sound"><span class="card-title">Воспроизвести звук</span><span class="card-sub">из набора Jarvis</span></button>
<button type="button" class="card" data-action="system"><span class="card-title">Системное действие</span><span class="card-sub">звук, выход</span></button>
<button type="button" class="card" data-action="shell"><span class="card-title">PowerShell</span><span class="card-sub">любая shell-команда</span></button>
<button type="button" class="card" data-action="sleep"><span class="card-title">Пауза</span><span class="card-sub">только в составе</span></button>
<button type="button" class="card" data-action="multi"><span class="card-title">Последовательность</span><span class="card-sub">несколько шагов</span></button>
</div>
<div class="actions">
<button type="button" class="ghost" data-go="1">Назад</button>
<button type="button" class="primary" data-go="3" id="to-step-3">Далее</button>
</div>
</div>
<div class="tab-content" data-tab-content="llm" hidden>
<h2>Опишите команду на русском</h2>
<p class="hint">Модель предложит набор шагов. Их можно отредактировать перед сохранением.</p>
<textarea id="llm-prompt" rows="4" placeholder="например, открой ютуб и приглуши звук"></textarea>
<div class="actions">
<button type="button" class="primary" id="llm-generate">Сгенерировать</button>
</div>
<div id="llm-status" class="row-hint"></div>
<div id="llm-suggestions"></div>
</div>
</section>
<section class="panel" data-panel="3" hidden>
<h2>Параметры действия</h2>
<div id="params-form"></div>
<div class="actions">
<button type="button" class="ghost" data-go="2">Назад</button>
<button type="button" class="primary" data-go="4">Далее</button>
</div>
</section>
<section class="panel" data-panel="4" hidden>
<h2>Предпросмотр и сохранение</h2>
<div class="field">
<label>YAML</label>
<pre id="yaml-preview"></pre>
</div>
<div id="save-status" class="row-hint"></div>
<div class="actions">
<button type="button" class="ghost" data-go="3">Назад</button>
<button type="button" class="primary" id="save-btn">Сохранить</button>
<button type="button" class="ghost" id="cancel-btn">Сброс</button>
</div>
</section>
</main>
<div id="toast" class="toast" hidden></div>
</div>
<script src="vendor/js-yaml.min.js"></script>
<script src="app.js"></script>
</body>
</html>

View file

@ -0,0 +1,392 @@
:root {
--bg: #0a1214;
--bg-elev: #0f1a1d;
--bg-elev-2: #142327;
--border: #1d3036;
--border-strong: #294349;
--text: #d6e4e6;
--text-dim: #8aa1a5;
--accent: #52fefe;
--accent-dim: #2cb8b8;
--danger: #ff6b6b;
--ok: #6bf2a4;
--radius: 6px;
--radius-lg: 8px;
}
* { box-sizing: border-box; }
html, body {
margin: 0;
padding: 0;
background: var(--bg);
color: var(--text);
font-family: "Inter", "Segoe UI", system-ui, sans-serif;
font-size: 14px;
line-height: 1.45;
}
.app {
display: flex;
flex-direction: column;
min-height: 100vh;
}
.topbar {
padding: 18px 24px;
border-bottom: 1px solid var(--border);
background: var(--bg-elev);
}
.topbar h1 {
margin: 0 0 14px 0;
font-size: 17px;
font-weight: 600;
letter-spacing: 0.3px;
color: var(--accent);
}
.steps {
display: flex;
gap: 6px;
flex-wrap: wrap;
}
.step {
display: inline-flex;
align-items: center;
gap: 8px;
padding: 6px 12px;
border-radius: var(--radius);
background: var(--bg-elev-2);
border: 1px solid var(--border);
color: var(--text-dim);
font-size: 12px;
cursor: pointer;
font-family: inherit;
}
.step span {
width: 18px;
height: 18px;
border-radius: 50%;
background: var(--border);
color: var(--text);
display: inline-flex;
align-items: center;
justify-content: center;
font-size: 11px;
font-weight: 600;
}
.step.active {
border-color: var(--accent-dim);
color: var(--text);
}
.step.active span {
background: var(--accent);
color: #062023;
}
.main {
flex: 1;
padding: 22px 24px 32px;
max-width: 880px;
width: 100%;
margin: 0 auto;
}
.panel h2 {
margin: 0 0 6px 0;
font-size: 15px;
font-weight: 600;
}
.hint {
margin: 0 0 18px 0;
color: var(--text-dim);
font-size: 12.5px;
}
.field {
margin-bottom: 14px;
}
.field label {
display: block;
margin-bottom: 6px;
color: var(--text-dim);
font-size: 12px;
text-transform: uppercase;
letter-spacing: 0.5px;
}
input[type="text"],
input[type="number"],
textarea,
select {
width: 100%;
padding: 9px 11px;
background: var(--bg-elev);
border: 1px solid var(--border);
border-radius: var(--radius);
color: var(--text);
font-family: inherit;
font-size: 13.5px;
outline: none;
}
input[type="text"]:focus,
input[type="number"]:focus,
textarea:focus,
select:focus {
border-color: var(--accent-dim);
}
textarea {
resize: vertical;
}
.row-hint {
margin-top: 6px;
font-size: 11.5px;
color: var(--text-dim);
min-height: 14px;
}
.row-hint.error {
color: var(--danger);
}
.row-hint.ok {
color: var(--ok);
}
.actions {
margin-top: 20px;
display: flex;
gap: 10px;
justify-content: flex-end;
flex-wrap: wrap;
}
button {
cursor: pointer;
border: none;
border-radius: var(--radius);
padding: 9px 16px;
font-family: inherit;
font-size: 13px;
font-weight: 500;
transition: background 80ms ease, border-color 80ms ease;
}
.primary {
background: var(--accent);
color: #062023;
}
.primary:hover {
background: #7affff;
}
.primary:disabled {
background: var(--border);
color: var(--text-dim);
cursor: not-allowed;
}
.ghost {
background: transparent;
border: 1px solid var(--border-strong);
color: var(--text);
}
.ghost:hover {
border-color: var(--accent-dim);
}
#extra-phrases .extra-row {
display: flex;
gap: 6px;
margin-bottom: 8px;
}
#extra-phrases .extra-row input {
flex: 1;
}
#extra-phrases .extra-row button {
background: transparent;
border: 1px solid var(--border-strong);
color: var(--text-dim);
padding: 6px 10px;
font-size: 12px;
}
.tabs {
display: flex;
gap: 4px;
border-bottom: 1px solid var(--border);
margin-bottom: 18px;
}
.tab {
background: transparent;
border: none;
border-bottom: 2px solid transparent;
border-radius: 0;
padding: 8px 14px;
color: var(--text-dim);
font-size: 13px;
}
.tab.active {
color: var(--text);
border-bottom-color: var(--accent);
}
.grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(180px, 1fr));
gap: 10px;
}
.card {
display: flex;
flex-direction: column;
gap: 4px;
padding: 14px;
background: var(--bg-elev);
border: 1px solid var(--border);
border-radius: var(--radius-lg);
color: var(--text);
text-align: left;
min-height: 70px;
}
.card:hover {
border-color: var(--accent-dim);
}
.card.selected {
border-color: var(--accent);
background: var(--bg-elev-2);
}
.card-title {
font-weight: 600;
font-size: 13.5px;
}
.card-sub {
font-size: 11.5px;
color: var(--text-dim);
}
pre {
background: #06101294;
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 12px 14px;
overflow-x: auto;
font-family: "JetBrains Mono", "Cascadia Code", Consolas, monospace;
font-size: 12.5px;
color: #c4ecec;
white-space: pre-wrap;
}
.toast {
position: fixed;
right: 22px;
bottom: 22px;
padding: 10px 16px;
background: var(--bg-elev-2);
border: 1px solid var(--accent-dim);
border-radius: var(--radius);
color: var(--accent);
font-size: 13px;
box-shadow: 0 4px 14px rgba(0,0,0,0.45);
}
.toast.error {
border-color: var(--danger);
color: var(--danger);
}
.params-row {
display: flex;
gap: 10px;
align-items: flex-end;
flex-wrap: wrap;
}
.params-row .field {
flex: 1;
min-width: 200px;
margin-bottom: 0;
}
.checkbox {
display: inline-flex;
align-items: center;
gap: 7px;
color: var(--text-dim);
font-size: 12.5px;
margin-top: 6px;
cursor: pointer;
}
.checkbox input {
accent-color: var(--accent);
}
.steps-list {
display: flex;
flex-direction: column;
gap: 10px;
margin-bottom: 14px;
}
.step-card {
background: var(--bg-elev);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 12px;
}
.step-card .step-head {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 8px;
}
.step-card .step-head .step-label {
font-size: 12px;
color: var(--text-dim);
text-transform: uppercase;
letter-spacing: 0.4px;
}
.step-card .step-head .icon-btn {
background: transparent;
border: 1px solid var(--border-strong);
color: var(--text-dim);
padding: 3px 8px;
font-size: 11px;
}
.suggested-block {
background: var(--bg-elev);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 12px;
margin-bottom: 10px;
}
.suggested-block pre {
margin: 6px 0;
font-size: 12px;
}

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,91 @@
from __future__ import annotations
import io
import os
import shutil
from pathlib import Path
from typing import Optional
from ruamel.yaml import YAML
from .schema import CommandEntry, SchemaError, command_from_dict
DEFAULT_YAML_PATH = Path("C:/Jarvis/python/commands.yaml")
def _make_yaml() -> YAML:
y = YAML()
y.preserve_quotes = True
y.indent(mapping=2, sequence=4, offset=2)
y.width = 4096
return y
def load_yaml(path: Path = DEFAULT_YAML_PATH):
y = _make_yaml()
with open(path, "rt", encoding="utf-8") as f:
return y.load(f) or {}
def list_command_ids(path: Path = DEFAULT_YAML_PATH) -> list:
try:
data = load_yaml(path)
except FileNotFoundError:
return []
return list(data.keys())
def _dump_to_string(data) -> str:
y = _make_yaml()
buf = io.StringIO()
y.dump(data, buf)
return buf.getvalue()
def _atomic_write(path: Path, text: str) -> None:
tmp = path.with_suffix(path.suffix + ".tmp")
with open(tmp, "wt", encoding="utf-8", newline="\n") as f:
f.write(text)
os.replace(tmp, path)
def _backup(path: Path) -> Optional[Path]:
if not path.exists():
return None
bak = path.with_suffix(path.suffix + ".bak")
shutil.copy2(path, bak)
return bak
def append_command(entry: CommandEntry, path: Path = DEFAULT_YAML_PATH) -> None:
try:
data = load_yaml(path)
except FileNotFoundError:
data = {}
if entry.cmd_id in data:
raise SchemaError(f"command id already exists: {entry.cmd_id}")
data[entry.cmd_id] = entry.to_dict()
_backup(path)
text = _dump_to_string(data)
_atomic_write(path, text)
def upsert_command(entry: CommandEntry, path: Path = DEFAULT_YAML_PATH) -> None:
try:
data = load_yaml(path)
except FileNotFoundError:
data = {}
data[entry.cmd_id] = entry.to_dict()
_backup(path)
text = _dump_to_string(data)
_atomic_write(path, text)
def validate_file(path: Path = DEFAULT_YAML_PATH) -> list:
data = load_yaml(path)
out = []
for cmd_id, body in data.items():
plain_body = body
out.append(command_from_dict(cmd_id, dict(plain_body)))
return out

37
tts.py
View file

@ -1,8 +1,12 @@
import time
import numpy as np
import sounddevice as sd
import torch
import config
import effects
language = 'ru'
model_id = 'ru_v3'
sample_rate = 48000 # 48000
@ -19,18 +23,39 @@ model, _ = torch.hub.load(repo_or_dir='snakers4/silero-models',
model.to(device)
# воспроизводим
def va_speak(what: str):
def _post_process(audio):
if not getattr(config, 'TTS_EFFECTS_ENABLED', False):
return audio
arr = np.asarray(audio, dtype=np.float32)
low = getattr(config, 'TTS_BANDPASS_LOW_HZ', 0)
high = getattr(config, 'TTS_BANDPASS_HIGH_HZ', 0)
bandpass = (low, high) if low and high and high > low else None
wet = float(getattr(config, 'TTS_REVERB_WET', 0.0))
decay = int(getattr(config, 'TTS_REVERB_DECAY_MS', 0))
reverb = (wet, decay) if wet > 0 and decay > 0 else None
pitch = int(getattr(config, 'TTS_PITCH_SEMITONES', 0))
return effects.process(arr, sample_rate,
bandpass=bandpass,
reverb=reverb,
pitch_semitones=pitch)
def synthesize(what: str):
audio = model.apply_tts(text=what + "..",
speaker=speaker,
sample_rate=sample_rate,
put_accent=put_accent,
put_yo=put_yo)
return _post_process(audio)
def va_speak(what: str):
audio = synthesize(what)
sd.play(audio, sample_rate * 1.05)
time.sleep((len(audio) / sample_rate) + 0.5)
sd.stop()
# sd.play(audio, sample_rate)
# time.sleep(len(audio) / sample_rate)
# sd.stop()

55
utils/preview_effects.py Normal file
View file

@ -0,0 +1,55 @@
import os
import sys
import numpy as np
import soundfile as sf
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import config
import effects
import tts
PHRASE = "Приветствую, сэр. Все системы в норме."
OUT_RAW = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
'sound', '_preview_silero_raw.wav')
OUT_FX = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
'sound', '_preview_silero_fx.wav')
def _render_raw(text: str) -> np.ndarray:
audio = tts.model.apply_tts(text=text + "..",
speaker=tts.speaker,
sample_rate=tts.sample_rate,
put_accent=tts.put_accent,
put_yo=tts.put_yo)
return np.asarray(audio, dtype=np.float32)
def _render_fx(raw: np.ndarray) -> np.ndarray:
return effects.process(
raw, tts.sample_rate,
bandpass=(config.TTS_BANDPASS_LOW_HZ, config.TTS_BANDPASS_HIGH_HZ),
reverb=(config.TTS_REVERB_WET, config.TTS_REVERB_DECAY_MS),
pitch_semitones=config.TTS_PITCH_SEMITONES,
)
def main():
print(f"Synthesizing: {PHRASE!r}")
raw = _render_raw(PHRASE)
fx = _render_fx(raw)
sf.write(OUT_RAW, raw, tts.sample_rate, subtype='PCM_16')
sf.write(OUT_FX, fx, tts.sample_rate, subtype='PCM_16')
dur_raw = len(raw) / tts.sample_rate
dur_fx = len(fx) / tts.sample_rate
print(f"raw: {OUT_RAW} ({len(raw)} samples, {dur_raw:.2f}s)")
print(f"fx : {OUT_FX} ({len(fx)} samples, {dur_fx:.2f}s)")
print("Open both in a media player and A/B compare.")
if __name__ == '__main__':
main()