feat: vision + macros + scheduler in Python (129 → 142 commands)

Three big rust packs ported. Python parity now covers the FULL imba lineup
except codebase_qa and github_pr.

vision_handler.py (new, ~125 lines)
  - Screenshot via PowerShell System.Drawing → temp PNG → base64 → Groq
    vision API (llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
  - do_vision_describe   "что на экране" / "опиши экран"
  - do_vision_read_error "прочитай ошибку"
  - Requires GROQ_TOKEN; speaks "Не удалось" on missing.

macros_store.py (new, ~170 lines)
  - JSON store at <here>/macros.json (atomic write-through).
  - start(name) / save() / cancel() / replay(name, dispatch_fn)
  - is_recording / recording_name / list_names / get / delete
  - record_step(phrase) called by execute_cmd hook (see main.py)
  - is_macro_control filter blocks "запиши макрос ..." from recording itself
    (no infinite recursion).
  - replay spawns daemon thread, fires each step with 800ms delay.

scheduler_store.py (new, ~220 lines)
  - JSON store at <here>/schedule.json.
  - parse_schedule(spec) handles "daily HH:MM", "at HH:MM",
    "every/in N minutes|hours|seconds" — Russian (минут/час/секунд) + English.
  - Background daemon thread ticks every 30s, fires Speak actions through the
    same speak callback used by main.py.
  - Once-tasks auto-delete after firing; daily/interval cycle forever.
  - add/remove/clear/remove_by_text/list_all.

extensions.py (+13 handlers)
  do_vision_describe / do_vision_read_error → thin proxies to vision_handler
  do_macros_start / save / cancel / replay / list / delete
  do_scheduler_add_reminder / add_recurring / list / clear / cancel_by_text

main.py
  - extensions.init_background_services() at startup → scheduler thread boots.
  - Wraps execute_cmd so every successful dispatch feeds macros_store.record_step.
  - set_dispatch_fn(_execute_phrase_for_macro) enables macro replay through the
    normal command pipeline (recognize_cmd → execute_cmd).

commands.yaml (+13 entries, 129 → 142)
  vision_describe, vision_read_error, macros_start, macros_save, macros_cancel,
  macros_replay, macros_list, macros_delete, scheduler_reminder,
  scheduler_recurring, scheduler_list, scheduler_clear, scheduler_cancel_text.

Tests: ast.parse passes for all five .py files. yaml.safe_load = 142 entries.

Python parity status vs rust (72 packs):
  ✓ memory_pack, profile_switch, vision, scheduler, macros, llm switch
    (via env), llm_context (basic), media_keys (existing), screenshot, net_info,
    unit_convert, generators, disk, date_math, sleep_timer (via shutdown),
    interesting_fact, mailto, self_check, ssl_check, intro/echo
  ✗ codebase_qa, github_pr (require gh CLI + file walk — TODO)
  ✗ pomodoro / daily_briefing / habit_nudge / quick_search / diagnostics
    (could be ported but need scheduler integration + LLM glue)
This commit is contained in:
Bossiara13 2026-05-16 00:00:22 +03:00
parent 02c6f2f90d
commit 6a328077bc
6 changed files with 1015 additions and 0 deletions

34
main.py
View file

@ -248,6 +248,7 @@ def _set_clipboard(text: str):
# Wire extensions module to our voice + clipboard helpers
extensions.set_speak(_speak)
extensions.set_clipboard_fn(_set_clipboard)
extensions.init_background_services() # boots scheduler thread
_CODEGEN_TRIGGERS = (
@ -1615,6 +1616,39 @@ print(f" J.A.R.V.I.S. v{config.VA_VER} [Python edition]")
print(f" Total commands: {len(VA_CMD_LIST)}")
print("=" * 60)
# Hook macros to the dispatch path: registers a callback so macros::replay
# can re-fire commands, and so the recorder can capture successful phrases.
def _execute_phrase_for_macro(phrase: str):
"""Replay step: re-fire a phrase through the normal dispatch path."""
try:
cmd = recognize_cmd(phrase)
except Exception as exc:
print(f"[macros] replay recognize: {exc}")
return
if cmd:
execute_cmd(cmd, phrase)
extensions.set_dispatch_fn(_execute_phrase_for_macro)
# Wrap execute_cmd to also feed the macro recorder on success.
_orig_execute_cmd = execute_cmd
def execute_cmd(cmd: str, voice: str): # noqa: F811
spec = VA_CMD_LIST.get(cmd)
_orig_execute_cmd(cmd, voice)
if spec:
# On every successful dispatch, feed the recorder. Filter inside
# macros_store.record_step blocks macro-control phrases.
try:
import macros_store
macros_store.record_step(voice)
except Exception as exc:
print(f"[macros] record_step: {exc}")
recorder = PvRecorder(device_index=config.MICROPHONE_INDEX, frame_length=512)
recorder.start()
print('Using device: %s' % recorder.selected_device)