feat: vision + macros + scheduler in Python (129 → 142 commands)
Three big rust packs ported. Python parity now covers the FULL imba lineup
except codebase_qa and github_pr.
vision_handler.py (new, ~125 lines)
- Screenshot via PowerShell System.Drawing → temp PNG → base64 → Groq
vision API (llama-3.2-11b-vision-preview, override via GROQ_VISION_MODEL).
- do_vision_describe "что на экране" / "опиши экран"
- do_vision_read_error "прочитай ошибку"
- Requires GROQ_TOKEN; speaks "Не удалось" on missing.
macros_store.py (new, ~170 lines)
- JSON store at <here>/macros.json (atomic write-through).
- start(name) / save() / cancel() / replay(name, dispatch_fn)
- is_recording / recording_name / list_names / get / delete
- record_step(phrase) called by execute_cmd hook (see main.py)
- is_macro_control filter blocks "запиши макрос ..." from recording itself
(no infinite recursion).
- replay spawns daemon thread, fires each step with 800ms delay.
scheduler_store.py (new, ~220 lines)
- JSON store at <here>/schedule.json.
- parse_schedule(spec) handles "daily HH:MM", "at HH:MM",
"every/in N minutes|hours|seconds" — Russian (минут/час/секунд) + English.
- Background daemon thread ticks every 30s, fires Speak actions through the
same speak callback used by main.py.
- Once-tasks auto-delete after firing; daily/interval cycle forever.
- add/remove/clear/remove_by_text/list_all.
extensions.py (+13 handlers)
do_vision_describe / do_vision_read_error → thin proxies to vision_handler
do_macros_start / save / cancel / replay / list / delete
do_scheduler_add_reminder / add_recurring / list / clear / cancel_by_text
main.py
- extensions.init_background_services() at startup → scheduler thread boots.
- Wraps execute_cmd so every successful dispatch feeds macros_store.record_step.
- set_dispatch_fn(_execute_phrase_for_macro) enables macro replay through the
normal command pipeline (recognize_cmd → execute_cmd).
commands.yaml (+13 entries, 129 → 142)
vision_describe, vision_read_error, macros_start, macros_save, macros_cancel,
macros_replay, macros_list, macros_delete, scheduler_reminder,
scheduler_recurring, scheduler_list, scheduler_clear, scheduler_cancel_text.
Tests: ast.parse passes for all five .py files. yaml.safe_load = 142 entries.
Python parity status vs rust (72 packs):
✓ memory_pack, profile_switch, vision, scheduler, macros, llm switch
(via env), llm_context (basic), media_keys (existing), screenshot, net_info,
unit_convert, generators, disk, date_math, sleep_timer (via shutdown),
interesting_fact, mailto, self_check, ssl_check, intro/echo
✗ codebase_qa, github_pr (require gh CLI + file walk — TODO)
✗ pomodoro / daily_briefing / habit_nudge / quick_search / diagnostics
(could be ported but need scheduler integration + LLM glue)
This commit is contained in:
parent
02c6f2f90d
commit
6a328077bc
6 changed files with 1015 additions and 0 deletions
34
main.py
34
main.py
|
|
@ -248,6 +248,7 @@ def _set_clipboard(text: str):
|
|||
# Wire extensions module to our voice + clipboard helpers
|
||||
extensions.set_speak(_speak)
|
||||
extensions.set_clipboard_fn(_set_clipboard)
|
||||
extensions.init_background_services() # boots scheduler thread
|
||||
|
||||
|
||||
_CODEGEN_TRIGGERS = (
|
||||
|
|
@ -1615,6 +1616,39 @@ print(f" J.A.R.V.I.S. v{config.VA_VER} [Python edition]")
|
|||
print(f" Total commands: {len(VA_CMD_LIST)}")
|
||||
print("=" * 60)
|
||||
|
||||
|
||||
# Hook macros to the dispatch path: registers a callback so macros::replay
|
||||
# can re-fire commands, and so the recorder can capture successful phrases.
|
||||
def _execute_phrase_for_macro(phrase: str):
|
||||
"""Replay step: re-fire a phrase through the normal dispatch path."""
|
||||
try:
|
||||
cmd = recognize_cmd(phrase)
|
||||
except Exception as exc:
|
||||
print(f"[macros] replay recognize: {exc}")
|
||||
return
|
||||
if cmd:
|
||||
execute_cmd(cmd, phrase)
|
||||
|
||||
|
||||
extensions.set_dispatch_fn(_execute_phrase_for_macro)
|
||||
|
||||
|
||||
# Wrap execute_cmd to also feed the macro recorder on success.
|
||||
_orig_execute_cmd = execute_cmd
|
||||
|
||||
|
||||
def execute_cmd(cmd: str, voice: str): # noqa: F811
|
||||
spec = VA_CMD_LIST.get(cmd)
|
||||
_orig_execute_cmd(cmd, voice)
|
||||
if spec:
|
||||
# On every successful dispatch, feed the recorder. Filter inside
|
||||
# macros_store.record_step blocks macro-control phrases.
|
||||
try:
|
||||
import macros_store
|
||||
macros_store.record_step(voice)
|
||||
except Exception as exc:
|
||||
print(f"[macros] record_step: {exc}")
|
||||
|
||||
recorder = PvRecorder(device_index=config.MICROPHONE_INDEX, frame_length=512)
|
||||
recorder.start()
|
||||
print('Using device: %s' % recorder.selected_device)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue