Release 1.8.4a1 - #196
Closed
github-actions[bot] wants to merge 143 commits into
Closed
Release 1.8.4a1#196github-actions[bot] wants to merge 143 commits into
github-actions[bot] wants to merge 143 commits into
Conversation
Added AI disclosure section to README.md outlining the use of AI tools in project development and the maintenance of public records.
Add a self-contained probe that drives a real HotWordEngine over a single clip the way the listening loop does: a few seconds of leading silence to warm the engine's rolling feature window (openWakeWord et al. only emit once it is full, ~2.5 s — too little lead silently drops short positives and biases false rejects), then the clip streamed frame by frame, returning a detection decision plus latency and frames-to-detection. Unlike MiniVoiceLoop it needs no bus or [listener] extra — just the [bench] extra (numpy). Tolerates the HotWordEngine(lang) signature and the vestigial found_wake_word(frame) arg. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat: per-clip WakeWordProbe for benchmark/test harnesses
…us-on-stop fix: restore SessionManager.bus when MiniCroft stops
End2EndTest.execute() and from_message() only stopped the MiniCroft on the success path, so a failing assertion left SessionManager.bus, default_session and Configuration patched for every later test. Both now run stop() from a finally block. MiniCroft snapshots the whole default Session at boot and restores it in stop(), so inject_active activations and wire-folded session values no longer outlive the test that made them. Mock-TTS unduck timers are tracked, made daemon and cancelled in stop(). An orphaned timer could otherwise emit onto a closed bus and fold a stale session onto the global SessionManager during a later test. CaptureSession resets its eof state atomically, records a timed_out flag, and returns a copy from finish(). A capture timeout now fails with a clear message instead of surfacing as a message-count mismatch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BusCoverageTracker snapshotted the session-cumulative global collector and added it into per-test counts, so every later test inherited the invocations of every earlier one. The snapshot is now a baseline and the report uses the delta over the tracker's own lifetime, frozen at start_tracking() so the tracking window is not counted twice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cmd_run booted a MiniCroft but never assigned it to the test, so execute() booted a second managed one and both patched the same globals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
match() treated mycroft.skill.handler.start as a failure signal, but it fires on a SUCCESSFUL match — so a successful match returned None. It also checked the failure flag before the captured message and spun a watcher thread that polled at 20Hz forever after a timeout. match_result() now returns a discriminated matched/no-match/timeout outcome and waits on the events directly. assert_no_match() fails on a timeout instead of passing vacuously; match() keeps its old signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The docstring told callers to emit after calling the helper, which is impossible single-threaded because the helper blocks. It now takes an optional emit= message and sends it once the handlers are in place. A match that raced an intent failure could also be dropped; appends are guarded by a lock and re-read once before giving up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The side effect inspected `mock.url` on a MagicMock, so no configured URL
ever matched and json() always returned {}. It now lives on the patched
GET, which receives the URL.
OCPTest also waits for ovos.common_play.query.response instead of sleeping
half the timeout, and stops the MiniCroft from a finally block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AudioServiceHarness.__exit__ skipped bus.close() when shutdown() raised. ListenerHarness and MiniListener left their wildcard "message" capture handler on the bus, so a shared bus kept feeding a dead harness. PlaybackServiceHarness now restores the TTS.queue object it replaced and refuses a second concurrent harness, because TTS.queue is process-wide class state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A load failure was warned about and skipped, then resurfaced much later as an unrelated assert_emitted timeout. Loading now raises by default; pass tolerate_load_errors=True to keep going, in which case the errors are kept in load_errors and quoted in assert_emitted failures. MiniPHAL.__exit__ also detaches its capture handler and closes the bus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cached MiniCrofts were never stopped and two could be live at once, each clobbering the same globals. At most one stays live now, and an atexit hook stops the rest. _wait_for_m2v_sync removes its three listeners in a finally block and only pays the 3.5s pad when no m2v activity was observed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feed_file ignored the join() result, so a listener thread that outlived its stop() kept appending to _messages during the next run. A still-alive thread is now logged and replaced with a fresh listener object. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`except (ImportError, Exception): pass` turned a malformed pyproject.toml into an understated coverage number. TOMLDecodeError and OSError are now caught explicitly and recorded in EcosystemCoverageReport.parse_errors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The client was left in place on a ConnectionError, so its reconnect thread lived for the rest of the process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One test per defect, each written to fail against the pre-fix code: teardown on the failure path, default-session isolation, TTS timer lifecycle, bus-coverage deltas, CaptureSession races, pipeline match verdicts, wait_for_match subscription order, the OCP HTTP mock, harness teardown, PHAL load errors, coverage parse errors and the RemoteRecorder connect leak. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raise-by-default made the old warn-and-skip expectation wrong; cover both the default raise and the tolerate_load_errors opt-out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI showed two gaps: the session restore bailed out when boot replaced the default-session singleton, leaking exactly the state it exists to scrub — restore now targets whatever object holds the role at stop() time. And on Python 3.10 there is no stdlib tomllib, so a malformed pyproject.toml was silently ignored — depend on the tomli backport there. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ovos-bus-client 1.x has serialize/deserialize, 2.x to_dict/from_dict; the snapshot silently became None on 1.x and the restore no-opped. Support both and warn instead of failing silently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix: Han audit round 1 — teardown safety, session isolation, harness lifecycle
…n available cmd_validate always ran _basic_validate, contradicting docs/cli.md which documented it as preferring pydantic_helpers.validate_fixture when the pydantic extra is importable. Match the code to the documented behaviour instead of weakening the docs, and add tests covering both the pydantic-available and fallback paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mirror (#173) The default-session class attribute lives on ovos-spec-tools' SessionManager (bus-client grafts it in); ovos-spec-tools 1.10.5a2 dropped that mirror, so SessionManager.sessions["default"] (via get_default_session()) is now the only source of truth. Every MiniCroft boot raised AttributeError: type object 'SessionManager' has no attribute 'default_session'. Converted every SessionManager.default_session attribute read/write in get_minicroft()/MiniCroft.run()/MiniCroft.stop()/_restore_default_session() to SessionManager.get_default_session(), and did the same in the four existing unittests that referenced the attribute directly. Added a regression test that deletes the class attribute (when present) before booting and tearing down a MiniCroft. No floor pin bump: the accessor conversion is version-agnostic (works whether or not the attribute mirror exists), and a spec-tools mirror shim is landing separately. Fail-before: reverting only ovoscope/__init__.py made the new test fail with "AttributeError: type object 'SessionManager' has no attribute 'default_session'. Did you mean: 'get_default_session'?"; restoring the fix made it pass. Full suite: 671 passed, 10 skipped (baseline 670 passed/10 skipped plus the new regression test). Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Adds ovoscope.golden: loads a skill's golden-utterance corpus and runs
it through the real padatious/padacioso/nebulento OPM pipeline plugins
(register_intent/calc_intent/match_<tier>, no bus/MiniCroft boot) plus
an informational Model2Vec zero-shot fighter, producing a CI-gating
scoreboard and ovos-plugin-arena-shaped prediction rows.
_extract_name_conf branched on hasattr(result, "get") to pick a dict-style
reader, but MatchData/PadaciosoIntent/NebulentoIntent all expose a .get()
that is a slot accessor over matched entities, not name/conf — every
published confidence silently came back 0.0. Always read via getattr, and
populate PredictionRow.predicted_slots from the same matched-entity dict
instead of hardcoding None.
build_engine_intents never registered entities, so a template's {name}
slot reached the plugin as an unconstrained wildcard instead of the
closed set the shipping pipeline builds from a skill's .entity files.
Add build_engine_entities() and thread entities through
EngineAdapter.build()/run_golden_suite() so the runner registers the
same entities the real pipeline does.
DEFAULT_ENGINES adapter instances were mutated in place (m2v threshold,
_cls/_probe caches) whenever run_golden_suite() was called with its
default engines=None, permanently reconfiguring the process-wide
defaults for every later call; build fresh adapters per call instead.
GenericOPMAdapter.available() also discarded the plugin instance it
built just to check for an init error; cache it and let the first
build() reuse it. build()'s bare except around train() swallowed a
real training failure and reported it later as an all-unmatched gate
failure blamed on the skill; let it surface. Drop the unused
FighterSpec.gate field (aggregate threshold is hardcoded elsewhere).
CI installed extras never included "engines", so nebulento/ovos-m2v-
pipeline/ovos-spec-tools were absent and several golden.py tests failed
on every Python version; both build-tests.yml and coverage.yml now
install a "test" extra that covers every dependency test/unittests/
actually imports (including engines), so ".[test]" resolves the way
the docs/reviews already assumed it did. dev depends on it too, instead
of duplicating the list.
The write_predictions() m2v-exclusion test used a deliberately
unavailable m2v adapter, so it produced zero m2v rows regardless of
whether the exclusion in write_predictions() existed — deleting that
exclusion left the test green. Swap in a stub adapter that IS available
and produces rows, so the mutation is now caught. The gating-engine
tests that used the default engine set also reached the real
M2VPrototypeAdapter and hit the HF hub for potion-base-8M; give them an
offline m2v override like the other tests in the file, and stop them
(and every other test) sharing DEFAULT_ENGINES' own adapter instances —
a shallow dict() copy still shares the objects — in favour of building
fresh ones per test the same way run_golden_suite(engines=None) does.
Add regression tests for each of the above: a real (>=0.8) confidence
on a matched row, an entity-registration effect on slot scoring, the
m2v-exclusion mutation actually going red, and DEFAULT_ENGINES surviving
a run with its own state (threshold, resolved plugin class) untouched.
get_minicroft waited for mycroft.skills.trained whenever a loaded skill registered an intent. That reply is a private padatious-family signal, not a spec topic: the m2v and adapt pipelines never subscribe to mycroft.skills.train and never emit it, so a boot without a padatious-family plugin could only ever time out and raise. Enter the wait only when a mycroft.skills.train subscriber is on the bus; otherwise the croft is ready at READY. Fixes #179 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…#182) get_m2v_minicroft boots a MiniCroft whose intent stages are the model2vec classifier for the labels the model carries and model2vec prototype mode, built from the skills' shipped .intent files, for every other label. The prototype stage deny-lists the model's own labels, read from the model config, so each label has exactly one engine, and it runs ahead of the classifier at every tier because the classifier is confidently wrong on labels it never saw while prototype mode cannot fire on the labels it is denied. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
E2EPipelineHarness.setUp() (via detach_skill), register_padatious_intent,
register_padatious_entity, register_adapt_vocab, register_adapt_intent, and
detach_intent all emitted registration/deregistration/detach messages with
no context["skill_id"]. OVOS-INTENT-4 §3.1 requires every Message a skill
originates to carry context["skill_id"], and §3.2 requires it to equal the
payload skill_id for registration/deregistration traffic; a conformant
plugin (padatious#146, adapt#70, m2v#122, core#956) that enforces this drops
the harness's context-less messages, breaking test isolation between cases.
detach_skill already receives skill_id as an argument and now stamps it on
context unconditionally. The other five helpers gained a new *required*
keyword-only skill_id argument that is always stamped on context — there is
no derivation fallback from the name/entity_type/builder argument, because
Adapt vocab and intent names are conventionally unscoped ("Fruit",
"greeting", "R2TestIntent"): guessing a skill_id from a colon that may not
be there would silently register vocab under the wrong owner (or no owner),
which is the leak this fix exists to close. Every call site in ovoscope's
own tests and in docs/e2e-pipeline-harness.md now passes skill_id=
explicitly. Payload ("data") shapes are unchanged.
BREAKING CHANGE: register_padatious_intent, register_padatious_entity,
register_adapt_vocab, register_adapt_intent, and detach_intent now require
a skill_id keyword-only argument; calls omitting it raise TypeError.
Fail-before: reverting the ovoscope/e2e.py hunk while keeping the new tests
made all "requires_skill_id" assertions fail (no TypeError was raised, or
context["skill_id"] was absent); restoring the hunk turned them green.
Full suite: 698 passed, 25 skipped, 0 failed.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…#190) The e2e registration helpers took skill_id as a keyword-only argument with no default, which broke every existing caller the moment it shipped. Eight repositories are red on that alone, found one CI job at a time. The argument stays required in every new call, and every shim still stamps Message.context['skill_id'] so a conformant plugin accepts the registration. A caller that omits it now gets the unattributed message it asked for, plus one warning naming the release the argument becomes required in, instead of a TypeError out of setUpClass.
… say (#189) `mycroft.skills.trained` is emitted before the container it reports has finished compiling: padatious never trains on the calling thread, so the event fires while `needs_compile` is still true. A consumer that waits on the event and then matches an utterance races the compile, and loses under load. The wait now reads the trainer. A plugin offering `wait_until_trained` is asked directly, and `container.needs_compile` decides whether work is outstanding, so the deadline is pushed forward while a trainer still has work rather than expiring on a quiet bus. The wait is skipped when it cannot be needed: no skill registered an intent, or nothing on the bus trains one. The first of those is not merely an optimisation. A fresh container reports `needs_compile` before anything is registered in it, so a boot with no skills at all would otherwise wait for a compile nobody asked for -- measured at 4.08s against dev's 0.59s on this host, and the whole of ovoscope's own suite at 117s against dev's 113s once the skip is in place. `docs/minicroft.md` stated a 300-second pytest-timeout, which is below the wait's own worst case. That ceiling is `OVOSCOPE_TRAINED_MAX` plus `OVOSCOPE_TRAINED_TIMEOUT`, 780 seconds at the shipped defaults, and a framework timeout below it kills setup mid-wait and reads as a boot failure -- exactly the misdiagnosis that section exists to prevent. It now names the sum and asks for 900. Co-authored-by: Claude Opus 5 (claude-opus-5) <noreply@anthropic.com>
DEFAULT_KEEP_SRC named only the legacy fallback ping, so under the canonical spelling the membership test missed and source/destination fell through to the rolling comparison. The assertion still ran and still passed, against a different rule. both_spellings() expands any topic to its migration counterpart and is applied to caller-supplied lists too, which had the same silent flip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human review requested! This supersedes any earlier open release proposal for this repository.