v3.22.0: the test suites (scripts grouped, ordered, with setup/teardown)
A suite is the root object: every script belongs to exactly one suite, the
suite holds the ordered member list by reference - the optional setup first,
the members, the optional teardown last - and runs them in one card session,
without resets and without resume.
- test_suites.py: the suite store (test_suites.json, --test-suites), validated
and atomic like the script store: roles normalised (setup first, teardown
last, at most one of each, a script appears once per suite), per-member
on_fail stop/continue and the suite-level require_adm.
- test_scripts.py schema v2: suite_id/require_adm; list_by_suite()/orphans();
the import reports the id_map a bundle import remaps with. v3.21.0 scripts
without a suite are attached once to an auto-created "Imported scripts"
suite (migrate_scripts_to_suites, called from __main__).
- API: /api/test/suites (+ update|delete|import - delete refused while
non-empty; the bundle import imports the scripts first, remaps the suite
references and skips unresolvable ones); the scripts endpoints gain
suite_id/require_adm plus move|copy (delete detaches from the suite first;
a suite update refuses a membership change).
- runner: /api/test/run {suite_id} resolves the members and the preset, runs
the ADM gate, then setup -> members (per-member on_fail) -> teardown. The
teardown always runs on a stop or a failure (skipped only when the card is
gone; it clears the stop flag for its duration); a card reset stops the
suite, skips the remaining members and is flagged. Members' live steps land
in the shared snapshot, their records (status/steps/note/log excerpt/
counters) in _TEST_RUN['suite']; the summary carries the counts, wall time,
preset, the SCP80 counters before/after per keyset and the session flags.
_test_log buffers the semantic lines (bounded) for the report's per-member
excerpts.
- ADM gate (_ensure_adm): a suite/script require_adm verifies the matched
preset's key once per card session (the /api/verify-adm badge sets the same
latch); a failure refuses the start, stamps the session (no automatic
retry) and needs a manual verify.
- PWA: the Test script pill becomes four views - the suite list (default),
the suite editor (the script list: role badges, on_fail, up/down, per-script
Run/Edit/Move/Copy/Delete, Add script with the templates), the script editor
(the step list; a single run shows its result under the steps and is not
preserved) and the suite report (per-member blocks with the step table,
notes and run log; summary; export JSON/Markdown). Suite bundles export/
import; the legacy script export imports into "Imported scripts". EN/RU.
- tests: the suite store/migration, the runner orchestration (order, on_fail,
setup failure, stop, card reset, teardown, per-member logs) and the ADM gate
(Python); the suite HTTP paths; the PWA report/Markdown/wiring (Node).
Tests: 833 frontend + 794 Python; sw cache simple-v397.
This commit is contained in:
+20
-1
@@ -12,12 +12,31 @@ localStorage export imports as-is), and any API client can create or run them
|
||||
(`POST /api/test/run` with an inline `script` or a stored `script_id` - see
|
||||
`docs/api.md`). While a run is active the card belongs to the script.
|
||||
|
||||
**Test suites** (v3.22.0) group the scripts: a suite is the root object (every
|
||||
script belongs to exactly one suite, `suite_id` in the script store) and the
|
||||
runner executes its scripts in order - the optional **setup** first, the
|
||||
**member** scripts, the optional **teardown** last - in **one card session,
|
||||
without resets and without resume**. Per member `on_fail` decides whether a
|
||||
failure stops the suite (`stop`, default) or the next script runs
|
||||
(`continue`); the **teardown always runs** on a stop or a failure (its job is
|
||||
to leave the card ready for a new test) as long as the card is in the reader,
|
||||
and a card reset (session change) stops the suite and is flagged in the
|
||||
report. A suite (or a single script) may require the **ADM** verified: the
|
||||
matched preset's key is tried **once** at the start - a failure refuses the
|
||||
run and needs a manual verify (the ADM badge) before the next attempt. The
|
||||
suite run's report (the PWA's report view, exportable as JSON or Markdown)
|
||||
carries every member's step table, failing notes and run-log excerpt, plus the
|
||||
summary (pass/warn/fail/skipped, wall time, the SCP80 counters before/after
|
||||
per keyset and the card-session flag). The suites live in
|
||||
`~/.pysim-simple-server/test_suites.json` (`--test-suites`).
|
||||
|
||||
**Format**: server ≥ 3.15.0 (`kvn`, `por` and `files` checks), scripts stored
|
||||
server-side since 3.16.0, `menu-select` by text and the spec-order TERMINAL
|
||||
RESPONSE codings since 3.17.0, the semantic `event` action since 3.18.0, the
|
||||
`envelope` action's `src` override since 3.18.1, the inline `por` object, the
|
||||
`alpha`/`sms` checks, `scp80.params.format` and the case-insensitive
|
||||
`menu-select` text match since 3.19.0.
|
||||
`menu-select` text match since 3.19.0, the `proactive` cleanup action and the
|
||||
ok-answering bounded cleanup since 3.21.0, the suites since 3.22.0.
|
||||
|
||||
## Writing a test for an applet
|
||||
|
||||
|
||||
Reference in New Issue
Block a user