v3.23.0: the start-with-reset suite option
A suite may carry `start_reset` (store schema v2, default false; the editor's
"Start with card reset" checkbox): before the run, the server physically
resets the card and runs the full re-initialization - `_card_reset_reinit`
(extracted from the eSIM profile-switch reinit) does reset_card() + equip +
_apply_equipped_card + the shell registration probe.
The reset runs in _test_suite_run_request before the preset/ADM checks:
- the run's preset must still match the post-reset ICCID (a mismatch refuses
the start with a re-select hint),
- a require_adm suite re-verifies the ADM for the new card session (the
latch is per session).
The suite report carries start_reset + reset {ok, iccid, ms}; the PWA shows
"started with a card reset" in the report header and a RESET badge in the
suite list. Mid-run resets stay fatal.
Documented caveat: the init drain consumes the card's initial proactive
commands, so a script cannot catch the first command after the reset (they
stay in the proactive log).
Docs: test-scripts.md, api.md, help EN/RU, repo AGENTS.
Tests: 843 frontend + 827 Python; sw cache simple-v413.
This commit is contained in:
+27
-13
@@ -60,7 +60,7 @@ a 3.x PWA).
|
||||
| `/api/test/scripts/copy` | POST | Copy a script into another suite (`{id, suite_id, name?}`) |
|
||||
| `/api/test/scripts/import` | POST | Import scripts (`{scripts: [...], mode: merge\|replace, suite_id?}`; accepts the old localStorage export shape) |
|
||||
| `/api/test/suites` | GET | Test suite store (server-side `test_suites.json`): all suites + the store path |
|
||||
| `/api/test/suites` | POST | Create a suite (`{name, require_adm?, scripts?}` or `{suite}`) |
|
||||
| `/api/test/suites` | POST | Create a suite (`{name, require_adm?, start_reset?, scripts?}` or `{suite}`) |
|
||||
| `/api/test/suites/update` | POST | Update a suite (`{id, suite}`; the member set changes through the script endpoints) |
|
||||
| `/api/test/suites/delete` | POST | Delete a suite (`{id}`; refused while it holds scripts) |
|
||||
| `/api/test/suites/import` | POST | Import a suite bundle (`{suites, scripts, mode}`; script references remapped) |
|
||||
@@ -839,25 +839,31 @@ leave dangling references; delete those scripts first, or import with
|
||||
Test suites live next to the scripts (`~/.pysim-simple-server/test_suites.json`,
|
||||
`--test-suites PATH`): a suite is the ordered list of its scripts - the
|
||||
optional `setup` first, the `member` scripts, the optional `teardown` last -
|
||||
plus the per-member `on_fail` policy and the suite-level `require_adm`.
|
||||
plus the per-member `on_fail` policy, the suite-level `require_adm` and the
|
||||
optional **`start_reset`** (reset + fully re-initialize the card before the
|
||||
run - see `/api/test/run`).
|
||||
|
||||
```json
|
||||
{"path": "/home/user/.pysim-simple-server/test_suites.json", "count": 1, "version": 1,
|
||||
{"path": "/home/user/.pysim-simple-server/test_suites.json", "count": 1, "version": 2,
|
||||
"suites": [{"id": "…", "name": "alfa regression", "require_adm": true,
|
||||
"start_reset": false,
|
||||
"scripts": [{"script_id": "…", "role": "setup", "on_fail": "stop"},
|
||||
{"script_id": "…", "role": "member", "on_fail": "stop"},
|
||||
{"script_id": "…", "role": "teardown", "on_fail": "stop"}],
|
||||
"created": 1690000000.0, "updated": 1690000000.0}]}
|
||||
```
|
||||
|
||||
**Schema v2 (v3.23.0)** adds `start_reset` (default `false`); a v1 file loads
|
||||
with it `false`.
|
||||
|
||||
The store normalises the order (setup first, teardown last, at most one of
|
||||
each; a script appears once per suite) and validates `on_fail` as
|
||||
`stop` (default) or `continue` - "which failures cascade" is a visible
|
||||
per-suite decision.
|
||||
|
||||
`POST /api/test/suites` creates one (`{name, require_adm?, scripts?}` or
|
||||
`{suite: {...}}`), `POST /api/test/suites/update` replaces
|
||||
name/require_adm/scripts (`{id, suite}`; the **member set** changes through
|
||||
`POST /api/test/suites` creates one (`{name, require_adm?, start_reset?,
|
||||
scripts?}` or `{suite: {...}}`), `POST /api/test/suites/update` replaces
|
||||
name/require_adm/start_reset/scripts (`{id, suite}`; the **member set** changes through
|
||||
the script endpoints, not here), `POST /api/test/suites/delete` (`{id}`)
|
||||
removes it - refused while the suite still holds scripts - and
|
||||
`POST /api/test/suites/import` takes
|
||||
@@ -884,18 +890,26 @@ idempotent. Restore a bundle into a fresh store, or wipe the target first
|
||||
|
||||
`{script, script_id?, preset_id?, require_adm?}` runs one script (the PWA
|
||||
sends the edited copy; the server validates and runs it). `{suite_id,
|
||||
preset_id?}` runs a suite: its scripts in order, one card session, no resume
|
||||
- the ADM prerequisite first (suite- or member-level `require_adm`: the
|
||||
preset_id?}` runs a suite: its scripts in order, one card session, no resume.
|
||||
With `start_reset` the card is **physically reset and fully re-initialized**
|
||||
first (ICCID, network state, TERMINAL PROFILE, the equip drain); the run's
|
||||
preset must still match the card's ICCID afterwards (a mismatch refuses the
|
||||
start) and a required ADM is verified against the **new** card session. Then
|
||||
the ADM prerequisite (suite- or member-level `require_adm`: the
|
||||
matched preset's key is verified **once**; a failure refuses the start, is
|
||||
not retried automatically, and requires a manual verify), then setup,
|
||||
members (per-member `on_fail`), teardown. **The teardown runs on every stop**
|
||||
as long as the card is in the reader; a card reset (session change) stops the
|
||||
suite, skips the remaining members and is flagged in the report.
|
||||
not retried automatically, and requires a manual verify), setup,
|
||||
members (per-member `on_fail`), teardown. The initialization consumes the
|
||||
card's initial proactive commands (they stay in the proactive log), so a
|
||||
script cannot catch the first command after the reset. **The teardown runs
|
||||
on every stop** as long as the card is in the reader; a card reset (session
|
||||
change) stops the suite, skips the remaining members and is flagged in the
|
||||
report.
|
||||
|
||||
The suite run's snapshot adds `suite`: the member records (`status`, `steps`,
|
||||
`note`, `log` - the semantic run-log excerpt), the `summary` (ok/warning/
|
||||
error/skipped counts, `wall_ms`, `stopped`, `session_changed`, the SCP80
|
||||
`counters_before`/`counters_after` per keyset) and the run's `adm` state.
|
||||
`counters_before`/`counters_after` per keyset), the `start_reset` flag and
|
||||
the `reset` result (ICCID + ms) and the run's `adm` state.
|
||||
The report is exported from the PWA as JSON or Markdown; the run state lives
|
||||
in the server until `POST /api/test/clear` (no history).
|
||||
|
||||
|
||||
+15
-1
@@ -33,6 +33,19 @@ import always merges: importing a suite export **into the store it came
|
||||
from duplicates the suite and its scripts** (an id already present gets a
|
||||
fresh uuid) - restore into a fresh store, or wipe the target first.
|
||||
|
||||
A suite may **start with a card reset** (`start_reset`, the suite editor's
|
||||
*Start with card reset*, v3.23.0): before anything runs, the server
|
||||
physically resets the card and performs the full re-initialization (ICCID,
|
||||
network state, TERMINAL PROFILE, the equip drain), so the scripts run against
|
||||
a freshly initialized card. The run's preset must still match the card's
|
||||
ICCID after the reset (a mismatch refuses the start - re-select the preset
|
||||
and retry), and a `require_adm` suite verifies the ADM again for the new card
|
||||
session. The initialization **consumes the card's initial proactive
|
||||
commands** (the menu, the event list, a PLI), so a script cannot catch the
|
||||
"first command after reset" event - those commands stay in the proactive log
|
||||
as evidence. A reset *during* the run is still fatal: the suite stops and
|
||||
the remaining members are skipped (flagged in the report).
|
||||
|
||||
**Format**: server ≥ 3.15.0 (`kvn`, `por` and `files` checks), scripts stored
|
||||
server-side since 3.16.0, `menu-select` by text and the spec-order TERMINAL
|
||||
RESPONSE codings since 3.17.0, the semantic `event` action since 3.18.0, the
|
||||
@@ -41,7 +54,8 @@ RESPONSE codings since 3.17.0, the semantic `event` action since 3.18.0, the
|
||||
`menu-select` text match since 3.19.0, the `proactive-drain` action (the kind
|
||||
`proactive` until 3.22.1) and the
|
||||
ok-answering bounded cleanup since 3.21.0, the suites and the Simulator's
|
||||
event forms for all modelled events since 3.22.0.
|
||||
event forms for all modelled events since 3.22.0, a suite's `start_reset`
|
||||
since 3.23.0.
|
||||
|
||||
## Writing a test for an applet
|
||||
|
||||
|
||||
Reference in New Issue
Block a user