v3.22.0: the test suites (scripts grouped, ordered, with setup/teardown)

A suite is the root object: every script belongs to exactly one suite, the
suite holds the ordered member list by reference - the optional setup first,
the members, the optional teardown last - and runs them in one card session,
without resets and without resume.

- test_suites.py: the suite store (test_suites.json, --test-suites), validated
  and atomic like the script store: roles normalised (setup first, teardown
  last, at most one of each, a script appears once per suite), per-member
  on_fail stop/continue and the suite-level require_adm.
- test_scripts.py schema v2: suite_id/require_adm; list_by_suite()/orphans();
  the import reports the id_map a bundle import remaps with.  v3.21.0 scripts
  without a suite are attached once to an auto-created "Imported scripts"
  suite (migrate_scripts_to_suites, called from __main__).
- API: /api/test/suites (+ update|delete|import - delete refused while
  non-empty; the bundle import imports the scripts first, remaps the suite
  references and skips unresolvable ones); the scripts endpoints gain
  suite_id/require_adm plus move|copy (delete detaches from the suite first;
  a suite update refuses a membership change).
- runner: /api/test/run {suite_id} resolves the members and the preset, runs
  the ADM gate, then setup -> members (per-member on_fail) -> teardown.  The
  teardown always runs on a stop or a failure (skipped only when the card is
  gone; it clears the stop flag for its duration); a card reset stops the
  suite, skips the remaining members and is flagged.  Members' live steps land
  in the shared snapshot, their records (status/steps/note/log excerpt/
  counters) in _TEST_RUN['suite']; the summary carries the counts, wall time,
  preset, the SCP80 counters before/after per keyset and the session flags.
  _test_log buffers the semantic lines (bounded) for the report's per-member
  excerpts.
- ADM gate (_ensure_adm): a suite/script require_adm verifies the matched
  preset's key once per card session (the /api/verify-adm badge sets the same
  latch); a failure refuses the start, stamps the session (no automatic
  retry) and needs a manual verify.
- PWA: the Test script pill becomes four views - the suite list (default),
  the suite editor (the script list: role badges, on_fail, up/down, per-script
  Run/Edit/Move/Copy/Delete, Add script with the templates), the script editor
  (the step list; a single run shows its result under the steps and is not
  preserved) and the suite report (per-member blocks with the step table,
  notes and run log; summary; export JSON/Markdown).  Suite bundles export/
  import; the legacy script export imports into "Imported scripts".  EN/RU.
- tests: the suite store/migration, the runner orchestration (order, on_fail,
  setup failure, stop, card reset, teardown, per-member logs) and the ADM gate
  (Python); the suite HTTP paths; the PWA report/Markdown/wiring (Node).
  Tests: 833 frontend + 794 Python; sw cache simple-v397.
This commit is contained in:
2026-10-06 00:26:50 +03:00
parent e6ced47323
commit 5976ef89b4
17 changed files with 3178 additions and 288 deletions
+88 -15
View File
@@ -48,15 +48,22 @@ a 3.x PWA).
| `/api/ram-install-app` | POST | Single INSTALL [for install] / [for make selectable] for an already loaded package (no re-load) |
| `/api/cap-info` | POST | Validate a `.cap` archive and estimate its code/NVRAM/RAM requirements (read-only) |
| `/api/cap-compat` | POST | CAP compatibility test: LOAD only to the block completing the Import component (nothing committed) |
| `/api/test/run` | POST | Start a test script (actions + proactive expectations; `script` inline or `script_id` from the store) |
| `/api/test/status` | GET | Test script run state and per-step results |
| `/api/test/stop` | POST | Request a running test script to stop |
| `/api/test/run` | POST | Start a test script (actions + proactive expectations; `script` inline or `script_id` from the store) or a test suite (`suite_id`; ordered scripts, one card session) |
| `/api/test/status` | GET | Test script/suite run state, per-step results and the suite report |
| `/api/test/stop` | POST | Request a running test script/suite to stop |
| `/api/test/clear` | POST | Clear the finished run report |
| `/api/test/scripts` | GET | Test script store (server-side `test_scripts.json`): all scripts + the store path |
| `/api/test/scripts` | POST | Create a test script (`{name, steps}` or `{script}`) |
| `/api/test/scripts/update` | POST | Update a test script (`{id, script}`) |
| `/api/test/scripts/delete` | POST | Delete a test script (`{id}`) |
| `/api/test/scripts/import` | POST | Import scripts (`{scripts: [...], mode: merge\|replace}`; accepts the old localStorage export shape) |
| `/api/test/scripts` | POST | Create a test script (`{name, steps, suite_id}` or `{script}`; the script joins the suite) |
| `/api/test/scripts/update` | POST | Update a test script (`{id, script}`; a changed `suite_id` is a move) |
| `/api/test/scripts/delete` | POST | Delete a test script (`{id}`; detaches it from its suite first) |
| `/api/test/scripts/move` | POST | Move a script to another suite (`{id, suite_id}`) |
| `/api/test/scripts/copy` | POST | Copy a script into another suite (`{id, suite_id, name?}`) |
| `/api/test/scripts/import` | POST | Import scripts (`{scripts: [...], mode: merge\|replace, suite_id?}`; accepts the old localStorage export shape) |
| `/api/test/suites` | GET | Test suite store (server-side `test_suites.json`): all suites + the store path |
| `/api/test/suites` | POST | Create a suite (`{name, require_adm?, scripts?}` or `{suite}`) |
| `/api/test/suites/update` | POST | Update a suite (`{id, suite}`; the member set changes through the script endpoints) |
| `/api/test/suites/delete` | POST | Delete a suite (`{id}`; refused while it holds scripts) |
| `/api/test/suites/import` | POST | Import a suite bundle (`{suites, scripts, mode}`; script references remapped) |
| `/api/sp-verify` | POST | Verify secured packet against pySim reference |
| `/api/menu` | GET | Current STK menu (title + items + active) |
| `/api/menu-select` | POST | ENVELOPE(Menu Selection) with item_id |
@@ -786,17 +793,83 @@ same engine the runner uses, so a stored script can never fail at run start for
a validation reason.
```json
{"path": "/home/user/.pysim-simple-server/test_scripts.json", "count": 1, "version": 1,
"scripts": [{"id": "…", "name": "applet RFM update", "steps": [...],
{"path": "/home/user/.pysim-simple-server/test_scripts.json", "count": 1, "version": 2,
"scripts": [{"id": "…", "name": "applet RFM update", "suite_id": "…",
"require_adm": false, "steps": [...],
"created": 1690000000.0, "updated": 1690000000.0}]}
```
`POST /api/test/scripts` creates one (`{name, steps}` or `{script: {...}}`),
`POST /api/test/scripts/update` replaces name/steps (`{id, script}`),
`POST /api/test/scripts/delete` removes (`{id}`) and
`POST /api/test/scripts/import` takes `{scripts: [...], mode: "merge"|"replace"}`
(the old localStorage export shape; invalid entries are reported in `errors`
instead of failing the whole import).
**Schema v2 (v3.22.0): every script belongs to exactly one test suite**
(`suite_id`; the suite is the root object) and may declare `require_adm`
(the run verifies the ADM before starting). A v1 entry loads with
`suite_id: null` and the server attaches it to an auto-created
"Imported scripts" suite on startup.
`POST /api/test/scripts` creates one (`{name, steps, suite_id, require_adm?}`
or `{script: {...}}`; a valid `suite_id` is required - the script joins the
suite's member list) and `POST /api/test/scripts/update` replaces
name/steps/require_adm (`{id, script}`). A `suite_id` that differs from the
current one is a **move** (both suites are updated in the request).
`POST /api/test/scripts/delete` (`{id}`) detaches the script from its suite
first - the suite's member list is the only reference, so no dangling id is
left. `POST /api/test/scripts/move` (`{id, suite_id}`) moves a script to
another suite (the target appends it as a member, keeping the `on_fail`
policy) and `POST /api/test/scripts/copy` (`{id, suite_id, name?}`) clones it
into the target. `POST /api/test/scripts/import` takes
`{scripts: [...], mode: "merge"|"replace", suite_id?}` (the old localStorage
export shape; invalid entries are reported in `errors` instead of failing the
whole import; `suite_id` attaches the imported scripts to that suite).
### `GET /api/test/suites` and the suite store
Test suites live next to the scripts (`~/.pysim-simple-server/test_suites.json`,
`--test-suites PATH`): a suite is the ordered list of its scripts - the
optional `setup` first, the `member` scripts, the optional `teardown` last -
plus the per-member `on_fail` policy and the suite-level `require_adm`.
```json
{"path": "/home/user/.pysim-simple-server/test_suites.json", "count": 1, "version": 1,
"suites": [{"id": "…", "name": "alfa regression", "require_adm": true,
"scripts": [{"script_id": "…", "role": "setup", "on_fail": "stop"},
{"script_id": "…", "role": "member", "on_fail": "stop"},
{"script_id": "…", "role": "teardown", "on_fail": "stop"}],
"created": 1690000000.0, "updated": 1690000000.0}]}
```
The store normalises the order (setup first, teardown last, at most one of
each; a script appears once per suite) and validates `on_fail` as
`stop` (default) or `continue` - "which failures cascade" is a visible
per-suite decision.
`POST /api/test/suites` creates one (`{name, require_adm?, scripts?}` or
`{suite: {...}}`), `POST /api/test/suites/update` replaces
name/require_adm/scripts (`{id, suite}`; the **member set** changes through
the script endpoints, not here), `POST /api/test/suites/delete` (`{id}`)
removes it - refused while the suite still holds scripts - and
`POST /api/test/suites/import` takes
`{suites: [...], scripts: [...], mode: "merge"|"replace"}`: the scripts are
imported first (ids preserved; a collision gets a fresh uuid and the suites'
references are remapped), then the suites; an unresolvable reference is
reported in `errors` and the suite is skipped.
### `POST /api/test/run` (single script or suite)
`{script, script_id?, preset_id?, require_adm?}` runs one script (the PWA
sends the edited copy; the server validates and runs it). `{suite_id,
preset_id?}` runs a suite: its scripts in order, one card session, no resume
- the ADM prerequisite first (suite- or member-level `require_adm`: the
matched preset's key is verified **once**; a failure refuses the start, is
not retried automatically, and requires a manual verify), then setup,
members (per-member `on_fail`), teardown. **The teardown runs on every stop**
as long as the card is in the reader; a card reset (session change) stops the
suite, skips the remaining members and is flagged in the report.
The suite run's snapshot adds `suite`: the member records (`status`, `steps`,
`note`, `log` - the semantic run-log excerpt), the `summary` (ok/warning/
error/skipped counts, `wall_ms`, `stopped`, `session_changed`, the SCP80
`counters_before`/`counters_after` per keyset) and the run's `adm` state.
The report is exported from the PWA as JSON or Markdown; the run state lives
in the server until `POST /api/test/clear` (no history).
### `GET /api/test/status`
+20 -1
View File
@@ -12,12 +12,31 @@ localStorage export imports as-is), and any API client can create or run them
(`POST /api/test/run` with an inline `script` or a stored `script_id` - see
`docs/api.md`). While a run is active the card belongs to the script.
**Test suites** (v3.22.0) group the scripts: a suite is the root object (every
script belongs to exactly one suite, `suite_id` in the script store) and the
runner executes its scripts in order - the optional **setup** first, the
**member** scripts, the optional **teardown** last - in **one card session,
without resets and without resume**. Per member `on_fail` decides whether a
failure stops the suite (`stop`, default) or the next script runs
(`continue`); the **teardown always runs** on a stop or a failure (its job is
to leave the card ready for a new test) as long as the card is in the reader,
and a card reset (session change) stops the suite and is flagged in the
report. A suite (or a single script) may require the **ADM** verified: the
matched preset's key is tried **once** at the start - a failure refuses the
run and needs a manual verify (the ADM badge) before the next attempt. The
suite run's report (the PWA's report view, exportable as JSON or Markdown)
carries every member's step table, failing notes and run-log excerpt, plus the
summary (pass/warn/fail/skipped, wall time, the SCP80 counters before/after
per keyset and the card-session flag). The suites live in
`~/.pysim-simple-server/test_suites.json` (`--test-suites`).
**Format**: server ≥ 3.15.0 (`kvn`, `por` and `files` checks), scripts stored
server-side since 3.16.0, `menu-select` by text and the spec-order TERMINAL
RESPONSE codings since 3.17.0, the semantic `event` action since 3.18.0, the
`envelope` action's `src` override since 3.18.1, the inline `por` object, the
`alpha`/`sms` checks, `scp80.params.format` and the case-insensitive
`menu-select` text match since 3.19.0.
`menu-select` text match since 3.19.0, the `proactive` cleanup action and the
ok-answering bounded cleanup since 3.21.0, the suites since 3.22.0.
## Writing a test for an applet
+2 -1
View File
@@ -562,11 +562,12 @@
<section class="mb-10">
<h3 id="test-script" class="text-lg font-medium mb-2">8.9 Тест-скрипт</h3>
<p class="text-sm mb-2"><strong>Тест-скрипт</strong> &mdash; упорядоченный список <strong>действий</strong> и <strong>ожиданий</strong> проактивных команд, выполняемый на подключённой карте на стороне сервера. Пока запуск активен, карта принадлежит скрипту: остальные операции с картой отклоняются (HTTP&nbsp;409), фоновый опрос STATUS приостанавливается; отчёт о запуске хранится в памяти (в том числе после перезагрузки страницы) до нажатия <strong>Очистить отчёт</strong>. Скрипты хранятся в <strong>хранилище на сервере</strong> (<code class="font-mono text-sm">~/.pysim-simple-server/test_scripts.json</code>, <code class="font-mono text-sm">--test-scripts</code>), поэтому все браузеры работают с одним набором; изменения записываются в хранилище по мере правки (кнопки &laquo;Сохранить&raquo; нет &mdash; черновик без валидного шага остаётся локальным); экспорт/импорт JSON остаётся для резервных копий и переноса скриптов между стендами (старый экспорт из localStorage импортируется как есть).</p>
<p class="text-sm mb-2">У вкладки <strong>четыре представления</strong>: <strong>список наборов</strong> (по умолчанию; <em>Добавить набор</em> / <em>Редактировать</em> / <em>Запустить</em> / <em>Экспорт</em> / <em>Удалить</em>), <strong>редактор набора</strong> (скрипты набора в порядке выполнения &mdash; необязательный <em>setup</em> первым и <em>teardown</em> последним, с пометками ролей; у каждого скрипта <em>Запустить</em>/<em>Редактировать</em>/<em>Переместить</em>/<em>Скопировать</em>/<em>Удалить</em>, политика <em>при ошибке</em> <em>остановить</em> (по умолчанию) / <em>продолжить</em> и <em>Требуется ADM</em>), <strong>редактор скрипта</strong> (список шагов; кнопка <em>Старт</em> выполняет только этот скрипт и показывает результат под шагами &mdash; он не сохраняется) и <strong>отчёт набора</strong> (после запуска набора: статус каждого скрипта, таблица шагов с ожидаемым/фактическим, примечание об ошибке и выдержка из журнала запуска; сводка с успешно/предупреждения/ошибки/пропущено, время выполнения, счётчики SCP80 до&nbsp;&rarr;&nbsp;после по каждому keyset, пресет и предупреждение о смене сессии карты; экспорт в <em>JSON</em> или <em>Markdown</em>). <strong>Запуск набора</strong> выполняет его скрипты по порядку в <strong>одной сессии карты, без сбросов и без продолжения с места остановки</strong>; <em>при ошибке</em> решает, остановится ли набор на этом скрипте или запустится следующий, <strong>teardown выполняется всегда</strong> при остановке или ошибке (пока карта в ридере), чтобы следующий тест начался с чистого состояния, а сброс карты останавливает набор и отмечается в отчёте. <strong>Требуется ADM</strong> (у набора или скрипта) проверяет ключ ADM из подобранного пресета <strong>один раз</strong> перед запуском &mdash; ошибка отклоняет запуск (проверьте вручную через значок ADM; автоматических повторов нет). Скрипты хранятся в хранилище наборов (<code class="font-mono text-sm">~/.pysim-simple-server/test_suites.json</code>, <code class="font-mono text-sm">--test-suites</code>); каждый скрипт принадлежит ровно одному набору, и скрипт можно переместить или скопировать в другой набор (экспорт набора &mdash; это пакет с набором и его скриптами; старый экспорт только скриптов импортируется в автоматически созданный набор &laquo;Imported scripts&raquo;).</p>
<ul class="list-disc list-inside text-sm space-y-1 mb-3">
<li><strong>Действия</strong> &mdash; <em>Envelope</em> (Event Download: сырой тип события + данные, с опциональным переопределением источника &mdash; 82 терминал / 83 сеть) или <em>Event</em> (семантические формы: Location status, Access technology change, Network rejection, Data connection status change &mdash; поля редактируются как объект JSON в редакторе шагов, а для остальных событий форма события на вкладке «Телефон» показывает готовый hex), <em>Menu Selection</em> (item id 1&ndash;255 или текст пункта с точным/частичным совпадением, по умолчанию без учёта регистра &mdash; выбор <em>«Учёт регистра»</em> в редакторе шагов включает точное совпадение; текст подбирается по меню карты, которое следует за параметрами установки апплета), <em>UPDATE/READ файла</em> (путь + режим; чтение проверяет данные файла), <em>сырой APDU</em> (без авто-обработчика), <em>SCP80</em> (C-APDU или готовый защищённый пакет, опциональные переопределения номера keyset (kvn) / TAR / SPI1 / SPI2 и формат команды &mdash; компактный или обёрнутый в шаблон Command Scripting TS 102 226 &sect;5.2.1 <em>expanded</em> (AA) / <em>expanded AE&nbsp;80</em>; ключи и счётчик берутся из пресета карты, подобранного по ICCID подключённой карты &mdash; запуск называет пресет в хранилище, поэтому всегда используются его keysets, TAR и счётчик &mdash; а счётчик записывается обратно после запуска), <em>STATUS</em> (при <em>попытках</em> &gt; 1 опрашивает, пока карта не объявит команду) и <em>Очистка proactive</em> (опрос STATUS и поглощение всех объявленных команд &mdash; результат TERMINAL RESPONSE для каждой, опционально другой результат для первой; граница попыток/интервала; опциональная проверка <em>require</em> по поглощённому. Пустой опрос проходит, финальный STATUS должен быть idle, а оставшаяся команда передаётся следующему шагу &mdash; используйте в начале, середине или конце скрипта, чтобы убедиться, что карта перестала слать команды). У каждого действия есть проверка ответа: ожидаемые SW и данные (точно или маска с подстановкой <code class="font-mono text-sm">?</code> &mdash; <code class="font-mono text-sm">91??</code>) и проверка PoR для SCP80 (<em>любой / без PoR / por_ok</em> или своя проверка декодированного PoR &mdash; статус / SW / данные; транспорт через SEND SHORT MESSAGE работает так же).</li>
<li><strong>Ожидания</strong> &mdash; карта должна объявить проактивную команду на предыдущем шаге (SW <code class="font-mono text-sm">91XX</code>, TS 102 221 &sect;7.4.2.1 / TS 102 223 &sect;6.3); <code class="font-mono text-sm">9000</code> означает &laquo;команды нет&raquo;: шаг падает на своём уровне ошибок &mdash; Ошибка останавливает скрипт, Предупреждение продолжает (терпимая форма &laquo;поглотить команду, если она есть&raquo;) &mdash; а действие STATUS с <em>попытками</em> &gt; 1 опрашивает, когда апплет выдаёт команду при опросе. Проверяются тип команды, квалификатор и содержимое: текст / пункт / raw, проверка <em>Alpha</em> (идентификатор Alpha команды, тег <code class="font-mono text-sm">05</code>/<code class="font-mono text-sm">85</code> &mdash; отличается от текстовой строки <code class="font-mono text-sm">8D</code>), проверка <em>SMS TPDU</em> (submit-TPDU команды SEND SHORT MESSAGE: получатель / PID / DCS / UDL / данные &mdash; для собственного SMS апплета), проверка <em>PoR</em> (SEND SHORT MESSAGE несёт PoR, когда пакет запросил PoR-in-submit, бит SPI2 <code class="font-mono text-sm">20</code>: статус ответа, SW R-APDU и данные ответа; перед этим должен выполниться шаг SCP80 &mdash; SW R-APDU находится <em>внутри</em> полезной нагрузки ответа: собственный TAR апплета возвращает данные приложения, поэтому проверяйте данные, а SW оставляйте пустым; отклонённый ENVELOPE (<code class="font-mono text-sm">62XX</code>/<code class="font-mono text-sm">63XX</code>) может вовсе не нести пакет ответа &mdash; для такого шага оставляйте проверку PoR «любой» или ожидайте «без PoR») и проверка <em>Files</em> (File List команды REFRESH, например квалификатор <code class="font-mono text-sm">00</code> = NAA initialization with full file change notification). <strong>TERMINAL RESPONSE</strong> (результат, идентификатор пункта, текст, доп. TLV) является частью инструкции.</li>
<li><strong>Уровни ошибок</strong> &mdash; несовпадение помечает шаг как <em>Ошибка</em> (скрипт останавливается) или <em>Предупреждение</em> (продолжает); уровень применяется к любой ошибке шага, включая ожидание без объявленной команды; каждая проверка показывается с ожидаемым и фактическим значением.</li>
<li><strong>Шаблоны</strong> &mdash; Empty, <em>Обзор меню STK</em>, <em>Апплет через SCP80</em>; редактор шагов — формный (поля действия Event вводятся объектом JSON), отдельные скрипты экспортируются/импортируются как JSON.</li>
<li><strong>Шаблоны</strong> &mdash; Empty, <em>Обзор меню STK</em>, <em>Апплет через SCP80</em>; редактор шагов — формный (поля действия Event вводятся объектом JSON), а набор экспортируется/импортируется пакетом JSON (набор и его скрипты).</li>
</ul>
<p class="text-sm mb-3">Каждый шаг пишет семантическую строку в журнал сервера под своим номером (<code class="font-mono text-sm">TEST-RUN step N: …</code>), запрос перед ответом: отправленный APDU/данные и полученный SW, для SCP80 — открытый C-APDU, собранный защищённый пакет и R-APDU из PoR, для ожидания — тройка FETCH / команда / TERMINAL RESPONSE, очистка proactive — каждый раунд STATUS/FETCH/команда/TR, а очистка в конце запуска — поглощённые команды (отвечаются <code class="font-mono text-sm">ok</code> &mdash; cancel отправляется только по явному запросу скрипта), а для выбора пункта меню — полный ENVELOPE с определённым пунктом. Запустите сервер с <code class="font-mono text-sm">--apdu-trace</code>, чтобы добавить и журнал сырых транспортных APDU.</p>
+2 -1
View File
@@ -561,11 +561,12 @@
<section class="mb-10">
<h3 id="test-script" class="text-lg font-medium mb-2">8.9 Test script</h3>
<p class="text-sm mb-2">A <strong>test script</strong> is an ordered list of <strong>actions</strong> and proactive-command <strong>expectations</strong> executed server-side on the equipped card. While a run is active the card belongs to the script: other card actions are refused (HTTP&nbsp;409) and background STATUS polling is paused; the run report is kept in memory (also after reloading the page) until <strong>Clear report</strong>. Scripts live in the <strong>server-side store</strong> (<code class="font-mono text-sm">~/.pysim-simple-server/test_scripts.json</code>, <code class="font-mono text-sm">--test-scripts</code>), so every browser manages the same set; edits are written to the store as you change them (there is no Save button &mdash; a draft without a valid step stays local until it is valid); JSON export/import remains for backup and for moving scripts between benches (the old localStorage export imports as-is).</p>
<p class="text-sm mb-2">The pill has <strong>four views</strong>: the <strong>suite list</strong> (default; <em>Add suite</em> / <em>Edit</em> / <em>Run</em> / <em>Export</em> / <em>Delete</em>), the <strong>suite editor</strong> (the suite's scripts in run order &mdash; the optional <em>setup</em> first and the <em>teardown</em> last, marked with role badges; per script <em>Run</em>/<em>Edit</em>/<em>Move</em>/<em>Copy</em>/<em>Delete</em>, the per-member <em>on failure</em> policy <em>stop</em> (default) / <em>continue</em>, and <em>ADM required</em>), the <strong>script editor</strong> (the step list; its <em>Start</em> runs that single script and shows the result under the steps &mdash; not preserved) and the <strong>suite report</strong> (after a suite run: every script's status, its step table with expected/actual, the failing note and a run-log excerpt; the summary with passed/warnings/failed/skipped, wall time, the SCP80 counters before&nbsp;&rarr;&nbsp;after per keyset, the preset and a card-session-changed warning; export as <em>JSON</em> or <em>Markdown</em>). A <strong>suite run</strong> executes its scripts in order in <strong>one card session, without resets and without resume</strong>; <em>on failure</em> decides whether the suite stops at that script or runs the next one, the <strong>teardown always runs</strong> on a stop or a failure (while the card is in the reader) so the next test starts clean, and a card reset stops the suite and is flagged in the report. <strong>ADM required</strong> (on the suite or a script) verifies the matched preset's ADM key <strong>once</strong> before the start &mdash; a failure refuses the run (verify manually with the ADM badge; nothing is retried automatically). Scripts live in the suite's store (<code class="font-mono text-sm">~/.pysim-simple-server/test_suites.json</code>, <code class="font-mono text-sm">--test-suites</code>); every script belongs to exactly one suite, and a script can be moved or copied to another suite (a bundle export carries the suite plus its scripts; the old script-only export imports into an auto-created &ldquo;Imported scripts&rdquo; suite).</p>
<ul class="list-disc list-inside text-sm space-y-1 mb-3">
<li><strong>Actions</strong> &mdash; <em>Envelope</em> (Event Download: raw event type + data, with an optional source override &mdash; 82 terminal / 83 network) or <em>Event</em> (the semantic forms: Location status, Access technology change, Network rejection, Data connection status change &mdash; the fields are edited as a JSON object in the step editor, and the Phone tab's event form shows the built hex for the other events), <em>Menu Selection</em> (item id 1&ndash;255, or the item's text with an exact/contains match, case-insensitive by default &mdash; the step editor's <em>Match case</em> choice switches to an exact-case match; the text is resolved against the card's menu, which follows the applet's install parameters), <em>UPDATE/READ file</em> (path + mode; reads verify the file data), <em>raw APDU</em> (no auto-handler), <em>SCP80</em> (C-APDU or a pre-built secured packet, optional keyset number (kvn) / TAR / SPI1 / SPI2 overrides and a command format &mdash; compact, or the TS 102 226 &sect;5.2.1 Command Scripting template <em>expanded</em> (AA) / <em>expanded AE&nbsp;80</em> around the C-APDU; the keys and the counter come from the card preset matched by the equipped card's ICCID &mdash; the run names the stored preset, so it always uses its keysets, TARs and counter &mdash; and the counter is written back to it after the run), <em>STATUS</em> (with <em>attempts</em> &gt; 1 it polls until the card announces a command) and <em>Proactive cleanup</em> (poll STATUS and consume every announced command &mdash; the TERMINAL RESPONSE result for each, optionally a different result for the first; the attempt/interval bound; an optional <em>require</em> assertion over what was drained. An empty poll passes, the final STATUS must be idle, and a still-pending command is handed to the next step &mdash; use it at the start, in the middle or at the end of a script to make sure the card has stopped sending). Every action has a response check: expected SW and data (exact, or a mask with <code class="font-mono text-sm">?</code> nibble wildcards &mdash; <code class="font-mono text-sm">91??</code>) plus a PoR check for SCP80 (<em>any / no PoR / por_ok</em>, or a custom assertion of the decoded PoR &mdash; status / SW / data; the SEND SHORT MESSAGE transport works the same).</li>
<li><strong>Expectations</strong> &mdash; the card must have announced a proactive command in the previous step (SW <code class="font-mono text-sm">91XX</code>, TS 102 221 &sect;7.4.2.1 / TS 102 223 &sect;6.3); a <code class="font-mono text-sm">9000</code> means &ldquo;no command&rdquo;: the step fails at its fail level &mdash; Error stops the script, Warning continues (the tolerant &ldquo;consume a command if there is one&rdquo; form) &mdash; and a STATUS action with <em>attempts</em> &gt; 1 polls when the applet delivers on poll. The command type, qualifier and contents are verified: text / item / raw checks, an <em>Alpha</em> check (the command's Alpha identifier, tag <code class="font-mono text-sm">05</code>/<code class="font-mono text-sm">85</code> &mdash; distinct from the <code class="font-mono text-sm">8D</code> text string), an <em>SMS TPDU</em> check (the submit TPDU of a SEND SHORT MESSAGE: destination / PID / DCS / UDL / user data &mdash; for an applet's own SMS), a <em>PoR</em> check (a SEND SHORT MESSAGE carries the PoR when the packet requested PoR-in-submit, SPI2 bit <code class="font-mono text-sm">20</code>: the response status, the R-APDU SW and the response data; make sure an SCP80 step ran first &mdash; the R-APDU SW lives <em>inside</em> the response payload: an applet's own TAR returns application data, so check its data and leave the SW empty; a refused ENVELOPE (<code class="font-mono text-sm">62XX</code>/<code class="font-mono text-sm">63XX</code>) may carry no response packet at all &mdash; leave the PoR check at <em>any</em> or expect <em>no PoR</em> for such a step) and a <em>Files</em> check (the File List of a REFRESH, e.g. qualifier <code class="font-mono text-sm">00</code> = NAA initialization with full file change notification). The scripted <strong>TERMINAL RESPONSE</strong> (result, item identifier, text, extra TLVs) is part of the instruction.</li>
<li><strong>Fail levels</strong> &mdash; a mismatch flags the step as <em>Error</em> (terminates the script) or <em>Warning</em> (continues); every step failure honours it, including an expectation that finds no pending command; each check is reported with expected vs actual values.</li>
<li><strong>Templates</strong> &mdash; Empty, <em>STK menu browsing</em>, <em>Applet via SCP80</em>; the step editor is form-based (the event action's fields are a JSON object) and individual scripts export/import as JSON.</li>
<li><strong>Templates</strong> &mdash; Empty, <em>STK menu browsing</em>, <em>Applet via SCP80</em>; the step editor is form-based (the event action's fields are a JSON object), and a suite exports/imports as a JSON bundle (the suite plus its scripts).</li>
</ul>
<p class="text-sm mb-3">Every step logs a semantic line to the server log under its step number (<code class="font-mono text-sm">TEST-RUN step N: …</code>), request before response: the APDU/data sent and the SW received, for SCP80 the plaintext C-APDU, the built secured packet and the PoR's R-APDU, for an expectation the FETCH / command / TERMINAL RESPONSE trio, a proactive cleanup every STATUS/FETCH/command/TR round and the run cleanup the commands it drained (answered <code class="font-mono text-sm">ok</code> &mdash; a cancel is only sent when a script asks for it), and for a menu selection the complete ENVELOPE with the resolved item. Start the server with <code class="font-mono text-sm">--apdu-trace</code> to add the raw transport APDU log as well.</p>
+763 -196
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1,4 +1,4 @@
const CACHE = 'simple-v396';
const CACHE = 'simple-v397';
const URLS = [
'index.html',
'help.html',
+39 -21
View File
@@ -40,6 +40,8 @@ eval(extractFunc(html, 'testStepSummary'));
eval(extractFunc(html, 'testCommandOptions'));
globalThis.t = s => s;
globalThis.esc = s => String(s);
globalThis.testScripts = () => _testScripts || [];
globalThis.testSuites = () => _testSuites || [];
eval(extractFunc(html, 'testFormRow'));
eval(extractFunc(html, 'testFormInput'));
eval(extractFunc(html, 'testFormSelect'));
@@ -53,11 +55,14 @@ eval(extractFunc(html, 'testScriptsWriteback', true));
eval(extractFunc(html, 'testScriptsWritebackQueued'));
eval(extractFunc(html, 'testScriptsSave'));
eval(extractFunc(html, 'testRunStart', true));
eval(extractFunc(html, 'testDelete', true));
eval(extractFunc(html, 'testSuiteDeleteScript', true));
eval(extractFunc(html, 'testSuitesRefetch', true));
eval(extractFunc(html, 'testChecksCollect'));
eval('var _testEditStep = null; var _testEditChecks = []; var _testEditStepIndex = -1;'
+ ' var _testScripts = null; var _testCurrentIdx = -1; var _testRunState = null;'
+ ' var _testSaveTimer = null; var _testWriteChains = new WeakMap(); var _testLastPresetIdx = -1;');
+ ' var _testScripts = null; var _testSuites = []; var _testScriptId = null;'
+ ' var _testScriptDraft = null; var _testRunState = null; var _testSaveTimer = null;'
+ ' var _testWriteChains = new WeakMap(); var _testLastPresetIdx = -1;'
+ ' var _testRunScriptRef = null; var _testSuiteIdx = -1;');
globalThis.localStorage = { getItem: () => null, setItem: () => {}, removeItem: () => {} };
function fakeForm(values) {
@@ -443,12 +448,14 @@ test('two quick saves of a draft create one script, then update it', async () =>
};
globalThis.ioStatus = () => {};
_testScripts = [{ name: 'one', steps: [{ type: 'action', kind: 'status', params: { attempts: 1 } }] }];
_testCurrentIdx = 0;
globalThis.testCurrent = () => _testScripts[_testCurrentIdx];
_testScriptId = null;
globalThis.testCurrentScript = () => _testScripts[0];
testScriptsSave(true);
testScriptsSave(true);
await new Promise(r => setTimeout(r, 40));
assert.deepStrictEqual(calls, ['/api/test/scripts', '/api/test/scripts/update']);
// the create is followed by a suites refetch (the suite gained the member)
assert.deepStrictEqual(calls, ['/api/test/scripts', '/api/test/suites',
'/api/test/scripts/update']);
assert.strictEqual(_testScripts[0].id, 'd'.repeat(32));
});
@@ -458,15 +465,21 @@ test('deleting a script cancels a pending debounced save', async () => {
globalThis.ioStatus = () => {};
globalThis.confirm = () => true;
globalThis.testRender = () => {};
_testScripts = [{ name: 'draft', steps: [{ type: 'action', kind: 'status', params: { attempts: 1 } }] }];
_testCurrentIdx = 0;
globalThis.testCurrent = () => _testScripts[_testCurrentIdx];
globalThis.testStoresFetch = async () => {};
const script = {id: 'e'.repeat(32), name: 'edited',
steps: [{type: 'action', kind: 'status', params: {attempts: 1}}]};
_testScripts = [script];
globalThis.testCurrentScript = () => script;
globalThis.testScriptById = () => script;
globalThis.testCurrentSuite = () => ({id: 'a'.repeat(32),
scripts: [{script_id: script.id, role: 'member', on_fail: 'stop'}]});
globalThis.testSuiteEntry = () => ({script_id: script.id});
globalThis.testSuites = () => [];
testScriptsSave(); // schedules the 600 ms write
await testDelete(); // a draft: local removal only
await testSuiteDeleteScript(0); // the delete path
await new Promise(r => setTimeout(r, 700));
assert.deepStrictEqual(calls, []); // the queued save never fired
assert.strictEqual(_testScripts.length, 1);
assert.strictEqual(_testScripts[0].name, 'New test script');
assert.deepStrictEqual(calls, ['/api/test/scripts/delete']);
assert.strictEqual(script._deleted, true);
});
test('the STATUS step leaves the SW check to the server default', () => {
@@ -606,12 +619,16 @@ test('the run sends the stored preset id, not the removed flat fields', () => {
test('the run refuses without a matched or complete preset', () => {
globalThis.cards = [];
globalThis.cardsMatchedPreset = () => null;
assert.match(testRunPreset().error, /No card preset matches/);
assert.match(testRunPreset(true).error, /No card preset matches/);
const preset = { id: 'p2' };
globalThis.cards = [preset];
globalThis.cardsMatchedPreset = () => preset;
globalThis.cardsScp80Complete = () => false;
assert.match(testRunPreset().error, /incomplete SCP80/);
assert.match(testRunPreset(true).error, /incomplete SCP80/);
// an ADM-only run needs the preset but not the SCP80 completeness
assert.strictEqual(testRunPreset(false).preset_id, 'p2');
globalThis.cardsScp80Complete = () => true;
assert.strictEqual(testRunPreset(true).preset_id, 'p2');
});
test('the client-side problem check bounds the keyset number', () => {
@@ -681,10 +698,11 @@ test('a new script is created in the server store and gets its id', async () =>
assert.strictEqual(calls[0][0], '/api/test/scripts');
assert.strictEqual(calls[0][1].name, 'new');
assert.strictEqual(s.id, 'a'.repeat(32)); // the store id is adopted
assert.strictEqual(calls[1][0], '/api/test/suites'); // the member refetch
// every later write updates by id
await testScriptsWriteback(s);
assert.strictEqual(calls[1][0], '/api/test/scripts/update');
assert.strictEqual(calls[1][1].id, 'a'.repeat(32));
assert.strictEqual(calls[2][0], '/api/test/scripts/update');
assert.strictEqual(calls[2][1].id, 'a'.repeat(32));
});
test('an incomplete draft is not written to the store', async () => {
@@ -701,11 +719,11 @@ test('an immediate save writes the current script through', async () => {
globalThis.pysimFetch = async (path) => { calls.push(path); return {ok: true, script: {id: 'c'.repeat(32)}}; };
globalThis.ioStatus = () => {};
_testScripts = [{name: 'one', steps: [{type: 'action', kind: 'status', params: {attempts: 1}}]}];
_testCurrentIdx = 0;
globalThis.testCurrent = () => _testScripts[_testCurrentIdx];
_testScriptId = null;
globalThis.testCurrentScript = () => _testScripts[0];
testScriptsSave(true);
await new Promise(r => setTimeout(r, 0)); // the writeback is async
assert.deepStrictEqual(calls, ['/api/test/scripts']);
assert.deepStrictEqual(calls, ['/api/test/scripts', '/api/test/suites']);
assert.strictEqual(_testScripts[0].id, 'c'.repeat(32));
});
@@ -723,7 +741,7 @@ test('the run sends the stored script id with the edited script', async () => {
sent = {path, body};
return {running: true, name: 'saved', index: 0, total: 1, steps: [], status: null};
};
globalThis.testCurrent = () => ({id: 'b'.repeat(32), name: 'saved',
globalThis.testCurrentScript = () => ({id: 'b'.repeat(32), name: 'saved',
steps: [{type: 'action', kind: 'status', params: {attempts: 1}}]});
globalThis.testRunPreset = () => { throw new Error('no preset for a status-only script'); };
globalThis.testRunMessage = () => {};
+151
View File
@@ -0,0 +1,151 @@
// Test suite view/report tests (v3.22.0): the report renderers, the Markdown
// export and the markup wiring of the four views under the Test script pill.
const { test } = require('node:test');
const assert = require('node:assert');
const fs = require('node:fs');
const path = require('node:path');
const html = fs.readFileSync(path.join(__dirname, '..', 'index.html'), 'utf8');
function extractFunc(src, name, asyncFn) {
const re = new RegExp('(?:async\\s+)?function\\s+' + name + '\\s*\\([^)]*\\)\\s*\\{');
const m = re.exec(src);
if (!m) throw new Error('function ' + name + ' not found');
let i = m.index + m[0].length - 1;
let depth = 0;
for (; i < src.length; i++) {
if (src[i] === '{') depth++;
else if (src[i] === '}') {
depth--;
if (depth === 0) break;
}
}
return src.slice(m.index, i + 1);
}
let code = '';
for (const fn of ['testStatusBadge', 'testRunStepHtml', 'testReportMemberHtml',
'testReportMarkdown', 'testSuiteProgress', 'testRoleOptions', 'testFailOptions']) {
code += extractFunc(html, fn) + '\n';
}
eval(code);
globalThis.t = s => s;
globalThis.esc = s => String(s);
function snapshot() {
return {
name: 'regression', status: 'error', preset: 'SE test1',
started: 1, finished: 2,
suite: {
id: 'a'.repeat(32), name: 'regression', require_adm: true,
session_start: 1, session_end: 1, session_changed: false,
counters_before: {'2': '0000000010'}, counters_after: {'2': '0000000012'},
members: [
{script_id: 'b'.repeat(32), name: 'setup', role: 'setup',
on_fail: 'stop', status: 'ok',
steps: [{index: 0, type: 'action', label: 'STATUS', status: 'ok',
checks: [{label: 'SW', ok: true, expected: '9000', actual: '9000'}]}],
log: ['TEST-RUN [1/3 setup setup] step 1: STATUS 1/1 -> 9000'], log_truncated: false},
{script_id: 'c'.repeat(32), name: 'alfa-dsa', role: 'member',
on_fail: 'stop', status: 'error',
note: 'step 2 failed',
steps: [{index: 0, type: 'action', label: 'SCP80', status: 'ok', checks: []},
{index: 1, type: 'expect', label: 'REFRESH', status: 'error',
checks: [{label: 'Command', ok: false, expected: 'REFRESH',
actual: '(none)', level: 'error'}]}],
log: ['TEST-RUN [2/3 member alfa-dsa] step 1: SCP80 -> SW=9103'], log_truncated: true},
{script_id: 'd'.repeat(32), name: 'teardown', role: 'teardown',
on_fail: 'stop', status: 'skipped', steps: [], log: [], log_truncated: false},
],
summary: {ok: 1, warning: 0, error: 1, skipped: 1, stopped: false,
session_changed: false, wall_ms: 12345,
counters_before: {'2': '0000000010'}, counters_after: {'2': '0000000012'},
setup: 'ok', teardown: 'skipped'},
},
adm: {required: true, verified: true},
};
}
test('testReportMarkdown renders the members, steps, notes and logs', () => {
const md = testReportMarkdown(snapshot());
assert.match(md, /# Test suite report: regression/);
assert.match(md, /- status: error/);
assert.match(md, /- Preset: SE test1/);
assert.match(md, /passed: 1, warnings: 0, failed: 1, skipped: 1/);
assert.match(md, /wall time: 12\.3 s/);
assert.match(md, /SCP80: 2 0000000010 -> 0000000012/);
assert.match(md, /## 2\. alfa-dsa \(member, on_fail: stop\) - error/);
assert.match(md, /> step 2 failed/);
assert.match(md, /### Step 2: REFRESH - error/);
assert.match(md, /- ✗ Command: expected REFRESH, actual \(none\)/);
assert.match(md, /TEST-RUN \[2\/3 member alfa-dsa\] step 1: SCP80 -> SW=9103/);
assert.match(md, /## 3\. teardown \(teardown, on_fail: stop\) - skipped/);
});
test('testReportMarkdown tolerates an empty snapshot', () => {
assert.match(testReportMarkdown(null), /# Test suite report: /);
});
test('testReportMemberHtml carries the status, note and log details', () => {
const member = snapshot().suite.members[1];
const htmlOut = testReportMemberHtml(member, 1);
assert.match(htmlOut, /alfa-dsa/);
assert.match(htmlOut, /step 2 failed/);
assert.match(htmlOut, /Run log/);
assert.match(htmlOut, /truncated/);
assert.match(htmlOut, /Command: /);
const teardown = testReportMemberHtml(snapshot().suite.members[2], 2);
assert.match(teardown, /teardown/);
assert.match(teardown, /skipped/);
});
test('testSuiteProgress counts the finished members only', () => {
assert.strictEqual(testSuiteProgress(snapshot()), '3/3');
const snap = snapshot();
snap.suite.members[1].status = 'running';
snap.suite.members[2].status = 'pending';
assert.strictEqual(testSuiteProgress(snap), '1/3');
});
test('the role and on_fail selects mark the stored value', () => {
assert.match(testRoleOptions('setup'), /value="setup" selected/);
assert.match(testRoleOptions('member'), /value="member" selected/);
assert.doesNotMatch(testRoleOptions('member'), /value="setup" selected/);
assert.match(testFailOptions('continue'), /value="continue" selected/);
assert.match(testFailOptions('stop'), /value="stop" selected/);
});
test('the Test script pill hosts the four suite views and their wiring', () => {
for (const id of ['test-view-suites', 'test-view-suite', 'test-view-script', 'test-view-report']) {
assert.ok(html.includes('id="' + id + '"'), 'missing ' + id);
}
for (const fn of ['testSuiteNew', 'testSuiteOpen', 'testSuiteDelete', 'testSuiteExport',
'testSuiteRun', 'testSuiteAddScript', 'testSuiteDeleteScript', 'testSuiteMoveScript',
'testSuiteTargetGo', 'testOpenScript', 'testScriptBack', 'testReportExportJson',
'testReportExportMarkdown', 'testImportFile', 'testViewShow']) {
assert.ok(html.includes('function ' + fn) || html.includes('async function ' + fn),
'missing ' + fn);
}
assert.ok(/id="test-suite-name"[^>]*oninput="testSuiteNameChanged\(\)"/.test(html));
assert.ok(/id="test-suite-adm"[^>]*onchange="testSuiteAdmChanged\(\)"/.test(html));
assert.ok(/id="test-script-adm"[^>]*onchange="testScriptAdmChanged\(\)"/.test(html));
assert.ok(/id="test-suite-run-btn"[^>]*data-needs="card"/.test(html));
assert.ok(/id="test-run-btn"[^>]*data-needs="card"/.test(html));
});
test('the suite run opens the report view and names the preset', () => {
const src = extractFunc(html, 'testSuiteRun', true);
assert.match(src, /'\/api\/test\/run'/);
assert.match(src, /suite_id/);
assert.match(src, /_testView = 'report'/);
assert.match(src, /testRunPreset\(usesScp80\)/);
assert.match(src, /needsAdm/);
});
test('a single script run sends its require_adm flag and keeps its id', () => {
const src = extractFunc(html, 'testRunStart', true);
assert.match(src, /require_adm: !!cur\.require_adm/);
assert.match(src, /body\.script_id = cur\.id/);
assert.match(src, /testRenderRun\(resp\)/);
});
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "pysim-simple-server"
version = "3.21.0"
version = "3.22.0"
description = "HTTP REST server wrapping pysim for the SIMple PWA"
requires-python = ">=3.8"
# pysim is a git-only dependency installed explicitly by setup.bat/setup.sh.
+11 -1
View File
@@ -16,8 +16,9 @@ from . import fastinit
from . import gsmtap
from . import presets
from . import test_scripts
from . import test_suites
from . import testscript
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter, _test_command_type
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter, _test_command_type, migrate_scripts_to_suites
_server_start = 0
@@ -70,6 +71,8 @@ def main():
help='Card preset store (default: ~/.pysim-simple-server/card_presets.json)')
parser.add_argument('--test-scripts', default=None, metavar='PATH',
help='Test script store (default: ~/.pysim-simple-server/test_scripts.json)')
parser.add_argument('--test-suites', default=None, metavar='PATH',
help='Test suite store (default: ~/.pysim-simple-server/test_suites.json)')
parser.add_argument('--sms-oa', default='12345', metavar='DIGITS',
help='TP-Originating-Address (SMSC number) for the SMS-DELIVER TPDU (default: 12345)')
parser.add_argument('--sms-sm-sc', default='12345678912', metavar='DIGITS',
@@ -300,6 +303,13 @@ def main():
validator=lambda raw: testscript.normalise_script(raw, _test_command_type))
sys.stderr.write('TESTSCRIPTS: %s (%d scripts)\n'
% (server.test_scripts.path, len(server.test_scripts.list())))
# Test suites group the scripts (v3.22.0): a script belongs to exactly one
# suite, so a pre-suite store's scripts are attached to an auto-created
# "Imported scripts" suite once (idempotent - no orphans, nothing to do).
server.test_suites = test_suites.TestSuiteStore(opts.test_suites)
sys.stderr.write('TESTSUITES: %s (%d suites)\n'
% (server.test_suites.path, len(server.test_suites.list())))
migrate_scripts_to_suites(server.test_scripts, server.test_suites)
server.sim_menu = sim_menu
server.event_list = event_list
server.menu_active = False
File diff suppressed because it is too large Load Diff
+72 -15
View File
@@ -12,10 +12,20 @@ the editor, while the next save rewrites the file with it intact.
A stored script is the script JSON (``name`` + ``steps``, the same shape the
PWA exports) plus a stable uuid and timestamps::
{"version": 1, "scripts": [
{"id": "...", "name": "applet RFM update", "steps": [...],
{"version": 2, "scripts": [
{"id": "...", "name": "applet RFM update", "suite_id": "...",
"require_adm": false, "steps": [...],
"created": 1690000000.0, "updated": 1690000000.0}]}
Schema v2 (v3.22.0) adds ``suite_id``/``require_adm``: a script belongs to
exactly one test suite (the suite is the root object; the PWA creates scripts
only inside a suite) and can declare that it needs the ADM PIN verified. A
v1 entry loads with ``suite_id: None`` - the server attaches such orphans to
an auto-created "Imported scripts" suite on startup - and the next save
writes v2. The cross-store rules (the suite exists, a move updates both
stores, a referenced script cannot leave a dangling reference) live in the
HTTP layer, which holds both stores.
Validation runs on every mutation: the basic shape always, and - when the
server passes its ``validator`` - the full ``testscript.normalise_script``
pass (the same one the runner uses at start). Import accepts the old
@@ -44,7 +54,9 @@ from pathlib import Path
DEFAULT_DIR = '.pysim-simple-server'
DEFAULT_FILENAME = 'test_scripts.json'
SCHEMA_VERSION = 1
SCHEMA_VERSION = 2
_ID_RE = re.compile(r'[0-9a-f]{32}')
def default_path():
@@ -171,7 +183,16 @@ class TestScriptStore:
if not isinstance(step, dict) or step.get('type') not in ('action', 'expect'):
raise TestScriptError('step %d must be an action or an expectation'
% (i + 1))
script = {'name': name.strip(), 'steps': copy.deepcopy(steps)}
suite_id = raw.get('suite_id')
if suite_id in (None, ''):
suite_id = None
else:
suite_id = str(suite_id).strip().lower()
if not _ID_RE.fullmatch(suite_id):
raise TestScriptError('suite_id must be a suite uuid')
script = {'name': name.strip(), 'suite_id': suite_id,
'require_adm': bool(raw.get('require_adm')),
'steps': copy.deepcopy(steps)}
if validate and self._validator is not None:
try:
self._validator(copy.deepcopy(script))
@@ -189,7 +210,7 @@ class TestScriptStore:
script = self._check(entry, validate=validate)
now = time.time()
rid = str(entry.get('id') or '').strip().lower()
if not re.fullmatch(r'[0-9a-f]{32}', rid):
if not _ID_RE.fullmatch(rid):
rid = uuid.uuid4().hex
if keep_meta:
created = entry.get('created')
@@ -198,7 +219,10 @@ class TestScriptStore:
updated = updated if isinstance(updated, (int, float)) else created
else:
created = updated = now
return {'id': rid, 'name': script['name'], 'steps': script['steps'],
return {'id': rid, 'name': script['name'],
'suite_id': script['suite_id'],
'require_adm': script['require_adm'],
'steps': script['steps'],
'created': created, 'updated': updated}
def _find(self, sid):
@@ -218,6 +242,20 @@ class TestScriptStore:
s = self._find(sid)
return copy.deepcopy(s) if s else None
def list_by_suite(self, suite_id):
"""The scripts attached to one suite (the suite holds their order)."""
want = str(suite_id or '').strip().lower()
with self._lock:
return [copy.deepcopy(s) for s in self._scripts
if (s.get('suite_id') or '') == want]
def orphans(self):
"""Scripts without a suite (a v1 store, or a hand edit): the server
attaches them to the imported-scripts suite on startup."""
with self._lock:
return [copy.deepcopy(s) for s in self._scripts
if not s.get('suite_id')]
def add(self, script):
with self._lock:
s = self._normalise(script or {})
@@ -232,19 +270,25 @@ class TestScriptStore:
return copy.deepcopy(s)
def update(self, sid, fields):
"""Partial or full update: name/steps replaced, id and created kept."""
"""Partial or full update: name/steps/suite_id/require_adm replaced,
id and created kept. The suite_id change is the *move* operation's
write (the HTTP layer updates the source and target suites around
it)."""
with self._lock:
cur = self._find(sid)
if cur is None:
return None
merged = {'name': cur['name'], 'steps': copy.deepcopy(cur['steps'])}
merged = {'name': cur['name'], 'suite_id': cur.get('suite_id'),
'require_adm': cur.get('require_adm', False),
'steps': copy.deepcopy(cur['steps'])}
if isinstance(fields, dict):
if 'name' in fields:
merged['name'] = fields['name']
if 'steps' in fields:
merged['steps'] = fields['steps']
for key in ('name', 'steps', 'suite_id', 'require_adm'):
if key in fields:
merged[key] = fields[key]
script = self._check(merged)
new = {'id': cur['id'], 'name': script['name'],
'suite_id': script['suite_id'],
'require_adm': script['require_adm'],
'steps': script['steps'], 'created': cur['created'],
'updated': time.time()}
self._scripts[self._scripts.index(cur)] = new
@@ -262,12 +306,17 @@ class TestScriptStore:
% (s['name'] or '(unnamed)', s['id']))
return True
def import_scripts(self, items, mode='merge'):
def import_scripts(self, items, mode='merge', suite_id=None):
"""Import exported (or old localStorage) scripts: entries are validated
one by one; an already-used id gets a fresh one, invalid entries are
reported instead of failing the whole import. ``replace`` wipes the
store first."""
store first. ``suite_id`` attaches every imported script to that suite
(the "import into this suite" flow). Returns the ``id_map`` (input id
-> stored id, the bundle import's reference remap) and the stored ids
of the imported scripts (the target suite's member entries)."""
added, skipped, errors = 0, 0, []
id_map = {}
added_ids = []
with self._lock:
if mode == 'replace':
self._scripts = []
@@ -276,20 +325,28 @@ class TestScriptStore:
skipped += 1
continue
try:
entry = dict(entry)
if suite_id:
entry['suite_id'] = suite_id
s = self._normalise(entry)
src_id = str(entry.get('id') or '').strip().lower()
if self._find(s['id']):
s['id'] = uuid.uuid4().hex
if _ID_RE.fullmatch(src_id):
id_map[src_id] = s['id']
except TestScriptError as e:
skipped += 1
errors.append('%s: %s' % (entry.get('name') or ('#' + str(i + 1)), e))
continue
self._scripts.append(s)
added_ids.append(s['id'])
added += 1
if added or mode == 'replace':
self._save()
sys.stderr.write('TESTSCRIPTS: import %s: %d added, %d skipped\n'
% (mode, added, skipped))
return {'added': added, 'skipped': skipped, 'errors': errors[:10]}
return {'added': added, 'skipped': skipped, 'errors': errors[:10],
'id_map': id_map, 'added_ids': added_ids}
def info(self):
with self._lock:
+339
View File
@@ -0,0 +1,339 @@
"""Server-side test suite store: ordered groups of card test scripts.
The test scripts used to be a flat set; a suite groups them (a suite is the
root object - every script belongs to exactly one suite, ``suite_id`` in the
script store). The suite keeps the **ordered member list** by reference
(``script_id``), so the scripts stay the single source of truth and moving or
editing one never touches the suite file beyond the reference list. The
runner executes them in order: an optional ``setup`` script first, the
``member`` scripts, an optional ``teardown`` script last - the roles must be
in that order, so the store normalises the list (setup entries first,
teardown entries last, at most one of each) instead of rejecting a
mis-ordered edit.
A stored suite::
{"version": 1, "suites": [
{"id": "...", "name": "alfa regression", "require_adm": true,
"scripts": [{"script_id": "...", "role": "setup", "on_fail": "stop"},
{"script_id": "...", "role": "member", "on_fail": "stop"},
{"script_id": "...", "role": "member", "on_fail": "continue"},
{"script_id": "...", "role": "teardown", "on_fail": "stop"}],
"created": ..., "updated": ...}]}
``on_fail`` is the per-member policy: ``stop`` (default) ends the suite at
that member, ``continue`` runs the next one - "which failures cascade" is a
visible per-suite decision. The store validates the shape on every mutation
(the cross-store rules - the suite exists, the scripts exist, a non-empty
suite cannot be deleted - live in the HTTP layer, which holds both stores);
a load keeps an entry that no longer validates (a tightened rule or a hand
edit must never destroy it), it is served and fails at run start.
Layout (``--test-suites`` overrides the file; ``Path.home()`` resolves the
same way on Linux, macOS and Windows)::
~/.pysim-simple-server/test_suites.json
Writes are atomic (temp file + fsync + ``os.replace``) and every mutation
holds an ``RLock``. Suites carry a stable uuid - never array indices.
"""
import copy
import json
import os
import re
import sys
import tempfile
import threading
import time
import uuid
from pathlib import Path
DEFAULT_DIR = '.pysim-simple-server'
DEFAULT_FILENAME = 'test_suites.json'
SCHEMA_VERSION = 1
ROLES = ('setup', 'member', 'teardown')
ON_FAIL_LEVELS = ('stop', 'continue')
MAX_SCRIPTS = 200
_ID_RE = re.compile(r'[0-9a-f]{32}')
def default_path():
"""The default store: ``~/.pysim-simple-server/test_suites.json``."""
return Path.home() / DEFAULT_DIR / DEFAULT_FILENAME
class TestSuiteError(Exception):
"""A refused mutation (validation, unknown id, ...)."""
class TestSuiteStore:
"""The test suite store. All public methods are thread-safe and return
copies - callers can never mutate the store's own state."""
def __init__(self, path=None):
self.path = Path(path) if path else default_path()
self._lock = threading.RLock()
self._suites = []
self._load()
# ------------------------------------------------------------------ disk
def _load(self):
try:
raw = self.path.read_text(encoding='utf-8')
except FileNotFoundError:
self._suites = []
return
except OSError as e:
sys.stderr.write('TESTSUITES: cannot read %s: %s\n' % (self.path, e))
self._suites = []
return
try:
data = json.loads(raw)
except ValueError as e:
# Never silently overwrite an unreadable store: keep the file for
# the user, start empty (the next save would destroy it).
bad = self.path.with_name(self.path.name + '.bad-%s'
% time.strftime('%Y%m%d-%H%M%S'))
try:
self.path.replace(bad)
sys.stderr.write('TESTSUITES: %s is not valid JSON (%s) - kept as '
'%s, starting empty\n' % (self.path, e, bad))
except OSError:
sys.stderr.write('TESTSUITES: %s is not valid JSON (%s)\n'
% (self.path, e))
self._suites = []
return
items = data.get('suites') if isinstance(data, dict) else None
if not isinstance(items, list):
items = []
loaded = []
for entry in items:
try:
s = self._normalise(entry, keep_meta=True)
except TestSuiteError as e:
sys.stderr.write('TESTSUITES: dropping an unreadable entry: %s\n' % e)
continue
# A shape-valid entry is kept even when a referenced script is
# gone: served, the editor shows the missing reference, and the
# HTTP layer refuses a save/run until it is fixed.
loaded.append(s)
self._suites = loaded
def _save(self):
data = {'version': SCHEMA_VERSION, 'suites': self._suites}
self.path.parent.mkdir(parents=True, exist_ok=True)
tmp_fd, tmp_name = tempfile.mkstemp(prefix=self.path.name + '.',
dir=str(self.path.parent))
try:
with os.fdopen(tmp_fd, 'w', encoding='utf-8') as fh:
json.dump(data, fh, indent=2, ensure_ascii=False)
fh.write('\n')
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp_name, self.path)
except Exception:
try:
os.unlink(tmp_name)
except OSError:
pass
raise
try:
dir_fd = os.open(str(self.path.parent), os.O_RDONLY)
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)
except OSError:
pass
# ------------------------------------------------------------- shaping
@staticmethod
def _script_entry(raw, index):
"""One member entry: the referenced script id, its role and the
per-member failure policy."""
if not isinstance(raw, dict):
raise TestSuiteError('script entry %d must be an object' % (index + 1))
sid = str(raw.get('script_id') or '').strip().lower()
if not _ID_RE.fullmatch(sid):
raise TestSuiteError('script entry %d: script_id must be a suite '
'uuid' % (index + 1))
role = str(raw.get('role') or 'member').strip().lower()
if role not in ROLES:
raise TestSuiteError('script entry %d: role must be one of %s'
% (index + 1, ', '.join(ROLES)))
on_fail = str(raw.get('on_fail') or 'stop').strip().lower()
if on_fail not in ON_FAIL_LEVELS:
raise TestSuiteError('script entry %d: on_fail must be one of %s'
% (index + 1, ', '.join(ON_FAIL_LEVELS)))
return {'script_id': sid, 'role': role, 'on_fail': on_fail}
def _check(self, raw):
"""The suite dict, validated (shape only - the cross-store rules live
in the HTTP layer): name, the roles-ordered script list."""
if not isinstance(raw, dict):
raise TestSuiteError('suite must be an object')
name = raw.get('name')
if name is None:
name = ''
if not isinstance(name, str):
raise TestSuiteError('suite name must be a string')
scripts_raw = raw.get('scripts')
if scripts_raw is None:
scripts_raw = []
if not isinstance(scripts_raw, list):
raise TestSuiteError('suite scripts must be a list')
if len(scripts_raw) > MAX_SCRIPTS:
raise TestSuiteError('a suite may hold at most %d scripts' % MAX_SCRIPTS)
entries = [self._script_entry(e, i) for i, e in enumerate(scripts_raw)]
seen = set()
for e in entries:
if e['script_id'] in seen:
raise TestSuiteError('a script appears once per suite: %s'
% e['script_id'])
seen.add(e['script_id'])
setups = [e for e in entries if e['role'] == 'setup']
teardowns = [e for e in entries if e['role'] == 'teardown']
if len(setups) > 1:
raise TestSuiteError('a suite has at most one setup script')
if len(teardowns) > 1:
raise TestSuiteError('a suite has at most one teardown script')
# The roles define the order: setup first, teardown last (members keep
# their relative order in between).
ordered = setups + [e for e in entries if e['role'] == 'member'] + teardowns
return {'name': name.strip(), 'require_adm': bool(raw.get('require_adm')),
'scripts': ordered}
def _normalise(self, entry, keep_meta=False):
"""A stored entry: the validated suite plus id and timestamps. With
``keep_meta`` the stored id/timestamps are preserved (a load), else
fresh ones are made (an add)."""
if not isinstance(entry, dict):
raise TestSuiteError('suite must be an object')
suite = self._check(entry)
now = time.time()
rid = str(entry.get('id') or '').strip().lower()
if not _ID_RE.fullmatch(rid):
rid = uuid.uuid4().hex
if keep_meta:
created = entry.get('created')
updated = entry.get('updated')
created = created if isinstance(created, (int, float)) else now
updated = updated if isinstance(updated, (int, float)) else created
else:
created = updated = now
return {'id': rid, 'name': suite['name'],
'require_adm': suite['require_adm'],
'scripts': suite['scripts'],
'created': created, 'updated': updated}
def _find(self, sid):
sid = str(sid or '').strip().lower()
for s in self._suites:
if s['id'] == sid:
return s
return None
# -------------------------------------------------------------- public
def list(self):
with self._lock:
return [copy.deepcopy(s) for s in self._suites]
def get(self, sid):
with self._lock:
s = self._find(sid)
return copy.deepcopy(s) if s else None
def find_by_name(self, name):
want = str(name or '').strip()
with self._lock:
for s in self._suites:
if s['name'] == want:
return copy.deepcopy(s)
return None
def add(self, suite):
with self._lock:
s = self._normalise(suite or {})
if self._find(s['id']):
# a caller-supplied id that is already taken never wins: ids
# must stay unique (the same guard the import uses)
s['id'] = uuid.uuid4().hex
self._suites.append(s)
self._save()
sys.stderr.write('TESTSUITES: added %s (%s, %d scripts)\n'
% (s['name'] or '(unnamed)', s['id'], len(s['scripts'])))
return copy.deepcopy(s)
def update(self, sid, fields):
"""Partial or full update: name/require_adm/scripts replaced, id and
created kept. The order/role rules are re-applied (setup first,
teardown last)."""
with self._lock:
cur = self._find(sid)
if cur is None:
return None
merged = {'name': cur['name'], 'require_adm': cur['require_adm'],
'scripts': copy.deepcopy(cur['scripts'])}
if isinstance(fields, dict):
for key in ('name', 'require_adm', 'scripts'):
if key in fields:
merged[key] = fields[key]
suite = self._check(merged)
new = {'id': cur['id'], 'name': suite['name'],
'require_adm': suite['require_adm'],
'scripts': suite['scripts'],
'created': cur['created'], 'updated': time.time()}
self._suites[self._suites.index(cur)] = new
self._save()
return copy.deepcopy(new)
def remove(self, sid):
"""Remove a suite. The HTTP layer refuses this while it still holds
scripts (the caller checks first); the store itself deletes."""
with self._lock:
s = self._find(sid)
if s is None:
return False
self._suites.remove(s)
self._save()
sys.stderr.write('TESTSUITES: removed %s (%s)\n'
% (s['name'] or '(unnamed)', s['id']))
return True
def import_suites(self, items, mode='merge'):
"""Import exported suites: entries are validated one by one; an
already-used id gets a fresh one, invalid entries are reported instead
of failing the whole import. The caller remaps script references (a
bundle import that had to regenerate a script's uuid rewrites the
references before this call). ``replace`` wipes the store first."""
added, skipped, errors = 0, 0, []
with self._lock:
if mode == 'replace':
self._suites = []
for i, entry in enumerate(items or []):
if not isinstance(entry, dict):
skipped += 1
continue
try:
s = self._normalise(entry)
if self._find(s['id']):
s['id'] = uuid.uuid4().hex
except TestSuiteError as e:
skipped += 1
errors.append('%s: %s' % (entry.get('name') or ('#' + str(i + 1)), e))
continue
self._suites.append(s)
added += 1
if added or mode == 'replace':
self._save()
sys.stderr.write('TESTSUITES: import %s: %d added, %d skipped\n'
% (mode, added, skipped))
return {'added': added, 'skipped': skipped, 'errors': errors[:10]}
def info(self):
with self._lock:
return {'path': str(self.path), 'count': len(self._suites),
'version': SCHEMA_VERSION}
+228 -6
View File
@@ -13,6 +13,7 @@ import json
import pathlib
import re
import threading
import time
import unittest
import urllib.error
import urllib.request
@@ -1158,6 +1159,7 @@ class TestScriptStoreHttpTests(unittest.TestCase):
def setUp(self):
import tempfile
from pysim_simple_server import test_scripts
from pysim_simple_server import test_suites
from pysim_simple_server import testscript
from pysim_simple_server import server as srv
self.srv = srv
@@ -1165,15 +1167,24 @@ class TestScriptStoreHttpTests(unittest.TestCase):
self.store = test_scripts.TestScriptStore(
pathlib.Path(self.tmp.name) / 'test_scripts.json',
validator=lambda raw: testscript.normalise_script(raw, srv._test_command_type))
self.suites = test_suites.TestSuiteStore(
pathlib.Path(self.tmp.name) / 'test_suites.json')
self.server = _build_http_server('127.0.0.1', 0, PysimHandler)
self.server.log_requests = False
self.server.app = None
self.server.sl = None
self.server.scc = object()
self.server.test_scripts = self.store
self.server.test_suites = self.suites
threading.Thread(target=self.server.serve_forever, daemon=True).start()
self.port = self.server.server_address[1]
def _suite(self, **kw):
"""A stored suite (the script store's scripts belong to one)."""
s = {'name': 'suite'}
s.update(kw)
return self.suites.add(s)
def tearDown(self):
self.server.shutdown()
self.server.server_close()
@@ -1210,20 +1221,29 @@ class TestScriptStoreHttpTests(unittest.TestCase):
def test_the_get_route_is_card_free_and_reports_the_store(self):
self.assertIn('/api/test/scripts', _CARD_FREE_GET)
self.assertIn('/api/test/suites', _CARD_FREE_GET)
with _CARD_LOCK:
status, resp = self._get('/api/test/scripts')
self.assertEqual(status, 200, resp)
self.assertEqual(resp['scripts'], [])
self.assertEqual(resp['path'], str(self.store.path))
status, resp = self._get('/api/test/suites')
self.assertEqual(status, 200, resp)
self.assertEqual(resp['suites'], [])
self.assertEqual(resp['path'], str(self.suites.path))
def test_create_update_delete_round_trip(self):
status, resp = self._post('/api/test/scripts', self._script())
suite = self._suite()
status, resp = self._post('/api/test/scripts',
self._script(suite_id=suite['id']))
self.assertEqual(status, 200, resp)
sid = resp['script']['id']
self.assertTrue(sid)
self.assertEqual(self.suites.get(suite['id'])['scripts'],
[{'script_id': sid, 'role': 'member', 'on_fail': 'stop'}])
status, resp = self._get('/api/test/scripts')
self.assertEqual(len(resp['scripts']), 1)
self.assertEqual(resp['scripts'][0]['id'], sid)
self.assertEqual(resp['scripts'][0]['suite_id'], suite['id'])
status, resp = self._post('/api/test/scripts/update',
{'id': sid, 'script': {'name': 'renamed'}})
@@ -1238,16 +1258,130 @@ class TestScriptStoreHttpTests(unittest.TestCase):
self.assertEqual(status, 200, resp)
self.assertTrue(resp['removed'])
self.assertEqual(self.store.list(), [])
# the suite's member list no longer references it (no dangling id)
self.assertEqual(self.suites.get(suite['id'])['scripts'], [])
def test_create_requires_a_suite(self):
status, resp = self._post('/api/test/scripts', self._script())
self.assertEqual(status, 400, resp)
self.assertIn('belongs to a suite', resp['error'])
status, resp = self._post('/api/test/scripts',
self._script(suite_id='deadbeefdeadbeefdeadbeefdeadbeef'))
self.assertEqual(status, 400, resp)
self.assertIn('suite_id of an existing suite', resp['error'])
self.assertEqual(self.store.list(), [])
def test_create_validates_with_the_engine(self):
status, resp = self._post('/api/test/scripts', {'name': 'x', 'steps': []})
suite = self._suite()
sid = suite['id']
status, resp = self._post('/api/test/scripts',
{'name': 'x', 'suite_id': sid, 'steps': []})
self.assertEqual(status, 400, resp)
self.assertIn('at least one step', resp['error'])
status, resp = self._post('/api/test/scripts', {'name': 'x', 'steps': [
{'type': 'action', 'kind': 'nonsense', 'params': {}}]})
status, resp = self._post('/api/test/scripts',
{'name': 'x', 'suite_id': sid, 'steps': [
{'type': 'action', 'kind': 'nonsense', 'params': {}}]})
self.assertEqual(status, 400, resp)
self.assertIn('unknown action kind', resp['error'])
def test_suite_crud_and_the_delete_guard(self):
status, resp = self._get('/api/test/suites')
self.assertEqual(status, 200, resp)
self.assertEqual(resp['suites'], [])
status, resp = self._post('/api/test/suites', {})
self.assertEqual(status, 200, resp)
sid = resp['suite']['id']
status, resp = self._post('/api/test/suites/update',
{'id': sid, 'suite': {'name': 'renamed',
'require_adm': True}})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['suite']['name'], 'renamed')
self.assertTrue(resp['suite']['require_adm'])
# a non-empty suite refuses deletion
status, resp = self._post('/api/test/scripts', self._script(suite_id=sid))
self.assertEqual(status, 200, resp)
status, resp = self._post('/api/test/suites/delete', {'id': sid})
self.assertEqual(status, 400, resp)
self.assertIn('still holds 1 script', resp['error'])
self._post('/api/test/scripts/delete', {'id': self.store.list()[0]['id']})
status, resp = self._post('/api/test/suites/delete', {'id': sid})
self.assertEqual(status, 200, resp)
self.assertTrue(resp['removed'])
def test_suite_update_refuses_a_membership_change(self):
a = self._suite(name='a')
b = self._suite(name='b')
status, resp = self._post('/api/test/scripts', self._script(suite_id=a['id']))
sid = resp['script']['id']
# the suite's own list may be reordered/retagged, but the member set is
# owned by the script endpoints
status, resp = self._post('/api/test/suites/update', {
'id': a['id'], 'suite': {'scripts': [
{'script_id': sid, 'role': 'setup', 'on_fail': 'stop'}]}})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['suite']['scripts'][0]['role'], 'setup')
other = self._suite()['id']
status, resp = self._post('/api/test/scripts', self._script(suite_id=other))
sid2 = resp['script']['id']
status, resp = self._post('/api/test/suites/update', {
'id': a['id'], 'suite': {'scripts': [
{'script_id': sid, 'role': 'member', 'on_fail': 'stop'},
{'script_id': sid2, 'role': 'member', 'on_fail': 'stop'}]}})
self.assertEqual(status, 400, resp)
self.assertIn('member set changes through the script endpoints', resp['error'])
def test_move_and_copy_between_suites(self):
a = self._suite(name='a')
b = self._suite(name='b')
status, resp = self._post('/api/test/scripts', self._script(suite_id=a['id']))
sid = resp['script']['id']
# a move keeps the on_fail policy and updates both suites
self.suites.update(a['id'], {'scripts': [
{'script_id': sid, 'role': 'member', 'on_fail': 'continue'}]})
status, resp = self._post('/api/test/scripts/move',
{'id': sid, 'suite_id': b['id']})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['script']['suite_id'], b['id'])
self.assertEqual(self.suites.get(a['id'])['scripts'], [])
self.assertEqual(self.suites.get(b['id'])['scripts'],
[{'script_id': sid, 'role': 'member', 'on_fail': 'continue'}])
# a copy lands in the target as a member with a fresh id
status, resp = self._post('/api/test/scripts/copy',
{'id': sid, 'suite_id': a['id'], 'name': 'cloned'})
self.assertEqual(status, 200, resp)
copy_id = resp['script']['id']
self.assertNotEqual(copy_id, sid)
self.assertEqual(resp['script']['name'], 'cloned')
self.assertEqual(self.suites.get(a['id'])['scripts'][0]['script_id'], copy_id)
self.assertEqual(len(self.store.list()), 2)
def test_bundle_import_remaps_script_references(self):
# an exported bundle: one suite + its script
s = self.store.add({'name': 'one', 'steps': [
{'type': 'action', 'kind': 'status', 'params': {'attempts': 1}}]})
bundle_suite = self.suites.add({'name': 'bundle', 'scripts': [
{'script_id': s['id'], 'role': 'setup', 'on_fail': 'stop'}]})
bundle = {'suites': [dict(bundle_suite)], 'scripts': [dict(self.store.get(s['id']))]}
# importing into the same store duplicates both (new ids), and the
# suite references the copies
status, resp = self._post('/api/test/suites/import', bundle)
self.assertEqual(status, 200, resp)
self.assertEqual(resp['scripts']['added'], 1)
self.assertEqual(resp['added'], 1)
new_scripts = [x for x in self.store.list() if x['id'] != s['id']]
self.assertEqual(len(new_scripts), 1)
new_suite = [x for x in self.suites.list() if x['id'] != bundle_suite['id']][0]
self.assertEqual(new_suite['scripts'][0]['script_id'], new_scripts[0]['id'])
self.assertEqual(new_suite['scripts'][0]['role'], 'setup')
self.assertEqual(new_scripts[0]['suite_id'], new_suite['id'])
# a reference that resolves to nothing is reported, not imported
status, resp = self._post('/api/test/suites/import', {
'suites': [{'name': 'bad', 'scripts': [
{'script_id': 'a' * 32, 'role': 'member', 'on_fail': 'stop'}]}]})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['added'], 0)
self.assertIn('unknown script', ' '.join(resp.get('errors') or []))
def test_import_accepts_the_old_localstorage_export(self):
status, resp = self._post('/api/test/scripts/import', {'scripts': [
{'name': 'A', 'steps': [{'type': 'action', 'kind': 'status',
@@ -1260,25 +1394,113 @@ class TestScriptStoreHttpTests(unittest.TestCase):
self.assertEqual(self.store.list()[0]['name'], 'A')
status, resp = self._post('/api/test/scripts/import', {'scripts': 'nope'})
self.assertEqual(status, 400, resp)
# an import into a named suite attaches the imported scripts there
suite = self._suite(name='target')
status, resp = self._post('/api/test/scripts/import', {
'suite_id': suite['id'],
'scripts': [{'name': 'B', 'steps': [
{'type': 'action', 'kind': 'status', 'params': {'attempts': 1}}]}]})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['added'], 1)
self.assertEqual(len(self.suites.get(suite['id'])['scripts']), 1)
def test_the_run_rejects_an_unknown_script_id(self):
status, resp = self._post('/api/test/run', {'script_id': 'nope'})
self.assertEqual(status, 404, resp)
self.assertIn('not found', resp['error'])
def test_suite_run_starts_and_reports_members(self):
suite = self._suite(name='regression')
status, resp = self._post('/api/test/scripts', self._script(suite_id=suite['id']))
self.assertEqual(status, 200, resp)
status, resp = self._post('/api/test/run', {'suite_id': suite['id']})
self.assertEqual(status, 200, resp)
self.assertIsNotNone(resp['suite'])
self.assertEqual(resp['suite']['name'], 'regression')
self.assertEqual(len(resp['suite']['members']), 1)
self.assertEqual(resp['suite']['members'][0]['role'], 'member')
snap = resp
for _ in range(100):
status, snap = self._get('/api/test/status')
if not snap['running']:
break
time.sleep(0.05)
self.assertFalse(snap['running'], snap)
# the fake scc has no transport: the member fails, the report closes
self.assertEqual(len(snap['suite']['members']), 1)
self.assertIn(snap['suite']['members'][0]['status'],
('ok', 'warning', 'error'))
self.assertIsNotNone(snap['suite']['summary'])
self.assertIn('wall_ms', snap['suite']['summary'])
# the report carries the run-log buffer and the member's slice
self.assertIn('log', snap)
self.assertIn('steps', snap['suite']['members'][0])
def test_suite_run_refuses_missing_and_unknown(self):
status, resp = self._post('/api/test/run',
{'suite_id': 'deadbeefdeadbeefdeadbeefdeadbeef'})
self.assertEqual(status, 404, resp)
self.assertIn('unknown test suite', resp['error'])
suite = self._suite(name='empty')
status, resp = self._post('/api/test/run', {'suite_id': suite['id']})
self.assertEqual(status, 400, resp)
self.assertIn('no scripts', resp['error'])
# a shape-valid but dangling reference (a hand edit) is refused
status, resp = self._post('/api/test/scripts', self._script(suite_id=suite['id']))
self.assertEqual(status, 200, resp)
sid = resp['script']['id']
self.suites.update(suite['id'], {'scripts': [
{'script_id': 'f' * 32, 'role': 'member', 'on_fail': 'stop'}]})
status, resp = self._post('/api/test/run', {'suite_id': suite['id']})
self.assertEqual(status, 400, resp)
self.assertIn('unknown test script', resp['error'])
# restore: run with the real member is fine
self.suites.update(suite['id'], {'scripts': [
{'script_id': sid, 'role': 'member', 'on_fail': 'stop'}]})
def test_suite_run_adm_gate_refuses_without_a_key(self):
self.server.app = object() # the gate needs a card (app + scc)
suite = self._suite(name='gated', require_adm=True)
status, resp = self._post('/api/test/scripts', self._script(suite_id=suite['id']))
self.assertEqual(status, 200, resp)
status, resp = self._post('/api/test/run', {
'suite_id': suite['id'], 'preset': {'name': 'no-adm'}})
self.assertEqual(status, 400, resp)
self.assertIn('no ADM key', resp['error'])
# a failed verification is not retried automatically
with mock.patch.object(self.srv, '_verify_adm',
return_value={'ok': False, 'sw': '63C2',
'attempts_left': 3}):
status, resp = self._post('/api/test/run', {
'suite_id': suite['id'], 'preset': {'name': 'with-adm', 'adm': 'AABBCCDD'}})
self.assertEqual(status, 400, resp)
self.assertIn('63C2', resp['error'])
self.assertIn('3 attempt(s) left', resp['error'])
with mock.patch.object(self.srv, '_verify_adm') as v:
status, resp = self._post('/api/test/run', {
'suite_id': suite['id'], 'preset': {'name': 'with-adm', 'adm': 'AABBCCDD'}})
self.assertEqual(status, 400, resp)
self.assertIn('verify it manually', resp['error'])
v.assert_not_called()
def test_a_store_write_failure_answers_a_json_500(self):
suite = self._suite()
with mock.patch.object(self.store, 'add',
side_effect=OSError('read-only file system')):
status, resp = self._post('/api/test/scripts', self._script())
status, resp = self._post('/api/test/scripts',
self._script(suite_id=suite['id']))
self.assertEqual(status, 500, resp)
self.assertIn('test script store write failed', resp['error'])
def test_the_store_endpoints_answer_503_without_a_store(self):
self.server.test_scripts = None
self.server.test_suites = None
status, resp = self._get('/api/test/scripts')
self.assertEqual(status, 503, resp)
status, resp = self._post('/api/test/scripts', self._script())
self.assertEqual(status, 503, resp)
status, resp = self._get('/api/test/suites')
self.assertEqual(status, 503, resp)
if __name__ == '__main__':
+64
View File
@@ -221,3 +221,67 @@ class StoreInfoTests(unittest.TestCase):
self.assertEqual(info['path'], str(store.path))
self.assertEqual(info['count'], 1)
self.assertEqual(info['version'], test_scripts.SCHEMA_VERSION)
class SuiteAttachmentTests(unittest.TestCase):
"""v3.22.0: a script belongs to a suite (suite_id), declares require_adm,
and the import reports the ids a bundle import needs."""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.path = pathlib.Path(self.tmp.name) / 'test_scripts.json'
self.store = test_scripts.TestScriptStore(self.path)
def tearDown(self):
self.tmp.cleanup()
def test_suite_id_and_require_adm_round_trip(self):
sid = 'a' * 32
s = self.store.add(dict(script('one'), suite_id=sid, require_adm=True))
self.assertEqual(s['suite_id'], sid)
self.assertTrue(s['require_adm'])
got = test_scripts.TestScriptStore(self.path).get(s['id'])
self.assertEqual(got['suite_id'], sid)
self.assertTrue(got['require_adm'])
with self.assertRaises(test_scripts.TestScriptError):
self.store.add(dict(script('two'), suite_id='short'))
def test_orphans_and_list_by_suite(self):
sid = 'a' * 32
s1 = self.store.add(dict(script('one'), suite_id=sid))
s2 = self.store.add(script('two'))
self.assertEqual([x['id'] for x in self.store.list_by_suite(sid)], [s1['id']])
self.assertEqual([x['id'] for x in self.store.orphans()], [s2['id']])
# a move is an update
moved = self.store.update(s2['id'], {'suite_id': sid})
self.assertEqual(moved['suite_id'], sid)
self.assertEqual(self.store.orphans(), [])
self.assertEqual(len(self.store.list_by_suite(sid)), 2)
def test_import_reports_the_id_map_and_attaches_to_a_suite(self):
sid = 'a' * 32
s = self.store.add(script('one'))
result = self.store.import_scripts([self.store.get(s['id'])])
self.assertEqual(result['added'], 1)
self.assertEqual(result['id_map'][s['id']], result['added_ids'][0])
self.assertNotEqual(result['added_ids'][0], s['id'])
# import into a named suite: every added script belongs to it
result = self.store.import_scripts(
[script('two'), script('three')], suite_id=sid)
self.assertEqual(result['added'], 2)
for sid_added in result['added_ids']:
self.assertEqual(self.store.get(sid_added)['suite_id'], sid)
def test_a_v1_store_loads_with_no_suite(self):
self.store.add(script('one'))
data = json.loads(self.path.read_text())
data['version'] = 1
for e in data['scripts']:
e.pop('suite_id', None)
e.pop('require_adm', None)
self.path.write_text(json.dumps(data))
again = test_scripts.TestScriptStore(self.path)
got = again.list()[0]
self.assertIsNone(got['suite_id'])
self.assertFalse(got['require_adm'])
self.assertEqual(len(again.orphans()), 1)
+227
View File
@@ -0,0 +1,227 @@
"""Test suite store tests (server-side suites, v3.22.0).
A suite groups the test scripts: the root object, holding the ordered member
list by reference (setup first, members, teardown last - the store
normalises the order), the per-member on_fail policy and the suite-level
require_adm flag. The cross-store rules live in the HTTP layer; this file
covers the store's own shape, ordering, persistence and import behaviour.
"""
import json
import pathlib
import tempfile
import unittest
from pysim_simple_server import test_suites
SID_A = 'a' * 32
SID_B = 'b' * 32
SID_C = 'c' * 32
def make_store(tmp):
return test_suites.TestSuiteStore(pathlib.Path(tmp) / 'test_suites.json')
def suite(name='demo', scripts=None, **kw):
s = {'name': name, 'scripts': scripts if scripts is not None else []}
s.update(kw)
return s
class StoreCrudTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.store = make_store(self.tmp.name)
def tearDown(self):
self.tmp.cleanup()
def test_add_list_get_and_remove(self):
s = self.store.add(suite())
self.assertEqual(len(s['id']), 32)
self.assertEqual(s['name'], 'demo')
self.assertEqual(s['require_adm'], False)
self.assertEqual(self.store.list()[0]['id'], s['id'])
self.assertTrue(self.store.remove(s['id']))
self.assertEqual(self.store.list(), [])
self.assertFalse(self.store.remove(s['id']))
self.assertIsNone(self.store.get('nope'))
def test_callers_never_mutate_the_store(self):
s = self.store.add(suite(scripts=[
{'script_id': SID_A, 'role': 'member', 'on_fail': 'stop'}]))
s['scripts'][0]['on_fail'] = 'continue'
s['name'] = 'hacked'
self.assertEqual(self.store.get(s['id'])['name'], 'demo')
self.assertEqual(self.store.get(s['id'])['scripts'][0]['on_fail'], 'stop')
def test_roles_are_ordered_setup_first_teardown_last(self):
# a caller may send any order; the store normalises it and marks the
# roles - setup first, teardown last, members keep their order
s = self.store.add(suite(scripts=[
{'script_id': SID_C, 'role': 'teardown', 'on_fail': 'stop'},
{'script_id': SID_A, 'role': 'member', 'on_fail': 'continue'},
{'script_id': SID_B, 'role': 'member', 'on_fail': 'stop'},
{'script_id': SID_A.replace('a', 'd'), 'role': 'setup', 'on_fail': 'stop'},
]))
ids = [e['script_id'] for e in s['scripts']]
self.assertEqual(ids[0], 'd' * 32)
self.assertEqual(ids[-1], SID_C)
self.assertEqual(ids[1:3], [SID_A, SID_B])
roles = [e['role'] for e in s['scripts']]
self.assertEqual(roles, ['setup', 'member', 'member', 'teardown'])
self.assertEqual(s['scripts'][1]['on_fail'], 'continue')
def test_shape_is_validated(self):
with self.assertRaises(test_suites.TestSuiteError):
self.store.add('nope')
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts='nope'))
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[{'script_id': 'short'}]))
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[{'script_id': SID_A, 'role': 'boss'}]))
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[{'script_id': SID_A, 'on_fail': 'maybe'}]))
# one script appears once per suite
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[
{'script_id': SID_A}, {'script_id': SID_A}]))
# at most one setup / teardown
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[
{'script_id': SID_A, 'role': 'setup'},
{'script_id': SID_B, 'role': 'setup'}]))
with self.assertRaises(test_suites.TestSuiteError):
self.store.add(suite(scripts=[
{'script_id': SID_A, 'role': 'teardown'},
{'script_id': SID_B, 'role': 'teardown'}]))
def test_update_keeps_id_and_created(self):
s = self.store.add(suite(name='one'))
updated = self.store.update(s['id'], {
'name': 'two', 'require_adm': True,
'scripts': [{'script_id': SID_A, 'role': 'member', 'on_fail': 'stop'}]})
self.assertEqual(updated['id'], s['id'])
self.assertEqual(updated['created'], s['created'])
self.assertEqual(updated['name'], 'two')
self.assertTrue(updated['require_adm'])
self.assertEqual(len(updated['scripts']), 1)
self.assertIsNone(self.store.update('nope', {'name': 'x'}))
def test_find_by_name(self):
self.store.add(suite(name='alpha'))
self.assertEqual(self.store.find_by_name('alpha')['name'], 'alpha')
self.assertIsNone(self.store.find_by_name('beta'))
class StorePersistenceTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.path = pathlib.Path(self.tmp.name) / 'test_suites.json'
self.store = test_suites.TestSuiteStore(self.path)
def tearDown(self):
self.tmp.cleanup()
def test_round_trip_keeps_ids_and_timestamps(self):
s = self.store.add(suite(require_adm=True, scripts=[
{'script_id': SID_A, 'role': 'member', 'on_fail': 'stop'}]))
again = test_suites.TestSuiteStore(self.path)
got = again.get(s['id'])
self.assertEqual(got['name'], s['name'])
self.assertEqual(got['created'], s['created'])
self.assertTrue(got['require_adm'])
self.assertEqual(got['scripts'], s['scripts'])
def test_the_file_carries_the_schema_version(self):
self.store.add(suite())
data = json.loads(self.path.read_text())
self.assertEqual(data['version'], test_suites.SCHEMA_VERSION)
def test_an_entry_with_a_dangling_reference_is_kept(self):
# a referenced script removed by hand (or an import that lost it): the
# suite keeps loading - it is served (the editor shows the missing
# reference) and the HTTP layer refuses a save/run until it is fixed
self.store.add(suite(scripts=[{'script_id': SID_A, 'role': 'member',
'on_fail': 'stop'}]))
again = test_suites.TestSuiteStore(self.path)
self.assertEqual(len(again.list()), 1)
self.assertEqual(again.list()[0]['scripts'][0]['script_id'], SID_A)
def test_a_shape_invalid_entry_is_dropped_with_a_log(self):
# an unreadable entry (a malformed script id) is not servable: unlike
# the engine-validated scripts, the store's own shape check drops it
self.store.add(suite())
raw = json.loads(self.path.read_text())
raw['suites'][0]['scripts'] = [{'script_id': 'short', 'role': 'member',
'on_fail': 'stop'}]
self.path.write_text(json.dumps(raw))
again = test_suites.TestSuiteStore(self.path)
self.assertEqual(again.list(), [])
class StoreImportTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.store = make_store(self.tmp.name)
def tearDown(self):
self.tmp.cleanup()
def test_import_merge_and_replace(self):
self.store.add(suite(name='old'))
result = self.store.import_suites([suite(name='new', scripts=[
{'script_id': SID_A, 'role': 'member', 'on_fail': 'stop'}])])
self.assertEqual(result['added'], 1)
self.assertEqual(result['skipped'], 0)
self.assertEqual([s['name'] for s in self.store.list()], ['old', 'new'])
result = self.store.import_suites([suite(name='third')], 'replace')
self.assertEqual(result['added'], 1)
self.assertEqual([s['name'] for s in self.store.list()], ['third'])
def test_import_keeps_a_given_id_and_reassigns_a_duplicate(self):
s = self.store.add(suite(name='one'))
entry = dict(s, name='copy')
result = self.store.import_suites([entry])
self.assertEqual(result['added'], 1)
ids = sorted(x['id'] for x in self.store.list())
self.assertEqual(len(set(ids)), 2)
self.assertIn(s['id'], ids)
def test_import_reports_invalid_entries(self):
result = self.store.import_suites([suite(name='bad', scripts=[
{'script_id': 'nope'}]), suite(name='ok')])
self.assertEqual(result['added'], 1)
self.assertEqual(result['skipped'], 1)
self.assertTrue(result['errors'])
class MigrationTests(unittest.TestCase):
"""v3.21.0 -> v3.22.0: scripts without a suite are attached to an
auto-created "Imported scripts" suite, once (idempotent)."""
def test_orphans_are_attached_to_the_imported_suite(self):
from pysim_simple_server import server as srv
from pysim_simple_server import test_scripts
with tempfile.TemporaryDirectory() as tmp:
scripts = test_scripts.TestScriptStore(pathlib.Path(tmp) / 's.json')
suites = test_suites.TestSuiteStore(pathlib.Path(tmp) / 'u.json')
step = {'type': 'action', 'kind': 'status', 'params': {'attempts': 1}}
a = scripts.add({'name': 'a', 'steps': [step]})
b = scripts.add({'name': 'b', 'steps': [step]})
self.assertEqual(srv.migrate_scripts_to_suites(scripts, suites), 2)
suite = suites.find_by_name('Imported scripts')
self.assertIsNotNone(suite)
self.assertEqual([e['script_id'] for e in suite['scripts']],
[a['id'], b['id']])
self.assertEqual([e['role'] for e in suite['scripts']],
['member', 'member'])
self.assertEqual(scripts.get(a['id'])['suite_id'], suite['id'])
# idempotent: nothing left to attach
self.assertEqual(srv.migrate_scripts_to_suites(scripts, suites), 0)
if __name__ == '__main__':
unittest.main()
+251 -2
View File
@@ -404,6 +404,7 @@ class FakeServer:
self.sms_oa = '12345'
self.sms_sc = '12345678912'
self.card_session = 1
self.card_present = True
self.stk_pending = None
self.menu_active = False
@@ -428,7 +429,8 @@ class RunnerTestCase(unittest.TestCase):
S._TEST_RUN.update({'running': True, 'stop': stop, 'name': 'test',
'status': None, 'steps': [], 'index': 0,
'total': len(script['steps']), 'scp80_counter': None,
'started': time.time(), 'finished': None, 'error': None})
'started': time.time(), 'finished': None, 'error': None,
'suite': None, 'log': [], 'log_prefix': '', 'adm': None})
S._TEST_RUNNING = True
try:
S._test_run_execute(server, script, preset or {})
@@ -1022,7 +1024,8 @@ class TestRunnerActions(RunnerTestCase):
self.assertIsNone(err)
# script_id alone resolves from the store
raw, sid, err = S._test_run_script_from_body(server, {'script_id': entry['id']})
self.assertEqual(raw, {'name': 'stored', 'steps': entry['steps']})
self.assertEqual(raw, {'name': 'stored', 'steps': entry['steps'],
'require_adm': False})
self.assertEqual(sid, entry['id'])
self.assertIsNone(err)
raw, sid, err = S._test_run_script_from_body(server, {'script_id': 'nope'})
@@ -1485,5 +1488,251 @@ class TestRunnerActions(RunnerTestCase):
self.assertEqual(S._increment_counter_hex(''), '')
OK_STEP = {'type': 'action', 'kind': 'apdu', 'params': {'apdu': '00A40000023F00'},
'check': {'sw': '9000'}}
BAD_STEP = {'type': 'action', 'kind': 'apdu', 'params': {'apdu': '00B0000000'},
'check': {'sw': '9000'}} # no queue -> 6D00
class SuiteRunnerTestCase(RunnerTestCase):
"""Suite runner tests (v3.22.0): setup -> members (per-member on_fail) ->
teardown, one card session, no resume."""
def members(self, specs):
"""specs: [(role, ok, on_fail)] -> the resolved member list."""
out = []
for i, spec in enumerate(specs):
role, ok = spec[0], spec[1]
on_fail = spec[2] if len(spec) > 2 else 'stop'
step = dict(OK_STEP if ok else BAD_STEP)
out.append(({'script_id': '%032x' % (i + 1), 'role': role,
'on_fail': on_fail},
T.normalise_script({'name': 'script %d' % (i + 1),
'steps': [step]}, S._test_command_type)))
return out
def suite_run(self, server, members, suite=None, preset=None, preset_id=None):
suite = suite or {'name': 'suite'}
S._TEST_RUN.clear()
S._TEST_RUN.update({'running': True, 'stop': False, 'name': 'suite',
'status': None, 'session': server.card_session,
'index': 0, 'total': 0, 'steps': [],
'started': time.time(), 'finished': None, 'error': None,
'scp80_counter': None, 'scp80_counters': {},
'script_id': None, 'preset': None,
'suite': S._test_suite_state(suite, members,
preset or {}, preset_id,
server.card_session),
'log': [], 'log_prefix': '', 'adm': None})
S._TEST_RUNNING = True
try:
S._test_suite_execute(server, suite, members, preset or {}, preset_id)
finally:
S._TEST_RUNNING = False
return S._TEST_RUN
def push_ok(self, scc, n):
for _ in range(n):
scc.push('00A4', '', '9000')
class TestSuiteRunner(SuiteRunnerTestCase):
def test_setup_members_teardown_run_in_order(self):
scc = FakeScc()
self.push_ok(scc, 4)
members = self.members([('setup', True), ('member', True),
('member', True), ('teardown', True)])
run = self.suite_run(FakeServer(scc), members)
self.assertEqual(run['status'], 'ok')
state = run['suite']
self.assertEqual([m['role'] for m in state['members']],
['setup', 'member', 'member', 'teardown'])
self.assertEqual([m['status'] for m in state['members']],
['ok', 'ok', 'ok', 'ok'])
self.assertEqual(state['summary']['setup'], 'ok')
self.assertEqual(state['summary']['teardown'], 'ok')
self.assertEqual(state['summary']['ok'], 4)
self.assertEqual(state['summary']['error'], 0)
self.assertFalse(state['summary']['stopped'])
self.assertEqual(state['summary']['counters_before'], {})
# the member steps are copied into the report (the live view is reset
# between members)
self.assertEqual(len(state['members'][1]['steps']), 1)
self.assertEqual(state['members'][1]['steps'][0]['status'], 'ok')
# the run log carries the member prefix and is sliced per member
joined = '\n'.join(state['members'][1]['log'])
self.assertIn('[2/4 member script 2]', joined)
def test_on_fail_stop_skips_the_rest_and_still_runs_teardown(self):
scc = FakeScc()
self.push_ok(scc, 1) # setup ok
self.push_ok(scc, 1) # first member ok
# the failing member gets no queue entry -> 6D00 (error)
self.push_ok(scc, 1) # teardown ok
members = self.members([('setup', True), ('member', True),
('member', False, 'stop'), ('member', True),
('teardown', True)])
run = self.suite_run(FakeServer(scc), members)
self.assertEqual(run['status'], 'error')
statuses = [m['status'] for m in run['suite']['members']]
self.assertEqual(statuses, ['ok', 'ok', 'error', 'skipped', 'ok'])
self.assertEqual(run['suite']['members'][3]['note'],
'a previous script failed (on_fail: stop)')
# no member ran after the error: the queue holds exactly the teardown's
self.assertEqual(sum(1 for a in scc.sent if a.startswith('00A4')), 3)
def test_on_fail_continue_runs_the_next_member(self):
scc = FakeScc()
self.push_ok(scc, 1) # member 1 ok
# member 2 fails (no queue entry)
self.push_ok(scc, 1) # member 3 ok
members = self.members([('member', True), ('member', False, 'continue'),
('member', True)])
run = self.suite_run(FakeServer(scc), members)
self.assertEqual(run['status'], 'error') # the failure is reported
self.assertEqual([m['status'] for m in run['suite']['members']],
['ok', 'error', 'ok'])
def test_setup_failure_skips_members_but_runs_teardown(self):
scc = FakeScc()
# setup fails (no queue), members never run, teardown ok
self.push_ok(scc, 1)
members = self.members([('setup', False), ('member', True),
('teardown', True)])
run = self.suite_run(FakeServer(scc), members)
self.assertEqual([m['status'] for m in run['suite']['members']],
['error', 'skipped', 'ok'])
self.assertEqual(run['suite']['members'][1]['note'],
'the setup script failed')
def test_stop_skips_the_members_and_runs_the_teardown(self):
scc = FakeScc()
self.push_ok(scc, 1) # setup ok (runs first)
self.push_ok(scc, 1) # teardown ok
members = self.members([('setup', True), ('member', True),
('teardown', True)])
server = FakeServer(scc)
real = S._test_run_script
calls = []
def wrapped(srv, script, preset):
ctx, stopped = real(srv, script, preset)
calls.append(script['name'])
if srv is server and len(calls) == 1:
S._TEST_RUN['stop'] = True # the operator stops here
return ctx, stopped
with mock.patch.object(S, '_test_run_script', wrapped):
run = self.suite_run(server, members)
statuses = [m['status'] for m in run['suite']['members']]
self.assertEqual(statuses, ['ok', 'skipped', 'ok'])
self.assertTrue(run['suite']['summary']['stopped'])
self.assertEqual(run['status'], 'stopped')
def test_a_card_reset_stops_the_suite_and_flags_it(self):
scc = FakeScc()
self.push_ok(scc, 1) # member 1 ok
self.push_ok(scc, 1) # teardown ok (new session)
members = self.members([('member', True), ('member', True),
('teardown', True)])
server = FakeServer(scc)
real = S._test_run_script
def wrapped(srv, script, preset):
ctx, stopped = real(srv, script, preset)
if srv is server and len(scc.sent) <= 1:
srv.card_session += 1 # the card reset mid-suite
return ctx, stopped
with mock.patch.object(S, '_test_run_script', wrapped):
run = self.suite_run(server, members)
state = run['suite']
self.assertEqual([m['status'] for m in state['members']],
['ok', 'skipped', 'ok'])
self.assertEqual(state['members'][1]['note'],
'card session changed (card removed or re-equipped)')
self.assertTrue(state['session_changed'])
self.assertEqual(state['session_start'], 1)
self.assertEqual(state['session_end'], 2)
self.assertTrue(state['summary']['session_changed'])
def test_teardown_is_skipped_when_the_card_is_gone(self):
scc = FakeScc()
self.push_ok(scc, 1)
members = self.members([('member', True), ('teardown', True)])
server = FakeServer(scc)
server.card_present = False
run = self.suite_run(server, members)
self.assertEqual([m['status'] for m in run['suite']['members']],
['ok', 'skipped'])
self.assertEqual(run['suite']['members'][1]['note'],
'no card in the reader')
class TestAdmGate(unittest.TestCase):
"""The run prerequisite: a script/suite flagged `require_adm` runs only
with the ADM verified for the current card session - the preset's key is
tried once, never retried automatically."""
def setUp(self):
self.server = FakeServer(FakeScc())
self.server.app = object()
def test_not_needed_passes(self):
ok, info, err = S._ensure_adm(self.server, {}, False)
self.assertTrue(ok)
self.assertIsNone(err)
self.assertFalse(info['required'])
def test_verified_latch_passes(self):
self.server.adm_session = 1
ok, info, err = S._ensure_adm(self.server, {}, True)
self.assertTrue(ok)
self.assertTrue(info['verified'])
def test_verifies_once_with_the_preset_key(self):
with mock.patch.object(S, '_verify_adm', return_value={'ok': True,
'sw': '9000'}) as v:
ok, info, err = S._ensure_adm(self.server, {'adm': 'AABBCCDD'}, True)
self.assertTrue(ok, err)
self.assertTrue(info['verified'])
self.assertTrue(info['checked'])
self.assertEqual(self.server.adm_session, self.server.card_session)
v.assert_called_once()
# the second gate in the same session does not verify again
with mock.patch.object(S, '_verify_adm', return_value={'ok': True}) as v2:
ok, _info, _err = S._ensure_adm(self.server, {'adm': 'AABBCCDD'}, True)
self.assertTrue(ok)
v2.assert_not_called()
def test_a_failure_is_not_retried_automatically(self):
with mock.patch.object(S, '_verify_adm', return_value={
'ok': False, 'sw': '63C2', 'attempts_left': 2}) as v:
ok, info, err = S._ensure_adm(self.server, {'adm': 'AABBCCDD'}, True)
self.assertFalse(ok)
self.assertIn('63C2', err)
self.assertIn('2 attempt(s) left', err)
self.assertEqual(info['attempts_left'], 2)
self.assertEqual(self.server.adm_failed_session, self.server.card_session)
# the next gate refuses without touching the card again
with mock.patch.object(S, '_verify_adm') as v2:
ok, _info, err = S._ensure_adm(self.server, {'adm': 'AABBCCDD'}, True)
self.assertFalse(ok)
self.assertIn('verify it manually', err)
v2.assert_not_called()
def test_no_key_in_the_preset_refuses(self):
ok, _info, err = S._ensure_adm(self.server, {}, True)
self.assertFalse(ok)
self.assertIn('no ADM key', err)
def test_a_new_session_clears_the_failure_stamp(self):
self.server.adm_failed_session = 1
self.server.card_session = 2
with mock.patch.object(S, '_verify_adm', return_value={'ok': True}):
ok, _info, _err = S._ensure_adm(self.server, {'adm': 'AABBCCDD'}, True)
self.assertTrue(ok)
if __name__ == '__main__':
unittest.main()