v3.22.0: the test suites (scripts grouped, ordered, with setup/teardown)

A suite is the root object: every script belongs to exactly one suite, the
suite holds the ordered member list by reference - the optional setup first,
the members, the optional teardown last - and runs them in one card session,
without resets and without resume.

- test_suites.py: the suite store (test_suites.json, --test-suites), validated
  and atomic like the script store: roles normalised (setup first, teardown
  last, at most one of each, a script appears once per suite), per-member
  on_fail stop/continue and the suite-level require_adm.
- test_scripts.py schema v2: suite_id/require_adm; list_by_suite()/orphans();
  the import reports the id_map a bundle import remaps with.  v3.21.0 scripts
  without a suite are attached once to an auto-created "Imported scripts"
  suite (migrate_scripts_to_suites, called from __main__).
- API: /api/test/suites (+ update|delete|import - delete refused while
  non-empty; the bundle import imports the scripts first, remaps the suite
  references and skips unresolvable ones); the scripts endpoints gain
  suite_id/require_adm plus move|copy (delete detaches from the suite first;
  a suite update refuses a membership change).
- runner: /api/test/run {suite_id} resolves the members and the preset, runs
  the ADM gate, then setup -> members (per-member on_fail) -> teardown.  The
  teardown always runs on a stop or a failure (skipped only when the card is
  gone; it clears the stop flag for its duration); a card reset stops the
  suite, skips the remaining members and is flagged.  Members' live steps land
  in the shared snapshot, their records (status/steps/note/log excerpt/
  counters) in _TEST_RUN['suite']; the summary carries the counts, wall time,
  preset, the SCP80 counters before/after per keyset and the session flags.
  _test_log buffers the semantic lines (bounded) for the report's per-member
  excerpts.
- ADM gate (_ensure_adm): a suite/script require_adm verifies the matched
  preset's key once per card session (the /api/verify-adm badge sets the same
  latch); a failure refuses the start, stamps the session (no automatic
  retry) and needs a manual verify.
- PWA: the Test script pill becomes four views - the suite list (default),
  the suite editor (the script list: role badges, on_fail, up/down, per-script
  Run/Edit/Move/Copy/Delete, Add script with the templates), the script editor
  (the step list; a single run shows its result under the steps and is not
  preserved) and the suite report (per-member blocks with the step table,
  notes and run log; summary; export JSON/Markdown).  Suite bundles export/
  import; the legacy script export imports into "Imported scripts".  EN/RU.
- tests: the suite store/migration, the runner orchestration (order, on_fail,
  setup failure, stop, card reset, teardown, per-member logs) and the ADM gate
  (Python); the suite HTTP paths; the PWA report/Markdown/wiring (Node).
  Tests: 833 frontend + 794 Python; sw cache simple-v397.
This commit is contained in:
2026-10-06 00:26:50 +03:00
parent e6ced47323
commit 5976ef89b4
17 changed files with 3178 additions and 288 deletions
+64
View File
@@ -221,3 +221,67 @@ class StoreInfoTests(unittest.TestCase):
self.assertEqual(info['path'], str(store.path))
self.assertEqual(info['count'], 1)
self.assertEqual(info['version'], test_scripts.SCHEMA_VERSION)
class SuiteAttachmentTests(unittest.TestCase):
"""v3.22.0: a script belongs to a suite (suite_id), declares require_adm,
and the import reports the ids a bundle import needs."""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.path = pathlib.Path(self.tmp.name) / 'test_scripts.json'
self.store = test_scripts.TestScriptStore(self.path)
def tearDown(self):
self.tmp.cleanup()
def test_suite_id_and_require_adm_round_trip(self):
sid = 'a' * 32
s = self.store.add(dict(script('one'), suite_id=sid, require_adm=True))
self.assertEqual(s['suite_id'], sid)
self.assertTrue(s['require_adm'])
got = test_scripts.TestScriptStore(self.path).get(s['id'])
self.assertEqual(got['suite_id'], sid)
self.assertTrue(got['require_adm'])
with self.assertRaises(test_scripts.TestScriptError):
self.store.add(dict(script('two'), suite_id='short'))
def test_orphans_and_list_by_suite(self):
sid = 'a' * 32
s1 = self.store.add(dict(script('one'), suite_id=sid))
s2 = self.store.add(script('two'))
self.assertEqual([x['id'] for x in self.store.list_by_suite(sid)], [s1['id']])
self.assertEqual([x['id'] for x in self.store.orphans()], [s2['id']])
# a move is an update
moved = self.store.update(s2['id'], {'suite_id': sid})
self.assertEqual(moved['suite_id'], sid)
self.assertEqual(self.store.orphans(), [])
self.assertEqual(len(self.store.list_by_suite(sid)), 2)
def test_import_reports_the_id_map_and_attaches_to_a_suite(self):
sid = 'a' * 32
s = self.store.add(script('one'))
result = self.store.import_scripts([self.store.get(s['id'])])
self.assertEqual(result['added'], 1)
self.assertEqual(result['id_map'][s['id']], result['added_ids'][0])
self.assertNotEqual(result['added_ids'][0], s['id'])
# import into a named suite: every added script belongs to it
result = self.store.import_scripts(
[script('two'), script('three')], suite_id=sid)
self.assertEqual(result['added'], 2)
for sid_added in result['added_ids']:
self.assertEqual(self.store.get(sid_added)['suite_id'], sid)
def test_a_v1_store_loads_with_no_suite(self):
self.store.add(script('one'))
data = json.loads(self.path.read_text())
data['version'] = 1
for e in data['scripts']:
e.pop('suite_id', None)
e.pop('require_adm', None)
self.path.write_text(json.dumps(data))
again = test_scripts.TestScriptStore(self.path)
got = again.list()[0]
self.assertIsNone(got['suite_id'])
self.assertFalse(got['require_adm'])
self.assertEqual(len(again.orphans()), 1)