v3.22.0: the test suites (scripts grouped, ordered, with setup/teardown)
A suite is the root object: every script belongs to exactly one suite, the
suite holds the ordered member list by reference - the optional setup first,
the members, the optional teardown last - and runs them in one card session,
without resets and without resume.
- test_suites.py: the suite store (test_suites.json, --test-suites), validated
and atomic like the script store: roles normalised (setup first, teardown
last, at most one of each, a script appears once per suite), per-member
on_fail stop/continue and the suite-level require_adm.
- test_scripts.py schema v2: suite_id/require_adm; list_by_suite()/orphans();
the import reports the id_map a bundle import remaps with. v3.21.0 scripts
without a suite are attached once to an auto-created "Imported scripts"
suite (migrate_scripts_to_suites, called from __main__).
- API: /api/test/suites (+ update|delete|import - delete refused while
non-empty; the bundle import imports the scripts first, remaps the suite
references and skips unresolvable ones); the scripts endpoints gain
suite_id/require_adm plus move|copy (delete detaches from the suite first;
a suite update refuses a membership change).
- runner: /api/test/run {suite_id} resolves the members and the preset, runs
the ADM gate, then setup -> members (per-member on_fail) -> teardown. The
teardown always runs on a stop or a failure (skipped only when the card is
gone; it clears the stop flag for its duration); a card reset stops the
suite, skips the remaining members and is flagged. Members' live steps land
in the shared snapshot, their records (status/steps/note/log excerpt/
counters) in _TEST_RUN['suite']; the summary carries the counts, wall time,
preset, the SCP80 counters before/after per keyset and the session flags.
_test_log buffers the semantic lines (bounded) for the report's per-member
excerpts.
- ADM gate (_ensure_adm): a suite/script require_adm verifies the matched
preset's key once per card session (the /api/verify-adm badge sets the same
latch); a failure refuses the start, stamps the session (no automatic
retry) and needs a manual verify.
- PWA: the Test script pill becomes four views - the suite list (default),
the suite editor (the script list: role badges, on_fail, up/down, per-script
Run/Edit/Move/Copy/Delete, Add script with the templates), the script editor
(the step list; a single run shows its result under the steps and is not
preserved) and the suite report (per-member blocks with the step table,
notes and run log; summary; export JSON/Markdown). Suite bundles export/
import; the legacy script export imports into "Imported scripts". EN/RU.
- tests: the suite store/migration, the runner orchestration (order, on_fail,
setup failure, stop, card reset, teardown, per-member logs) and the ADM gate
(Python); the suite HTTP paths; the PWA report/Markdown/wiring (Node).
Tests: 833 frontend + 794 Python; sw cache simple-v397.
This commit is contained in:
@@ -221,3 +221,67 @@ class StoreInfoTests(unittest.TestCase):
|
||||
self.assertEqual(info['path'], str(store.path))
|
||||
self.assertEqual(info['count'], 1)
|
||||
self.assertEqual(info['version'], test_scripts.SCHEMA_VERSION)
|
||||
|
||||
|
||||
class SuiteAttachmentTests(unittest.TestCase):
|
||||
"""v3.22.0: a script belongs to a suite (suite_id), declares require_adm,
|
||||
and the import reports the ids a bundle import needs."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.path = pathlib.Path(self.tmp.name) / 'test_scripts.json'
|
||||
self.store = test_scripts.TestScriptStore(self.path)
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_suite_id_and_require_adm_round_trip(self):
|
||||
sid = 'a' * 32
|
||||
s = self.store.add(dict(script('one'), suite_id=sid, require_adm=True))
|
||||
self.assertEqual(s['suite_id'], sid)
|
||||
self.assertTrue(s['require_adm'])
|
||||
got = test_scripts.TestScriptStore(self.path).get(s['id'])
|
||||
self.assertEqual(got['suite_id'], sid)
|
||||
self.assertTrue(got['require_adm'])
|
||||
with self.assertRaises(test_scripts.TestScriptError):
|
||||
self.store.add(dict(script('two'), suite_id='short'))
|
||||
|
||||
def test_orphans_and_list_by_suite(self):
|
||||
sid = 'a' * 32
|
||||
s1 = self.store.add(dict(script('one'), suite_id=sid))
|
||||
s2 = self.store.add(script('two'))
|
||||
self.assertEqual([x['id'] for x in self.store.list_by_suite(sid)], [s1['id']])
|
||||
self.assertEqual([x['id'] for x in self.store.orphans()], [s2['id']])
|
||||
# a move is an update
|
||||
moved = self.store.update(s2['id'], {'suite_id': sid})
|
||||
self.assertEqual(moved['suite_id'], sid)
|
||||
self.assertEqual(self.store.orphans(), [])
|
||||
self.assertEqual(len(self.store.list_by_suite(sid)), 2)
|
||||
|
||||
def test_import_reports_the_id_map_and_attaches_to_a_suite(self):
|
||||
sid = 'a' * 32
|
||||
s = self.store.add(script('one'))
|
||||
result = self.store.import_scripts([self.store.get(s['id'])])
|
||||
self.assertEqual(result['added'], 1)
|
||||
self.assertEqual(result['id_map'][s['id']], result['added_ids'][0])
|
||||
self.assertNotEqual(result['added_ids'][0], s['id'])
|
||||
# import into a named suite: every added script belongs to it
|
||||
result = self.store.import_scripts(
|
||||
[script('two'), script('three')], suite_id=sid)
|
||||
self.assertEqual(result['added'], 2)
|
||||
for sid_added in result['added_ids']:
|
||||
self.assertEqual(self.store.get(sid_added)['suite_id'], sid)
|
||||
|
||||
def test_a_v1_store_loads_with_no_suite(self):
|
||||
self.store.add(script('one'))
|
||||
data = json.loads(self.path.read_text())
|
||||
data['version'] = 1
|
||||
for e in data['scripts']:
|
||||
e.pop('suite_id', None)
|
||||
e.pop('require_adm', None)
|
||||
self.path.write_text(json.dumps(data))
|
||||
again = test_scripts.TestScriptStore(self.path)
|
||||
got = again.list()[0]
|
||||
self.assertIsNone(got['suite_id'])
|
||||
self.assertFalse(got['require_adm'])
|
||||
self.assertEqual(len(again.orphans()), 1)
|
||||
|
||||
Reference in New Issue
Block a user