catarrh/simple
1
0
Fork 0
Code Issues Pull Requests Actions 8 Packages Projects Releases Wiki Activity

v3.22.0: the test suites (scripts grouped, ordered, with setup/teardown)

Browse Source 
A suite is the root object: every script belongs to exactly one suite, the
suite holds the ordered member list by reference - the optional setup first,
the members, the optional teardown last - and runs them in one card session,
without resets and without resume.

- test_suites.py: the suite store (test_suites.json, --test-suites), validated
  and atomic like the script store: roles normalised (setup first, teardown
  last, at most one of each, a script appears once per suite), per-member
  on_fail stop/continue and the suite-level require_adm.
- test_scripts.py schema v2: suite_id/require_adm; list_by_suite()/orphans();
  the import reports the id_map a bundle import remaps with.  v3.21.0 scripts
  without a suite are attached once to an auto-created "Imported scripts"
  suite (migrate_scripts_to_suites, called from __main__).
- API: /api/test/suites (+ update|delete|import - delete refused while
  non-empty; the bundle import imports the scripts first, remaps the suite
  references and skips unresolvable ones); the scripts endpoints gain
  suite_id/require_adm plus move|copy (delete detaches from the suite first;
  a suite update refuses a membership change).
- runner: /api/test/run {suite_id} resolves the members and the preset, runs
  the ADM gate, then setup -> members (per-member on_fail) -> teardown.  The
  teardown always runs on a stop or a failure (skipped only when the card is
  gone; it clears the stop flag for its duration); a card reset stops the
  suite, skips the remaining members and is flagged.  Members' live steps land
  in the shared snapshot, their records (status/steps/note/log excerpt/
  counters) in _TEST_RUN['suite']; the summary carries the counts, wall time,
  preset, the SCP80 counters before/after per keyset and the session flags.
  _test_log buffers the semantic lines (bounded) for the report's per-member
  excerpts.
- ADM gate (_ensure_adm): a suite/script require_adm verifies the matched
  preset's key once per card session (the /api/verify-adm badge sets the same
  latch); a failure refuses the start, stamps the session (no automatic
  retry) and needs a manual verify.
- PWA: the Test script pill becomes four views - the suite list (default),
  the suite editor (the script list: role badges, on_fail, up/down, per-script
  Run/Edit/Move/Copy/Delete, Add script with the templates), the script editor
  (the step list; a single run shows its result under the steps and is not
  preserved) and the suite report (per-member blocks with the step table,
  notes and run log; summary; export JSON/Markdown).  Suite bundles export/
  import; the legacy script export imports into "Imported scripts".  EN/RU.
- tests: the suite store/migration, the runner orchestration (order, on_fail,
  setup failure, stop, card reset, teardown, per-member logs) and the ADM gate
  (Python); the suite HTTP paths; the PWA report/Markdown/wiring (Node).
  Tests: 833 frontend + 794 Python; sw cache simple-v397.
This commit is contained in:
Антон Трошин
2026-10-06 00:26:50 +03:00
parent e6ced47323
commit 5976ef89b4
17 changed files with 3178 additions and 288 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pysim_simple_server/__main__.py
+11 -1
View File
@@ -16,8 +16,9 @@ from . import fastinit
from . import gsmtap
from . import presets
from . import test_scripts
from . import test_suites
from . import testscript
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter, _test_command_type
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter, _test_command_type, migrate_scripts_to_suites
_server_start = 0
@@ -70,6 +71,8 @@ def main():
help='Card preset store (default: ~/.pysim-simple-server/card_presets.json)')
parser.add_argument('--test-scripts', default=None, metavar='PATH',
help='Test script store (default: ~/.pysim-simple-server/test_scripts.json)')
parser.add_argument('--test-suites', default=None, metavar='PATH',
help='Test suite store (default: ~/.pysim-simple-server/test_suites.json)')
parser.add_argument('--sms-oa', default='12345', metavar='DIGITS',
help='TP-Originating-Address (SMSC number) for the SMS-DELIVER TPDU (default: 12345)')
parser.add_argument('--sms-sm-sc', default='12345678912', metavar='DIGITS',
@@ -300,6 +303,13 @@ def main():
validator=lambda raw: testscript.normalise_script(raw, _test_command_type))
sys.stderr.write('TESTSCRIPTS: %s (%d scripts)\n'
% (server.test_scripts.path, len(server.test_scripts.list())))
# Test suites group the scripts (v3.22.0): a script belongs to exactly one
# suite, so a pre-suite store's scripts are attached to an auto-created
# "Imported scripts" suite once (idempotent - no orphans, nothing to do).
server.test_suites = test_suites.TestSuiteStore(opts.test_suites)
sys.stderr.write('TESTSUITES: %s (%d suites)\n'
% (server.test_suites.path, len(server.test_suites.list())))
migrate_scripts_to_suites(server.test_scripts, server.test_suites)
server.sim_menu = sim_menu
server.event_list = event_list
server.menu_active = False
pysim_simple_server/server.py
+919 -27
View File
File diff suppressed because it is too large Load Diff
pysim_simple_server/test_scripts.py
+72 -15
View File
@@ -12,10 +12,20 @@ the editor, while the next save rewrites the file with it intact.
A stored script is the script JSON (``name`` + ``steps``, the same shape the
PWA exports) plus a stable uuid and timestamps::
{"version": 1, "scripts": [
{"id": "...", "name": "applet RFM update", "steps": [...],
{"version": 2, "scripts": [
{"id": "...", "name": "applet RFM update", "suite_id": "...",
"require_adm": false, "steps": [...],
"created": 1690000000.0, "updated": 1690000000.0}]}
Schema v2 (v3.22.0) adds ``suite_id``/``require_adm``: a script belongs to
exactly one test suite (the suite is the root object; the PWA creates scripts
only inside a suite) and can declare that it needs the ADM PIN verified. A
v1 entry loads with ``suite_id: None`` - the server attaches such orphans to
an auto-created "Imported scripts" suite on startup - and the next save
writes v2. The cross-store rules (the suite exists, a move updates both
stores, a referenced script cannot leave a dangling reference) live in the
HTTP layer, which holds both stores.
Validation runs on every mutation: the basic shape always, and - when the
server passes its ``validator`` - the full ``testscript.normalise_script``
pass (the same one the runner uses at start). Import accepts the old
@@ -44,7 +54,9 @@ from pathlib import Path
DEFAULT_DIR = '.pysim-simple-server'
DEFAULT_FILENAME = 'test_scripts.json'
SCHEMA_VERSION = 1
SCHEMA_VERSION = 2
_ID_RE = re.compile(r'[0-9a-f]{32}')
def default_path():
@@ -171,7 +183,16 @@ class TestScriptStore:
if not isinstance(step, dict) or step.get('type') not in ('action', 'expect'):
raise TestScriptError('step %d must be an action or an expectation'
% (i + 1))
script = {'name': name.strip(), 'steps': copy.deepcopy(steps)}
suite_id = raw.get('suite_id')
if suite_id in (None, ''):
suite_id = None
else:
suite_id = str(suite_id).strip().lower()
if not _ID_RE.fullmatch(suite_id):
raise TestScriptError('suite_id must be a suite uuid')
script = {'name': name.strip(), 'suite_id': suite_id,
'require_adm': bool(raw.get('require_adm')),
'steps': copy.deepcopy(steps)}
if validate and self._validator is not None:
try:
self._validator(copy.deepcopy(script))
@@ -189,7 +210,7 @@ class TestScriptStore:
script = self._check(entry, validate=validate)
now = time.time()
rid = str(entry.get('id') or '').strip().lower()
if not re.fullmatch(r'[0-9a-f]{32}', rid):
if not _ID_RE.fullmatch(rid):
rid = uuid.uuid4().hex
if keep_meta:
created = entry.get('created')
@@ -198,7 +219,10 @@ class TestScriptStore:
updated = updated if isinstance(updated, (int, float)) else created
else:
created = updated = now
return {'id': rid, 'name': script['name'], 'steps': script['steps'],
return {'id': rid, 'name': script['name'],
'suite_id': script['suite_id'],
'require_adm': script['require_adm'],
'steps': script['steps'],
'created': created, 'updated': updated}
def _find(self, sid):
@@ -218,6 +242,20 @@ class TestScriptStore:
s = self._find(sid)
return copy.deepcopy(s) if s else None
def list_by_suite(self, suite_id):
"""The scripts attached to one suite (the suite holds their order)."""
want = str(suite_id or '').strip().lower()
with self._lock:
return [copy.deepcopy(s) for s in self._scripts
if (s.get('suite_id') or '') == want]
def orphans(self):
"""Scripts without a suite (a v1 store, or a hand edit): the server
attaches them to the imported-scripts suite on startup."""
with self._lock:
return [copy.deepcopy(s) for s in self._scripts
if not s.get('suite_id')]
def add(self, script):
with self._lock:
s = self._normalise(script or {})
@@ -232,19 +270,25 @@ class TestScriptStore:
return copy.deepcopy(s)
def update(self, sid, fields):
"""Partial or full update: name/steps replaced, id and created kept."""
"""Partial or full update: name/steps/suite_id/require_adm replaced,
id and created kept. The suite_id change is the *move* operation's
write (the HTTP layer updates the source and target suites around
it)."""
with self._lock:
cur = self._find(sid)
if cur is None:
return None
merged = {'name': cur['name'], 'steps': copy.deepcopy(cur['steps'])}
merged = {'name': cur['name'], 'suite_id': cur.get('suite_id'),
'require_adm': cur.get('require_adm', False),
'steps': copy.deepcopy(cur['steps'])}
if isinstance(fields, dict):
if 'name' in fields:
merged['name'] = fields['name']
if 'steps' in fields:
merged['steps'] = fields['steps']
for key in ('name', 'steps', 'suite_id', 'require_adm'):
if key in fields:
merged[key] = fields[key]
script = self._check(merged)
new = {'id': cur['id'], 'name': script['name'],
'suite_id': script['suite_id'],
'require_adm': script['require_adm'],
'steps': script['steps'], 'created': cur['created'],
'updated': time.time()}
self._scripts[self._scripts.index(cur)] = new
@@ -262,12 +306,17 @@ class TestScriptStore:
% (s['name'] or '(unnamed)', s['id']))
return True
def import_scripts(self, items, mode='merge'):
def import_scripts(self, items, mode='merge', suite_id=None):
"""Import exported (or old localStorage) scripts: entries are validated
one by one; an already-used id gets a fresh one, invalid entries are
reported instead of failing the whole import. ``replace`` wipes the
store first."""
store first. ``suite_id`` attaches every imported script to that suite
(the "import into this suite" flow). Returns the ``id_map`` (input id
-> stored id, the bundle import's reference remap) and the stored ids
of the imported scripts (the target suite's member entries)."""
added, skipped, errors = 0, 0, []
id_map = {}
added_ids = []
with self._lock:
if mode == 'replace':
self._scripts = []
@@ -276,20 +325,28 @@ class TestScriptStore:
skipped += 1
continue
try:
entry = dict(entry)
if suite_id:
entry['suite_id'] = suite_id
s = self._normalise(entry)
src_id = str(entry.get('id') or '').strip().lower()
if self._find(s['id']):
s['id'] = uuid.uuid4().hex
if _ID_RE.fullmatch(src_id):
id_map[src_id] = s['id']
except TestScriptError as e:
skipped += 1
errors.append('%s: %s' % (entry.get('name') or ('#' + str(i + 1)), e))
continue
self._scripts.append(s)
added_ids.append(s['id'])
added += 1
if added or mode == 'replace':
self._save()
sys.stderr.write('TESTSCRIPTS: import %s: %d added, %d skipped\n'
% (mode, added, skipped))
return {'added': added, 'skipped': skipped, 'errors': errors[:10]}
return {'added': added, 'skipped': skipped, 'errors': errors[:10],
'id_map': id_map, 'added_ids': added_ids}
def info(self):
with self._lock:
pysim_simple_server/test_suites.py
+339
View File
@@ -0,0 +1,339 @@
"""Server-side test suite store: ordered groups of card test scripts.
The test scripts used to be a flat set; a suite groups them (a suite is the
root object - every script belongs to exactly one suite, ``suite_id`` in the
script store). The suite keeps the **ordered member list** by reference
(``script_id``), so the scripts stay the single source of truth and moving or
editing one never touches the suite file beyond the reference list. The
runner executes them in order: an optional ``setup`` script first, the
``member`` scripts, an optional ``teardown`` script last - the roles must be
in that order, so the store normalises the list (setup entries first,
teardown entries last, at most one of each) instead of rejecting a
mis-ordered edit.
A stored suite::
{"version": 1, "suites": [
{"id": "...", "name": "alfa regression", "require_adm": true,
"scripts": [{"script_id": "...", "role": "setup", "on_fail": "stop"},
{"script_id": "...", "role": "member", "on_fail": "stop"},
{"script_id": "...", "role": "member", "on_fail": "continue"},
{"script_id": "...", "role": "teardown", "on_fail": "stop"}],
"created": ..., "updated": ...}]}
``on_fail`` is the per-member policy: ``stop`` (default) ends the suite at
that member, ``continue`` runs the next one - "which failures cascade" is a
visible per-suite decision. The store validates the shape on every mutation
(the cross-store rules - the suite exists, the scripts exist, a non-empty
suite cannot be deleted - live in the HTTP layer, which holds both stores);
a load keeps an entry that no longer validates (a tightened rule or a hand
edit must never destroy it), it is served and fails at run start.
Layout (``--test-suites`` overrides the file; ``Path.home()`` resolves the
same way on Linux, macOS and Windows)::
~/.pysim-simple-server/test_suites.json
Writes are atomic (temp file + fsync + ``os.replace``) and every mutation
holds an ``RLock``. Suites carry a stable uuid - never array indices.
"""
import copy
import json
import os
import re
import sys
import tempfile
import threading
import time
import uuid
from pathlib import Path
DEFAULT_DIR = '.pysim-simple-server'
DEFAULT_FILENAME = 'test_suites.json'
SCHEMA_VERSION = 1
ROLES = ('setup', 'member', 'teardown')
ON_FAIL_LEVELS = ('stop', 'continue')
MAX_SCRIPTS = 200
_ID_RE = re.compile(r'[0-9a-f]{32}')
def default_path():
"""The default store: ``~/.pysim-simple-server/test_suites.json``."""
return Path.home() / DEFAULT_DIR / DEFAULT_FILENAME
class TestSuiteError(Exception):
"""A refused mutation (validation, unknown id, ...)."""
class TestSuiteStore:
"""The test suite store. All public methods are thread-safe and return
copies - callers can never mutate the store's own state."""
def __init__(self, path=None):
self.path = Path(path) if path else default_path()
self._lock = threading.RLock()
self._suites = []
self._load()
# ------------------------------------------------------------------ disk
def _load(self):
try:
raw = self.path.read_text(encoding='utf-8')
except FileNotFoundError:
self._suites = []
return
except OSError as e:
sys.stderr.write('TESTSUITES: cannot read %s: %s\n' % (self.path, e))
self._suites = []
return
try:
data = json.loads(raw)
except ValueError as e:
# Never silently overwrite an unreadable store: keep the file for
# the user, start empty (the next save would destroy it).
bad = self.path.with_name(self.path.name + '.bad-%s'
% time.strftime('%Y%m%d-%H%M%S'))
try:
self.path.replace(bad)
sys.stderr.write('TESTSUITES: %s is not valid JSON (%s) - kept as '
'%s, starting empty\n' % (self.path, e, bad))
except OSError:
sys.stderr.write('TESTSUITES: %s is not valid JSON (%s)\n'
% (self.path, e))
self._suites = []
return
items = data.get('suites') if isinstance(data, dict) else None
if not isinstance(items, list):
items = []
loaded = []
for entry in items:
try:
s = self._normalise(entry, keep_meta=True)
except TestSuiteError as e:
sys.stderr.write('TESTSUITES: dropping an unreadable entry: %s\n' % e)
continue
# A shape-valid entry is kept even when a referenced script is
# gone: served, the editor shows the missing reference, and the
# HTTP layer refuses a save/run until it is fixed.
loaded.append(s)
self._suites = loaded
def _save(self):
data = {'version': SCHEMA_VERSION, 'suites': self._suites}
self.path.parent.mkdir(parents=True, exist_ok=True)
tmp_fd, tmp_name = tempfile.mkstemp(prefix=self.path.name + '.',
dir=str(self.path.parent))
try:
with os.fdopen(tmp_fd, 'w', encoding='utf-8') as fh:
json.dump(data, fh, indent=2, ensure_ascii=False)
fh.write('\n')
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp_name, self.path)
except Exception:
try:
os.unlink(tmp_name)
except OSError:
pass
raise
try:
dir_fd = os.open(str(self.path.parent), os.O_RDONLY)
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)
except OSError:
pass
# ------------------------------------------------------------- shaping
@staticmethod
def _script_entry(raw, index):
"""One member entry: the referenced script id, its role and the
per-member failure policy."""
if not isinstance(raw, dict):
raise TestSuiteError('script entry %d must be an object' % (index + 1))
sid = str(raw.get('script_id') or '').strip().lower()
if not _ID_RE.fullmatch(sid):
raise TestSuiteError('script entry %d: script_id must be a suite '
'uuid' % (index + 1))
role = str(raw.get('role') or 'member').strip().lower()
if role not in ROLES:
raise TestSuiteError('script entry %d: role must be one of %s'
% (index + 1, ', '.join(ROLES)))
on_fail = str(raw.get('on_fail') or 'stop').strip().lower()
if on_fail not in ON_FAIL_LEVELS:
raise TestSuiteError('script entry %d: on_fail must be one of %s'
% (index + 1, ', '.join(ON_FAIL_LEVELS)))
return {'script_id': sid, 'role': role, 'on_fail': on_fail}
def _check(self, raw):
"""The suite dict, validated (shape only - the cross-store rules live
in the HTTP layer): name, the roles-ordered script list."""
if not isinstance(raw, dict):
raise TestSuiteError('suite must be an object')
name = raw.get('name')
if name is None:
name = ''
if not isinstance(name, str):
raise TestSuiteError('suite name must be a string')
scripts_raw = raw.get('scripts')
if scripts_raw is None:
scripts_raw = []
if not isinstance(scripts_raw, list):
raise TestSuiteError('suite scripts must be a list')
if len(scripts_raw) > MAX_SCRIPTS:
raise TestSuiteError('a suite may hold at most %d scripts' % MAX_SCRIPTS)
entries = [self._script_entry(e, i) for i, e in enumerate(scripts_raw)]
seen = set()
for e in entries:
if e['script_id'] in seen:
raise TestSuiteError('a script appears once per suite: %s'
% e['script_id'])
seen.add(e['script_id'])
setups = [e for e in entries if e['role'] == 'setup']
teardowns = [e for e in entries if e['role'] == 'teardown']
if len(setups) > 1:
raise TestSuiteError('a suite has at most one setup script')
if len(teardowns) > 1:
raise TestSuiteError('a suite has at most one teardown script')
# The roles define the order: setup first, teardown last (members keep
# their relative order in between).
ordered = setups + [e for e in entries if e['role'] == 'member'] + teardowns
return {'name': name.strip(), 'require_adm': bool(raw.get('require_adm')),
'scripts': ordered}
def _normalise(self, entry, keep_meta=False):
"""A stored entry: the validated suite plus id and timestamps. With
``keep_meta`` the stored id/timestamps are preserved (a load), else
fresh ones are made (an add)."""
if not isinstance(entry, dict):
raise TestSuiteError('suite must be an object')
suite = self._check(entry)
now = time.time()
rid = str(entry.get('id') or '').strip().lower()
if not _ID_RE.fullmatch(rid):
rid = uuid.uuid4().hex
if keep_meta:
created = entry.get('created')
updated = entry.get('updated')
created = created if isinstance(created, (int, float)) else now
updated = updated if isinstance(updated, (int, float)) else created
else:
created = updated = now
return {'id': rid, 'name': suite['name'],
'require_adm': suite['require_adm'],
'scripts': suite['scripts'],
'created': created, 'updated': updated}
def _find(self, sid):
sid = str(sid or '').strip().lower()
for s in self._suites:
if s['id'] == sid:
return s
return None
# -------------------------------------------------------------- public
def list(self):
with self._lock:
return [copy.deepcopy(s) for s in self._suites]
def get(self, sid):
with self._lock:
s = self._find(sid)
return copy.deepcopy(s) if s else None
def find_by_name(self, name):
want = str(name or '').strip()
with self._lock:
for s in self._suites:
if s['name'] == want:
return copy.deepcopy(s)
return None
def add(self, suite):
with self._lock:
s = self._normalise(suite or {})
if self._find(s['id']):
# a caller-supplied id that is already taken never wins: ids
# must stay unique (the same guard the import uses)
s['id'] = uuid.uuid4().hex
self._suites.append(s)
self._save()
sys.stderr.write('TESTSUITES: added %s (%s, %d scripts)\n'
% (s['name'] or '(unnamed)', s['id'], len(s['scripts'])))
return copy.deepcopy(s)
def update(self, sid, fields):
"""Partial or full update: name/require_adm/scripts replaced, id and
created kept. The order/role rules are re-applied (setup first,
teardown last)."""
with self._lock:
cur = self._find(sid)
if cur is None:
return None
merged = {'name': cur['name'], 'require_adm': cur['require_adm'],
'scripts': copy.deepcopy(cur['scripts'])}
if isinstance(fields, dict):
for key in ('name', 'require_adm', 'scripts'):
if key in fields:
merged[key] = fields[key]
suite = self._check(merged)
new = {'id': cur['id'], 'name': suite['name'],
'require_adm': suite['require_adm'],
'scripts': suite['scripts'],
'created': cur['created'], 'updated': time.time()}
self._suites[self._suites.index(cur)] = new
self._save()
return copy.deepcopy(new)
def remove(self, sid):
"""Remove a suite. The HTTP layer refuses this while it still holds
scripts (the caller checks first); the store itself deletes."""
with self._lock:
s = self._find(sid)
if s is None:
return False
self._suites.remove(s)
self._save()
sys.stderr.write('TESTSUITES: removed %s (%s)\n'
% (s['name'] or '(unnamed)', s['id']))
return True
def import_suites(self, items, mode='merge'):
"""Import exported suites: entries are validated one by one; an
already-used id gets a fresh one, invalid entries are reported instead
of failing the whole import. The caller remaps script references (a
bundle import that had to regenerate a script's uuid rewrites the
references before this call). ``replace`` wipes the store first."""
added, skipped, errors = 0, 0, []
with self._lock:
if mode == 'replace':
self._suites = []
for i, entry in enumerate(items or []):
if not isinstance(entry, dict):
skipped += 1
continue
try:
s = self._normalise(entry)
if self._find(s['id']):
s['id'] = uuid.uuid4().hex
except TestSuiteError as e:
skipped += 1
errors.append('%s: %s' % (entry.get('name') or ('#' + str(i + 1)), e))
continue
self._suites.append(s)
added += 1
if added or mode == 'replace':
self._save()
sys.stderr.write('TESTSUITES: import %s: %d added, %d skipped\n'
% (mode, added, skipped))
return {'added': added, 'skipped': skipped, 'errors': errors[:10]}
def info(self):
with self._lock:
return {'path': str(self.path), 'count': len(self._suites),
'version': SCHEMA_VERSION}