test runner: add the stuck-card watchdog
A card that stops answering (no SW, no driver error) leaves the runner inside a blocking SCardTransmit while it holds the card lock - Stop cannot take effect, the presence monitor waits for the lock and a load test hangs in that iteration without explanation. The runner now beats a progress heartbeat (each step start/finish and each attempt of a bounded STATUS/drain loop); after _TEST_STEP_TIMEOUT (30 s) the watchdog flags the run `stuck`, records the step and the idle time, sets the stop flag and tries to cancel the blocked transmit by disconnecting the PC/SC link on a daemon thread. The disconnect must run in the background: pcsc-lite serializes it behind the pending transmit (both take the per-context lock), so a synchronous call would block the watchdog instead of cancelling anything. Where a platform does cancel, the step fails through the ordinary on_fail path; either way the finalisers map `stuck` to `error` - a stuck card is not a user stop - and the pending drain is skipped on the dead link. The transport is marked stale once the disconnect returns, so the next equip rebuilds it. The run status gains `stuck` and `heartbeat`. Help EN/RU, docs/test-scripts.md, docs/api.md, the CHANGELOG (folded into the unreleased 3.37.0) and both AGENTS describe the behaviour and the background-disconnect caveat; the sw cache is simple-v476 (help is precached). Tests: tests/test_test_watchdog.py (17: the check rule, the abort, the final status of the single/load/suite runs and a blocked-runner integration case). 1058 Python + 1069 frontend.
This commit is contained in:
1 parent
2181ef6bbf
commit
643c6609f3
8 files changed
+527
-16
No files matched your search
+9
-1
@@ -1039,7 +1039,15 @@ The current (or last) run:
|
||||
```
|
||||
|
||||
`status` becomes `ok`/`warning`/`error`/`stopped` when the run finishes;
|
||||
expected/actual pairs are reported per check.
|
||||
expected/actual pairs are reported per check. A card that stops answering is
|
||||
not waited on silently: after 30 s without runner progress the watchdog sets
|
||||
`stuck: true`, `stop` and `error` (`the card stopped responding (step N, no
|
||||
answer for N s)`) and tries to cancel the blocked transmit by disconnecting
|
||||
the PC/SC link on a background thread (pcsc-lite serializes the disconnect
|
||||
behind the pending transmit, so the call returns only when the driver's card
|
||||
timeout fires). Once the step returns, the run finalises as `error` - a stuck
|
||||
card is a failure, not a user stop - and the next equip rebuilds the link
|
||||
(the transport is marked stale).
|
||||
|
||||
### `POST /api/test/stop`
|
||||
|
||||
|
||||
@@ -242,6 +242,29 @@ between iterations is an error. The SCP80 counter state is shared across
|
||||
iterations, so packets continue from what the previous iteration consumed.
|
||||
The iteration count is a run parameter - the stored script/suite is unchanged.
|
||||
|
||||
## A card that stops answering
|
||||
|
||||
pyscard's `SCardTransmit` is blocking and has no timeout: a card that stops
|
||||
answering (no status word, no driver error) would leave the runner inside the
|
||||
step while it holds the card lock - Stop could not take effect, the presence
|
||||
monitor would wait for the lock, and a load test would hang in that iteration.
|
||||
The runner's **watchdog** watches a progress heartbeat: after 30 s without one
|
||||
(a bounded STATUS/drain loop beats once per attempt, so a long poll is not
|
||||
mistaken for a stuck card) it marks the run `stuck`, sets the error
|
||||
`the card stopped responding (step N, no answer for N s)` and the stop flag,
|
||||
and **tries** to cancel the blocked transmit by disconnecting the PC/SC link
|
||||
on a background thread. Where PC/SC can cancel it, the blocking call raises,
|
||||
the step fails through the ordinary path (`on_fail` applies) and the run
|
||||
finalises as an **error** - never as `stopped`, even though the stop flag
|
||||
ended the loop. The disconnect runs in the background because pcsc-lite
|
||||
serializes it behind the pending transmit (both take the per-context lock), so
|
||||
on that stack the call returns only when the driver's own card timeout fires
|
||||
(libccid derives it from the card's ATR); the run is already flagged and the
|
||||
status names the step meanwhile. The transport is marked stale once the
|
||||
disconnect returns, so the next **Equip** rebuilds the link. A physical card
|
||||
removal is unaffected: the driver aborts the transmit and the presence monitor
|
||||
clears the card session.
|
||||
|
||||
## Document shape
|
||||
|
||||
```json
|
||||
|
||||
Reference in new issue
Block a user