The former 'passthru' mode is now 'redirect': it pins the configured target
and every BIP channel is connected there (the card's requested address is
only logged; host/port required). The name 'passthru' now means the new mode:
no listener and no target - each channel dials the destination the card
requests in OPEN CHANNEL (Other address + Transport level port, TCP client,
remote, 0x02 only). The specs define no default port (TS 102 223 8.59), so an
incomplete or non-TCP request fails the channel with result 3A and an
open-fail log reason.
BipTerminal gains a mode (enable(host, port, mode=...), open(..., proto=...)),
reports it in status(), and the control API/status expose redirect (target)
and passthru (per-channel targets). The PWA mode selector shows four modes
with per-mode notes and disables Host/Port in passthru; the status line shows
each channel's actual target. Docs, help (EN/RU), READMEs and the AGENTS notes
updated; SW cache otaman-v179.
Breaking API change: mode:'passthru' has the new semantics - use
mode:'redirect' for the previous behavior (no alias).
TERMINAL PROFILE:
- GET /api/terminal-profile returns the profile in effect + the CLI default;
POST /api/terminal-profile validates ({profile}, hex, even, 1-255 bytes),
stores it in memory and re-sends it, resetting the STK session like
/api/rescue (the shared _resend_terminal_profile helper; rescue now
delegates to it). __main__ keeps server.cli_terminal_profile.
- Phone tab: a TERMINAL PROFILE block next to STATUS and Polling with the
current hex/byte count, Send (re-send) and Configure. The Configure
dialog has a device-model preset selector, a hex field and a per-bit
form generated from a 264-entry table for TS 102 223 V18.3.0 5.2 bytes
1-33 (pySim's table as scaffold, later bytes/3GPP bits added from the
spec; beyond the table generic RFU labels). Form <-> hex sync both
ways, hex authoritative, bits preserved. Apply posts the new value.
- Presets: Xiaomi Mi A1 (project default), Quectel GSM module example.
In-memory only, no persistence.
Custom files:
- Canonical paths rooted at MF / ADF.USIM / ADF.ISIM; entries are
{path, name, kind}. The editor now uses root + parent-DF selector +
4-hex FID + alias, requires the parent DF to be defined first, rejects
duplicates, rewrites descendants when a DF's path changes and cascades
deletes after a confirmation.
- Legacy forms are normalized on load/import (3F00/... -> MF/..., relative
a153/4954 resolved against the custom DFs); unresolvable entries are
dropped and reported in the list.
- Tree injection matches by exact parent path via the new pysimFsNodePath
(same-FID DFs under different parents no longer collide);
profilerCustomNameForPath uses the same normalization.
SW cache otaman-v167; help EN/RU + docs/api.md + AGENTS updated.
Tests: 236 Python + 371 frontend.
SCP81:
- New listener mode "passthru": no local listener - the card's BIP
channels connect straight to a configured external platform
(host/port required), which terminates TLS and runs the dialog.
The status API reports mode/target (_SCP81_MODE/_SCP81_TARGET) and
clears them on stop. PWA mode select, hint, Start validation; TLS PSK
stays the default.
Proactive command decoding (Phone tab log):
- SEND SHORT MESSAGE (0x13) is now decoded: alpha, address (TON/NPI +
number), 3GPP-SMS TPDU (type, TP-MR, TP-DA, TP-PID 0x7F flagged as
SIM data download, TP-DCS, TP-VP, TP-UDL), UDH concatenation IEs, and
the TP-UD as text (GSM-7 with a septet unpacker, UCS2, 8-bit) or as a
TS 31.115 secured packet for PID 0x7F; malformed TPDUs fall back to
the raw hex line.
- PROVIDE LOCAL INFORMATION qualifier names completed per TS 102 223
V18.3.0: ESN (07), MEID (0B), Supported RATs (1A); 05 relabelled
"Reserved for GSM (Timing Advance)". Fixed in the server dict and
both frontend tables.
Header:
- Compact ADM badge next to the card indicator: "ADM ✓" green when
pySim's adm_verified is set, "ADM ✗" red otherwise, hidden without a
card session or when the server is down; updated ahead of the card
state-key early return so it never disturbs the connect/reset flow.
File manager:
- Sort pills (FID/Name), Probe all files button and progress line are
pinned above the tree instead of scrolling with it; the tree box cap
grows from 420px to 65vh.
SW cache otaman-v165; help EN/RU updated; tests 234 Python + 361 frontend.
Cards / SCP81:
- Cards is a top-level tab; presets gain PSK identity + key, HTTP-OTA
column, Edit/Update and a live PSK-map push into a running listener.
- SCP81 has Listener/Scripts pills; scripts are named local APDU lists
(Empty / Explore / Install from .cap / Delete templates), sent to the
server explicitly at start. The listener takes mode/host/port/script
only; PSK inputs and the .cap row are gone.
- Multi-PSK TLS listener: identity -> key lookup from the card presets
(POST /api/scp81/psk-map updates a running listener), unknown
identities log tls-psk-unknown and fail; handshake logs carry psk_match.
- Script engine: execution tracking (next/done/pending/results), a
resumed dialog sends only the unexecuted tail (unreported APDU is
resent), a fresh dialog restarts, listing continuation pages are
tracked separately (pending.pages/complete). Restart script button.
- POST /api/scp81/gen-install replaces the SCP81 ram-install queueing
(generation only; the .cap is never stored).
Profiler / snapshots:
- Snapshot comparison is always exact (mask checkboxes removed; the
first-4-bytes mask remains a profile-creation option).
- "matching records" line shows count + #record numbers.
- New Clone action: copy named "Copy of <profile>", opened in the editor.
- Matched-record count/numbers fix ("1 из 8 (#8)").
SCP80:
- Configurable / auto-fitted LOAD block size: each LOAD APDU encodes into
one SMS (pySim rejects secured packets above 140 octets, so a 240-byte
block could never be sent). Response reports the effective size and
clamps; encode failures are reported per step with the pySim message.
SW cache otaman-v161; docs/api.md, scp81-findings and help EN/RU updated.
Tests: 226 Python + 356 frontend.
The R-APDU TLV length was read as a raw byte, so a listing page above 127
bytes (AF 80 23 81 FC <252 bytes> 00 00) was cut to its first 127 bytes with
a bogus status word (the data's last two bytes: CAFE/0001/9F70 instead of
the real 63 10 "more data"). The bogus SW also stopped the SW CAFE/6310
pagination, which is why later registry entries - e.g. the installed package
AA1902BC225801 - never showed up. Uses httpota.ber_len_read now.
Tests: 250-byte long-form page and short-form regression (204 python);
findings updated; service worker v154
- continuation repeats the SAME GET STATUS command with P2.b1 set (the
pagination state lives in the card); changing the 4F criterion is a match
filter, not a position - P2=03 with the last AID is rejected with 6A80 and
P2=02 with it returns that single match (the earlier duplicate)
- handle the standard "more data available" warning SW 63 10 (Table 11-38)
in addition to the live card's proprietary CA FE
- explore script: P1=40 is applications+SDs, P1=20 the ELF registry, P1=10
ELF+modules (Table 11-33) - the ELF-only registry was never queried, which
hid the installed package; labels and the results decoder show C4 (ELF AID)
and CC (SD AID) too
- UICC_SPECS.md: GET STATUS P1/P2 tables made explicit with the pagination
rule, plus BER length coding notes for the scripting templates and the
TS 102 223 channel data TLV (the two >127-byte traps)
201 python + 346 frontend; service worker v153
P2=02 means "get first or all occurrence(s)" (Table 11-34), so every
continuation re-returned the first listing (the criterion's single match),
pagination stopped after one extra page and the installed package
AA1902BC225801 never showed up in the ELF registry. P2=03 = "get next
occurrence(s)" is the correct value for the SW CAFE continuation.
Tests updated with the new continuation bytes; findings documented;
service worker v152
- POST /api/scp81/queue takes an explicit APDU list (or single APDU) and
queues it as the SCP81 script; entries that already are Command Scripting
templates (AA.../AE80..., the expanded format) are sent verbatim instead of
being wrapped again
- Remote APDU -> RAM chain: "To expanded" builds each command in the
TS 102 226 expanded form (AA definite / AE80 indefinite selector) and
"Queue in SCP81" queues the built commands for the next card POST, so the
full-featured RAM/INSTALL [for install] form (AIDs, privileges, TK/STK
parameters) can drive the HTTP OTA install
- RU strings; api.md; service worker v151
The block slicer used the block number as a character offset
(loadfile_tlv[i * 2:(i + 240) * 2]), so every LOAD block after the first was
a 1-byte-shifted copy of the previous one - the cap header repeated every
239 bytes on the wire. A live install accepted three blocks, failed block 4
with SW 6400, then 6985, and INSTALL [for install] answered 6A88. The same
slicing was inherited by the SCP81 helper from the SCP80 path, so
multi-block caps could not install there either; both are fixed.
Tests: blocks are consecutive and reassemble the C4 TLV byte-for-byte
(200 python); findings updated; service worker v149.
The C-APDU TLV length was written as a raw byte: a 245-byte LOAD command
produced 'AE 80 22 F5 ...', which BER reads as a long-form marker, so the
card mis-parsed every script command over 127 bytes. Small commands worked,
which made a RAM install look alive: the card answered the LOAD steps with a
degenerate 'AF 80' body (no R-APDU), and the final INSTALL [for install]
failed with 6A88 because the package never loaded.
Both the indefinite ('22' TLV) and definite ('AA' outer) lengths are now
BER-encoded (same rule as the BIP channel data TLV fix). Tests: 245-byte
LOAD body, short form, definite variant (200 python, 346 frontend);
findings doc updated; service worker v148.
- shared _cap_apdu_sequence helper (INSTALL [for load] -> 240-byte LOAD
blocks -> INSTALL [for install]); the SCP80 /api/ram-install path now uses
it too (one source of truth; byte-level tests pin the APDUs)
- POST /api/scp81/ram-install: parses the .cap server-side and queues the
APDU sequence as the SCP81 command script (one C-APDU per POST, runs on the
card's next push); refused while a script is mid-run unless force
- /api/scp81/script reports the script kind (explore/none/custom/ram-install)
- tab: RAM install row (CAP file + SD AID + Queue button); RU strings
- docs: api.md, findings, AGENTS; service worker v145
Long GET STATUS listings answer SW CAFE with 127-byte pages. The script
responder now extracts the last complete AID from the page and inserts a
next-occurrence GET STATUS (80F2 <P1> 02 <Lc> 4F <len> <AID> 00) as the
next command, until the listing ends. Pages are logged (script-page), a
repeated page logs script-page-stalled and stops, inserted continuations are
dropped at session start.
Live-verified: ELF registry and applications collected completely in two
pages each (7/7 commands, all script-status ok).
- tests: page parsing (truncated tails, live FC-prefixed junk), continuation
bytes, auto-insert, stall guard, per-session purge (192 python + 337 frontend)
- UI: page number and script-status log rendering; service worker v143
- scp81.py: PSK TLS listener (stdlib ssl PSK callbacks) speaking the GP
HTTP administration dialog; configurable framing (chunked/Content-Length,
TLS record split, Apache-style/compact headers, Connection header,
keep-alive, Next-URI template with %d, TLS version/cipher, answer delay,
keylog for capture decryption)
- server.py: script responder + Response Scripting parsing (AF/AB, 80/23
TLVs), memory decoder, SCP81 start options, terminal-side timer
management, background-mode BIP events, permissive OPEN CHANNEL
- BIP fix: the RECEIVE DATA channel-data TLV length is BER long form
(36 81 <len>) above 127 bytes; a raw length byte is mis-parsed on the
card, so the large TLS records never reached its stack (a live card
fetched the script response and silently never processed it - endless
resume). The card now executes scripts and returns R-APDUs: memory
(13 applets, 50646 B NV free, 2402 B volatile), ISD, stored HTTP OTA
parameters, ELF and application registries
- frontend: SCP81 tab (listener, script selection, HTTP OTA log), phone
event forms, i18n; service worker v141
- docs: api.md, scp81-findings.md (attempt matrix + root cause analysis);
tools/scp81_decrypt.py decrypts listener captures via the keylog
- tests: 187 python + 337 frontend
Groundwork for HTTP OTA emulation (GP RAM over HTTP / SCP81): the server
now emulates the terminal side of Bearer Independent Protocol and redirects
the card's TCP channel to a locally configured target.
- new pysim_otaman_server/httpota.py: BER-TLV parser for proactive commands,
BipTerminal (per-channel TCP client with rx buffering, redirect target,
bounded event log) and TcpDumpServer (raw capture listener used to
calibrate the card's TLS ClientHello before the PSK platform exists)
- server.py: BIP commands 0x40-0x43 (+0x44) are handled in
_handle_proactive_chain with TR payloads matching the captured real
terminal traces (result first, Channel status with link-established bit,
Bearer description, Buffer size, Channel data/length); connection
failures return result 3A/00; names and decoders added for the proactive
log; /api/scp81/status, /log, /log-clear and /bip (start/stop dump mode)
- tests: TR byte vectors from the traces, TLV parsing, open/send/receive/
close flow over a local peer, dump listener; 123 Python tests pass
pySim's lchan.select() resolves names against global selectables (self +
parent chain + MF children + applications) and falls back to probe_file(),
which blindly SELECTs an unknown FID and permanently injects a dynamically
named DF.XXXX/EF.XXXX into the running filesystem model. Probing a whole tree
or scanning a snapshot with custom files therefore polluted the model, made
tree branches show children of the wrong object, and could persist phantom
files into snapshots.
- server: new _select_with_parent()/_select_path() walk the requested parent
path (new parent_path field, parent_sel kept as legacy fallback) strictly
through the model and call lchan.select_file() only; model-unknown 4-hex
segments are probed only with allow_probe and the temporary child pySim
adds is detached again via the cleanup callable that the four handlers
(/api/tree|select|read|write) now run in a finally block
- frontend: getParentPath() builds the segment chain (MF, ADF names, FIDs)
and all tree/select/read/write bodies plus the snapshot/profile walker send
parent_path; allow_probe is set only for custom files; the blind retries
in the file manager were dropped
- tests: tests/test_select_scope.py (duplicate-FID resolution, no APDU for
unknown non-custom files, probe+detach, model unchanged); fs_load/fs_probe
assertions for parent_path and allow_probe; docs/api.md and AGENTS.md
document the contract; SW cache v117 -> v118.
The stuck-card pattern: after a Back TR the card re-issues the parent menu
as a new SELECT ITEM (91XX -> FETCH, paused, awaiting TR), but the UI's
back/timeout branch ignored that response and showed the cached top menu;
the next item click then sent ENVELOPE(Menu Selection) while the card was
waiting for the TR. The card answers such an ENVELOPE with 9000 (not the
usual 91XX), and menu-select cleared the pending command without a TR,
leaving an unfinished proactive session until reset/equip.
- server: _finish_pending_menu() answers a paused command with a cancel TR
(0x10) before /api/menu-select sends its ENVELOPE and drains a 91XX
follow-up, so a new selection can never shadow an unanswered FETCH
- frontend: stkMenuRespond('back'|'timeout') renders the follow-up command
from the server response (SELECT ITEM / DISPLAY TEXT) and shows the
cached top menu only when the TR answer carries no command (9000)
- tests: finish-pending cancel TR + chain drain (Python); stkMenuRespond
back/timeout/cancel/ok rendering (frontend); help/AGENTS updated;
SW cache v107 -> v108.
Server measures every classified APDU (A4 select, B0 read binary, B2 read
record) from command to response: _collect_apdu_times() enables the tracer
(reattaching it if pySim nulled it) and /api/select + /api/read return
'apdu_times': [{type, ms}]. Collection is safe: handlers hold _CARD_LOCK.
Snapshots store per-file {select_ms, read_ms}, a ms value per record and
snapshot-level stats {select, read_binary, read_record}: {min, max, avg,
count} plus total_ms (wall time of the scan). The snapshot view gets a
summary under the title (files/records counts, scan time, min/avg/max per
command type) and shows select/read per file and read time per record.
Timings are display-only: checks, snapshots comparison and imports ignore
them (old snapshots simply show 'No timing data').
Tests: Python classifier/collection (tests/test_apdu_timing.py) and
frontend stats/accumulator/format/build-file/summary. SW cache v99 ->
v100; help, README and docs/api.md updated.
- _apply_equipped_card() centralizes the post-equip refresh + TERMINAL
PROFILE (shared by the /api/command equip branch and auto-equip)
- server tracks card_session (bumped on equip and disconnect) and
equipping; /api/status exposes connected, card_present, card_session,
equipping, auto_equip and is exempt from _CARD_LOCK (pure cached state)
- auto-equip is on by default (--no-auto-equip; off with --no-card-init):
the presence observer spawns a one-shot worker after insertion, which
runs equip under _CARD_LOCK and applies the same refresh; the monitor
starts after the startup init so pyscard's initial 'already present'
event does not re-equip a fresh session
- UI: /api/status polls every 2s (other views stay at 5s); when
card_session changes it runs pysimResetCardData() (STK overlay, file
tree, events, proactive log, PLI, status) — the same reset as a manual
Equip; messages: initializing / press Equip / no card
Tests for the observer and auto-equip rules, session bumps,
_apply_equipped_card, and the UI state machine. SW cache v91 -> v92.
The UI only noticed a removed card when some user action ran a real card
command (e.g. Check status); /api/status is a cached-state read that kept
returning the old card, and _handle_card_disconnect() did not clear
app.card/rs.
- start_card_monitor() registers a pyscard CardObserver for our reader;
it only polls SCardGetStatusChange (no APDU, no connection, no extra
process), and on removal sets server.card_present=False and calls
_handle_card_disconnect() under _CARD_LOCK
- /api/status now exposes connected (session usable) and card_present
(physically inserted) and masks card/profile/atr/selection when not
connected; _CARD_CONNECTED is initialized from card presence instead of
being unconditionally True
- the 2s UI poll includes /api/status; on disconnect it switches to the
existing 'No card detected. Insert card and click Equip' state, or the
new 'Card inserted — press Equip' hint when the card is back; the old
_hadData heuristic is gone
Tests for the observer (filtering, removal, insertion) and the UI state
transitions. SW cache v90 -> v91.
A probe can leave a card in a context where CLA-00 file access returns
6d00, so the software MF select fails (stock pysim init survives only
because it resets physically after probing). Fast init now recovers the
same way, on demand:
- FastRuntimeState.reset() falls back to hard_reset() on
SwMatchError/ProtocolError (logged as FAST-RESET)
- init_card_fast() retries once after sl.reset_card() (FAST-INIT)
- __main__ falls back to stock pysim init once and skips
TERMINAL PROFILE/drain when no card was initialized (no more
6d00/6985 noise after a failed init)
- do_equip_fast() no longer pre-unregisters command sets; PysimApp.equip
does that after a successful init, so a failed equip keeps the
previous card/rs instead of leaving the app unequipped
Tests for the escalation, the retry and failed-equip state retention.
No card-model special cases.
Every FETCHed proactive command must be answered, otherwise the card is
left in an unfinished session and stops issuing commands (e.g. it will
not deliver a PoR for SEND SM). The menu handlers now share
_menu_send_response, and a server-side watchdog (_arm_menu_timeout /
_menu_timeout_fire, --menu-timeout, default 60s, 0 disables) sends the
timeout result (0x12) when the user never answers. The timer is armed
while a command is pending and cancelled on any response, equip, rescue
and card disconnect.
Also fixes docs/api.md, which had back (0x11) and timeout (0x12) codes
swapped. Tests for arm/cancel/clamping and the flat timeout TR. SW cache
v89 -> v90.
pysim's init_card() resets the card once per profile candidate in
CardProfile.pick(), once in RuntimeState.__init__ and again in
PysimApp.equip(); on typical readers each reconnect costs ~1.3s and a
normal init/equip does 7-8 of them (measured: equip 9.85s, startup
~14.8s).
fastinit.py mirrors pySim.app.init_card() with the resets removed:
pick_profile_no_reset() runs all profile probes back-to-back on one
connection, FastRuntimeState.reset() is a software reset (select MF,
clear selected_adf/scp, ATR from the transport) and the equip/reset
commands are routed through do_equip_fast (one reconnect via
wait_for_card) and do_reset_fast (always a physical reset). Enabled
with --fast-init; stock behavior remains the default.
Tests for the reset-free pick, the soft reset and the explicit reset
paths. Docs updated; SW cache v88 -> v89.
--poll-interval 0 now really disables background STATUS polling (it was
clamped to 1s, and the equip branch force-enabled it anyway). Request
handlers and the poll thread now share _CARD_LOCK so a poll can never
interleave a FETCH/TERMINAL RESPONSE pair — the baseline log showed
AUTO-STATUS chains and duplicate FETCHes inside the equip TP chain.
--timing adds elapsed timestamps, per-reset logging (RESET #n) and
phase durations for startup (init_reader, card_init, pysim_app,
terminal_profile_drain) and equip (onecmd, terminal profile).
AGENTS.md documents the proactive-TR invariant: never fetch without
answering, never fetch twice, one APDU conversation at a time.
Incoming: _parse_sms_concat parses UDH for IEI 0x00 (8-bit ref) and
IEI 0x08 (16-bit ref). PoRSubmitHandler accumulates segments, sorts by
num, and reassembles complete payload when all parts arrive.
Outgoing: MAX_ENVELOPE_SEGMENTS=5 limits the secured packet to 5
ENVELOPE chunks (650B max at 130B/chunk).
10 new tests for concat parsing and reassembly.
- pysimSendOta: Response Parser filled strictly from decoded PoR
(por.decoded) for both SPI2 variants; envelope SW/failures no longer
leak into it — they render in a new inline result line (#sp-send-result)
next to Send to Card, always including SW on failure
- Default-level stderr tracing per send (no flags needed):
'OTA SEND: SPI .. KIc .. KID .. TAR .. CNTR .. LEN ..B CHUNKS N',
'OTA SEND FAILED: chunk N SW xxxx' and
'OTA PoR[envelope|sms-submit]: status=.. TAR=.. CNTR=.. PCNTR=..
RPL=.. RHL=..' (+ compact summary / undecodable raw / none fallbacks)
- _decode_por: surface every parsed PoR field verbatim (cntr, rpl, rhl,
cc_rc, raw) instead of status/tar/pcntr only
- genSp: CNTR normalization padEnd -> padStart so short input like '1'
becomes 0000000001, not 1000000000 (counter is big-endian 5 bytes)
- tests: TestDecodePor completeness + cntr_low field-report cases
Version 1.9.3 -> 1.9.4 everywhere; SW cache otaman-v14 -> otaman-v15
Server:
- Track all TERMINAL RESPONSEs (chain, TP, pySim auto-handler, menu-respond
link to paused entry) with shared _build_tr/_record_tr
- Log entry ids + cmd_num; server-side decode of fetched commands and TRs
(compact PLI port); Result CTLV extracted as tr_result/tr_result_name
- _LoggingApduTracer captures TR SW for pySim auto-handler responses
- _DefaultProactiveHandler answers PLI with editor data (no re-request)
- VERSION 1.8.0
Frontend:
- Expandable proactive log rows (click to expand, state survives re-render)
- Response section: Result name + hex, decoded fields, width-restrained
readonly hex inputs; no SW display
- Rename Expanded Script tab/pill/button/docs; proactive UICC view font
sizing increase; card reader pills text-sm
catarrh