catarrh/otaman
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scp81: PSK TLS server and command scripting (phases B/C) + BIP fix (v2.1.5)

Browse Source 
- scp81.py: PSK TLS listener (stdlib ssl PSK callbacks) speaking the GP
  HTTP administration dialog; configurable framing (chunked/Content-Length,
  TLS record split, Apache-style/compact headers, Connection header,
  keep-alive, Next-URI template with %d, TLS version/cipher, answer delay,
  keylog for capture decryption)
- server.py: script responder + Response Scripting parsing (AF/AB, 80/23
  TLVs), memory decoder, SCP81 start options, terminal-side timer
  management, background-mode BIP events, permissive OPEN CHANNEL
- BIP fix: the RECEIVE DATA channel-data TLV length is BER long form
  (36 81 <len>) above 127 bytes; a raw length byte is mis-parsed on the
  card, so the large TLS records never reached its stack (a live card
  fetched the script response and silently never processed it - endless
  resume). The card now executes scripts and returns R-APDUs: memory
  (13 applets, 50646 B NV free, 2402 B volatile), ISD, stored HTTP OTA
  parameters, ELF and application registries
- frontend: SCP81 tab (listener, script selection, HTTP OTA log), phone
  event forms, i18n; service worker v141
- docs: api.md, scp81-findings.md (attempt matrix + root cause analysis);
  tools/scp81_decrypt.py decrypts listener captures via the keylog
- tests: 187 python + 337 frontend
This commit is contained in:
Антон Трошин
2026-09-16 01:50:29 +03:00
parent ea1730b206
commit 175ca934d8
18 changed files with 3049 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/api.md
+95
View File
@@ -46,6 +46,10 @@ connect and warns if versions are incompatible.
| `/api/pli-qualifiers` | GET | List of qualifier codes with descriptions |
| `/api/pli-dict` | GET | Current dictionary (hex values per qualifier) |
| `/api/pli-dict` | POST | Update dictionary entries |
| `/api/scp81/bip` | POST | Start/stop the HTTP OTA listener (dump capture or PSK TLS server) |
| `/api/scp81/status` | GET | BIP terminal + listener state (channels, PSK identity seen) |
| `/api/scp81/log` | GET | HTTP OTA event log (`?after=<seq>`) |
| `/api/scp81/log-clear` | POST | Clear the HTTP OTA event log |
## Endpoint details
@@ -378,6 +382,13 @@ optional hex for events that carry data. Returns the SW and any response data:
{"sw": "9000", "data": "..."}
```
Channel status (event `0x0A`, TS 102 223 §8.56) carries the Channel status TLV
`B8 02 <status> <info>`, where the status byte is the channel id (1–7) OR-ed
with the state bits (0x00 link not established / 0x40 TCP LISTEN / 0x80 link
established) and the info byte is `00` (no further info) or `05` (link
dropped). The server also sends this event automatically when a BIP link drops
outside a proactive command and the card subscribed to `0x0A`.
### `GET /api/proactive-log`
Returns the last 50 proactive commands fetched during CAT sessions, newest
@@ -444,3 +455,87 @@ Returns the current PLI data dictionary as a qualifier-code map.
Updates dictionary entries. Body is a map of qualifier code to hex value; keys
must be known qualifiers and values valid hex, otherwise they are ignored.
Returns the updated dictionary.
### `POST /api/scp81/bip`
Starts or stops the local target the card's BIP channel is redirected to.
Dump mode (default) captures whatever the card sends (e.g. its TLS
ClientHello) without answering:
```json
{"action": "start", "mode": "dump", "host": "127.0.0.1", "port": 8443}
```
TLS mode runs the Phase B PSK TLS server (GPC v2.2 Amendment B): the PSK key
and optional identity are applied to the TLS handshake, and the GP HTTP
administration dialog (`X-Admin-*` headers, 200 with a command string or 204
No Content) is served. `psk_hex` is required (the previous key is reused when
omitted); `psk_identity` restricts the accepted identity. The key is never
stored or logged.
```json
{"action": "start", "mode": "tls", "host": "127.0.0.1", "port": 8443,
"psk_hex": "00112233445566778899aabbccddeeff",
"psk_identity": "89012345678901234567"}
```
Stop either mode with `{"action": "stop"}` (also disables the BIP terminal).
### `GET /api/scp81/status`
```json
{"bip": {"enabled": true, "target": "127.0.0.1:8443", "channels": [], "seq": 12},
"listener": {"mode": "tls", "host": "127.0.0.1", "port": 8443,
"psk_identity": null, "identity_seen": "89012345678901234567"}}
```
### `GET /api/scp81/log`
Returns the BIP/TLS event log (open/close, SEND/RECEIVE DATA hex, TLS
handshake and HTTP request/response records). `?after=<seq>` returns only
newer entries; `seq` echoes the latest sequence number.
### `GET /api/scp81/script`
Returns the active command script and the R-APDUs collected so far:
```json
{"script": ["80CAFF2100", "80F28002024F0000"], "sent": 1,
"results": [{"index": 1, "sw": "9000", "rapdu": "FF210C810102..."}]}
```
The script is selected when starting the TLS listener with the `script`
parameter: `explore` (default — the reference administration server's command
sequence: GET DATA FF21 extended resources / free memory, GET STATUS P1=80
Issuer Security Domain, GET DATA 0085 HTTP administration parameters, GET
STATUS P1=40 executable load files and P1=10 applications), `none` (answer
every POST with 204), or an explicit list of APDU hex strings. Each APDU is
delivered in an `AE 80 22 <len> <apdu> 00 00` Command Scripting template
(TS 102 226 §5.2.1) with `X-Admin-Next-URI`; the card returns its R-APDUs in
the next POST's Response Scripting template, which is parsed and logged
(`script-rapdu`, `script-memory`).
TLS mode also accepts `chunked` (**default `true`** — the reference server's
chunked framing; the card rejects a chunked response that also carries a
Content-Length) and `chunk_size` (default `0` — the whole response in one TLS
record, as in the decrypted reference session; a positive value writes the
head and each body piece as its own record). Both are echoed by
`GET /api/scp81/status`.
`keep_alive` (default `true`, matching the reference session: the card sends
all its POSTs on one connection until the 204) ends the TLS connection after
each response
(after the card drained the BIP buffer, with `close_notify`, so the card
processes the script and opens a new connection for its next POST);
`compact_headers` (default `false`) drops the space after each header colon,
`apache_headers` (default `true`) adds Date/Server/X-Powered-By like the
reference servers and puts Transfer-Encoding before Content-Type,
`conn_header` (default `'none'` = omit the header, like the reference)
declares the connection fate, `tls_version` pins `1.1`/`1.0` for cards that
only speak the older record layer, `cipher` pins one suite, `next_uri`
overrides the per-command `X-Admin-Next-URI` (`%d` = command id; empty string
omits the header), `link_events` (default `true`) controls the automatic
Channel status events, `answer_delay` waits before answering a request. `keylog` writes the TLS traffic secrets to
the given file (SSLKEYLOGFILE format) for debugging captures — it contains key
material, use a temporary path.
docs/scp81-findings.md
+200
View File
@@ -0,0 +1,200 @@
# SCP81 / HTTP OTA live-card findings
Living debug log for the HTTP OTA (RAM over HTTP) work against the live UICC.
Purpose: record **every attempted configuration and its outcome**, so the same
variations are not repeated. Add rows as tests are run; keep the confirmed
rules section current.
Setup: `pysim_otaman_server` with a PC/SC reader, the PWA SCP81 tab (or
`POST /api/scp81/bip`), the card triggered by its SMS-PP push / the Location
status event. Server log at `GET /api/scp81/log`, script state at
`GET /api/scp81/script`, proactive history at `GET /api/proactive-log`.
## RESOLVED 2026-09-16: the card never received the response - BIP TLV bug
**Root cause:** our RECEIVE DATA TERMINAL RESPONSE encoded the channel-data
TLV length as a raw byte (`36 ED ...` for a 237-byte chunk). BER requires the
long form for lengths >127: **`36 81 ED ...`** (the reference terminal traces
use exactly that, e.g. `push_3311_success_req2.pcapng`). The card's BIP layer
silently mis-parsed the malformed TLV, so the TLS record bytes never reached
its TLS stack: no alert, no script processing, and the SD kept resuming its
dialog ("no complete script received") forever. Every delivery <=127 bytes
(handshake records, 204 responses) always worked - which is why the handshake
succeeded and only the large script responses "vanished".
**Fix:** `_handle_bip_command` (cmd 0x42) BER-encodes the channel data length
(`36 81 <len>` above 127); regression test
`test_receive_data_tlv_long_form_length`.
**Result with the live card** (one push, `explore` script, 5/5 commands):
```
#1 80CAFF2100 SW 9000 FF210B 81010D 8202C5D6 83020962 (13 applets,
free NV 50646 B, free volatile 2402 B)
#2 80F28002024F0000 SW 9000 ISD A000000003000000 + D276000005AAFFCAFE00
#3 80CA008500 SW 9000 stored HTTP OTA parameters
#4 80F24002024F0000 SW CAFE 127-byte ELF registry page (more available)
#5 80F21002024F0000 SW CAFE 127-byte applications page (more available)
```
Every command returned `X-Admin-Script-Status: ok` on the card's own POST to
the incremented `X-Admin-Next-URI`, on the same keep-alive connection, and the
session ended with 204 + mutual close_notify - exactly the reference flow.
`SW CAFE` marks a truncated 127-byte page: the remaining entries need a
continuation GET STATUS (P2=02 with the last AID as search criterion).
## Live card facts (verified via the reader, 2026-09-16)
- `80CAFF2100` (GET DATA extended card resources) **works**:
`FF21 0B 81 01 0D 82 02 C5 D6 83 02 09 62` -> 13 applets installed,
free NV memory `0xC5D6` = 50646 B, free volatile `0x0962` = 2402 B.
- `80CA008500` (GET DATA HTTP administration parameters) **works** and returns
the SD's stored OTA configuration: `8A 09 "localhost"`, `8B 14 <agent id>`,
`8C 01 "/"` (stored URI), `85 14 <PSK identity>`, `86 07 00 01 25 03 00 10 00`
(retry counter 1, timer **10 minutes**), `02 40 01` (KVN/KID), APN-ish
`C7 04 03 47 50 42`, destination `BE 05 21 5B D5 05 02` = 91.213.5.2.
- `80F28002/80F24002/80F21002 ...4F0000` return `6985` through the reader when
the ISD is not the current DF; the reference platform sends
`80F28002024F0000` over HTTP, where the SD executes inside the ISD.
- `SELECT` of the ISD (`00A4040008A000000003000000`) returns `6112`;
a subsequent GET RESPONSE (`00C0000012`) returns `6D00`.
- BIP device identities: OPEN CHANNEL uses destination `0x82`; SEND/RECEIVE
DATA carry channel `0x21..0x27` (e.g. `82 02 81 22` = channel 2).
- Subscribed events (`99 03`): `03` location status, `09` data available,
`0A` channel status.
- A Location status event re-triggers the OTA session only while the last
session is incomplete; after a clean session end the card waits for a push.
- The SD stores a 10-minute retry timer (`25 03 00 10 00`).
## Confirmed rules (with evidence)
1. **The card needs a clean TLS close, with the close_notify actually
fetched.** Keep-alive (no close) -> fatal `unexpected_message` after it
fetched the response. `close_notify` sent *after* the buffer drained is
never fetched (the card ends the dialog on its own first). Correct order:
send it while the response still waits, then wait for the drain, then
close.
2. **The card's abort alert is `fatal unexpected_message`** - decrypted with
the listener's `keylog` option (see `tools/scp81_decrypt.py`).
3. **A dropped link must be signalled (TS 102 223 7.5.11), and only after the
buffered data was fetched.** Signalling the drop while bytes are still in
the BIP buffer makes the card abort the fetch mid-record and end the
session. Omitting the signal entirely hangs the SD: after a listener
restart dropped the channel silently, the card ignored pushes and location
events for minutes; a manual `ENVELOPE (Channel status, B8 02 02 05)`
immediately made it start a fresh session.
4. **The Next-URI shape matters.** A path-only or absolute Next-URI (`/`,
`/1`, `http://127.0.0.1:8443/api/scp81`) draws the fatal
`unexpected_message`; the reference-style relative path **with a query**
(`/adminserver?PHPSESSID=...&apdu_id=101`) does not.
5. **The reference administration server** (`samples/HTTP_OTA/
httpota_adminserver_php_v2`) uses: command script
`AE 80 22 <len> <apdu> 00 00`; response `200` with
`X-Admin-Protocol`, `X-Admin-Next-URI: /adminserver?PHPSESSID=<id>&apdu_id=<n>`,
`Content-Type: ...;version=1.0`, **chunked** body (100-byte chunks);
the card returns the R-APDU as the body of its next POST with
`X-Admin-Script-Status: ok`; the server ends with `204`.
Its log proves the card followed the Next-URI three times within 1-2 s per
step (`Got next request ... Script status is 'ok' - storing R-APDU data`).
6. **`chunked=false` (Content-Length) has never produced an R-APDU.** All
sessions that ended silently (clean close, no alert, no POST) used
`Content-Length`. Hypothesis: the card only treats a chunked body as a
command script; with Content-Length it sees an empty script, executes
nothing and ends the session gracefully.
## The one fully successful session trace (ground truth)
`traces/HTTPOTA_session_3311_success1.pcap` (2019, **plain HTTP on port 80**,
one TCP connection for the whole session, card `3311` - *not* our UICC):
```
POST /server/adminagent?cmd=1 <- card (trigger URI, with query!)
200 OK + Date/Server + X-Admin-Protocol
+ X-Admin-Next-URI: /Download?req=1 + Content-Length: 11
+ Content-Type: .../card-content-mgt;version=1.0
body: ae 80 22 05 80 ca 00 85 00 00 00 (script: GET DATA 0085)
POST /Download?req=1 <- card, SAME connection
X-Admin-Script-Status: ok
Content-Type: .../card-content-mgt-response;version=1.0
Transfer-Encoding: chunked
body: "8
" af 80 23 02 6a 88 00 00 "0
" (R-APDU SW 6A88)
200 OK + X-Admin-Next-URI: /Download?req=2 + Content-Length: 14
body: ae 80 22 08 80 f2 80 02 02 4f 00 00 00 00 (GET STATUS P1=80)
POST /Download?req=2 -> X-Admin-Script-Status: ok, chunked
body: "1F
" af 80 23 19 <25-byte R-APDU ... 90 00> 00 00 "0
"
200 OK + /Download?req=3 + 11-byte script
POST /Download?req=3 -> status ok, R-APDU 23 02 6d 00 (SW 6D00)
204 No Content <- session ends
```
Confirmed from it: the card echoes the `X-Admin-Next-URI` (path *and* query)
verbatim; its response POST goes on the **same TCP connection**; its response
is the `AF 80 23 <len> <R-APDU> 00 00` indefinite Response Scripting template
(in a chunked body, with `X-Admin-Script-Status`); the server's script
`AE 80 22 <len> <APDU> 00 00` matches ours byte for byte; the server uses
`Content-Length` (not chunked), no `Connection` header (implicit keep-alive),
and ends with 204.
## Attempt matrix
| # | transport | framing | Next-URI | close | link events | outcome |
|---|-----------|---------|----------|-------|-------------|---------|
| 1 | dump mode only | - | - | - | off | OPEN CHANNEL + ClientHello captured (Phase A) |
| 2 | TLS, 204 only | - | - | yes | off | session completes cleanly, no alert (Phase B, live) |
| 3 | TLS + script | chunked 100 | `/N` | early (raced fetch) | on | fetch truncated (237/399); card re-opened and repeated its POST with `X-Admin-Resume: true` -> breakdown-resume works |
| 4 | TLS + script | chunked 100 / single | `/1`, `/`, absolute | keep-alive | off | full fetch, then fatal `unexpected_message` (Next-URI shape) |
| 5 | TLS + script | single | none (`""`) | keep-alive | off | no alert, no POST, session left open (spec: no Next-URI -> no response) |
| 6 | TLS + script | chunked 100 | reference | close_notify after drain | off | full fetch, alert (notify never fetched) |
| 7 | TLS + script | chunked 100 | reference | close_notify before drain | off | full fetch, alert (head split into its own record) |
| 8 | TLS + script | **single record** | reference | drain + close_notify | off | **no alert**, card CLOSE CHANNELs, no R-APDU (`chunked=false` -> suspected empty script) |
| 9 | TLS + script | single record | reference | keep-alive (no close) | off | fatal `unexpected_message` (close required) |
| 10 | TLS + script | chunked 100 | reference | drain + close_notify | off | full fetch, then alert; later the SD hung until a manual link-dropped event |
| 11 | TLS + script | single record | reference | keep-alive | off | fatal `unexpected_message` after the full fetch (no close) |
| 12 | TLS + script | single record | reference | drain + close_notify | off | **no alert**, card CLOSE CHANNELs, no R-APDU (`Content-Length`) |
| 13 | TLS + script | chunked100 + single | reference | drain + close_notify | off | no alert, no R-APDU |
| 14 | TLS + script | single record | reference | keep-alive | off | alert again |
| 15 | TLS + script | chunked 100 | reference | keep-alive | on | alert (small records, ruled out record size) |
| 16 | TLS + script | single record | reference | keep-alive, no `Connection` header | on | alert |
| 17 | TLS + script (RFM! `00D6` write-probe) | chunked, single | reference | drain + close_notify | on | no alert, no R-APDU; EF.SPN unchanged - **RFM result is void**: the ISD only accepts RAM commands |
All script attempts used the `explore` list, except #8-#17 which used only
`80CAFF2100` (or the RFM probe). #3-#17 ran with the card's PSK identity
`89390…903` (push trigger) or `89701…` (event trigger).
**Status after #17 (superseded by the 2026-09-16 resolution above):** the
failures were caused by the BIP TLV length bug, not by the HTTP/TLS details;
resume mode was a symptom (the working session even started as a resume). The
key working recipe (also now the server default): one keep-alive connection,
Apache-style headers, `Transfer-Encoding: chunked` body with the script in
one TLS record, no Connection header, `X-Admin-Next-URI` with a query whose
command id increments.
**Also confirmed:** a TLS half-close (close_notify then keep reading for the
card's POST which RFC 5246 leaves open in practice) cannot be done with
CPython's `ssl`: `SSLSocket.unwrap()` with a short timeout raises and poisons
the session (tested), so the `half_close` option is a documented no-op.
## Next tests / work
1. **Continuation pages:** follow `SW CAFE` (127-byte listing pages) with
GET STATUS P1=40/10 P2=02 using the last returned AID as the search
criterion, and append the pages to the result set (memory + full ELF and
application registries).
2. **UI:** show the decoded memory/applications results (and page merging) in
the SCP81 tab; expose the framing options there.
3. Load/store operations (RAM INSTALL/LOAD) over SCP81 using the same recipe.
## Tooling
- `tools/scp81_decrypt.py <log.json> <keys.log>` - decrypts the dialog from
`GET /api/scp81/log` plus the listener's `keylog` file (SSLKEYLOGFILE
format; PSK-AES128-CBC-SHA256, TLS 1.2 PRF + OpenSSL CLI). Shows each
record's plaintext and any alert level/description.
- Start the listener with `"keylog": "/tmp/.../scp81.keys"` to collect the
secrets (contains key material - use a temp path, never commit).