v3.31.0: the EF_PBR (phonebook reference) support

EF_PBR ('4F30', TS 31.102 4.4.2.1) lists the phonebook's files as
BER-TLVs, and those files are not in pySim's model - so they were
invisible in the file tree.

- The PWA gains the decoder (efDecPbr, registered as EF.PBR/4f30): the
  'A8'/'A9'/'AA' type-1/2/3 containers, the 'C0'-'CC' kinds, the 02/03
  value lengths (FID / FID+SFI) and the master EF (the first FID after
  the first 'A8'), tolerant of FF padding, truncation, unknown tags and
  the spec's vague 'AA' path form (kept raw + noted, never a throw).
- The server reads the three PBR copies once per card session
  (best-effort, exactly like EF.ARR) into server.pbr and serves the raw
  records from the card-free GET /api/pbr.
- The file tree is populated SERVER-SIDE (no frontend involvement): a
  small Python parser reads the cached records and /api/tree appends the
  referenced files as children of the DF that holds the PBR
  ({'name','fid','isDir':false,'pbr':true,'sfi'}, deduplicated against
  the model), and /api/tree|select|read|write auto-allow the probe for
  exactly those FIDs, so the injected nodes are clickable without any
  PWA flag.  The node-to-PBR match is by the requested node's normalised
  path: a leading MF before an ADF segment is dropped and an ADF
  addressed by AID (the Profiler's walk) folds onto its name, so both
  DF_PHONEBOOK copies resolve to their own PBR; a full path never falls
  back to another DF's copy, and an ambiguous bare 5F3A injects nothing.
- Drift guard: the JS and Python parsers share
  frontend/tests/pbr_vectors.json, which includes the TS 31.102 Annex G
  record (its printed A8 length is internally inconsistent - 10
  primitives x 5 bytes = 0x32, not the printed 0x2D - so the corrected
  length is used, noted in the case name).

Docs: api.md (/api/pbr, the tree children, the auto-probe, the AA path
form), the repo AGENTS.  953 frontend + 989 Python; sw simple-v455.
This commit is contained in:
2026-10-18 01:47:04 +03:00
parent 30a742618e
commit 9ae9c39e55
9 changed files with 1237 additions and 10 deletions
+60 -1
View File
@@ -25,7 +25,7 @@ a 3.x PWA).
| `/api/command` | POST | pySim command (equip, status, tree, etc.) |
| `/api/commands` | GET | List available pySim commands |
| `/api/cardinfo` | GET | pySim `cardinfo` output |
| `/api/tree` | POST | File tree browser for given FID/name |
| `/api/tree` | POST | File tree browser for given FID/name (the DF holding EF_PBR gains the PBR-referenced files) |
| `/api/select` | POST | Select a file by name or FID |
| `/api/read` | POST | Read file content |
| `/api/write` | POST | Write raw hex data to a file |
@@ -80,6 +80,7 @@ a 3.x PWA).
| `/api/net-state` | GET | Cached network-state monitor (service, location, per-file state) |
| `/api/net-state-refresh` | POST | Re-read the monitored files (optional `files` list) and return the updated monitor state |
| `/api/arr` | GET | Cached EF.ARR copies (the FCP '8B' security attribute references): records per ARR file |
| `/api/pbr` | GET | Cached EF.PBR phonebook reference records (the raw TS 31.102 4.4.2.1 structure) |
| `/api/mcc-mnc` | GET | Search the optional MCC/MNC operator list (`?q=`; `?random=1&exclude=`) |
| `/api/proactive-log` | GET | Last 50 proactive commands |
| `/api/status-poll` | POST | Manual STATUS poll + FETCH if 91XX |
@@ -1457,6 +1458,34 @@ Returns:
{"exists": true, "name": "MF", "fid": "3F00", "file_type": "df", "children": [{"name": "EF.ICCID", "fid": "2fe2", "isDir": false}]}
```
**PBR-derived children**: when the requested node is the DF that holds the
card's **EF_PBR** (`4F30`, TS 31.102 §4.4.2.1 — `MF/7F10/5F3A`,
`ADF.USIM/5F3A` or `ADF.USIM-NON-IMSI/5F3A`), the server parses the cached
PBR records (see `GET /api/pbr`) and appends the phonebook files the PBR
references as children — the files are not in pySim's model, so they would
otherwise be invisible:
```json
{"name": "EF.ADN", "fid": "4F3A", "isDir": false, "pbr": true, "sfi": "02"}
```
`fid` is uppercase; `pbr: true` marks a PBR-derived node and `sfi` carries
the file's short file identifier when the PBR declares one (`null`
otherwise). A file pySim's model already lists is not added twice, and the
matching is by the requested node's full normalised path (uppercase, `3F00`
→ `MF`); a bare DF FID that cannot be told apart (the three PBR copies all
sit in a DF with FID `5F3A`) injects nothing. The PWA renders these nodes
like any other tree node — no frontend change.
Because the PBR files are not in the model, `POST /api/tree`, `/api/select`,
`/api/read` and `/api/write` **auto-allow the probe** for exactly the FIDs
the parsed PBR references (the PWA only sets `allow_probe` for its own custom
files), so the injected nodes can be selected and read without any client
flag. The probe stays scoped to those FIDs and the temporary model file is
detached again after the request. An explicit `allow_probe: false` does not
disable it — the server knows the file exists, so a caller cannot make the
tree's own nodes unselectable (the flag remains the PWA's custom-file switch).
### `GET /api/events`
Returns the event list captured from the card's SET UP EVENT LIST (an array of
@@ -1598,6 +1627,36 @@ FCI against this cache and shows the referenced record's access rules as
indented lines; snapshots resolve the same references from their own
captured `EF.ARR` copy.
### `GET /api/pbr`
Returns the card's cached **EF.PBR records** — the phonebook reference file
(`4F30`, TS 31.102 §4.4.2.1), whose BER-TLV structure describes the
phonebook files (the `'A8'`/`'A9'`/`'AA'` containers and the primitive
phonebook kinds). The copies are read once per card session when a card is
equipped (`MF/7F10/5F3A/4F30`, `ADF.USIM/5F3A/4F30`,
`ADF.USIM-NON-IMSI/5F3A/4F30`, best effort, at most `PBR_RECORD_LIMIT` = 16
records each) and served from memory, so this endpoint performs no card I/O;
a disconnect clears the cache.
```json
{"ok": true, "session": 12, "pbrs": [
{"fid": "4F30", "path": "MF/7F10/5F3A/4F30", "record_len": 32,
"num_records": 1, "records": ["A80AC0034F3A02C5034F0904AA0A..."]}]}
```
`ok` is `false` (with an empty `pbrs`) when the card has no readable PBR.
The records are served raw: the PWA owns the decoder for the decoded file
views (`efDecPbr`, TS 31.102 §4.4.2.1 — the containers, the primitive tags,
the master EF and the EF_IAP reference). The spec leaves the `'AA'`
container's "path to the file" form unspecified; a container that does not
parse as primitive tags is kept raw (the decoder notes it) and its files are
neither resolved nor injected into the tree — only same-directory FIDs are. The server keeps a small parser of
the same structure only to populate the file tree — `/api/tree` appends the
referenced phonebook files as children of the DF that holds the PBR and
auto-allows the probe for exactly those FIDs (see the `/api/tree` section).
The two parsers share the decode vectors in `frontend/tests/pbr_vectors.json`
(drift-guarded from both suites).
### `GET /api/proactive-log`
Returns the last 50 proactive commands fetched during CAT sessions, newest
+80 -2
View File
@@ -1943,7 +1943,7 @@
// ===== Version =====
// Single source of truth for the PWA version: shown in the header and used
// by the server version check in pysimConnect().
const SIMPLE_VERSION = '3.30.4';
const SIMPLE_VERSION = '3.31.0';
document.getElementById('app-version').textContent = 'v' + SIMPLE_VERSION;
// ===== view history (the browser Back/Forward buttons) =====
@@ -22553,6 +22553,8 @@ const EF_UST_SERVICES = {1: "Local Phone Book", 2: "Fixed Dialling Numbers (FDN)
const EF_SST_SERVICES = {1: "CHV1 disable function", 2: "Abbreviated Dialling Numbers (ADN)", 3: "Fixed Dialling Numbers (FDN)", 4: "Short Message Storage (SMS)", 5: "Advice of Charge (AoC)", 6: "Capability Configuration Parameters (CCP)", 7: "PLMN selector", 8: "RFU", 9: "MSISDN", 10: "Extension1", 11: "Extension2", 12: "SMS Parameters", 13: "Last Number Dialled (LND)", 14: "Cell Broadcast Message Identifier", 15: "Group Identifier Level 1", 16: "Group Identifier Level 2", 17: "Service Provider Name", 18: "Service Dialling Numbers (SDN)", 19: "Extension3", 20: "RFU", 21: "VGCS Group Identifier List (EFVGCS and EFVGCSS)", 22: "VBS Group Identifier List (EFVBS and EFVBSS)", 23: "enhanced Multi-Level Precedence and Pre-emption Service", 24: "Automatic Answer for eMLPP", 25: "Data download via SMS-CB", 26: "Data download via SMS-PP", 27: "Menu selection", 28: "Call control", 29: "Proactive SIM", 30: "Cell Broadcast Message Identifier Ranges", 31: "Barred Dialling Numbers (BDN)", 32: "Extension4", 33: "De-personalization Control Keys", 34: "Co-operative Network List", 35: "Short Message Status Reports", 36: "Network's indication of alerting in the MS", 37: "Mobile Originated Short Message control by SIM", 38: "GPRS", 39: "Image (IMG)", 40: "SoLSA (Support of Local Service Area)", 41: "USSD string data object supported in Call Control", 42: "RUN AT COMMAND command", 43: "User controlled PLMN Selector with Access Technology", 44: "Operator controlled PLMN Selector with Access Technology", 45: "HPLMN Selector with Access Technology", 46: "CPBCCH Information", 47: "Investigation Scan", 48: "Extended Capability Configuration Parameters", 49: "MExE", 50: "Reserved and shall be ignored", 51: "PLMN Network Name", 52: "Operator PLMN List", 53: "Mailbox Dialling Numbers", 54: "Message Waiting Indication Status", 55: "Call Forwarding Indication Status", 56: "Service Provider Display Information", 57: "Multimedia Messaging Service (MMS)", 58: "Extension 8", 59: "MMS User Connectivity Parameters"};
const EF_IST_SERVICES = {1: "P-CSCF address", 2: "Generic Bootstrapping Architecture (GBA)", 3: "HTTP Digest", 4: "GBA-based Local Key Establishment Mechanism", 5: "Support of P-CSCF discovery for IMS Local Break Out", 6: "Short Message Storage (SMS)", 7: "Short Message Status Reports (SMSR)", 8: "Support for SM-over-IP including data download via SMS-PP as defined in TS 31.111 [31]", 9: "Communication Control for IMS by ISIM", 10: "Support of UICC access to IMS", 11: "URI support by UICC", 12: "Media Type support", 13: "IMS call disconnection cause", 14: "URI support for MO SHORT MESSAGE CONTROL", 15: "MCPTT", 16: "URI support for SMS-PP DOWNLOAD as defined in 3GPP TS 31.111 [31]", 17: "From Preferred", 18: "IMS configuration data", 19: "XCAP Configuration Data", 20: "WebRTC URI", 21: "MuD and MiD configuration data", 22: "IMS Data Channel indication"};
const EF_PHASE_NAMES = {0: "phase 1", 2: "phase 2", 3: "phase 2 and higher"};
// TS 31.102 §4.4.2.1 EF_PBR primitive tags (the phonebook kinds).
const EF_PBR_KINDS = { 0xC0: 'EF.ADN', 0xC1: 'EF.IAP', 0xC2: 'EF.EXT1', 0xC3: 'EF.SNE', 0xC4: 'EF.ANR', 0xC5: 'EF.PBC', 0xC6: 'EF.GRP', 0xC7: 'EF.AAS', 0xC8: 'EF.GAS', 0xC9: 'EF.UID', 0xCA: 'EF.EMAIL', 0xCB: 'EF.CCP1', 0xCC: 'EF.PURI' };
// ===== EF content decoders (client-side, offline) =====
// Decode the contents of common EFs for the file manager, the snapshot view,
@@ -22583,6 +22585,7 @@ const EF_FIELD_LABELS = {
lac_range_high: 'LAC range high', show_in_hplmn: 'Show in HPLMN', hide_in_oplmn: 'Hide in OPLMN',
location_update_status: 'Location update status', eps_update_status: 'EPS update status',
update_status: 'Update status', access_tech: 'Access technology', p_tmsi_signature: 'P-TMSI signature',
master: 'Master EF', iap: 'IAP (EF.IAP)', links: 'Links', files: 'Files', note: 'Note',
};
const EF_GSM7_EXT = { 0x0A: '\n', 0x14: '^', 0x28: '{', 0x29: '}', 0x2F: '\\', 0x3C: '[', 0x3D: '~', 0x3E: ']', 0x40: '|', 0x65: '\u20ac' };
@@ -22618,6 +22621,8 @@ function efRstripFf(b) {
return b.subarray(0, end);
}
// Returns {tag, value, end} - `end` is the index just past the TLV, so a
// caller can tell a complete walk from a truncated tail (efDecPbr).
function efTlv(data) {
const out = [];
let i = 0;
@@ -22631,7 +22636,7 @@ function efTlv(data) {
for (let k = 0; k < n; k++) len = (len << 8) | data[i++];
}
if (i + len > data.length) break;
out.push({ tag: tag, value: data.slice(i, i + len) });
out.push({ tag: tag, value: data.slice(i, i + len), end: i + len });
i += len;
}
return out;
@@ -23280,6 +23285,77 @@ function efDecSume(b) {
return Object.keys(out).length ? out : { raw: efHex(b) };
}
// TS 31.102 §4.4.2.1 EF_PBR (4F30): the phonebook reference record - a
// sequence of BER-TLVs describing the phonebook structure (unused trailing
// bytes are FF). Three constructed containers: 'A8' type 1 (one record per
// master-EF record, mapped 1:1), 'A9' type 2 (mapped through the pointers in
// EF_IAP, in the order of the FIDs after the tag) and 'AA' type 3 (addressed
// by record identifiers). Each holds primitive tags whose value length says
// whether an SFI is present (02 = FID, 03 = FID + SFI). The first FID after
// the first 'A8' is the master EF; all linked files inherit its access
// conditions. One record covers at most 254 entries - a larger phonebook
// uses further records, decoded one at a time by the caller.
function efPbrHex(tag) {
return tag.toString(16).toUpperCase().padStart(2, '0');
}
function efDecPbr(b) {
if (efAllFf(b)) return { empty: true };
const data = efRstripFf(b);
const out = { master: null, iap: null, links: [], files: [] };
const notes = [];
const tlvs = efTlv(data);
// A trailing TLV whose length runs past the record: keep what parsed and
// say so instead of throwing (a corrupt/partial PBR must stay readable).
if (data.length && (!tlvs.length || tlvs[tlvs.length - 1].end !== data.length)) {
notes.push('truncated TLV at the end of the record');
}
let seenA8 = false;
for (const t of tlvs) {
if (t.tag !== 0xA8 && t.tag !== 0xA9 && t.tag !== 0xAA) {
notes.push('unexpected top-level tag ' + efPbrHex(t.tag));
continue;
}
const link = t.tag === 0xA8 ? 1 : (t.tag === 0xA9 ? 2 : 3);
const firstA8 = t.tag === 0xA8 && !seenA8;
if (t.tag === 0xA8) seenA8 = true;
const inner = efTlv(t.value);
const complete = t.value.length === 0
|| (inner.length > 0 && inner[inner.length - 1].end === t.value.length);
if (t.tag === 0xAA && !complete) {
// The spec's tag length may point at a path instead of a primitive
// list (TS 31.102 4.4.2.1): keep the raw bytes, never throw.
out.links.push({ link: link, tag: 'AA', kind: null, fid: null, sfi: null, raw: efHex(t.value) });
notes.push('the AA (type 3) container is not a primitive list - kept raw');
continue;
}
if (!complete) notes.push('truncated TLV inside the ' + efPbrHex(t.tag) + ' container');
for (const p of inner) {
const kind = EF_PBR_KINDS[p.tag] || null;
let fid = null, sfi = null;
if (p.value.length === 2) fid = efHex(p.value);
else if (p.value.length === 3) {
fid = efHex(p.value.subarray(0, 2));
sfi = efHex(p.value.subarray(2, 3));
}
if (!kind || fid === null) {
// An unknown or malformed primitive tag is preserved raw.
out.links.push({ link: link, tag: efPbrHex(p.tag), kind: kind, fid: fid, sfi: sfi, raw: efHex(p.value) });
notes.push('unknown or malformed primitive tag ' + efPbrHex(p.tag)
+ ' (' + efHex(p.value) + ') - kept raw');
continue;
}
out.links.push({ link: link, tag: efPbrHex(p.tag), kind: kind, fid: fid, sfi: sfi });
if (!out.files.includes(fid)) out.files.push(fid);
if (firstA8 && !out.master) out.master = fid;
if (p.tag === 0xC1 && !out.iap) out.iap = fid;
}
}
if (!out.links.length) return { raw: efHex(b), note: notes.join('; ') || undefined };
if (notes.length) out.note = notes.join('; ');
return out;
}
// ---- registry ----
const EF_DECODERS = [
@@ -23336,6 +23412,7 @@ const EF_DECODERS = [
{ name: 'EF.EXT4', fid: '6f4e', fn: efDecExt1 },
{ name: 'EF.EXT5', fid: '6f55', fn: efDecExt1 },
{ name: 'EF.SUME', fid: '6f54', fn: efDecSume },
{ name: 'EF.PBR', fid: '4f30', fn: efDecPbr },
];
function efFindDecoder(name, fid) {
@@ -23417,6 +23494,7 @@ function efFlatten(data) {
else if (it && it.n !== undefined && it.name !== undefined) rows.push(['Service ' + it.n, String(it.name)]);
else if (it && it.mcc !== undefined) rows.push(['PLMN ' + (i + 1), it.mcc + '-' + it.mnc + (it.access_tech ? ' (' + it.access_tech + ')' : '')]);
else if (it && it.aid !== undefined) rows.push(['Application ' + (i + 1), (it.label ? it.label + ' \u2014 ' : '') + it.aid + (it.eap ? ' [' + (it.eap.eap_types || []).join(', ') + ']' : '')]);
else if (it && it.link !== undefined && it.tag !== undefined) rows.push([label + ' ' + (i + 1), 'Type ' + it.link + ' \u00b7 ' + (it.kind || ('tag ' + it.tag)) + (it.fid ? ' \u2014 ' + it.fid + (it.sfi ? ' (SFI ' + it.sfi + ')' : '') : (it.raw ? ' \u2014 ' + it.raw : ''))]);
else rows.push([label + ' ' + (i + 1), JSON.stringify(it)]);
});
};
+1 -1
View File
@@ -1,4 +1,4 @@
const CACHE = 'simple-v454';
const CACHE = 'simple-v455';
const URLS = [
'index.html',
'help.html',
+102 -2
View File
@@ -29,7 +29,7 @@ const FNS = [
'efDecPhase', 'efDecCbmi', 'efDecCbmir', 'efDecEcc', 'efDecOpl', 'efDecAd', 'efDecAcl',
'efDecSmsp', 'efDecSpdi', 'efDecNai', 'efDecDir', 'efDecArr', 'efDecPnn', 'efDecAdn',
'pinKeyRefName', 'efArrAmLabel', 'efArrSc', 'efArrAmDo',
'efDecExt1', 'efDecSms', 'efDecSume', 'efFindDecoder', 'efFidFromPath', 'efDecodeBytes',
'efDecExt1', 'efDecSms', 'efDecSume', 'efPbrHex', 'efDecPbr', 'efFindDecoder', 'efFidFromPath', 'efDecodeBytes',
'efDecodeFile', 'efDecodeHex', 'efIsRawOnly', 'efFieldLabel', 'efPrimitive', 'efFlatten',
'efDiffData', 'efDataSummary', 'efContentDiff', 'gsm7Decode', 'efRenderFieldsHtml',
'pysimFsDecodedHtml', 'profilerRenderReport', 'profilerLabelText', 'profilerResultAspects',
@@ -42,7 +42,7 @@ for (const f of FNS) code += extractFunc(html, f) + '\n';
for (const c of ['GSM7_ALPHABET', 'GSM7_EXT_MAP', 'FCP_ARR_AM_BITS']) {
code += html.match(new RegExp('const ' + c + ' = [\\s\\S]*?\\n\\];'))[0].replace('const ', 'var ') + '\n';
}
for (const c of ['EF_UST_SERVICES', 'EF_SST_SERVICES', 'EF_IST_SERVICES', 'EF_PHASE_NAMES']) {
for (const c of ['EF_UST_SERVICES', 'EF_SST_SERVICES', 'EF_IST_SERVICES', 'EF_PHASE_NAMES', 'EF_PBR_KINDS']) {
code += html.match(new RegExp('const ' + c + ' = \\{.*\\};'))[0].replace('const ', 'var ') + '\n';
}
code += html.match(/const EF_FIELD_LABELS = \{[\s\S]*?\n\};/)[0].replace('const ', 'var ') + '\n';
@@ -132,6 +132,106 @@ test('EF.SUME decodes the SET UP MENU title and icon TLVs (TS 51.011 10.3.34)',
{ title: 'Menu' });
});
test('EF.PBR decodes the phonebook structure containers (TS 31.102 4.4.2.1)', () => {
// pySim's EF_PBR fixture: A8 { C0 03 4F3A 02, C5 03 4F09 04 }
// + AA { CB 03 4F3D 07, C2 03 4F4A 06 } - all with an SFI.
const d = efDecPbr(efBytes('A80AC0034F3A02C5034F0904AA0ACB034F3D07C2034F4A06'));
assert.strictEqual(d.master, '4F3A');
assert.strictEqual(d.iap, null);
assert.strictEqual(d.note, undefined);
assert.deepStrictEqual(d.links, [
{ link: 1, tag: 'C0', kind: 'EF.ADN', fid: '4F3A', sfi: '02' },
{ link: 1, tag: 'C5', kind: 'EF.PBC', fid: '4F09', sfi: '04' },
{ link: 3, tag: 'CB', kind: 'EF.CCP1', fid: '4F3D', sfi: '07' },
{ link: 3, tag: 'C2', kind: 'EF.EXT1', fid: '4F4A', sfi: '06' },
]);
assert.deepStrictEqual(d.files, ['4F3A', '4F09', '4F3D', '4F4A']);
});
test('EF.PBR: the no-SFI form and the type-2 (A9) container with EF_IAP', () => {
// A8 { C0 02 4F3A (FID only, no SFI), C1 02 4F09 (EF_IAP) }
// + A9 { C6 03 4F0B 06, C7 03 4F0C 07 } - the type-2 files are addressed
// through the EF_IAP pointers, in the order of the FIDs after 'A9'.
const d = efDecPbr(efBytes('A808C0024F3AC1024F09A90AC6034F0B06C7034F0C07'));
assert.strictEqual(d.master, '4F3A');
assert.strictEqual(d.iap, '4F09');
assert.deepStrictEqual(d.links, [
{ link: 1, tag: 'C0', kind: 'EF.ADN', fid: '4F3A', sfi: null },
{ link: 1, tag: 'C1', kind: 'EF.IAP', fid: '4F09', sfi: null },
{ link: 2, tag: 'C6', kind: 'EF.GRP', fid: '4F0B', sfi: '06' },
{ link: 2, tag: 'C7', kind: 'EF.AAS', fid: '4F0C', sfi: '07' },
]);
assert.deepStrictEqual(d.files, ['4F3A', '4F09', '4F0B', '4F0C']);
assert.strictEqual(d.note, undefined);
});
test('EF.PBR: FF padding is not a bogus link; a truncated TLV is a note, not a throw', () => {
const padded = efDecPbr(efBytes('A80AC0034F3A02C5034F0904AA0ACB034F3D07C2034F4A06' + 'FF'.repeat(8)));
assert.strictEqual(padded.links.length, 4);
assert.strictEqual(padded.note, undefined);
assert.deepStrictEqual(padded.files, ['4F3A', '4F09', '4F3D', '4F4A']);
// a trailing 'C5' tag with no length byte: the parsed links are kept
const trunc = efDecPbr(efBytes('A80AC0034F3A02C5034F0904C5'));
assert.strictEqual(trunc.master, '4F3A');
assert.strictEqual(trunc.links.length, 2);
assert.match(trunc.note, /truncated TLV/);
// a container whose value runs past the record: the earlier links are kept
const bad = efDecPbr(efBytes('A80AC0034F3A02C5034F0904AA0ACB034F3D07'));
assert.strictEqual(bad.links.length, 2);
assert.match(bad.note, /truncated TLV/);
});
test('EF.PBR: unknown primitive tags are preserved raw with a note', () => {
// A8 { C0 03 4F3A 02 (master), CE 03 4F09 04 (not a phonebook kind) }
const d = efDecPbr(efBytes('A80AC0034F3A02CE034F0904'));
assert.strictEqual(d.master, '4F3A');
assert.deepStrictEqual(d.links, [
{ link: 1, tag: 'C0', kind: 'EF.ADN', fid: '4F3A', sfi: '02' },
{ link: 1, tag: 'CE', kind: null, fid: '4F09', sfi: '04', raw: '4F0904' },
]);
assert.deepStrictEqual(d.files, ['4F3A']);
assert.match(d.note, /CE/);
});
test('EF.PBR: the AA non-primitive fallback keeps the raw bytes and notes it', () => {
const d = efDecPbr(efBytes('AA03C0FF01'));
assert.deepStrictEqual(d.links, [
{ link: 3, tag: 'AA', kind: null, fid: null, sfi: null, raw: 'C0FF01' },
]);
assert.match(d.note, /not a primitive list/);
// an all-FF record is the unused marker
assert.deepStrictEqual(efDecPbr(efBytes('FFFFFFFF')), { empty: true });
});
test('EF.PBR matches the shared decode vectors (drift guard with the server parser)', () => {
// frontend/tests/pbr_vectors.json is asserted by tests/test_pbr.py too:
// the PWA decoder and the server-side PBR parser must agree.
const vectors = JSON.parse(fs.readFileSync(path.join(__dirname, 'pbr_vectors.json'), 'utf8'));
assert.ok(vectors.cases.length >= 5);
for (const c of vectors.cases) {
const d = efDecPbr(efBytes(c.records[0]));
assert.strictEqual(d.master, c.master, c.name + ': master');
assert.deepStrictEqual(d.links, c.links, c.name + ': links');
assert.deepStrictEqual(d.files, c.files, c.name + ': files');
}
});
test('EF.PBR is registered by name and FID; efFlatten renders link rows', () => {
assert.strictEqual(efFindDecoder('EF.PBR', null).fid, '4f30');
assert.strictEqual(efFindDecoder(null, '4f30').name, 'EF.PBR');
assert.strictEqual(efFindDecoder('MF/7F10/5F3A/EF.PBR', null).fn, efDecPbr);
assert.deepStrictEqual(efFlatten({ master: '4F3A', iap: '4F09', links: [
{ link: 1, tag: 'C0', kind: 'EF.ADN', fid: '4F3A', sfi: '02' },
{ link: 3, tag: 'AA', kind: null, fid: null, sfi: null, raw: 'C0FF01' },
], files: ['4F3A'] }), [
['Master EF', '4F3A'],
['IAP (EF.IAP)', '4F09'],
['Links 1', 'Type 1 \u00b7 EF.ADN \u2014 4F3A (SFI 02)'],
['Links 2', 'Type 3 \u00b7 tag AA \u2014 C0FF01'],
['Files', '4F3A'],
]);
});
test('ADN-format records: alpha + BCD number + CCP/EXT1', () => {
const d = efDecAdn(efBytes('42204841203120536963FFFFFFFFFFFF06810628560810FFFFFFFFFFFFFF'));
assert.strictEqual(d.name, 'B HA 1 Sic');
+303
View File
@@ -0,0 +1,303 @@
{
"_comment": "EF.PBR (TS 31.102 4.4.2.1) decode vectors, shared by the PWA decoder (frontend/tests/ef_decode.test.js, efDecPbr) and the server parser (tests/test_pbr.py, _pbr_parse) - the drift guard for the two implementations. 'records' are raw EF_PBR record hex strings (as /api/pbr serves them); 'master' is the first FID of the first 'A8' container (the PWA decoder only - the server parser does not need it); 'links' is the PWA decoder's link list (tag/kind/fid/sfi, plus 'raw' for a preserved unknown tag); 'files' is the referenced FID list in record order, deduplicated by FID - the shape the server parser returns.",
"cases": [
{
"name": "pySim's EF_PBR fixture: type 1 and type 3 containers, every primitive with an SFI",
"records": [
"A80AC0034F3A02C5034F0904AA0ACB034F3D07C2034F4A06"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": "02"
},
{
"link": 1,
"tag": "C5",
"kind": "EF.PBC",
"fid": "4F09",
"sfi": "04"
},
{
"link": 3,
"tag": "CB",
"kind": "EF.CCP1",
"fid": "4F3D",
"sfi": "07"
},
{
"link": 3,
"tag": "C2",
"kind": "EF.EXT1",
"fid": "4F4A",
"sfi": "06"
}
],
"files": [
"4F3A",
"4F09",
"4F3D",
"4F4A"
]
},
{
"name": "type 2 ('A9') addressed through EF_IAP (a 'C1' inside the 'A8'); FID-only primitives (no SFI)",
"records": [
"A808C0024F3AC1024F09A90AC6034F0B06C7034F0C07"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": null
},
{
"link": 1,
"tag": "C1",
"kind": "EF.IAP",
"fid": "4F09",
"sfi": null
},
{
"link": 2,
"tag": "C6",
"kind": "EF.GRP",
"fid": "4F0B",
"sfi": "06"
},
{
"link": 2,
"tag": "C7",
"kind": "EF.AAS",
"fid": "4F0C",
"sfi": "07"
}
],
"files": [
"4F3A",
"4F09",
"4F0B",
"4F0C"
]
},
{
"name": "unused trailing bytes are FF: the padding is not a link",
"records": [
"A80AC0034F3A02C5034F0904AA0ACB034F3D07C2034F4A06FFFFFFFFFFFFFFFF"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": "02"
},
{
"link": 1,
"tag": "C5",
"kind": "EF.PBC",
"fid": "4F09",
"sfi": "04"
},
{
"link": 3,
"tag": "CB",
"kind": "EF.CCP1",
"fid": "4F3D",
"sfi": "07"
},
{
"link": 3,
"tag": "C2",
"kind": "EF.EXT1",
"fid": "4F4A",
"sfi": "06"
}
],
"files": [
"4F3A",
"4F09",
"4F3D",
"4F4A"
]
},
{
"name": "a truncated trailing TLV (a lone 'C5' tag): the links before it are kept",
"records": [
"A80AC0034F3A02C5034F0904C5"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": "02"
},
{
"link": 1,
"tag": "C5",
"kind": "EF.PBC",
"fid": "4F09",
"sfi": "04"
}
],
"files": [
"4F3A",
"4F09"
]
},
{
"name": "an unknown primitive tag ('CE') is preserved raw and is not a phonebook file",
"records": [
"A80AC0034F3A02CE034F0904"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": "02"
},
{
"link": 1,
"tag": "CE",
"kind": null,
"fid": "4F09",
"sfi": "04",
"raw": "4F0904"
}
],
"files": [
"4F3A"
]
},
{
"name": "TS 31.102 Annex G Table G.2 Rec 1 (every kind tag; the printed A8 length is internally inconsistent - the table lists 10 primitives of 5 bytes = 50 = 0x32 - so the corrected length is used)",
"records": [
"A832C0034F3A01C5034F0902C6034F2603C4034F1104C4034F1305C4034F1506C3034F1907C9034F2112CA034F5009CC034F5416AA0FC7034F4B14C8034F4C15C2034F4A08"
],
"master": "4F3A",
"links": [
{
"link": 1,
"tag": "C0",
"kind": "EF.ADN",
"fid": "4F3A",
"sfi": "01"
},
{
"link": 1,
"tag": "C5",
"kind": "EF.PBC",
"fid": "4F09",
"sfi": "02"
},
{
"link": 1,
"tag": "C6",
"kind": "EF.GRP",
"fid": "4F26",
"sfi": "03"
},
{
"link": 1,
"tag": "C4",
"kind": "EF.ANR",
"fid": "4F11",
"sfi": "04"
},
{
"link": 1,
"tag": "C4",
"kind": "EF.ANR",
"fid": "4F13",
"sfi": "05"
},
{
"link": 1,
"tag": "C4",
"kind": "EF.ANR",
"fid": "4F15",
"sfi": "06"
},
{
"link": 1,
"tag": "C3",
"kind": "EF.SNE",
"fid": "4F19",
"sfi": "07"
},
{
"link": 1,
"tag": "C9",
"kind": "EF.UID",
"fid": "4F21",
"sfi": "12"
},
{
"link": 1,
"tag": "CA",
"kind": "EF.EMAIL",
"fid": "4F50",
"sfi": "09"
},
{
"link": 1,
"tag": "CC",
"kind": "EF.PURI",
"fid": "4F54",
"sfi": "16"
},
{
"link": 3,
"tag": "C7",
"kind": "EF.AAS",
"fid": "4F4B",
"sfi": "14"
},
{
"link": 3,
"tag": "C8",
"kind": "EF.GAS",
"fid": "4F4C",
"sfi": "15"
},
{
"link": 3,
"tag": "C2",
"kind": "EF.EXT1",
"fid": "4F4A",
"sfi": "08"
}
],
"files": [
"4F3A",
"4F09",
"4F26",
"4F11",
"4F13",
"4F15",
"4F19",
"4F21",
"4F50",
"4F54",
"4F4B",
"4F4C",
"4F4A"
]
}
]
}
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "pysim-simple-server"
version = "3.30.4"
version = "3.31.0"
description = "HTTP REST server wrapping pysim for the SIMple PWA"
requires-python = ">=3.8"
# pysim is a git-only dependency installed explicitly by setup.bat/setup.sh.
+11 -1
View File
@@ -18,7 +18,7 @@ from . import presets
from . import test_scripts
from . import test_suites
from . import testscript
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _read_arr, _netstate_read, _netstate_install, _LineFilter, _test_command_type, migrate_scripts_to_suites
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _read_arr, _read_pbr, _netstate_read, _netstate_install, _LineFilter, _test_command_type, migrate_scripts_to_suites
_server_start = 0
@@ -253,6 +253,7 @@ def main():
iccid = None
netstate_files = None
arr_files = None
pbr_files = None
if scc and card is not None and hasattr(scc, '_tp'):
scc._tp.apdu_tracer = _LoggingApduTracer()
try:
@@ -275,6 +276,12 @@ def main():
if arr_files:
sys.stderr.write('INIT: EF.ARR %s\n' % ', '.join(
'%s (%d rec)' % (a['fid'], a['num_records']) for a in arr_files))
# EF.PBR (the phonebook reference record): same reason - the
# startup init never runs _apply_equipped_card.
pbr_files = _read_pbr(app)
if pbr_files:
sys.stderr.write('INIT: EF.PBR %s\n' % ', '.join(
'%s (%d rec)' % (p['fid'], p['num_records']) for p in pbr_files))
t_phase = time.time()
sys.stderr.write('INIT: sending TERMINAL PROFILE %s (CLA=%s)\n' % (opts.terminal_profile, scc.cat_cla))
sm, el = _send_terminal_profile(scc, opts.terminal_profile)
@@ -364,6 +371,9 @@ def main():
# EF.ARR cache (v3.24.0): read by the startup init above (a card present
# at server start); a later equip refreshes it in _apply_equipped_card.
server.arr = arr_files
# EF.PBR cache (TS 31.102 4.4.2.1): read by the startup init above; a
# later equip refreshes it in _apply_equipped_card.
server.pbr = pbr_files
# Network state monitor: install the state read during the startup init
# (right after the ICCID, before the TERMINAL PROFILE). No readable
# ICCID means the card is considered unusable - give up.
+264 -2
View File
@@ -37,7 +37,7 @@ from osmocom.tlv import BER_TLV_IE
from osmocom.utils import rpad
VERSION = '3.30.4'
VERSION = '3.31.0'
MAX_ENVELOPE_SEGMENTS = 5 # max SMS segments for outgoing C-APDU in ENVELOPE
@@ -555,6 +555,226 @@ def _arr_refresh_after_write(server, app, fid):
return True
# EF.PBR (TS 31.102 4.4.2.1): the phonebook reference record - the BER-TLV
# structure that describes the phonebook files (the 'A8'/'A9'/'AA' containers
# and the primitive phonebook kinds). It is read once per card session like
# EF.ARR; the records are served raw because the PWA owns the single decoder
# (one record covers at most 254 entries - a larger phonebook uses further
# records, hence the cap).
PBR_READ_PATHS = ('MF/7F10/5F3A/4F30', 'ADF.USIM/5F3A/4F30',
'ADF.USIM-NON-IMSI/5F3A/4F30')
PBR_RECORD_LIMIT = 16
def _read_pbr(app):
"""Best-effort EF.PBR reads (PBR_READ_PATHS): the phonebook structure
records. Returns a list of
{'fid','path','record_len','num_records','records'} dicts (records as hex,
1-based) or None; never raises."""
if not app or not getattr(app, 'rs', None):
return None
lchan = app.rs.lchan[0]
out = []
for path in PBR_READ_PATHS:
cleanup = None
try:
_, cleanup = _select_path(lchan, path, app)
if _get_file_type(lchan, lchan.selected_file) not in ('linear_fixed', 'cyclic'):
continue
n = min(lchan.selected_file_num_of_rec() or 1, PBR_RECORD_LIMIT)
records = []
for i in range(1, n + 1):
rv = lchan.read_record(i)
data = rv[0] if isinstance(rv, tuple) else rv
records.append((data or '').upper())
out.append({'fid': path.rsplit('/', 1)[-1].upper(),
'path': path,
'record_len': _fcp_value(lchan, 'selected_file_record_len'),
'num_records': len(records),
'records': records})
except Exception:
continue
finally:
if cleanup:
try:
cleanup()
except Exception:
pass
return out or None
def _refresh_pbr_cache(server):
"""Read the EF.PBR records into the session cache (best effort); a failed
read clears the previous session's cache."""
server.pbr = _read_pbr(getattr(server, 'app', None))
if server.pbr:
_tlog('equip: EF.PBR ' + ', '.join(
'%s (%d rec)' % (p['fid'], p['num_records']) for p in server.pbr))
# The EF.PBR primitive tags and the constructed containers (TS 31.102 4.4.2.1);
# the JS decoder (efDecPbr in the PWA) carries the same tables - the shared
# fixture frontend/tests/pbr_vectors.json guards the two against drift.
PBR_KINDS = {
'C0': 'EF.ADN', 'C1': 'EF.IAP', 'C2': 'EF.EXT1', 'C3': 'EF.SNE',
'C4': 'EF.ANR', 'C5': 'EF.PBC', 'C6': 'EF.GRP', 'C7': 'EF.AAS',
'C8': 'EF.GAS', 'C9': 'EF.UID', 'CA': 'EF.EMAIL', 'CB': 'EF.CCP1',
'CC': 'EF.PURI',
}
PBR_CONTAINERS = {'A8': 1, 'A9': 2, 'AA': 3}
def _pbr_tlvs(data):
"""BER-TLV walk of a hex string: [(tag, value_hex)]. Stops cleanly at a
truncated TLV or at the trailing FF padding - never raises."""
out = []
i = 0
n = len(data)
while i + 4 <= n:
tag = data[i:i + 2]
length = int(data[i + 2:i + 4], 16)
i += 4
if length & 0x80:
k = length & 0x7F
if k == 0 or i + 2 * k > n:
break
length = int(data[i:i + 2 * k], 16)
i += 2 * k
if i + 2 * length > n:
break
out.append((tag, data[i:i + 2 * length]))
i += 2 * length
return out
def _pbr_parse(records):
"""Parse EF.PBR records (TS 31.102 4.4.2.1) into the referenced phonebook
files: a list of {'fid','kind','link','sfi'} in record order, deduplicated
by FID (first occurrence wins). The containers 'A8'/'A9'/'AA' are link
types 1/2/3; a primitive value of 2 bytes is the FID, 3 bytes FID + SFI.
Pure and tolerant (FF padding / a truncated TLV are skipped); never
raises, [] when nothing parses."""
files = []
seen = set()
for rec in records or []:
data = re.sub(r'[^0-9A-Fa-f]', '', rec or '').upper()
while len(data) >= 2 and data.endswith('FF'):
data = data[:-2]
for tag, value in _pbr_tlvs(data):
link = PBR_CONTAINERS.get(tag)
if link is None:
continue
for ptag, pval in _pbr_tlvs(value):
kind = PBR_KINDS.get(ptag)
if not kind:
continue
if len(pval) == 4:
fid, sfi = pval, None
elif len(pval) == 6:
fid, sfi = pval[:4], pval[4:]
else:
continue
if fid in seen:
continue
seen.add(fid)
files.append({'fid': fid, 'kind': kind, 'link': link, 'sfi': sfi})
return files
def _pbr_norm_path(segments, adf_aids=None):
"""Normalise a tree path (uppercase segments, 3F00 -> MF) for the PBR
path comparison: the cache paths look like 'MF/7F10/5F3A/4F30' and
'ADF.USIM/5F3A/4F30'.
An ADF is rooted at itself, but a tree request carries the whole chain
(MF/ADF.USIM/5F3A), so a leading 'MF' before an ADF segment is dropped -
otherwise the ADF's DF_PHONEBOOK could never match its cache entry (the
two-PBR card case). The file manager addresses an ADF by name and the
Profiler's tree walk by AID, so `adf_aids` ({AID: name}) folds the AID
form onto the name form; the cache paths are name-based."""
out = []
for seg in segments or []:
s = str(seg or '').strip().upper()
if not s:
continue
if adf_aids and s in adf_aids:
s = adf_aids[s]
out.append('MF' if s in ('MF', '3F00') else s)
if len(out) > 1 and out[0] == 'MF' and out[1].startswith('ADF.'):
out.pop(0)
return out
def _pbr_files_for_node(server, path):
"""The parsed PBR files when `path` (normalised) is the DF holding a cached
EF.PBR, else None. A full-path match wins; with several cached PBRs under
the same DF FID the path disambiguates, and an ambiguous path is accepted
only when a single cached entry exists (the three PBR copies all sit in a
DF with FID 5F3A)."""
pbrs = getattr(server, 'pbr', None)
if not pbrs or not path:
return None
wanted = '/'.join(path)
df_fid = path[-1]
exact, by_fid = [], []
for p in pbrs:
if not isinstance(p, dict):
continue
ppath = _pbr_norm_path((p.get('path') or '').split('/'))
if len(ppath) < 2:
continue
if '/'.join(ppath[:-1]) == wanted:
exact.append(p)
elif ppath[-2] == df_fid:
by_fid.append(p)
if exact:
return _pbr_parse(exact[0].get('records'))
# The FID-only fallback is for a bare DF request (the caller could not name
# the DF): a full path that failed the exact match must never be answered by
# another DF's copy (e.g. the ADF's PBR for a MF/7F10/5F3A request).
if len(path) == 1 and len(by_fid) == 1:
return _pbr_parse(by_fid[0].get('records'))
return None
def _pbr_known_fid(server, seg):
"""True when `seg` is a 4-hex FID the cached EF.PBR references. These
files are not in pySim's model, so /api/tree, /api/select, /api/read and
/api/write auto-allow the probe for exactly them - that is what makes the
server-injected tree nodes clickable without any PWA flag (the PWA only
sets allow_probe for its own custom nodes)."""
fid = str(seg or '').strip().upper()
if not _fid4(fid):
return False
for p in getattr(server, 'pbr', None) or []:
if not isinstance(p, dict):
continue
for f in _pbr_parse(p.get('records')):
if f['fid'] == fid:
return True
return False
def _pbr_inject_children(server, path, children):
"""Append the EF.PBR-referenced phonebook files to a /api/tree children
list when `path` (normalised) is the DF that holds a cached EF.PBR. A
file pySim's model already lists is not added twice. Returns the number
of appended children."""
pbr_files = _pbr_files_for_node(server, path)
if not pbr_files:
return 0
known = {str(c.get('fid') or '').upper() for c in children if c.get('fid')}
added = 0
for pf in pbr_files:
if pf['fid'] in known:
continue
known.add(pf['fid'])
children.append({'name': pf['kind'], 'fid': pf['fid'], 'isDir': False,
'pbr': True, 'sfi': pf['sfi']})
added += 1
return added
_MCC_MNC_CACHE = {'path': None, 'data': None}
@@ -5169,6 +5389,7 @@ def _handle_card_disconnect(stale=False):
_server_ref.sim_menu = None
_server_ref.iccid = None
_server_ref.arr = None
_server_ref.pbr = None
_server_ref.net_state = None
_server_ref.equipping = False
_server_ref.card_session = getattr(_server_ref, 'card_session', 0) + 1
@@ -5244,6 +5465,7 @@ def _apply_equipped_card(server):
server.card_session = getattr(server, 'card_session', 0) + 1
server.iccid = None
server.arr = None
server.pbr = None
# Read the ICCID before the TERMINAL PROFILE starts a CAT session: the
# PWA auto-selects the matching card preset (SCP80 views) from it.
server.iccid = _read_iccid(server.app)
@@ -5256,6 +5478,9 @@ def _apply_equipped_card(server):
# EF.ARR (FCP '8B' references): read once per card session - the
# file manager resolves the referenced record's rules from it.
_refresh_arr_cache(server)
# EF.PBR (the phonebook reference record): read once per card session
# too; the PWA decodes the raw records.
_refresh_pbr_cache(server)
else:
server.net_state = None
_tlog('equip: ICCID not readable - network state skipped')
@@ -6802,7 +7027,7 @@ _CARD_FREE_GET = frozenset([
'/api/stk-status', '/api/poll-status', '/api/proactive-log',
'/api/menu', '/api/events', '/api/terminal-profile',
'/api/pli-qualifiers', '/api/pli-dict', '/api/commands',
'/api/net-state', '/api/arr', '/api/mcc-mnc', '/api/presets', '/api/test/scripts',
'/api/net-state', '/api/arr', '/api/pbr', '/api/mcc-mnc', '/api/presets', '/api/test/scripts',
'/api/test/suites',
'/api/scp81/status', '/api/bip/status', '/api/scp81/script',
'/api/scp81/log', '/api/bip/log',
@@ -8888,6 +9113,17 @@ class PysimHandler(BaseHTTPRequestHandler):
'session': getattr(self.server, 'card_session', 0),
'arrs': arrs or []})
self._log_resp({'ok': bool(arrs), 'arrs': len(arrs or [])})
elif self.path == '/api/pbr':
# EF.PBR session cache (TS 31.102 4.4.2.1): read once per card
# session in _refresh_pbr_cache; the PWA decodes the raw records
# (the server does not parse the PBR). Cache-only - served
# without the card lock (see _CARD_FREE_GET).
self._log_req()
pbrs = getattr(self.server, 'pbr', None)
self._send_json({'ok': bool(pbrs),
'session': getattr(self.server, 'card_session', 0),
'pbrs': pbrs or []})
self._log_resp({'ok': bool(pbrs), 'pbrs': len(pbrs or [])})
elif self.path == '/api/status':
self._log_req()
app = self.server.app
@@ -9478,6 +9714,9 @@ class PysimHandler(BaseHTTPRequestHandler):
parent_sel = body.get('parent_sel')
parent_path = body.get('parent_path')
allow_probe = bool(body.get('allow_probe'))
# PBR-referenced files are not in pySim's model (see _pbr_known_fid)
if not allow_probe:
allow_probe = _pbr_known_fid(self.server, fid if fid else name)
rs = app.rs
if not rs:
self._send_json({'error': _err('no_card_state', lang)}, 503)
@@ -9530,6 +9769,9 @@ class PysimHandler(BaseHTTPRequestHandler):
parent_path = body.get('parent_path')
allow_probe = bool(body.get('allow_probe'))
mode = body.get('mode', 'raw')
# PBR-referenced files are not in pySim's model (see _pbr_known_fid)
if not allow_probe:
allow_probe = _pbr_known_fid(self.server, fid if fid else name)
rs = app.rs
if not rs:
self._send_json({'error': _err('no_card_state', lang)}, 503)
@@ -9617,6 +9859,9 @@ class PysimHandler(BaseHTTPRequestHandler):
parent_path = body.get('parent_path')
allow_probe = bool(body.get('allow_probe'))
path = body.get('path')
# PBR-referenced files are not in pySim's model (see _pbr_known_fid)
if not allow_probe:
allow_probe = _pbr_known_fid(self.server, fid if fid else name)
rs = app.rs
if not rs:
self._send_json({'error': _err('no_card_state', lang)}, 503)
@@ -9687,6 +9932,9 @@ class PysimHandler(BaseHTTPRequestHandler):
parent_sel = body.get('parent_sel')
parent_path = body.get('parent_path')
allow_probe = bool(body.get('allow_probe'))
# PBR-referenced files are not in pySim's model (see _pbr_known_fid)
if not allow_probe:
allow_probe = _pbr_known_fid(self.server, fid if fid else name)
rs = app.rs
if not rs:
self._send_json({'error': _err('no_card_state', lang)}, 503)
@@ -9710,6 +9958,20 @@ class PysimHandler(BaseHTTPRequestHandler):
app.stdout = old_stdout
sys.stderr = old_stderr
children = _parse_tree_output(output)
# EF.PBR (TS 31.102 4.4.2.1): the phonebook files the DF's PBR
# references are not in pySim's model, so they are appended to
# the tree here - the PWA renders them like any other node (no
# frontend change) and the select/read/write handlers
# auto-allow the probe for exactly these FIDs (_pbr_known_fid).
adf_aids = {}
try:
for aid, adf in (rs.mf.applications or {}).items():
if getattr(adf, 'name', None):
adf_aids[str(aid).upper()] = adf.name.upper()
except Exception:
adf_aids = {}
node_path = _pbr_norm_path(list(parent_path or []) + [fid if fid else name], adf_aids)
_pbr_inject_children(self.server, node_path, children)
sels = lchan.selected_file.get_selectables() if lchan and lchan.selected_file else {}
for child in children:
if child['isDir'] and child['name'] in sels:
+415
View File
@@ -0,0 +1,415 @@
# coding=utf-8
"""EF.PBR session cache (TS 31.102 4.4.2.1): the equip-time read of the
phonebook reference records, the /api/pbr route and the cache lifecycle."""
import json
import pathlib
import threading
import unittest
import urllib.request
from unittest import mock
from pysim_simple_server.__main__ import _build_http_server
from pysim_simple_server.server import (PBR_READ_PATHS, PBR_RECORD_LIMIT,
PysimHandler, _CARD_FREE_GET,
_pbr_files_for_node, _pbr_inject_children,
_pbr_known_fid, _pbr_norm_path,
_pbr_parse, _read_pbr,
_refresh_pbr_cache)
ROOT = pathlib.Path(__file__).resolve().parents[1]
SRC = (ROOT / 'pysim_simple_server' / 'server.py').read_text(encoding='utf-8')
# The PBR_READ_PATHS the frontend's uicc_files.json lists for EF.PBR (4F30).
FRONTEND_PATHS = (ROOT / 'frontend' / 'uicc_files.json').read_text(encoding='utf-8')
# The shared decode vectors: the PWA decoder (ef_decode.test.js) asserts the
# same file, so the two PBR parsers cannot drift apart.
PBR_VECTORS = json.loads((ROOT / 'frontend' / 'tests' / 'pbr_vectors.json').read_text(encoding='utf-8'))
FIXTURE = 'A80AC0034F3A02C5034F0904AA0ACB034F3D07C2034F4A06'
class FakeFile:
def __init__(self, fid):
self.fid = fid
class FakeLchan:
"""One selected PBR file: read_record serves its records."""
def __init__(self, files):
self.files = files # path -> [record hex]
self._cur = None
self.selected_file = None
def selected_file_num_of_rec(self):
return len(self.files.get(self._cur) or [])
def selected_file_record_len(self):
return 32
def read_record(self, n):
return (self.files[self._cur][n - 1], '9000')
def _select(lchan, path, app):
if path not in lchan.files:
raise RuntimeError('file not found')
lchan._cur = path
lchan.selected_file = FakeFile(path.rsplit('/', 1)[-1])
return lchan.selected_file, None
def _app(files):
lchan = FakeLchan(files)
app = mock.Mock()
app.rs.lchan = [lchan]
return app, lchan
def _read(app, file_type='linear_fixed'):
with mock.patch('pysim_simple_server.server._select_path', _select), \
mock.patch('pysim_simple_server.server._get_file_type', return_value=file_type):
return _read_pbr(app)
class ReadPbrTests(unittest.TestCase):
def test_reads_all_paths(self):
app, _ = _app({'MF/7F10/5F3A/4F30': ['A80AC0034F3A02C5034F0904'],
'ADF.USIM/5F3A/4F30': ['A80AC0034F3A02'],
'ADF.USIM-NON-IMSI/5F3A/4F30': ['A80AC0034F3A02']})
pbrs = _read(app)
self.assertEqual([p['fid'] for p in pbrs], ['4F30', '4F30', '4F30'])
self.assertEqual([p['path'] for p in pbrs], list(PBR_READ_PATHS))
self.assertEqual(pbrs[0]['records'], ['A80AC0034F3A02C5034F0904'])
self.assertEqual(pbrs[0]['num_records'], 1)
self.assertEqual(pbrs[0]['record_len'], 32)
def test_a_missing_copy_is_skipped(self):
app, _ = _app({'ADF.USIM/5F3A/4F30': ['A80AC0034F3A02']})
pbrs = _read(app)
self.assertEqual([p['path'] for p in pbrs], ['ADF.USIM/5F3A/4F30'])
def test_no_pbr_returns_none(self):
app, _ = _app({})
self.assertIsNone(_read(app))
def test_a_transparent_file_is_skipped(self):
app, _ = _app({'MF/7F10/5F3A/4F30': ['A80AC0034F3A02']})
self.assertIsNone(_read(app, file_type='transparent'))
def test_a_failing_read_skips_the_file(self):
app, lchan = _app({'MF/7F10/5F3A/4F30': ['A80AC0034F3A02'],
'ADF.USIM/5F3A/4F30': ['A80AC0034F3A02']})
orig = lchan.read_record
def flaky(n):
if lchan._cur == 'MF/7F10/5F3A/4F30':
raise RuntimeError('6A82')
return orig(n)
lchan.read_record = flaky
pbrs = _read(app)
self.assertEqual([p['path'] for p in pbrs], ['ADF.USIM/5F3A/4F30'])
def test_the_record_count_is_capped(self):
app, _ = _app({'MF/7F10/5F3A/4F30': ['A80AC0034F3A02'] * 100})
pbrs = _read(app)
self.assertEqual(pbrs[0]['num_records'], PBR_RECORD_LIMIT)
self.assertEqual(PBR_RECORD_LIMIT, 16)
def test_records_are_uppercased_hex(self):
app, _ = _app({'MF/7F10/5F3A/4F30': ['a80ac0034f3a02']})
pbrs = _read(app)
self.assertEqual(pbrs[0]['records'], ['A80AC0034F3A02'])
def test_no_app_returns_none(self):
self.assertIsNone(_read_pbr(None))
self.assertIsNone(_read_pbr(mock.Mock(rs=None)))
def test_the_paths_match_the_frontend_file_list(self):
# the read paths must be the paths uicc_files.json ships for EF.PBR
for path in PBR_READ_PATHS:
self.assertIn('"%s"' % path, FRONTEND_PATHS, path)
class CacheTests(unittest.TestCase):
def test_refresh_stores_and_clears(self):
s = mock.Mock()
with mock.patch('pysim_simple_server.server._read_pbr',
return_value=[{'fid': '4F30', 'num_records': 1}]), \
mock.patch('pysim_simple_server.server._tlog'):
_refresh_pbr_cache(s)
self.assertEqual(s.pbr, [{'fid': '4F30', 'num_records': 1}])
with mock.patch('pysim_simple_server.server._read_pbr', return_value=None):
_refresh_pbr_cache(s)
self.assertIsNone(s.pbr)
def test_equip_and_disconnect_wiring(self):
# the equip hook reads the cache; a disconnect clears it
seg = SRC[SRC.index('def _apply_equipped_card'):SRC.index('def _card_reset_reinit')]
self.assertIn('_refresh_pbr_cache(server)', seg)
self.assertIn('server.pbr = None', seg)
seg = SRC[SRC.index('def _handle_card_disconnect'):SRC.index('def _ensure_transport')]
self.assertIn('_server_ref.pbr = None', seg)
def test_the_startup_init_reads_the_pbr(self):
# a card present at server start never runs _apply_equipped_card, so
# the startup init reads the PBR in its own CAT-free window
src = (ROOT / 'pysim_simple_server' / '__main__.py').read_text(encoding='utf-8')
self.assertIn('pbr_files = _read_pbr(app)', src)
self.assertIn('server.pbr = pbr_files', src)
imp = src[src.index('from .server import'):]
self.assertIn('_read_pbr', imp[:imp.index('\n')])
class PbrParserTests(unittest.TestCase):
"""The pure server-side PBR parser (mirrors the PWA decoder)."""
def test_the_shared_vectors(self):
# every case in pbr_vectors.json: the parser must agree with the PWA
# decoder's links (fid/kind/link/sfi - a raw-preserved unknown tag is
# not a phonebook file) and with the FID list.
self.assertTrue(PBR_VECTORS['cases'])
for c in PBR_VECTORS['cases']:
parsed = _pbr_parse(c['records'])
expected = []
for l in c['links']:
if not l.get('kind'):
continue # a raw-preserved unknown tag is not a file
if any(e['fid'] == l['fid'] for e in expected):
continue # the parser deduplicates by FID (first wins)
expected.append({'fid': l['fid'], 'kind': l['kind'], 'link': l['link'],
'sfi': l['sfi']})
self.assertEqual(parsed, expected, c['name'])
self.assertEqual([f['fid'] for f in parsed], c['files'], c['name'])
def test_the_pysim_fixture(self):
self.assertEqual(_pbr_parse([FIXTURE]), [
{'fid': '4F3A', 'kind': 'EF.ADN', 'link': 1, 'sfi': '02'},
{'fid': '4F09', 'kind': 'EF.PBC', 'link': 1, 'sfi': '04'},
{'fid': '4F3D', 'kind': 'EF.CCP1', 'link': 3, 'sfi': '07'},
{'fid': '4F4A', 'kind': 'EF.EXT1', 'link': 3, 'sfi': '06'},
])
def test_the_no_sfi_form_and_the_type_2_container(self):
parsed = _pbr_parse(['A808C0024F3AC1024F09A90AC6034F0B06C7034F0C07'])
self.assertEqual([f['sfi'] for f in parsed], [None, None, '06', '07'])
self.assertEqual([f['link'] for f in parsed], [1, 1, 2, 2])
self.assertEqual(parsed[1]['kind'], 'EF.IAP')
def test_ff_padding_is_ignored(self):
self.assertEqual(_pbr_parse([FIXTURE + 'FF' * 8]), _pbr_parse([FIXTURE]))
def test_a_truncated_tlv_keeps_the_earlier_links(self):
self.assertEqual([f['fid'] for f in _pbr_parse(['A80AC0034F3A02C5034F0904C5'])],
['4F3A', '4F09'])
# a container whose value runs past the record parses to nothing
self.assertEqual(_pbr_parse(['A80AC0034F3A02C5034F09']), [])
def test_an_unknown_primitive_tag_is_skipped(self):
self.assertEqual([f['fid'] for f in _pbr_parse(['A80AC0034F3A02CE034F0904'])],
['4F3A'])
def test_a_bad_primitive_length_is_skipped(self):
# 'CE 05' with five bytes: not a FID (2) and not FID + SFI (3)
self.assertEqual([f['fid'] for f in _pbr_parse(['A80CC0034F3A02CE05AABBCCDDEE'])],
['4F3A'])
def test_dedupe_first_occurrence_wins(self):
# a phonebook over 254 entries uses further records; a FID repeated in
# a later record must not appear twice
parsed = _pbr_parse([FIXTURE, 'A80AC0034F3A02'])
self.assertEqual([f['fid'] for f in parsed], ['4F3A', '4F09', '4F3D', '4F4A'])
self.assertEqual(len(parsed), 4)
def test_tolerances(self):
self.assertEqual(_pbr_parse(None), [])
self.assertEqual(_pbr_parse([]), [])
self.assertEqual(_pbr_parse(['']), [])
self.assertEqual(_pbr_parse(['FFFFFFFF']), [])
self.assertEqual(_pbr_parse(['zz']), [])
self.assertEqual(_pbr_parse(['a808c0024f3ac1024f09'])[0]['fid'], '4F3A')
self.assertEqual(_pbr_parse(['A808C0024F3AC1024F09', '4F'])[0]['fid'], '4F3A')
PBR_ENTRY = {'fid': '4F30', 'path': 'MF/7F10/5F3A/4F30',
'record_len': 32, 'num_records': 1, 'records': [FIXTURE]}
def _server(pbrs):
s = mock.Mock()
s.pbr = pbrs
return s
class PbrTreeTests(unittest.TestCase):
"""The /api/tree injection: the DF holding a cached EF.PBR gains the
referenced phonebook files as children."""
def test_norm_path(self):
self.assertEqual(_pbr_norm_path(['3f00', '7f10', '5f3a', '4f30']),
['MF', '7F10', '5F3A', '4F30'])
self.assertEqual(_pbr_norm_path(['mf', '']), ['MF'])
self.assertEqual(_pbr_norm_path(None), [])
self.assertEqual(_pbr_norm_path(['ADF.USIM', '5F3A']), ['ADF.USIM', '5F3A'])
def test_the_df_with_a_pbr_gets_the_children(self):
s = _server([PBR_ENTRY])
children = [{'name': 'EF.PBR', 'fid': '4f30', 'isDir': False}]
self.assertEqual(_pbr_inject_children(s, ['MF', '7F10', '5F3A'], children), 4)
self.assertEqual([c['fid'] for c in children],
['4f30', '4F3A', '4F09', '4F3D', '4F4A'])
self.assertEqual(children[1], {'name': 'EF.ADN', 'fid': '4F3A',
'isDir': False, 'pbr': True, 'sfi': '02'})
def test_a_full_df_path_is_never_answered_by_another_dfs_copy(self):
# Only the ADF copy is cached: the MF/DF.TELECOM phonebook DF (a
# different DF that happens to share the FID) must NOT be answered by
# it - the FID-only fallback is for a bare DF request only.
s = _server([dict(PBR_ENTRY, path='ADF.USIM/5F3A/4F30')])
self.assertIsNone(_pbr_files_for_node(s, ['MF', '7F10', '5F3A']))
self.assertIsNone(_pbr_files_for_node(s, ['MF', '5F3A']))
children = []
self.assertEqual(_pbr_inject_children(s, ['MF', '7F10', '5F3A'], children), 0)
# ...while the bare DF request still resolves the single cached copy
self.assertIsNotNone(_pbr_files_for_node(s, ['5F3A']))
def test_an_adf_addressed_by_aid_resolves_to_its_name(self):
# The Profiler's tree walk addresses an ADF by AID (the file manager by
# name): both must fold onto the cache's name-based path.
aid = 'A0000000871002FF33FF0181FFFF8900'
self.assertEqual(_pbr_norm_path(['MF', aid, '5F3A'], {aid: 'ADF.USIM'}),
['ADF.USIM', '5F3A'])
self.assertEqual(_pbr_norm_path(['MF', 'ADF.USIM', '5F3A'], {aid: 'ADF.USIM'}),
['ADF.USIM', '5F3A'])
# with both copies cached the AID-shaped request still picks its own
s = _server([dict(PBR_ENTRY, path='MF/7F10/5F3A/4F30'),
dict(PBR_ENTRY, path='ADF.USIM/5F3A/4F30')])
children = []
self.assertEqual(_pbr_inject_children(
s, _pbr_norm_path(['MF', aid, '5F3A'], {aid: 'ADF.USIM'}), children), 4)
def test_the_pwa_style_adf_path_matches_with_two_cached_pbrs(self):
# The PWA sends the whole chain (MF/ADF.USIM/5F3A) while the cache path
# is rooted at the ADF; with both PBR copies cached the ADF's DF must
# still resolve to its own entry (the normaliser drops the leading MF).
s = _server([dict(PBR_ENTRY, path='MF/7F10/5F3A/4F30'),
dict(PBR_ENTRY, path='ADF.USIM/5F3A/4F30')])
children = []
self.assertEqual(_pbr_inject_children(s, _pbr_norm_path(['MF', 'ADF.USIM', '5F3A']),
children), 4)
self.assertEqual(_pbr_norm_path(['MF', 'ADF.USIM', '5F3A']), ['ADF.USIM', '5F3A'])
# ...and the MF/DF.TELECOM copy keeps matching exactly
children2 = []
self.assertEqual(_pbr_inject_children(s, _pbr_norm_path(['MF', '7F10', '5F3A']),
children2), 4)
# a bare FID stays ambiguous between the two copies: nothing is injected
children3 = []
self.assertEqual(_pbr_inject_children(s, ['5F3A'], children3), 0)
self.assertEqual(children3, [])
def test_the_adf_path_matches(self):
s = _server([dict(PBR_ENTRY, path='ADF.USIM/5F3A/4F30')])
children = []
self.assertEqual(_pbr_inject_children(s, ['ADF.USIM', '5F3A'], children), 4)
self.assertEqual(children[0]['fid'], '4F3A')
def test_an_unrelated_df_is_untouched(self):
s = _server([PBR_ENTRY])
children = [{'name': 'EF.ADN', 'fid': '4f3a', 'isDir': False}]
self.assertEqual(_pbr_inject_children(s, ['MF', '7F10', '5F3B'], children), 0)
self.assertEqual(len(children), 1)
self.assertEqual(_pbr_inject_children(_server(None), ['MF', '7F10', '5F3A'],
children), 0)
def test_a_known_fid_is_not_added_twice(self):
# pySim's model already lists EF.ADN under DF.PHONEBOOK
s = _server([PBR_ENTRY])
children = [{'name': 'EF.ADN', 'fid': '4F3A', 'isDir': False}]
self.assertEqual(_pbr_inject_children(s, ['MF', '7F10', '5F3A'], children), 3)
self.assertEqual([c['fid'] for c in children], ['4F3A', '4F09', '4F3D', '4F4A'])
def test_an_ambiguous_fid_only_path_is_not_guessed(self):
# the three PBR copies all sit in a DF with FID 5F3A: a bare '5F3A'
# cannot say which one, so nothing is injected
s = _server([PBR_ENTRY, dict(PBR_ENTRY, path='ADF.USIM/5F3A/4F30'),
dict(PBR_ENTRY, path='ADF.USIM-NON-IMSI/5F3A/4F30')])
children = []
self.assertEqual(_pbr_inject_children(s, ['5F3A'], children), 0)
self.assertEqual(children, [])
def test_a_single_cached_entry_answers_a_bare_fid(self):
children = []
self.assertEqual(_pbr_inject_children(_server([PBR_ENTRY]), ['5F3A'], children), 4)
def test_an_empty_path_leaves_children_untouched(self):
children = []
self.assertEqual(_pbr_inject_children(_server([PBR_ENTRY]), [], children), 0)
self.assertEqual(children, [])
def test_known_fid(self):
s = _server([PBR_ENTRY])
self.assertTrue(_pbr_known_fid(s, '4f3a'))
self.assertTrue(_pbr_known_fid(s, '4F4A'))
self.assertFalse(_pbr_known_fid(s, '4F30')) # EF.PBR itself: model-known
self.assertFalse(_pbr_known_fid(s, 'EF.ADN'))
self.assertFalse(_pbr_known_fid(s, None))
self.assertFalse(_pbr_known_fid(_server(None), '4F3A'))
def test_the_handlers_inject_and_auto_allow_the_probe(self):
# the /api/tree handler appends the PBR children...
tree = SRC[SRC.index("elif self.path == '/api/tree':"):
SRC.index("elif self.path == '/api/menu-select':")]
self.assertIn('_pbr_inject_children(self.server, node_path, children)', tree)
# ...and every tree/select/read/write handler auto-allows the probe for
# the PBR FIDs (after the caller's own allow_probe flag)
for marker in ("elif self.path == '/api/select':", "elif self.path == '/api/read':",
"elif self.path == '/api/write':", "elif self.path == '/api/tree':"):
i = SRC.index(marker)
seg = SRC[i:i + 3000]
self.assertIn('_pbr_known_fid(self.server', seg, marker)
self.assertLess(seg.index("allow_probe = bool(body.get('allow_probe'))"),
seg.index('_pbr_known_fid(self.server'), marker)
class PbrRouteTests(unittest.TestCase):
"""GET /api/pbr serves the session cache without touching the card."""
def setUp(self):
self.server = _build_http_server('127.0.0.1', 0, PysimHandler)
self.server.log_requests = False
self.server.app = None
threading.Thread(target=self.server.serve_forever, daemon=True).start()
self.port = self.server.server_address[1]
def tearDown(self):
self.server.shutdown()
self.server.server_close()
def _get(self, path):
with urllib.request.urlopen('http://127.0.0.1:%d%s' % (self.port, path),
timeout=5) as res:
return res.status, json.loads(res.read() or b'{}')
def test_pbr_is_a_card_free_get(self):
self.assertIn('/api/pbr', _CARD_FREE_GET)
def test_empty_cache_answers_not_found(self):
status, resp = self._get('/api/pbr')
self.assertEqual(status, 200, resp)
self.assertFalse(resp['ok'])
self.assertEqual(resp['pbrs'], [])
def test_the_cache_is_served_with_the_session(self):
self.server.pbr = [{'fid': '4F30', 'path': 'MF/7F10/5F3A/4F30',
'records': ['A80AC0034F3A02']}]
self.server.card_session = 9
status, resp = self._get('/api/pbr')
self.assertEqual(status, 200, resp)
self.assertTrue(resp['ok'])
self.assertEqual(resp['session'], 9)
self.assertEqual(resp['pbrs'][0]['fid'], '4F30')
if __name__ == '__main__':
unittest.main()