diff --git a/docs/decoded-editing.md b/docs/decoded-editing.md index 63b43f5..5436722 100644 --- a/docs/decoded-editing.md +++ b/docs/decoded-editing.md @@ -303,16 +303,24 @@ de-allocates. - **Filesystem check** (implemented, step 4b): many services declare a DF or EF (ADN -> EF.ADN 6F3A, SPN -> EF.SPN 6F46, GSM Access -> DF.GSM-ACCESS 5F3B, GPRS -> EF.KcGPRS, …); a second curated table (`SERVICES_FS_MAP`, keyed by - feature name) maps a feature to the FID(s) its `Filesystem` cell declares. - A **Check filesystem** action writes the services, probes the file-manager - tree once (`pysimFsProbeAll`) and flags mismatches — a declared service whose - file is absent, or a present file whose service is not declared. Services - that map to a capability rather than a file (CHV1 disable, Proactive SIM, - …) carry no file check. This first pass matches **FID-level**, not - (DF, FID)-exact as finding 6 asks: the documented FID collisions are not - disambiguated and the report says so (`docs/services-editor.md` §B.0). The - last report is kept in memory and dropped with the cached reads on a - card-session change. + feature name) maps a feature to the **symbolic files** its `Filesystem` cell + declares, resolved through the standard file list's `(name, root) -> path` + index — path-exact, so the same FID under another DF (EF.PBR/EF.SAI both + 4F30, EF.SST/EF.UST both 6F38) no longer explains a service. A **Check + filesystem** action probes the file-manager tree once + (`pysimFsProbeAll`) and flags mismatches — a declared service whose file is + absent, or a present file whose service is not declared in any checked tree; + a declared feature the standard list cannot resolve is reported separately as + *unchecked*. Services that map to a capability rather than a file (CHV1 + disable, Proactive SIM, …) carry no file check. The declarations are read + per application (SST -> MF, UST/EST -> ADF.USIM, IST -> ADF.ISIM) and a + USIM/ISIM role may resolve into the common DF.TELECOM subtree (the MF tree), + except the GSM application's mapped copies (EF.ADN/EF.EXT1/EF.ECCP, TS 31.102 + 4.5). The follow-ups are the `ADF.USIM-NON-IMSI` root alias and the files + missing from the standard list (`DF.GRAPHICS`/`EF.IMG`, the DF.TELECOM + DF.SoLSA/DF.MExE, EF.3GPPPSDATAOFFservicelist) — `docs/services-editor.md` + §B.0. The last report is kept in memory (it carries its resolved paths) and + dropped with the cached reads on a card-session change. ### 6.1 Candidates for the same cross-application treatment diff --git a/docs/services-editor.md b/docs/services-editor.md index 118ca09..18d9691 100644 --- a/docs/services-editor.md +++ b/docs/services-editor.md @@ -10,8 +10,8 @@ > Step 4a (the cross-file Services editor: §A's cross-reference, read, render, > per-file save) is implemented against §A; the counts below were verified > against the PWA's constants by `frontend/tests/services.test.js`. Step 4b -> (§B, the filesystem check) is implemented as well - see §B.0 for the -> representation and the FID-level rule it uses. +> (§B, the filesystem check) is implemented as well — see §B.0 for the +> representation and the path-exact `(name, root)` rule it uses. **Counts:** SST **59** / UST **150** / IST **22** / EST **3** services → **164 feature rows** (10 SIM-only, 85 USIM-only, 5 ISIM-only, 61 shared, @@ -231,44 +231,83 @@ The FS column in §A is deliverable B, merged for readability. Legend: `frontend/index.html` carries the column as `SERVICES_FS_MAP`, a map keyed by the §A feature name (the cross-reference's row identity) whose value is the -list of **every FID the cell mentions**, in the cell's order: +list of **symbolic file names the cell mentions**, in the cell's order: -- `EF.FDN 6F3B (+ EF.EXT2 6F4B)` → `['6F3B', '6F4B']` -- `DF.GRAPHICS 5F50 → EF.IMG 4F20` → `['5F50', '4F20']` (a DF's FID counts: - the DF itself is the file the service gates) -- `EF.NIA 6F51 (SIM) / 6FD3 (USIM)` → both trees' FIDs +- `EF.FDN 6F3B (+ EF.EXT2 6F4B)` → `['EF.FDN', 'EF.EXT2']` +- `DF.GRAPHICS 5F50 → EF.IMG 4F20` → `['DF.GRAPHICS', 'EF.IMG']` (a DF's own + name counts: the DF is the file the service gates) +- `EF.NIA 6F51 (SIM) / 6FD3 (USIM)` → `['EF.NIA']` — one name, a different + path per tree - `capability` → `[]`; the parenthetical references a capability cell carries (`gate: EF.CBMID 6F48`, `menu title: EF.SUME 6F54`, the URI set, `SUCI data`) are informational — the service does not declare a file of its own. The §A.4 RFU/reserved rows are not features and therefore have no entry. +The names are the standard file list's (`frontend/uicc_files.json`, built from +pySim by `pysim_simple_server/uicc_files.py`) `name` values, so the check can +resolve them **path-exactly**: a FID alone is ambiguous across applications +(`4F30` is EF.PBR *and* EF.SAI, `6F38` EF.SST *and* EF.UST, `6F4E` EF.EXT4 +under DF.TELECOM *and* EF.EXT5 under ADF.USIM). `servicesPathIndexBuild()` +turns the list into a `NAME|ROOT → path` index (uppercased both sides; the one +duplicate — pySim models `EF.SUCI_Calc_Info` twice under ADF.USIM, in DF.5GS +`4F07` and in the eSIM profile package's DF.SAIP `4F01` — takes the list's +first entry, the DF.5GS one). `servicesPathIndex()` loads it once per page +through `uiccStdFilesLoad()`. + The **Check filesystem** button (Services pill, needs a card) reads the four service tables, probes the file manager's tree once (`pysimFsProbeAll`; the `/api/tree` model lists files without testing them, so a loaded-but-unprobed -tree proves nothing), and compares the two sides: +tree proves nothing), collects the tree's **present paths** (every node with +`exists === true`, DFs included, as canonical `pysimFsNodePath` paths) and +compares the two sides: -- `missing` — a **declared** feature (any of its service bits set in a present - file; EF.SST only at *allocated + activated*, the state the editor shows) - whose mapped FIDs are absent from the card. In §A order. -- `extra` — a **present** FID that no declared feature maps. Sorted by FID, - naming every feature that maps it (all of them undeclared). A FID **any** - declared feature maps counts as explained and is not reported: several - features share one file (`4F01` under the ACDC/MCS/5GS/V2X/5G ProSe/5MBS/A2X - DFs, `6F4A`/`6F4B` across EXT1/EXT2 and BDN) and the rule cannot tell which - DF the present copy sits in. +- **declared per tree** — `servicesFsDeclaredTrees()`: EF.SST declares in the + **MF** tree, EF.UST/EF.EST in **ADF.USIM**, EF.IST in **ADF.ISIM** (the state + the editor shows: EF.SST only at *allocated + activated*). A tree whose + service table is not on the card declares nothing there (its declarations + are unknown, so it is not checked at all). +- `missing` — a declared feature whose resolved paths are absent, in §A order + then MF / ADF.USIM / ADF.ISIM; each entry carries the declaring `tree` and + the absent `paths` (in the cell's order). +- `extra` — a present path whose mapped features are declared in **none** of + the checked trees, sorted by path, with the path's own `tree`, its `role` + name and every mapping feature. A path **any** declared feature maps is + explained: several features share one file (DF.GSM-ACCESS across "GSM + Access"/"GSM security context", EXT2 across FDN/Extension 2, the 5G DFs' + `4F01`/`4F02`) and a declaration in one application may gate a file of + another (a UST service gating a DF.TELECOM DF — see below). +- `unchecked` — a declared feature with **no** path at all: the standard list + is missing the file, so its presence cannot be verified. Today: + `Image (IMG)` (`DF.GRAPHICS`/`EF.IMG` — pySim does not model DF.GRAPHICS), + `SoLSA` and `MExE` in the MF tree (the DF.TELECOM copies of DF.SoLSA/DF.MExE + are not in the list) and `3GPP PS Data Off Service List` + (EF.3GPPPSDATAOFFservicelist). Reported instead of guessed — the follow-up + is to add those files to `SUPPLEMENT` in `pysim_simple_server/uicc_files.py`. -The check is **FID-level, not (DF, FID)-exact** (this first pass; finding 6 in -`docs/decoded-editing.md` asks for DF-scoped resolution). The FID collisions -of Notes §3 — `6F4E` EXT4/EXT5, `6F4F` ECCP/CCP2, `6F07` IMSI/IST, `6F38` -SST/UST, `6FD7` MSK/GBANL, `5F3B` DF.GSM-ACCESS/DF.MULTIMEDIA — are therefore -not disambiguated, and the report says so. `EF.PBR 4F30` also exists in -several DFs, so a present `4F30` explains `ADN`/`SoLSA` regardless of which -copy it is. +**Resolution per declaring tree.** A role resolves in the declaring tree's +root first, then — for a USIM/ISIM declaration — in the **MF** tree, because +several UST-gated DFs are children of the common DF.TELECOM (TS 31.102 4.6.0: +DF.MULTIMEDIA `5F3B`, DF.MCS `5F3D`, DF.V2X `5F3E`, DF.A2X `5F3F`, +DF.GRAPHICS `5F50`), i.e. `MF/7F10/…` paths. The exceptions +(`SERVICES_FS_GSM_MAPPED`) are the files TS 31.102 4.5 lists as *mappings* of a +DF.PHONEBOOK file into DF.TELECOM "in case of a present GSM application" — +EF.ADN `6F3A`, EF.EXT1 `6F4A`, EF.ECCP `6F4F`: a USIM declaration's phonebook +is DF.PHONEBOOK + EF.PBR (ADR's `EF.ADN` is the SIM's file), so those are +never expected in a USIM/ISIM tree. A role the list knows for another root +only is skipped, not a failure — a feature's files are the union of its +applications' files (ADN = EF.ADN for the SIM, DF.PHONEBOOK + EF.PBR for the +USIM). -The pure pieces (`servicesFsDeclared`, `servicesFsPresentFids`, -`servicesFsReport`, `servicesFsReportHtml`) are DOM-free and unit-tested; the -last report is kept in memory and redrawn on a language switch, and it is -dropped with the cached reads on a card-session change. +**Not covered (follow-ups):** `ADF.USIM-NON-IMSI` — the list mirrors the +ADF.USIM files under that root, but a card's tree names the application +`ADF.USIM-non-IMSI`, so a non-IMSI USIM card's UST/EST paths are not matched +yet (resolve it as an alias of ADF.USIM); and the registry gaps listed above. + +The pure pieces (`servicesFsDeclaredTrees`, `servicesFsPresentPaths`, +`servicesPathIndexBuild`, `servicesFsRolePath`, `servicesFsReport`, +`servicesFsReportHtml`) are DOM/fetch-free and unit-tested; the last report is +kept in memory (it carries its resolved paths) and redrawn on a language +switch, and it is dropped with the cached reads on a card-session change. ### B.1 `capability` entries (no single file) @@ -371,19 +410,31 @@ carries **147–150** (5MBS UE pre-configuration / Operator Controlled Signal Threshold per Access Technology / A2X / IMS Data Channel Indication), matching `UICC_FILES.md` §6.1 and TS 31.102 §4.2.8, so the tables above are complete. -### 5. Source-coverage gaps (cardless list only) +### 5. Source-coverage gaps (the standard file list) -The Services editor is **card-only**: it probes the equipped card by FID and -writes via `/api/write`, so it does **not** depend on `frontend/uicc_files.json` -(that is the *cardless* standard-file list feeding the SELECT picker and the -test-script file picker). The FS check probes the card directly, so the files -below missing from `uicc_files.json` are not a blocker — they matter only if a -cardless picker must offer them: EF.IMG 4F20 / DF.GRAPHICS 5F50, DF.MExE 5F3C + -EF.MExE-ST/ORPK/ARPK/TPRPK 4F40–4F43, DF.SoLSA 5F70 + EF.SAI 4F30/EF.SLL 4F31, -DF.BCAST 5F80, DF.ACDC 5FA0 + EF.ACDC_LIST 4F01, DF.TV 5FB0 + EF.TVUSD, -EF.TVCONFIG 6FFB, EF.3GPPPSDATAOFF 6FF9 / EF.3GPPPSDATAOFFservicelist 6FFA, -EF.IAL 6FF0 / EF.IPD 6FF2 (pairing), EF.PSISMSC 6FE5, and the 5G/A2X/5MBS -DFs (5FC0/5F3F/5F3E/5FF0/5FF1/5FE0). +The Services editor is **card-only**: it probes the equipped card's service +tables by path and writes via `/api/write`. The **FS check** additionally +needs the *cardless* standard file list (`frontend/uicc_files.json`) to resolve +a feature's symbolic files to paths (`(name, root) -> path`, §B.0) — the card's +own tree supplies presence, the list supplies the paths. Part 1 of step 4b +added the service-gated files pySim's model omits to `SUPPLEMENT` +(`pysim_simple_server/uicc_files.py`): EF.LRPLMNSI, EF.IAL, EF.IPD, +EF.TVCONFIG, EF.3GPPPSDATAOFF, DF.MExE + its files, DF.SoLSA + EF.SAI/EF.SLL, +DF.BCAST, DF.ACDC + EF.ACDC_LIST, DF.TV, DF.A2X + EF.AST/EF.A2X_CONFIG. + +Still missing from the list, and therefore reported as *unchecked* by the +check (a declared feature whose files have no path) rather than guessed: + +- **DF.GRAPHICS 5F50 → EF.IMG 4F20** — the DF.TELECOM child (TS 31.102 4.6.0); + pySim's SIM model does not carry DF.GRAPHICS at all. +- **DF.SoLSA 5F70 / DF.MExE 5F3C under DF.TELECOM** — the list carries them + under ADF.USIM only, so an *SST* declaration of SoLSA (service 40) or MExE + (49) cannot be checked in the MF tree. +- **EF.3GPPPSDATAOFFservicelist 6FFA** — a pySim TODO. + +A non-IMSI USIM (`ADF.USIM-NON-IMSI`) is mirrored in the list, but a card's +tree names the application `ADF.USIM-non-IMSI`; resolving the two as one root +is the follow-up (§B.0). ### 6. Deliberately unmapped diff --git a/frontend/index.html b/frontend/index.html index ff05eec..dc88fe8 100644 --- a/frontend/index.html +++ b/frontend/index.html @@ -23301,192 +23301,208 @@ const SERVICES_FILES = [ const SERVICES_TABLES = { ust: EF_UST_SERVICES, est: EF_EST_SERVICES, ist: EF_IST_SERVICES }; -// Feature -> the FID(s) its Filesystem cell declares, transcribed from -// docs/services-editor.md §A (the merged "Filesystem" column, legend in §B): -// every FID the cell mentions, in order - `EF.FDN 6F3B (+ EF.EXT2 6F4B)` -// -> ['6F3B', '6F4B'], `DF.GRAPHICS 5F50 -> EF.IMG 4F20` -> ['5F50', '4F20'] -// (a DF's FID counts: the DF itself is the file the service gates), -// `EF.NIA 6F51 (SIM) / 6FD3 (USIM)` -> both trees' FIDs. +// Feature -> the *symbolic* files its Filesystem cell declares, transcribed +// from docs/services-editor.md §A (the merged "Filesystem" column, legend in +// §B): the file names only - `EF.FDN 6F3B (+ EF.EXT2 6F4B)` -> ['EF.FDN', +// 'EF.EXT2'], `DF.GRAPHICS 5F50 -> EF.IMG 4F20` -> ['DF.GRAPHICS', 'EF.IMG'] +// (a DF's own name counts: the DF is the file the service gates), `EF.NIA 6F51 +// (SIM) / 6FD3 (USIM)` -> ['EF.NIA'] (one name, a different path per tree). +// The names are the standard file list's (frontend/uicc_files.json) `name` +// values, so a role resolves to a path through (name, root) - the FID alone +// is ambiguous across applications (EF.PBR and EF.SAI are both 4F30, EF.SST +// and EF.UST are both 6F38). // `capability` cells (procedural/terminal-side features with no file of their // own) map to [] - their parenthetical references (a gate, a title, a URI // set) are informational, not a file the service declares - and so do the // §A.4 RFU/reserved rows, which are not features and therefore absent here. // The map is keyed by feature name because that is the cross-reference's key // (the editor's row identity); SERVICES_FILES above is the four *service -// table* descriptors, so the name is taken. Coverage is pinned by -// frontend/tests/services.test.js. -// -// The check is deliberately FID-level, not (DF, FID)-exact (docs/decoded- -// editing.md finding 6 asks for DF-scoped resolution; this first pass accepts -// the coarser rule, visible to the user as the report's note): the known -// collisions - 6F4E EXT4/EXT5, 6F4F ECCP/CCP2, 6F07 IMSI/IST, 6F38 SST/UST, -// 6FD7 MSK/GBANL, 5F3B DF.GSM-ACCESS/DF.MULTIMEDIA - are matched by FID -// anywhere in the card's tree, so a file carrying a colliding FID can explain -// a service it does not actually gate. +// table* descriptors, so the name is taken. Coverage and the (name, root) +// resolution are pinned by frontend/tests/services.test.js. const SERVICES_FS_MAP = { 'CHV1 disable function': [], - 'Local Phone Book (ADN)': ['6F3A', '5F3A', '4F30'], - 'Fixed Dialling Numbers (FDN)': ['6F3B', '6F4B'], - 'Short Message Storage (SMS)': ['6F3C'], - 'Advice of Charge (AoC)': ['6F39', '6F37', '6F41'], - 'Capability Configuration Parameters (CCP)': ['6F3D'], - 'Capability Configuration Parameters 2 (CCP2)': ['6F4F'], - 'PLMN selector': ['6F30'], - 'MSISDN': ['6F40'], - 'Extension 1': ['6F4A'], - 'Extension 2': ['6F4B'], - 'SMS Parameters (SMSP)': ['6F42'], - 'Last Number Dialled (LND)': ['6F44'], - 'Cell Broadcast Message Identifier (CBMI)': ['6F45'], - 'Group Identifier Level 1 (GID1)': ['6F3E'], - 'Group Identifier Level 2 (GID2)': ['6F3F'], - 'Service Provider Name (SPN)': ['6F46'], - 'Service Dialling Numbers (SDN)': ['6F49', '6F4C'], - 'Extension 3': ['6F4C'], - 'VGCS Group Identifier List': ['6FB1', '6FB2'], - 'VBS Group Identifier List': ['6FB3', '6FB4'], - 'enhanced Multi-Level Precedence and Pre-emption (eMLPP)': ['6FB5'], - 'Automatic Answer for eMLPP (AAeM)': ['6FB6'], + 'Local Phone Book (ADN)': ['EF.ADN', 'DF.PHONEBOOK', 'EF.PBR'], + 'Fixed Dialling Numbers (FDN)': ['EF.FDN', 'EF.EXT2'], + 'Short Message Storage (SMS)': ['EF.SMS'], + 'Advice of Charge (AoC)': ['EF.ACM', 'EF.ACMmax', 'EF.PUCT'], + 'Capability Configuration Parameters (CCP)': ['EF.CCP'], + 'Capability Configuration Parameters 2 (CCP2)': ['EF.CCP2'], + 'PLMN selector': ['EF.PLMNsel'], + 'MSISDN': ['EF.MSISDN'], + 'Extension 1': ['EF.EXT1'], + 'Extension 2': ['EF.EXT2'], + 'SMS Parameters (SMSP)': ['EF.SMSP'], + 'Last Number Dialled (LND)': ['EF.LND'], + 'Cell Broadcast Message Identifier (CBMI)': ['EF.CBMI'], + 'Group Identifier Level 1 (GID1)': ['EF.GID1'], + 'Group Identifier Level 2 (GID2)': ['EF.GID2'], + 'Service Provider Name (SPN)': ['EF.SPN'], + 'Service Dialling Numbers (SDN)': ['EF.SDN', 'EF.EXT3'], + 'Extension 3': ['EF.EXT3'], + 'VGCS Group Identifier List': ['EF.VGCS', 'EF.VGCSS'], + 'VBS Group Identifier List': ['EF.VBS', 'EF.VBSS'], + 'enhanced Multi-Level Precedence and Pre-emption (eMLPP)': ['EF.eMLPP'], + 'Automatic Answer for eMLPP (AAeM)': ['EF.AAeM'], 'Data download via SMS-CB': [], 'Data download via SMS-PP': [], 'Menu selection': [], 'Call control (by SIM/USIM)': [], 'Proactive SIM': [], - 'Cell Broadcast Message Identifier Ranges (CBMIR)': ['6F50'], - 'Barred Dialling Numbers (BDN)': ['6F4D', '6F4E', '6F55'], - 'Extension 4': ['6F4E', '6F55'], - 'De-personalisation Control Keys (DCK)': ['6F2C'], - 'Co-operative Network List (CNL)': ['6F32'], - 'Short Message Status Reports (SMSR)': ['6F47'], - 'Network\'s indication of alerting in the MS (NIA)': ['6F51', '6FD3'], + 'Cell Broadcast Message Identifier Ranges (CBMIR)': ['EF.CBMIR'], + 'Barred Dialling Numbers (BDN)': ['EF.BDN', 'EF.EXT4'], + 'Extension 4': ['EF.EXT4'], + 'De-personalisation Control Keys (DCK)': ['EF.DCK'], + 'Co-operative Network List (CNL)': ['EF.CNL'], + 'Short Message Status Reports (SMSR)': ['EF.SMSR'], + 'Network\'s indication of alerting in the MS (NIA)': ['EF.NIA'], 'MO-SMS control (by SIM/USIM)': [], - 'GPRS': ['6F52', '7F20'], - 'Image (IMG)': ['5F50', '4F20'], - 'SoLSA': ['5F70', '4F30', '4F31'], + 'GPRS': ['EF.KcGPRS', 'DF.GSM'], + 'Image (IMG)': ['DF.GRAPHICS', 'EF.IMG'], + 'SoLSA': ['DF.SoLSA', 'EF.SAI', 'EF.SLL'], 'USSD string data object supported in Call Control': [], 'RUN AT COMMAND command': [], - 'User controlled PLMN selector with Access Technology': ['6F60'], - 'Operator controlled PLMN selector with Access Technology': ['6F61'], - 'HPLMN selector with Access Technology': ['6F62'], - 'CPBCCH Information': ['6F63', '4F63'], - 'Investigation Scan': ['6F64', '4F64'], - 'Extended Capability Configuration Parameters (ECCP)': ['6F4F'], - 'MExE': ['5F3C', '4F40'], - 'PLMN Network Name (PNN)': ['6FC5'], - 'Operator PLMN List (OPL)': ['6FC6'], - 'Mailbox Dialling Numbers (MBDN)': ['6FC7', '6FC8'], - 'Message Waiting Indication Status (MWIS)': ['6FCA', '6FC9'], - 'Call Forwarding Indication Status (CFIS)': ['6FCB', '6FCC'], - 'Service Provider Display Information (SPDI)': ['6FCD'], - 'Multimedia Messaging Service (MMS)': ['6FCE'], - 'Extension 8': ['6FCF'], - 'MMS User Connectivity Parameters': ['6FD0', '6FD2', '6FD1'], - 'Outgoing Call Information (OCI and OCT)': ['6F81', '6F83'], - 'Incoming Call Information (ICI and ICT)': ['6F80', '6F82'], - 'GSM Access': ['5F3B'], - 'Enabled Services Table': ['6F56'], - 'APN Control List (ACL)': ['6F57'], - 'GSM security context': ['4F20', '4F52', '5F3B'], - 'Extension 5': ['6F4E'], + 'User controlled PLMN selector with Access Technology': ['EF.PLMNwAcT'], + 'Operator controlled PLMN selector with Access Technology': ['EF.OPLMNwAcT'], + 'HPLMN selector with Access Technology': ['EF.HPLMNwAcT'], + 'CPBCCH Information': ['EF.CPBCCH'], + 'Investigation Scan': ['EF.InvScan'], + 'Extended Capability Configuration Parameters (ECCP)': ['EF.ECCP'], + 'MExE': ['DF.MExE', 'EF.MExE-ST'], + 'PLMN Network Name (PNN)': ['EF.PNN'], + 'Operator PLMN List (OPL)': ['EF.OPL'], + 'Mailbox Dialling Numbers (MBDN)': ['EF.MBDN', 'EF.EXT6'], + 'Message Waiting Indication Status (MWIS)': ['EF.MWIS', 'EF.MBI'], + 'Call Forwarding Indication Status (CFIS)': ['EF.CFIS', 'EF.EXT7'], + 'Service Provider Display Information (SPDI)': ['EF.SPDI'], + 'Multimedia Messaging Service (MMS)': ['EF.MMSN'], + 'Extension 8': ['EF.EXT8'], + 'MMS User Connectivity Parameters': ['EF.MMSICP', 'EF.MMSUCP', 'EF.MMSUP'], + 'Outgoing Call Information (OCI and OCT)': ['EF.OCI', 'EF.OCT'], + 'Incoming Call Information (ICI and ICT)': ['EF.ICI', 'EF.ICT'], + 'GSM Access': ['DF.GSM-ACCESS'], + 'Enabled Services Table': ['EF.EST'], + 'APN Control List (ACL)': ['EF.ACL'], + 'GSM security context': ['EF.Kc', 'EF.KcGPRS', 'DF.GSM-ACCESS'], + 'Extension 5': ['EF.EXT5'], 'Call control on GPRS by USIM': [], - 'Pseudonym': ['4F41', '5F40'], - 'User Controlled PLMN selector for I-WLAN access': ['4F42'], - 'Operator Controlled PLMN selector for I-WLAN access': ['4F43'], - 'User controlled WSID list': ['4F44'], - 'Operator controlled WSID list': ['4F45'], - 'VGCS security': ['6FD4'], - 'VBS security': ['6FD5'], - 'WLAN Reauthentication Identity': ['4F46'], - 'Multimedia Messages Storage': ['4F47', '4F48', '5F3B'], - 'Generic Bootstrapping Architecture (GBA)': ['6FD6', '6FD5', '6FDA', '6FD7', '6FDD'], - 'MBMS security': ['6FD7', '6FD8'], + 'Pseudonym': ['EF.Pseudo', 'DF.WLAN'], + 'User Controlled PLMN selector for I-WLAN access': ['EF.UPLMNWLAN'], + 'Operator Controlled PLMN selector for I-WLAN access': ['EF.OPLMNWLAN'], + 'User controlled WSID list': ['EF.UWSIDL'], + 'Operator controlled WSID list': ['EF.OWSIDL'], + 'VGCS security': ['EF.VGCSCA'], + 'VBS security': ['EF.VBSCA'], + 'WLAN Reauthentication Identity': ['EF.WRI'], + 'Multimedia Messages Storage': ['EF.MML', 'EF.MMDF', 'DF.MULTIMEDIA'], + 'Generic Bootstrapping Architecture (GBA)': ['EF.GBABP', 'EF.GBANL', 'EF.NAFKCA'], + 'MBMS security': ['EF.MSK', 'EF.MUK'], 'Data download via USSD and USSD application mode': [], - 'Equivalent HPLMN': ['6FD9'], + 'Equivalent HPLMN': ['EF.EHPLMN'], 'Additional TERMINAL PROFILE after UICC activation': [], - 'Equivalent HPLMN Presentation Indication': ['6FDB'], - 'Last RPLMN Selection Indication': ['6FDC'], - 'OMA BCAST Smart Card Profile': ['5F80'], + 'Equivalent HPLMN Presentation Indication': ['EF.EHPLMNPI'], + 'Last RPLMN Selection Indication': ['EF.LRPLMNSI'], + 'OMA BCAST Smart Card Profile': ['DF.BCAST'], 'GBA-based Local Key Establishment Mechanism': [], 'Terminal Applications': [], - 'Service Provider Name Icon': ['6FDE'], - 'PLMN Network Name Icon': ['6FDF'], - 'Connectivity Parameters for USIM IP connections': ['6FE2'], - 'Home I-WLAN Specific Identifier List': ['4F47'], - 'I-WLAN Equivalent HPLMN Presentation Indication': ['4F48'], - 'I-WLAN HPLMN Priority Indication': ['4F49'], - 'I-WLAN Last Registered PLMN': ['4F4A'], - 'EPS Mobility Management Information': ['6FE3', '6FE4'], - 'Allowed CSG Lists and corresponding indications': ['4F81', '4F82', '4F83'], + 'Service Provider Name Icon': ['EF.SPNI'], + 'PLMN Network Name Icon': ['EF.PNNI'], + 'Connectivity Parameters for USIM IP connections': ['EF.NCP-IP'], + 'Home I-WLAN Specific Identifier List': ['EF.HWSIDL'], + 'I-WLAN Equivalent HPLMN Presentation Indication': ['EF.WEHPLMNPI'], + 'I-WLAN HPLMN Priority Indication': ['EF.WHPI'], + 'I-WLAN Last Registered PLMN': ['EF.WLRPLMN'], + 'EPS Mobility Management Information': ['EF.EPSLOCI', 'EF.EPSNSC'], + 'Allowed CSG Lists and corresponding indications': ['EF.ACSGL', 'EF.CSGT', 'EF.HNBN'], 'Call control on EPS PDN connection by USIM': [], - 'HPLMN Direct Access': ['4F4B'], - 'eCall Data': ['6F49', '6F3B'], - 'Operator CSG Lists and corresponding indications': ['4F84', '4F85', '4F86'], + 'HPLMN Direct Access': ['EF.HPLMNDAI'], + 'eCall Data': ['EF.SDN', 'EF.FDN'], + 'Operator CSG Lists and corresponding indications': ['EF.OCSGL', 'EF.OCSGT', 'EF.OHNBN'], 'Support for SM-over-IP': [], 'Support of CSG Display Control': [], 'Communication Control for IMS': [], 'Extended Terminal Applications': [], 'Support of UICC access to IMS': [], - 'Non-Access Stratum configuration by USIM': ['6FE8'], - 'PWS configuration by USIM': ['6FEC'], + 'Non-Access Stratum configuration by USIM': ['EF.NASCONFIG'], + 'PWS configuration by USIM': ['EF.PWS'], 'URI support by UICC': [], 'Extended EARFCN support': [], - 'ProSe': ['5F90', '4F10'], - 'USAT Application Pairing': ['6FF1', '6FF0', '6FF2'], + 'ProSe': ['DF.ProSe', 'EF.PST'], + 'USAT Application Pairing': ['EF.IPS', 'EF.IAL', 'EF.IPD'], 'Media Type support': [], 'IMS call disconnection cause': [], 'URI support for MO SHORT MESSAGE CONTROL': [], - 'ePDG configuration Information support': ['6FF4'], - 'ePDG configuration Information configured': ['6FF3'], - 'ACDC support': ['5FA0', '4F01'], - 'MCPTT': ['5F3D', '4F01'], - 'ePDG configuration Information for Emergency Service support': ['6FF6'], - 'ePDG configuration Information for Emergency Service configured': ['6FF5'], - 'eCall Data over IMS': ['6FEF', '6FED'], + 'ePDG configuration Information support': ['EF.ePDGSelection'], + 'ePDG configuration Information configured': ['EF.ePDGId'], + 'ACDC support': ['DF.ACDC', 'EF.ACDC_LIST'], + 'MCPTT': ['DF.MCS', 'EF.MST'], + 'ePDG configuration Information for Emergency Service support': ['EF.ePDGSelectionEm'], + 'ePDG configuration Information for Emergency Service configured': ['EF.ePDGIdEm'], + 'eCall Data over IMS': ['EF.SDNURI', 'EF.FDNURI'], 'URI support for SMS-PP DOWNLOAD': [], - 'From Preferred': ['6FF7'], - 'IMS configuration data': ['6FF8'], - 'TV configuration': ['6FFB', '5FB0'], - '3GPP PS Data Off': ['6FF9'], - '3GPP PS Data Off Service List': ['6FFA'], - 'V2X': ['5F3E', '4F01', '4F02'], - 'XCAP Configuration Data': ['6FFC'], - 'EARFCN list for MTC/NB-IOT UEs': ['6FFD'], - '5GS Mobility Management Information': ['5FC0', '4F01', '4F02'], - '5G Security Parameters': ['4F05', '4F03', '4F04'], + 'From Preferred': ['EF.FromPreferred'], + 'IMS configuration data': ['EF.IMSConfigData'], + 'TV configuration': ['EF.TVCONFIG', 'DF.TV'], + '3GPP PS Data Off': ['EF.3GPPPSDATAOFF'], + '3GPP PS Data Off Service List': ['EF.3GPPPSDATAOFFservicelist'], + 'V2X': ['DF.V2X', 'EF.VST', 'EF.V2X_CONFIG'], + 'XCAP Configuration Data': ['EF.XCAPConfigData'], + 'EARFCN list for MTC/NB-IOT UEs': ['EF.EARFCNList'], + '5GS Mobility Management Information': ['DF.5GS', 'EF.5GS3GPPLOCI', 'EF.5GSN3GPPLOCI'], + '5G Security Parameters': ['EF.5GAUTHKEYS', 'EF.5GS3GPPNSC', 'EF.5GSN3GPPNSC'], 'Subscription identifier privacy support': [], - 'SUCI calculation by the USIM': ['4F07'], - 'UAC Access Identities support': ['4F06'], + 'SUCI calculation by the USIM': ['EF.SUCI_Calc_Info'], + 'UAC Access Identities support': ['EF.UAC_AIC'], 'Expect control plane-based Steering of Roaming information during initial registration in VPLMN': [], 'Call control on PDU Session by USIM': [], - '5GS Operator PLMN List': ['4F08'], - 'Support for SUPI of type NSI or GLI or GCI': ['4F09'], + '5GS Operator PLMN List': ['EF.OPL5G'], + 'Support for SUPI of type NSI or GLI or GCI': ['EF.SUPI_NAI'], '3GPP PS Data Off separate Home and Roaming lists': [], - 'Support for URSP by USIM': ['4F0B'], + 'Support for URSP by USIM': ['EF.URSP'], '5G Security Parameters extended': [], - 'MuD and MiD configuration data': ['6FFE'], + 'MuD and MiD configuration data': ['EF.MuDMiDConfigData'], 'Support for Trusted non-3GPP access networks by USIM': [], - 'Support for multiple records of NAS security context storage for multiple registration': ['4F03', '4F04'], - 'Pre-configured CAG information list': ['4F0D'], - 'SOR-CMCI storage in USIM': ['4F0E'], - '5G ProSe': ['5FF0', '4F01'], - 'Storage of disaster roaming information in USIM': ['4F0F'], - 'Pre-configured eDRX parameters': ['4F10'], - '5G NSWO support': ['4F11'], - 'PWS configuration for SNPN in USIM': ['4F01', '5FE0'], - 'Multiplier Coefficient for Higher Priority PLMN search via NG-RAN satellite access': ['4F15'], - 'K_AUSF derivation configuration': ['4F16'], - 'Network Identifier for SNPN (NID)': ['4F02', '5FE0'], - '5MBS UE pre-configuration': ['5FF1', '4F01'], - 'Operator Controlled Signal Threshold per Access Technology': ['6F02'], - 'A2X': ['5F3F', '4F01'], - 'IMS Data Channel Indication': ['6F0B'], - 'P-CSCF address': ['6F09'], + 'Support for multiple records of NAS security context storage for multiple registration': ['EF.5GS3GPPNSC', 'EF.5GSN3GPPNSC'], + 'Pre-configured CAG information list': ['EF.CAG'], + 'SOR-CMCI storage in USIM': ['EF.SOR-CMCI'], + '5G ProSe': ['DF.5G_ProSe', 'EF.5G_PROSE_ST'], + 'Storage of disaster roaming information in USIM': ['EF.DRI'], + 'Pre-configured eDRX parameters': ['EF.5GSEDRX'], + '5G NSWO support': ['EF.5GNSWO_CONF'], + 'PWS configuration for SNPN in USIM': ['EF.PWS_SNPN', 'DF.SNPN'], + 'Multiplier Coefficient for Higher Priority PLMN search via NG-RAN satellite access': ['EF.MCHPPLMN'], + 'K_AUSF derivation configuration': ['EF.KAUSF_DERIVATION'], + 'Network Identifier for SNPN (NID)': ['EF.NID', 'DF.SNPN'], + '5MBS UE pre-configuration': ['DF.5MBSUECONFIG', 'EF.5MBSUECONFIG'], + 'Operator Controlled Signal Threshold per Access Technology': ['EF.OCST'], + 'A2X': ['DF.A2X', 'EF.AST', 'EF.A2X_CONFIG'], + 'IMS Data Channel Indication': ['EF.IMSDCI'], + 'P-CSCF address': ['EF.P-CSCF'], 'HTTP Digest': [], 'Support of P-CSCF discovery for IMS Local Break Out': [], - 'WebRTC URI': ['6FFA'], - 'IMS Data Channel indication': ['6F0B'], + 'WebRTC URI': ['EF.WebRTCURI'], + 'IMS Data Channel indication': ['EF.IMSDCI'], }; +// The application whose service table declares a feature (the report's +// `tree`): EF.SST lists the SIM (MF) services, EF.UST/EF.EST the USIM's, +// EF.IST the ISIM's. +const SERVICES_FS_TREES = { sst: 'MF', ust: 'ADF.USIM', est: 'ADF.USIM', ist: 'ADF.ISIM' }; +// Report order of the trees: the SIM's first, then the applications. +const SERVICES_FS_TREE_ORDER = ['MF', 'ADF.USIM', 'ADF.ISIM']; +// A USIM/ISIM-declared file may live in the common DF.TELECOM subtree instead +// of the application's own DF (TS 31.102 4.6.0: DF.MULTIMEDIA '5F3B', +// DF.MCS '5F3D', DF.V2X '5F3E', DF.A2X '5F3F', DF.GRAPHICS '5F50' are +// children of DF.TELECOM, i.e. MF paths), so a role that has no path under +// the declaring application's root falls back to the MF tree. +// +// The exceptions are the files TS 31.102 4.5 lists as *mappings* of a +// DF.PHONEBOOK file into DF.TELECOM ("in case of a present GSM application"): +// EF.ADN '6F3A', EF.EXT1 '6F4A' and EF.ECCP '6F4F' under DF.TELECOM are the +// GSM application's backwards-compatible copies, so a USIM/ISIM declaration +// (whose phonebook lives in DF.PHONEBOOK, EF.PBR-referenced) must not expect +// them. A SIM (SST) declaration does, and resolves them in the MF tree. +const SERVICES_FS_GSM_MAPPED = ['EF.ADN', 'EF.EXT1', 'EF.ECCP']; + function servicesDecode(key, bytes) { if (key === 'sst') return efDecSst(bytes); return efDecServiceTable(bytes, SERVICES_TABLES[key]); @@ -23606,40 +23622,103 @@ function servicesFileBytes(key, values, files) { } // --- filesystem check (docs/services-editor.md §B) --- -// Three pure pieces: which features the card declares (from the same decoded -// service tables the grid renders), which FIDs the card actually has (from -// the file manager's probed tree) and the mismatch report between the two. +// The check is path-exact: a feature's symbolic files (SERVICES_FS_MAP) are +// resolved through the standard file list's (name, root) -> path index, and +// compared with the paths the card's probed tree confirms. The pure pieces +// are DOM/fetch-free: the index builder, the per-tree declaration sets, the +// present-path collector and the report. -// Feature names whose service bit is set in at least one of the present -// service files - the "declared" side of the check. A file that is absent -// from the card leaves its column null (servicesFeatureState), so it declares -// nothing; EF.SST counts only at 'on' (allocated + activated), as the editor -// renders it. -function servicesFsDeclared(files) { - const dec = key => (files && files[key] && files[key].present) ? files[key].dec : null; - const sstDec = dec('sst'), ustDec = dec('ust'), estDec = dec('est'), istDec = dec('ist'); - const out = []; +// The (name, root) key of the path index. Both sides are uppercased so a +// card's tree names and the standard list's names compare directly. +function servicesFsPathKey(name, root) { + return String(name || '').toUpperCase() + '|' + String(root || '').toUpperCase(); +} + +// Canonical form of a path: uppercase, so the registry's and the tree's +// paths compare directly. +function servicesFsNormPath(path) { + return String(path || '').toUpperCase(); +} + +// NAME|ROOT -> canonical path, from the standard file list +// (frontend/uicc_files.json, loaded by uiccStdFilesLoad). A name is unique +// per root except for EF.SUCI_Calc_Info under ADF.USIM, which pySim models +// twice (DF.5GS/4F07, the UST service 125 file, and the eSIM profile +// package's DF.SAIP/4F01 copy) - the list's first entry wins, which is the +// DF.5GS one (the list is sorted, so this is deterministic). +function servicesPathIndexBuild(files) { + const index = new Map(); + for (const e of (files || [])) { + if (!e || !e.name || !e.root || !e.path) continue; + const key = servicesFsPathKey(e.name, e.root); + if (!index.has(key)) index.set(key, servicesFsNormPath(e.path)); + } + return index; +} + +let _servicesFsIndex = null; + +// The path index, loaded once per page from the standard file list (static +// and precached by the service worker, so the check works without a server +// round-trip for the list itself). +async function servicesPathIndex() { + if (!_servicesFsIndex) _servicesFsIndex = servicesPathIndexBuild(await uiccStdFilesLoad()); + return _servicesFsIndex; +} + +// The path a role resolves to in a declaring tree: the tree's own root first, +// then the common DF.TELECOM subtree (the MF tree) for a USIM/ISIM-declared +// role - except the GSM application's mapped copies, which only a SIM +// declaration expects (see SERVICES_FS_GSM_MAPPED). undefined when the +// standard list has no such file (the report's `unchecked`). +function servicesFsRolePath(index, role, root) { + if (!index) return undefined; + const own = index.get(servicesFsPathKey(role, root)); + if (own) return own; + if (root === 'MF') return undefined; + if (SERVICES_FS_GSM_MAPPED.indexOf(String(role || '').toUpperCase()) >= 0) return undefined; + return index.get(servicesFsPathKey(role, 'MF')); +} + +// Feature names the card declares, per declaring tree: { MF: [...], +// 'ADF.USIM': [...], 'ADF.ISIM': [...] } - the same decoded tables the grid +// renders, in cross-reference order. A tree is present only while its +// service table is on the card (without one the card declares nothing there, +// and every file of that area would look unused); EF.SST counts a feature +// only at 'on' (allocated + activated), as the editor renders it. +function servicesFsDeclaredTrees(files) { + const dec = key => (files && files[key] && files[key].present) ? (files[key].dec || null) : null; + const decs = { sst: dec('sst'), ust: dec('ust'), est: dec('est'), ist: dec('ist') }; + const out = {}; for (const f of SERVICES_CROSS_REF) { - const st = servicesFeatureState(f, sstDec, ustDec, estDec, istDec); - if (st.sst === 'on' || st.ust === true || st.est === true || st.ist === true) out.push(f.f); + const st = servicesFeatureState(f, decs.sst, decs.ust, decs.est, decs.ist); + for (const spec of SERVICES_FILES) { + if (!decs[spec.key]) continue; + if (spec.key === 'sst' ? st.sst !== 'on' : st[spec.key] !== true) continue; + const root = SERVICES_FS_TREES[spec.key]; + const list = out[root] || (out[root] = []); + if (list.indexOf(f.f) < 0) list.push(f.f); + } } return out; } -// FIDs present on the card: any node of the file-manager tree that the probe +// Paths present on the card: every node of the file-manager tree the probe // confirmed (exists === true), DFs included - a service gating a whole DF -// declares the DF's FID. A node that was never probed (exists === null) is -// not counted, so an unprobed tree reports nothing as present, and a DF -// confirmed absent is not descended into. -function servicesFsPresentFids(root) { +// declares the DF's path. A node that was never probed (exists === null) is +// not counted, and an absent DF is not descended into (the probe does not +// fetch its children either). +function servicesFsPresentPaths(root) { const out = []; - const seen = {}; + const seen = new Set(); const walk = node => { for (const ch of ((node && node.children) || [])) { - const fid = String(ch.fid || '').toUpperCase(); - if (ch.exists === true && /^[0-9A-F]{4}$/.test(fid) && !seen[fid]) { - seen[fid] = true; - out.push(fid); + if (ch.exists === true) { + const p = servicesFsNormPath(pysimFsNodePath(ch)); + if (p && !seen.has(p)) { + seen.add(p); + out.push(p); + } } if (ch.isDir && ch.exists !== false) walk(ch); } @@ -23648,86 +23727,111 @@ function servicesFsPresentFids(root) { return out; } -// File names per FID from the same tree, for the report: a FID may carry one -// name per DF (the collisions the note names), so the value is a list. -function servicesFsFidNames(root) { - const names = {}; - const walk = node => { - for (const ch of ((node && node.children) || [])) { - const fid = String(ch.fid || '').toUpperCase(); - if (/^[0-9A-F]{4}$/.test(fid) && ch.name) { - if (!names[fid]) names[fid] = []; - if (names[fid].indexOf(ch.name) < 0) names[fid].push(ch.name); - } - if (ch.isDir && ch.exists !== false) walk(ch); - } - }; - walk(root); - return names; -} - // The mismatch report, deterministic and DOM-free: -// missing: [{ feature, fids }] - declared features with absent files, in -// cross-reference order, `fids` in the map's (doc) order; -// extra: [{ fid, features }] - present FIDs no declared feature explains, -// sorted by FID, `features` = every feature mapping that FID (all -// of them undeclared). A FID that any declared feature maps is -// explained and not reported: several features legitimately share -// one file (4F01 under the 5G/ACDC/MCPTT DFs, 6F4A/6F4B across -// EXT1/EXT2 and BDN), and the FID-level rule cannot tell which DF -// the present file sits in. -function servicesFsReport(declared, presentFids) { - const declaredSet = {}; - for (const name of (declared || [])) declaredSet[name] = true; - const presentSet = {}; - for (const fid of (presentFids || [])) presentSet[String(fid).toUpperCase()] = true; - const missing = []; - const byFid = {}; - for (const f of SERVICES_CROSS_REF) { - const fids = SERVICES_FS_MAP[f.f] || []; - for (const fid of fids) { - if (!byFid[fid]) byFid[fid] = []; - byFid[fid].push(f.f); - } - if (!declaredSet[f.f]) continue; - const absent = fids.filter(fid => !presentSet[fid]); - if (absent.length) missing.push({ feature: f.f, fids: absent }); +// missing: [{ feature, tree, paths }] - a declared feature whose resolved +// paths are absent, in cross-reference order then +// SERVICES_FS_TREE_ORDER, `paths` in the map's (doc) role order; +// `tree` is the application whose service table declares it; +// extra: [{ path, role, tree, features }] - a present path whose mapped +// features are declared in none of the checked trees, sorted by +// path; `tree` is the path's own root and `features` every +// feature mapping that path (all of them undeclared). A path +// any declared feature maps is explained and not reported: several +// features share one file (the 5G DFs' 4F01/4F02, EXT2 across +// FDN/Bdn, the DF.TELECOM DFs a USIM service gates); +// unchecked: [{ feature, tree, roles }] - a declared feature whose roles +// resolve to no path in that tree at all (the standard list is +// missing the file: DF.GRAPHICS/EF.IMG, the DF.TELECOM copies of +// DF.SoLSA/DF.MExE, EF.3GPPPSDATAOFFservicelist), so its presence +// cannot be verified - reported instead of guessed. +// A role the list has for another root only is skipped, not a failure: a +// feature's files are the union of its applications' files (ADN's EF.ADN is +// the SIM's, its USIM copy is DF.PHONEBOOK + EF.PBR). +function servicesFsReport(declaredByTree, index, presentPaths) { + const declared = declaredByTree || {}; + const present = new Set((presentPaths || []).map(servicesFsNormPath)); + const trees = SERVICES_FS_TREE_ORDER.filter(root => Array.isArray(declared[root])); + const declaredIn = {}; + const allDeclared = new Set(); + for (const root of trees) { + declaredIn[root] = new Set(declared[root]); + for (const name of declared[root]) allDeclared.add(name); } - const extra = Object.keys(byFid).sort() - .filter(fid => presentSet[fid] && !byFid[fid].some(name => declaredSet[name])) - .map(fid => ({ fid: fid, features: byFid[fid] })); - return { missing: missing, extra: extra }; + const missing = []; + const unchecked = []; + const reverse = new Map(); + for (const f of SERVICES_CROSS_REF) { + const roles = SERVICES_FS_MAP[f.f] || []; + if (!roles.length) continue; + for (const root of trees) { + const paths = []; + for (const role of roles) { + const path = servicesFsRolePath(index, role, root); + if (!path) continue; + if (paths.indexOf(path) < 0) paths.push(path); + let rev = reverse.get(path); + if (!rev) { + rev = { path: path, role: role, tree: path.split('/')[0], features: [] }; + reverse.set(path, rev); + } + if (rev.features.indexOf(f.f) < 0) rev.features.push(f.f); + } + if (!declaredIn[root].has(f.f)) continue; + if (!paths.length) { + unchecked.push({ feature: f.f, tree: root, roles: roles.slice() }); + continue; + } + const absent = paths.filter(path => !present.has(path)); + if (absent.length) missing.push({ feature: f.f, tree: root, paths: absent }); + } + } + const extra = Array.from(reverse.values()) + .filter(e => present.has(e.path) && !e.features.some(name => allDeclared.has(name))) + .sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0)) + .map(e => ({ path: e.path, role: e.role, tree: e.tree, features: e.features.slice() })); + return { missing: missing, extra: extra, unchecked: unchecked }; } -function servicesFsReportHtml(report, names) { +function servicesFsReportHtml(report) { const note = '
' + - esc(t('The check is FID-level: a FID is counted anywhere in the card\'s tree, so the collisions across DFs (6F4E EXT4/EXT5, 6F4F ECCP/CCP2, 6F07 IMSI/IST, 6F38 SST/UST) are not disambiguated.')) + '
'; + esc(t('The check is path-exact: each service file is resolved to its full path in the standard file list, so a file with the same FID under another DF no longer explains it.')) + ''; const head = (text, n) => '
' + esc(text) + ' (' + n + ')
'; const item = text => '
  • ' + text + '
    • '; - // the tree's own name(s) for the FID, when the check loaded it - const fidLabel = fid => { - const list = names && names[fid]; - return '' + esc(fid) + '' + - ((list && list.length) ? ' (' + esc(list.join(' / ')) + ')' : ''); - }; - const fidList = fids => fids.map(fidLabel).join(', '); - if (!report || (!report.missing.length && !report.extra.length)) { + const pathLabel = path => '' + esc(path) + ''; + const missing = (report && report.missing) || []; + const extra = (report && report.extra) || []; + const unchecked = (report && report.unchecked) || []; + if (!missing.length && !extra.length && !unchecked.length) { return '
    ' + esc(t('The filesystem matches the declared services')) + '
    ' + note; } let html = ''; - if (report.missing.length) { - html += head(t('Declared services whose files are missing'), report.missing.length); + if (missing.length) { + html += head(t('Declared services whose files are missing'), missing.length); html += ''; } - if (report.extra.length) { - html += head(t('Present files whose service is not declared'), report.extra.length); + if (extra.length) { + html += head(t('Present files whose service is not declared'), extra.length); html += ''; } + if (unchecked.length) { + html += head(t('Declared services that cannot be checked'), unchecked.length); + html += ''; + html += '
    ' + + esc(t('The standard file list has no path for these files, so their presence cannot be verified.')) + '
    '; + } return html + note; } @@ -23737,10 +23841,9 @@ function servicesFsReportHtml(report, names) { let _pysimServicesCache = null; let _pysimServicesBusy = false; // The last filesystem-check report, kept so a re-render (language switch) can -// redraw it; cleared with the cache on a card-session change. The names map -// annotates the report's FIDs with the tree's file names. +// redraw it; cleared with the cache on a card-session change. It carries the +// resolved paths itself, so a redraw needs no card or file list. let _pysimServicesFsReport = null; -let _pysimServicesFsNames = null; let _pysimServicesFsBusy = false; async function pysimServicesReadFile(spec) { @@ -23786,7 +23889,6 @@ async function pysimServicesOpen(force) { function pysimServicesReset() { _pysimServicesCache = null; _pysimServicesFsReport = null; - _pysimServicesFsNames = null; pysimServicesRender(); if (isViewVisible('pysim-sub-services')) pysimServicesOpen(); } @@ -23794,14 +23896,16 @@ function pysimServicesReset() { function pysimServicesFsRender() { const el = document.getElementById('pysim-services-fs-report'); if (!el) return; - el.innerHTML = _pysimServicesFsReport ? servicesFsReportHtml(_pysimServicesFsReport, _pysimServicesFsNames) : ''; + el.innerHTML = _pysimServicesFsReport ? servicesFsReportHtml(_pysimServicesFsReport) : ''; } // "Check filesystem": compare the declared services with the card's probed -// tree. The file manager loads its tree lazily and only probes presence on -// demand, so the check runs the full probe once per tree (a loaded but -// unprobed tree claims every listed file as present - /api/tree does not test -// it). +// tree, path-exactly. The file manager loads its tree lazily and only probes +// presence on demand, so the check runs the full probe once per tree (a +// loaded but unprobed tree claims every listed file as present - /api/tree +// does not test it). The standard file list's (name, root) index turns each +// feature's symbolic files into paths the probe's results can be compared +// with. async function pysimServicesFsCheck() { const statusEl = document.getElementById('pysim-services-fs-status'); const reportEl = document.getElementById('pysim-services-fs-report'); @@ -23810,7 +23914,6 @@ async function pysimServicesFsCheck() { if (_pysimFsProbe) { setStatus(t('A file probe is already running')); return; } _pysimServicesFsBusy = true; _pysimServicesFsReport = null; - _pysimServicesFsNames = null; reportEl.innerHTML = ''; setStatus(t('Loading...')); try { @@ -23825,9 +23928,15 @@ async function pysimServicesFsCheck() { setStatus(t('Probing:')); await pysimFsProbeAll(); } + const index = await servicesPathIndex(); + if (!index.size) { + reportEl.innerHTML = '
    ' + + esc(t('The standard file list could not be loaded')) + '
    '; + return; + } _pysimServicesFsReport = servicesFsReport( - servicesFsDeclared(cache.files), servicesFsPresentFids(pysimFsTreeRoot)); - _pysimServicesFsNames = servicesFsFidNames(pysimFsTreeRoot); + servicesFsDeclaredTrees(cache.files), index, + servicesFsPresentPaths(pysimFsTreeRoot)); pysimServicesFsRender(); } catch (e) { reportEl.innerHTML = '
    ' + @@ -24973,12 +25082,14 @@ function efFidFromPath(path) { // The ADF roots a snapshot/profile path uses are the DF name (an AID) the card // reports in its FCI, e.g. A0000000871002FF49FF0589 - but the decoders key -// their DF-specific entries by the symbolic name. The two standard -// application RIDs (TS 101 220; ADF.USIM / ADF.ISIM per TS 31.102/31.103 §4) -// fold onto those names, whatever the AID's PIX is. +// their DF-specific entries by the symbolic name. The standard application +// RIDs (TS 101 220; ADF.USIM / ADF.ISIM per TS 31.102/31.103 §4, and the +// non-IMSI USIM of TS 31.102 Annex N, which repeats the ADF.USIM file set +// under its own root) fold onto those names, whatever the AID's PIX is. const EF_ADF_RIDS = [ ['A0000000871002', 'ADF.USIM'], ['A0000000871004', 'ADF.ISIM'], + ['A000000087100B', 'ADF.USIM-NON-IMSI'], ]; // The parent DF of a canonical path: "MF/7F20/6FAD" -> "7F20", @@ -27614,7 +27725,10 @@ const LANG_RU = { 'Present files whose service is not declared': 'Присутствующие файлы, сервис которых не заявлен', 'The filesystem matches the declared services': 'Файловая система соответствует заявленным сервисам', 'A file probe is already running': 'Опрос файлов уже выполняется', - 'The check is FID-level: a FID is counted anywhere in the card\'s tree, so the collisions across DFs (6F4E EXT4/EXT5, 6F4F ECCP/CCP2, 6F07 IMSI/IST, 6F38 SST/UST) are not disambiguated.': 'Проверка выполняется на уровне FID: FID учитывается в любом месте дерева карты, поэтому совпадения FID в разных DF (6F4E EXT4/EXT5, 6F4F ECCP/CCP2, 6F07 IMSI/IST, 6F38 SST/UST) не различаются.', + 'Declared services that cannot be checked': 'Заявленные сервисы, которые нельзя проверить', + 'The standard file list has no path for these files, so their presence cannot be verified.': 'В стандартном списке файлов нет пути для этих файлов, поэтому их наличие проверить нельзя.', + 'The standard file list could not be loaded': 'Не удалось загрузить стандартный список файлов', + 'The check is path-exact: each service file is resolved to its full path in the standard file list, so a file with the same FID under another DF no longer explains it.': 'Проверка точна по пути: каждый файл сервиса сопоставляется с полным путём в стандартном списке файлов, поэтому файл с тем же FID в другом DF больше не считается объяснением.', 'No service table found on this card': 'На этой карте не найдено ни одной таблицы сервисов', 'Stop': 'Остановить', 'Probing:': 'Проверка:', diff --git a/frontend/tests/ef_decode.test.js b/frontend/tests/ef_decode.test.js index c5ad951..c951ffe 100644 --- a/frontend/tests/ef_decode.test.js +++ b/frontend/tests/ef_decode.test.js @@ -400,6 +400,9 @@ test('efDfFromPath folds an ADF AID root onto its symbolic ADF name', () => { assert.strictEqual(efDfFromPath('a0000000871002ff49ff0589/6fad'), 'ADF.USIM'); assert.strictEqual(efDfFromPath('A0000000871004/6F07'), 'ADF.ISIM'); assert.strictEqual(efDfFromPath('A0000000871004FF49FF0589/6F07'), 'ADF.ISIM'); + // the non-IMSI USIM (TS 31.102 Annex N) repeats the ADF.USIM file set + assert.strictEqual(efDfFromPath('A000000087100B/6F38'), 'ADF.USIM-NON-IMSI'); + assert.strictEqual(efDfFromPath('a000000087100bff49ff0589/6f38'), 'ADF.USIM-NON-IMSI'); // an unrelated AID is not an ADF name: it passes through as itself assert.strictEqual(efDfFromPath('A0000000090003FFFF/6F07'), 'A0000000090003FFFF'); assert.strictEqual(efDfFromPath('F0414C46416001/6F07'), 'F0414C46416001'); diff --git a/frontend/tests/services.test.js b/frontend/tests/services.test.js index 7bdac43..6771b1d 100644 --- a/frontend/tests/services.test.js +++ b/frontend/tests/services.test.js @@ -45,14 +45,16 @@ const FNS = [ 'servicesSstAllocated', 'servicesSstPreserved', 'servicesBitsPreserved', 'servicesSstValues', 'servicesTableValues', 'servicesFileBytes', 'pysimServicesColumns', 'pysimServicesGridHtml', 'pysimServicesSaveRowHtml', 'pysimServicesCollect', - 'servicesFsDeclared', 'servicesFsPresentFids', 'servicesFsFidNames', 'servicesFsReport', - 'servicesFsReportHtml', + 'servicesFsDeclaredTrees', 'servicesFsPresentPaths', 'servicesFsReport', + 'servicesFsReportHtml', 'servicesFsPathKey', 'servicesFsNormPath', 'servicesFsRolePath', + 'servicesPathIndexBuild', 'pysimFsNodePath', ]; let code = 'var _pysimServicesCache = null;\nvar _pysimServicesBusy = false;\n'; for (const f of FNS) code += extractFunc(html, f) + '\n'; for (const c of ['EF_UST_SERVICES', 'EF_SST_SERVICES', 'EF_IST_SERVICES', 'EF_EST_SERVICES', - 'SERVICES_CROSS_REF', 'SERVICES_FILES', 'SERVICES_FS_MAP', 'SERVICES_TABLES']) { + 'SERVICES_CROSS_REF', 'SERVICES_FILES', 'SERVICES_FS_MAP', 'SERVICES_FS_TREES', + 'SERVICES_FS_TREE_ORDER', 'SERVICES_FS_GSM_MAPPED', 'SERVICES_TABLES']) { code += extractConst(html, c); } code += 'globalThis.esc = s => String(s);\nglobalThis.t = k => k;\n'; @@ -305,20 +307,24 @@ test('an absent file yields no bytes', () => { assert.strictEqual(servicesFileBytes('ust', {}, {}), null); }); -// ---- filesystem check: the feature -> FID table (docs/services-editor.md §B) ---- +// ---- filesystem check: the feature -> symbolic files table (docs/services-editor.md §B) ---- const UICC_FILES = JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'uicc_files.json'), 'utf8')); +const FS_INDEX = servicesPathIndexBuild(UICC_FILES.files); +const fsResolve = (role, root) => FS_INDEX.get(servicesFsPathKey(role, root)); -test('SERVICES_FS_MAP covers every feature exactly once and carries 4-hex FIDs', () => { +test('SERVICES_FS_MAP covers every feature exactly once and carries symbolic file names', () => { const names = SERVICES_CROSS_REF.map(f => f.f); assert.deepStrictEqual(Object.keys(SERVICES_FS_MAP), names, 'one entry per cross-reference row, in table order'); let backed = 0, capability = 0; - for (const [name, fids] of Object.entries(SERVICES_FS_MAP)) { - assert.ok(Array.isArray(fids), 'not an array: ' + name); - assert.strictEqual(new Set(fids).size, fids.length, 'duplicate FID: ' + name); - for (const fid of fids) assert.match(fid, /^[0-9A-F]{4}$/, name + ': ' + fid); - if (fids.length) backed++; else capability++; + for (const [name, roles] of Object.entries(SERVICES_FS_MAP)) { + assert.ok(Array.isArray(roles), 'not an array: ' + name); + assert.strictEqual(new Set(roles).size, roles.length, 'duplicate role: ' + name); + // the standard file list's own names, never FIDs: a FID is ambiguous + // across applications (4F30 = EF.PBR and EF.SAI) + for (const role of roles) assert.match(role, /^(EF|DF)\.[A-Za-z0-9_+.-]+$/, name + ': ' + role); + if (roles.length) backed++; else capability++; } // docs/services-editor.md §B.1 lists 34 `capability` cells; the remaining // 127 rows declare at least one file. @@ -339,31 +345,124 @@ test('the capability features carry no file check', () => { } }); -test('the well-known files map to their documented FIDs', () => { - assert.deepStrictEqual(SERVICES_FS_MAP['Fixed Dialling Numbers (FDN)'], ['6F3B', '6F4B']); - assert.deepStrictEqual(SERVICES_FS_MAP['Service Provider Name (SPN)'], ['6F46']); - assert.deepStrictEqual(SERVICES_FS_MAP['GSM Access'], ['5F3B']); - // ADN: the EF plus the USIM's DF.PHONEBOOK and EF.PBR - assert.ok(SERVICES_FS_MAP['Local Phone Book (ADN)'].includes('6F3A')); - assert.ok(SERVICES_FS_MAP['Short Message Storage (SMS)'].includes('6F3C')); - assert.deepStrictEqual(SERVICES_FS_MAP['Image (IMG)'], ['5F50', '4F20']); - assert.deepStrictEqual(SERVICES_FS_MAP['Advice of Charge (AoC)'], ['6F39', '6F37', '6F41']); - assert.deepStrictEqual(SERVICES_FS_MAP['Extension 4'], ['6F4E', '6F55'], - 'both trees\' FIDs, per the doc\'s "(SIM) / 6F55 (USIM)"'); +test('the well-known features map to their documented files', () => { + assert.deepStrictEqual(SERVICES_FS_MAP['Fixed Dialling Numbers (FDN)'], ['EF.FDN', 'EF.EXT2']); + assert.deepStrictEqual(SERVICES_FS_MAP['Service Provider Name (SPN)'], ['EF.SPN']); + assert.deepStrictEqual(SERVICES_FS_MAP['GSM Access'], ['DF.GSM-ACCESS']); + assert.deepStrictEqual(SERVICES_FS_MAP['Local Phone Book (ADN)'], ['EF.ADN', 'DF.PHONEBOOK', 'EF.PBR']); + assert.deepStrictEqual(SERVICES_FS_MAP['SoLSA'], ['DF.SoLSA', 'EF.SAI', 'EF.SLL']); + assert.deepStrictEqual(SERVICES_FS_MAP['MExE'], ['DF.MExE', 'EF.MExE-ST']); + assert.deepStrictEqual(SERVICES_FS_MAP['A2X'], ['DF.A2X', 'EF.AST', 'EF.A2X_CONFIG']); + assert.deepStrictEqual(SERVICES_FS_MAP['Advice of Charge (AoC)'], ['EF.ACM', 'EF.ACMmax', 'EF.PUCT']); + assert.deepStrictEqual(SERVICES_FS_MAP['Extension 4'], ['EF.EXT4'], + 'one name: the SIM and the USIM copy have different FIDs but the same name'); }); -test('every mapped FID is a known file (pySim\'s cardless list or the documented gap)', () => { - const known = new Set(UICC_FILES.files.map(e => e.fid)); - const mapped = new Set(); - for (const fids of Object.values(SERVICES_FS_MAP)) for (const fid of fids) mapped.add(fid); - // docs/services-editor.md §5: files UICC_FILES.md documents but pySim's - // model does not carry (the FS check probes the card, so they are not a - // blocker there). 6FDC/6FF0/6FF2 are also outside pySim's profile. - const gap = new Set(['4F31', '4F40', '5F3C', '5F3F', '5F70', '5F80', '5FA0', '5FB0', - '6FDC', '6FF0', '6FF2', '6FF9', '6FFB']); - assert.deepStrictEqual(Array.from(mapped).filter(fid => !known.has(fid)).sort(), - Array.from(gap).sort(), 'mapped FIDs outside the cardless list'); - assert.ok(mapped.size >= 150, 'distinct mapped FIDs: ' + mapped.size); +test('a role outside its feature\'s declaring tree lives in the shared DF.TELECOM subtree', () => { + // A UST/IST-gated DF may be a child of DF.TELECOM (TS 31.102 4.6.0), i.e. + // an MF path - the resolver's fallback. Anything else would be a mapping + // error, and the GSM application's mapped copies (EF.ADN/EF.EXT1/EF.ECCP) + // are deliberately not expected there. + const features = new Set(); + for (const f of SERVICES_CROSS_REF) { + const trees = []; + if (f.sst !== null) trees.push('MF'); + if (f.ust !== null || f.est !== null) trees.push('ADF.USIM'); + if (f.ist !== null) trees.push('ADF.ISIM'); + for (const role of (SERVICES_FS_MAP[f.f] || [])) { + if (trees.some(root => fsResolve(role, root))) continue; + const path = fsResolve(role, 'MF'); + if (!path) continue; // a registry gap, pinned above + assert.match(path, /^MF\/7F10\//, f.f + ': ' + role + ' -> ' + path); + assert.ok(SERVICES_FS_GSM_MAPPED.indexOf(role) < 0, f.f + ': ' + role); + features.add(f.f); + } + } + assert.deepStrictEqual(Array.from(features), + ['Multimedia Messages Storage', 'MCPTT', 'V2X', 'A2X']); +}); + +test('every mapped role is a standard file, except the documented registry gaps', () => { // The names that resolve in none of the checked trees: files the standard + // list does not carry yet (pySim omits DF.GRAPHICS and the DF.TELECOM + // copies of DF.SoLSA/DF.MExE; EF.3GPPPSDATAOFFservicelist is a pySim TODO). + // The report lists these as `unchecked` instead of guessing a path. + const gaps = { + 'Image (IMG)': ['DF.GRAPHICS', 'EF.IMG'], + '3GPP PS Data Off Service List': ['EF.3GPPPSDATAOFFservicelist'], + }; + const unresolved = {}; + for (const [name, roles] of Object.entries(SERVICES_FS_MAP)) { + for (const role of roles) { + if (SERVICES_FS_TREE_ORDER.some(root => fsResolve(role, root))) continue; + (unresolved[name] = unresolved[name] || []).push(role); + } + } + assert.deepStrictEqual(unresolved, gaps); + // a mapped role that does resolve resolves through exactly one list entry + for (const roles of Object.values(SERVICES_FS_MAP)) { + for (const role of roles) { + for (const root of SERVICES_FS_TREE_ORDER) { + const path = fsResolve(role, root); + if (!path) continue; + const entries = UICC_FILES.files.filter(f => f.name === role + && f.root.toUpperCase() === root && f.path.toUpperCase() === path); + assert.strictEqual(entries.length, 1, role + ' ' + root + ' -> ' + path); + } + } + } +}); + +// ---- filesystem check: the (name, root) -> path index ---- + +test('servicesPathIndexBuild keys the standard list by (name, root)', () => { + assert.ok(FS_INDEX.size >= 200, 'indexed files: ' + FS_INDEX.size); + // every value is one of the list's paths, and the list's roots are the keys + for (const [key, path] of FS_INDEX) { + const [name, root] = key.split('|'); + assert.ok(UICC_FILES.files.some(f => f.name.toUpperCase() === name && f.root.toUpperCase() === root + && f.path.toUpperCase() === path), key + ' -> ' + path); + } + assert.deepStrictEqual(servicesPathIndexBuild(null), new Map()); + assert.deepStrictEqual(servicesPathIndexBuild([null, {}, { name: 'EF.X' }]), new Map()); + // case-insensitive on both sides + assert.strictEqual(FS_INDEX.get(servicesFsPathKey('ef.spn', 'mf')), 'MF/7F20/6F46'); +}); + +test('the same FID under different names resolves to its own path', () => { + // EF.PBR and EF.SAI are both 4F30 + assert.strictEqual(fsResolve('EF.PBR', 'ADF.USIM'), 'ADF.USIM/5F3A/4F30'); + assert.strictEqual(fsResolve('EF.SAI', 'ADF.USIM'), 'ADF.USIM/5F70/4F30'); + // EF.PBR exists in both trees, under different parents + assert.strictEqual(fsResolve('EF.PBR', 'MF'), 'MF/7F10/5F3A/4F30'); + // a DF.TELECOM child resolves in the MF tree + assert.strictEqual(fsResolve('DF.A2X', 'MF'), 'MF/7F10/5F3F'); + assert.strictEqual(fsResolve('EF.A2X_CONFIG', 'MF'), 'MF/7F10/5F3F/4F02'); + // EF.SST and EF.UST are both 6F38 + assert.strictEqual(fsResolve('EF.UST', 'ADF.USIM'), 'ADF.USIM/6F38'); + assert.strictEqual(fsResolve('EF.IST', 'ADF.ISIM'), 'ADF.ISIM/6F07'); + // the one duplicate (name, root): pySim models EF.SUCI_Calc_Info twice + // under ADF.USIM (DF.5GS/4F07 - the UST 125 file - and the eSIM profile + // package's DF.SAIP/4F01); the list's first entry, DF.5GS, wins + const suci = UICC_FILES.files.filter(f => f.name === 'EF.SUCI_Calc_Info' && f.root === 'ADF.USIM'); + assert.strictEqual(suci.length, 2); + assert.strictEqual(fsResolve('EF.SUCI_Calc_Info', 'ADF.USIM'), 'ADF.USIM/5FC0/4F07'); +}); + +test('servicesFsRolePath falls back to the shared DF.TELECOM subtree', () => { + // DF.MCS is a child of DF.TELECOM (TS 31.102 4.6.0): a UST declaration + // resolves it in the MF tree + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'DF.MCS', 'ADF.USIM'), 'MF/7F10/5F3D'); + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'EF.MST', 'ADF.USIM'), 'MF/7F10/5F3D/4F01'); + // an own-tree path wins over the fallback + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'EF.FDN', 'ADF.USIM'), 'ADF.USIM/6F3B'); + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'EF.FDN', 'MF'), 'MF/7F10/6F3B'); + // the GSM application's mapped copies are not expected by a USIM + // declaration (TS 31.102 4.5) ... + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'EF.ADN', 'ADF.USIM'), undefined); + assert.ok(SERVICES_FS_GSM_MAPPED.indexOf('EF.ADN') >= 0); + // ... and a SIM (MF) declaration never falls back to an application + assert.strictEqual(servicesFsRolePath(FS_INDEX, 'DF.SoLSA', 'MF'), undefined); + assert.strictEqual(servicesFsRolePath(null, 'EF.SPN', 'MF'), undefined); }); // ---- filesystem check: the report ---- @@ -375,130 +474,188 @@ const svcFiles = (sst, ust, est, ist) => ({ ist: ist ? { present: true, dec: ist } : { present: false }, }); -test('servicesFsPresentFids collects the probed nodes\' FIDs, recursively', () => { - const root = { - children: [ - { name: 'MF/7F20', fid: '7f20', isDir: true, exists: true, children: [ - { name: 'EF.SPN', fid: '6f46', exists: true }, - { name: 'EF.LND', fid: '6F44', exists: false }, - { name: 'EF.ADN', fid: '6F3A', exists: true }, - ] }, - { name: 'EF.ICCID', fid: '2FE2', exists: null }, // never probed - { name: 'DF.GSM-ACCESS', fid: '5F3B', isDir: false, exists: true }, - { name: 'no fid', exists: true }, - { name: 'MF/2F06', fid: '2F06', isDir: true, exists: false, children: [ - { name: 'EF.SST', fid: '6F38', exists: true }, // under an absent DF - ] }, - ], +// A file-manager tree with parent links (pysimFsNodePath walks up them). +function fsTree(children) { + const mk = (spec, parent) => { + const node = { + name: spec.name, fid: spec.fid, isDir: !!spec.isDir, exists: spec.exists, + children: null, parent: parent, + }; + if (spec.children) node.children = spec.children.map(c => mk(c, node)); + return node; }; - assert.deepStrictEqual(servicesFsPresentFids(root), ['7F20', '6F46', '6F3A', '5F3B']); - assert.deepStrictEqual(servicesFsPresentFids(null), []); - // the same walk supplies the report's file names, one list per collided FID - assert.deepStrictEqual(servicesFsFidNames(root), { - '7F20': ['MF/7F20'], '6F46': ['EF.SPN'], '6F44': ['EF.LND'], '6F3A': ['EF.ADN'], - '2FE2': ['EF.ICCID'], '5F3B': ['DF.GSM-ACCESS'], '2F06': ['MF/2F06'], - }); - assert.deepStrictEqual(servicesFsFidNames(null), {}); + const root = { name: 'MF', path: 'MF', fid: '3F00', isDir: true, exists: true, parent: null, children: null }; + root.children = (children || []).map(c => mk(c, root)); + return root; +} + +test('servicesFsPresentPaths collects the probed nodes\' canonical paths', () => { + const root = fsTree([ + { name: 'DF.TELECOM', fid: '7F10', isDir: true, exists: true, children: [ + { name: 'EF.SMS', fid: '6f3c', exists: true }, + { name: 'EF.LND', fid: '6F44', exists: false }, + ] }, + { name: 'EF.ICCID', fid: '2FE2', exists: null }, // never probed + { name: 'ADF.USIM', isDir: true, exists: true, children: [ + { name: 'EF.UST', fid: '6F38', exists: true }, + ] }, + { name: 'DF.ABSENT', fid: '5F80', isDir: true, exists: false, children: [ + { name: 'EF.BCAST', fid: '4F01', exists: true }, // under an absent DF + ] }, + ]); + assert.deepStrictEqual(servicesFsPresentPaths(root), + ['MF/7F10', 'MF/7F10/6F3C', 'ADF.USIM', 'ADF.USIM/6F38']); + assert.deepStrictEqual(servicesFsPresentPaths(null), []); }); -test('servicesFsDeclared reads the union of the present service files', () => { +test('servicesFsDeclaredTrees groups the declared features by declaring application', () => { const files = svcFiles( sstDec({ 3: [1, 1], 4: [1, 0] }), // FDN on, SMS allocated-off bitDec([2, 10]), // UST FDN + SMS null, // no EF.EST on the card bitDec([6])); // IST SMS - const declared = servicesFsDeclared(files); - assert.ok(declared.includes('Fixed Dialling Numbers (FDN)'), declared.join(' | ')); - assert.ok(declared.includes('Short Message Storage (SMS)'), declared.join(' | ')); - // an allocated-off SST service declares nothing on its own - const off = servicesFsDeclared(svcFiles(sstDec({ 4: [1, 0] }), null, null, null)); - assert.deepStrictEqual(off, []); - // no files at all: nothing declared - assert.deepStrictEqual(servicesFsDeclared(svcFiles(null, null, null, null)), []); + const trees = servicesFsDeclaredTrees(files); + assert.deepStrictEqual(Object.keys(trees), ['MF', 'ADF.USIM', 'ADF.ISIM']); + assert.ok(trees.MF.includes('Fixed Dialling Numbers (FDN)'), trees.MF.join(' | ')); + assert.ok(!trees.MF.includes('Short Message Storage (SMS)'), 'an allocated-off SST service declares nothing'); + assert.ok(trees['ADF.USIM'].includes('Fixed Dialling Numbers (FDN)')); + assert.ok(trees['ADF.USIM'].includes('Short Message Storage (SMS)')); + assert.ok(trees['ADF.ISIM'].includes('Short Message Storage (SMS)')); + // the tree's own order: cross-reference order, deduplicated + assert.deepStrictEqual(trees['ADF.USIM'], ['Fixed Dialling Numbers (FDN)', 'Short Message Storage (SMS)']); + // a tree whose service table is absent is not checked at all + assert.deepStrictEqual(servicesFsDeclaredTrees(svcFiles(null, bitDec([2]), null, null)), + { 'ADF.USIM': ['Fixed Dialling Numbers (FDN)'] }); + assert.deepStrictEqual(servicesFsDeclaredTrees(svcFiles(null, null, null, null)), {}); }); -test('servicesFsReport: a declared feature with an absent file', () => { - const rep = servicesFsReport(['Fixed Dialling Numbers (FDN)'], ['6F3B']); - assert.deepStrictEqual(rep.missing, [{ feature: 'Fixed Dialling Numbers (FDN)', fids: ['6F4B'] }]); +const fsReport = (declaredByTree, present) => servicesFsReport(declaredByTree, FS_INDEX, present); + +test('servicesFsReport: a declared feature with an absent path', () => { + const rep = fsReport({ MF: ['Fixed Dialling Numbers (FDN)'] }, ['MF/7F10/6F3B']); + assert.deepStrictEqual(rep.missing, + [{ feature: 'Fixed Dialling Numbers (FDN)', tree: 'MF', paths: ['MF/7F10/6F4B'] }]); assert.deepStrictEqual(rep.extra, []); - // both files gone - assert.deepStrictEqual(servicesFsReport(['Fixed Dialling Numbers (FDN)'], []).missing, - [{ feature: 'Fixed Dialling Numbers (FDN)', fids: ['6F3B', '6F4B'] }]); + assert.deepStrictEqual(rep.unchecked, []); + // both files gone, in the map's (doc) role order + assert.deepStrictEqual(fsReport({ MF: ['Fixed Dialling Numbers (FDN)'] }, []).missing, + [{ feature: 'Fixed Dialling Numbers (FDN)', tree: 'MF', paths: ['MF/7F10/6F3B', 'MF/7F10/6F4B'] }]); }); -test('servicesFsReport: a present file whose service is undeclared', () => { - const rep = servicesFsReport([], ['6F3C']); +test('servicesFsReport: the same FID in another DF no longer explains a service', () => { + // EF.PBR and EF.SAI are both 4F30: a card holding the phonebook copy but + // no DF.SoLSA leaves SoLSA's paths missing + const rep = fsReport({ 'ADF.USIM': ['SoLSA'] }, ['ADF.USIM/5F3A', 'ADF.USIM/5F3A/4F30']); + assert.deepStrictEqual(rep.missing, [{ feature: 'SoLSA', tree: 'ADF.USIM', + paths: ['ADF.USIM/5F70', 'ADF.USIM/5F70/4F30', 'ADF.USIM/5F70/4F31'] }]); + // the phonebook paths are present but ADN is not declared + assert.deepStrictEqual(rep.extra.map(e => e.path), ['ADF.USIM/5F3A', 'ADF.USIM/5F3A/4F30']); + assert.deepStrictEqual(rep.extra[1], + { path: 'ADF.USIM/5F3A/4F30', role: 'EF.PBR', tree: 'ADF.USIM', features: ['Local Phone Book (ADN)'] }); + // with the SoLSA paths present the report is clean + const ok = fsReport({ 'ADF.USIM': ['SoLSA', 'Local Phone Book (ADN)'] }, + ['ADF.USIM/5F70', 'ADF.USIM/5F70/4F30', 'ADF.USIM/5F70/4F31', 'ADF.USIM/5F3A', 'ADF.USIM/5F3A/4F30']); + assert.deepStrictEqual(ok, { missing: [], extra: [], unchecked: [] }); +}); + +test('servicesFsReport: ADN\'s USIM files are the phonebook pair, not the GSM copy', () => { + // TS 31.102 4.5: EF.ADN under DF.TELECOM is the GSM application's mapping + const usim = fsReport({ 'ADF.USIM': ['Local Phone Book (ADN)'] }, ['ADF.USIM/5F3A', 'ADF.USIM/5F3A/4F30']); + assert.deepStrictEqual(usim, { missing: [], extra: [], unchecked: [] }); + // ... so a USIM card without the mapped 6F3A is not missing anything + assert.deepStrictEqual(fsReport({ 'ADF.USIM': ['Local Phone Book (ADN)'] }, ['ADF.USIM/5F3A/4F30']).missing, + [{ feature: 'Local Phone Book (ADN)', tree: 'ADF.USIM', paths: ['ADF.USIM/5F3A'] }]); + // the SIM declaration does expect it + const sim = fsReport({ MF: ['Local Phone Book (ADN)'] }, ['MF/7F10/5F3A', 'MF/7F10/5F3A/4F30']); + assert.deepStrictEqual(sim.missing, + [{ feature: 'Local Phone Book (ADN)', tree: 'MF', paths: ['MF/7F10/6F3A'] }]); + assert.deepStrictEqual(sim.extra, []); +}); + +test('servicesFsReport: a UST-declared DF.TELECOM file resolves in the MF tree', () => { + // MCPTT's DF.MCS is a child of DF.TELECOM (TS 31.102 4.6.0) + assert.deepStrictEqual(fsReport({ 'ADF.USIM': ['MCPTT'] }, ['MF/7F10/5F3D', 'MF/7F10/5F3D/4F01']), + { missing: [], extra: [], unchecked: [] }); + assert.deepStrictEqual(fsReport({ 'ADF.USIM': ['MCPTT'] }, []).missing, + [{ feature: 'MCPTT', tree: 'ADF.USIM', paths: ['MF/7F10/5F3D', 'MF/7F10/5F3D/4F01'] }]); +}); + +test('servicesFsReport: a declared feature the standard list cannot resolve is unchecked', () => { + const rep = fsReport({ 'ADF.USIM': ['Image (IMG)'] }, []); + assert.deepStrictEqual(rep, { missing: [], extra: [], unchecked: [ + { feature: 'Image (IMG)', tree: 'ADF.USIM', roles: ['DF.GRAPHICS', 'EF.IMG'] }] }); + // a capability row has no files and is never unchecked + assert.deepStrictEqual(fsReport({ MF: ['CHV1 disable function'] }, []), + { missing: [], extra: [], unchecked: [] }); + // a tree the card does not have declares nothing there + assert.deepStrictEqual(fsReport({}, []), { missing: [], extra: [], unchecked: [] }); +}); + +test('servicesFsReport: a shared file is explained by any declared feature', () => { + // DF.GSM-ACCESS is gated by both "GSM Access" and "GSM security context"; + // declaring either one explains the present path + const present = ['ADF.USIM/5F3B', 'ADF.USIM/5F3B/4F20', 'ADF.USIM/5F3B/4F52']; + assert.deepStrictEqual(fsReport({ 'ADF.USIM': ['GSM security context'] }, present), + { missing: [], extra: [], unchecked: [] }); + // with neither declared the path is an extra, naming every mapping feature + assert.deepStrictEqual(fsReport({ 'ADF.USIM': [] }, ['ADF.USIM/5F3B']).extra, + [{ path: 'ADF.USIM/5F3B', role: 'DF.GSM-ACCESS', tree: 'ADF.USIM', + features: ['GSM Access', 'GSM security context'] }]); + // a lower-case present path matches the list's upper-case one + assert.deepStrictEqual(fsReport({ MF: ['Service Provider Name (SPN)'] }, ['mf/7f20/6f46']), + { missing: [], extra: [], unchecked: [] }); +}); + +test('servicesFsReport: a present path whose service is undeclared', () => { + const rep = fsReport({ MF: [] }, ['MF/7F10/6F3C']); assert.deepStrictEqual(rep.missing, []); - assert.deepStrictEqual(rep.extra, [{ fid: '6F3C', features: ['Short Message Storage (SMS)'] }]); - // a FID several features map lists all of them, in table order - assert.deepStrictEqual(servicesFsReport([], ['4F02']).extra, - [{ fid: '4F02', features: ['V2X', '5GS Mobility Management Information', 'Network Identifier for SNPN (NID)'] }]); -}); - -test('servicesFsReport: an empty pair reports no mismatch', () => { - assert.deepStrictEqual(servicesFsReport([], []), { missing: [], extra: [] }); - // a declared feature whose files are all present is not a mismatch either - assert.deepStrictEqual(servicesFsReport(['Service Provider Name (SPN)'], ['6F46']), - { missing: [], extra: [] }); -}); - -test('servicesFsReport: a capability feature is never flagged', () => { - const rep = servicesFsReport(['CHV1 disable function', 'Proactive SIM', 'Menu selection'], []); - assert.deepStrictEqual(rep, { missing: [], extra: [] }); -}); - -test('servicesFsReport: a shared feature flags on its own mapped FIDs', () => { - // SMS is declared and present; ADN is declared, nothing of it is on the card - const rep = servicesFsReport(['Local Phone Book (ADN)', 'Short Message Storage (SMS)'], ['6F3C']); - assert.deepStrictEqual(rep.missing, [{ feature: 'Local Phone Book (ADN)', fids: ['6F3A', '5F3A', '4F30'] }]); - assert.deepStrictEqual(rep.extra, [], 'the present SMS file is explained by its declared service'); - // a present FID any declared feature maps is explained by it - assert.deepStrictEqual(servicesFsReport(['MCPTT'], ['4F01']), - { missing: [{ feature: 'MCPTT', fids: ['5F3D'] }], extra: [] }); - // 4F01 present while nothing that maps it is declared: one extra entry - assert.deepStrictEqual(servicesFsReport([], ['4F01']).extra[0].fid, '4F01'); - assert.ok(servicesFsReport([], ['4F01']).extra[0].features.includes('ACDC support')); + assert.deepStrictEqual(rep.extra, + [{ path: 'MF/7F10/6F3C', role: 'EF.SMS', tree: 'MF', features: ['Short Message Storage (SMS)'] }]); + // extra is sorted by path, and a path without a mapped feature is ignored + const extra = fsReport({ MF: ['Service Provider Name (SPN)'] }, + ['MF/7F20/6F45', 'MF/7F10/6F44', 'MF/7F10/6F3C', 'MF/7F20/6F46', 'MF/2F06']).extra; + assert.deepStrictEqual(extra.map(e => e.path), ['MF/7F10/6F3C', 'MF/7F10/6F44', 'MF/7F20/6F45']); + assert.deepStrictEqual(extra[2].features, ['Cell Broadcast Message Identifier (CBMI)']); }); test('servicesFsReport is deterministic and order-stable', () => { - const declared = ['Service Provider Name (SPN)', 'Fixed Dialling Numbers (FDN)']; - const present = ['6f3b', '6F46']; - const a = servicesFsReport(declared, present); - const b = servicesFsReport(declared.slice().reverse(), present); + const present = ['ADF.USIM/6F3B']; + const a = fsReport({ 'ADF.USIM': ['Service Provider Name (SPN)', 'Fixed Dialling Numbers (FDN)'] }, present); + const b = fsReport({ 'ADF.USIM': ['Fixed Dialling Numbers (FDN)', 'Service Provider Name (SPN)'] }, present); assert.deepStrictEqual(a, b, 'cross-reference order, not input order'); - assert.deepStrictEqual(a.missing, [{ feature: 'Fixed Dialling Numbers (FDN)', fids: ['6F4B'] }]); - // the lower-case present FIDs matched - assert.deepStrictEqual(a.extra, []); - // extra is sorted by FID - const extra = servicesFsReport(['Service Provider Name (SPN)'], ['6F81', '6F3C', '6F45']).extra; - assert.deepStrictEqual(extra.map(e => e.fid), ['6F3C', '6F45', '6F81']); + assert.deepStrictEqual(a.missing, [ + { feature: 'Fixed Dialling Numbers (FDN)', tree: 'ADF.USIM', paths: ['ADF.USIM/6F4B'] }, + { feature: 'Service Provider Name (SPN)', tree: 'ADF.USIM', paths: ['ADF.USIM/6F46'] }, + ]); }); -test('servicesFsReportHtml renders the two headings, the match line and the note', () => { - const note = 'FID-level'; - const empty = servicesFsReportHtml(servicesFsReport([], [])); +test('servicesFsReportHtml renders the paths and the notes', () => { + const note = 'path-exact'; + const empty = servicesFsReportHtml(fsReport({ MF: [] }, [])); assert.ok(empty.includes('The filesystem matches the declared services'), empty); assert.ok(empty.includes(note), empty); - const rep = servicesFsReport(['Fixed Dialling Numbers (FDN)'], ['6F3C']); + assert.ok(servicesFsReportHtml(null).includes('The filesystem matches the declared services')); + const rep = fsReport({ MF: ['Fixed Dialling Numbers (FDN)'] }, ['MF/7F10/6F3C']); const html = servicesFsReportHtml(rep); assert.ok(html.includes('Declared services whose files are missing'), html); assert.ok(html.includes('Fixed Dialling Numbers (FDN)'), html); - assert.ok(html.includes('6F4B'), html); + assert.ok(html.includes('MF/7F10/6F4B'), 'the missing path is shown'); assert.ok(html.includes('Present files whose service is not declared'), html); - assert.ok(html.includes('6F3C'), html); + assert.ok(html.includes('MF/7F10/6F3C'), 'the extra path is shown'); + assert.ok(html.includes('EF.SMS'), 'the extra path names its role'); assert.ok(html.includes('Short Message Storage (SMS)'), html); assert.ok(html.includes(note), html); assert.ok(!html.includes('The filesystem matches the declared services'), html); // a single-sided report renders only its own heading - const one = servicesFsReportHtml(servicesFsReport(['Fixed Dialling Numbers (FDN)'], ['6F3B'])); + const one = servicesFsReportHtml(fsReport({ MF: ['Fixed Dialling Numbers (FDN)'] }, ['MF/7F10/6F3B'])); assert.ok(one.includes('Declared services whose files are missing'), one); assert.ok(!one.includes('Present files whose service is not declared'), one); - // the tree's names annotate the FIDs when the check passes its names map - const named = servicesFsReportHtml(rep, { '6F3C': ['EF.SMS'], '6F4B': ['EF.EXT2'] }); - assert.ok(named.includes('6F4B') && named.includes('(EF.EXT2)'), named); - assert.ok(named.includes('6F3C') && named.includes('(EF.SMS)'), named); - // ... and a FID the map lacks stays bare - assert.ok(!named.includes('6F3A ('), named); + assert.ok(!one.includes('Declared services that cannot be checked'), one); + // the unchecked bucket names the roles and explains itself + const unc = servicesFsReportHtml(fsReport({ 'ADF.USIM': ['Image (IMG)'] }, [])); + assert.ok(unc.includes('Declared services that cannot be checked'), unc); + assert.ok(unc.includes('Image (IMG)') && unc.includes('DF.GRAPHICS') && unc.includes('EF.IMG'), unc); + assert.ok(unc.includes('cannot be verified'), unc); }); // ---- filesystem check: the wiring ---- @@ -518,11 +675,20 @@ test('the check probes the tree once per loaded session and keeps the report acr assert.match(html, /_pysimFsProbed = !probe\.stop && !error;/); assert.match(html, /async function pysimFsRefresh\(\) \{[\s\S]{0,200}_pysimFsProbed = false;/); // the check loads the tree if missing, probes when unprobed, and renders - // the pure report + // the pure report from the per-tree declarations, the standard list's path + // index and the tree's present paths const fn = extractFunc(html, 'pysimServicesFsCheck'); assert.ok(fn.indexOf('await pysimFsRefresh()') > 0, fn); assert.ok(fn.indexOf('await pysimFsProbeAll()') > 0, fn); + assert.ok(fn.indexOf('servicesFsDeclaredTrees(cache.files)') > 0, fn); + assert.ok(fn.indexOf('await servicesPathIndex()') > 0, fn); + assert.ok(fn.indexOf('index.size') > 0, 'an unloadable list is an error, not a wall of unchecked rows'); + assert.ok(fn.indexOf('servicesFsPresentPaths(pysimFsTreeRoot)') > 0, fn); assert.ok(fn.indexOf('servicesFsReport(') > 0, fn); + // the path index is loaded once from the standard file list + const loader = extractFunc(html, 'servicesPathIndex'); + assert.ok(loader.indexOf('uiccStdFilesLoad()') > 0, loader); + assert.ok(loader.indexOf('servicesPathIndexBuild(') > 0, loader); // the grid's render redraws the kept report (language switch) const render = extractFunc(html, 'pysimServicesRender'); assert.ok(render.indexOf('pysimServicesFsRender()') > 0, render);