Files view: name the phonebook files from the card's EF_PBR (v2.7.7)

- EF_PBR (TS 31.102 §4.4.2.1) is the card's declaration of its phonebook:
  the constructed tags A8/A9/AA announce the type-1/2/3 groups and each nested
  primitive tag C0…CC names the file kind plus its FID — and, in a three-byte
  object, its SFI (length 2 = FID only).  pySim has no parser for it (its
  EF_PBR class still carries a "# TODO: a80ac0034f3a…" stub), so the decoder
  follows the spec: new simtrace_analyser/pbr.py
- the tracker decodes every EF_PBR record it observes (a READ RECORD's data
  or an UPDATE RECORD's body) and uses it for the DF it belongs to: the
  phonebook files get their symbolic names (4FA1 = EF_PBC, 4F3A = EF_ADN,
  4F01 = EF_IAP, 4F4A = EF_EXT1, 4F3D = EF_CCP1, 4F61 = EF_EMAIL, …), an
  earlier placeholder name is upgraded when the PBR arrives later, and the
  declared SFIs feed the DF's SFI→FID map (so SFI-addressed reads resolve
  without waiting for an FCP)
- verified on the live sessions 47/193:
  DF_TELECOM/DF_PHONEBOOK/EF_PBC (4FA1) and the SFI map
  {2→4F3A, 4→4FA1, 6→4F4A, 7→4F4F}
- also fixed: the "SFI = FID's five low bits" fallback (TS 102 221
  §11.1.1.4.8) no longer applies to a DF/ADF's FCP, which has no SFI
- tests: tests/test_pbr.py (the two real records, SFI-less objects, padding,
  truncation, merging, the shape check) and tracker cases (names, path,
  SFI map, the later-PBR upgrade); 537 Python + 76 Node tests
- README/README_RUS document the PBR naming; PWA_VERSION/pyproject/
  version.py 2.7.7, SW cache simtrace-analyser-v87

README.md

@@ -228,6 +228,15 @@ and the bytes the operation first observed in another, the changed/new byte
 counts, the APDU reference and a colour legend below the views that lists only
 the marks actually shown.
 
+Phonebook files have issuer-assigned FIDs, so the dictionary cannot name
+them: when the trace carries the card's **EF_PBR** records, they are decoded
+(TS 31.102 §4.4.2.1 — the `A8`/`A9`/`AA` type groups and the `C0`…`CC`
+primitive tags, each with its FID and, for a three-byte object, its SFI) and
+the Files view names the files from the card's own declaration — e.g.
+`DF_TELECOM/DF_PHONEBOOK/EF_PBC (4FA1)` — while the declared SFIs feed the
+DF's SFI→FID map (the `88`-less FCP fallback of TS 102 221 §11.1.1.4.8 still
+applies for other files).
+
 The images come from `GET /api/sessions/<id>/files/image?key=…&change=…` and
 are reconstructed from the current image by undoing the later changes
 (`filetrack.FileTracker.change_images`) — nothing is duplicated per event.

README_RUS.md

@@ -233,6 +233,15 @@ Files — это таймлайн файловых операций по все
 счётчики изменённых/новых байт, ссылка на APDU и легенда цветов под
 представлениями, перечисляющая только реально показанные пометки.
 
+У файлов телефонной книги FID назначает карта, поэтому словарь их не
+знает: если в трассе есть записи **EF_PBR**, они разбираются
+(TS 31.102 §4.4.2.1 — группы типов `A8`/`A9`/`AA` и примитивные теги `C0`…`CC`
+с FID и, для трёхбайтового объекта, SFI), и Files называет файлы по
+объявлению самой карты — например
+`DF_TELECOM/DF_PHONEBOOK/EF_PBC (4FA1)` — а объявленные SFI пополняют карту
+SFI→FID этого DF (правило «SFI = 5 младших бит FID» из TS 102 221 §11.1.1.4.8
+для остальных файлов остаётся).
+
 Изображения отдаёт `GET /api/sessions/<id>/files/image?key=…&change=…`; они
 восстанавливаются из текущего образа отменой более поздних изменений
 (`filetrack.FileTracker.change_images`) — поэтому ничего не дублируется по

frontend/index.html

@@ -3386,7 +3386,7 @@ function onSearchChange() {
 }
 
 // ──────────────────── Versioning ────────────────────
-const PWA_VERSION = '2.7.6';
+const PWA_VERSION = '2.7.7';
 const MIN_SERVER_VERSION = '2.3.0';
 
 function parseVersion(v) {

frontend/sw.js

@@ -1,4 +1,4 @@
-const CACHE = 'simtrace-analyser-v86';
+const CACHE = 'simtrace-analyser-v87';
 
 const PRECACHE = [
     'index.html',

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.backends._legacy:_Backend"
 
 [project]
 name = "simtrace-analyser"
-version = "2.7.6"
+version = "2.7.7"
 description = "SIMtrace Analyser — APDU trace analyser for GSMTAP/PCAP captures (bundles the simtrace2-pysniff utility)"
 readme = {file = "README.md", content-type = "text/markdown"}
 license = {text = "GPL-2.0-or-later"}

simtrace_analyser/decode.py

@@ -1472,6 +1472,11 @@ def _decode_fcp(data):
         elif tag == 0xAB:
             result['security_attr_template'] = value.hex().upper()
     if not sfi_seen and fid:
+        # TS 102 221 §11.1.1.4.8: without the '88' DO the SFI is the file
+        # identifier's five least-significant bits — an EF rule; a DF/ADF's
+        # FCP has no SFI at all.
+        ft = (result.get('file_descriptor') or {}).get('file_type')
+        if ft != 'DF or ADF':
             sfi = int(fid, 16) & 0x1F
             result['sfi'] = sfi if 1 <= sfi <= 30 else None
     return result

simtrace_analyser/filetrack.py

@@ -42,6 +42,7 @@ from .database import _SUCCESS_SW1
 from .decode import (DF_CHILDREN, KNOWN_FIDS, _file_name, _is_df_fid,
                      select_target_fid, selected_df_fid, selected_df_state,
                      sfi_table)
+from .pbr import parse_pbr_record
 
 IMAGE_CAP = 64 * 1024      # observed bytes kept per file
 EVENTS_CAP = 1000          # change events kept per file
@@ -250,6 +251,7 @@ class FileTracker:
         self.files = {}            # key -> _FileEntry
         self.timeline = []         # one item per read/update/card change
         self.sfi_relations = {}    # df -> {sfi: fid}
+        self.pb_names = {}         # df -> {fid: EF_*} from the card's EF_PBR
         self.aid_labels = {}       # AID (upper hex) -> application label
         self._index = 0
         self._pending_read = {}    # channel -> {'d', 'df'} read awaiting its data
@@ -287,7 +289,8 @@ class FileTracker:
             # A file first seen via its FCP (or an operation whose decode had
             # no name) still gets its DF-scoped dictionary name.
             entry = _FileEntry(key, df, fid,
-                               (name or _file_name(fid, df)) if fid else '?',
+                               (name or self.pb_names.get(df, {}).get(fid)
+                                or _file_name(fid, df)) if fid else '?',
                                image_cap=self._image_cap, df_path=df_path)
             self.files[key] = entry
         elif df_path and len(df_path) > len(entry.df_path) and \
@@ -476,8 +479,35 @@ class FileTracker:
                        if idx is not None else None)
         else:
             self._push('read', entry, index, msg, record=record)
+        self._observe_pbr(df, fid, data)
         self._note_dir_content(d)
 
+    def _observe_pbr(self, df, fid, data):
+        """Learn the phonebook structure from an EF_PBR record.
+
+        The card declares each phonebook file's FID, its role (EF_ADN, EF_PBC,
+        …) and — when the object is three bytes long — its SFI
+        (TS 31.102 §4.4.2.1); the files then get their symbolic names and the
+        SFI relations resolve without waiting for an FCP.
+        """
+        if fid != '4f30' or not data:
+            return
+        parsed = parse_pbr_record(bytes(data))
+        if not parsed['groups']:
+            return
+        names = self.pb_names.setdefault(df, {})
+        for group in parsed['groups']:
+            for item in group['files']:
+                if 'fid' not in item:
+                    continue
+                names[item['fid']] = item['name']
+                if item.get('sfi'):
+                    self._learn_sfi(df, item['sfi'], item['fid'])
+                entry = self.files.get((df, item['fid']))
+                if entry is not None and entry.name in (entry.fid,
+                                                        entry.fid.upper()):
+                    entry.name = item['name']
+
     def _observe_write(self, d, df, index, msg, df_path=None):
         data = _body_bytes(d)
         if not data or not _success(d):
@@ -502,6 +532,7 @@ class FileTracker:
                 change_ref = {'file': _key_str(entry.key), 'idx': idx}
         self._push('update', entry, index, msg, record=record, change_ref=change_ref,
                    no_change=(not diffs and compared_all))
+        self._observe_pbr(df, fid, data)
 
     # -- helpers ---------------------------------------------------------
 

simtrace_analyser/pbr.py

@@ -0,0 +1,107 @@
+"""EF_PBR (Phone Book Reference file) record decoding — TS 31.102 §4.4.2.1.
+
+The phonebook's files (ADN, IAP, ANR, PBC, …) have issuer-assigned FIDs, so
+the static EF dictionary cannot name them.  The card declares them here:
+
+* the constructed tags 'A8', 'A9' and 'AA' announce a type-1/2/3 file group
+  (the first FID under 'A8' is the master EF, normally the ADN file);
+* each nested primitive tag ('C0' … 'CC') names the kind of file and carries
+  its FID and — when the object is three bytes long — its SFI
+  (length '02' = FID, length '03' = FID + SFI);
+* a record is padded with 'FF' after the last object.
+
+The decoder is deliberately tolerant: unknown primitives are reported raw and
+a truncated object ends the record (real cards pad and occasionally carry
+longer, path-shaped type-3 references).
+"""
+import collections
+
+# Table 4.2 — the primitive tags naming the phonebook files.
+PBR_NAMES = {
+    'C0': 'EF_ADN', 'C1': 'EF_IAP', 'C2': 'EF_EXT1', 'C3': 'EF_SNE',
+    'C4': 'EF_ANR', 'C5': 'EF_PBC', 'C6': 'EF_GRP', 'C7': 'EF_AAS',
+    'C8': 'EF_GAS', 'C9': 'EF_UID', 'CA': 'EF_EMAIL', 'CB': 'EF_CCP1',
+    'CC': 'EF_PURI',
+}
+# Table 4.1 — the constructed tags: the file-linking type.
+PBR_TYPES = {'A8': 1, 'A9': 2, 'AA': 3}
+
+
+def _objects(data):
+    """Iterate (tag, value) pairs; stop at the 'FF' padding or a truncation."""
+    i = 0
+    while i + 1 < len(data):
+        tag = data[i]
+        if tag == 0xFF:
+            return
+        length = data[i + 1]
+        j = i + 2
+        if length & 0x80:                       # BER long form (defensive)
+            octets = length & 0x7F
+            if j + octets > len(data):
+                return
+            length = int.from_bytes(data[j:j + octets], 'big')
+            j += octets
+        value = data[j:j + length]
+        if len(value) < length:
+            return
+        yield tag, value
+        i = j + length
+
+
+def _files(value):
+    out = []
+    for tag, v in _objects(value):
+        name = PBR_NAMES.get(f'{tag:02X}')
+        if name is None:
+            continue
+        if len(v) == 2:                         # FID only
+            item = {'tag': f'{tag:02X}', 'name': name, 'fid': v.hex()}
+        elif len(v) == 3:                       # FID + SFI
+            item = {'tag': f'{tag:02X}', 'name': name, 'fid': v[:2].hex(),
+                    'sfi': v[2]}
+        else:                                   # a path-shaped reference
+            item = {'tag': f'{tag:02X}', 'name': name, 'path': v.hex()}
+        out.append(item)
+    return out
+
+
+def parse_pbr_record(raw):
+    """Decode one EF_PBR record into its type groups (and the master EF)."""
+    if not raw:
+        return {'groups': [], 'master': None}
+    groups = []
+    master = None
+    for tag, value in _objects(raw):
+        kind = PBR_TYPES.get(f'{tag:02X}')
+        if kind is None:
+            continue
+        files = _files(value)
+        groups.append({'type': kind, 'tag': f'{tag:02X}', 'files': files})
+        if master is None and kind == 1 and files and 'fid' in files[0]:
+            master = files[0]['fid']
+    return {'groups': groups, 'master': master}
+
+
+def parse_pbr(records):
+    """Merge several EF_PBR records (the structure repeats per record)."""
+    out = collections.OrderedDict()
+    master = None
+    for raw in records or []:
+        parsed = parse_pbr_record(raw)
+        if master is None:
+            master = parsed['master']
+        for group in parsed['groups']:
+            for item in group['files']:
+                if 'fid' not in item:
+                    continue
+                prev = out.setdefault(item['fid'], dict(item))
+                prev.setdefault('types', [])
+                if group['type'] not in prev['types']:
+                    prev['types'].append(group['type'])
+    return {'files': out, 'master': master}
+
+
+def looks_like_pbr(raw):
+    """Cheap shape check: the record starts with a constructed A8/A9/AA tag."""
+    return bool(raw) and f'{raw[0]:02X}' in PBR_TYPES and len(raw) >= 4

simtrace_analyser/version.py

@@ -1,3 +1,3 @@
 """Single source of truth for the SIMtrace Analyser version."""
 
-__version__ = '2.7.6'
+__version__ = '2.7.7'

tests/test_decode.py

@@ -323,10 +323,14 @@ class TestFcpResponse(unittest.TestCase):
         self.assertIsNone(fcp['sfi'])
 
     def test_fcp_sfi_absent_uses_fid_lsbs(self):
-        # No SFI tag → SFI = 5 LSBs of the FID (TS 102 221 §11.1.2).
+        # No SFI tag → SFI = 5 LSBs of the FID (TS 102 221 §11.1.1.4.8).
         from simtrace_analyser.decode import _decode_fcp
         fcp = _decode_fcp(bytes.fromhex('8202412183022f06'))
         self.assertEqual(fcp['sfi'], 0x06)
+        # …but only for an EF: a DF/ADF's FCP carries no SFI.
+        df = _decode_fcp(bytes.fromhex('82017883025f3a'))
+        self.assertEqual(df['file_descriptor']['file_type'], 'DF or ADF')
+        self.assertNotIn('sfi', df)
 
     def test_fcp_summary_transparent(self):
         # GET RESPONSE (FCP) for a transparent EF of 9 bytes

tests/test_filetrack.py

@@ -560,6 +560,37 @@ class TestDfChain(TrackerCase):
         self.assertEqual(entry['n_records'], 100)
 
 
+class TestPbrNames(TrackerCase):
+    def test_pbr_names_the_phonebook_files(self):
+        # The card's EF_PBR declares the phonebook files' FIDs, roles and SFIs
+        # (TS 31.102 §4.4.2.1): 4FA1 = EF_PBC with SFI 4.
+        pbr_rec = 'a80ac0034f3a02c5034fa104aa0ac2034f4a06cb034f4f07'
+        data = self.track(
+            sel_path('7f105f3a'), gr(fcp('5f3a')),
+            sel_fid('4f30'), gr(fcp('4f30', rec_len=24, n_rec=1)),
+            read_record(1, pbr_rec),
+            sel_fid('4fa1'), gr(fcp('4fa1', rec_len=44, n_rec=100)),
+        )
+        entry = data['files']['5f3a/4fa1']
+        self.assertEqual(entry['name'], 'EF_PBC')
+        self.assertEqual(entry['path'],
+                         'DF_TELECOM/DF_PHONEBOOK/EF_PBC (4FA1)')
+        # The PBR's SFI feeds the DF's SFI relation map.
+        self.assertEqual(data['sfi_relations'].get('5f3a', {}).get('4'), '4fa1')
+
+    def test_pbr_name_upgrades_an_earlier_placeholder(self):
+        # A file seen before its PBR record is named once the record arrives.
+        pbr_rec = 'a80ac0034f3a02c5034fa104aa0ac2034f4a06cb034f4f07'
+        data = self.track(
+            sel_path('7f105f3a'), gr(fcp('5f3a')),
+            sel_fid('4fa1'), gr(fcp('4fa1', rec_len=44, n_rec=100)),
+            sel_fid('4f30'), gr(fcp('4f30', rec_len=24, n_rec=1)),
+            read_record(1, pbr_rec),
+        )
+        entry = data['files']['5f3a/4fa1']
+        self.assertEqual(entry['name'], 'EF_PBC')
+
+
 class TestFilePath(unittest.TestCase):
     def test_paths(self):
         from simtrace_analyser.filetrack import _file_path

tests/test_pbr.py

@@ -0,0 +1,78 @@
+"""EF_PBR record decoding (TS 31.102 §4.4.2.1)."""
+
+import unittest
+
+from simtrace_analyser.pbr import (looks_like_pbr, parse_pbr,
+                                   parse_pbr_record)
+
+# The records seen in the local test cards' traces.
+REC_SESSION_47 = bytes.fromhex(
+    'a80fc0034f3a04c1034f0108c5034fa11ea905ca034f6113'
+    'aa0ac2034f4a06cb034f3d12' + 'ff' * 12)
+REC_SESSION_193 = bytes.fromhex(
+    'a80ac0034f3a02c5034fa104' 'aa0ac2034f4a06cb034f4f07')
+
+
+class TestPbrRecord(unittest.TestCase):
+    def test_session_47_record(self):
+        # A8 = type 1 (ADN 4F3A/SFI 4, IAP 4F01/SFI 8, PBC 4FA1/SFI 30);
+        # A9 = type 2 (EMAIL 4F61/SFI 19); AA = type 3 (EXT1 4F4A/SFI 6,
+        # CCP1 4F3D/SFI 18).  The FF padding ends the record.
+        parsed = parse_pbr_record(REC_SESSION_47)
+        self.assertEqual(parsed['master'], '4f3a')
+        types = [g['type'] for g in parsed['groups']]
+        self.assertEqual(types, [1, 2, 3])
+        type1 = {f['tag']: (f['fid'], f.get('sfi'))
+                 for f in parsed['groups'][0]['files']}
+        self.assertEqual(type1['C0'], ('4f3a', 4))
+        self.assertEqual(type1['C1'], ('4f01', 8))
+        self.assertEqual(type1['C5'], ('4fa1', 30))
+        self.assertEqual(parsed['groups'][1]['files'][0]['name'], 'EF_EMAIL')
+        self.assertEqual(parsed['groups'][2]['files'][0]['name'], 'EF_EXT1')
+
+    def test_session_193_record(self):
+        parsed = parse_pbr_record(REC_SESSION_193)
+        self.assertEqual(parsed['master'], '4f3a')
+        files = {f['fid']: f for g in parsed['groups'] for f in g['files']}
+        self.assertEqual(files['4fa1']['name'], 'EF_PBC')
+        self.assertEqual(files['4fa1']['sfi'], 4)
+        self.assertEqual(files['4f4a']['name'], 'EF_EXT1')
+        self.assertEqual(files['4f4f']['name'], 'EF_CCP1')
+
+    def test_objects_without_an_sfi(self):
+        # Length 2 = FID only (TS 31.102 §4.4.2.1): no SFI is declared.
+        parsed = parse_pbr_record(bytes.fromhex('a808c0024f3ac5024fa1'))
+        files = parsed['groups'][0]['files']
+        self.assertEqual([(f['fid'], f.get('sfi')) for f in files],
+                         [('4f3a', None), ('4fa1', None)])
+
+    def test_truncated_and_unknown(self):
+        # A truncated record ends parsing instead of inventing objects.
+        self.assertEqual(parse_pbr_record(bytes.fromhex('a80ac0034f3a')),
+                         {'groups': [], 'master': None})
+        self.assertEqual(parse_pbr_record(bytes.fromhex('a804c0034f3a'))['groups'],
+                         [{'type': 1, 'tag': 'A8', 'files': []}])
+        # Unknown primitive tags are skipped; 'FF' padding ends the record.
+        parsed = parse_pbr_record(bytes.fromhex('a809c0034f3a04d102abcd'))
+        self.assertEqual([f['fid'] for f in parsed['groups'][0]['files']],
+                         ['4f3a'])
+        self.assertEqual(parse_pbr_record(b'\xff' * 8)['groups'], [])
+        self.assertEqual(parse_pbr_record(b''),
+                         {'groups': [], 'master': None})
+
+    def test_parse_pbr_merges_records(self):
+        parsed = parse_pbr([REC_SESSION_47, REC_SESSION_193])
+        self.assertEqual(parsed['master'], '4f3a')
+        self.assertEqual(parsed['files']['4fa1']['name'], 'EF_PBC')
+        self.assertEqual(parsed['files']['4f01']['name'], 'EF_IAP')
+        # A later record may add files; the first declaration is kept.
+        self.assertEqual(parsed['files']['4fa1']['sfi'], 30)
+
+    def test_shape_check(self):
+        self.assertTrue(looks_like_pbr(REC_SESSION_193))
+        self.assertFalse(looks_like_pbr(b''))
+        self.assertFalse(looks_like_pbr(bytes.fromhex('c0034f3a')))
+
+
+if __name__ == '__main__':
+    unittest.main()

tests/test_pcap_import.py

@@ -224,7 +224,7 @@ class TestImportEndpoint(unittest.TestCase):
 class TestVersion(unittest.TestCase):
     def test_package_version(self):
         from simtrace2_pysniff import __version__
-        self.assertEqual(__version__, '2.7.6')
+        self.assertEqual(__version__, '2.7.7')
 
     def test_status_includes_version(self):
         import os