diff --git a/AGENTS.md b/AGENTS.md index 14a5941..82db650 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -173,8 +173,9 @@ The edge-read path had two bugs: Rejected: `0x20`-`0x3F` (ISO-reserved); `0x60`-`0x6F`/`0x70`-`0x7F` (the SM-indicating extended classes -- `0x60`-`0x6F` overlaps printable ASCII and the text byte `0x61` started a bogus frame on xiaomi_mi_a1_coldboot -when it was accepted); `0x81`-`0x8F`/`0xA1`-`0xAF` (the `'8X'`/`'AX'` -standard-channel variants for channels 0-3 -- the FCP byte `0xA5` did the +when it was accepted); `0x81`-`0x83`/`0x85`-`0x8F`/`0xA1`-`0xAF` (the +`'8X'`/`'AX'` standard-channel variants for channels 0-3 other than the +legacy `0x80`/`0x84`/`0xA0` -- the FCP byte `0xA5` did the same); `0x90`-`0x9F`, `0xD0`-`0xDF`, `0xF0`-`0xFF`. They are spec-coded but cost frame-start precision; add any of them back when a real trace requires it. @@ -347,7 +348,8 @@ the empirical precision of the mis-frame filter preserved. channel 4 = `0xC0`) and higher channels in the no-SM `'4X'` form decode. Rejected after testing: `0x60`-`0x6F`/`0x70`-`0x7F` (the SM-indicating interindustry classes) and the `'8X'`/`'AX'` channel variants - `0x81`-`0x8F`/`0xA1`-`0xAF`. They are spec-coded, but they cost + `0x81`-`0x83`/`0x85`-`0x8F`/`0xA1`-`0xAF`. They are spec-coded, but + they cost frame-start precision on the mis-framed traces: `0x60`-`0x6F` overlaps printable ASCII, and the menu-text byte `0x61` ('a') started a bogus frame on xiaomi_mi_a1_coldboot whose P3 swallowed the following exchanges @@ -365,21 +367,23 @@ the empirical precision of the mis-frame filter preserved. now (the duplicate inline condition is gone) and the stale ISO `§12.1.1` citation was corrected to `§5.1.2`. - **Idle-noise gate.** The 6+ set-bits discard runs before the CLA check, - so it would swallow the legitimately bit-dense extended classes - (`0xCF`/`0xEB`-`0xEF`) and kept `0x5F` dead (`plausible_cla()` accepted + so it would swallow the legitimately bit-dense classes (`0xCF`, `0xE7`, + `0xEB`, `0xED`-`0xEF`) and kept `0x5F` dead (`plausible_cla()` accepted it, the gate always discarded it). The gate is now `is_idle_noise_byte()`: only bytes no spec allows are idle noise; valid CLAs take the normal header path. - **CONCAT detector unchanged by design.** `EMBEDDED_CLA` no longer derives from `plausible_cla()`; it is pinned to the empirically tuned 36 values (`0x00`-`0x1F` + `0x80`/`0x84`/`0xA0`/`0xB0`) in both `pd.py` and - `tools/vs_reader.py`, with a membership-pin test -- the CONCAT scan runs + `tools/vs_reader.py`, with a membership-pin test and a `pd.py` ↔ + `vs_reader.py` mirror test -- the CONCAT scan runs across payload bytes where the extended classes false-hit (the Alcor `"40 70 70 61 15"` run). - **Tests.** CLA accept/reject vectors (including the two regressing bytes `0x61`/`0xA5` and the reserved ranges), extended-channel `validate_t0_apdu`/`is_desynced` vectors, the odd-INS vectors, the - idle-noise vectors and the EMBEDDED_CLA pin (75 -> 82 tests); + idle-noise vectors, the EMBEDDED_CLA membership pin, the `ODD_INS` set + pin and the `pd.py` ↔ `vs_reader.py` mirror test (75 -> 84 tests); `tools/vs_reader.py` mirrors the predicate, `ODD_INS` and the pinned set. Validation: all 10 traces identical to the v1.9.2 baseline (APDU counts, @@ -764,7 +768,8 @@ Accepted (v1.9.3): Rejected: `0x20`-`0x3F` (ISO-reserved); `0x60`-`0x6F`/`0x70`-`0x7F` (SM in b6 — `0x60`-`0x6F` overlaps printable ASCII, and text bytes started bogus frames on the mis-framed phone traces); the `'8X'`/`'AX'` standard-channel -variants `0x81`-`0x8F`/`0xA1`-`0xAF` (channels 0–3; same trade-off); +variants `0x81`-`0x83`/`0x85`-`0x8F`/`0xA1`-`0xAF` (channels 0–3; same +trade-off); `0x90`-`0x9F`, `0xD0`-`0xDF`, `0xF0`-`0xFF`. Examples: `0x00` (standard), `0x01` (channel 1), `0x44` (channel 8), diff --git a/README_RUS.md b/README_RUS.md index b74a485..93fd7a7 100644 --- a/README_RUS.md +++ b/README_RUS.md @@ -232,8 +232,8 @@ USB у FX2 при длительной высокой частоте дискр - Десинхронизированные TPDU (обмены с ошибкой кадрирования) сохраняются, но помечаются `GSMTAP_FLAG_BAD_FCS` в байте `res` заголовка GSMTAP — видно в Wireshark как предупреждение BAD-FCS. -- Заголовок T=0 со структурно некорректным INS (ISO 7816-4 §12.1.1: - `0x00`, `0x6x`/`0x9x` или нечётный младший бит) почти всегда сдвинут на +- Заголовок T=0 со структурно некорректным INS (ISO 7816-4 §5.1.2: + `0x00` или `0x6x`/`0x9x`) почти всегда сдвинут на один байт из-за лишнего ведущего байта (например, дочитанного остатка SW2 после PACKETEND). Вместо молчаливого отбрасывания всего 5-байтового окна (из-за чего следующий байт становился фиктивным CLA и проглатывал diff --git a/pd.py b/pd.py index a469183..06e3a5e 100644 --- a/pd.py +++ b/pd.py @@ -127,9 +127,11 @@ def plausible_cla(cla): 0x00-0x1F first interindustry: logical channels 0-3 in b2b1, secure messaging in b4b3, chaining in b5 [ISO 7816-4 §5.1.1 Table 2]. - 0x40-0x5F further interindustry: logical channels 4-19 (b4..b1 - encode the channel number minus four), with the legacy - chaining variants [Table 3; TS 102 221 Table 10.4a]. + 0x40-0x4F further interindustry: logical channels 4-19 (b4..b1 + encode the channel number minus four) + [ISO Table 3; TS 102 221 Table 10.4a]. + 0x50-0x5F the legacy command-chaining variants of 0x40-0x4F + [ISO Table 3 only: TS 102 221 keeps b5 = 0]. 0x80/0x84/0xA0/0xB0 SIM/UICC classes (0xB0 has no pinned-spec basis; kept for backward compatibility). 0xC0-0xCF/0xE0-0xEF the UICC extended classes 'CX'/'EX': logical @@ -146,8 +148,9 @@ def plausible_cla(cla): Extended channels without CLA-level SM ('4X') and the 'CX'/'EX' classes are accepted instead. 0x70-0x7F chaining + secure messaging (not used by UICC: b5=0). - 0x81-0x8F/0xA1-0xAF the '8X'/'AX' standard-channel variants - (channels 0-3): no trace uses them, and the same + 0x81-0x83/0x85-0x8F/0xA1-0xAF the '8X'/'AX' standard-channel + variants (channels 0-3) other than the legacy + 0x80/0x84/0xA0: no trace uses them, and the same precision trade-off applies (the FCP byte 0xA5 on xiaomi_mi_a1_coldboot). 0x90-0x9F, 0xD0-0xDF, 0xF0-0xFF not SIM/UICC classes. @@ -198,10 +201,9 @@ def is_idle_noise_byte(first_byte): command class byte. Idle-line noise is an all-1s byte with a few dropped bits, so 6 or - more set bits. A spec-valid CLA can be bit-dense as well (extended - logical channels: 0x5F, 0xCF, 0xEB-0xEF), so plausibility wins over - density: such a byte takes the normal CLA/header path instead of the - noise resync.''' + more set bits. Valid CLAs can be bit-dense as well (0x5F, 0xCF, + 0xE7, 0xEB, 0xED-0xEF), so plausibility wins over density: such a + byte takes the normal CLA/header path instead of the noise resync.''' return bin(first_byte).count('1') >= 6 and not plausible_cla(first_byte) @@ -288,8 +290,9 @@ def embedded_exchanges(buf): embedded inside a packet's data region (index 5 onward). Pattern: header (CLA INS P1 P2 P3) + P3 data bytes + interior SW1/SW2 - before the packet's own final status word. CLA must be structurally - valid (same rule as plausible_cla); INS must be a real command byte; + before the packet's own final status word. CLA must be one of the + pinned EMBEDDED_CLA classes (deliberately narrower than + plausible_cla()); INS must be a real command byte; SW1 is accepted within a small slack window after the nominal data end and must be interior (trailing bytes follow it).''' n = len(buf) @@ -2300,10 +2303,10 @@ class Decoder(srd.Decoder): elif is_idle_noise_byte(firstByte): # Idle-line noise: an all-1s byte with a few dropped bits. # Re-sync instead of mis-parsing as a T=0 header (which - # would pack idle noise into garbage APDUs). Spec-valid - # high-density CLAs (0x5F/0xCF/0xEB-0xEF: extended logical - # channels) are not noise and fall through to the - # CLA/header path below. + # would pack idle noise into garbage APDUs). Valid + # bit-dense CLAs (0x5F/0xCF/0xE7/0xEB/0xED-0xEF: + # extended logical channels) are not noise and fall + # through to the CLA/header path below. self._note_discard(firstByte, 'idle-line noise (6+ bits set)', idle=True) self._consume_first() self._resync = True diff --git a/tests/baseline.txt b/tests/baseline.txt index 029d53d..6a7f09a 100644 --- a/tests/baseline.txt +++ b/tests/baseline.txt @@ -1,4 +1,4 @@ -=== Baseline captured 2026-10-17T23:18:56+03:00 === +=== Baseline captured 2026-10-17T23:51:01+03:00 === === test_8 === ('=== iso7816 decoder v1.9.3 ===',) @@ -411,6 +411,7 @@ test_invalid_ins_zero (__main__.TestValidateT0Apdu.test_invalid_ins_zero) ... ok test_invalid_oversized (__main__.TestValidateT0Apdu.test_invalid_oversized) ... ok test_invalid_sw1 (__main__.TestValidateT0Apdu.test_invalid_sw1) ... ok test_invalid_too_short (__main__.TestValidateT0Apdu.test_invalid_too_short) ... ok +test_odd_ins_set_pinned (__main__.TestValidateT0Apdu.test_odd_ins_set_pinned) ... ok test_valid_case1_direct_sw (__main__.TestValidateT0Apdu.test_valid_case1_direct_sw) ... ok test_valid_case2_with_response (__main__.TestValidateT0Apdu.test_valid_case2_with_response) ... ok test_valid_case3_with_command_data (__main__.TestValidateT0Apdu.test_valid_case3_with_command_data) ... ok @@ -420,8 +421,9 @@ test_valid_cla_interindustry_classes (__main__.TestValidateT0Apdu.test_valid_cla test_valid_cla_uicc_classes (__main__.TestValidateT0Apdu.test_valid_cla_uicc_classes) ... ok test_valid_direct_sw_with_p3_nonzero (__main__.TestValidateT0Apdu.test_valid_direct_sw_with_p3_nonzero) ... ok test_valid_ins_odd_variants (__main__.TestValidateT0Apdu.test_valid_ins_odd_variants) ... ok +test_vs_reader_mirror (__main__.TestValidateT0Apdu.test_vs_reader_mirror) ... ok ---------------------------------------------------------------------- -Ran 82 tests in 0.023s +Ran 84 tests in 0.025s OK diff --git a/tests/test_gsmtap.py b/tests/test_gsmtap.py index d8bd6bc..aab3ed6 100644 --- a/tests/test_gsmtap.py +++ b/tests/test_gsmtap.py @@ -102,9 +102,9 @@ class TestDesyncHelpers(unittest.TestCase): # v1.9.3: higher logical channels and the UICC extended classes # are clean traffic, not desync. 0x44 = SELECT on channel 8, # 0x4F = SELECT on channel 19, 0xC0 = STATUS on channel 4 - # ('CX'), 0xE2 = READ BINARY on channel 6 with SM ('EX'). + # ('CX'), 0xE2 = STATUS on channel 6 with SM ('EX'). for h in ('44a40004009000', '4fa40004009000', - 'c0f20000009000', 'e2b000000212349000'): + 'c0f20000009000', 'e2f20000009000'): with self.subTest(h=h): self.assertFalse(self.pd.is_desynced(bytes.fromhex(h))) @@ -114,10 +114,9 @@ class TestDesyncHelpers(unittest.TestCase): for b in (0x6f, 0x7f, 0xaf, 0xbf, 0xf7, 0xfc, 0xfd, 0xfe): with self.subTest(b=b): self.assertTrue(self.pd.is_idle_noise_byte(b)) - # ... but spec-valid, bit-dense CLAs are command bytes: extended - # logical channels 0x5F/0xCF/0xEB-0xEF must not be swallowed by - # the noise resync. - for b in (0x5f, 0xcf, 0xeb, 0xee, 0xef): + # ... but valid, bit-dense CLAs are command bytes: 0x5F/0xCF/ + # 0xE7/0xEB/0xED-0xEF must not be swallowed by the noise resync. + for b in (0x5f, 0xcf, 0xe7, 0xeb, 0xed, 0xee, 0xef): with self.subTest(b=b): self.assertFalse(self.pd.is_idle_noise_byte(b)) # A sparse invalid class is not "idle noise" either (it takes the @@ -210,12 +209,12 @@ class TestValidateT0Apdu(unittest.TestCase): def test_valid_cla_extended_channel_exchanges(self): # Real higher-channel shapes from TS 102 221 Tables 10.4a/10.5: # SELECT on channel 8 ('4X'), SELECT on channel 19 ('4X'), - # STATUS on channel 4 ('CX'), READ BINARY on channel 6 with SM + # STATUS on channel 4 ('CX'), STATUS on channel 6 with SM # ('EX' -- the UICC extended class accepts b6 = SM). self._valid(b'\x44\xa4\x00\x04\x00\x90\x00') self._valid(b'\x4f\xa4\x00\x04\x00\x90\x00') self._valid(b'\xc0\xf2\x00\x00\x00\x90\x00') - self._valid(b'\xe2\xb0\x00\x00\x02\x12\x34\x90\x00') + self._valid(b'\xe2\xf2\x00\x00\x00\x90\x00') def test_valid_cla_channel1_select_after_manage_channel(self): # The command that follows MANAGE CHANNEL (00 70 00 00 01) opening @@ -239,15 +238,41 @@ class TestValidateT0Apdu(unittest.TestCase): # cost frame-start precision stay rejected. 0x60-0x6F/0x70-0x7F # are the SM-indicating extended classes -- 0x60-0x6F overlaps # printable ASCII ('a'-'o'), and the text byte 0x61 started a bogus - # frame on xiaomi_mi_a1_coldboot; 0x81-0x8F/0xA1-0xAF are the - # '8X'/'AX' standard-channel variants (channels 0-3), where the - # FCP byte 0xA5 did the same. + # frame on xiaomi_mi_a1_coldboot; 0x81-0x83/0x85-0x8F/0xA1-0xAF + # are the '8X'/'AX' standard-channel variants (channels 0-3) other + # than the legacy 0x80/0x84/0xA0, where the FCP byte 0xA5 did the + # same. for cla in (0x60, 0x61, 0x6F, 0x70, 0x73, 0x7F, 0x81, 0x85, 0x8F, 0xA1, 0xA5, 0xAF): pkt = bytes([cla, 0xa4, 0x00, 0x04, 0x00, 0x90, 0x00]) with self.subTest(cla=cla): self._invalid(pkt, 'invalid CLA') + def test_odd_ins_set_pinned(self): + # Exactly the odd codes the pinned specs define: the ISO 7816-4 + # Table 4 pairs (0F ERASE BINARY, 21 VERIFY, 87 GENERAL + # AUTHENTICATE, A1 SEARCH BINARY, B1 READ BINARY, B3 READ RECORD, + # C3 ENVELOPE, CB GET DATA, D1 WRITE BINARY, D7 UPDATE BINARY, + # DB PUT DATA, DD UPDATE RECORD) plus TS 102 221 AUTHENTICATE 89. + self.assertEqual(self.pd.ODD_INS, frozenset(( + 0x0F, 0x21, 0x87, 0x89, 0xA1, 0xB1, 0xB3, + 0xC3, 0xCB, 0xDB, 0xD1, 0xD7, 0xDD))) + + def test_vs_reader_mirror(self): + # tools/vs_reader.py duplicates the acceptance predicates for its + # standalone pcap analysis; the copies must not drift from pd.py + # (a drift would make its "suspicious CLA" / validity metrics and + # the --pcap-only verdict disagree with the decoder). + path = os.path.join(REPO, 'tools', 'vs_reader.py') + spec = importlib.util.spec_from_file_location('vs_reader', path) + vs = importlib.util.module_from_spec(spec) + spec.loader.exec_module(vs) + self.assertEqual( + vs.VALID_CLA, + frozenset(c for c in range(256) if self.pd.plausible_cla(c))) + self.assertEqual(vs.ODD_INS, self.pd.ODD_INS) + self.assertEqual(vs.EMBEDDED_CLA, self.pd.EMBEDDED_CLA) + def test_invalid_too_short(self): self._invalid(b'\x00\xa4\x00\x04\x00\x90', 'too short') diff --git a/tools/vs_reader.py b/tools/vs_reader.py index 35a2a5e..fe1b2fb 100644 --- a/tools/vs_reader.py +++ b/tools/vs_reader.py @@ -95,8 +95,9 @@ def embedded_exchanges(buf): embedded inside a packet's data region (index 5 onward). Pattern: header (CLA INS P1 P2 P3) + P3 data bytes + interior SW1/SW2 - before the packet's own final status word. CLA must be structurally - valid (same rule as _plausible_cla); INS must be a real command byte; + before the packet's own final status word. CLA must be one of the + pinned EMBEDDED_CLA classes (deliberately narrower than + _plausible_cla()); INS must be a real command byte; SW1 is accepted within a small slack window after the nominal data end and must be interior (trailing bytes follow it). """