diff --git a/AGENTS.md b/AGENTS.md index bdfa140..ad1d8a0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -304,6 +304,21 @@ Known limitation: no on-disk fixture reproduces the exact live failure (all real captures contain a 372 ATR), so end-to-end proof needs a re-captured `.sr` from the failing phone; the unit tests cover the loop-escape logic. +### v1.8.0: PPS emitted as standard request + response packets + +The PPS exchange was previously emitted as a single packet under sub_type +0x02 (`GSMTAP_SIM_PPS_REQ`) carrying the request + response bytes +concatenated — a deviation from libosmocore's gsmtap.h. It is now split +into the two standard subtypes: 0x02 (`PPS_REQ`, raw request bytes, span +`ss..es_req`) and 0x03 (`PPS_RSP`, raw response bytes, span +`es_req..end`). Wireshark now dissects both halves, and simtrace2-pysniff +needs no change: its pcap importer already maps 0x03 to `pps` and +`_decode_pps()` parses any 0xFF-leading frame (the response frame also +starts with 0xFF). The PPS annotation and stderr log lines are unchanged. +`tests/baseline.txt` regenerated — the only differences were the version +line and the unit-test listing, no decode-metric changes. 68/68 unit +tests. VERSION -> 1.8.0. + ### v1.7.1: accept EOFError as clean end-of-stream (newer libsigrokdecode) Newer libsigrokdecode signals end-of-capture from `wait()` by raising @@ -619,18 +634,19 @@ dirty-wire tail (GARBAGE=1 / RESULT UNCLEAN) and was dropped from the suite. ### GSMTAP wire-format extensions -The GSMTAP-SIM output deviates from libosmocore's gsmtap.h in three -documented ways (see README.md "GSMTAP events" and the comments in -`gsmtap_stream.py`): sub_type 0x02 (`GSMTAP_SIM_PPS`) carries the PPS -request + response combined under the spec's `PPS_REQ` value (the spec -splits into `PPS_REQ` 0x02 / `PPS_RSP` 0x03; 0x03 is unused here); -0x10/0x11 (`RST_EVENT`/`VCC_EVENT`) are custom line-event subtypes; and +The GSMTAP-SIM output uses the standard sub_types 0x00-0x03 from +libosmocore's gsmtap.h with standard payloads -- APDU, ATR, and the PPS +request/response as separate packets (see README.md "GSMTAP events" and the +comments in `gsmtap_stream.py`). It deviates in two documented ways: +0x10/0x11 (`RST_EVENT`/`VCC_EVENT`) are custom line-event subtypes, and `GSMTAP_FLAG_BAD_FCS` writes flags into the header `res` byte, which -official gsmtap.h reserves — a simtrace2-sniff convention. +official gsmtap.h reserves. The `res`-byte carriage is this decoder's own +extension (upstream simtrace2 never sets `res`); the semantics mirror +simtrace2's per-message USB data flags. ### Versioning -The decoder version is defined in `pd.py` as `VERSION = '1.7.1'`. +The decoder version is defined in `pd.py` as `VERSION = '1.8.0'`. The version is printed to the log on decoder startup. ### Testing after decoder changes diff --git a/README.md b/README.md index d7fce63..1773b86 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,8 @@ GSMTAP header: version 2, type SIM (0x04), UDP port 4729. |----------|------|---------| | 0x00 | APDU | raw T=0/T=1 packet bytes | | 0x01 | ATR | raw ATR bytes | -| 0x02 | PPS | PPS request + response bytes concatenated (deviation, see below) | +| 0x02 | PPS request | raw PPS request bytes | +| 0x03 | PPS response | raw PPS response bytes | | 0x10 | RST event (custom) | `[direction, level, 0x00]` | | 0x11 | VCC event (custom) | `[direction, level, 0x00]` | @@ -91,17 +92,15 @@ Event payload byte semantics: - VCC: `direction` 1 = power applied (low→high), 0 = removed (high→low); `level` = line level after the transition. -0x00/0x01 match libosmocore's gsmtap.h: standard values and payloads. -0x02 reuses the spec's `GSMTAP_SIM_PPS_REQ` value but carries the request -and response bytes concatenated in one packet; the spec splits PPS into -`PPS_REQ` (0x02) / `PPS_RSP` (0x03), so Wireshark dissects these are -"PPS request" and `PPS_RSP` (0x03) is never sent. 0x10/0x11 are custom -extensions not defined in libosmocore's gsmtap.h. +0x00-0x03 match libosmocore's gsmtap.h: standard values and payloads +(PPS request and response are emitted as separate packets). 0x10/0x11 +are custom extensions not defined in libosmocore's gsmtap.h. Header extensions: flags such as `GSMTAP_FLAG_BAD_FCS` are written into -the header `res` byte, which official gsmtap.h marks reserved (RFU) — -flag usage there follows the simtrace2-sniff convention and is not part -of the libosmocore spec. +the header `res` byte, which official gsmtap.h marks reserved (RFU). +This carriage is this decoder's own extension (upstream simtrace2-sniff +never sets `res`); the flag semantics mirror simtrace2's per-message USB +data flags and are not part of the libosmocore spec. ## Usage diff --git a/README_RUS.md b/README_RUS.md index 2ce9ede..2bd8c61 100644 --- a/README_RUS.md +++ b/README_RUS.md @@ -84,7 +84,8 @@ GSMTAP pcap-файлы (включая PCAP, который этот декод |----------|------|---------| | 0x00 | APDU | сырые байты пакета T=0/T=1 | | 0x01 | ATR | сырые байты ATR | -| 0x02 | PPS | запрос + ответ PPS, склеенные (отклонение от спеки, см. ниже) | +| 0x02 | Запрос PPS | сырые байты запроса PPS | +| 0x03 | Ответ PPS | сырые байты ответа PPS | | 0x10 | Событие RST (кастомное) | `[direction, level, 0x00]` | | 0x11 | Событие VCC (кастомное) | `[direction, level, 0x00]` | @@ -94,18 +95,16 @@ GSMTAP pcap-файлы (включая PCAP, который этот декод - VCC: `direction` 1 = питание подано (низкий→высокий), 0 = снято (высокий→низкий); `level` = уровень линии после перехода. -0x00/0x01 совпадают с gsmtap.h из libosmocore: стандартные значения и -полезные нагрузки. 0x02 переиспользует значение `GSMTAP_SIM_PPS_REQ` из -спеки, но несёт запрос и ответ, склеенные в одном пакете; спека разделяет -PPS на `PPS_REQ` (0x02) / `PPS_RSP` (0x03), поэтому Wireshark разбирает -такие пакеты как «PPS request», а `PPS_RSP` (0x03) не отправляется -никогда. 0x10/0x11 — кастомные расширения, не определённые в gsmtap.h из +0x00–0x03 совпадают с gsmtap.h из libosmocore: стандартные значения и +полезные нагрузки (запрос и ответ PPS отправляются отдельными пакетами). +0x10/0x11 — кастомные расширения, не определённые в gsmtap.h из libosmocore. Расширения заголовка: флаги вроде `GSMTAP_FLAG_BAD_FCS` пишутся в байт `res` заголовка, который официальный gsmtap.h помечает как зарезервированный -(RFU) — такое использование флагов следует конвенции simtrace2-sniff и не -является частью спецификации libosmocore. +(RFU). Это собственное расширение данного декодера (upstream simtrace2-sniff +никогда не заполняет `res`); семантика флагов повторяет флаги данных (data +flags) USB-протокола simtrace2 и не является частью спецификации libosmocore. ## Использование diff --git a/gsmtap_stream.py b/gsmtap_stream.py index c01e4e8..32fd034 100644 --- a/gsmtap_stream.py +++ b/gsmtap_stream.py @@ -18,15 +18,14 @@ GSMTAP_VERSION = 0x02 GSMTAP_HDR_LEN = 4 # in 32-bit words (16 bytes) GSMTAP_TYPE_SIM = 0x04 -# sub_type values. 0x00/0x01 are standard GSMTAP-SIM types with standard -# payloads (libosmocore gsmtap.h). 0x02 uses the spec's GSMTAP_SIM_PPS_REQ -# value but carries request + response bytes concatenated in one packet; -# the spec splits PPS into PPS_REQ (0x02) / PPS_RSP (0x03), so Wireshark -# dissects 0x02 as "PPS request" and GSMTAP_SIM_PPS_RSP (0x03) is -# intentionally unused. 0x10/0x11 are custom extensions (not in the spec). +# sub_type values. 0x00-0x03 are standard GSMTAP-SIM types with standard +# payloads (libosmocore gsmtap.h): the PPS request and response are emitted +# as separate packets (PPS_REQ 0x02 / PPS_RSP 0x03). 0x10/0x11 are custom +# extensions (not in the spec) for RST/VCC line events. GSMTAP_SIM_APDU = 0x00 GSMTAP_SIM_ATR = 0x01 -GSMTAP_SIM_PPS = 0x02 # request + response combined (this decoder) +GSMTAP_SIM_PPS_REQ = 0x02 # raw PPS request bytes +GSMTAP_SIM_PPS_RSP = 0x03 # raw PPS response bytes GSMTAP_SIM_RST_EVENT = 0x10 # payload: [direction, level, reserved] GSMTAP_SIM_VCC_EVENT = 0x11 # payload: [direction, level, reserved] @@ -39,9 +38,11 @@ GSMTAP_SIM_VCC_EVENT = 0x11 # payload: [direction, level, reserved] GSMTAP_UDP_PORT = 4729 -# libosmocore gsmtap.h: per-packet flags live in the header `res` byte, -# which the spec marks reserved (RFU). Flag usage here is a documented -# extension following the simtrace2-sniff convention, not part of gsmtap.h. +# Per-packet flags are carried in the header `res` byte, which libosmocore's +# gsmtap.h defines as reserved (RFU). This carriage is our own extension -- +# upstream simtrace2-sniff/cardem never set `res` -- although the flag +# semantics mirror simtrace2's per-message USB data flags (the wire problem +# is reported out-of-band instead of being encoded in the APDU bytes). GSMTAP_FLAG_BAD_FCS = 0x01 # "Any data checksum is wrong" -- reused here to # mark decoder-desynced TPDUs (see README.md, # "GSMTAP events"). @@ -60,8 +61,8 @@ def build_packet(sub_type, data, flags=0): GSMTAP_TYPE_SIM, # type 0, # timeslot 0, # arfcn - 0, # noise_db - 0, # signal_db + 0, # signal_dbm (libosmocore gsmtap.h field order) + 0, # snr_db 0, # frame_number sub_type, # sub_type 0, # antenna_nr diff --git a/pd.py b/pd.py index 19a0a03..1fbbc46 100644 --- a/pd.py +++ b/pd.py @@ -24,11 +24,11 @@ import traceback from collections import deque from .gsmtap_stream import (GsmtapStreamSender, - GSMTAP_SIM_APDU, GSMTAP_SIM_ATR, GSMTAP_SIM_PPS, + GSMTAP_SIM_APDU, GSMTAP_SIM_ATR, GSMTAP_SIM_PPS_REQ, GSMTAP_SIM_PPS_RSP, GSMTAP_SIM_RST_EVENT, GSMTAP_SIM_VCC_EVENT, GSMTAP_FLAG_BAD_FCS) -VERSION = '1.7.1' +VERSION = '1.8.0' @@ -1921,7 +1921,11 @@ class Decoder(srd.Decoder): if (lrc != 0): self.put(ss, self.samplenum, self.out_ann, [0, ["INVALID Checksum on PPS Request, got={got:02x} expected={expected:02x}".format(got=pck,expected=(lrc ^ pps ^ pps0))]]) self.log("INVALID Checksum on PPS Request", hex(lrc)) - + # End of the request frame: the response starts on the next byte. + # Keep the boundary sample so the two halves can be emitted as the + # separate standard GSMTAP subtypes (PPS_REQ / PPS_RSP). + es_req = self.samplenum + r_lrc = 0 r_pps = self.read_byte() pps_rsp = [r_pps] @@ -2019,7 +2023,10 @@ class Decoder(srd.Decoder): self.log("INVALID PPS. Request & Response not matching.", hex(r_lrc)) self.put(ss, self.samplenum, self.out_ann, [0, ["INVALID PPS. Request & Response not matching"]]) self.put(ss, self.samplenum, self.out_ann, [3, ["PPS", "PPS DI={di} FI={fi} clock_skip={clock_skip}".format(di=self.di,fi=self.fi,clock_skip=self.clock_skip)]]) - self.emit_packet(GSMTAP_SIM_PPS, bytes(pps_req + pps_rsp), ss, self.samplenum) + # Standard GSMTAP-SIM framing: request and response are separate + # packets (sub_type 0x02 / 0x03), each with its own sample span. + self.emit_packet(GSMTAP_SIM_PPS_REQ, bytes(pps_req), ss, es_req) + self.emit_packet(GSMTAP_SIM_PPS_RSP, bytes(pps_rsp), es_req, self.samplenum) def _at_eof(self): diff --git a/tests/baseline.txt b/tests/baseline.txt index 83be604..bdd3665 100644 --- a/tests/baseline.txt +++ b/tests/baseline.txt @@ -1,7 +1,7 @@ -=== Baseline captured 2026-09-09T14:15:00+03:00 === +=== Baseline captured 2026-09-17T21:58:59+03:00 === === test_8 === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('etu recovered (samples):', 1237) iso7816-1: RST deasserted ('RST deasserted',) @@ -36,7 +36,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === test_7 === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('etu recovered (samples):', 1236) iso7816-1: RST deasserted ('RST deasserted',) @@ -77,7 +77,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === xiaomi_phone_sample === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('etu recovered (samples):', 53) (11673977, 'CHKSUM ERROR: ', 0, 'bits: ', [0, 0, 0, 1, 0, 0, 1, 1, 1, 1]) iso7816-1: CHKSUM ERROR bits=[0, 0, 0, 1, 0, 0, 1, 1, 1, 1] @@ -118,7 +118,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === xiaomi_mi_a1_coldboot === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -145,7 +145,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === 4gmodem_coldboot === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -186,7 +186,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === samsung_A55_coldboot_new_cable === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -213,7 +213,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === samsung_A55_coldboot_new_cable_sim2 === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -240,7 +240,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === samsung_S21p_coldboot_new_cable === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -267,7 +267,7 @@ Degradation metrics: RESULT: OK - capture fully explained (reader + CAT + valid retries) === samsung_S21p_coldboot_new_cable_sim2 === -('=== iso7816 decoder v1.5.0 ===',) +('=== iso7816 decoder v1.8.0 ===',) ('RST low at start: expecting cold-boot ATR',) iso7816-1: RST deasserted ('RST deasserted',) @@ -348,6 +348,19 @@ RESULT: OK - capture fully explained (reader + CAT + valid retries) Transitions: 3 Transitions: 47161 === unit tests === +test_advancing_is_not_eof (__main__.TestAtEof.test_advancing_is_not_eof) ... ok +test_explicit_eof_wins (__main__.TestAtEof.test_explicit_eof_wins) ... ok +test_pending_replay_is_bounded (__main__.TestAtEof.test_pending_replay_is_bounded) ... ok +test_pending_replay_is_not_eof (__main__.TestAtEof.test_pending_replay_is_not_eof) ... ok +test_stuck_with_nothing_queued_is_eof (__main__.TestAtEof.test_stuck_with_nothing_queued_is_eof) ... ok +test_all_ff_idle_loop_is_bounded (__main__.TestAtrHuntEscape.test_all_ff_idle_loop_is_bounded) ... ok +test_idle_ff_retries_before_cap (__main__.TestAtrHuntEscape.test_idle_ff_retries_before_cap) ... ok +test_idle_resets_capped_then_resume (__main__.TestAtrHuntEscape.test_idle_resets_capped_then_resume) ... ok +test_no_recovery_when_etu_known (__main__.TestAtrHuntEscape.test_no_recovery_when_etu_known) ... ok +test_non_ff_resumes_when_count_high (__main__.TestAtrHuntEscape.test_non_ff_resumes_when_count_high) ... ok +test_recover_etu_in_hunt_runs_measurement (__main__.TestAtrHuntEscape.test_recover_etu_in_hunt_runs_measurement) ... ok +test_recovery_attempts_capped_then_hard_limit (__main__.TestAtrHuntEscape.test_recovery_attempts_capped_then_hard_limit) ... ok +test_retry_then_recover_after_two_failures (__main__.TestAtrHuntEscape.test_retry_then_recover_after_two_failures) ... ok test_glitch_low_pulse_removed (__main__.TestDeglitchEdges.test_glitch_low_pulse_removed) ... ok test_no_edges (__main__.TestDeglitchEdges.test_no_edges) ... ok test_real_pulses_preserved (__main__.TestDeglitchEdges.test_real_pulses_preserved) ... ok @@ -372,16 +385,24 @@ test_clean_select_6a82 (__main__.TestEmbeddedExchanges.test_clean_select_6a82) . test_concat_packets_are_not_is_desynced (__main__.TestEmbeddedExchanges.test_concat_packets_are_not_is_desynced) ... ok test_samsung_record_burst_concat (__main__.TestEmbeddedExchanges.test_samsung_record_burst_concat) ... ok test_xiaomi_select_getresp_swallow (__main__.TestEmbeddedExchanges.test_xiaomi_select_getresp_swallow) ... ok +test_eoferror_propagates (__main__.TestEndOfStream.test_eoferror_propagates) ... ok +test_systemerror_propagates (__main__.TestEndOfStream.test_systemerror_propagates) ... ok test_build_packet_bad_fcs_flag (__main__.TestGsmtapFlags.test_build_packet_bad_fcs_flag) ... ok test_build_packet_payload_passthrough (__main__.TestGsmtapFlags.test_build_packet_payload_passthrough) ... ok test_build_packet_plain_res_zero (__main__.TestGsmtapFlags.test_build_packet_plain_res_zero) ... ok test_flag_constant_value (__main__.TestGsmtapFlags.test_flag_constant_value) ... ok +test_pps_subtypes_are_standard (__main__.TestGsmtapFlags.test_pps_subtypes_are_standard) ... ok test_accepts_full_hold_fall (__main__.TestNextStartFall.test_accepts_full_hold_fall) ... ok test_finds_fall_after_threshold (__main__.TestNextStartFall.test_finds_fall_after_threshold) ... ok test_finds_fall_exactly_at_threshold (__main__.TestNextStartFall.test_finds_fall_exactly_at_threshold) Regression test for Samsung 16 CLK/ETU concatenation. ... ok test_rejects_phantom_sub_etu_fall (__main__.TestNextStartFall.test_rejects_phantom_sub_etu_fall) ... ok test_returns_none_when_no_fall (__main__.TestNextStartFall.test_returns_none_when_no_fall) ... ok +test_advancing_sample_is_not_stall (__main__.TestStallCheck.test_advancing_sample_is_not_stall) ... ok +test_none_pins_is_eof (__main__.TestStallCheck.test_none_pins_is_eof) ... ok +test_repeated_same_sample_is_bounded_stall (__main__.TestStallCheck.test_repeated_same_sample_is_bounded_stall) ... ok +test_untracked_level_wait_never_stalls (__main__.TestStallCheck.test_untracked_level_wait_never_stalls) ... ok +test_untracked_resets_stall_window (__main__.TestStallCheck.test_untracked_resets_stall_window) ... ok test_invalid_cla (__main__.TestValidateT0Apdu.test_invalid_cla) ... ok test_invalid_cla_proprietary_unseen (__main__.TestValidateT0Apdu.test_invalid_cla_proprietary_unseen) ... ok test_invalid_ins_lsb_odd (__main__.TestValidateT0Apdu.test_invalid_ins_lsb_odd) ... ok @@ -398,6 +419,6 @@ test_valid_cla_interindustry_classes (__main__.TestValidateT0Apdu.test_valid_cla test_valid_direct_sw_with_p3_nonzero (__main__.TestValidateT0Apdu.test_valid_direct_sw_with_p3_nonzero) ... ok ---------------------------------------------------------------------- -Ran 47 tests in 0.017s +Ran 68 tests in 0.027s OK diff --git a/tests/test_gsmtap.py b/tests/test_gsmtap.py index 0804829..119532c 100644 --- a/tests/test_gsmtap.py +++ b/tests/test_gsmtap.py @@ -66,6 +66,12 @@ class TestGsmtapFlags(unittest.TestCase): def test_flag_constant_value(self): self.assertEqual(self.gs.GSMTAP_FLAG_BAD_FCS, 0x01) + def test_pps_subtypes_are_standard(self): + # PPS is emitted as the two standard GSMTAP-SIM subtypes + # (libosmocore gsmtap.h), request and response as separate packets. + self.assertEqual(self.gs.GSMTAP_SIM_PPS_REQ, 0x02) + self.assertEqual(self.gs.GSMTAP_SIM_PPS_RSP, 0x03) + class TestDesyncHelpers(unittest.TestCase): @classmethod