catarrh/sigrok_iso7816_stream
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

v1.8.0: PPS as standard request/response packets; fix res-byte flag docs

Browse Source 
- Split the PPS exchange into the two standard GSMTAP-SIM subtypes:
  0x02 (PPS_REQ, raw request bytes, span ss..es_req) and 0x03 (PPS_RSP,
  raw response bytes, span es_req..end), replacing the single 0x02 packet
  with concatenated request+response that deviated from libosmocore's
  gsmtap.h.  Wireshark now dissects both halves; simtrace2-pysniff needs
  no change (its importer already maps 0x03 to 'pps' and _decode_pps()
  parses the 0xFF-leading response frame).  PPS annotation/log unchanged.
- gsmtap_stream.py: GSMTAP_SIM_PPS_REQ/RSP constants; header field labels
  aligned with libosmocore's struct (signal_dbm/snr_db).
- Docs: GSMTAP event tables (EN/RU) updated, PPS deviation note removed;
  corrected the res-byte flag attribution -- upstream simtrace2 never
  sets res, so the GSMTAP_FLAG_BAD_FCS carriage is this decoder's own
  extension (semantics mirror simtrace2's USB data flags).
- Tests: assert the PPS subtype values; baseline regenerated (only the
  version line and test listing differed -- no decode-metric changes).
  68/68 unit tests.
This commit is contained in:
catarrh committed 2026-09-17 23:01:40 +03:00
1 parent c222eecb3c
commit e5a2b7c83b
7 files changed
+102 -53
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+24 -8
View File
@@ -304,6 +304,21 @@ Known limitation: no on-disk fixture reproduces the exact live failure (all
real captures contain a 372 ATR), so end-to-end proof needs a re-captured
`.sr` from the failing phone; the unit tests cover the loop-escape logic.
### v1.8.0: PPS emitted as standard request + response packets
The PPS exchange was previously emitted as a single packet under sub_type
0x02 (`GSMTAP_SIM_PPS_REQ`) carrying the request + response bytes
concatenated — a deviation from libosmocore's gsmtap.h. It is now split
into the two standard subtypes: 0x02 (`PPS_REQ`, raw request bytes, span
`ss..es_req`) and 0x03 (`PPS_RSP`, raw response bytes, span
`es_req..end`). Wireshark now dissects both halves, and simtrace2-pysniff
needs no change: its pcap importer already maps 0x03 to `pps` and
`_decode_pps()` parses any 0xFF-leading frame (the response frame also
starts with 0xFF). The PPS annotation and stderr log lines are unchanged.
`tests/baseline.txt` regenerated — the only differences were the version
line and the unit-test listing, no decode-metric changes. 68/68 unit
tests. VERSION -> 1.8.0.
### v1.7.1: accept EOFError as clean end-of-stream (newer libsigrokdecode)
Newer libsigrokdecode signals end-of-capture from `wait()` by raising
@@ -619,18 +634,19 @@ dirty-wire tail (GARBAGE=1 / RESULT UNCLEAN) and was dropped from the suite.
### GSMTAP wire-format extensions
The GSMTAP-SIM output deviates from libosmocore's gsmtap.h in three
documented ways (see README.md "GSMTAP events" and the comments in
`gsmtap_stream.py`): sub_type 0x02 (`GSMTAP_SIM_PPS`) carries the PPS
request + response combined under the spec's `PPS_REQ` value (the spec
splits into `PPS_REQ` 0x02 / `PPS_RSP` 0x03; 0x03 is unused here);
0x10/0x11 (`RST_EVENT`/`VCC_EVENT`) are custom line-event subtypes; and
The GSMTAP-SIM output uses the standard sub_types 0x00-0x03 from
libosmocore's gsmtap.h with standard payloads -- APDU, ATR, and the PPS
request/response as separate packets (see README.md "GSMTAP events" and the
comments in `gsmtap_stream.py`). It deviates in two documented ways:
0x10/0x11 (`RST_EVENT`/`VCC_EVENT`) are custom line-event subtypes, and
`GSMTAP_FLAG_BAD_FCS` writes flags into the header `res` byte, which
official gsmtap.h reserves — a simtrace2-sniff convention.
official gsmtap.h reserves. The `res`-byte carriage is this decoder's own
extension (upstream simtrace2 never sets `res`); the semantics mirror
simtrace2's per-message USB data flags.
### Versioning
The decoder version is defined in `pd.py` as `VERSION = '1.7.1'`.
The decoder version is defined in `pd.py` as `VERSION = '1.8.0'`.
The version is printed to the log on decoder startup.
### Testing after decoder changes
README.md
+9 -10
View File
@@ -81,7 +81,8 @@ GSMTAP header: version 2, type SIM (0x04), UDP port 4729.
|----------|------|---------|
| 0x00 | APDU | raw T=0/T=1 packet bytes |
| 0x01 | ATR | raw ATR bytes |
| 0x02 | PPS | PPS request + response bytes concatenated (deviation, see below) |
| 0x02 | PPS request | raw PPS request bytes |
| 0x03 | PPS response | raw PPS response bytes |
| 0x10 | RST event (custom) | `[direction, level, 0x00]` |
| 0x11 | VCC event (custom) | `[direction, level, 0x00]` |
@@ -91,17 +92,15 @@ Event payload byte semantics:
- VCC: `direction` 1 = power applied (low→high), 0 = removed (high→low);
`level` = line level after the transition.
0x00/0x01 match libosmocore's gsmtap.h: standard values and payloads.
0x02 reuses the spec's `GSMTAP_SIM_PPS_REQ` value but carries the request
and response bytes concatenated in one packet; the spec splits PPS into
`PPS_REQ` (0x02) / `PPS_RSP` (0x03), so Wireshark dissects these are
"PPS request" and `PPS_RSP` (0x03) is never sent. 0x10/0x11 are custom
extensions not defined in libosmocore's gsmtap.h.
0x00-0x03 match libosmocore's gsmtap.h: standard values and payloads
(PPS request and response are emitted as separate packets). 0x10/0x11
are custom extensions not defined in libosmocore's gsmtap.h.
Header extensions: flags such as `GSMTAP_FLAG_BAD_FCS` are written into
the header `res` byte, which official gsmtap.h marks reserved (RFU) —
flag usage there follows the simtrace2-sniff convention and is not part
of the libosmocore spec.
the header `res` byte, which official gsmtap.h marks reserved (RFU).
This carriage is this decoder's own extension (upstream simtrace2-sniff
never sets `res`); the flag semantics mirror simtrace2's per-message USB
data flags and are not part of the libosmocore spec.
## Usage
README_RUS.md
+8 -9
View File
@@ -84,7 +84,8 @@ GSMTAP pcap-файлы (включая PCAP, который этот декод
|----------|------|---------|
| 0x00 | APDU | сырые байты пакета T=0/T=1 |
| 0x01 | ATR | сырые байты ATR |
| 0x02 | PPS | запрос + ответ PPS, склеенные (отклонение от спеки, см. ниже) |
| 0x02 | Запрос PPS | сырые байты запроса PPS |
| 0x03 | Ответ PPS | сырые байты ответа PPS |
| 0x10 | Событие RST (кастомное) | `[direction, level, 0x00]` |
| 0x11 | Событие VCC (кастомное) | `[direction, level, 0x00]` |
@@ -94,18 +95,16 @@ GSMTAP pcap-файлы (включая PCAP, который этот декод
- VCC: `direction` 1 = питание подано (низкий→высокий), 0 = снято (высокий→низкий);
`level` = уровень линии после перехода.
0x00/0x01 совпадают с gsmtap.h из libosmocore: стандартные значения и
полезные нагрузки. 0x02 переиспользует значение `GSMTAP_SIM_PPS_REQ` из
спеки, но несёт запрос и ответ, склеенные в одном пакете; спека разделяет
PPS на `PPS_REQ` (0x02) / `PPS_RSP` (0x03), поэтому Wireshark разбирает
такие пакеты как «PPS request», а `PPS_RSP` (0x03) не отправляется
никогда. 0x10/0x11 — кастомные расширения, не определённые в gsmtap.h из
0x00–0x03 совпадают с gsmtap.h из libosmocore: стандартные значения и
полезные нагрузки (запрос и ответ PPS отправляются отдельными пакетами).
0x10/0x11 — кастомные расширения, не определённые в gsmtap.h из
libosmocore.
Расширения заголовка: флаги вроде `GSMTAP_FLAG_BAD_FCS` пишутся в байт
`res` заголовка, который официальный gsmtap.h помечает как зарезервированный
(RFU) — такое использование флагов следует конвенции simtrace2-sniff и не
является частью спецификации libosmocore.
(RFU). Это собственное расширение данного декодера (upstream simtrace2-sniff
никогда не заполняет `res`); семантика флагов повторяет флаги данных (data
flags) USB-протокола simtrace2 и не является частью спецификации libosmocore.
## Использование
gsmtap_stream.py
+13 -12
View File
@@ -18,15 +18,14 @@ GSMTAP_VERSION = 0x02
GSMTAP_HDR_LEN = 4 # in 32-bit words (16 bytes)
GSMTAP_TYPE_SIM = 0x04
# sub_type values. 0x00/0x01 are standard GSMTAP-SIM types with standard
# payloads (libosmocore gsmtap.h). 0x02 uses the spec's GSMTAP_SIM_PPS_REQ
# value but carries request + response bytes concatenated in one packet;
# the spec splits PPS into PPS_REQ (0x02) / PPS_RSP (0x03), so Wireshark
# dissects 0x02 as "PPS request" and GSMTAP_SIM_PPS_RSP (0x03) is
# intentionally unused. 0x10/0x11 are custom extensions (not in the spec).
# sub_type values. 0x00-0x03 are standard GSMTAP-SIM types with standard
# payloads (libosmocore gsmtap.h): the PPS request and response are emitted
# as separate packets (PPS_REQ 0x02 / PPS_RSP 0x03). 0x10/0x11 are custom
# extensions (not in the spec) for RST/VCC line events.
GSMTAP_SIM_APDU = 0x00
GSMTAP_SIM_ATR = 0x01
GSMTAP_SIM_PPS = 0x02 # request + response combined (this decoder)
GSMTAP_SIM_PPS_REQ = 0x02 # raw PPS request bytes
GSMTAP_SIM_PPS_RSP = 0x03 # raw PPS response bytes
GSMTAP_SIM_RST_EVENT = 0x10 # payload: [direction, level, reserved]
GSMTAP_SIM_VCC_EVENT = 0x11 # payload: [direction, level, reserved]
@@ -39,9 +38,11 @@ GSMTAP_SIM_VCC_EVENT = 0x11 # payload: [direction, level, reserved]
GSMTAP_UDP_PORT = 4729
# libosmocore gsmtap.h: per-packet flags live in the header `res` byte,
# which the spec marks reserved (RFU). Flag usage here is a documented
# extension following the simtrace2-sniff convention, not part of gsmtap.h.
# Per-packet flags are carried in the header `res` byte, which libosmocore's
# gsmtap.h defines as reserved (RFU). This carriage is our own extension --
# upstream simtrace2-sniff/cardem never set `res` -- although the flag
# semantics mirror simtrace2's per-message USB data flags (the wire problem
# is reported out-of-band instead of being encoded in the APDU bytes).
GSMTAP_FLAG_BAD_FCS = 0x01 # "Any data checksum is wrong" -- reused here to
# mark decoder-desynced TPDUs (see README.md,
# "GSMTAP events").
@@ -60,8 +61,8 @@ def build_packet(sub_type, data, flags=0):
GSMTAP_TYPE_SIM, # type
0, # timeslot
0, # arfcn
0, # noise_db
0, # signal_db
0, # signal_dbm (libosmocore gsmtap.h field order)
0, # snr_db
0, # frame_number
sub_type, # sub_type
0, # antenna_nr
pd.py
+10 -3
View File
@@ -24,11 +24,11 @@ import traceback
from collections import deque
from .gsmtap_stream import (GsmtapStreamSender,
GSMTAP_SIM_APDU, GSMTAP_SIM_ATR, GSMTAP_SIM_PPS,
GSMTAP_SIM_APDU, GSMTAP_SIM_ATR, GSMTAP_SIM_PPS_REQ, GSMTAP_SIM_PPS_RSP,
GSMTAP_SIM_RST_EVENT, GSMTAP_SIM_VCC_EVENT,
GSMTAP_FLAG_BAD_FCS)
VERSION = '1.7.1'
VERSION = '1.8.0'
@@ -1921,6 +1921,10 @@ class Decoder(srd.Decoder):
if (lrc != 0):
self.put(ss, self.samplenum, self.out_ann, [0, ["INVALID Checksum on PPS Request, got={got:02x} expected={expected:02x}".format(got=pck,expected=(lrc ^ pps ^ pps0))]])
self.log("INVALID Checksum on PPS Request", hex(lrc))
# End of the request frame: the response starts on the next byte.
# Keep the boundary sample so the two halves can be emitted as the
# separate standard GSMTAP subtypes (PPS_REQ / PPS_RSP).
es_req = self.samplenum
r_lrc = 0
r_pps = self.read_byte()
@@ -2019,7 +2023,10 @@ class Decoder(srd.Decoder):
self.log("INVALID PPS. Request & Response not matching.", hex(r_lrc))
self.put(ss, self.samplenum, self.out_ann, [0, ["INVALID PPS. Request & Response not matching"]])
self.put(ss, self.samplenum, self.out_ann, [3, ["PPS", "PPS DI={di} FI={fi} clock_skip={clock_skip}".format(di=self.di,fi=self.fi,clock_skip=self.clock_skip)]])
self.emit_packet(GSMTAP_SIM_PPS, bytes(pps_req + pps_rsp), ss, self.samplenum)
# Standard GSMTAP-SIM framing: request and response are separate
# packets (sub_type 0x02 / 0x03), each with its own sample span.
self.emit_packet(GSMTAP_SIM_PPS_REQ, bytes(pps_req), ss, es_req)
self.emit_packet(GSMTAP_SIM_PPS_RSP, bytes(pps_rsp), es_req, self.samplenum)
def _at_eof(self):
tests/baseline.txt
+32 -11
View File
@@ -1,7 +1,7 @@
=== Baseline captured 2026-09-09T14:15:00+03:00 ===
=== Baseline captured 2026-09-17T21:58:59+03:00 ===
=== test_8 ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('etu recovered (samples):', 1237)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -36,7 +36,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== test_7 ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('etu recovered (samples):', 1236)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -77,7 +77,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== xiaomi_phone_sample ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('etu recovered (samples):', 53)
(11673977, 'CHKSUM ERROR: ', 0, 'bits: ', [0, 0, 0, 1, 0, 0, 1, 1, 1, 1])
iso7816-1: CHKSUM ERROR bits=[0, 0, 0, 1, 0, 0, 1, 1, 1, 1]
@@ -118,7 +118,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== xiaomi_mi_a1_coldboot ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -145,7 +145,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== 4gmodem_coldboot ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -186,7 +186,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== samsung_A55_coldboot_new_cable ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -213,7 +213,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== samsung_A55_coldboot_new_cable_sim2 ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -240,7 +240,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== samsung_S21p_coldboot_new_cable ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -267,7 +267,7 @@ Degradation metrics:
RESULT: OK - capture fully explained (reader + CAT + valid retries)
=== samsung_S21p_coldboot_new_cable_sim2 ===
('=== iso7816 decoder v1.5.0 ===',)
('=== iso7816 decoder v1.8.0 ===',)
('RST low at start: expecting cold-boot ATR',)
iso7816-1: RST deasserted
('RST deasserted',)
@@ -348,6 +348,19 @@ RESULT: OK - capture fully explained (reader + CAT + valid retries)
Transitions: 3
Transitions: 47161
=== unit tests ===
test_advancing_is_not_eof (__main__.TestAtEof.test_advancing_is_not_eof) ... ok
test_explicit_eof_wins (__main__.TestAtEof.test_explicit_eof_wins) ... ok
test_pending_replay_is_bounded (__main__.TestAtEof.test_pending_replay_is_bounded) ... ok
test_pending_replay_is_not_eof (__main__.TestAtEof.test_pending_replay_is_not_eof) ... ok
test_stuck_with_nothing_queued_is_eof (__main__.TestAtEof.test_stuck_with_nothing_queued_is_eof) ... ok
test_all_ff_idle_loop_is_bounded (__main__.TestAtrHuntEscape.test_all_ff_idle_loop_is_bounded) ... ok
test_idle_ff_retries_before_cap (__main__.TestAtrHuntEscape.test_idle_ff_retries_before_cap) ... ok
test_idle_resets_capped_then_resume (__main__.TestAtrHuntEscape.test_idle_resets_capped_then_resume) ... ok
test_no_recovery_when_etu_known (__main__.TestAtrHuntEscape.test_no_recovery_when_etu_known) ... ok
test_non_ff_resumes_when_count_high (__main__.TestAtrHuntEscape.test_non_ff_resumes_when_count_high) ... ok
test_recover_etu_in_hunt_runs_measurement (__main__.TestAtrHuntEscape.test_recover_etu_in_hunt_runs_measurement) ... ok
test_recovery_attempts_capped_then_hard_limit (__main__.TestAtrHuntEscape.test_recovery_attempts_capped_then_hard_limit) ... ok
test_retry_then_recover_after_two_failures (__main__.TestAtrHuntEscape.test_retry_then_recover_after_two_failures) ... ok
test_glitch_low_pulse_removed (__main__.TestDeglitchEdges.test_glitch_low_pulse_removed) ... ok
test_no_edges (__main__.TestDeglitchEdges.test_no_edges) ... ok
test_real_pulses_preserved (__main__.TestDeglitchEdges.test_real_pulses_preserved) ... ok
@@ -372,16 +385,24 @@ test_clean_select_6a82 (__main__.TestEmbeddedExchanges.test_clean_select_6a82) .
test_concat_packets_are_not_is_desynced (__main__.TestEmbeddedExchanges.test_concat_packets_are_not_is_desynced) ... ok
test_samsung_record_burst_concat (__main__.TestEmbeddedExchanges.test_samsung_record_burst_concat) ... ok
test_xiaomi_select_getresp_swallow (__main__.TestEmbeddedExchanges.test_xiaomi_select_getresp_swallow) ... ok
test_eoferror_propagates (__main__.TestEndOfStream.test_eoferror_propagates) ... ok
test_systemerror_propagates (__main__.TestEndOfStream.test_systemerror_propagates) ... ok
test_build_packet_bad_fcs_flag (__main__.TestGsmtapFlags.test_build_packet_bad_fcs_flag) ... ok
test_build_packet_payload_passthrough (__main__.TestGsmtapFlags.test_build_packet_payload_passthrough) ... ok
test_build_packet_plain_res_zero (__main__.TestGsmtapFlags.test_build_packet_plain_res_zero) ... ok
test_flag_constant_value (__main__.TestGsmtapFlags.test_flag_constant_value) ... ok
test_pps_subtypes_are_standard (__main__.TestGsmtapFlags.test_pps_subtypes_are_standard) ... ok
test_accepts_full_hold_fall (__main__.TestNextStartFall.test_accepts_full_hold_fall) ... ok
test_finds_fall_after_threshold (__main__.TestNextStartFall.test_finds_fall_after_threshold) ... ok
test_finds_fall_exactly_at_threshold (__main__.TestNextStartFall.test_finds_fall_exactly_at_threshold)
Regression test for Samsung 16 CLK/ETU concatenation. ... ok
test_rejects_phantom_sub_etu_fall (__main__.TestNextStartFall.test_rejects_phantom_sub_etu_fall) ... ok
test_returns_none_when_no_fall (__main__.TestNextStartFall.test_returns_none_when_no_fall) ... ok
test_advancing_sample_is_not_stall (__main__.TestStallCheck.test_advancing_sample_is_not_stall) ... ok
test_none_pins_is_eof (__main__.TestStallCheck.test_none_pins_is_eof) ... ok
test_repeated_same_sample_is_bounded_stall (__main__.TestStallCheck.test_repeated_same_sample_is_bounded_stall) ... ok
test_untracked_level_wait_never_stalls (__main__.TestStallCheck.test_untracked_level_wait_never_stalls) ... ok
test_untracked_resets_stall_window (__main__.TestStallCheck.test_untracked_resets_stall_window) ... ok
test_invalid_cla (__main__.TestValidateT0Apdu.test_invalid_cla) ... ok
test_invalid_cla_proprietary_unseen (__main__.TestValidateT0Apdu.test_invalid_cla_proprietary_unseen) ... ok
test_invalid_ins_lsb_odd (__main__.TestValidateT0Apdu.test_invalid_ins_lsb_odd) ... ok
@@ -398,6 +419,6 @@ test_valid_cla_interindustry_classes (__main__.TestValidateT0Apdu.test_valid_cla
test_valid_direct_sw_with_p3_nonzero (__main__.TestValidateT0Apdu.test_valid_direct_sw_with_p3_nonzero) ... ok
----------------------------------------------------------------------
Ran 47 tests in 0.017s
Ran 68 tests in 0.027s
OK
tests/test_gsmtap.py
+6
View File
@@ -66,6 +66,12 @@ class TestGsmtapFlags(unittest.TestCase):
def test_flag_constant_value(self):
self.assertEqual(self.gs.GSMTAP_FLAG_BAD_FCS, 0x01)
def test_pps_subtypes_are_standard(self):
# PPS is emitted as the two standard GSMTAP-SIM subtypes
# (libosmocore gsmtap.h), request and response as separate packets.
self.assertEqual(self.gs.GSMTAP_SIM_PPS_REQ, 0x02)
self.assertEqual(self.gs.GSMTAP_SIM_PPS_RSP, 0x03)
class TestDesyncHelpers(unittest.TestCase):
@classmethod