Commit Graph
41 Commits
Author SHA1 Message Date
catarrh 1c2d6af9e4 Decode OTA secured command/response packets in SMS TP-UD; collapse RFU P1/P2 2026-08-16 23:08:54 +03:00
catarrh a056acb753 Bump version to 1.4.0 2026-08-16 22:18:24 +03:00
catarrh 58af838e72 Decode P1/P2 for all commands; fix PIN/SEARCH RECORD references
- Always emit raw P1/P2 in decode_message (with name/bits when the spec
  provides them), so FETCH/ENVELOPE/GET RESPONSE/etc. show P1/P2.
- Add PIN key-reference table (TS 102 221 Table 9.3) and move PIN
  reference from P1 to P2 for VERIFY/CHANGE/DISABLE/ENABLE/UNBLOCK PIN.
- Fix SEARCH RECORD P2 mode bits (forward/backward/enhanced/proprietary).
- Add MANAGE CHANNEL P2 (channel number), RETRIEVE/SET DATA block mode,
  GET/PUT DATA tag, STATUS/DELETE FILE P2.
- Frontend renders raw-only P1/P2 cleanly.
2026-08-16 22:15:20 +03:00
catarrh 0fdaed503e Deepen AUTHENTICATE decode: P2 auth context + AUTN/AUTS decomposition
- Add a P2 spec for AUTHENTICATE (auth context bits + specific-reference-data
  flag), so P2 is displayed instead of only leaking through cmd.context.
- Fix P1 name to 'No indication' (was the misleading 'Run GSM algorithm').
- Decompose AUTN into SQN(xor)AK / AMF / MAC and AUTS into SQN(xor)AK / MAC-S
  per TS 33.102; surface 'specific reference data' key flag.
2026-08-16 21:44:57 +03:00
catarrh 35067f8d9d Color-code SW in APDU list + fix empty ISIM record decode
- SW column now shows green (success 90/92-9F), blue (pending 61/91/6C),
  amber (warning 62/63), red (error 64-6F) per ISO 7816-4 / TS 102 221.
- _decode_nai: treat all-FF records as empty instead of emitting
  replacement characters; strip FF padding before the ASCII fallback.
2026-08-16 13:22:01 +03:00
catarrh 747974460e Harden selection tracking against dropped/failed SELECTs
- Only update the selected-file state on a successful SELECT (SW1 in
  61/62/63/90/91/92/9E/9F); a failed SELECT (e.g. 6A82) leaves the
  current file unchanged.
- Track the file structure from the FCP (GET RESPONSE after SELECT).
- Skip file decoding for record ops (READ/UPDATE/SEARCH RECORD) when the
  selected file is transparent/DF, marking the result 'stale' instead of
  decoding garbage — this catches silently-dropped SELECTs.
2026-08-16 00:20:32 +03:00
catarrh 91cb4e6e02 Add file-aware decoding for READ/UPDATE/SEARCH RECORD
- Track the currently selected file (SELECT by FID/path) across the
  session and pass it to the decoder; reset on ATR/card change; seed via
  a bounded backward scan on paged views.
- Decode READ BINARY/RECORD, UPDATE BINARY/RECORD and (via GET RESPONSE)
  SEARCH RECORD bodies using a FILE_DECODERS registry: IMSI, ICCID,
  LI/PL, ADN-format (MSISDN/SDN/FDN/...), SMS records (SMSC + TPDU),
  UST/SST/IST service tables, PLMN lists (+access tech), EF_DIR,
  CBMI/R, ARR, PNN, ECC, SPN, LOCI, ACC, Phase, ISIM IMPI/DOMAIN/IMPU.
- File formats verified against TS 51.011 / TS 102 221 / TS 31.102
  (PLMNwAcT is 5-byte entries; EF_SMS = status + SMSC + TPDU; ISIM
  identities are 0x80 TLV).
- Offset != 0 READ BINARY/UPDATE BINARY is skipped (fragment reads).
- SEARCH RECORD results (record numbers) decoded in GET RESPONSE.
- GET RESPONSE summary now shows FCP file type/structure/size.
- New 'File data' section in the detail view.
2026-08-15 23:11:31 +03:00
catarrh 143e1854e2 Add ATR/PPS + SMS TPDU decoding, command summaries, SW column
- Decode ATR (convention, interface bytes, historical, TCK) and PPS (Fi/Di, PCK)
- Decode ENVELOPE SMS-PP DOWNLOAD + SEND SHORT MESSAGE TPDU: SCTS, VP,
  DCS/PID, UDH IEIs, best-effort 8-bit text
- Fix Duration data object ([time unit][interval], not a 16-bit int)
- List view: command summary instead of raw hex, SW column, raw APDU
  first line in detail view
- Expand FID tables (full pySim USIM/ISIM/SIM set) + context-aware
  SELECT-path resolution (7FFF = current ADF)
- Fix SELECT P1 names (0x09 = path from current DF)
- Bump version to 1.3.0
2026-08-15 21:24:50 +03:00
catarrh 97a7e6034a Show status in trace empty-state; discard empty captures; polish capture-mode
Frontend:
- trace empty-state now shows connection status + capture mode (boxed,
  colored) in the center, updated live via checkConnection
- capture-mode uses amber warning style (gsmtap/disabled), green for direct
- stopCapture clears currentSessionId

Server:
- gsmtap read_packet catches OSError on socket close (no more EBADF
  traceback on stop)
- stop_session joins the loop and discards a session with zero messages;
  /api/capture/stop reports discarded
- startup prints server version

Tests: 122 (gsmtap close, empty/non-empty capture lifecycle).
2026-08-15 08:48:07 +03:00
catarrh 0bb3bc82c7 Add --host bind flag + gitignore tools/ (Docker demo scaffolding)
- server --host (default 127.0.0.1) so it can bind 0.0.0.0 inside a
  container
- gitignore tools/ (untracked local Docker/export scripts)
2026-08-15 02:31:34 +03:00
catarrh c21b655fe1 Fix pcapng timestamp resolution + imported-session duration
- pcap.py: _parse_tsresol now pads options to a 32-bit boundary
  ((-length) % 4) and uses 2** for the MSB-set (power-of-2) case, so
  nanosecond if_tsresol is read correctly (was defaulting to µs and
  scaling timestamps 1000x)
- database/server: set imported session started/ended from the first/last
  packet timestamps (falling back to wall-clock for synthetic SPB)
- frontend: formatDuration renders hours
- tests: tsresol (ns after padded option, default µs, ns round-trip) and
  session-window/fallback (119 total)
2026-08-15 02:02:44 +03:00
catarrh 5ba55ac3a9 Decode EVENT DOWNLOAD location/caller and CELL BROADCAST DOWNLOAD CB page
- EVENT DOWNLOAD: decode location information (PLMN/LAC/Cell), caller
  number (MT call Address), transaction identifier, subaddress
- CELL BROADCAST DOWNLOAD: decode CB page header (serial, message ID,
  DCS, page) with raw content hex
- frontend: render Caller/Transaction/Subaddress and CB page fields

Tests: 114.
2026-08-15 00:55:07 +03:00
catarrh 9362bef869 Add server-source selector (same-origin default) and drop SIMTRACE_EMBEDDED
- Settings: 'Server' selector — 'Same origin' (default) vs 'Custom URL'
  (URL input revealed only in custom mode, placeholder 127.0.0.1:8081)
- serverUrl defaults to '' (same-origin); persisted per choice
- remove SIMTRACE_EMBEDDED frontend check and the server's _EMBEDDED_MARKER
  index.html injection (no longer needed)
- i18n: Server / Same origin / Custom URL (EN/RU)
2026-08-15 00:24:58 +03:00
catarrh 24d0decd51 Add --capture disabled mode + capture-mode status in PWA
Server:
- --capture accepts 'disabled'; no backend is created, capture_mode exposed
- /api/status returns capture_mode; capture start/stop return 403 when
  disabled; latest/status/delete_session are None-safe

PWA:
- captureMode state; conn-status shows server version
- new capture-mode line (Settings) explaining gsmtap/direct/disabled
- hide capture + New capture UI when disabled; trace empty-state shows
  'Capture unavailable'
- 7 new EN/RU i18n keys

Version 1.2.0 (server + PWA); tests: 111.
2026-08-14 23:49:10 +03:00
catarrh b900a8a3e1 Merge simtrace-analyser PWA into frontend/ (monorepo); serve it same-origin
- frontend/: simtrace-analyser PWA (index.html, sw.js, manifest.json,
  icons, tests/index.test.html) copied in fresh
- server.py: static file serving (_STATIC_MIME, _serve_static) with
  path-traversal guard; injects window.SIMTRACE_EMBEDDED='1' into
  index.html so the PWA defaults to a relative API base
- server __main__: --web-dir (default <repo>/frontend)
- frontend: embedded same-origin mode; sw.js relative precache paths +
  cache v4; GitHub link + version-warn point at simtrace2-pysniff
- version 1.1.0 everywhere (__version__, pyproject, PWA_VERSION,
  manifest); MIN_SERVER_VERSION stays 1.0.0
- README/scripts/.gitignore updated

Tests: 107.
2026-08-14 16:26:28 +03:00
catarrh 8b7b10b8fa Version 1.0.0; expose version in /api/status
First published version. __version__ and pyproject bumped to 1.0.0,
/api/status now returns 'version' so the PWA can check compatibility.

Tests: 107 (status includes version, package version).
2026-08-14 10:24:24 +03:00
catarrh 73f40ad570 server: send Access-Control-Allow-Private-Network on CORS responses
Lets a public-hosted PWA reach a local simtrace2-pysniff-server
(http://127.0.0.1:8081); without it the browser blocks the preflight
before any request is sent.
2026-08-14 00:45:52 +03:00
catarrh e5a6000bfe parse_tlv: support BER-TLV long-form lengths
Proactive commands with a body over 127 bytes (e.g. SET UP MENU with a
long item list) use a long-form length (0x81 NN / 0x82 NN NN) in their
D0 wrapper.  Previously parse_tlv bailed on those, so the FETCH decoded
as a bare 'FETCH' without its command name or items.

Tests: 105 (long-form 1/2-byte, real SET UP MENU sample).
2026-08-13 10:24:59 +03:00
catarrh dd34c25899 Add PCAP/PCAPNG import via POST /api/sessions/import
- pcap.py: parse_pcap (micro/nano, both byte orders) and parse_pcapng
  (SHB byte-order, IDB linktype + if_tsresol, EPB/SPB) readers that
  unwrap Ethernet/RAW/IPv4/NULL/SLL/SLL2 to UDP and extract GSMTAP-SIM
  packets (APDU->tpdu, ATR->atr, PPS->pps), with a heuristic fallback
- database: insert_messages bulk-insert helper
- server: import endpoint reads raw body, auto-detects format, skips
  non-GSMTAP packets; 400 'No GSMTAP data found in trace' (no session)
  when empty, else creates a 'pcap' session named after the file

Tests: 102 (pcap round-trip, pcapng, no-GSMTAP, endpoint 200/400).
2026-08-13 10:15:22 +03:00
catarrh 6b2bcefbac decode_cla: label standard ETSI class bytes precisely
CLA 0x80 is the ETSI/3GPP-defined UICC/USIM class (TS 102 221 Table 10.3),
not ISO 7816-4 'proprietary'. Relabel 0x80 -> ETSI-defined (UICC/USIM),
0xA0 -> ETSI-defined (SIM/GSM), 0x00 -> inter-industry (ISO 7816-4);
drop the now-redundant note field.
2026-08-13 09:14:41 +03:00
catarrh 4e67948dce Add CAT deep decode: TR local info (PLI) + proactive command bodies
TERMINAL RESPONSE additional information:
- PLI local-info decoders by tag: Location (PLMN/LAC/Cell), IMEI (BCD),
  date-time (TS 23.040 SCTS), language, access technology, timing
  advance, battery state, network search mode, NMR
- Duration (POLL INTERVAL) and Item Identifier (SELECT ITEM)

Proactive commands (FETCH, D0 TLV, tag&0x7F normalized):
- Annex A text (0x80/0x81/0x82 UCS2 variants, else GSM-7 default) and
  DCS text decoders
- SET UP MENU / SELECT ITEM title + items, SEND SHORT MESSAGE TPDU,
  SET UP EVENT LIST events, POLL INTERVAL duration, DISPLAY TEXT etc.

Tests: 96 (TR PLI, proactive, Annex A variants, DCS text).
2026-08-13 09:00:14 +03:00
catarrh 0785e4d1df Add deeper command-body decoding: AUTHENTICATE, ENVELOPE, SM TPDU
- _decode_auth_cmd: AUTHENTICATE command body → RAND/AUTN (GSM/3G,
  context from P2)
- EVENT_TYPES + event download decode (Event List → event name,
  Location status → Normal/Limited/No service)
- _decode_envelope: D1 SMS-PP (SMSC + SMS TPDU), D2 Cell Broadcast
  (structure), D5 MO SM control (SMSC + TP-DA + location), D3/D4/D7 basic
- _decode_sm_tpdu: SMS-DELIVER/SMS-SUBMIT → TP-MTI, BCD address, TP-PID,
  TP-DCS (encoding + class), text (GSM 7-bit / UCS2 only; 8-bit stays
  hex; PID 0x7F → secured packet hex)
- helpers: _decode_bcd_address, _decode_gsm7, GSM 7-bit alphabet

decode_message stores command-body decode in decoded.cmd for
AUTHENTICATE (0x88/0x89) and ENVELOPE (0xC2).

tests: 79 total (auth-cmd, event download, SM TPDU, BCD, GSM-7).
2026-08-12 22:23:39 +03:00
catarrh 7556401618 CLI --pcap: only write ATR/TPDU messages (skip PPS/CHANGE/FIDI)
The PCAP branch fell back to GSMTAP_SIM_APDU for every non-ATR message,
so PPS/CHANGE/FIDI were written as 'APDU'. Align with the UDP branch and
the server export: only atr→ATR and tpdu→APDU are written.
2026-08-12 21:37:06 +03:00
catarrh cb4c07f3c2 Fix GSMTAP linktype + type/sub-type values so Wireshark decodes PCAP
pcap.py: LINKTYPE_GSMTAP (155) was wrong — 155 is DLT_USER8, so
Wireshark showed 'USER 8 / DLT=155'. Now write Ethernet-encapsulated
PCAP (LINKTYPE 1): each GSMTAP message wrapped in Ethernet + IPv4 +
UDP (port 4729), with correct checksums — matching a real tcpdump
capture of GSMTAP-over-UDP and how Wireshark's gsmtap dissector is keyed.

gsmtap.py: GSMTAP_TYPE_SIM was 0x07 (WiMAX burst) — corrected to 0x04;
GSMTAP_SIM_APDU was 0x02 (PPS req) — corrected to 0x00; added PPS_REQ/
PPS_RSP. These match libosmocore/Wireshark packet-gsmtap.h.

Verified with tshark: PCAP now decodes as GSM SIM (SELECT/STATUS/ATR)
instead of 'USER 8'.

tests: pcap tests assert Ethernet linktype + framing + GSMTAP payload.
2026-08-12 21:32:21 +03:00
catarrh e09aac4cb7 Add session export to PCAP
pcap.py: build_pcap(packets) — build PCAP bytes in memory (same format
as PcapWriter)
database.py: get_messages_raw() — raw elapsed/type/data without decode
server.py: GET /api/sessions/<id>/pcap generates a GSMTAP-encapsulated
PCAP (ATR/TPDU) with real timestamps and downloads it as
<session-name>.pcap (ASCII filename fallback + RFC 5987 UTF-8 for
spaces/unicode via _content_disposition)

tests: build_pcap + _content_disposition (67 total)
2026-08-12 21:16:11 +03:00
catarrh a1bbd0cf0f Add session stats and VACUUM on deletion
database.py: count_sessions(), db_size() (db+WAL+SHM), vacuum()
(VACUUM + wal_checkpoint(TRUNCATE) to actually reclaim space in WAL mode)

server.py: /api/sessions now returns total_sessions + db_size;
DELETE /api/sessions/<id> runs vacuum() after deletion
2026-08-12 21:05:54 +03:00
catarrh 31011301ea Add PLI qualifier description to PROVIDE LOCAL INFORMATION decode
- PLI_QUALIFIERS: TS 102 223 qualifier values (Location Info, IMEI,
  NMR, Date/time, Timing Advance, Access Technology, etc.)
- _command_details_name(): appends the qualifier description to
  PROVIDE LOCAL INFORMATION (type 0x26) in Command Details
- decode_cat (FETCH) and decode_tr_command (TR) now use it, so both
  'cat_command' and 'response_to' carry the qualifier (e.g.
  'PROVIDE LOCAL INFORMATION — IMEI')

tests: PLI qualifier cases (TR/FETCH, IMEI/location/date, non-PLI)
2026-08-12 20:58:15 +03:00
catarrh f30fe41d75 Add per-command response decoders (FCP, AUTH, TERMINAL RESPONSE)
decode.py:
- _decode_fcp: FCP/FCI/FMD template — full bit-level file descriptor
  (TS 102 221 Table 11.5: file type, structure, shareable, data coding,
  record length, num records), file ID (+ KNOWN_FIDS name), DF name,
  SFI, life cycle
- _decode_auth: GSM SRES+Kc, or 3G tag DB (RES/CK/IK/KC) / DC (AUTS)
- _decode_tr_result: TERMINAL RESPONSE Result TLV (code + name)
- _decode_response_for(ins, data): GET RESPONSE decodes its body using
  the previous command's INS (SELECT→FCP, AUTHENTICATE→auth)
- decode_message wires response decoding for TR and GET RESPONSE

database.py: prev context now carries 'ins' (INS byte) via
_context_from_decoded, used by GET RESPONSE response decoding

tests: FCP, AUTH (GSM/3G), TR result, GET RESPONSE wiring (57 total)
2026-08-12 20:45:06 +03:00
catarrh e68df163a9 TERMINAL RESPONSE: label as 'TERMINAL RESPONSE', keep command type as context
decode.py: split decode_tr_command() out of decode_cat(); TR (0x14)
now stores the echoed proactive command type in 'response_to' instead
of 'cat_command', so it no longer overrides the INS name.

FETCH/ENVELOPE still use cat_command as the primary label.

tests: TR now asserts response_to + no cat_command
2026-08-12 20:02:45 +03:00
catarrh 083924c976 Add context-aware decoding for GET RESPONSE and TERMINAL RESPONSE
decode.py:
- decode_cat: TERMINAL RESPONSE (0x14) now extracts the proactive
  command type from its echoed Command Details (tag 81) — no history
  lookup needed since TR carries its own type
- decode_message gains prev param; GET RESPONSE (0xC0) resolves
  'response_for' from the previous command when its SW1 was 0x61,
  else None (orphaned)
- decode_sniff_msg threads prev through

database.py:
- _row_to_message accepts prev; new _decode_rows tracks previous-TPDU
  context in sequence; get_messages/get_messages_after seed context
  from the message before the window/cursor

tests: TR cat_command + GET RESPONSE context/orphaned cases (49 total)
2026-08-12 19:41:57 +03:00
catarrh 2a58fec0a3 Fix dropped change flags + FIDI parsing
- capture.py: DirectSniffer/GsmtapListener yield (type, data, flags),
  CaptureManager passes flags to insert_message (change flags were
  previously dropped, leaving every change row with flags=0)
- protocol.py: add _parse_fidi (1-byte value) — FIDI was mis-mapped to
  _parse_change (4-byte flags), so FIDI messages were silently dropped
- database.py: pass flags into decode_sniff_msg
- decode.py: decode_change(flags) reads flags directly; decode_sniff_msg
  gains flags parameter; change messages no longer require non-empty data
- tests: fix FIDI test, add change/fidi decode tests (43 total)
2026-08-12 19:25:14 +03:00
catarrh 11d0f29b6a Add CAT (TS 102 223) command decoding + fix AUTHENTICATE naming
- CAT_COMMAND_TYPES: complete Type of Command table from TS 102 223 §9.4
- ENVELOPE_TYPES: D1-D7 envelope tags
- parse_tlv(): minimal BER-TLV parser
- decode_cat(): FETCH → proactive command name (via D0→81 type byte);
  ENVELOPE → envelope type name
- AUTHENTICATE (0x88/0x89) unified to 'AUTHENTICATE' (no suffix)
- decoded dict gains 'cat_command' field for FETCH/ENVELOPE
- tests/test_decode.py: 14 new tests (TLV, naming, CAT, tables)
2026-08-12 18:26:58 +03:00
catarrh 41b5c0218e Defensive: try/except around decode in _row_to_message
Decode errors now log to stderr instead of crashing the entire
API response. This prevents a single malformed message from hiding
all subsequent messages in the same query.
2026-08-12 17:49:05 +03:00
catarrh e211dd91bc Add decode for change, fidi, atr, pps message types
decode_sniff_msg() dispatches to type-specific decoders:
- change: 4-byte flags bitmask → card insert/eject/reset/timeout
- fidi: 1-byte Fi/Di index → decoded values
- atr/pps: raw hex value
database.py now decodes all message types, not just tpdu.
2026-08-12 17:46:03 +03:00
catarrh 7ada4345b8 Add server-side APDU decoder (decode.py)
Pure Python, no dependencies. Decodes raw TPDU bytes into
structured JSON: CLA bitfield, INS->name via APDU_SPEC table,
P1/P2 field breakdown, body extraction, SW name lookup.
~40 command types, 40+ SW codes, known FID notes.

database.py: _row_to_message() calls decode_message() for tpdu
messages, adding 'decoded' field to API responses.

Decoding happens at query time, not storage time — no schema
migration, all past sessions get decode retroactively.
2026-08-12 17:21:25 +03:00
catarrh 17eeda27f3 Merge simtrace-analyser-server into simtrace2_pysniff.server
Add simtrace2-pysniff-server CLI — HTTP API for simtrace-analyser PWA.
Two capture modes: GSMTAP UDP listener (compatible with simtrace2-sniff)
or direct SIMtrace2 hardware capture. SQLite storage with sessions +
messages tables. REST API with CORS.

server/ sub-package: database.py, capture.py, server.py, __main__.py

README updated with server usage section.
pyproject.toml: simtrace2-pysniff-server script entry added.
2026-08-12 15:59:51 +03:00
catarrh a130b97cce Add GsmtapReceiver — receive GSMTAP-SIM over UDP
UDP socket on port 4729, parses GSMTAP header, validates
GSMTAP_TYPE_SIM, returns (sub_type, data). Compatible with
simtrace2-sniff and pySim's GsmtapApduSource.
2026-08-12 12:31:52 +03:00
catarrh f8e9cb1018 SIGHUP: ignore only when in background (stdin not a tty)
Use sys.stdin.isatty() to auto-detect: foreground sessions still
respond to SIGHUP (terminal close kills the process as expected),
background/piped processes ignore it and survive.
2026-08-11 01:02:07 +03:00
catarrh fb5c1fbf6a Ignore SIGHUP — survive terminal closure in background
__main__.py: add signal.SIGHUP → SIG_IGN before setting up
SIGINT/SIGTERM handlers. Sniffer no longer dies when the terminal
closes or the shell session ends.

README: add Background / Daemon section with nohup examples.
2026-08-11 00:37:41 +03:00
catarrh ec129c4463 Add --format option: timestamp and atr-time TPDU output modes
dump.py: tpdu_timestamp_prefix(), tpdu_atr_time_prefix(),
  format_message() accepts tpdu_prefix parameter,
  FileDumper supports fmt + ATR time tracking

__main__.py: --format/-f flag (hex|timestamp|atr-time),
  atr_time_start tracking in message loop

sniff-start.sh: pass through --format/-f and FORMAT env var
2026-08-10 23:53:38 +03:00
catarrh 73b0277ef7 Initial commit: simtrace2-pysniff v0.1.0
Python-based SIMtrace2 sniffer — drop-in replacement for simtrace2-sniff.
Single dependency (pyusb), auto-reconnect, GSMTAP/PCAP output, library API.

Modules:
  protocol.py — SIMtrace2 USB sniffing protocol parser
  device.py   — USB discovery + SniffSession with exponential backoff reconnect
  gsmtap.py   — GSMTAP UDP sender for Wireshark
  pcap.py     — PCAP file writer (LINKTYPE_GSMTAP)
  dump.py     — hex dump formatters + file output
  __main__.py — CLI entry point

Startup script sniff-start.sh handles udev permission checks and shows
install instructions. udev rule uses TAG+=uaccess (systemd-logind ACL).
2026-08-10 23:17:04 +03:00