simtrace-analyser/README.md

# SIMtrace Analyser

English | [Русский](README_RUS.md)

APDU trace analyser for the GSMTAP-SIM format — captures SIM/UICC
communication from a UDP GSMTAP stream (or imports PCAP/PCAPNG files),
decodes it, and serves an installable web app for browsing.  The repository
also bundles **simtrace2-pysniff**, a Python replacement for
`simtrace2-sniff` that captures from Osmocom SIMtrace2 hardware and can feed
GSMTAP into this analyser.

![SIMtrace Analyser PWA](frontend.png)

## Components

- **Analysis server + PWA** (`simtrace-analyser-server`) — stores captured
  traffic in SQLite, decodes it, and serves the PWA ([`frontend/`](frontend/))
  plus the HTTP API on a single origin.  Three capture modes: a GSMTAP
  listener, direct SIMtrace2 hardware capture, or disabled (browse existing
  captures only).
- **Capture utility** (`simtrace2-pysniff`) — captures SIMtrace2 hardware
  traffic and writes it to stdout, GSMTAP (Wireshark), or PCAP files.  It is
  one GSMTAP source of several that the analyser accepts (see also
  [sigrok_iso7816_stream](https://github.com/anttro/sigrok_iso7816_stream)).
- **PWA** ([`frontend/`](frontend/)) — trace browser with decoded APDU,
  SIM Toolkit, SMS and SCP80 OTA detail; sessions, PCAP import, EN/RU UI.

## Features

**APDU decoder** (server-side, rendered by the PWA):

- Full APDU decode — CLA per ISO 7816-4 / ETSI coding (logical channel,
  secure messaging, command chaining), INS names from a spec table,
  SFI-aware P1/P2 for READ/UPDATE BINARY and record commands
- Status words — full ISO 7816-4 set plus UICC-specific values:
  `91XX` (proactive command pending), `62F1/F2/F3`, `63F1/F2` (more data),
  `9300` (SAT busy), `9850`, `9862–64`
- SIM Toolkit — SAT (GSM 11.14), CAT (TS 102 223) and USAT (TS 31.111)
  proactive commands with decoded qualifiers, TERMINAL RESPONSE, ENVELOPE
  (Menu Selection, Call Control, SMS-PP download…); tolerates known
  non-compliant legacy-card quirks and preserves unrecognized TLVs
- SMS TPDU — SMS-DELIVER/SUBMIT, SCTS timestamps, UDH information elements
- SCP80 OTA secured packets — SPI bits (ciphering, PoR requirement/mode,
  RC/CC/DS, counter), KIc/KID algorithm + key set, TAR, CNTR/PCNTR;
  Response Packet (PoR) decode on GET RESPONSE; **optional client-side
  decryption** of ciphered packets — paste the card's 3DES/AES key in the
  detail view to reveal the real CNTR and secured application data (the
  key stays in the browser, in-memory only; ciphered on-wire CNTR/PCNTR/RC
  are ciphertext and labeled as such); the secured application message is
  decoded as a stream of **C-APDUs** (SELECT, UPDATE BINARY, …) with
  full CLA/INS/P1/P2/Lc/Le breakdown — plaintext immediately, or after
  decryption for ciphered packets
- ATR parse (clock-rate conversion, Fi/Di, T-bitmask) and FCP/FCI TLV

**SIMtrace Analyser PWA** ([`frontend/`](frontend/)):

- Trace view with Map/List modes — proactive, TERMINAL RESPONSE, ENVELOPE
  and AUTH commands get distinctive styling in map view
- Sessions with deep links (`#session=N`), PCAP import (`.pcap/.pcapng/.cap`),
  search and type filtering
- Installable PWA, dark theme, English/Russian UI

## Prerequisites

- **Python 3.9+**
- **PyUSB** — `pip install pyusb` (needed only for capture from SIMtrace2
  hardware; GSMTAP/UDP capture and PCAP import work without it)
- **libusb** — system library, present by default on Linux; on Windows the
  SIMtrace2 driver must be swapped (see [USB Device Access](#usb-device-access))
- **SIMtrace2 hardware** running firmware in *trace* mode (only for direct
  capture; VID `1d50`, PID `60e3`, USB class `ff`, subclass `01`). For
  building and flashing trace firmware, see the
  [upstream SIMtrace2 project](https://gitea.osmocom.org/sim-card/simtrace2).

No other dependencies. PCAP, GSMTAP, and hex dump output use only Python stdlib.

## Install

```sh
git clone https://github.com/anttro/simtrace-analyser.git
cd simtrace-analyser
pip install pyusb
```

Or install the package into your environment:

```sh
pip install -e .
# then run: simtrace-analyser-server [opts]
# or:       simtrace2-pysniff [opts]
```

## Running

### Analysis server + PWA (`simtrace-analyser-server`)

Stores captured APDU traffic in SQLite and serves it over HTTP, together with
the bundled **SIMtrace Analyser** PWA (`frontend/`) on the same origin.  Three
capture modes:

```sh
# Listen for GSMTAP from simtrace2-pysniff (or original simtrace2-sniff):
simtrace-analyser-server --capture gsmtap

# Capture directly from SIMtrace2 hardware (no external tool needed):
simtrace-analyser-server --capture direct

# Browse/analyse existing captures without a capture backend:
simtrace-analyser-server --capture disabled
```

The server logs its capture liveness to stderr (each line timestamped
with the local time): `Capture started` on begin, a periodic
`Capture alive: session=N messages=M bytes=B dropped=D` heartbeat, and
`Capture stopped` (or `ERROR: capture thread stopped on error …` if the
capture thread died unexpectedly). A mis-framed or non-SIM GSMTAP
datagram is dropped and counted (`dropped`) rather than silently
killing the capture. The heartbeat interval defaults to 60 s and is
tunable via `--log-interval SECONDS`.

The GSMTAP listener also accepts the **sigrok-iso7816-stream** decoder
(https://github.com/anttro/sigrok_iso7816_stream) — an FX2-logic-analyzer
passive SIM sniffer that emits the same GSMTAP-SIM format on UDP 4729. Its
standard PPS request/response sub-types (`0x02`/`0x03`) are decoded as PPS,
and its custom sub-types for RST/VCC line events (`0x10`/`0x11`, added for
compatibility with that project) are decoded as `RESET ASSERTED` /
`RST de-asserted — ATR follows` and `VCC ON/OFF` timeline markers, and a
reset-assert or power-removal
correctly invalidates the file-selection tracking. When an RST event carries
the measured CLK frequency (decoder v1.9.0+, optional payload extension), the
following ATR is annotated with the CLK frequency, data rate and ETU derived
from its TA1 Fi/Di (`data_rate = clk_hz × D / F`); the measurement is
consumed by the ATR so a later ATR without a reset gets no stale rate (a
late-arriving measurement is backfilled onto the ATR it belongs to). GSMTAP
packets carrying the `GSMTAP_FLAG_BAD_FCS` desync flag (set by that decoder
on mis-framed exchanges) are surfaced as a flagged/desynced entry in the
capture timeline, so such artefacts are easy to spot when reviewing a capture
rather than silently altering the decode.

Server options:

```
--host ADDR          HTTP bind address (default: 127.0.0.1)
--port PORT          HTTP server port (default: 8081)
--db FILE            SQLite database path (default: ~/.simtrace-analyser/sessions.db)
--capture MODE       gsmtap | direct | disabled  (default: gsmtap)
--gsmtap-port PORT   UDP port for the GSMTAP listener (default: 4729)
--web-dir PATH       PWA static files directory (default: <repo>/frontend)
```

Then open **http://127.0.0.1:8081/** in a browser — the PWA is served by the
server, so the UI and the API share an origin and no CORS/Private-Network-Access
setup is involved.  Use `--web-dir PATH` to serve a different PWA directory
(default: `<repo>/frontend`).

### Hosted PWA (landing page)

A standalone copy of the PWA is hosted at **https://simtrace.atroshin.ru**.
It is a pure frontend: point it (Settings → Server URL) at a locally running
`simtrace-analyser-server`.

> **Browser restriction:** when the PWA is served from a public HTTPS host,
> reaching a local server (`http://127.0.0.1:8081`) requires two things: the
> server must send `Access-Control-Allow-Private-Network: true` (this server
> does), and the browser must be allowed to access the local network — in
> Chrome/Edge/Vivaldi: Site settings → Local network access → allow the site
> (or accept the permission prompt).  Without the browser permission, the
> request to `127.0.0.1` is blocked before any preflight is sent.

### Capture utility (`simtrace2-pysniff`)

Captures SIMtrace2 hardware traffic and writes it to stdout, GSMTAP
(Wireshark), or PCAP files.  It is the bundled replacement for the original
Osmocom `simtrace2-sniff`.

The `./sniff-start.sh` startup script performs a pre-flight USB permission
check (prints udev install instructions if the SIMtrace2 device is not
writable) and then launches the Python module. On Windows, use
`sniff-start.bat` with the same options (no permission check needed).

```sh
./sniff-start.sh
./sniff-start.sh --format timestamp
./sniff-start.sh --gsmtap 127.0.0.1:4729
./sniff-start.sh --pcap trace.pcap --format atr-time
```

OPTIONS can also be set via environment variables:

```sh
FORMAT=timestamp GSMTAP=127.0.0.1 ./sniff-start.sh
```

Cross-platform module invocation:

```sh
python -m simtrace2_pysniff
python -m simtrace2_pysniff --format timestamp --gsmtap 127.0.0.1:4729
python -m simtrace2_pysniff --pcap trace.pcap
```

If installed via `pip install -e .`, use the console script directly:

```sh
simtrace2-pysniff --format atr-time
```

## USB Device Access

Required for capture from SIMtrace2 hardware (the utility and the server's
`--capture direct` mode); not needed for GSMTAP or PCAP sources.

### Linux

A udev rule is required for non-root USB access to the SIMtrace2 device.

```sh
sudo cp 70-simtrace2-pysniff.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger
```

Then reconnect the SIMtrace2 USB cable. The rule uses `TAG+="uaccess"`
(systemd-logind ACL) — no group membership or world-writable permissions needed.

### Windows

Windows does not ship a libusb-compatible driver for SIMtrace2. Use
[Zadig](https://zadig.akeo.ie/) to replace the default driver:

1. Download and run **Zadig**
2. Plug in the SIMtrace2 device
3. Select it in the dropdown (VID `1D50`, PID `60E3` — "Osmocom SIMtrace 2")
4. Choose **WinUSB** (or libusbK) as the replacement driver
5. Click **Replace Driver** (one-time operation)

After that, `python -m simtrace2_pysniff` works identically to Linux.

## Utility CLI Options

```
python -m simtrace2_pysniff [OPTIONS]

  --format, -f FORMAT       Output format (default: hex)
                            hex       — plain hex dump
                            timestamp — TPDU lines prefixed with [HH:MM:SS.mmm]
                            atr-time  — TPDU lines prefixed with [SSSS.mmm] from last ATR
  --gsmtap HOST[:PORT]      Send ATR/TPDU as GSMTAP over UDP (default port 4729)
  --pcap FILE               Write sniffed data to PCAP file for Wireshark
  --output, -o FILE         Write hex dump to file
  --no-stdout               Suppress hex dump to stdout
  --vendor HEX              USB vendor ID (default: 0x1d50)
  --product HEX             USB product ID (default: auto-detect)

Recovery options:
  --no-reconnect            Exit on USB disconnect instead of reconnecting
  --reconnect-delay-min SEC Minimum reconnect delay (default: 1.0)
  --reconnect-delay-max SEC Maximum reconnect delay cap (default: 30.0)
  --backoff-factor N        Exponential backoff multiplier (default: 1.5)
  --inactivity-timeout SEC  Reconnect after N seconds of silence (default: disabled)
```

## Output Formats

- **`hex`** (default): plain hex dump — `ATR: 3b 9e ...`, `TPDU: a0 a4 00 00 02 3f 00`
- **`timestamp`**: TPDU lines prefixed with local time — `[14:32:05.123] TPDU: a0 a4 00 00 02 3f 00`
- **`atr-time`**: TPDU lines prefixed with seconds since last ATR — `[0001.234] TPDU: a0 a4 00 00 02 3f 00`

Non-TPDU messages (ATR, PPS, card state changes, Fi/Di) print identically in all formats.

- **GSMTAP**: ATR and APDU messages over UDP (port 4729) to Wireshark
- **PCAP**: ATR/TPDU messages as Ethernet/IP/UDP/GSMTAP packets, openable in Wireshark

## Background / Daemon

SIGHUP is automatically ignored when stdin is not a terminal (i.e. run in
background or via a pipe). Foreground terminal sessions still respond to
SIGHUP normally.

To run persistently in the background with output to files:

```sh
nohup ./sniff-start.sh --output /tmp/sniff.log &> /tmp/sniff-err.log &
# or
FORMAT=timestamp nohup python -m simtrace2_pysniff --output /tmp/sniff.log &> /tmp/sniff-err.log &
```

## Library Usage

```python
from simtrace2_pysniff import SniffSession

session = SniffSession(inactivity_timeout=30.0)
for msg in session.iter_messages():
    print(f"{msg.type}: {msg.data.hex()}")
```

## Recovery

The utility survives hardware resets, cable disconnects, and firmware hangs
by default — it reconnects with exponential backoff (1s → 30s cap).
Use `--inactivity-timeout` to also trigger reconnect on silent firmware hangs.

## Related

- **[simtrace2](https://gitea.osmocom.org/sim-card/simtrace2)** — the upstream
  SIMtrace2 hardware/firmware project that the capture utility sniffs from.
- **[sigrok_iso7816_stream](https://github.com/anttro/sigrok_iso7816_stream)** —
  FX2LP-based passive ISO 7816 sniffer that emits the same GSMTAP-SIM stream
  on UDP 4729 (accepted by this project's GSMTAP listener).