catarrh/simple
1
0
Fork 0
Code Issues Pull Requests Actions 8 Packages Projects Releases Wiki Activity

feat: card presets move to a server-side store; the server owns the SCP80 counter (v3.8.0)

Browse Source 
The presets lived in the browser's localStorage and the PWA wrote the SCP80
counter back after every operation, so an increment could be lost when the
response never reached the tab, when two browser contexts (or the demo origin
plus a local one) held separate copies, or when the tab was closed mid-run.
The counter is card state and the server performs every card operation, so the
store now lives next to the server:

- pysim_simple_server/presets.py: one JSON file
  (~/.pysim-simple-server/card_presets.json, --card-presets overrides; Path.home()
  resolves the same way on Windows), atomic writes (temp + fsync + os.replace),
  an RLock, stable uuid ids, the ICCID normalisation the PWA uses (digits /
  spaced / raw EF hex), a duplicate-ICCID refusal and a monotonic set_counter;
  every counter change is appended to card_presets.json.audit.jsonl.
- API: GET /api/presets (card-free), POST /api/presets and
  /api/presets/update|delete|import; the import also takes the old localStorage
  export shape - that is the migration path, there is no automatic migration.
- Operations name a preset_id and the server persists the counter they
  consumed: /api/send-ota (the plain SCP80 path reports final_cntr now too),
  the RAM install/install-app/compat steps (_ram_send_gp_apdu) and the test
  runner's SCP80 steps (_preset_counter_persist).
- PWA: the Cards tab is a thin client of the store (load on connect, CRUD via
  the API, export/import), the counter write-backs became local display syncs
  (the SCP80 form's counter input stays a deliberate human edit) and the
  SCP80/RAM bodies carry preset_id; a new pysimFetchOk() surfaces the server's
  error bodies instead of silently using them.

Policy: card keys in logs and local files are not a risk for this workbench
tool (AGENTS.md) - the store is not encrypted and no redaction is required.

Tests: tests/test_presets.py (store: ICCID forms, counter monotonicity and
wrap, atomic writes, corrupt-file recovery, import), the preset endpoints in
tests/test_fetch_routes.py (incl. the counter persistence via /api/send-ota)
and the updated PWA tests.  sw cache simple-v334.
This commit is contained in:
Антон Трошин
2026-09-30 13:59:47 +03:00
parent 212bdf6fa9
commit 795a1fea49
14 changed files with 1189 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/api.md
+34
View File
@@ -31,6 +31,11 @@ a 3.x PWA).
| `/api/write` | POST | Write raw hex data to a file | | `/api/write` | POST | Write raw hex data to a file |
| `/api/apdu` | POST | Raw APDU send | | `/api/apdu` | POST | Raw APDU send |
| `/api/verify-adm` | POST | Verify the card's ADM PIN (from the matched card preset) | | `/api/verify-adm` | POST | Verify the card's ADM PIN (from the matched card preset) |
| `/api/presets` | GET | Card preset store: all presets + the store path |
| `/api/presets` | POST | Create a card preset (server-side store) |
| `/api/presets/update` | POST | Update a preset (`{id, fields}`; the counter may be raised) |
| `/api/presets/delete` | POST | Delete a preset (`{id}`) |
| `/api/presets/import` | POST | Import presets (`{presets: [...], mode: merge\|replace}`) |
| `/api/esim/chip` | GET | eUICC chip details (EID, EUICCInfo1/2, configured addresses) | | `/api/esim/chip` | GET | eUICC chip details (EID, EUICCInfo1/2, configured addresses) |
| `/api/esim/profiles` | GET | Installed eSIM profiles with their metadata | | `/api/esim/profiles` | GET | Installed eSIM profiles with their metadata |
| `/api/esim/notifications` | GET | Pending eSIM notifications (read-only) | | `/api/esim/notifications` | GET | Pending eSIM notifications (read-only) |
@@ -183,6 +188,35 @@ On a wrong key (`63Cx`, x attempts left):
A blocked ADM (`6983`/`9804`) reports `{"ok": false, "sw": "9804", A blocked ADM (`6983`/`9804`) reports `{"ok": false, "sw": "9804",
"blocked": true}` and cannot be recovered without the card's unblock key. "blocked": true}` and cannot be recovered without the card's unblock key.
### Card presets (server-side store)
Card presets (KIc/KID + keys, SPI, per-target TARs, counter, PSK pair, ADM)
live on the server in `~/.pysim-simple-server/card_presets.json`
(`--card-presets PATH` overrides it; `Path.home()` resolves the same way on
Linux, macOS and Windows). The file is written atomically and every counter
change is appended to `card_presets.json.audit.jsonl`.
The store is the source of truth for the SCP80 counter: the operations below
accept a `preset_id` and **persist the counter they consumed themselves**
(monotonic - a stale value never regresses it), so a closed tab, a lost
response or a second browser window can no longer lose an increment. The
Cards tab's export/import uses this API; the import also accepts presets
exported by the older localStorage-based builds (the store assigns ids and
applies the form defaults).
- `GET /api/presets` — `{"path": "…", "count": 2, "version": 1, "presets": [{…}]}`
(card-free: it answers while a long card operation runs).
- `POST /api/presets` — body = the preset fields; returns `{"ok": true, "preset": {…}}`.
A duplicate ICCID (digits, spaced or raw EF hex are normalised to one form)
is refused with `400 {"error": "card with this ICCID already exists: …"}`.
- `POST /api/presets/update` — `{"id": "…", "fields": {…}}` (partial update;
a plain `{id, name, …}` body works too). The counter is written as given -
this is the deliberate human edit; unknown ids answer `404`.
- `POST /api/presets/delete` — `{"id": "…"}` → `{"ok": true, "removed": true}`.
- `POST /api/presets/import` — `{"presets": […], "mode": "merge"}` (or
`"replace"`) → `{"ok": true, "added": N, "skipped": M, "errors": […]}`;
invalid entries and duplicates are skipped and reported, never fatal.
### eSIM / LPA (local ES10 operations) ### eSIM / LPA (local ES10 operations)
These endpoints work only when the equipped card is an eUICC (SGP.22/32); These endpoints work only when the equipped card is an eUICC (SGP.22/32);
frontend/help-ru.html
+1 -1
View File
@@ -340,7 +340,7 @@
</section> </section>
<section class="mb-10"> <section class="mb-10">
<h2 id="cards" class="text-xl font-semibold mb-3 border-b border-gray-300 dark:border-slate-700 pb-1">4. Вкладка &laquo;Карты&raquo;</h2> <h2 id="cards" class="text-xl font-semibold mb-3 border-b border-gray-300 dark:border-slate-700 pb-1">4. Вкладка &laquo;Карты&raquo;</h2>
<p class="mb-2">Хранит предустановки карт локально в браузере (<code class="font-mono text-sm">localStorage</code>), чтобы представление Secured Packet могло автоматически подставлять ключи и параметры, а слушатель SCP81 HTTP OTA — находить PSK-ключи. &laquo;Карты&raquo; — верхнеуровневая вкладка. При подключении карты читается её EF.ICCID, и предустановка с тем же ICCID автоматически выбирается в обоих представлениях SCP80. Поля формы сгруппированы в два блока с рамкой: <strong>SCP80 (GSM 03.48, ETSI TS 102 225)</strong> (KIc/KID, SPI1/SPI2, TAR-ы ISD/UICC RFM/ADF RFM, счётчик и ключи) и <strong>SCP81 (HTTP OTA)</strong> (пара PSK identity/ключ с пояснением о выборе ключа). Необязательное поле <strong>ADM</strong> находится в верхней строке и сохраняется для последующего использования файловым менеджером. В шапке отображаются серые маркеры <strong>SCP80</strong>/<strong>SCP81</strong> и значок ключа на бейдже ADM, когда в предустановке с ICCID подключённой карты эти поля заполнены.</p> <p class="mb-2">Хранит предустановки карт на сервере (<code class="font-mono text-sm">~/.pysim-simple-server/card_presets.json</code>; переопределяется <code class="font-mono text-sm">--card-presets</code>), чтобы представление Secured Packet могло автоматически подставлять ключи и параметры, слушатель SCP81 HTTP OTA — находить PSK-ключи, а счётчик SCP80, израсходованный операцией, сохранялся самой операцией &mdash; закрытая вкладка или второе окно браузера больше не могут потерять инкремент. Список загружается при подключении PWA и экспортируется/импортируется как JSON (импорт принимает и предустановки, выгруженные старыми сборками с хранением в браузере). &laquo;Карты&raquo; — верхнеуровневая вкладка. При подключении карты читается её EF.ICCID, и предустановка с тем же ICCID автоматически выбирается в обоих представлениях SCP80. Поля формы сгруппированы в два блока с рамкой: <strong>SCP80 (GSM 03.48, ETSI TS 102 225)</strong> (KIc/KID, SPI1/SPI2, TAR-ы ISD/UICC RFM/ADF RFM, счётчик и ключи) и <strong>SCP81 (HTTP OTA)</strong> (пара PSK identity/ключ с пояснением о выборе ключа). Необязательное поле <strong>ADM</strong> находится в верхней строке и сохраняется для последующего использования файловым менеджером. В шапке отображаются серые маркеры <strong>SCP80</strong>/<strong>SCP81</strong> и значок ключа на бейдже ADM, когда в предустановке с ICCID подключённой карты эти поля заполнены.</p>
<table class="w-full text-sm mb-3 border-collapse"> <table class="w-full text-sm mb-3 border-collapse">
<thead><tr class="border-b border-gray-300 dark:border-slate-700"><th class="text-left py-1 px-2">Поле</th><th class="text-left py-1 px-2">Описание</th></tr></thead> <thead><tr class="border-b border-gray-300 dark:border-slate-700"><th class="text-left py-1 px-2">Поле</th><th class="text-left py-1 px-2">Описание</th></tr></thead>
<tbody> <tbody>
frontend/help.html
+1 -1
View File
@@ -339,7 +339,7 @@
</section> </section>
<section class="mb-10"> <section class="mb-10">
<h2 id="cards" class="text-xl font-semibold mb-3 border-b border-gray-300 dark:border-slate-700 pb-1">4. Cards tab</h2> <h2 id="cards" class="text-xl font-semibold mb-3 border-b border-gray-300 dark:border-slate-700 pb-1">4. Cards tab</h2>
<p class="mb-2">Stores card presets locally in the browser (<code class="font-mono text-sm">localStorage</code>) so the Secured Packet view can auto-fill keys and parameters, and so the SCP81 HTTP OTA listener can look up PSK keys. Cards is a top-level tab. When the card is equipped its EF.ICCID is read and the preset with the same ICCID is selected automatically in both SCP80 views. The form groups the fields into two bordered blocks: <strong>SCP80 (GSM 03.48, ETSI TS 102 225)</strong> (KIc/KID, SPI1/SPI2, the ISD/UICC RFM/ADF RFM TARs, counter and keys) and <strong>SCP81 (HTTP OTA)</strong> (the PSK identity/key pair with the key-selection note). The optional <strong>ADM</strong> field sits in the top row and is stored for later use by the file manager. The header shows gray <strong>SCP80</strong>/<strong>SCP81</strong> markers and a key glyph on the ADM badge when the preset matching the equipped card's ICCID has those settings filled in.</p> <p class="mb-2">Stores card presets on the server (<code class="font-mono text-sm">~/.pysim-simple-server/card_presets.json</code>; <code class="font-mono text-sm">--card-presets</code> overrides it) so the Secured Packet view can auto-fill keys and parameters, the SCP81 HTTP OTA listener can look up PSK keys, and the SCP80 counter an operation consumed is persisted with the operation itself &mdash; a closed tab or a second browser window can no longer lose an increment. The list is loaded when the PWA connects and can be exported/imported as JSON (the import also takes presets exported by older browser-storage builds). Cards is a top-level tab. When the card is equipped its EF.ICCID is read and the preset with the same ICCID is selected automatically in both SCP80 views. The form groups the fields into two bordered blocks: <strong>SCP80 (GSM 03.48, ETSI TS 102 225)</strong> (KIc/KID, SPI1/SPI2, the ISD/UICC RFM/ADF RFM TARs, counter and keys) and <strong>SCP81 (HTTP OTA)</strong> (the PSK identity/key pair with the key-selection note). The optional <strong>ADM</strong> field sits in the top row and is stored for later use by the file manager. The header shows gray <strong>SCP80</strong>/<strong>SCP81</strong> markers and a key glyph on the ADM badge when the preset matching the equipped card's ICCID has those settings filled in.</p>
<table class="w-full text-sm mb-3 border-collapse"> <table class="w-full text-sm mb-3 border-collapse">
<thead><tr class="border-b border-gray-300 dark:border-slate-700"><th class="text-left py-1 px-2">Field</th><th class="text-left py-1 px-2">Description</th></tr></thead> <thead><tr class="border-b border-gray-300 dark:border-slate-700"><th class="text-left py-1 px-2">Field</th><th class="text-left py-1 px-2">Description</th></tr></thead>
<tbody> <tbody>
frontend/index.html
+125 -61
View File
@@ -1750,7 +1750,7 @@
// ===== Version ===== // ===== Version =====
// Single source of truth for the PWA version: shown in the header and used // Single source of truth for the PWA version: shown in the header and used
// by the server version check in pysimConnect(). // by the server version check in pysimConnect().
const SIMPLE_VERSION = '3.7.1'; const SIMPLE_VERSION = '3.8.0';
document.getElementById('app-version').textContent = 'v' + SIMPLE_VERSION; document.getElementById('app-version').textContent = 'v' + SIMPLE_VERSION;
// ===== Tab switching ===== // ===== Tab switching =====
@@ -1768,7 +1768,7 @@ function switchTab(name) {
active.classList.add('bg-blue-600', 'text-white', 'dark:bg-blue-500', 'dark:text-white'); active.classList.add('bg-blue-600', 'text-white', 'dark:bg-blue-500', 'dark:text-white');
if (name === 'c-apdu') cApduSwitchSubtab('sim'); if (name === 'c-apdu') cApduSwitchSubtab('sim');
else if (name === 'scp80') setHelpAnchor(scp80HelpAnchor); else if (name === 'scp80') setHelpAnchor(scp80HelpAnchor);
else if (name === 'cards') { cardsRender(); setHelpAnchor('cards'); } else if (name === 'cards') { cardsRender(); cardsFetch(); setHelpAnchor('cards'); }
else if (name === 'pysim') { setHelpAnchor(pysimHelpAnchor); pysimFsFitTree(); } else if (name === 'pysim') { setHelpAnchor(pysimHelpAnchor); pysimFsFitTree(); }
else if (name === 'profiler') { profilerSetView('list'); setHelpAnchor('profiler'); } else if (name === 'profiler') { profilerSetView('list'); setHelpAnchor('profiler'); }
else if (name === 'phone') phoneSwitchSubtab('phone'); else if (name === 'phone') phoneSwitchSubtab('phone');
@@ -7507,8 +7507,8 @@ function testWriteBackCounter() {
if (!preset) return; if (!preset) return;
_testLastPresetIdx = idx; _testLastPresetIdx = idx;
if (String(preset.cntr || '').toUpperCase() === next) return; if (String(preset.cntr || '').toUpperCase() === next) return;
preset.cntr = next; preset.cntr = next; // the server persisted it with the run (v3.8.0)
cardsSave(); cardsRender();
ioStatus('test-io-status', t('SCP80 counter saved to the preset:') + ' ' + next); ioStatus('test-io-status', t('SCP80 counter saved to the preset:') + ' ' + next);
} }
@@ -7559,6 +7559,14 @@ async function pysimFetch(path, body) {
return await res.json(); return await res.json();
} }
// The same call, but a response carrying `error` (the server's 400/404/503
// bodies) throws - the caller shows the message instead of silently using it.
async function pysimFetchOk(path, body) {
const data = await pysimFetch(path, body);
if (data && data.error) throw new Error(data.error);
return data;
}
// ===== UI availability (server / card) ===== // ===== UI availability (server / card) =====
let _pysimServerAvailable = null; // null until probed let _pysimServerAvailable = null; // null until probed
let _pysimCardEquipped = false; let _pysimCardEquipped = false;
@@ -7981,6 +7989,7 @@ async function pysimConnect() {
pysimFsRefresh(); pysimFsRefresh();
pysimPollStatusInit(); pysimPollStatusInit();
pysimStartBackendPoll(); pysimStartBackendPoll();
cardsFetch(); // the server store holds the card presets (v3.8.0)
btn.textContent = t('Connected'); btn.textContent = t('Connected');
stkCheckMenu(); stkCheckMenu();
} catch (e) { } catch (e) {
@@ -8099,20 +8108,49 @@ function spNextCntr(value) {
return ((v + 1) % 0x10000000000).toString(16).toUpperCase().padStart(10, '0'); return ((v + 1) % 0x10000000000).toString(16).toUpperCase().padStart(10, '0');
} }
function spCntrSyncPreset() { // The index of the preset selected in the SCP80 (Secured packet) view.
// Persist the counter into the selected card preset: the OTA counter is function spPresetIdx() {
// part of the card's state and must survive reloads.
const sel = document.getElementById('sp-card-sel'); const sel = document.getElementById('sp-card-sel');
const idx = sel ? parseInt(sel.value, 10) : NaN; const idx = sel ? parseInt(sel.value, 10) : NaN;
if (isNaN(idx) || !cards[idx]) return false; return (!isNaN(idx) && cards[idx]) ? idx : -1;
cards[idx].cntr = document.getElementById('sp-cntr').value; }
cardsSave();
// The SCP80 form's counter input: a deliberate human edit, written to the
// server-side preset (v3.8.0). The counter an *operation* accepted is
// persisted by the server itself - the send/RAM flows only sync the local copy
// through spCntrLocalSync().
function spCntrSyncPreset() {
const idx = spPresetIdx();
if (idx < 0) return false;
const preset = cards[idx];
const value = (document.getElementById('sp-cntr').value || '').trim().toUpperCase();
preset.cntr = value;
cardsRender(); cardsRender();
if (preset.id) {
pysimFetch('/api/presets/update', { id: preset.id, fields: { cntr: value } })
.then(res => {
if (res && res.error) throw new Error(res.error);
// the store normalises the counter (fixed 10-hex form)
if (res && res.preset) { preset.cntr = res.preset.cntr; cardsRender(); }
})
.catch(e => ioStatus('cards-io', t('Counter not saved:') + ' ' + e.message));
}
return true; return true;
} }
// Keep the selected preset in step with a counter the server already persisted
// (the operations report it as final_cntr): display only, no write.
function spCntrLocalSync(value) {
const idx = spPresetIdx();
if (idx < 0) return;
cards[idx].cntr = value;
cardsRender();
}
function spParams() { function spParams() {
const idx = spPresetIdx();
const p = { const p = {
preset_id: idx >= 0 ? (cards[idx].id || '') : '',
spi1: document.getElementById('sp-spi1').value, spi1: document.getElementById('sp-spi1').value,
spi2: document.getElementById('sp-spi2-hex').value, spi2: document.getElementById('sp-spi2-hex').value,
kic: document.getElementById('sp-kic-hex').value, kic: document.getElementById('sp-kic-hex').value,
@@ -8219,8 +8257,8 @@ async function pysimSendOta() {
msg += ' | CNTR -> ' + cntrEl.value; msg += ' | CNTR -> ' + cntrEl.value;
document.getElementById('sp-result').value = ''; document.getElementById('sp-result').value = '';
spShowSizeInfo(); spShowSizeInfo();
// keep the selected card preset in sync with the new counter // the server persisted the accepted counter; sync the display
spCntrSyncPreset(); spCntrLocalSync(cntrEl.value);
} else { } else {
msg += ' | CNTR unchanged (' + ((por && por.response_status) || 'PoR error') + ')'; msg += ' | CNTR unchanged (' + ((por && por.response_status) || 'PoR error') + ')';
} }
@@ -8692,7 +8730,13 @@ function scriptsCapFileChanged() {
// Collect SP params from the SP form (populated by ramApplyCard -> cardsApply) // Collect SP params from the SP form (populated by ramApplyCard -> cardsApply)
function getRamSpParams() { function getRamSpParams() {
// `preset_id` names the server-side preset the operation belongs to: the
// server persists the counter it consumes there (v3.8.0).
const sel = document.getElementById('ram-card-sel');
const idx = sel ? parseInt(sel.value, 10) : NaN;
const preset = (!isNaN(idx) && cards[idx]) ? cards[idx] : null;
return { return {
preset_id: (preset && preset.id) || '',
spi1: document.getElementById('sp-spi1').value, spi1: document.getElementById('sp-spi1').value,
spi2: document.getElementById('sp-spi2-hex').value, spi2: document.getElementById('sp-spi2-hex').value,
kic: document.getElementById('sp-kic-hex').value, kic: document.getElementById('sp-kic-hex').value,
@@ -8710,12 +8754,13 @@ function ramIncrementCntr(cntr) {
} }
function ramSaveCntr(cntr) { function ramSaveCntr(cntr) {
// Keep the local copy in step with the counter the server persisted for the
// operation (v3.8.0): display only - the store already holds it.
const el = document.getElementById('sp-cntr'); const el = document.getElementById('sp-cntr');
el.value = cntr; if (el) el.value = cntr;
const selIdx = parseInt(document.getElementById('ram-card-sel').value, 10); const selIdx = parseInt((document.getElementById('ram-card-sel') || {}).value, 10);
if (!isNaN(selIdx) && cards[selIdx]) { if (!isNaN(selIdx) && cards[selIdx]) {
cards[selIdx].cntr = cntr; cards[selIdx].cntr = cntr;
cardsSave();
cardsRender(); cardsRender();
ramRender(); ramRender();
} }
@@ -10226,6 +10271,7 @@ async function ramCompatTest(sp) {
const body = { const body = {
cap_hex: await ramReadFileHex(file), cap_hex: await ramReadFileHex(file),
probe_imports: parsed.map, probe_additions: parsed.additions, probe_imports: parsed.map, probe_additions: parsed.additions,
preset_id: sp.preset_id,
spi1: sp.spi1, spi2: '01', kic: sp.kic, kid: sp.kid, spi1: sp.spi1, spi2: '01', kic: sp.kic, kid: sp.kid,
tar: sp.tar, cntr: sp.cntr, kicKey: sp.kicKey, kidKey: sp.kidKey, tar: sp.tar, cntr: sp.cntr, kicKey: sp.kicKey, kidKey: sp.kidKey,
}; };
@@ -10298,6 +10344,7 @@ async function ramInstallCap(sp) {
const body = { const body = {
cap_hex: capHex, cap_hex: capHex,
preset_id: sp.preset_id,
sd_aid: (document.getElementById('ram-sd-aid').value || '').replace(/[^0-9a-fA-F]/g, ''), sd_aid: (document.getElementById('ram-sd-aid').value || '').replace(/[^0-9a-fA-F]/g, ''),
privileges: (document.getElementById('rc-priv').value || '00').replace(/[^0-9a-fA-F]/g, ''), privileges: (document.getElementById('rc-priv').value || '00').replace(/[^0-9a-fA-F]/g, ''),
install_params: (ramInstallParamsPrefix() + install_params: (ramInstallParamsPrefix() +
@@ -10350,6 +10397,7 @@ async function ramInstallApp(sp, op) {
} }
const body = { const body = {
mode: mode, mode: mode,
preset_id: sp.preset_id,
loadfile_aid: loadfile, loadfile_aid: loadfile,
module_aid: module, module_aid: module,
instance_aid: instance, instance_aid: instance,
@@ -10839,16 +10887,35 @@ let cards = [];
let _cardsEditIdx = null; let _cardsEditIdx = null;
function cardsLoad() { function cardsLoad() {
try { // Presets live on the server now (v3.8.0): /api/presets fills this array
const d = localStorage.getItem('simple_cards'); // on connect. localStorage is deliberately ignored - the Cards tab's
cards = d ? JSON.parse(d) : []; // export/import (the same JSON shape) moves old browser presets over.
} catch (e) { cards = []; } cards = [];
cardsRebuildSelect(); cardsRebuildSelect();
cardsRender(); cardsRender();
} }
// Load the presets from the server store. Called on connect, after every
// Cards-tab mutation and when the tab is opened; operations keep the local
// copy in step through the counter they report back.
async function cardsFetch() {
try {
const data = await pysimFetchOk('/api/presets');
cards = Array.isArray(data.presets) ? data.presets : [];
} catch (e) {
return false; // server optional at this point; the tab fills on connect
}
cardsRebuildSelect();
cardsRender();
scp81PushPskMap();
return true;
}
function cardsSave() { function cardsSave() {
localStorage.setItem('simple_cards', JSON.stringify(cards)); // Nothing to persist here: the server store is the source of truth (the
// Cards tab writes through /api/presets*, operations persist their own
// accepted counters). Kept as the "the list changed" hook for the
// selects, the table and the SCP81 PSK map.
cardsRebuildSelect(); cardsRebuildSelect();
cardsRender(); cardsRender();
scp81PushPskMap(); scp81PushPskMap();
@@ -10924,7 +10991,7 @@ function cardsClearForm() {
}); });
} }
function cardsAdd() { async function cardsAdd() {
const v = cardsFormValues(); const v = cardsFormValues();
// empty TAR fields fall back to the per-target spec defaults // empty TAR fields fall back to the per-target spec defaults
v.tar = cardsTarValue('tar', v.tar); v.tar = cardsTarValue('tar', v.tar);
@@ -10946,16 +11013,17 @@ function cardsAdd() {
alert(t('Card with this ICCID already exists') + ': ' + (cards[dupIdx].name || ('Card ' + dupIdx))); alert(t('Card with this ICCID already exists') + ': ' + (cards[dupIdx].name || ('Card ' + dupIdx)));
return; return;
} }
if (_cardsEditIdx !== null && cards[_cardsEditIdx]) { // The server store is the source of truth (v3.8.0): create or update
cards[_cardsEditIdx] = Object.assign({}, cards[_cardsEditIdx], v); // through the API, then reload - it also enforces the duplicate-ICCID rule.
} else { const editing = (_cardsEditIdx !== null && cards[_cardsEditIdx]) ? cards[_cardsEditIdx] : null;
cards.push(Object.assign({}, v, { try {
spi1: v.spi1 || '16', if (editing) await pysimFetchOk('/api/presets/update', { id: editing.id, fields: v });
spi2: v.spi2 || '01', else await pysimFetchOk('/api/presets', v);
cntr: v.cntr || '0000000001', } catch (e) {
})); alert(t('Save error') + ': ' + e.message);
return;
} }
cardsSave(); await cardsFetch();
cardsCancelEdit(); cardsCancelEdit();
} }
@@ -10990,12 +11058,19 @@ function ramCardIdxAfterRemove(idx, removedIdx) {
return idx > removedIdx ? idx - 1 : idx; return idx > removedIdx ? idx - 1 : idx;
} }
function cardsRemove(i) { async function cardsRemove(i) {
const c = cards[i];
if (!c) return;
try {
await pysimFetchOk('/api/presets/delete', { id: c.id });
} catch (e) {
alert(t('Delete error') + ': ' + e.message);
return;
}
_ramCardIdx = ramCardIdxAfterRemove(_ramCardIdx, i); _ramCardIdx = ramCardIdxAfterRemove(_ramCardIdx, i);
cards.splice(i, 1);
if (_cardsEditIdx === i) _cardsEditIdx = null; if (_cardsEditIdx === i) _cardsEditIdx = null;
else if (_cardsEditIdx !== null && _cardsEditIdx > i) _cardsEditIdx--; else if (_cardsEditIdx !== null && _cardsEditIdx > i) _cardsEditIdx--;
cardsSave(); await cardsFetch();
} }
function cardsRender() { function cardsRender() {
@@ -11322,36 +11397,22 @@ function cardsExportFile() {
downloadJson('simple-cards.json', cards); downloadJson('simple-cards.json', cards);
} }
function cardsImport(text) { async function cardsImport(text) {
let imported;
try { try {
const imported = JSON.parse(text); imported = JSON.parse(text);
if (!Array.isArray(imported)) throw new Error('Expected an array'); if (!Array.isArray(imported)) throw new Error('Expected an array');
let added = 0; } catch (e) {
for (const entry of imported) { alert('Import error: ' + e.message);
if (!entry.name) continue; return;
if (entry.iccid && cards.some(c => c.iccid === entry.iccid)) continue; }
cards.push({ try {
name: entry.name, // The server validates and normalises the entries (the old localStorage
iccid: entry.iccid, // export shape included) and reports what it skipped.
adm: (entry.adm || '').replace(/\s/g, '').toUpperCase(), const res = await pysimFetchOk('/api/presets/import', { presets: imported });
kic: entry.kic || '15', await cardsFetch();
kid: entry.kid || '15', ioStatus('cards-io', 'Imported ' + res.added + ' cards'
spi1: entry.spi1 || '16', + (res.skipped ? ' (' + res.skipped + ' skipped)' : '') + '.');
spi2: entry.spi2 || '01',
tar: cardsTarValue('tar', entry.tar),
uiccTar: cardsTarValue('uiccTar', entry.uiccTar),
usimTar: cardsTarValue('usimTar', entry.usimTar),
cntr: entry.cntr || '0000000001',
kicKey: entry.kicKey || '',
kidKey: entry.kidKey || '',
pskIdentity: entry.pskIdentity || '',
pskKey: (entry.pskKey || '').replace(/\s/g, '').toUpperCase(),
});
added++;
}
cardsSave();
cardsRender();
ioStatus('cards-io', 'Imported ' + added + ' cards.');
} catch (e) { } catch (e) {
alert('Import error: ' + e.message); alert('Import error: ' + e.message);
} }
@@ -19035,6 +19096,9 @@ const LANG_RU = {
'SCP81 HTTP OTA: the listener picks the key by the PSK identity the card sends in the TLS handshake. Cards without a PSK are ignored by the listener.': 'SCP81 HTTP OTA: слушатель выбирает ключ по PSK identity, которую карта присылает в TLS-рукопожатии. Карты без PSK слушателем игнорируются.', 'SCP81 HTTP OTA: the listener picks the key by the PSK identity the card sends in the TLS handshake. Cards without a PSK are ignored by the listener.': 'SCP81 HTTP OTA: слушатель выбирает ключ по PSK identity, которую карта присылает в TLS-рукопожатии. Карты без PSK слушателем игнорируются.',
'Card with this ICCID already exists': 'Карта с таким ICCID уже есть в списке', 'Card with this ICCID already exists': 'Карта с таким ICCID уже есть в списке',
'Card equipped but its ICCID is not readable': 'Карта подключена, но её ICCID не читается', 'Card equipped but its ICCID is not readable': 'Карта подключена, но её ICCID не читается',
'Save error': 'Ошибка сохранения',
'Delete error': 'Ошибка удаления',
'Counter not saved:': 'Счётчик не сохранён:',
'Saved.': 'Сохранено.', 'Saved.': 'Сохранено.',
'Invalid APDU': 'Некорректный APDU', 'Invalid APDU': 'Некорректный APDU',
'Add at least one APDU': 'Добавьте хотя бы один APDU', 'Add at least one APDU': 'Добавьте хотя бы один APDU',
frontend/sw.js
+1 -1
View File
@@ -1,4 +1,4 @@
const CACHE = 'simple-v333'; const CACHE = 'simple-v334';
const URLS = [ const URLS = [
'index.html', 'index.html',
'help.html', 'help.html',
frontend/tests/cards_form.test.js
+90 -24
View File
@@ -5,7 +5,7 @@ const path = require('node:path');
const html = fs.readFileSync(path.join(__dirname, '..', 'index.html'), 'utf8'); const html = fs.readFileSync(path.join(__dirname, '..', 'index.html'), 'utf8');
function extractFunc(src, name) { function extractFunc(src, name, asyncFn) {
const re = new RegExp('function\\s+' + name + '\\s*\\([^)]*\\)\\s*\\{'); const re = new RegExp('function\\s+' + name + '\\s*\\([^)]*\\)\\s*\\{');
const m = re.exec(src); const m = re.exec(src);
if (!m) throw new Error('function ' + name + ' not found'); if (!m) throw new Error('function ' + name + ' not found');
@@ -18,15 +18,18 @@ function extractFunc(src, name) {
if (depth === 0) break; if (depth === 0) break;
} }
} }
return src.slice(m.index, i + 1); return (asyncFn ? 'async ' : '') + src.slice(m.index, i + 1);
} }
let code = ''; let code = '';
const ASYNC_FNS = ['cardsAdd', 'cardsImport', 'cardsRemove'];
for (const fn of ['cardsTarValue', 'cardsFormValues', 'cardsClearForm', 'cardsEdit', for (const fn of ['cardsTarValue', 'cardsFormValues', 'cardsClearForm', 'cardsEdit',
'cardsAdd', 'cardsImport', 'cardsApplyFields', 'cardsApply', 'ramApplyCard', 'cardsAdd', 'cardsImport', 'cardsRemove', 'ramCardIdxAfterRemove',
'spPresetIdx', 'spCntrSyncPreset', 'spCntrLocalSync', 'ramSaveCntr',
'cardsApplyFields', 'cardsApply', 'ramApplyCard',
'cardsScp80Complete', 'cardsScp81Complete', 'cardsAdmPresent', 'cardsScp80Complete', 'cardsScp81Complete', 'cardsAdmPresent',
'spTarKeyForPack', 'spPresetTar', 'packToSp']) { 'spTarKeyForPack', 'spPresetTar', 'packToSp']) {
code += extractFunc(html, fn) + '\n'; code += extractFunc(html, fn, ASYNC_FNS.indexOf(fn) >= 0) + '\n';
} }
eval(code); eval(code);
@@ -51,12 +54,15 @@ function fakeEl() {
}; };
} }
let apiCalls = [];
function setup() { function setup() {
const els = {}; const els = {};
for (const id of CARD_IDS.concat(SP_IDS, CHAIN_IDS)) els[id] = fakeEl(); for (const id of CARD_IDS.concat(SP_IDS, CHAIN_IDS)) els[id] = fakeEl();
globalThis.document = { getElementById: id => els[id] || null, querySelectorAll: () => [] }; globalThis.document = { getElementById: id => els[id] || null, querySelectorAll: () => [] };
globalThis.cards = []; globalThis.cards = [];
globalThis._cardsEditIdx = null; globalThis._cardsEditIdx = null;
globalThis._ramCardIdx = null;
globalThis._spTarKey = 'tar'; globalThis._spTarKey = 'tar';
globalThis.t = s => s; globalThis.t = s => s;
globalThis.alert = () => {}; globalThis.alert = () => {};
@@ -75,6 +81,19 @@ function setup() {
globalThis.spInvalidate = () => {}; globalThis.spInvalidate = () => {};
globalThis.updateSp = () => {}; globalThis.updateSp = () => {};
globalThis.genSp = () => {}; globalThis.genSp = () => {};
globalThis.ramRender = () => {};
// the server store is the source of truth (v3.8.0): the Cards tab talks to
// /api/presets* and reloads through cardsFetch()
apiCalls = [];
globalThis.pysimFetchOk = async (path, body) => {
apiCalls.push({ path: path, body: body });
return { ok: true, added: 1, skipped: 0 };
};
globalThis.pysimFetch = async (path, body) => {
apiCalls.push({ path: path, body: body });
return { ok: true };
};
globalThis.cardsFetch = async () => true;
return els; return els;
} }
@@ -152,7 +171,7 @@ test('the form values carry the ADM and per-target TAR fields', () => {
assert.strictEqual(v.adm, '00112233'); assert.strictEqual(v.adm, '00112233');
}); });
test('saving a preset stores spec defaults for empty TARs', () => { test('saving a preset posts the form with spec-default TARs', async () => {
const els = setup(); const els = setup();
els['cards-name'].value = 'New card'; els['cards-name'].value = 'New card';
els['cards-kic'].value = '15'; els['cards-kic'].value = '15';
@@ -163,30 +182,77 @@ test('saving a preset stores spec defaults for empty TARs', () => {
els['cards-uicc-tar'].value = ''; els['cards-uicc-tar'].value = '';
els['cards-usim-tar'].value = ''; els['cards-usim-tar'].value = '';
els['cards-adm'].value = 'aa bb'; els['cards-adm'].value = 'aa bb';
cardsAdd(); await cardsAdd();
assert.strictEqual(cards.length, 1); assert.strictEqual(apiCalls.length, 1, JSON.stringify(apiCalls));
assert.strictEqual(cards[0].tar, '000000'); assert.strictEqual(apiCalls[0].path, '/api/presets');
assert.strictEqual(cards[0].uiccTar, 'B00000'); assert.strictEqual(apiCalls[0].body.name, 'New card');
assert.strictEqual(cards[0].usimTar, 'B00001'); assert.strictEqual(apiCalls[0].body.tar, '000000');
assert.strictEqual(cards[0].adm, 'AABB'); assert.strictEqual(apiCalls[0].body.uiccTar, 'B00000');
assert.strictEqual(cards[0].spi1, '16'); assert.strictEqual(apiCalls[0].body.usimTar, 'B00001');
assert.strictEqual(apiCalls[0].body.adm, 'AABB');
}); });
test('import fills missing TARs with spec defaults and normalizes ADM', () => { test('editing a preset updates it by its store id', async () => {
const els = setup();
globalThis.cards = [{ id: 'abc123', name: 'Old', kic: '15', kid: '15',
kicKey: 'AA', kidKey: 'BB' }];
globalThis._cardsEditIdx = 0;
els['cards-name'].value = 'Renamed';
els['cards-kic'].value = '15';
els['cards-kid'].value = '15';
els['cards-kic-key'].value = 'AA';
els['cards-kid-key'].value = 'BB';
await cardsAdd();
assert.strictEqual(apiCalls[0].path, '/api/presets/update');
assert.strictEqual(apiCalls[0].body.id, 'abc123');
assert.strictEqual(apiCalls[0].body.fields.name, 'Renamed');
});
test('removing a preset deletes it in the store', async () => {
setup(); setup();
cardsImport(JSON.stringify([ globalThis.cards = [{ id: 'abc123', name: 'A' }, { id: 'def456', name: 'B' }];
await cardsRemove(0);
assert.strictEqual(apiCalls.length, 1);
assert.strictEqual(apiCalls[0].path, '/api/presets/delete');
assert.strictEqual(apiCalls[0].body.id, 'abc123');
});
test('import posts the entries to the server store', async () => {
setup();
await cardsImport(JSON.stringify([
{ name: 'Imported', kic: '15', kid: '15', adm: ' aa bb ' }, { name: 'Imported', kic: '15', kid: '15', adm: ' aa bb ' },
{ name: 'Explicit', tar: '000009', uiccTar: 'B0000C', usimTar: 'B0000D', adm: 'cc' }, { name: 'Explicit', tar: '000009' },
])); ]));
assert.strictEqual(cards.length, 2); assert.strictEqual(apiCalls.length, 1);
assert.strictEqual(cards[0].tar, '000000'); assert.strictEqual(apiCalls[0].path, '/api/presets/import');
assert.strictEqual(cards[0].uiccTar, 'B00000'); assert.strictEqual(apiCalls[0].body.presets.length, 2);
assert.strictEqual(cards[0].usimTar, 'B00001'); assert.strictEqual(apiCalls[0].body.presets[0].name, 'Imported');
assert.strictEqual(cards[0].adm, 'AABB'); // a payload that is not an array never reaches the server
assert.strictEqual(cards[1].tar, '000009'); apiCalls = [];
assert.strictEqual(cards[1].uiccTar, 'B0000C'); await cardsImport('{"nope": 1}');
assert.strictEqual(cards[1].usimTar, 'B0000D'); assert.strictEqual(apiCalls.length, 0);
assert.strictEqual(cards[1].adm, 'CC'); });
test('the SCP80 counter edit is written to the store, a reported counter only locally', () => {
const els = setup();
globalThis.cards = [{ id: 'abc123', name: 'C', cntr: '0000000001' }];
els['sp-card-sel'].value = '0';
els['sp-cntr'].value = '0000000009';
assert.strictEqual(spCntrSyncPreset(), true);
assert.strictEqual(cards[0].cntr, '0000000009');
assert.strictEqual(apiCalls.length, 1);
assert.strictEqual(apiCalls[0].path, '/api/presets/update');
assert.strictEqual(apiCalls[0].body.id, 'abc123');
assert.deepStrictEqual(apiCalls[0].body.fields, { cntr: '0000000009' });
// a counter the server already persisted: display only, no second write
apiCalls = [];
spCntrLocalSync('000000000A');
assert.strictEqual(cards[0].cntr, '000000000A');
assert.strictEqual(apiCalls.length, 0);
// no preset selected: nothing to write
els['sp-card-sel'].value = '';
assert.strictEqual(spCntrSyncPreset(), false);
assert.strictEqual(apiCalls.length, 0);
}); });
test('applying a preset uses the TAR of the current operation', () => { test('applying a preset uses the TAR of the current operation', () => {
frontend/tests/ram_counter_flow.test.js
+4 -3
View File
@@ -50,7 +50,7 @@ function fakeEnv(por, success) {
els['sp-kid-key'].value = 'BB'; els['sp-kid-key'].value = 'BB';
els['sp-spi2-hex'].value = '01'; els['sp-spi2-hex'].value = '01';
globalThis.document = { getElementById: id => els[id] || null }; globalThis.document = { getElementById: id => els[id] || null };
globalThis.cards = [{ name: 'C', cntr: '0000000005', kicKey: 'AA', kidKey: 'BB' }]; globalThis.cards = [{ id: 'preset-1', name: 'C', cntr: '0000000005', kicKey: 'AA', kidKey: 'BB' }];
const calls = { saved: 0, explored: null, removed: null, sent: null }; const calls = { saved: 0, explored: null, removed: null, sent: null };
globalThis.cardsSave = () => { calls.saved++; }; globalThis.cardsSave = () => { calls.saved++; };
globalThis.cardsRender = () => {}; globalThis.cardsRender = () => {};
@@ -63,7 +63,7 @@ function fakeEnv(por, success) {
return cards[0].cntr; return cards[0].cntr;
}; };
globalThis.ramSendOta = async (apdu, sp) => { globalThis.ramSendOta = async (apdu, sp) => {
calls.sent = { apdu: apdu, cntr: sp.cntr, spi2: sp.spi2 }; calls.sent = { apdu: apdu, cntr: sp.cntr, spi2: sp.spi2, preset_id: sp.preset_id };
return { success: success !== false, por: por }; return { success: success !== false, por: por };
}; };
globalThis.ramExplore = async sp => { calls.explored = sp.cntr; }; globalThis.ramExplore = async sp => { calls.explored = sp.cntr; };
@@ -84,7 +84,8 @@ test('accepted delete persists the consumed counter and drops the record', async
assert.strictEqual(cards[0].cntr, '0000000006', assert.strictEqual(cards[0].cntr, '0000000006',
'the preset must carry the counter the card consumed'); 'the preset must carry the counter the card consumed');
assert.strictEqual(els['sp-cntr'].value, '0000000006'); assert.strictEqual(els['sp-cntr'].value, '0000000006');
assert.ok(calls.saved > 0, 'cardsSave() must persist it'); assert.strictEqual(calls.sent.preset_id, 'preset-1',
'the operation must name the server-side preset that persists the counter');
assert.deepStrictEqual(calls.removed, { aid: 'F0414C46416101', cascade: false }); assert.deepStrictEqual(calls.removed, { aid: 'F0414C46416101', cascade: false });
assert.strictEqual(calls.explored, null, 'no re-explore: the record is dropped locally'); assert.strictEqual(calls.explored, null, 'no re-explore: the record is dropped locally');
assert.strictEqual(els['ram-result'].textContent, 'OK'); assert.strictEqual(els['ram-result'].textContent, 'OK');
frontend/tests/testscripts.test.js
+7 -7
View File
@@ -207,9 +207,9 @@ test('hex fields strip mask wildcards, check values keep them', () => {
}); });
test('the counter write-back finds the preset by name after a reload', () => { test('the counter write-back finds the preset by name after a reload', () => {
let saved = 0; let renders = 0;
globalThis.cards = [{ name: 'Card 1', iccid: '8970119000004600098', cntr: '0000000A' }]; globalThis.cards = [{ name: 'Card 1', iccid: '8970119000004600098', cntr: '0000000A' }];
globalThis.cardsSave = () => { saved++; }; globalThis.cardsRender = () => { renders++; };
globalThis.ioStatus = () => {}; globalThis.ioStatus = () => {};
// the name lookup must win: the ICCID helpers are not even called // the name lookup must win: the ICCID helpers are not even called
globalThis.cardsNormIccid = () => { throw new Error('ICCID lookup for a name'); }; globalThis.cardsNormIccid = () => { throw new Error('ICCID lookup for a name'); };
@@ -218,15 +218,15 @@ test('the counter write-back finds the preset by name after a reload', () => {
_testRunState = { running: false, scp80_counter: '0000000B', preset: 'Card 1' }; _testRunState = { running: false, scp80_counter: '0000000B', preset: 'Card 1' };
testWriteBackCounter(); testWriteBackCounter();
assert.strictEqual(cards[0].cntr, '0000000B'); assert.strictEqual(cards[0].cntr, '0000000B');
assert.strictEqual(saved, 1); assert.strictEqual(renders, 1);
testWriteBackCounter(); // idempotent testWriteBackCounter(); // idempotent
assert.strictEqual(saved, 1); assert.strictEqual(renders, 1);
}); });
test('the counter write-back falls back to an ICCID snapshot', () => { test('the counter write-back falls back to an ICCID snapshot', () => {
let saved = 0; let renders = 0;
globalThis.cards = [{ name: 'Other', iccid: '8970119000004600098', cntr: '0000000A' }]; globalThis.cards = [{ name: 'Other', iccid: '8970119000004600098', cntr: '0000000A' }];
globalThis.cardsSave = () => { saved++; }; globalThis.cardsRender = () => { renders++; };
globalThis.ioStatus = () => {}; globalThis.ioStatus = () => {};
globalThis.cardsNormIccid = v => String(v).replace(/\D/g, ''); globalThis.cardsNormIccid = v => String(v).replace(/\D/g, '');
globalThis.cardsFindByIccid = v => (globalThis.cardsNormIccid(v) === '8970119000004600098' ? 0 : -1); globalThis.cardsFindByIccid = v => (globalThis.cardsNormIccid(v) === '8970119000004600098' ? 0 : -1);
@@ -234,7 +234,7 @@ test('the counter write-back falls back to an ICCID snapshot', () => {
_testRunState = { running: false, scp80_counter: '0000000C', preset: '8970119000004600098' }; _testRunState = { running: false, scp80_counter: '0000000C', preset: '8970119000004600098' };
testWriteBackCounter(); testWriteBackCounter();
assert.strictEqual(cards[0].cntr, '0000000C'); assert.strictEqual(cards[0].cntr, '0000000C');
assert.strictEqual(saved, 1); assert.strictEqual(renders, 1);
}); });
test('the item text check offers contains/exact only', () => { test('the item text check offers contains/exact only', () => {
pyproject.toml
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "pysim-simple-server" name = "pysim-simple-server"
version = "3.7.1" version = "3.8.0"
description = "HTTP REST server wrapping pysim for the SIMple PWA" description = "HTTP REST server wrapping pysim for the SIMple PWA"
requires-python = ">=3.8" requires-python = ">=3.8"
# pysim is a git-only dependency installed explicitly by setup.bat/setup.sh. # pysim is a git-only dependency installed explicitly by setup.bat/setup.sh.
pysim_simple_server/__main__.py
+9
View File
@@ -14,6 +14,7 @@ from pySim.cards import UiccCardBase
from .shell import load_pysim_app from .shell import load_pysim_app
from . import fastinit from . import fastinit
from . import gsmtap from . import gsmtap
from . import presets
from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter from .server import PysimHandler, StderrApduTracer, _LoggingApduTracer, VERSION, _send_terminal_profile, _DefaultProactiveHandler, _handle_proactive_chain, _send_status, _init_proactive_session, _timing_on, _tlog, _set_menu_timeout, start_card_monitor, set_auto_equip, _read_iccid, _netstate_read, _netstate_install, _LineFilter
@@ -63,6 +64,8 @@ def main():
parser.add_argument('--mcc-mnc-list', default=_default_mcc_mnc_list(), metavar='PATH', parser.add_argument('--mcc-mnc-list', default=_default_mcc_mnc_list(), metavar='PATH',
help='Worldwide MCC/MNC operator list (JSON) for the network-simulation operator picker (default: the bundled data/mcc-mnc-list.json)') help='Worldwide MCC/MNC operator list (JSON) for the network-simulation operator picker (default: the bundled data/mcc-mnc-list.json)')
parser.add_argument('--log-requests', action='store_true', default=False, help='Log request/response payloads to stderr') parser.add_argument('--log-requests', action='store_true', default=False, help='Log request/response payloads to stderr')
parser.add_argument('--card-presets', default=None, metavar='PATH',
help='Card preset store (default: ~/.pysim-simple-server/card_presets.json)')
parser.add_argument('--sms-oa', default='12345', metavar='DIGITS', parser.add_argument('--sms-oa', default='12345', metavar='DIGITS',
help='TP-Originating-Address (SMSC number) for the SMS-DELIVER TPDU (default: 12345)') help='TP-Originating-Address (SMSC number) for the SMS-DELIVER TPDU (default: 12345)')
parser.add_argument('--sms-sm-sc', default='12345678912', metavar='DIGITS', parser.add_argument('--sms-sm-sc', default='12345678912', metavar='DIGITS',
@@ -278,6 +281,12 @@ def main():
server.cli_terminal_profile = opts.terminal_profile server.cli_terminal_profile = opts.terminal_profile
server.web_dir = opts.web_dir server.web_dir = opts.web_dir
server.mcc_mnc_path = opts.mcc_mnc_list server.mcc_mnc_path = opts.mcc_mnc_list
# Card presets live server-side (the SCP80 counter is card state the server
# persists with the operation that consumed it); --card-presets overrides
# the default store in the user's home.
server.card_presets = presets.PresetStore(opts.card_presets)
sys.stderr.write('PRESETS: %s (%d presets)\n'
% (server.card_presets.path, len(server.card_presets.list())))
server.sim_menu = sim_menu server.sim_menu = sim_menu
server.event_list = event_list server.event_list = event_list
server.menu_active = False server.menu_active = False
pysim_simple_server/presets.py
+366
View File
@@ -0,0 +1,366 @@
"""Server-side card presets: SCP80/SCP81 key material, counters and per-card
parameters, kept in one JSON file.
The presets used to live in the browser's localStorage and the PWA wrote the
SCP80 counter back after every operation. The counter is card state (the card
rejects a packet whose counter is not above its own), so the server - which
performs every card operation and already computes the accepted counter - now
owns the store: an accepted counter is persisted in the same operation, and a
closed tab, a lost response or a second browser window can no longer lose an
increment.
Layout (``--card-presets`` overrides the file; ``Path.home()`` resolves the
same way on Linux, macOS and Windows)::
~/.pysim-simple-server/card_presets.json
~/.pysim-simple-server/card_presets.json.audit.jsonl (counter changes)
Writes are atomic (temp file + fsync + ``os.replace``) and every mutation
holds an ``RLock``, so request handlers and the operation threads can share one
store. Presets carry a stable uuid - never array indices. The JSON is
deliberately human-readable: hand-editing a counter or moving the file between
machines is part of the workbench workflow, and the Cards tab's export/import
uses the same shape (which is also how old localStorage presets move over -
there is no automatic migration).
"""
import json
import os
import re
import sys
import tempfile
import threading
import time
import uuid
from pathlib import Path
DEFAULT_DIR = '.pysim-simple-server'
DEFAULT_FILENAME = 'card_presets.json'
SCHEMA_VERSION = 1
# The fields a preset carries: the localStorage shape plus the id. Defaults
# mirror the Cards form (spi1/spi2/counter and the per-target TARs).
PRESET_FIELDS = ('name', 'iccid', 'adm', 'kic', 'kid', 'spi1', 'spi2',
'tar', 'uiccTar', 'usimTar', 'cntr', 'kicKey', 'kidKey',
'pskIdentity', 'pskKey')
HEX_FIELDS = ('adm', 'kic', 'kid', 'spi1', 'spi2', 'tar', 'uiccTar', 'usimTar',
'cntr', 'kicKey', 'kidKey', 'pskKey')
TAR_DEFAULTS = {'tar': '000000', 'uiccTar': 'B00000', 'usimTar': 'B00001'}
COUNTER_BITS = 40 # the SCP80 counter is 5 bytes (TS 31.115)
COUNTER_WIDTH = 10 # ... shown as 10 hex digits
def default_path():
"""The default store: ``~/.pysim-simple-server/card_presets.json``."""
return Path.home() / DEFAULT_DIR / DEFAULT_FILENAME
def _swap_nibbles(s):
"""Nibble-swap a hex string; an odd trailing nibble becomes 'F' (the JS
``swapNibbles`` the PWA uses)."""
out = []
for i in range(0, len(s), 2):
out.append(s[i + 1] if i + 1 < len(s) else 'F')
out.append(s[i])
return ''.join(out).upper()
def normalize_iccid(value):
"""ICCID -> plain digits, accepting every stored form (the PWA's
``cardsNormIccid``): digits, spaced digits, or the raw nibble-swapped EF
hex - the swap applies only when the value carries A-F, because a raw
value without hex letters is indistinguishable from digits. Empty string
means "no ICCID"."""
s = re.sub(r'[\s-]', '', str(value or '')).upper()
if not s:
return ''
if re.search(r'[A-F]', s):
s = _swap_nibbles(re.sub(r'[^0-9A-F]', '', s))
s = re.sub(r'F+$', '', s)
return re.sub(r'^0+', '', re.sub(r'\D', '', s))
def counter_value(cntr):
"""The counter as an int, or None when it is not hex."""
try:
return int(re.sub(r'[^0-9A-Fa-f]', '', str(cntr or '')), 16)
except ValueError:
return None
def counter_hex(value):
"""The fixed-width 10-hex-digit counter form the card protocol uses."""
v = counter_value(value)
if v is None:
return str(value or '').strip().upper()
return '%0*X' % (COUNTER_WIDTH, v % (1 << COUNTER_BITS))
def counter_ahead(old, new):
"""True when ``new`` is a plausible forward step from ``old``: the counter
only moves forward, and a value that wrapped around the 40-bit field
(top of the range -> small value) counts as forward."""
o, n = counter_value(old), counter_value(new)
if o is None or n is None:
return False
if n > o:
return True
return (o - n) > (1 << (COUNTER_BITS - 1))
class PresetError(Exception):
"""A refused mutation (validation, duplicate ICCID, ...)."""
class PresetStore:
"""The card preset store. All public methods are thread-safe and return
copies - callers can never mutate the store's own state."""
def __init__(self, path=None):
self.path = Path(path) if path else default_path()
self.audit_path = self.path.with_name(self.path.name + '.audit.jsonl')
self._lock = threading.RLock()
self._presets = []
self._load()
# ------------------------------------------------------------------ disk
def _load(self):
try:
raw = self.path.read_text(encoding='utf-8')
except FileNotFoundError:
self._presets = []
return
except OSError as e:
sys.stderr.write('PRESETS: cannot read %s: %s\n' % (self.path, e))
self._presets = []
return
try:
data = json.loads(raw)
except ValueError as e:
# Never silently overwrite an unreadable store: keep the file for
# the user, start empty (the next save would destroy it).
bad = self.path.with_name(self.path.name + '.bad-%s'
% time.strftime('%Y%m%d-%H%M%S'))
try:
self.path.replace(bad)
sys.stderr.write('PRESETS: %s is not valid JSON (%s) - kept as %s, '
'starting empty\n' % (self.path, e, bad))
except OSError:
sys.stderr.write('PRESETS: %s is not valid JSON (%s)\n' % (self.path, e))
self._presets = []
return
items = data.get('presets') if isinstance(data, dict) else None
if not isinstance(items, list):
items = []
self._presets = [self._normalise(p) for p in items if isinstance(p, dict)]
def _save(self):
data = {'version': SCHEMA_VERSION, 'presets': self._presets}
self.path.parent.mkdir(parents=True, exist_ok=True)
tmp_fd, tmp_name = tempfile.mkstemp(prefix=self.path.name + '.',
dir=str(self.path.parent))
try:
with os.fdopen(tmp_fd, 'w', encoding='utf-8') as fh:
json.dump(data, fh, indent=2, ensure_ascii=False)
fh.write('\n')
fh.flush()
os.fsync(fh.fileno())
os.replace(tmp_name, self.path)
except Exception:
try:
os.unlink(tmp_name)
except OSError:
pass
raise
try:
# Make the rename durable. Best effort: opening a directory is not
# portable (Windows) - a failure here never fails the mutation.
dir_fd = os.open(str(self.path.parent), os.O_RDONLY)
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)
except OSError:
pass
def _audit(self, pid, name, old, new, source):
"""One JSONL line per counter change: the forensic trail for "did an
increment get lost?" questions."""
try:
line = json.dumps({'ts': time.strftime('%Y-%m-%dT%H:%M:%S'),
'id': pid, 'name': name, 'old': old, 'new': new,
'source': source}, ensure_ascii=False)
self.path.parent.mkdir(parents=True, exist_ok=True)
with open(self.audit_path, 'a', encoding='utf-8') as fh:
fh.write(line + '\n')
except OSError as e:
sys.stderr.write('PRESETS: audit write failed: %s\n' % e)
# ------------------------------------------------------------- shaping
def _normalise(self, fields):
"""The stored shape: the known fields, cleaned; unknown keys dropped."""
out = {}
for k in PRESET_FIELDS:
v = fields.get(k)
v = '' if v is None else str(v).strip()
if k in HEX_FIELDS:
v = re.sub(r'\s', '', v).upper()
out[k] = v
out['spi1'] = out['spi1'] or '16'
out['spi2'] = out['spi2'] or '01'
out['cntr'] = counter_hex(out['cntr'] or '1')
for k, d in TAR_DEFAULTS.items():
out[k] = out[k] or d
pid = str(fields.get('id') or '').strip().lower()
out['id'] = pid or uuid.uuid4().hex
return out
def _validate(self, p, skip_id=None):
if not p['name']:
raise PresetError('name is required')
for key in ('kic', 'kid', 'kicKey', 'kidKey'):
if not p[key]:
raise PresetError('%s is required' % key)
if bool(p['pskIdentity']) != bool(p['pskKey']):
raise PresetError('PSK identity and PSK key must be set together')
if p['pskKey'] and not re.fullmatch(r'[0-9A-F]{32}', p['pskKey']):
raise PresetError('PSK key must be 32 hex characters')
norm = normalize_iccid(p['iccid'])
if norm:
for other in self._presets:
if other['id'] == skip_id:
continue
if normalize_iccid(other['iccid']) == norm:
raise PresetError('card with this ICCID already exists: %s'
% (other['name'] or other['id']))
return p
def _find(self, pid):
want = str(pid or '').strip().lower()
if not want:
return None
for p in self._presets:
if p['id'] == want:
return p
return None
# -------------------------------------------------------------- public
def list(self):
with self._lock:
return [dict(p) for p in self._presets]
def get(self, pid):
with self._lock:
p = self._find(pid)
return dict(p) if p else None
def find_by_iccid(self, iccid):
norm = normalize_iccid(iccid)
if not norm:
return None
with self._lock:
for p in self._presets:
if normalize_iccid(p['iccid']) == norm:
return dict(p)
return None
def add(self, fields):
with self._lock:
p = self._normalise(fields or {})
self._validate(p)
self._presets.append(p)
self._save()
sys.stderr.write('PRESETS: added %s (%s)\n' % (p['name'], p['id']))
return dict(p)
def update(self, pid, fields):
"""Partial or full update (the human edit path): the counter is written
as given - the Cards tab is the place to resync it deliberately. The
counter an *operation* accepted goes through set_counter()."""
with self._lock:
cur = self._find(pid)
if cur is None:
return None
merged = dict(cur)
for k in PRESET_FIELDS:
if isinstance(fields, dict) and k in fields:
merged[k] = fields[k]
new = self._normalise(merged)
new['id'] = cur['id']
self._validate(new, skip_id=cur['id'])
old_cntr = cur['cntr']
self._presets[self._presets.index(cur)] = new
self._save()
if new['cntr'] != old_cntr:
self._audit(new['id'], new['name'], old_cntr, new['cntr'], 'edit')
sys.stderr.write('PRESETS: counter %s %s -> %s (edit)\n'
% (new['name'], old_cntr, new['cntr']))
return dict(new)
def remove(self, pid):
with self._lock:
p = self._find(pid)
if p is None:
return False
self._presets.remove(p)
self._save()
sys.stderr.write('PRESETS: removed %s (%s)\n' % (p['name'], p['id']))
return True
def set_counter(self, pid, cntr, source='operation'):
"""Persist the counter an operation accepted. Monotonic: the counter
only moves forward, so a stale caller can never regress it. Returns
the stored preset (or None when the id is unknown)."""
with self._lock:
p = self._find(pid)
if p is None:
return None
new = counter_hex(cntr)
if new == p['cntr']:
return dict(p)
if not counter_ahead(p['cntr'], new):
sys.stderr.write('PRESETS: ignoring backwards counter %s -> %s (%s)\n'
% (p['cntr'], new, source))
return dict(p)
old = p['cntr']
p['cntr'] = new
self._save()
self._audit(p['id'], p['name'], old, new, source)
sys.stderr.write('PRESETS: counter %s %s -> %s (%s)\n'
% (p['name'], old, new, source))
return dict(p)
def import_presets(self, items, mode='merge'):
"""Import exported (or old localStorage) presets: entries are validated
one by one; a duplicate ICCID or an already-used id is skipped, invalid
entries are reported instead of failing the whole import. ``replace``
wipes the store first."""
added, skipped, errors = 0, 0, []
with self._lock:
if mode == 'replace':
self._presets = []
for i, entry in enumerate(items or []):
if not isinstance(entry, dict):
skipped += 1
continue
try:
p = self._normalise(entry)
if self._find(p['id']):
p['id'] = uuid.uuid4().hex
self._validate(p)
except PresetError as e:
skipped += 1
errors.append('%s: %s' % (entry.get('name') or ('#' + str(i + 1)), e))
continue
self._presets.append(p)
added += 1
if added or mode == 'replace':
self._save()
sys.stderr.write('PRESETS: import %s: %d added, %d skipped\n'
% (mode, added, skipped))
return {'added': added, 'skipped': skipped, 'errors': errors[:10]}
def info(self):
with self._lock:
return {'path': str(self.path), 'count': len(self._presets),
'version': SCHEMA_VERSION}
pysim_simple_server/server.py
+132 -11
View File
@@ -21,6 +21,7 @@ from pysim_simple_server import scp81
from pysim_simple_server import esim from pysim_simple_server import esim
from pysim_simple_server import capmem from pysim_simple_server import capmem
from pysim_simple_server import testscript from pysim_simple_server import testscript
from pysim_simple_server import presets
from smartcard.CardMonitoring import CardMonitor, CardObserver from smartcard.CardMonitoring import CardMonitor, CardObserver
from cmd2.exceptions import CommandSetRegistrationError from cmd2.exceptions import CommandSetRegistrationError
@@ -31,7 +32,7 @@ from osmocom.tlv import BER_TLV_IE
from osmocom.utils import rpad from osmocom.utils import rpad
VERSION = '3.7.1' VERSION = '3.8.0'
MAX_ENVELOPE_SEGMENTS = 5 # max SMS segments for outgoing C-APDU in ENVELOPE MAX_ENVELOPE_SEGMENTS = 5 # max SMS segments for outgoing C-APDU in ENVELOPE
@@ -1411,6 +1412,24 @@ def _ram_next_cntr(cntr, advance):
return '%010X' % ((int(cntr, 16) + 1) % (2 ** 40)) return '%010X' % ((int(cntr, 16) + 1) % (2 ** 40))
def _preset_counter_persist(server, preset_id, cntr, source):
"""Persist the counter an operation accepted into the server-side preset
store (v3.8.0). The counter is card state, so the server writes it with
the operation that consumed it - a closed tab, a lost response or a second
browser window can no longer lose an increment. The store itself is
monotonic (a stale value never regresses it). Best effort: a store
failure must never fail the card operation."""
if not preset_id or not cntr:
return
store = getattr(server, 'card_presets', None)
if store is None:
return
try:
store.set_counter(preset_id, cntr, source)
except Exception as e:
sys.stderr.write('PRESETS: counter persist failed (%s): %s\n' % (source, e))
# RAM command formats (TS 102 226 5.2.1): the bare C-APDU (compact) or the # RAM command formats (TS 102 226 5.2.1): the bare C-APDU (compact) or the
# Command TLV '22' inside the 'AA' scripting template (expanded) - the form # Command TLV '22' inside the 'AA' scripting template (expanded) - the form
# the reference terminal traces use. The format is detected per operation # the reference terminal traces use. The format is detected per operation
@@ -1448,7 +1467,7 @@ def _ram_detect_format(server, scc, sp, state):
for fmt, apdu in (('compact', _RAM_PROBE_APDU), for fmt, apdu in (('compact', _RAM_PROBE_APDU),
('expanded', _wrap_expanded_apdu(_RAM_PROBE_APDU))): ('expanded', _wrap_expanded_apdu(_RAM_PROBE_APDU))):
scratch = {'steps': [], 'encode_error': None, 'failure': {}, scratch = {'steps': [], 'encode_error': None, 'failure': {},
'cntr': state['cntr']} 'cntr': state['cntr'], 'preset_id': state.get('preset_id')}
ok = _ram_send_gp_apdu(server, scc, sp, scratch, ok = _ram_send_gp_apdu(server, scc, sp, scratch,
'FORMAT CHECK (%s)' % fmt, apdu) 'FORMAT CHECK (%s)' % fmt, apdu)
state['steps'].extend(scratch['steps']) state['steps'].extend(scratch['steps'])
@@ -1605,6 +1624,11 @@ def _ram_send_gp_apdu(server, scc, sp, state, step_name, apdu_hex, silent=False)
# Advance the counter only for an accepted packet # Advance the counter only for an accepted packet
state['cntr'] = _ram_next_cntr( state['cntr'] = _ram_next_cntr(
state['cntr'], step.get('por_status') in ('por_ok', 'no_por')) state['cntr'], step.get('por_status') in ('por_ok', 'no_por'))
# ... and persist it right away: the server owns the counter now, so a
# crash mid-install cannot leave the preset behind the card (v3.8.0).
if step.get('por_status') in ('por_ok', 'no_por'):
_preset_counter_persist(server, state.get('preset_id'),
state['cntr'], 'ram-%s' % step_name.split()[0].lower())
if step_error: if step_error:
if not silent: if not silent:
state['failure']['error'] = '%s: %s' % (step_name, step_error) state['failure']['error'] = '%s: %s' % (step_name, step_error)
@@ -5105,7 +5129,7 @@ _CARD_FREE_GET = frozenset([
'/api/stk-status', '/api/poll-status', '/api/proactive-log', '/api/stk-status', '/api/poll-status', '/api/proactive-log',
'/api/menu', '/api/events', '/api/terminal-profile', '/api/menu', '/api/events', '/api/terminal-profile',
'/api/pli-qualifiers', '/api/pli-dict', '/api/commands', '/api/pli-qualifiers', '/api/pli-dict', '/api/commands',
'/api/net-state', '/api/mcc-mnc', '/api/net-state', '/api/mcc-mnc', '/api/presets',
'/api/scp81/status', '/api/bip/status', '/api/scp81/script', '/api/scp81/status', '/api/bip/status', '/api/scp81/script',
'/api/scp81/log', '/api/bip/log', '/api/scp81/log', '/api/bip/log',
]) ])
@@ -5296,8 +5320,12 @@ def _test_run_scp80(server, step, ctx):
source, tar or '-', spi1, spi2, counter) source, tar or '-', spi1, spi2, counter)
if sw: if sw:
# The card answered, so the SCP80 counter was consumed: advance the # The card answered, so the SCP80 counter was consumed: advance the
# working value (the PWA writes the final one back to the preset). # working value and persist it into the server-side preset (v3.8.0 -
# the server owns the counter, so the run survives a lost response and
# the PWA's counter write-back dance is gone).
ctx['counter'] = _increment_counter_hex(counter) ctx['counter'] = _increment_counter_hex(counter)
_preset_counter_persist(server, (preset or {}).get('id'),
ctx['counter'], 'test-run')
return data, sw, sent, por, counter return data, sw, sent, por, counter
@@ -5595,6 +5623,17 @@ class PysimHandler(BaseHTTPRequestHandler):
length = int(self.headers.get('Content-Length', 0)) length = int(self.headers.get('Content-Length', 0))
return json.loads(self.rfile.read(length)) return json.loads(self.rfile.read(length))
def _preset_store_or_503(self):
"""The card preset store, or None after answering 503 (a server built
without one - e.g. a test harness - must not crash the endpoints)."""
store = getattr(self.server, 'card_presets', None)
if store is None:
resp = {'error': 'preset store not initialized'}
self._send_json(resp, 503)
self._log_resp(resp)
return None
return store
def _log_req(self, body=None): def _log_req(self, body=None):
if self.server.log_requests: if self.server.log_requests:
if body is not None: if body is not None:
@@ -5958,6 +5997,15 @@ class PysimHandler(BaseHTTPRequestHandler):
resp = _scp81_script_state() resp = _scp81_script_state()
self._send_json(resp) self._send_json(resp)
self._log_resp(resp) self._log_resp(resp)
elif self.path == '/api/presets':
self._log_req()
store = self._preset_store_or_503()
if store is None:
return
resp = store.info()
resp['presets'] = store.list()
self._send_json(resp)
self._log_resp(resp)
elif self.path.startswith('/api/'): elif self.path.startswith('/api/'):
self._send_json({'error': _err('not_found', lang)}, 404) self._send_json({'error': _err('not_found', lang)}, 404)
self._log_resp({'error': _err('not_found', lang)}) self._log_resp({'error': _err('not_found', lang)})
@@ -6688,7 +6736,8 @@ class PysimHandler(BaseHTTPRequestHandler):
ram_format = _ram_normalize_format(body.get('ram_format')) ram_format = _ram_normalize_format(body.get('ram_format'))
if ram_format == 'auto': if ram_format == 'auto':
probe_state = {'steps': [], 'encode_error': None, probe_state = {'steps': [], 'encode_error': None,
'failure': {}, 'cntr': cntr} 'failure': {}, 'cntr': cntr,
'preset_id': body.get('preset_id')}
sp_probe = {'spi1': spi1, 'spi2': spi2, 'kic': kic, sp_probe = {'spi1': spi1, 'spi2': spi2, 'kic': kic,
'kid': kid, 'tar': tar, 'kic_key': kic_key, 'kid': kid, 'tar': tar, 'kic_key': kic_key,
'kid_key': kid_key, 'kid_key': kid_key,
@@ -6701,6 +6750,7 @@ class PysimHandler(BaseHTTPRequestHandler):
else: else:
# Regular SCP80: use pre-built secured packet # Regular SCP80: use pre-built secured packet
sp_hex = sp sp_hex = sp
cntr = body.get('cntr', '')
spi2_val = int(body.get('spi2', '00'), 16) spi2_val = int(body.get('spi2', '00'), 16)
# Capture the PoR from either transport: inline in the # Capture the PoR from either transport: inline in the
# ENVELOPE response or as a proactive SEND SHORT MESSAGE (some # ENVELOPE response or as a proactive SEND SHORT MESSAGE (some
@@ -6797,14 +6847,18 @@ class PysimHandler(BaseHTTPRequestHandler):
sys.stderr.write('OTA PoR[%s]: undecodable raw=%s\n' % (por_src, str(por_hex))) sys.stderr.write('OTA PoR[%s]: undecodable raw=%s\n' % (por_src, str(por_hex)))
else: else:
sys.stderr.write('OTA PoR[%s]: none\n' % por_src) sys.stderr.write('OTA PoR[%s]: none\n' % por_src)
accepted = ((por is None) or por.get('response_status') in
('por_ok', 'actual_response_sms_submit'))
if ram_format is not None: if ram_format is not None:
accepted = ((por is None) or por.get('response_status') in
('por_ok', 'actual_response_sms_submit'))
resp['ram_format'] = ram_format resp['ram_format'] = ram_format
resp['final_cntr'] = _ram_next_cntr(cntr, accepted) # The counter to use next, reported for the RAM and the
# plain SCP80 path alike; the server persists it (v3.8.0).
resp['final_cntr'] = _ram_next_cntr(cntr, accepted)
finally: finally:
if submit_handler and hasattr(scc, '_tp'): if submit_handler and hasattr(scc, '_tp'):
scc._tp.proactive_handler = old_proactive scc._tp.proactive_handler = old_proactive
_preset_counter_persist(self.server, body.get('preset_id'),
resp.get('final_cntr'), 'send-ota')
self._send_json(resp) self._send_json(resp)
self._log_resp(resp) self._log_resp(resp)
except Exception as e: except Exception as e:
@@ -6896,7 +6950,7 @@ class PysimHandler(BaseHTTPRequestHandler):
steps = [] steps = []
state = {'steps': steps, 'encode_error': None, 'failure': {}, state = {'steps': steps, 'encode_error': None, 'failure': {},
'cntr': cntr} 'cntr': cntr, 'preset_id': body.get('preset_id')}
sp_state = {'spi1': spi1, 'spi2': spi2, 'kic': kic, 'kid': kid, sp_state = {'spi1': spi1, 'spi2': spi2, 'kic': kic, 'kid': kid,
'tar': tar, 'kic_key': kic_key, 'kid_key': kid_key, 'tar': tar, 'kic_key': kic_key, 'kid_key': kid_key,
'include_cpi': body.get('includeCpi', True)} 'include_cpi': body.get('includeCpi', True)}
@@ -7030,7 +7084,7 @@ class PysimHandler(BaseHTTPRequestHandler):
'kic_key': body.get('kicKey', ''), 'kid_key': body.get('kidKey', ''), 'kic_key': body.get('kicKey', ''), 'kid_key': body.get('kidKey', ''),
'include_cpi': body.get('includeCpi', True)} 'include_cpi': body.get('includeCpi', True)}
state = {'steps': [], 'encode_error': None, 'failure': {}, state = {'steps': [], 'encode_error': None, 'failure': {},
'cntr': sp_state['cntr']} 'cntr': sp_state['cntr'], 'preset_id': body.get('preset_id')}
# Live progress for the PWA's operation modal (see /api/ram-install) # Live progress for the PWA's operation modal (see /api/ram-install)
_ram_progress_begin('install-app', 2) _ram_progress_begin('install-app', 2)
try: try:
@@ -7136,7 +7190,7 @@ class PysimHandler(BaseHTTPRequestHandler):
kid_key = body.get('kidKey', '') kid_key = body.get('kidKey', '')
steps = [] steps = []
state = {'steps': steps, 'encode_error': None, 'failure': {}, state = {'steps': steps, 'encode_error': None, 'failure': {},
'cntr': cntr} 'cntr': cntr, 'preset_id': body.get('preset_id')}
sp_state = {'spi1': spi1, 'spi2': spi2, 'kic': kic, 'kid': kid, sp_state = {'spi1': spi1, 'spi2': spi2, 'kic': kic, 'kid': kid,
'tar': tar, 'kic_key': kic_key, 'kid_key': kid_key, 'tar': tar, 'kic_key': kic_key, 'kid_key': kid_key,
'include_cpi': body.get('includeCpi', True)} 'include_cpi': body.get('includeCpi', True)}
@@ -7317,6 +7371,73 @@ class PysimHandler(BaseHTTPRequestHandler):
resp = _test_state_snapshot() resp = _test_state_snapshot()
self._send_json(resp) self._send_json(resp)
self._log_resp(resp) self._log_resp(resp)
elif self.path == '/api/presets':
body = self._read_body()
self._log_req(body)
store = self._preset_store_or_503()
if store is None:
return
try:
preset = store.add(body)
except presets.PresetError as e:
resp = {'error': str(e)}
self._send_json(resp, 400)
self._log_resp(resp)
return
resp = {'ok': True, 'preset': preset}
self._send_json(resp)
self._log_resp(resp)
elif self.path == '/api/presets/update':
body = self._read_body()
self._log_req(body)
store = self._preset_store_or_503()
if store is None:
return
fields = body.get('fields')
if not isinstance(fields, dict):
fields = {k: v for k, v in body.items() if k not in ('id', 'fields')}
try:
preset = store.update(body.get('id'), fields)
except presets.PresetError as e:
resp = {'error': str(e)}
self._send_json(resp, 400)
self._log_resp(resp)
return
if preset is None:
resp = {'error': 'unknown preset id'}
self._send_json(resp, 404)
self._log_resp(resp)
return
resp = {'ok': True, 'preset': preset}
self._send_json(resp)
self._log_resp(resp)
elif self.path == '/api/presets/delete':
body = self._read_body()
self._log_req(body)
store = self._preset_store_or_503()
if store is None:
return
removed = store.remove(body.get('id'))
resp = {'ok': True, 'removed': removed}
self._send_json(resp)
self._log_resp(resp)
elif self.path == '/api/presets/import':
body = self._read_body()
self._log_req(body)
store = self._preset_store_or_503()
if store is None:
return
items = body.get('presets')
if not isinstance(items, list) or len(items) > 1000:
resp = {'error': 'presets must be a list of at most 1000 entries'}
self._send_json(resp, 400)
self._log_resp(resp)
return
mode = body.get('mode') if body.get('mode') in ('merge', 'replace') else 'merge'
resp = store.import_presets(items, mode)
resp['ok'] = True
self._send_json(resp)
self._log_resp(resp)
else: else:
self._send_json({'error': _err('not_found', lang)}, 404) self._send_json({'error': _err('not_found', lang)}, 404)
self._log_resp({'error': _err('not_found', lang)}) self._log_resp({'error': _err('not_found', lang)})
tests/test_fetch_routes.py
+142 -2
View File
@@ -25,10 +25,11 @@ ROOT = pathlib.Path(__file__).resolve().parents[1]
def _pwa_calls(): def _pwa_calls():
"""[(path, has_body)] for every literal pysimFetch() call in the PWA.""" """[(path, has_body)] for every literal pysimFetch()/pysimFetchOk() call in
the PWA."""
html = (ROOT / 'frontend' / 'index.html').read_text(encoding='utf-8') html = (ROOT / 'frontend' / 'index.html').read_text(encoding='utf-8')
return [(m.group(1).split('?')[0], m.group(2) == ',') return [(m.group(1).split('?')[0], m.group(2) == ',')
for m in re.finditer(r"pysimFetch\(\s*'([^']+)'\s*(,|\))", html)] for m in re.finditer(r"pysimFetch(?:Ok)?\(\s*'([^']+)'\s*(,|\))", html)]
def _server_routes(): def _server_routes():
@@ -427,5 +428,144 @@ class CntrLowGuardHttpTests(unittest.TestCase):
self.assertEqual(resp['suggested_cntr'], '00000000BC') self.assertEqual(resp['suggested_cntr'], '00000000BC')
class PresetStoreHttpTests(unittest.TestCase):
"""The card preset endpoints (v3.8.0): the PWA's Cards tab talks to the
server store instead of localStorage, and the counter an operation
accepted is persisted server-side."""
def setUp(self):
import tempfile
from pysim_simple_server import presets
from pysim_simple_server import server as srv
self.srv = srv
self.tmp = tempfile.TemporaryDirectory()
self.store = presets.PresetStore(pathlib.Path(self.tmp.name) / 'card_presets.json')
self.server = _build_http_server('127.0.0.1', 0, PysimHandler)
self.server.log_requests = False
self.server.app = None
self.server.sl = None
self.server.scc = object()
self.server.sms_oa = '12345'
self.server.sms_sc = '12345678912'
self.server.card_presets = self.store
threading.Thread(target=self.server.serve_forever, daemon=True).start()
self.port = self.server.server_address[1]
def tearDown(self):
self.server.shutdown()
self.server.server_close()
self.tmp.cleanup()
def _post(self, path, body):
req = urllib.request.Request(
'http://127.0.0.1:%d%s' % (self.port, path),
data=json.dumps(body).encode(),
headers={'Content-Type': 'application/json'}, method='POST')
try:
with urllib.request.urlopen(req, timeout=5) as res:
return res.status, json.loads(res.read() or b'{}')
except urllib.error.HTTPError as e:
body = json.loads(e.read() or b'{}')
e.close()
return e.code, body
def _get(self, path, timeout=5):
try:
with urllib.request.urlopen(
'http://127.0.0.1:%d%s' % (self.port, path), timeout=timeout) as res:
return res.status, json.loads(res.read() or b'{}')
except urllib.error.HTTPError as e:
body = json.loads(e.read() or b'{}')
e.close()
return e.code, body
def _preset(self, **kw):
p = {'name': 'C', 'kic': '15', 'kid': '15', 'kicKey': 'AA', 'kidKey': 'BB'}
p.update(kw)
return p
def test_the_get_route_is_card_free_and_reports_the_store(self):
self.assertIn('/api/presets', _CARD_FREE_GET)
with _CARD_LOCK:
status, resp = self._get('/api/presets')
self.assertEqual(status, 200, resp)
self.assertEqual(resp['presets'], [])
self.assertEqual(resp['path'], str(self.store.path))
def test_create_update_delete_round_trip(self):
status, resp = self._post('/api/presets', self._preset(iccid='8970119000004600098'))
self.assertEqual(status, 200, resp)
pid = resp['preset']['id']
self.assertTrue(pid)
self.assertEqual(resp['preset']['cntr'], '0000000001')
# a duplicate ICCID (raw-hex form) is refused
status, resp = self._post('/api/presets',
self._preset(name='Other', iccid='980711090000640090F8'))
self.assertEqual(status, 400, resp)
self.assertIn('already exists', resp['error'])
status, resp = self._post('/api/presets/update',
{'id': pid, 'fields': {'name': 'Renamed'}})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['preset']['name'], 'Renamed')
self.assertEqual(resp['preset']['id'], pid)
status, resp = self._post('/api/presets/update', {'id': 'nope', 'fields': {}})
self.assertEqual(status, 404, resp)
status, resp = self._post('/api/presets/delete', {'id': pid})
self.assertEqual(status, 200, resp)
self.assertTrue(resp['removed'])
self.assertEqual(self.store.list(), [])
def test_import_accepts_the_old_localstorage_export(self):
status, resp = self._post('/api/presets/import', {'presets': [
{'name': 'A', 'iccid': '8970119000004600098', 'kic': '15', 'kid': '15',
'kicKey': 'AA', 'kidKey': 'BB', 'cntr': '0000000007'},
{'name': 'Broken'},
]})
self.assertEqual(status, 200, resp)
self.assertEqual(resp['added'], 1)
self.assertEqual(resp['skipped'], 1)
self.assertEqual(self.store.find_by_iccid('8970119000004600098')['cntr'], '0000000007')
status, resp = self._post('/api/presets/import', {'presets': 'nope'})
self.assertEqual(status, 400, resp)
def test_the_counter_an_operation_accepted_is_persisted(self):
p = self.store.add(self._preset(iccid='8970119000004600098'))
patches = [
mock.patch.object(self.srv, '_send_secured_packet', lambda *a, **k: {
'success': True, 'sw': '9000', 'bytes': 34, 'segments': 1,
'response_data': '027100000000'}),
mock.patch.object(self.srv, '_decode_por', lambda *a, **k: {
'response_status': 'por_ok', 'tar': '000000', 'cntr': '0000000001',
'pcntr': 0, 'rpl': 11, 'rhl': 10, 'raw': '',
'decoded': {'number_of_commands': 1, 'last_status_word': '9000',
'last_response_data': ''}}),
]
self.server.app = object()
for patch in patches:
patch.start()
try:
status, resp = self._post('/api/send-ota', {
'sp': '00', 'spi1': '16', 'spi2': '01', 'kic': '15', 'kid': '15',
'tar': '000000', 'cntr': '0000000001', 'preset_id': p['id']})
finally:
for patch in reversed(patches):
patch.stop()
self.assertEqual(status, 200, resp)
self.assertEqual(resp['final_cntr'], '0000000002')
self.assertEqual(self.store.get(p['id'])['cntr'], '0000000002')
def test_the_store_endpoints_answer_503_without_a_store(self):
self.server.card_presets = None
status, resp = self._get('/api/presets')
self.assertEqual(status, 503, resp)
status, resp = self._post('/api/presets', self._preset())
self.assertEqual(status, 503, resp)
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
tests/test_presets.py
+276
View File
@@ -0,0 +1,276 @@
"""Card preset store tests (server-side presets).
The store replaces the localStorage presets; the ICCID normalisation must stay
byte-compatible with the PWA's `cardsNormIccid` (digits / spaced / raw
nibble-swapped EF hex), the counter must only move forward, and the file must
survive a corrupt store without silently destroying it.
"""
import json
import pathlib
import tempfile
import unittest
from pysim_simple_server import presets
DIGITS = '8970119000004600098'
RAW_HEX = '980711090000640090F8' # the PWA's fixture for DIGITS
def make_store(tmp):
return presets.PresetStore(pathlib.Path(tmp) / 'card_presets.json')
def preset(**kw):
p = {'name': 'C', 'kic': '15', 'kid': '15', 'kicKey': 'AA', 'kidKey': 'BB'}
p.update(kw)
return p
class IccidNormalisationTests(unittest.TestCase):
def test_digits_spaces_and_raw_hex_normalise_alike(self):
self.assertEqual(presets.normalize_iccid(DIGITS), DIGITS)
self.assertEqual(presets.normalize_iccid(' 89 70 1190-0000 4600 098 '), DIGITS)
self.assertEqual(presets.normalize_iccid(RAW_HEX), DIGITS)
self.assertEqual(presets.normalize_iccid(RAW_HEX.lower()), DIGITS)
def test_empty_values_mean_no_iccid(self):
for value in ('', None, ' ', '---'):
self.assertEqual(presets.normalize_iccid(value), '')
def test_leading_zeros_are_stripped(self):
self.assertEqual(presets.normalize_iccid('008970119000004600098'),
'8970119000004600098')
def test_odd_length_raw_hex_gets_the_js_fallback_nibble(self):
# swapNibbles pads the missing nibble with 'f', then trailing Fs go
self.assertEqual(presets.normalize_iccid('980711090000640090F8'), DIGITS)
self.assertEqual(presets._swap_nibbles('123'), '21F3')
class CounterTests(unittest.TestCase):
def test_counter_hex_is_the_fixed_width_protocol_form(self):
self.assertEqual(presets.counter_hex('1'), '0000000001')
self.assertEqual(presets.counter_hex('abc'), '0000000ABC')
self.assertEqual(presets.counter_hex('FFFFFFFFFF'), 'FFFFFFFFFF')
def test_counter_ahead_is_forward_only_with_wrap(self):
self.assertTrue(presets.counter_ahead('0000000001', '0000000002'))
self.assertFalse(presets.counter_ahead('0000000002', '0000000001'))
self.assertFalse(presets.counter_ahead('0000000002', '0000000002'))
# the 40-bit field wrapped past the top
self.assertTrue(presets.counter_ahead('FFFFFFFFFF', '0000000000'))
self.assertFalse(presets.counter_ahead('nonsense', '0000000002'))
class StoreCrudTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.store = make_store(self.tmp.name)
def tearDown(self):
self.tmp.cleanup()
def test_add_applies_the_form_defaults_and_assigns_an_id(self):
p = self.store.add(preset(iccid=DIGITS))
self.assertTrue(p['id'])
self.assertEqual(p['spi1'], '16')
self.assertEqual(p['spi2'], '01')
self.assertEqual(p['cntr'], '0000000001')
self.assertEqual(p['tar'], '000000')
self.assertEqual(p['uiccTar'], 'B00000')
self.assertEqual(p['usimTar'], 'B00001')
self.assertEqual([x['id'] for x in self.store.list()], [p['id']])
def test_validation_requires_the_scp80_fields(self):
with self.assertRaises(presets.PresetError):
self.store.add(preset(name=''))
with self.assertRaises(presets.PresetError):
self.store.add(preset(kicKey=''))
with self.assertRaises(presets.PresetError):
self.store.add(preset(pskIdentity='id'))
with self.assertRaises(presets.PresetError):
self.store.add(preset(pskIdentity='id', pskKey='00'))
def test_duplicate_iccid_is_refused_across_stored_forms(self):
self.store.add(preset(iccid=DIGITS))
with self.assertRaises(presets.PresetError) as ctx:
self.store.add(preset(name='Other', iccid=RAW_HEX))
self.assertIn('already exists', str(ctx.exception))
# an empty ICCID is optional and never a duplicate
self.store.add(preset(name='No ICCID'))
self.store.add(preset(name='No ICCID 2'))
self.assertEqual(len(self.store.list()), 3)
def test_update_is_partial_and_keeps_the_id(self):
p = self.store.add(preset(iccid=DIGITS))
up = self.store.update(p['id'], {'name': 'Renamed', 'cntr': '0000000009'})
self.assertEqual(up['id'], p['id'])
self.assertEqual(up['name'], 'Renamed')
self.assertEqual(up['cntr'], '0000000009')
self.assertEqual(up['kic'], '15')
self.assertIsNone(self.store.update('nope', {'name': 'x'}))
def test_remove(self):
p = self.store.add(preset())
self.assertTrue(self.store.remove(p['id']))
self.assertFalse(self.store.remove(p['id']))
self.assertEqual(self.store.list(), [])
def test_find_by_iccid(self):
p = self.store.add(preset(iccid=DIGITS))
self.assertEqual(self.store.find_by_iccid(RAW_HEX)['id'], p['id'])
self.assertEqual(self.store.find_by_iccid(' 89 70 1190-0000 4600 098 ')['id'], p['id'])
self.assertIsNone(self.store.find_by_iccid('1234567890123456789'))
self.assertIsNone(self.store.find_by_iccid(''))
def test_callers_get_copies(self):
p = self.store.add(preset())
p['name'] = 'mutated'
self.assertEqual(self.store.get(p['id'])['name'], 'C')
items = self.store.list()
items[0]['name'] = 'mutated'
self.assertEqual(self.store.list()[0]['name'], 'C')
class StoreCounterTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.store = make_store(self.tmp.name)
self.p = self.store.add(preset(iccid=DIGITS))
def tearDown(self):
self.tmp.cleanup()
def test_set_counter_advances_and_ignores_stale_values(self):
self.store.set_counter(self.p['id'], '0000000005', 'send-ota')
self.assertEqual(self.store.get(self.p['id'])['cntr'], '0000000005')
# a stale caller can never regress the counter
self.store.set_counter(self.p['id'], '0000000003', 'send-ota')
self.assertEqual(self.store.get(self.p['id'])['cntr'], '0000000005')
# equal values are a no-op
self.store.set_counter(self.p['id'], '0000000005', 'send-ota')
self.assertEqual(self.store.get(self.p['id'])['cntr'], '0000000005')
def test_set_counter_allows_a_wrapped_value(self):
self.store.update(self.p['id'], {'cntr': 'FFFFFFFFFF'})
self.store.set_counter(self.p['id'], '0000000000', 'send-ota')
self.assertEqual(self.store.get(self.p['id'])['cntr'], '0000000000')
def test_set_counter_on_unknown_id_is_none(self):
self.assertIsNone(self.store.set_counter('nope', '0000000002'))
def test_counter_changes_are_audited(self):
self.store.set_counter(self.p['id'], '0000000005', 'send-ota')
lines = self.store.audit_path.read_text(encoding='utf-8').strip().splitlines()
self.assertEqual(len(lines), 1)
entry = json.loads(lines[0])
self.assertEqual(entry['old'], '0000000001')
self.assertEqual(entry['new'], '0000000005')
self.assertEqual(entry['source'], 'send-ota')
class StorePersistenceTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
def tearDown(self):
self.tmp.cleanup()
def test_the_file_round_trips(self):
store = make_store(self.tmp.name)
p = store.add(preset(iccid=DIGITS, pskIdentity='id', pskKey='00' * 16))
store.set_counter(p['id'], '0000000004', 'send-ota')
raw = json.loads(pathlib.Path(store.path).read_text(encoding='utf-8'))
self.assertEqual(raw['version'], presets.SCHEMA_VERSION)
again = make_store(self.tmp.name)
self.assertEqual([x['id'] for x in again.list()], [p['id']])
self.assertEqual(again.get(p['id'])['cntr'], '0000000004')
def test_no_temporary_files_are_left_behind(self):
store = make_store(self.tmp.name)
store.add(preset())
store.add(preset(name='Second'))
leftovers = [f.name for f in pathlib.Path(self.tmp.name).iterdir()
if '.json.' in f.name]
self.assertEqual(leftovers, [])
def test_a_corrupt_store_is_kept_and_the_store_starts_empty(self):
path = pathlib.Path(self.tmp.name) / 'card_presets.json'
path.write_text('{not json', encoding='utf-8')
store = make_store(self.tmp.name)
self.assertEqual(store.list(), [])
kept = [f.name for f in pathlib.Path(self.tmp.name).iterdir()
if '.bad-' in f.name]
self.assertEqual(len(kept), 1)
self.assertEqual(pathlib.Path(self.tmp.name, kept[0]).read_text(encoding='utf-8'),
'{not json')
def test_a_missing_file_is_an_empty_store(self):
self.assertEqual(make_store(self.tmp.name).list(), [])
class StoreImportTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.store = make_store(self.tmp.name)
def tearDown(self):
self.tmp.cleanup()
def test_import_accepts_the_old_localstorage_shape(self):
# an export from the browser: no ids, no defaults filled in
old = [{'name': 'A', 'iccid': DIGITS, 'kic': '15', 'kid': '15',
'kicKey': 'AA', 'kidKey': 'BB', 'cntr': '0000000007'},
{'name': 'B', 'iccid': '', 'kic': '25', 'kid': '25',
'kicKey': 'CC', 'kidKey': 'DD'}]
res = self.store.import_presets(old)
self.assertEqual(res['added'], 2)
self.assertEqual(res['skipped'], 0)
a = self.store.find_by_iccid(DIGITS)
self.assertEqual(a['cntr'], '0000000007')
self.assertTrue(a['id'])
def test_import_skips_duplicates_and_reports_invalid_entries(self):
self.store.add(preset(iccid=DIGITS))
res = self.store.import_presets([
preset(iccid=RAW_HEX), # same card in raw form
{'name': 'Broken'}, # no KIc/KID/keys
'not a dict',
])
self.assertEqual(res['added'], 0)
self.assertEqual(res['skipped'], 3)
self.assertTrue(any('already exists' in e for e in res['errors']))
self.assertEqual(len(self.store.list()), 1)
def test_import_replace_wipes_the_store(self):
self.store.add(preset(iccid=DIGITS))
res = self.store.import_presets([preset(name='Fresh', iccid='')], mode='replace')
self.assertEqual(res['added'], 1)
self.assertEqual([p['name'] for p in self.store.list()], ['Fresh'])
def test_an_existing_id_is_never_overwritten(self):
p = self.store.add(preset(iccid=DIGITS))
res = self.store.import_presets([dict(preset(name='Copy'), id=p['id'], iccid='')])
self.assertEqual(res['added'], 1)
ids = [x['id'] for x in self.store.list()]
self.assertEqual(len(set(ids)), 2)
class StoreInfoTests(unittest.TestCase):
def test_info_reports_the_path_and_count(self):
with tempfile.TemporaryDirectory() as tmp:
store = make_store(tmp)
store.add(preset())
info = store.info()
self.assertEqual(info['count'], 1)
self.assertEqual(info['path'], str(store.path))
self.assertEqual(info['version'], presets.SCHEMA_VERSION)
def test_default_path_is_the_home_dot_directory(self):
self.assertEqual(presets.default_path().name, 'card_presets.json')
self.assertEqual(presets.default_path().parent.name, '.pysim-simple-server')
if __name__ == '__main__':
unittest.main()