2 Commits

Author SHA1 Message Date
catarrh cdd3820064 v3.32.1: "Start counter value" in the outfile import
release / release (push) Waiting to run
The import created presets with the store's default counter (1).  Cards
whose counters are already past it had to be edited preset by preset
afterwards.

- The outfile preview form gains a "Start counter value" field (default
  1, the store's value).  The value is written into every created keyset
  of every imported card - parse_outfile's `start_counter`, 1-10 hex
  digits, whitespace stripped and uppercased; a bad value raises and the
  endpoint answers 400.  Without it the store's default still applies
  (the field is omitted), so the API keeps its old behaviour.
- The preview shows the counter; an import stores the protocol's
  10-digit form (counter_hex).
- Docs: api.md (the field), help EN/RU (the form).
- Version strings + the PWA cache bumped together (simple-v459).

Tests: the parser (every keyset, normalisation, unset, the bad values),
the endpoint (the preview's keysets, the 400, the stored padded form),
the PWA (the field's default, the controls' normalisation, both bodies).
1008 Python + 953 frontend green.
2026-10-21 13:20:28 +03:00
catarrh aa8c9c6f12 v3.32.0: the packaged-install updater + a repo CHANGELOG
release / release (push) Waiting to run
Reinstalling the package was slow because setup rebuilds pysim's
prerequisites (pyscard compiles from source on Linux), while a clone
updates with a plain `git pull`.

- tools/update.py (+ update.sh / update.bat in the package, and the
  release workflow shipping them) updates a packaged install in place:
  it downloads the latest release package, extracts it (zip-slip guard,
  permission bits restored - zipfile drops them), replaces the app files
  over the app directory and re-installs the app into the existing
  .venv.  The .venv and simple_lastrun.log are not in the package, so
  pysim and its prerequisites survive: seconds, not a rebuild.
  --check / --version / --zip (offline) / --no-pip / --force; the same
  version is a no-op.
- CHANGELOG.md (Keep a Changelog, from the first packaged release;
  earlier history is the git log).  The release workflow now uses the
  version's CHANGELOG section as the release body
  (tools/changelog_section.py, falling back to generated notes).
- The READMEs document the update path and the `git clone --depth 1`
  option for the clone path; setup prints the update hint.
- Version strings + the PWA cache bumped together (simple-v458).

Verified: 1007 Python + 953 frontend; the release-package test covers
the new files; the updater was exercised end to end (a v3.31.2 install
with a real venv updated to the 3.32.0 package: files replaced, the
venv marker kept, pip metadata 3.32.0, exec bits kept, a second run a
no-op, --check correct).
2026-10-21 00:35:47 +03:00
24 changed files with 617 additions and 26 deletions
+10 -2
View File
@@ -79,9 +79,17 @@ jobs:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
run: | run: |
TAG="${{ github.event.inputs.tag || github.ref_name }}" TAG="${{ github.event.inputs.tag || github.ref_name }}"
ZIP="dist/simple-${TAG#v}.zip" VER="${TAG#v}"
ZIP="dist/simple-$VER.zip"
# The release body is the CHANGELOG section of the version; a missing
# section falls back to generated notes.
if python tools/changelog_section.py "$VER" > /tmp/notes.md 2>/dev/null; then
NOTES="--notes-file /tmp/notes.md"
else
NOTES="--generate-notes"
fi
if gh release view "$TAG" >/dev/null 2>&1; then if gh release view "$TAG" >/dev/null 2>&1; then
gh release upload "$TAG" "$ZIP" --clobber gh release upload "$TAG" "$ZIP" --clobber
else else
gh release create "$TAG" "$ZIP" --title "$TAG" --generate-notes gh release create "$TAG" "$ZIP" --title "$TAG" $NOTES
fi fi
+73
View File
@@ -0,0 +1,73 @@
# Changelog
All notable changes to SIMple (the PWA and `pysim-simple-server`).
The format follows [Keep a Changelog](https://keepachangelog.com/);
versions use `MAJOR.MINOR.PATCH` (see the README).
History before the first packaged release (v3.31.1) lives in the git log —
every change carries a descriptive commit message.
## [Unreleased]
## [3.32.1] — 2026-10-21
### Added
- **"Start counter value"** in the outfile import: the counter every created keyset
starts from. It defaults to the store's value (`1`); raise it for cards whose
counters are already past it. The API takes `start_counter` (1–10 hex digits;
a bad value is a 400), and the preview carries the value.
## [3.32.0] — 2026-10-21
### Added
- **Updating a packaged install** — `update.sh` / `update.bat` (and `tools/update.py`
beneath them): downloads the latest release package, replaces the application files
in place and re-installs the app into the **existing `.venv`**. pysim and its
prerequisites (the slow part: pyscard builds from source on Linux) are kept, so an
update takes seconds. `--check` reports the current and latest versions, `--version`
picks a specific release, `--zip` updates from a local package (offline), `--no-pip`
refreshes the files only, `--force` re-applies the same version.
- The release workflow now takes the release notes from this file's matching version
section (falling back to generated notes when it is missing).
### Changed
- Installs made from the v3.31.x packages have no updater yet: copy `update.sh`,
`update.bat` and `tools/update.py` from this package once (or re-run `setup` from it).
Later updates use `update.sh` / `update.bat` — or `git pull` for a clone.
## [3.31.2] — 2026-10-20
### Changed
- The **SCP81 TLS-PSK listener states its Python requirement**: it needs Python 3.13+
(`ssl`'s PSK callbacks exist only since 3.13). Below that it raises a clear error
instead of `AttributeError`, and the PSK tests are skipped (they run on 3.13+).
- The READMEs and the setup scripts: **Windows strongly recommends Python 3.13**
(pyscard has wheels up to 3.13; on 3.14 pyscard and some pysim dependency wheels are
unavailable and build from source with MSVC).
## [3.31.1] — 2026-10-20
The first packaged release (`simple-3.31.1.zip`).
### Added
- **EF_PBR (phonebook reference) support** — the PBR is decoded in the decoded file
views (the `A8`/`A9`/`AA` containers, the `C0`–`CC` kinds, the master EF), and the
phonebook files it references (they are not in pySim's model) are added to the file
tree server-side, selectable and readable.
- **In-app dialogs** — the native alert/confirm/prompt dialogs are replaced by a shared
modal (queued, keyboard-aware, focus-trapped, ARIA-labelled).
- **Windows console encoding guard** — the `start.bat` PowerShell pipe can no longer
kill the server on a character the ANSI code page lacks.
- **GPL-2.0-or-later licensing** (`LICENSE`, `THIRD-PARTY-NOTICES`), matching pysim.
- The curated **release package** (`tools/build_release.py`) and the tag-triggered
`.github/workflows/release.yml`.
### Changed
- `setup.sh` / `setup.bat` and the READMEs: on Linux `pyscard` builds from source and
needs the PC/SC headers (`libpcsclite-dev`).
+9 -2
View File
@@ -30,11 +30,17 @@ The package is built from the tagged source — no git history, no development f
Git is still required: `setup` installs **pysim** from its upstream repository Git is still required: `setup` installs **pysim** from its upstream repository
(nothing from pysim or its dependencies is bundled). (nothing from pysim or its dependencies is bundled).
**Updates are fast:** `./update.sh` (or `update.bat`) downloads the latest release
package, replaces the application files in place and re-installs the app into the
existing `.venv` — pysim and its prerequisites are kept, so nothing is rebuilt.
`./update.sh --check` reports the current and latest versions, `--version X.Y.Z`
picks a specific release and `--zip file.zip` updates from a local package.
**2. Clone the repository (for development, or changes not released yet).** **2. Clone the repository (for development, or changes not released yet).**
```sh ```sh
git clone https://github.com/anttro/simple.git git clone https://github.com/anttro/simple.git # or --depth 1: no history,
cd simple cd simple # `git pull` still updates it
./setup.sh # or setup.bat on Windows ./setup.sh # or setup.bat on Windows
./start.sh # or start.bat ./start.sh # or start.bat
``` ```
@@ -748,6 +754,7 @@ The bundled Python server wraps [pySim](https://osmocom.org/projects/pysim/wiki)
|--------|-------------| |--------|-------------|
| `setup.sh` / `setup.bat` | Creates `.venv/`, installs pysim and the server. Run once after cloning or after unzipping the release package. | | `setup.sh` / `setup.bat` | Creates `.venv/`, installs pysim and the server. Run once after cloning or after unzipping the release package. |
| `start.sh` / `start.bat` | Starts the server from the venv (serves the PWA + API on `:8080`). | | `start.sh` / `start.bat` | Starts the server from the venv (serves the PWA + API on `:8080`). |
| `update.sh` / `update.bat` | Updates a packaged install in place: fetches the latest release package, replaces the app files and re-installs the app into the existing `.venv` (pysim is kept). `--check`, `--version X.Y.Z`, `--zip FILE`, `--no-pip`, `--force`. |
`start.sh` asks for the first PC/SC reader when `pcscd` is running; otherwise the server still auto-detects a PC/SC reader itself, and with none present pysim falls back to its serial default `/dev/ttyUSB0` (pass `-d <device>` for a serial/FTDI reader). `start.bat` always passes `-p 0` (PC/SC is built into Windows). Extra arguments are passed through to the server, e.g. `./start.sh --gsmtap` or `start.bat --gsmtap 10.0.0.5:4729`. If no reader is found the server still starts ("Reader: none") — initialize the card later via the **Equip** button. `start.sh` asks for the first PC/SC reader when `pcscd` is running; otherwise the server still auto-detects a PC/SC reader itself, and with none present pysim falls back to its serial default `/dev/ttyUSB0` (pass `-d <device>` for a serial/FTDI reader). `start.bat` always passes `-p 0` (PC/SC is built into Windows). Extra arguments are passed through to the server, e.g. `./start.sh --gsmtap` or `start.bat --gsmtap 10.0.0.5:4729`. If no reader is found the server still starts ("Reader: none") — initialize the card later via the **Equip** button.
+9 -2
View File
@@ -30,11 +30,17 @@ SIMple — инструментарий для специалистов по UIC
разработки. Git всё равно нужен: `setup` ставит **pysim** из его upstream-репозитория разработки. Git всё равно нужен: `setup` ставит **pysim** из его upstream-репозитория
(ни pysim, ни его зависимости в пакет не входят). (ни pysim, ни его зависимости в пакет не входят).
**Обновление быстрое:** `./update.sh` (или `update.bat`) скачивает последний релизный
пакет, заменяет файлы приложения на месте и переустанавливает приложение в существующий
`.venv` — pysim и его зависимости остаются, ничего не пересобирается.
`./update.sh --check` показывает текущую и последнюю версии, `--version X.Y.Z` —
конкретный релиз, `--zip file.zip` — обновление из локального пакета.
**2. Клонировать репозиторий (для разработки или ещё не выпущенных изменений).** **2. Клонировать репозиторий (для разработки или ещё не выпущенных изменений).**
```sh ```sh
git clone https://github.com/anttro/simple.git git clone https://github.com/anttro/simple.git # или --depth 1: без истории,
cd simple cd simple # `git pull` всё равно обновляет
./setup.sh # или setup.bat в Windows ./setup.sh # или setup.bat в Windows
./start.sh # или start.bat ./start.sh # или start.bat
``` ```
@@ -732,6 +738,7 @@ PWA проверяет версию сервера при подключении
|--------|-------------| |--------|-------------|
| `setup.sh` / `setup.bat` | Создаёт `.venv/`, ставит pysim и сервер. Запускать один раз после клонирования или распаковки релизного пакета. | | `setup.sh` / `setup.bat` | Создаёт `.venv/`, ставит pysim и сервер. Запускать один раз после клонирования или распаковки релизного пакета. |
| `start.sh` / `start.bat` | Запускает сервер из venv (раздаёт PWA + API на `:8080`). | | `start.sh` / `start.bat` | Запускает сервер из venv (раздаёт PWA + API на `:8080`). |
| `update.sh` / `update.bat` | Обновляет пакетную установку на месте: скачивает последний релизный пакет, заменяет файлы приложения и переустанавливает приложение в существующий `.venv` (pysim сохраняется). `--check`, `--version X.Y.Z`, `--zip FILE`, `--no-pip`, `--force`. |
`start.sh` при работающем `pcscd` запрашивает первый PC/SC-ридер; иначе сервер сам автоопределяет PC/SC-ридер, а если его нет — pysim откатывается на последовательное устройство по умолчанию `/dev/ttyUSB0` (для serial/FTDI-ридера укажите `-d <устройство>`). `start.bat` всегда передаёт `-p 0`. Дополнительные аргументы передаются серверу, например `./start.sh --gsmtap` или `start.bat --gsmtap 10.0.0.5:4729`. Без ридера сервер всё равно стартует («Reader: none») — карту можно инициализировать позже кнопкой **Equip**. `start.sh` при работающем `pcscd` запрашивает первый PC/SC-ридер; иначе сервер сам автоопределяет PC/SC-ридер, а если его нет — pysim откатывается на последовательное устройство по умолчанию `/dev/ttyUSB0` (для serial/FTDI-ридера укажите `-d <устройство>`). `start.bat` всегда передаёт `-p 0`. Дополнительные аргументы передаются серверу, например `./start.sh --gsmtap` или `start.bat --gsmtap 10.0.0.5:4729`. Без ридера сервер всё равно стартует («Reader: none») — карту можно инициализировать позже кнопкой **Equip**.
+4
View File
@@ -351,6 +351,10 @@ into a preview.
- `tar_keyset` — an optional **global default keyset** (1–15) for the TAR - `tar_keyset` — an optional **global default keyset** (1–15) for the TAR
entries that do not name one (the API's shorthand; the PWA sends per-row entries that do not name one (the API's shorthand; the PWA sends per-row
keysets instead). keysets instead).
- `start_counter` — the counter **every created keyset starts from** (1–10 hex
digits; whitespace stripped, uppercased). Without it the store's default
(`1`) applies. A bad value is a `400`; the import stores the protocol's
10-digit form.
Recognised columns (case-insensitive): `ICCID` (the digits + odd-nibble F Recognised columns (case-insensitive): `ICCID` (the digits + odd-nibble F
pad, the plain digits, spaced digits or the raw nibble-swapped EF hex all pad, the plain digits, spaced digits or the raw nibble-swapped EF hex all
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -371,7 +371,7 @@
</tbody> </tbody>
</table> </table>
<p class="text-sm mb-3">The <strong>SCP81</strong> column shows whether the preset supplies a usable PSK pair: <strong>&#10003;</strong> (green) — identity and key set; <strong>&#9888;</strong> (amber) — only one of the two, so the listener ignores the preset; <strong>&mdash;</strong> — no PSK. PSK identity and key must be set together. <strong>Add preset</strong> opens the editor with a fresh preset; <strong>Edit</strong> loads the row's preset into it so fields can be changed without re-entering the card; <strong>Save</strong> stores it and returns to the list, <strong>Cancel</strong> discards the edits; <strong>Remove</strong> deletes the row (list only). Presets can be shared with <strong>Export to file</strong> and restored with <strong>Import from file</strong>. The selected card preset auto-fills the Secured Packet form and sets the TAR for the operation (RAM/GP &rarr; ISD TAR, SIM RFM &rarr; UICC RFM TAR, USIM RFM &rarr; ADF RFM TAR); a TAR typed into the Secured Packet form by hand is kept when sending - the counter, keys and the TAR's MSL are still re-read from the preset; the SPI1 is seeded from the TAR's MSL and a lower value warns (the server returns it as a <code class="font-mono text-sm">warning</code> too); edits are pushed into a running SCP81 listener automatically.</p> <p class="text-sm mb-3">The <strong>SCP81</strong> column shows whether the preset supplies a usable PSK pair: <strong>&#10003;</strong> (green) — identity and key set; <strong>&#9888;</strong> (amber) — only one of the two, so the listener ignores the preset; <strong>&mdash;</strong> — no PSK. PSK identity and key must be set together. <strong>Add preset</strong> opens the editor with a fresh preset; <strong>Edit</strong> loads the row's preset into it so fields can be changed without re-entering the card; <strong>Save</strong> stores it and returns to the list, <strong>Cancel</strong> discards the edits; <strong>Remove</strong> deletes the row (list only). Presets can be shared with <strong>Export to file</strong> and restored with <strong>Import from file</strong>. The selected card preset auto-fills the Secured Packet form and sets the TAR for the operation (RAM/GP &rarr; ISD TAR, SIM RFM &rarr; UICC RFM TAR, USIM RFM &rarr; ADF RFM TAR); a TAR typed into the Secured Packet form by hand is kept when sending - the counter, keys and the TAR's MSL are still re-read from the preset; the SPI1 is seeded from the TAR's MSL and a lower value warns (the server returns it as a <code class="font-mono text-sm">warning</code> too); edits are pushed into a running SCP81 listener automatically.</p>
<p class="text-sm mb-3"><strong>Import outfile</strong> imports many cards at once from a card vendor's <em>outfile</em> — the personalization tool's plain-text dump: a banner and a configuration header, then a whitespace-separated <strong>column line</strong> (the tokens include <code class="font-mono text-sm">ICCID</code>; the first token may carry a <code class="font-mono text-sm">var_out:</code>-style prefix) and one row per card. Everything before the column line and every unused column is ignored. Every data row must carry a value for <strong>every declared column</strong> &mdash; the format is positional, so an empty value is indistinguishable from a missing one &mdash; and a <code class="font-mono text-sm">KICn</code>/<code class="font-mono text-sm">KIDn</code> pair named in the header must have a usable value in every row; a violation makes the whole file invalid (the preview reports the line and the import is refused). The parser reads <code class="font-mono text-sm">ICCID</code> (digits, the digits with an odd-nibble <code class="font-mono text-sm">F</code> pad, spaced digits or the raw EF hex — all normalised to one form), <code class="font-mono text-sm">PIN1</code>/<code class="font-mono text-sm">PIN2</code>/<code class="font-mono text-sm">PUK1</code>/<code class="font-mono text-sm">PUK2</code>, <code class="font-mono text-sm">ADM</code> (the key's hex value, 4&ndash;16 hex digits in byte pairs — e.g. <code class="font-mono text-sm">3632333239463337</code>), <code class="font-mono text-sm">PSK</code> (16 bytes; an optional <code class="font-mono text-sm">PSKID</code>/<code class="font-mono text-sm">PSK_ID</code> names its identity, defaulting to the card's ICCID) and the numbered key pairs <code class="font-mono text-sm">KICn</code>/<code class="font-mono text-sm">KIDn</code> — one keyset per pair, up to 15. The <strong>KIc/KID bytes are never in the file</strong>: the keyset number is the column number (<code class="font-mono text-sm">KIC2</code> &rarr; keyset 2) and the algorithm nibble follows the key size — 8 bytes DES, 24 bytes 3DES3, and for <strong>16 bytes</strong> the preview's choice, <strong>3DES2</strong> (default) or <strong>AES</strong>; any other size skips that keyset with a note. The TARs never come from the file either: the preview shows the TAR table the imported cards get (the three role entries with their spec TARs and MSL 16 by default, editable, with <strong>+ TAR</strong> for extra entries) and <strong>every row carries its own keyset</strong> — a select over the keysets found in the file (the first one is preselected; a non-zero MSL needs one, marked amber), so each TAR's packets use exactly the keyset you choose. A <strong>common name</strong> is applied to every imported card (empty falls back to the ICCID); <strong>Import mode</strong> chooses <strong>Merge</strong> (default — existing presets are kept, duplicate ICCIDs are skipped and reported) or <strong>Replace</strong> (the store is wiped first). The preview lists the parsed cards (ICCID, keysets, the codes found) and every skipped row/key with its reason before anything is stored.</p> <p class="text-sm mb-3"><strong>Import outfile</strong> imports many cards at once from a card vendor's <em>outfile</em> — the personalization tool's plain-text dump: a banner and a configuration header, then a whitespace-separated <strong>column line</strong> (the tokens include <code class="font-mono text-sm">ICCID</code>; the first token may carry a <code class="font-mono text-sm">var_out:</code>-style prefix) and one row per card. Everything before the column line and every unused column is ignored. Every data row must carry a value for <strong>every declared column</strong> &mdash; the format is positional, so an empty value is indistinguishable from a missing one &mdash; and a <code class="font-mono text-sm">KICn</code>/<code class="font-mono text-sm">KIDn</code> pair named in the header must have a usable value in every row; a violation makes the whole file invalid (the preview reports the line and the import is refused). The parser reads <code class="font-mono text-sm">ICCID</code> (digits, the digits with an odd-nibble <code class="font-mono text-sm">F</code> pad, spaced digits or the raw EF hex — all normalised to one form), <code class="font-mono text-sm">PIN1</code>/<code class="font-mono text-sm">PIN2</code>/<code class="font-mono text-sm">PUK1</code>/<code class="font-mono text-sm">PUK2</code>, <code class="font-mono text-sm">ADM</code> (the key's hex value, 4&ndash;16 hex digits in byte pairs — e.g. <code class="font-mono text-sm">3632333239463337</code>), <code class="font-mono text-sm">PSK</code> (16 bytes; an optional <code class="font-mono text-sm">PSKID</code>/<code class="font-mono text-sm">PSK_ID</code> names its identity, defaulting to the card's ICCID) and the numbered key pairs <code class="font-mono text-sm">KICn</code>/<code class="font-mono text-sm">KIDn</code> — one keyset per pair, up to 15. The <strong>KIc/KID bytes are never in the file</strong>: the keyset number is the column number (<code class="font-mono text-sm">KIC2</code> &rarr; keyset 2) and the algorithm nibble follows the key size — 8 bytes DES, 24 bytes 3DES3, and for <strong>16 bytes</strong> the preview's choice, <strong>3DES2</strong> (default) or <strong>AES</strong>; any other size skips that keyset with a note. The TARs never come from the file either: the preview shows the TAR table the imported cards get (the three role entries with their spec TARs and MSL 16 by default, editable, with <strong>+ TAR</strong> for extra entries) and <strong>every row carries its own keyset</strong> — a select over the keysets found in the file (the first one is preselected; a non-zero MSL needs one, marked amber), so each TAR's packets use exactly the keyset you choose. <strong>Start counter value</strong> sets the counter every created keyset starts from (default <code class="font-mono text-sm">1</code> &mdash; the store's value; raise it for cards already past it). A <strong>common name</strong> is applied to every imported card (empty falls back to the ICCID); <strong>Import mode</strong> chooses <strong>Merge</strong> (default — existing presets are kept, duplicate ICCIDs are skipped and reported) or <strong>Replace</strong> (the store is wiped first). The preview lists the parsed cards (ICCID, keysets, the codes found) and every skipped row/key with its reason before anything is stored.</p>
</section> </section>
<section class="mb-10"> <section class="mb-10">
+14 -4
View File
@@ -1943,7 +1943,7 @@
// ===== Version ===== // ===== Version =====
// Single source of truth for the PWA version: shown in the header and used // Single source of truth for the PWA version: shown in the header and used
// by the server version check in pysimConnect(). // by the server version check in pysimConnect().
const SIMPLE_VERSION = '3.31.2'; const SIMPLE_VERSION = '3.32.1';
document.getElementById('app-version').textContent = 'v' + SIMPLE_VERSION; document.getElementById('app-version').textContent = 'v' + SIMPLE_VERSION;
// ===== view history (the browser Back/Forward buttons) ===== // ===== view history (the browser Back/Forward buttons) =====
@@ -15154,7 +15154,7 @@ async function cardsOutfileFile(input) {
} }
_cardsOutfile = { text: text, file: String(file.name || ''), _cardsOutfile = { text: text, file: String(file.name || ''),
name: String(file.name || '').replace(/\.[^.]+$/, ''), name: String(file.name || '').replace(/\.[^.]+$/, ''),
algorithm16: '3des2', mode: 'merge', preview: null }; algorithm16: '3des2', cntr: '1', mode: 'merge', preview: null };
_cardsOutfileTars = 0; _cardsOutfileTars = 0;
cardsOutfileRender(); cardsOutfileRender();
await cardsOutfilePreview(); await cardsOutfilePreview();
@@ -15177,6 +15177,7 @@ function cardsOutfileControls() {
const val = id => { const el = document.getElementById(id); return el ? String(el.value) : ''; }; const val = id => { const el = document.getElementById(id); return el ? String(el.value) : ''; };
if (document.getElementById('cards-of-name')) o.name = val('cards-of-name').trim(); if (document.getElementById('cards-of-name')) o.name = val('cards-of-name').trim();
if (document.getElementById('cards-of-algo')) o.algorithm16 = val('cards-of-algo') || '3des2'; if (document.getElementById('cards-of-algo')) o.algorithm16 = val('cards-of-algo') || '3des2';
if (document.getElementById('cards-of-cntr')) o.cntr = val('cards-of-cntr').trim().toUpperCase();
if (document.getElementById('cards-of-mode')) { if (document.getElementById('cards-of-mode')) {
o.mode = val('cards-of-mode') === 'replace' ? 'replace' : 'merge'; o.mode = val('cards-of-mode') === 'replace' ? 'replace' : 'merge';
} }
@@ -15371,6 +15372,13 @@ function cardsOutfileRender() {
'<select id="cards-of-mode" class="block mt-0.5 border border-gray-300 dark:border-slate-600 text-sm rounded px-2 py-1 dark:bg-slate-800">' + '<select id="cards-of-mode" class="block mt-0.5 border border-gray-300 dark:border-slate-600 text-sm rounded px-2 py-1 dark:bg-slate-800">' +
'<option value="merge"' + (o.mode === 'replace' ? '' : ' selected') + '>' + esc(t('Merge')) + '</option>' + '<option value="merge"' + (o.mode === 'replace' ? '' : ' selected') + '>' + esc(t('Merge')) + '</option>' +
'<option value="replace"' + (o.mode === 'replace' ? ' selected' : '') + '>' + esc(t('Replace')) + '</option></select></label>'; '<option value="replace"' + (o.mode === 'replace' ? ' selected' : '') + '>' + esc(t('Replace')) + '</option></select></label>';
// The counter every created keyset starts from (v3.32.1): the store's
// default, editable for cards whose counter is already past it.
html += '<label class="text-xs text-gray-600 dark:text-slate-400" title="' +
esc(t('The counter every created keyset starts from (1-10 hex digits); the store default is 1')) + '">' +
esc(t('Start counter value')) +
'<input id="cards-of-cntr" class="block mt-0.5 w-24 border border-gray-300 dark:border-slate-600 text-sm rounded px-2 py-1 dark:bg-slate-800 font-mono" maxlength="10" value="' +
escHtml(o.cntr || '1') + '" onchange="cardsOutfilePreview()"></label>';
html += '</div>'; html += '</div>';
html += '<div class="text-xs text-gray-600 dark:text-slate-400 mb-1">' + esc(t('TAR table')) + '</div>'; html += '<div class="text-xs text-gray-600 dark:text-slate-400 mb-1">' + esc(t('TAR table')) + '</div>';
html += '<div id="cards-of-tars">' + cardsOutfileTarRowsHtml() + '</div>'; html += '<div id="cards-of-tars">' + cardsOutfileTarRowsHtml() + '</div>';
@@ -15393,7 +15401,7 @@ async function cardsOutfilePreview() {
cardsOutfileControls(); cardsOutfileControls();
const body = { const body = {
text: o.text, name: o.name, algorithm16: o.algorithm16, text: o.text, name: o.name, algorithm16: o.algorithm16,
tars: cardsOutfileTarsFromForm(), tars: cardsOutfileTarsFromForm(), start_counter: o.cntr,
}; };
try { try {
o.preview = await pysimFetchOk('/api/presets/import-outfile', body); o.preview = await pysimFetchOk('/api/presets/import-outfile', body);
@@ -15418,7 +15426,7 @@ async function cardsOutfileImport() {
if (btn) btn.disabled = true; if (btn) btn.disabled = true;
const body = { const body = {
text: o.text, name: o.name, algorithm16: o.algorithm16, text: o.text, name: o.name, algorithm16: o.algorithm16,
tars: cardsOutfileTarsFromForm(), mode: o.mode, tars: cardsOutfileTarsFromForm(), start_counter: o.cntr, mode: o.mode,
}; };
try { try {
const res = await pysimFetchOk('/api/presets/import-outfile', body); const res = await pysimFetchOk('/api/presets/import-outfile', body);
@@ -24012,6 +24020,8 @@ const LANG_RU = {
'Common name': 'Общее имя', 'Common name': 'Общее имя',
'16-byte keys': '16-байтные ключи', '16-byte keys': '16-байтные ключи',
'This TAR needs a keyset (its MSL requires security)': 'Этому TAR нужен keyset (его MSL требует защиты)', 'This TAR needs a keyset (its MSL requires security)': 'Этому TAR нужен keyset (его MSL требует защиты)',
'Start counter value': 'Начальное значение счётчика',
'The counter every created keyset starts from (1-10 hex digits); the store default is 1': 'Счётчик, с которого начинается каждый созданный keyset (1–10 hex-цифр); по умолчанию в хранилище — 1',
'The file is invalid - fix it and refresh the preview': 'Файл некорректен — исправьте его и обновите предпросмотр', 'The file is invalid - fix it and refresh the preview': 'Файл некорректен — исправьте его и обновите предпросмотр',
'Import mode': 'Режим импорта', 'Import mode': 'Режим импорта',
'Merge': 'Дополнить', 'Merge': 'Дополнить',
+1 -1
View File
@@ -1,4 +1,4 @@
const CACHE = 'simple-v457'; const CACHE = 'simple-v459';
const URLS = [ const URLS = [
'index.html', 'index.html',
'help.html', 'help.html',
+8 -2
View File
@@ -113,8 +113,8 @@ test('the preview and the import both post the outfile endpoint', () => {
// the keysets travel per TAR row (v3.29.1): no global tar_keyset // the keysets travel per TAR row (v3.29.1): no global tar_keyset
assert.ok(!preview.includes('tar_keyset'), 'the preview must not send a global keyset'); assert.ok(!preview.includes('tar_keyset'), 'the preview must not send a global keyset');
assert.ok(!imp.includes('tar_keyset'), 'the import must not send a global keyset'); assert.ok(!imp.includes('tar_keyset'), 'the import must not send a global keyset');
assert.match(preview, /tars: cardsOutfileTarsFromForm\(\)/); assert.match(preview, /tars: cardsOutfileTarsFromForm\(\), start_counter: o\.cntr/);
assert.match(imp, /tars: cardsOutfileTarsFromForm\(\), mode: o\.mode/); assert.match(imp, /tars: cardsOutfileTarsFromForm\(\), start_counter: o\.cntr, mode: o\.mode/);
}); });
test('the 16-byte algorithm values match the server map', () => { test('the 16-byte algorithm values match the server map', () => {
@@ -225,10 +225,12 @@ test('cardsOutfileControls reads the panel controls back into the state', () =>
setEl('cards-of-name', ' Batch 7 '); setEl('cards-of-name', ' Batch 7 ');
setEl('cards-of-algo', 'aes'); setEl('cards-of-algo', 'aes');
setEl('cards-of-mode', 'replace'); setEl('cards-of-mode', 'replace');
setEl('cards-of-cntr', ' 1a ');
app.cardsOutfileControls(); app.cardsOutfileControls();
assert.strictEqual(app.state.name, 'Batch 7'); assert.strictEqual(app.state.name, 'Batch 7');
assert.strictEqual(app.state.algorithm16, 'aes'); assert.strictEqual(app.state.algorithm16, 'aes');
assert.strictEqual(app.state.mode, 'replace'); assert.strictEqual(app.state.mode, 'replace');
assert.strictEqual(app.state.cntr, '1A'); // the store's uppercase hex form
}); });
test('picking a file previews it with the file name and the TAR defaults', async () => { test('picking a file previews it with the file name and the TAR defaults', async () => {
@@ -254,6 +256,10 @@ test('picking a file previews it with the file name and the TAR defaults', async
assert.match(els['cards-outfile-panel'].innerHTML, /id="cards-of-tar-0-keyset"/); assert.match(els['cards-outfile-panel'].innerHTML, /id="cards-of-tar-0-keyset"/);
assert.match(els['cards-outfile-panel'].innerHTML, /<option value="1" selected>Keyset 1 — 15\/15<\/option>/); assert.match(els['cards-outfile-panel'].innerHTML, /<option value="1" selected>Keyset 1 — 15\/15<\/option>/);
assert.match(els['cards-outfile-panel'].innerHTML, /8970119000004600098/); assert.match(els['cards-outfile-panel'].innerHTML, /8970119000004600098/);
// the start counter field defaults to the store's value and travels in the body
assert.match(els['cards-outfile-panel'].innerHTML, /id="cards-of-cntr"/);
assert.match(els['cards-outfile-panel'].innerHTML, /value="1"/);
assert.strictEqual(body.start_counter, '1');
}); });
test('a failing preview shows the server message in the panel', async () => { test('a failing preview shows the server message in the panel', async () => {
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "pysim-simple-server" name = "pysim-simple-server"
version = "3.31.2" version = "3.32.1"
description = "HTTP REST server wrapping pysim for the SIMple PWA" description = "HTTP REST server wrapping pysim for the SIMple PWA"
license = { text = "GPL-2.0-or-later" } license = { text = "GPL-2.0-or-later" }
requires-python = ">=3.9" requires-python = ">=3.9"
+19 -6
View File
@@ -184,13 +184,23 @@ def _hex_value(tok):
return v if v and len(v) % 2 == 0 else '' return v if v and len(v) % 2 == 0 else ''
def parse_outfile(text, name='', algorithm16='3des2', tars=None, tar_keyset=None): def parse_outfile(text, name='', algorithm16='3des2', tars=None, tar_keyset=None,
start_counter=None):
"""Parse an outfile into preset dicts (the store's shape). """Parse an outfile into preset dicts (the store's shape).
Returns ``{'presets': [...], 'report': {...}}``. Every skipped row or key ``start_counter`` (1-10 hex digits) is written into every created keyset -
is named in the report (up to ``SKIP_REPORT_MAX`` lines, then counted) - the counter an operation against that keyset starts from; without it the
nothing is silently dropped. Raises ValueError on a malformed TAR table. store's default ('1') applies. Returns
``{'presets': [...], 'report': {...}}``. Every skipped row or key is named
in the report (up to ``SKIP_REPORT_MAX`` lines, then counted) - nothing is
silently dropped. Raises ValueError on a malformed TAR table or counter.
""" """
counter = None
if start_counter not in (None, ''):
counter = re.sub(r'\s', '', str(start_counter)).upper()
if not re.fullmatch(r'[0-9A-F]{1,10}', counter):
raise ValueError('start_counter must be 1-10 hex digits: %r'
% (start_counter,))
columns, line_index = find_columns(text) columns, line_index = find_columns(text)
report = {'columns': columns or [], 'header_lines': max(line_index, 0), report = {'columns': columns or [], 'header_lines': max(line_index, 0),
'rows': 0, 'cards': 0, 'keysets': [], 'tars': 0, 'rows': 0, 'cards': 0, 'keysets': [], 'tars': 0,
@@ -256,8 +266,11 @@ def parse_outfile(text, name='', algorithm16='3des2', tars=None, tar_keyset=None
% (row_no, n, len(kic_key) // 2)) % (row_no, n, len(kic_key) // 2))
continue continue
byte = '%02X' % ((n << 4) | algo) byte = '%02X' % ((n << 4) | algo)
keysets.append({'kic': byte, 'kid': byte, keyset = {'kic': byte, 'kid': byte,
'kicKey': kic_key, 'kidKey': kid_key}) 'kicKey': kic_key, 'kidKey': kid_key}
if counter is not None:
keyset['cntr'] = counter
keysets.append(keyset)
if not keysets: if not keysets:
skip('line %d: no usable keyset' % row_no) skip('line %d: no usable keyset' % row_no)
continue continue
+3 -2
View File
@@ -37,7 +37,7 @@ from osmocom.tlv import BER_TLV_IE
from osmocom.utils import rpad from osmocom.utils import rpad
VERSION = '3.31.2' VERSION = '3.32.1'
MAX_ENVELOPE_SEGMENTS = 5 # max SMS segments for outgoing C-APDU in ENVELOPE MAX_ENVELOPE_SEGMENTS = 5 # max SMS segments for outgoing C-APDU in ENVELOPE
@@ -11610,7 +11610,8 @@ class PysimHandler(BaseHTTPRequestHandler):
name=str(body.get('name') or ''), name=str(body.get('name') or ''),
algorithm16=algorithm16, algorithm16=algorithm16,
tars=tars, tars=tars,
tar_keyset=tar_keyset) tar_keyset=tar_keyset,
start_counter=body.get('start_counter'))
except ValueError as e: except ValueError as e:
resp = {'error': str(e)} resp = {'error': str(e)}
self._send_json(resp, 400) self._send_json(resp, 400)
+1
View File
@@ -86,4 +86,5 @@ echo.
echo === Setup complete === echo === Setup complete ===
echo Run start.bat to start the server. echo Run start.bat to start the server.
echo Updating later: update.bat ^(reuses .venv - pysim is not re-installed^).
pause pause
+1
View File
@@ -63,3 +63,4 @@ fi
echo "" echo ""
echo "=== Setup complete ===" echo "=== Setup complete ==="
echo "Run ./start.sh to start the server." echo "Run ./start.sh to start the server."
echo "Updating later: ./update.sh (reuses .venv - pysim is not re-installed)."
+20
View File
@@ -979,6 +979,26 @@ class PresetStoreHttpTests(unittest.TestCase):
self.assertEqual(status, 400, resp) self.assertEqual(status, 400, resp)
self.assertIn('the file is invalid', resp['error']) self.assertIn('the file is invalid', resp['error'])
self.assertEqual(len(self.store.list()), before) self.assertEqual(len(self.store.list()), before)
# the import form's start counter (v3.32.1): every created keyset
# carries it; an unusable value is a 400
status, resp = self._post('/api/presets/import-outfile', {
'text': text, 'start_counter': '1A'})
self.assertEqual(status, 200, resp)
self.assertEqual([k.get('cntr') for k in resp['presets'][0]['keysets']],
['1A'] * len(resp['presets'][0]['keysets']))
status, resp = self._post('/api/presets/import-outfile', {
'text': text, 'start_counter': 'XYZ'})
self.assertEqual(status, 400, resp)
self.assertIn('start_counter', resp['error'])
# ...and an import stores the protocol's padded form (a fresh ICCID:
# merge mode skips a duplicate)
fresh = text.replace('8970119000004600098', '8970119000004600999')
status, resp = self._post('/api/presets/import-outfile', {
'text': fresh, 'start_counter': '1A', 'mode': 'merge'})
self.assertEqual(status, 200, resp)
got = [p for p in self.store.list()
if p['iccid'] == '8970119000004600999']
self.assertEqual([k['cntr'] for k in got[0]['keysets']], ['000000001A'])
# a non-list TAR table is the caller's error, never a silent fallback # a non-list TAR table is the caller's error, never a silent fallback
status, resp = self._post('/api/presets/import-outfile', { status, resp = self._post('/api/presets/import-outfile', {
'text': text, 'tars': 'nope'}) 'text': text, 'tars': 'nope'})
+20
View File
@@ -219,6 +219,26 @@ class OptionTests(unittest.TestCase):
with self.assertRaises(ValueError): with self.assertRaises(ValueError):
outfile.parse_outfile(text, tars=[{'tar': '000000', 'msl': '16', 'keyset': 'x'}]) outfile.parse_outfile(text, tars=[{'tar': '000000', 'msl': '16', 'keyset': 'x'}])
def test_the_start_counter_is_written_into_every_keyset(self):
"""The import form's "Start counter value": every created keyset
carries it (the store pads it to the protocol's 10-digit form); without
it the store's default ('1') applies. An unusable value is the
caller's error, never a silent default (v3.32.1)."""
res = outfile.parse_outfile(alfa_text(), start_counter='1a')
ks = res['presets'][0]['keysets']
self.assertGreaterEqual(len(ks), 2)
self.assertEqual([k.get('cntr') for k in ks], ['1A'] * len(ks))
# normalised: whitespace stripped, uppercase
res = outfile.parse_outfile(alfa_text(), start_counter=' a1 ')
self.assertEqual(res['presets'][0]['keysets'][0]['cntr'], 'A1')
# unset: no field at all - the store fills its default
res = outfile.parse_outfile(alfa_text())
self.assertNotIn('cntr', res['presets'][0]['keysets'][0])
# an unusable value refuses the whole parse
for bad in ('XYZ', '1G', '1234567890A', ' '):
with self.assertRaises(ValueError):
outfile.parse_outfile(alfa_text(), start_counter=bad)
def test_a_malformed_tar_table_raises(self): def test_a_malformed_tar_table_raises(self):
with self.assertRaises(ValueError): with self.assertRaises(ValueError):
outfile.parse_outfile(alfa_text(), tars=[{'tar': 'ZZ', 'msl': '16'}]) outfile.parse_outfile(alfa_text(), tars=[{'tar': 'ZZ', 'msl': '16'}])
+2
View File
@@ -69,6 +69,8 @@ class ReleasePackageTests(unittest.TestCase):
'pysim_simple_server/data/mcc-mnc-list.json', 'pysim_simple_server/data/mcc-mnc-list.json',
'pysim_simple_server/data/mcc-mnc-list.LICENSE', 'pysim_simple_server/data/mcc-mnc-list.LICENSE',
'setup.sh', 'setup.bat', 'start.sh', 'start.bat', 'setup.sh', 'setup.bat', 'start.sh', 'start.bat',
'update.sh', 'update.bat', 'tools/update.py',
'tools/changelog_section.py', 'CHANGELOG.md',
'pyproject.toml', 'requirements-pysim.txt', 'pyproject.toml', 'requirements-pysim.txt',
'README.md', 'README_RUS.md', 'LICENSE', 'THIRD-PARTY-NOTICES', 'README.md', 'README_RUS.md', 'LICENSE', 'THIRD-PARTY-NOTICES',
'docs/api.md'): 'docs/api.md'):
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env python3
"""Updating a packaged install in place (tools/update.py).
The update replaces the application files, keeps the .venv (pysim and its
prerequisites - the slow part of a reinstall) and the run log, preserves the
scripts' exec bits, and refuses a package whose version does not match or that
carries unsafe paths. tests/test_release_package.py pins that the update
scripts are in the package.
"""
import importlib.util
import os
import re
import subprocess
import sys
import tempfile
import unittest
import zipfile
from pathlib import Path
REPO = Path(__file__).resolve().parents[1]
UPDATE = REPO / 'tools' / 'update.py'
CHANGELOG_SECTION = REPO / 'tools' / 'changelog_section.py'
def _load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def _package(path, version, entries=None):
"""A minimal release package: simple-<version>/... with the real updater."""
root = 'simple-%s/' % version
with zipfile.ZipFile(path, 'w') as z:
def add(name, data, mode=0o644):
info = zipfile.ZipInfo(root + name)
info.external_attr = mode << 16
z.writestr(info, data)
add('pysim_simple_server/server.py', "VERSION = '%s'\n" % version)
add('README.md', '# readme %s\n' % version)
add('update.sh', '#!/bin/sh\n', 0o755)
add('tools/update.py', UPDATE.read_text(encoding='utf-8'))
for name, data in (entries or {}).items():
add(name, data)
return path
def _app(tmp):
"""A simulated packaged install: app files, a .venv and the run log."""
app = Path(tmp) / 'app'
(app / 'tools').mkdir(parents=True)
(app / 'tools' / 'update.py').write_text(UPDATE.read_text(encoding='utf-8'))
(app / 'pysim_simple_server').mkdir()
(app / 'pysim_simple_server' / 'server.py').write_text("VERSION = '1.0.0'\n")
(app / 'README.md').write_text('# old\n')
(app / '.venv').mkdir()
(app / '.venv' / 'MARKER').write_text('venv')
(app / 'simple_lastrun.log').write_text('log')
return app
class InPlaceUpdateTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.dir = Path(self.tmp.name)
self.app = _app(self.tmp.name)
self.zip = _package(self.dir / 'simple-2.0.0.zip', '2.0.0',
{'frontend/new-asset.txt': 'new'})
def tearDown(self):
self.tmp.cleanup()
def run_update(self, *args):
return subprocess.run(
[sys.executable, str(self.app / 'tools' / 'update.py')] + list(args),
capture_output=True, text=True)
def test_the_files_are_replaced_and_the_venv_and_log_kept(self):
r = self.run_update('--zip', str(self.zip), '--no-pip')
self.assertEqual(r.returncode, 0, r.stderr)
self.assertIn("VERSION = '2.0.0'",
(self.app / 'pysim_simple_server' / 'server.py').read_text())
self.assertEqual((self.app / 'README.md').read_text(), '# readme 2.0.0\n')
self.assertEqual((self.app / 'frontend' / 'new-asset.txt').read_text(), 'new')
# pysim lives in .venv - it must survive untouched, as does the run log
self.assertEqual((self.app / '.venv' / 'MARKER').read_text(), 'venv')
self.assertEqual((self.app / 'simple_lastrun.log').read_text(), 'log')
self.assertIn('restart the server', r.stdout)
@unittest.skipIf(os.name == 'nt', 'POSIX permission bits')
def test_the_scripts_keep_their_exec_bits(self):
self.run_update('--zip', str(self.zip), '--no-pip')
# zipfile drops the mode on extraction; the updater restores it
self.assertTrue(os.access(self.app / 'update.sh', os.X_OK))
def test_the_same_version_is_not_reapplied_unless_forced(self):
self.run_update('--zip', str(self.zip), '--no-pip')
r = self.run_update('--zip', str(self.zip), '--no-pip')
self.assertEqual(r.returncode, 0)
self.assertIn('already up to date', r.stdout)
(self.app / 'README.md').write_text('# edited\n')
r = self.run_update('--zip', str(self.zip), '--no-pip', '--force')
self.assertEqual(r.returncode, 0)
self.assertEqual((self.app / 'README.md').read_text(), '# readme 2.0.0\n')
def test_check_reports_both_versions_without_touching_anything(self):
r = self.run_update('--check', '--version', '2.0.0')
self.assertEqual(r.returncode, 0)
self.assertIn('current: 1.0.0', r.stdout)
self.assertIn('latest: 2.0.0', r.stdout)
self.assertEqual((self.app / 'README.md').read_text(), '# old\n')
def test_a_version_mismatch_is_refused(self):
bad = self.dir / 'simple-9.9.9.zip'
_package(bad, '8.8.8') # the file name says 9.9.9
r = self.run_update('--zip', str(bad), '--no-pip', '--version', '9.9.9')
self.assertNotEqual(r.returncode, 0)
self.assertIn('does not match', r.stderr)
self.assertEqual((self.app / 'pysim_simple_server' / 'server.py').read_text(),
"VERSION = '1.0.0'\n")
def test_an_unsafe_path_is_refused(self):
bad = self.dir / 'simple-2.0.0-evil.zip'
with zipfile.ZipFile(bad, 'w') as z:
z.writestr('simple-2.0.0/../../evil.txt', 'x')
z.writestr('simple-2.0.0/pysim_simple_server/server.py', "VERSION = '2.0.0'\n")
r = self.run_update('--zip', str(bad), '--no-pip')
self.assertNotEqual(r.returncode, 0)
self.assertIn('unsafe path', r.stderr)
self.assertFalse((self.dir / 'evil.txt').exists())
class ChangelogSectionTests(unittest.TestCase):
def setUp(self):
self.mod = _load(CHANGELOG_SECTION, 'changelog_section')
def test_the_current_version_has_a_section(self):
# the server version is the single source of truth
text = (REPO / 'pysim_simple_server' / 'server.py').read_text(encoding='utf-8')
ver = re.search(r"^VERSION = '([^']+)'$", text, re.M).group(1)
body = self.mod.section(ver)
self.assertIsNotNone(body, 'CHANGELOG.md has no [%s] section' % ver)
self.assertIn('### ', body)
def test_an_unknown_version_has_no_section(self):
self.assertIsNone(self.mod.section('0.0.0'))
def test_the_helper_exits_non_zero_for_an_unknown_version(self):
r = subprocess.run([sys.executable, str(CHANGELOG_SECTION), '0.0.0'],
capture_output=True, text=True)
self.assertNotEqual(r.returncode, 0)
if __name__ == '__main__':
unittest.main()
+2 -1
View File
@@ -37,8 +37,9 @@ PACKAGE_ENTRIES = (
'docs', 'docs',
'tools', 'tools',
'setup.sh', 'setup.bat', 'start.sh', 'start.bat', 'setup.sh', 'setup.bat', 'start.sh', 'start.bat',
'update.sh', 'update.bat',
'pyproject.toml', 'requirements-pysim.txt', 'pyproject.toml', 'requirements-pysim.txt',
'README.md', 'README_RUS.md', 'README.md', 'README_RUS.md', 'CHANGELOG.md',
'LICENSE', 'THIRD-PARTY-NOTICES', 'LICENSE', 'THIRD-PARTY-NOTICES',
) )
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env python3
"""Print the CHANGELOG.md section of a version (the release notes body).
Usage: python tools/changelog_section.py X.Y.Z
Exits non-zero when the version has no section - the workflow then falls back
to generated notes.
"""
import re
import sys
from pathlib import Path
CHANGELOG = Path(__file__).resolve().parent.parent / 'CHANGELOG.md'
def section(version):
text = CHANGELOG.read_text(encoding='utf-8')
pat = re.compile(r'^## \[%s\][^\n]*\n(.*?)(?=^## |\Z)'
% re.escape(version), re.M | re.S)
m = pat.search(text)
if not m:
return None
body = m.group(1).strip()
return body + '\n' if body else None
def main(argv):
if len(argv) != 2:
sys.exit('usage: changelog_section.py X.Y.Z')
body = section(argv[1])
if body is None:
sys.exit('error: no CHANGELOG.md section for %s' % argv[1])
sys.stdout.write(body)
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env python3
"""Update a packaged SIMple install in place.
The release package is a snapshot of the application, and the slow part of a
reinstall is pysim and its prerequisites (built into ``.venv`` by setup -
pyscard compiles from source on Linux). This script downloads the latest
release package, replaces the application files in place and re-installs the
app into the **existing** ``.venv``: pysim is kept, so an update takes seconds.
Usage:
python tools/update.py [--check] [--version X.Y.Z] [--zip FILE] [--no-pip] [--force]
"""
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
import urllib.request
import zipfile
from pathlib import Path
APP = Path(__file__).resolve().parent.parent
REPO_API = 'https://api.github.com/repos/anttro/simple/releases/latest'
ASSET_URL = 'https://github.com/anttro/simple/releases/download/v%s/simple-%s.zip'
UA = {'User-Agent': 'simple-update'}
# Never overwritten even if a package carried them (it never does).
SKIP_TOP = {'.venv', 'dist', '__pycache__', '.git'}
def current_version():
"""The installed app's VERSION - the same source the build reads."""
text = (APP / 'pysim_simple_server' / 'server.py').read_text(encoding='utf-8')
m = re.search(r"^VERSION = '([^']+)'$", text, re.M)
if not m:
sys.exit('error: cannot read VERSION from pysim_simple_server/server.py')
return m.group(1)
def latest_version():
req = urllib.request.Request(REPO_API, headers=UA)
with urllib.request.urlopen(req, timeout=20) as r:
tag = (json.load(r).get('tag_name') or '').strip()
if not tag.startswith('v') or len(tag) < 2:
sys.exit('error: unexpected release tag %r' % tag)
return tag[1:]
def download(version, dest):
url = ASSET_URL % (version, version)
req = urllib.request.Request(url, headers=UA)
with urllib.request.urlopen(req, timeout=120) as r, open(dest, 'wb') as f:
shutil.copyfileobj(r, f)
def safe_extract(zf, dest):
"""Extract, refusing paths outside dest and restoring the permission bits
(zipfile drops them, which would strip +x from setup.sh/start.sh/update.sh)."""
dest = Path(dest).resolve()
for info in zf.infolist():
target = (dest / info.filename).resolve()
if target != dest and dest not in target.parents:
sys.exit('error: unsafe path in the package: %s' % info.filename)
zf.extractall(dest)
for info in zf.infolist():
mode = (info.external_attr >> 16) & 0o777
if mode and not info.is_dir():
try:
os.chmod(dest / info.filename, mode)
except OSError:
pass
def verify(root, version):
"""The package must be the version it claims (the tag is the version)."""
src = root / 'pysim_simple_server' / 'server.py'
if not src.is_file():
sys.exit('error: the package lacks pysim_simple_server/server.py')
m = re.search(r"^VERSION = '([^']+)'$", src.read_text(encoding='utf-8'), re.M)
if not m or m.group(1) != version:
sys.exit('error: the package version (%s) does not match %s'
% (m.group(1) if m else '?', version))
def replace_tree(new_root):
"""Copy the new package over the app directory; the venv and the run log
(neither is in the package) stay untouched."""
count = 0
for src in sorted(new_root.rglob('*')):
if not src.is_file():
continue
rel = src.relative_to(new_root)
if rel.parts[0] in SKIP_TOP or '__pycache__' in rel.parts or src.suffix == '.pyc':
continue
dst = APP / rel
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
count += 1
return count
def venv_python():
for rel in ('bin/python', 'Scripts/python.exe'):
p = APP / '.venv' / rel
if p.exists():
return p
return None
def reinstall():
py = venv_python()
if py is None:
print('note: no .venv in %s - run setup.sh / setup.bat (or install manually)' % APP)
return False
# --no-build-isolation: setuptools is in the venv (setup installed it);
# this keeps the update working offline.
r = subprocess.run([str(py), '-m', 'pip', 'install', '--no-build-isolation',
'-e', str(APP)], capture_output=True, text=True)
if r.returncode != 0:
print('warning: the files were updated, but "pip install -e" failed:')
print((r.stderr or r.stdout).strip()[-800:])
return False
return True
def main():
ap = argparse.ArgumentParser(description='Update a packaged SIMple install in place.')
ap.add_argument('--check', action='store_true',
help='print the current and latest versions and exit')
ap.add_argument('--version', metavar='X.Y.Z',
help='update to this release (default: the latest)')
ap.add_argument('--zip', metavar='FILE',
help='update from a local package file (offline)')
ap.add_argument('--no-pip', action='store_true',
help='replace the files only (do not re-install into .venv)')
ap.add_argument('--force', action='store_true', help='re-apply the same version')
args = ap.parse_args()
cur = current_version()
version = args.version
if not version and not args.zip:
try:
version = latest_version()
except Exception as e:
sys.exit('error: cannot reach GitHub (%s); pass --version X.Y.Z' % e)
if args.check:
print('current: %s' % cur)
print('latest: %s' % (version or '?'))
return 0
if not version:
m = re.search(r'simple-([0-9][^\\/]*)\.zip$', str(args.zip))
if not m:
sys.exit('error: cannot tell the package version; pass --version')
version = m.group(1)
if version == cur and not args.force:
print('already up to date (%s)' % cur)
return 0
with tempfile.TemporaryDirectory(prefix='simple-update-') as tmp:
tmp = Path(tmp)
if args.zip:
zpath = Path(args.zip)
if not zpath.is_file():
sys.exit('error: %s not found' % zpath)
else:
zpath = tmp / 'package.zip'
print('downloading %s ...' % (ASSET_URL % (version, version)))
try:
download(version, zpath)
except Exception as e:
sys.exit('error: download failed (%s)' % e)
try:
with zipfile.ZipFile(zpath) as zf:
safe_extract(zf, tmp)
except zipfile.BadZipFile as e:
sys.exit('error: %s is not a valid package: %s' % (zpath, e))
root = tmp / ('simple-%s' % version)
if not root.is_dir():
found = [p for p in tmp.iterdir()
if p.is_dir() and (p / 'pysim_simple_server').is_dir()]
if len(found) != 1:
sys.exit('error: the package layout is unexpected')
root = found[0]
verify(root, version)
count = replace_tree(root)
print('updated %s -> %s (%d files)' % (cur, version, count))
if args.no_pip:
print('note: --no-pip - .venv was left untouched')
elif reinstall():
print('app re-installed into .venv (pysim kept)')
print('restart the server: ./start.sh (or start.bat)')
return 0
if __name__ == '__main__':
sys.exit(main())
Executable
+9
View File
@@ -0,0 +1,9 @@
@echo off
REM Update this packaged SIMple install in place (see tools\update.py --help).
REM The .venv is reused - pysim and its prerequisites are not re-installed.
if exist "%~dp0.venv\Scripts\python.exe" (
"%~dp0.venv\Scripts\python.exe" "%~dp0tools\update.py" %*
) else (
python "%~dp0tools\update.py" %*
)
pause
Executable
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Update this packaged SIMple install in place (see tools/update.py --help).
# The .venv is reused - pysim and its prerequisites are not re-installed.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
if [ -x "$SCRIPT_DIR/.venv/bin/python" ]; then
PY="$SCRIPT_DIR/.venv/bin/python"
elif command -v python3 >/dev/null 2>&1; then
PY=python3
else
PY=python
fi
exec "$PY" "$SCRIPT_DIR/tools/update.py" "$@"