Files view: change details with before/after images (v2.4.0)

- the timeline's Δ glyph became a **Details** button, and it now appears on
  Update rows too (not just card-side changes); EN "Changed (card)" ->
  "Changed on card", RU "Изменено (картой)" -> "Изменено картой"
- the details modal shows the affected unit — the whole file, or the single
  record that changed — **before and after** the change as hex views (16 bytes
  per row, ASCII column, never-observed bytes as `--`), with the bytes that
  changed and the bytes the operation *first observed* highlighted in two
  different colours, plus the changed/new byte counts and the APDU reference;
  the old before/after byte table is gone
- `GET /api/sessions/<id>/files/image?key=…&change=…` returns those two views
  (segments with offsets) plus the change's metadata; the images are
  reconstructed from the current image by undoing the later changes
  (`FileTracker.change_images`), so nothing is duplicated per event — every
  value change is an event, hence the values are exact (a byte only first
  observed later can appear as known in both views)
- tracker: change events carry `observed_ranges` (bytes that were unknown
  before the operation) next to `ranges`; an update that only observes new
  bytes now records an event (so it can be inspected); record-file ranges are
  in-record byte offsets (the record number stays in its own field), which the
  previous "record number as position" could not express for a hex highlight
- tests: change-images reconstruction (before/after per event, later changes
  undone, records isolated, observation-only events, unknown gaps), the image
  endpoint (200/404/400), and Node tests for the hex-image helper (highlight
  classes, unknown gaps, ASCII, row limit) and the modal rendering
- v2.4.0; SW cache simtrace-analyser-v65
This commit is contained in:
catarrh committed 2026-10-17 21:19:14 +03:00
1 parent 87b6811adf
commit cad3999307
12 files changed
+617 -120

No files matched your search

+14 -3
View File
@@ -191,9 +191,20 @@ in the stored wire shape and are skipped with a note in the import report.
The Trace tab's side panel has a view selector (*Map* / *Stats* / **Files**; The Trace tab's side panel has a view selector (*Map* / *Stats* / **Files**;
a drop-down, since three tab captions no longer fit). The Files view is a a drop-down, since three tab captions no longer fit). The Files view is a
trace-wide timeline of file operations — every **Read**, **Update** and trace-wide timeline of file operations — every **Read**, **Update** and
**Changed (card)** row shows the same file name as the Map view, and clicking **Changed on card** row shows the same file name as the Map view, and clicking
a row jumps to the APDU in the trace. A *Changed (card)* row carries a Δ a row jumps to the APDU in the trace. Rows that carry a recorded change (a
button that opens the changed ranges with the previous and the new bytes. card-side change, or an update that wrote or first observed bytes) also offer
**Details**: a modal showing the affected unit — the whole file, or the single
record that changed — **before and after** the change, as hex views (16 bytes
per row with an ASCII column, never-observed bytes drawn as `--`) with the
bytes that changed highlighted in one colour and the bytes the operation first
observed in another, plus the changed/new byte counts and the APDU reference.
The images come from `GET /api/sessions/<id>/files/image?key=…&change=…` and
are reconstructed from the current image by undoing the later changes
(`filetrack.FileTracker.change_images`) — nothing is duplicated per event.
Every value change is recorded, so the values are exact; a byte that was only
*first observed* after the selected change can appear as known in both views.
The data comes from `simtrace_analyser/filetrack.py` The data comes from `simtrace_analyser/filetrack.py`
(`GET /api/sessions/<id>/files`), a single-pass tracker over the decoded (`GET /api/sessions/<id>/files`), a single-pass tracker over the decoded
+15 -3
View File
@@ -196,9 +196,21 @@ Application Layer* — два отдельных поля выбора, имен
В боковой панели вкладки «Трассировка» появился выбор вида (*Map* / *Stats* / В боковой панели вкладки «Трассировка» появился выбор вида (*Map* / *Stats* /
**Files** — выпадающий список, три подписи-вкладки уже не помещались). Вид **Files** — выпадающий список, три подписи-вкладки уже не помещались). Вид
Files — это таймлайн файловых операций по всей трассе: строки **Read**, Files — это таймлайн файловых операций по всей трассе: строки **Read**,
**Update** и **Changed (card)** показывают те же имена файлов, что и вид Map, **Update** и **Changed on card** показывают те же имена файлов, что и вид Map,
а щелчок по строке переходит к APDU в трассе. У строки *Changed (card)* есть а щелчок по строке переходит к APDU в трассе. У строк с записанным изменением
кнопка Δ, открывающая изменённые диапазоны с прежними и новыми байтами. (изменение со стороны карты либо запись, изменившая или впервые показавшая
байты) есть кнопка **Details**: окно показывает затронутую единицу — файл
целиком или изменившуюся запись — **до и после** изменения в виде hex-вида
(16 байт в строке со столбцом ASCII, никогда не наблюдавшиеся байты — `--`),
причём изменённые байты подсвечены одним цветом, а впервые наблюдаемые —
другим; ниже — счётчики изменённых/новых байт и ссылка на APDU.
Изображения отдаёт `GET /api/sessions/<id>/files/image?key=…&change=…`; они
восстанавливаются из текущего образа отменой более поздних изменений
(`filetrack.FileTracker.change_images`) — поэтому ничего не дублируется по
событиям. Каждое изменение значения записывается, так что значения точны;
байт, который был лишь *впервые показан* после выбранного изменения, может
отображаться известным в обоих видах.
Данные даёт `simtrace_analyser/filetrack.py` (`GET /api/sessions/<id>/files`) — Данные даёт `simtrace_analyser/filetrack.py` (`GET /api/sessions/<id>/files`) —
однопроходный анализатор сессии, который ведёт по ключу (DF, FID) и **через однопроходный анализатор сессии, который ведёт по ключу (DF, FID) и **через
+138 -39
View File
@@ -188,13 +188,25 @@ body { font: 14px/1.5 system-ui, sans-serif; background: var(--bg); color: var(-
.file-op-read { color: var(--text-muted); } .file-op-read { color: var(--text-muted); }
.file-op-update { color: var(--accent); } .file-op-update { color: var(--accent); }
.file-op-card { color: var(--warn); } .file-op-card { color: var(--warn); }
.file-op .file-op-diff { margin-left: auto; background: none; border: none; color: var(--warn); cursor: pointer; font-size: 12px; padding: 0 2px; flex: 0 0 auto; } .file-op .file-op-diff { margin-left: auto; background: none; border: none; color: var(--accent); cursor: pointer; font-size: 12px; padding: 0 2px; flex: 0 0 auto; text-decoration: underline; }
.file-op .file-op-diff:hover { color: var(--accent); } .file-op .file-op-diff:hover { color: var(--warn); }
.file-diff-modal { max-width: 640px; } .file-diff-modal { max-width: 760px; }
.file-diff-body { max-height: 50vh; overflow-y: auto; margin-bottom: 16px; } .file-diff-body { max-height: 60vh; overflow-y: auto; margin-bottom: 16px; }
.file-diff-body table { font-size: 12px; } .hex-legend { display: flex; align-items: center; gap: 6px; font-size: 12px; color: var(--text-muted); margin-bottom: 6px; flex-wrap: wrap; }
.file-diff-body td { padding: 2px 8px 2px 0; vertical-align: top; } .hex-summary { font-size: 12px; color: var(--text-muted); margin-bottom: 10px; }
.file-diff-body .file-diff-hex { font-family: ui-monospace, monospace; word-break: break-all; } .hex-pair { display: flex; gap: 16px; flex-wrap: wrap; }
.hex-pair > div { flex: 1 1 320px; min-width: 0; }
.hex-pair h4 { font-size: 12px; text-transform: uppercase; letter-spacing: 1px; color: var(--accent); margin-bottom: 4px; }
.hex-view { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; line-height: 1.45; white-space: pre; overflow-x: auto; }
.hex-row { display: flex; align-items: baseline; }
.hex-addr { color: var(--text-muted); margin-right: 8px; }
.hex-cell { padding: 0 1px; border-radius: 2px; }
.hex-cell.hex-changed { background: var(--warn); color: #1a1a1a; font-weight: 600; }
.hex-cell.hex-new { background: var(--accent); color: #fff; font-weight: 600; }
.hex-cell.hex-unknown { color: var(--text-muted); }
.hex-ascii { color: var(--text-muted); margin-left: 8px; }
.hex-note { font-size: 12px; color: var(--text-muted); margin-top: 6px; }
.hex-error { font-size: 12px; color: var(--error); }
.map-item { padding: 3px 8px; cursor: pointer; border-radius: 4px; font-size: 13px; color: var(--text); word-break: break-word; } .map-item { padding: 3px 8px; cursor: pointer; border-radius: 4px; font-size: 13px; color: var(--text); word-break: break-word; }
.map-item:hover { background: var(--bg-input); } .map-item:hover { background: var(--bg-input); }
.map-files-toggle { color: var(--accent); cursor: pointer; } .map-files-toggle { color: var(--accent); cursor: pointer; }
@@ -438,7 +450,7 @@ body { font: 14px/1.5 system-ui, sans-serif; background: var(--bg); color: var(-
<!-- Card-side change detail (Files view) --> <!-- Card-side change detail (Files view) -->
<div class="modal-overlay" id="file-diff-modal"> <div class="modal-overlay" id="file-diff-modal">
<div class="modal file-diff-modal"> <div class="modal file-diff-modal">
<h3 data-l10n="Card-side change">Card-side change</h3> <h3 data-l10n="Change details">Change details</h3>
<p id="file-diff-sub"></p> <p id="file-diff-sub"></p>
<div class="file-diff-body" id="file-diff-body"></div> <div class="file-diff-body" id="file-diff-body"></div>
<div class="modal-buttons"> <div class="modal-buttons">
@@ -499,6 +511,7 @@ const STATE = {
sideView: 'map', sideView: 'map',
filesData: null, filesData: null,
filesSessionId: null, filesSessionId: null,
filesImages: {},
pendingDeleteId: null, pendingDeleteId: null,
serverAvailable: false, serverAvailable: false,
serverName: '', serverName: '',
@@ -1193,6 +1206,10 @@ const API = {
stopCapture() { return this._fetch('POST', '/api/capture/stop'); }, stopCapture() { return this._fetch('POST', '/api/capture/stop'); },
getLatest(after) { return this._fetch('GET', `/api/capture/latest?after=${after}`); }, getLatest(after) { return this._fetch('GET', `/api/capture/latest?after=${after}`); },
getSessionFiles(id) { return this._fetch('GET', `/api/sessions/${id}/files`); }, getSessionFiles(id) { return this._fetch('GET', `/api/sessions/${id}/files`); },
getSessionFileImage(id, key, change) {
return this._fetch('GET', `/api/sessions/${id}/files/image?key=` +
encodeURIComponent(key) + `&change=${change}`);
},
searchMessages(sessionId, query) { return this._fetch('GET', `/api/apdu/search?session_id=${sessionId}&q=${encodeURIComponent(query)}`); }, searchMessages(sessionId, query) { return this._fetch('GET', `/api/apdu/search?session_id=${sessionId}&q=${encodeURIComponent(query)}`); },
filterMessages(sessionId, type) { return this._fetch('GET', `/api/apdu/filter?session_id=${sessionId}&type=${type}`); }, filterMessages(sessionId, type) { return this._fetch('GET', `/api/apdu/filter?session_id=${sessionId}&type=${type}`); },
async _importPost(params, body, contentType) { async _importPost(params, body, contentType) {
@@ -2087,7 +2104,7 @@ function fileOpClass(kind) {
function fileOpVerb(kind) { function fileOpVerb(kind) {
if (kind === 'update') return t('Update'); if (kind === 'update') return t('Update');
if (kind === 'card_change') return t('Changed (card)'); if (kind === 'card_change') return t('Changed on card');
return t('Read'); return t('Read');
} }
@@ -2096,21 +2113,59 @@ function fileOpLabel(op) {
return `${fileOpVerb(op.kind)} ${name}`; return `${fileOpVerb(op.kind)} ${name}`;
} }
// Split a change's concatenated before/after hex into per-range rows. // Hex view of an image: 16 bytes per row across the observed segments, with
function fileDiffRows(change) { // gaps drawn as unknown cells. *ranges* (bytes that changed) and *observed*
const rows = []; // (bytes first observed by this change) mark the cells.
let at = 0; function hexImageRows(segments, ranges, observed, limit) {
for (const [pos, len] of (change.ranges || [])) { const maxBytes = limit || 2048;
const take = len * 2; const bytes = new Map();
rows.push({ let end = 0;
pos: pos, for (const seg of segments || []) {
len: len, const hex = (seg.hex || '').toLowerCase();
before: (change.before_hex || '').slice(at, at + take), for (let i = 0; i < hex.length / 2; i++) {
after: (change.after_hex || '').slice(at, at + take), bytes.set(seg.offset + i, hex.substr(i * 2, 2));
}); }
at += take; end = Math.max(end, seg.offset + Math.floor(hex.length / 2));
} }
return rows; const covered = (list, pos) => (list || []).some(([p, n]) => pos >= p && pos < p + n);
const rows = [];
let truncated = false;
for (let off = 0; off < end; off += 16) {
if (off >= maxBytes) {
truncated = true;
break;
}
const cells = [];
let ascii = '';
for (let i = 0; i < 16; i++) {
const pos = off + i;
const hex = bytes.get(pos);
if (hex === undefined) {
cells.push({hex: '--', cls: 'hex-unknown'});
ascii += ' ';
continue;
}
const cls = covered(ranges, pos) ? 'hex-changed'
: covered(observed, pos) ? 'hex-new' : '';
const value = parseInt(hex, 16);
cells.push({hex: hex, cls: cls});
ascii += (value >= 0x20 && value < 0x7f) ? String.fromCharCode(value) : '.';
}
rows.push({offset: off, cells: cells, ascii: ascii});
}
return {rows: rows, truncated: truncated, end: end};
}
function hexImageHtml(image, key) {
const segments = image[key] || [];
const view = hexImageRows(segments, image.ranges, image.observed_ranges);
const body = view.rows.map(r =>
`<div class="hex-row"><span class="hex-addr">${r.offset.toString(16).toUpperCase().padStart(4, '0')}</span>` +
r.cells.map(c => `<span class="hex-cell ${c.cls}">${c.hex}</span>`).join('') +
`<span class="hex-ascii">${escapeHtml(r.ascii)}</span></div>`
).join('') || `<div class="hex-row hex-unknown">${t('never observed')}</div>`;
return `<div class="hex-view">${body}</div>` +
(view.truncated ? `<div class="hex-note">${t('Showing the first {n} bytes of the image').replace('{n}', 2048)}</div>` : '');
} }
async function renderFileView() { async function renderFileView() {
@@ -2149,8 +2204,8 @@ async function renderFileView() {
return; return;
} }
view.innerHTML = timeline.map(op => { view.innerHTML = timeline.map(op => {
const diff = (op.kind === 'card_change' && op.change_ref) const diff = op.change_ref
? `<button class="file-op-diff" data-diff="${escapeHtml(op.change_ref.file)}|${op.change_ref.idx}" title="${t('Show the change')}">Δ</button>` ? `<button class="file-op-diff" data-diff="${escapeHtml(op.change_ref.file)}|${op.change_ref.idx}" title="${t('Show the change')}">${t('Details')}</button>`
: ''; : '';
return `<div class="file-op ${fileOpClass(op.kind)}" data-idx="${op.index}"> return `<div class="file-op ${fileOpClass(op.kind)}" data-idx="${op.index}">
<span>${escapeHtml(fileOpLabel(op))}</span>${diff} <span>${escapeHtml(fileOpLabel(op))}</span>${diff}
@@ -2158,20 +2213,56 @@ async function renderFileView() {
}).join(''); }).join('');
} }
function showFileDiff(fileKey, idx) { async function showFileDiff(fileKey, idx) {
const data = STATE.filesData; const data = STATE.filesData;
const entry = data && data.files ? data.files[fileKey] : null; const entry = data && data.files ? data.files[fileKey] : null;
const change = entry && entry.changes ? entry.changes[idx] : null; if (!entry) return;
if (!change) return; el('file-diff-sub').textContent = entry.name;
const rows = fileDiffRows(change); el('file-diff-body').innerHTML = `<p class="hex-note">${t('Loading…')}</p>`;
el('file-diff-sub').textContent = `${entry.name} — ${change.op}`;
el('file-diff-body').innerHTML = rows.map(r => `<table>
<tr><td>${change.record != null ? t('Record') : t('Offset')}:</td><td>${escapeHtml(change.record != null ? String(r.pos) : '0x' + r.pos.toString(16).toUpperCase())} (${r.len} B)</td></tr>
<tr><td>${t('Before')}:</td><td class="file-diff-hex">${escapeHtml(r.before.toUpperCase())}</td></tr>
<tr><td>${t('After')}:</td><td class="file-diff-hex">${escapeHtml(r.after.toUpperCase())}</td></tr>
<tr><td colspan="2" style="height:6px"></td></tr>
</table>`).join('');
el('file-diff-modal').classList.add('show'); el('file-diff-modal').classList.add('show');
const cacheKey = `${STATE.sessionViewId}|${fileKey}|${idx}`;
let image = STATE.filesImages[cacheKey];
try {
if (!image) {
image = await API.getSessionFileImage(STATE.sessionViewId, fileKey, idx);
if (!image || !image.before) {
throw new Error((image && image.error) || t('Failed to load files'));
}
STATE.filesImages[cacheKey] = image;
}
} catch (e) {
el('file-diff-body').innerHTML =
`<p class="hex-error">${t('Failed to load files')}: ${escapeHtml((e && e.message) || '')}</p>`;
return;
}
renderFileDiff(image);
}
function renderFileDiff(image) {
const changed = (image.ranges || []).reduce((n, r) => n + r[1], 0);
const fresh = (image.observed_ranges || []).reduce((n, r) => n + r[1], 0);
const meta = [];
if (image.op) meta.push(image.op);
if (image.record !== null && image.record !== undefined) {
meta.push(`${t('Record')} ${image.record}` +
(image.n_records ? ` / ${image.n_records}` : ''));
}
if (image.index !== null && image.index !== undefined) meta.push(`#${image.index}`);
if (image.elapsed !== null && image.elapsed !== undefined) meta.push(fmt_elapsed(image.elapsed));
el('file-diff-sub').textContent = `${image.name || ''} — ${meta.join(' · ')}`;
el('file-diff-body').innerHTML =
`<div class="hex-legend">
<span class="hex-cell hex-changed">aa</span> ${t('changed')}
<span class="hex-cell hex-new">aa</span> ${t('newly observed')}
<span class="hex-cell hex-unknown">--</span> ${t('never observed')}
</div>
<div class="hex-summary">${t('{c} bytes changed, {n} bytes newly observed')
.replace('{c}', changed).replace('{n}', fresh)}</div>
<div class="hex-pair">
<div><h4>${t('Before')}</h4>${hexImageHtml(image, 'before')}</div>
<div><h4>${t('After')}</h4>${hexImageHtml(image, 'after')}</div>
</div>` +
(image.image_truncated ? `<p class="hex-note">${t('The image is frozen (size cap reached) — later changes are not tracked.')}</p>` : '');
} }
// Load the pages up to *idx* (the message list is fetched in pages) and focus // Load the pages up to *idx* (the message list is fetched in pages) and focus
@@ -2810,6 +2901,7 @@ async function loadSession(id) {
STATE.sessionViewId = id; STATE.sessionViewId = id;
STATE.filesData = null; STATE.filesData = null;
STATE.filesSessionId = null; STATE.filesSessionId = null;
STATE.filesImages = {};
STATE.sessionStarted = data.session.started; STATE.sessionStarted = data.session.started;
STATE.captureActive = false; STATE.captureActive = false;
STATE.currentSessionName = data.session.name || t('Session #') + id; STATE.currentSessionName = data.session.name || t('Session #') + id;
@@ -3072,7 +3164,7 @@ function onFilterChange() {
} }
// ──────────────────── Versioning ──────────────────── // ──────────────────── Versioning ────────────────────
const PWA_VERSION = '2.3.1'; const PWA_VERSION = '2.4.0';
const MIN_SERVER_VERSION = '2.3.0'; const MIN_SERVER_VERSION = '2.3.0';
function parseVersion(v) { function parseVersion(v) {
@@ -3118,8 +3210,15 @@ const LANG_RU = {
'Files': 'Файлы', 'Files': 'Файлы',
'Read': 'Чтение', 'Read': 'Чтение',
'Update': 'Запись', 'Update': 'Запись',
'Changed (card)': 'Изменено (картой)', 'Changed on card': 'Изменено картой',
'Card-side change': 'Изменение на карте', 'Change details': 'Подробности изменения',
'Details': 'Подробнее',
'changed': 'изменено',
'newly observed': 'впервые наблюдается',
'never observed': 'не наблюдалось',
'{c} bytes changed, {n} bytes newly observed': 'Изменено байт: {c}; впервые наблюдается байт: {n}',
'Showing the first {n} bytes of the image': 'Показаны первые {n} байт образа',
'The image is frozen (size cap reached) — later changes are not tracked.': 'Образ заморожен (достигнут предел размера) — последующие изменения не отслеживаются.',
'Close': 'Закрыть', 'Close': 'Закрыть',
'Offset': 'Смещение', 'Offset': 'Смещение',
'Record': 'Запись №', 'Record': 'Запись №',
+1 -1
View File
@@ -1,4 +1,4 @@
const CACHE = 'simtrace-analyser-v64'; const CACHE = 'simtrace-analyser-v65';
const PRECACHE = [ const PRECACHE = [
'index.html', 'index.html',
+83 -16
View File
@@ -282,7 +282,8 @@ eval(
extractFunc(src, 'fileOpClass') + '\n' + extractFunc(src, 'fileOpClass') + '\n' +
extractFunc(src, 'fileOpVerb') + '\n' + extractFunc(src, 'fileOpVerb') + '\n' +
extractFunc(src, 'fileOpLabel') + '\n' + extractFunc(src, 'fileOpLabel') + '\n' +
extractFunc(src, 'fileDiffRows') extractFunc(src, 'escapeHtml') + '\n' +
extractFunc(src, 'hexImageRows')
); );
test('Files view: operation kinds map to classes and verbs', () => { test('Files view: operation kinds map to classes and verbs', () => {
@@ -291,24 +292,36 @@ test('Files view: operation kinds map to classes and verbs', () => {
assert.equal(fileOpClass('card_change'), 'file-op-card'); assert.equal(fileOpClass('card_change'), 'file-op-card');
assert.equal(fileOpVerb('read'), 'Read'); assert.equal(fileOpVerb('read'), 'Read');
assert.equal(fileOpVerb('update'), 'Update'); assert.equal(fileOpVerb('update'), 'Update');
assert.equal(fileOpVerb('card_change'), 'Changed (card)'); assert.equal(fileOpVerb('card_change'), 'Changed on card');
assert.equal(fileOpLabel({kind: 'read', file: {name: 'EF_ICCID'}}), 'Read EF_ICCID'); assert.equal(fileOpLabel({kind: 'read', file: {name: 'EF_ICCID'}}), 'Read EF_ICCID');
assert.equal(fileOpLabel({kind: 'card_change', file: {name: 'EF_LOCI'}}), assert.equal(fileOpLabel({kind: 'card_change', file: {name: 'EF_LOCI'}}),
'Changed (card) EF_LOCI'); 'Changed on card EF_LOCI');
assert.equal(fileOpLabel({kind: 'read', file: null}), 'Read ?'); assert.equal(fileOpLabel({kind: 'read', file: null}), 'Read ?');
}); });
test('Files view: the diff rows slice the concatenated hex', () => { test('Files view: the hex image rows mark changed, new and unknown bytes', () => {
const change = { // 'ABCD' then a gap, then 'ef' newly observed.
ranges: [[8, 2], [16, 1]], const view = hexImageRows(
before_hex: '95f9aa', [{offset: 0, hex: '41424344'}, {offset: 8, hex: '6566'}],
after_hex: '000001', [[2, 1]], // changed
}; [[8, 2]]); // newly observed
assert.deepEqual(fileDiffRows(change), [ assert.equal(view.truncated, false);
{pos: 8, len: 2, before: '95f9', after: '0000'}, assert.equal(view.end, 10);
{pos: 16, len: 1, before: 'aa', after: '01'}, assert.equal(view.rows.length, 1);
]); const cells = view.rows[0].cells;
assert.deepEqual(fileDiffRows({ranges: [], before_hex: '', after_hex: ''}), []); assert.deepEqual(cells[0], {hex: '41', cls: ''});
assert.deepEqual(cells[2], {hex: '43', cls: 'hex-changed'});
assert.deepEqual(cells[4], {hex: '--', cls: 'hex-unknown'}); // the gap
assert.deepEqual(cells[8], {hex: '65', cls: 'hex-new'});
// ASCII column: printable bytes stay, gaps and the unobserved tail are blank.
assert.equal(view.rows[0].ascii, 'ABCD ef ');
});
test('Files view: the hex image respects the row limit', () => {
const view = hexImageRows([{offset: 0, hex: 'aa'.repeat(64)}], [], [], 32);
assert.equal(view.truncated, true);
assert.equal(view.rows.length, 2);
assert.equal(view.end, 64);
}); });
test('Files view: side-panel dropdown and the timeline are wired', () => { test('Files view: side-panel dropdown and the timeline are wired', () => {
@@ -321,6 +334,10 @@ test('Files view: side-panel dropdown and the timeline are wired', () => {
assert.match(src, /\$on\('side-view-select', 'change'/); assert.match(src, /\$on\('side-view-select', 'change'/);
assert.match(src, /\$on\('file-view', 'click'/); assert.match(src, /\$on\('file-view', 'click'/);
assert.match(src, /API\.getSessionFiles\(sid\)/); assert.match(src, /API\.getSessionFiles\(sid\)/);
assert.match(src, />\$\{t\('Details'\)\}<\/button>/);
assert.match(src, /API\.getSessionFileImage\(STATE\.sessionViewId, fileKey, idx\)/);
assert.match(src, /getSessionFileImage\(id, key, change\)/);
assert.match(src, /files\/image\?key=/);
assert.match(src, /if \(row\) jumpToMessage\(Number\(row\.dataset\.idx\)\);/); assert.match(src, /if \(row\) jumpToMessage\(Number\(row\.dataset\.idx\)\);/);
assert.match(src, /getSessionFiles\(id\)\s*\{ return this\._fetch\('GET', `\/api\/sessions\/\$\{id\}\/files`\); \}/); assert.match(src, /getSessionFiles\(id\)\s*\{ return this\._fetch\('GET', `\/api\/sessions\/\$\{id\}\/files`\); \}/);
}); });
@@ -331,8 +348,12 @@ test('Files view: strings carry Russian translations', () => {
for (const m of lang.matchAll(/'((?:[^'\\]|\\.)*)'\s*:/g)) { for (const m of lang.matchAll(/'((?:[^'\\]|\\.)*)'\s*:/g)) {
keys.add(m[1].replace(/\\(['\\])/g, '$1')); keys.add(m[1].replace(/\\(['\\])/g, '$1'));
} }
for (const s of ['Files', 'Read', 'Update', 'Changed (card)', 'Card-side change', for (const s of ['Files', 'Read', 'Update', 'Changed on card', 'Change details',
'Close', 'Offset', 'Record', 'Before', 'After', 'Show the change', 'Details', 'Close', 'Offset', 'Record', 'Before', 'After', 'Show the change',
'changed', 'newly observed', 'never observed',
'{c} bytes changed, {n} bytes newly observed',
'Showing the first {n} bytes of the image',
'The image is frozen (size cap reached) — later changes are not tracked.',
'No file operations recorded', 'Failed to load files', 'Loading…']) { 'No file operations recorded', 'Failed to load files', 'Loading…']) {
assert.ok(keys.has(s), 'no LANG_RU entry for: ' + s); assert.ok(keys.has(s), 'no LANG_RU entry for: ' + s);
} }
@@ -431,3 +452,49 @@ test('Files view: a response for a session left meanwhile is ignored', async ()
assert.equal(STATE.filesData, null); assert.equal(STATE.filesData, null);
assert.equal(STATE.filesSessionId, null); assert.equal(STATE.filesSessionId, null);
}); });
// ── Change-details modal rendering (stubbed DOM) ───────────────────────
eval(
'function el(id) { return dlgEl(id); }\n' +
extractFunc(src, 'fmt_elapsed') + '\n' +
extractFunc(src, 'escapeHtml') + '\n' +
extractFunc(src, 'hexImageRows') + '\n' +
extractFunc(src, 'hexImageHtml') + '\n' +
extractFunc(src, 'renderFileDiff')
);
test('Change details modal: header, summary, legend and both views', () => {
renderFileDiff({
name: 'EF_ICCID',
op: 'READ BINARY',
record: null,
n_records: null,
index: 12,
elapsed: 3.25,
ranges: [[8, 2]],
observed_ranges: [[0, 4]],
before: [{offset: 0, hex: '41424344'}],
after: [{offset: 0, hex: '41424344'}, {offset: 8, hex: '9900'}],
image_truncated: false,
});
const sub = el('file-diff-sub').textContent;
assert.match(sub, /EF_ICCID — READ BINARY/);
assert.match(sub, /#12/);
const body = el('file-diff-body').innerHTML;
assert.match(body, /2 bytes changed, 4 bytes newly observed/);
assert.match(body, /hex-cell hex-changed/); // legend + the changed byte
assert.match(body, /hex-cell hex-new/);
assert.match(body, /hex-cell hex-unknown/);
assert.match(body, /Before/);
assert.match(body, /After/);
assert.equal((body.match(/class="hex-view"/g) || []).length, 2);
// The record header variant and the frozen-image note.
renderFileDiff({
name: 'EF_IMSI', op: 'READ RECORD', record: 2, n_records: 5, index: 40,
elapsed: 9.5, ranges: [[0, 1]], observed_ranges: [],
before: [{offset: 0, hex: 'aa'}], after: [{offset: 0, hex: 'bb'}],
image_truncated: true,
});
assert.match(el('file-diff-sub').textContent, /Record 2 \/ 5/);
assert.match(el('file-diff-body').innerHTML, /image is frozen/);
});
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.backends._legacy:_Backend"
[project] [project]
name = "simtrace-analyser" name = "simtrace-analyser"
version = "2.3.1" version = "2.4.0"
description = "SIMtrace Analyser — APDU trace analyser for GSMTAP/PCAP captures (bundles the simtrace2-pysniff utility)" description = "SIMtrace Analyser — APDU trace analyser for GSMTAP/PCAP captures (bundles the simtrace2-pysniff utility)"
readme = {file = "README.md", content-type = "text/markdown"} readme = {file = "README.md", content-type = "text/markdown"}
license = {text = "GPL-2.0-or-later"} license = {text = "GPL-2.0-or-later"}
+10 -1
View File
@@ -198,13 +198,22 @@ class Database:
Returns the tracker's result dict: the operation timeline, per-file Returns the tracker's result dict: the operation timeline, per-file
counters/images/changes, the SFI relations and the AID labels. counters/images/changes, the SFI relations and the AID labels.
""" """
tracker = self._file_tracker(session_id)
return tracker.result()
def change_images(self, session_id, key, idx):
"""Before/after images of the unit affected by change *idx* of *key*
(the Files view's Details modal), or None for an unknown file/index."""
return self._file_tracker(session_id).change_images(key, idx)
def _file_tracker(self, session_id):
from .filetrack import FileTracker from .filetrack import FileTracker
rows = self._conn.execute( rows = self._conn.execute(
'SELECT id, session_id, elapsed, type, data, flags FROM messages ' 'SELECT id, session_id, elapsed, type, data, flags FROM messages '
'WHERE session_id=? ORDER BY elapsed, id', (session_id,)).fetchall() 'WHERE session_id=? ORDER BY elapsed, id', (session_id,)).fetchall()
tracker = FileTracker() tracker = FileTracker()
self._decode_rows(rows, on_message=tracker.observe) self._decode_rows(rows, on_message=tracker.observe)
return tracker.result() return tracker
def get_messages_raw(self, session_id): def get_messages_raw(self, session_id):
rows = self._conn.execute( rows = self._conn.execute(
+127 -22
View File
@@ -77,6 +77,37 @@ def _key_str(key):
return f'{df}/{fid}' return f'{df}/{fid}'
def _parse_key(key):
df, _sep, fid = (key or '').rpartition('/')
return df, fid
def _apply_hex(buf, ranges, hex_str):
"""Write a concatenated hex string into *buf* at the given ranges."""
at = 0
for pos, length in ranges or []:
chunk = bytes.fromhex(hex_str[at:at + length * 2])
at += length * 2
if len(buf) < pos + length:
buf.extend(b'\x00' * (pos + length - len(buf)))
buf[pos:pos + length] = chunk
def _segments(buf, mask):
"""Observed runs of the image: ``[{'offset': n, 'hex': '…'}, …]``."""
out = []
i = 0
while i < len(mask):
if not mask[i]:
i += 1
continue
start = i
while i < len(mask) and mask[i]:
i += 1
out.append({'offset': start, 'hex': bytes(buf[start:i]).hex()})
return out
class _FileEntry: class _FileEntry:
"""One tracked file: counters, sparse image, change log.""" """One tracked file: counters, sparse image, change log."""
@@ -111,14 +142,18 @@ class _FileEntry:
def apply(self, record, offset, data): def apply(self, record, offset, data):
"""Apply observed bytes. """Apply observed bytes.
Returns ``(diffs, compared_all)``: *diffs* is a list of Returns ``(diffs, observed, compared_all)``:
``(position, before, after)`` for the positions whose known value
differed (position = byte offset for transparent files, record number * *diffs* — ``[(pos, before, after)]`` for positions whose known value
for record files); *compared_all* tells whether every written byte was differed (pos = the byte offset in the file, or *in the record* for
already known (so a zero diff proves a no-op). record files);
* *observed* — ``[(pos, len)]`` runs whose bytes were not known before
this operation (a first observation);
* *compared_all* — every written byte was already known (so an empty
diff proves a no-op).
""" """
if self.image_truncated: if self.image_truncated:
return [], False return [], [], False
if record is None: if record is None:
buf, mask, base = self.buf, self.mask, offset buf, mask, base = self.buf, self.mask, offset
needed = base + len(data) needed = base + len(data)
@@ -130,20 +165,24 @@ class _FileEntry:
if (needed > len(buf) if (needed > len(buf)
and self._tracked_bytes() + (needed - len(buf)) > self.image_cap): and self._tracked_bytes() + (needed - len(buf)) > self.image_cap):
self.image_truncated = True self.image_truncated = True
return [], False return [], [], False
if len(buf) < needed: if len(buf) < needed:
buf.extend(b'\x00' * (needed - len(buf))) buf.extend(b'\x00' * (needed - len(buf)))
mask.extend(b'\x00' * (needed - len(mask))) mask.extend(b'\x00' * (needed - len(mask)))
diffs = [] diffs = []
observed = []
compared_all = True compared_all = True
i = 0 i = 0
while i < len(data): while i < len(data):
pos = base + i pos = base + i
if not mask[pos]: if not mask[pos]:
compared_all = False compared_all = False
buf[pos] = data[i] start = pos
mask[pos] = 1 while i < len(data) and not mask[base + i]:
i += 1 buf[base + i] = data[i]
mask[base + i] = 1
i += 1
observed.append((start, i - (start - base)))
continue continue
if buf[pos] == data[i]: if buf[pos] == data[i]:
i += 1 i += 1
@@ -156,9 +195,8 @@ class _FileEntry:
after.append(data[i]) after.append(data[i])
buf[base + i] = data[i] buf[base + i] = data[i]
i += 1 i += 1
diffs.append((start if record is None else record, diffs.append((start, bytes(before), bytes(after)))
bytes(before), bytes(after))) return diffs, observed, compared_all
return diffs, compared_all
def coverage(self): def coverage(self):
known = sum(self.mask) + sum(sum(m) for m in self.rec_mask.values()) known = sum(self.mask) + sum(sum(m) for m in self.rec_mask.values())
@@ -291,7 +329,8 @@ class FileTracker:
self.timeline.append(item) self.timeline.append(item)
return item return item
def _add_change(self, entry, kind, index, msg, op, diffs, record=None): def _add_change(self, entry, kind, index, msg, op, diffs, observed=(),
record=None):
entry.changes_total += 1 entry.changes_total += 1
if len(entry.changes) >= self._events_cap: if len(entry.changes) >= self._events_cap:
return None return None
@@ -299,6 +338,7 @@ class FileTracker:
'kind': kind, 'op': op, 'index': index, 'msg_id': msg.get('id'), 'kind': kind, 'op': op, 'index': index, 'msg_id': msg.get('id'),
'elapsed': msg.get('elapsed'), 'record': record, 'elapsed': msg.get('elapsed'), 'record': record,
'ranges': [[pos, len(before)] for pos, before, _after in diffs], 'ranges': [[pos, len(before)] for pos, before, _after in diffs],
'observed_ranges': [[pos, length] for pos, length in observed],
'before_hex': b''.join(b for _p, b, _a in diffs).hex(), 'before_hex': b''.join(b for _p, b, _a in diffs).hex(),
'after_hex': b''.join(a for _p, _b, a in diffs).hex(), 'after_hex': b''.join(a for _p, _b, a in diffs).hex(),
} }
@@ -371,12 +411,13 @@ class FileTracker:
entry.sfi = sfi entry.sfi = sfi
entry.counters['read'] += 1 entry.counters['read'] += 1
record = self._record_number(d) record = self._record_number(d)
diffs, _compared = entry.apply(record, self._offset(d, record), data) diffs, observed, _compared = entry.apply(record, self._offset(d, record), data)
if entry.image_truncated: if entry.image_truncated:
return return
if diffs: if diffs:
idx = self._add_change(entry, 'card', index, msg, idx = self._add_change(entry, 'card', index, msg,
d.get('ins_name') or '', diffs, record=record) d.get('ins_name') or '', diffs,
observed=observed, record=record)
self._push('card_change', entry, index, msg, record=record, self._push('card_change', entry, index, msg, record=record,
change_ref={'file': _key_str(entry.key), 'idx': idx} change_ref={'file': _key_str(entry.key), 'idx': idx}
if idx is not None else None) if idx is not None else None)
@@ -396,13 +437,17 @@ class FileTracker:
entry.sfi = sfi entry.sfi = sfi
entry.counters['update'] += 1 entry.counters['update'] += 1
record = self._record_number(d) record = self._record_number(d)
diffs, compared_all = entry.apply(record, self._offset(d, record), data) diffs, observed, compared_all = entry.apply(record, self._offset(d, record), data)
if entry.image_truncated: if entry.image_truncated:
return return
if diffs: change_ref = None
self._add_change(entry, 'device', index, msg, if diffs or observed:
d.get('ins_name') or '', diffs, record=record) idx = self._add_change(entry, 'device', index, msg,
self._push('update', entry, index, msg, record=record, d.get('ins_name') or '', diffs,
observed=observed, record=record)
if idx is not None:
change_ref = {'file': _key_str(entry.key), 'idx': idx}
self._push('update', entry, index, msg, record=record, change_ref=change_ref,
no_change=(not diffs and compared_all)) no_change=(not diffs and compared_all))
# -- helpers --------------------------------------------------------- # -- helpers ---------------------------------------------------------
@@ -430,7 +475,67 @@ class FileTracker:
# -- result ---------------------------------------------------------- # -- result ----------------------------------------------------------
def result(self): def change_images(self, key, idx):
"""Before/after images of the unit affected by change *idx* of *key*.
The unit is the whole file for transparent files, the changed record
for record files. The images are reconstructed from the current image
by undoing the later events' changed bytes (every value change is an
event, so the values are exact; a byte that was only *first observed*
later can appear as known in both views). The requested event's own
``observed_ranges`` are shown as unknown in the *before* view — that is
what "newly observed" means.
"""
entry = self.files.get(_parse_key(key))
if entry is None or not 0 <= idx < len(entry.changes):
return None
event = entry.changes[idx]
record = event.get('record')
if record is None:
buf = bytearray(entry.buf)
mask = bytearray(entry.mask)
else:
buf = bytearray(entry.records.get(record, b''))
mask = bytearray(entry.rec_mask.get(record, b''))
for later in reversed(entry.changes[idx:]): # the event itself included
if later.get('record') != record:
continue
_apply_hex(buf, later['ranges'], later['before_hex'])
# The before view: this event's newly observed bytes were unknown then.
for pos, length in event.get('observed_ranges') or []:
for i in range(pos, min(pos + length, len(mask))):
mask[i] = 0
before = _segments(buf, mask)
# The after view: this event's changed bytes and its observations.
_apply_hex(buf, event['ranges'], event['after_hex'])
for pos, length in event.get('observed_ranges') or []:
for i in range(pos, min(pos + length, len(mask))):
mask[i] = 1
after = _segments(buf, mask)
return {
'key': _key_str(entry.key),
'name': entry.name,
'df': entry.df,
'fid': entry.fid,
'structure': entry.structure,
'size': entry.size,
'record_len': entry.record_len,
'n_records': entry.n_records,
'record': record,
'image_truncated': entry.image_truncated,
'coverage': entry.coverage(),
'kind': event['kind'],
'op': event['op'],
'index': event['index'],
'msg_id': event.get('msg_id'),
'elapsed': event.get('elapsed'),
'ranges': event['ranges'],
'observed_ranges': event.get('observed_ranges') or [],
'before': before,
'after': after,
}
def result(self, include_images=False):
files = {} files = {}
for key, entry in self.files.items(): for key, entry in self.files.items():
files[_key_str(key)] = { files[_key_str(key)] = {
+26
View File
@@ -141,6 +141,9 @@ class RequestHandler(BaseHTTPRequestHandler):
elif path.startswith('/api/sessions/') and path.endswith('/pcap'): elif path.startswith('/api/sessions/') and path.endswith('/pcap'):
session_id = int(path.split('/')[-2]) session_id = int(path.split('/')[-2])
self._handle_get_session_pcap(session_id) self._handle_get_session_pcap(session_id)
elif path.startswith('/api/sessions/') and path.endswith('/files/image'):
session_id = int(path.split('/')[-3])
self._handle_session_files_image(session_id, params)
elif path.startswith('/api/sessions/') and path.endswith('/files'): elif path.startswith('/api/sessions/') and path.endswith('/files'):
session_id = int(path.split('/')[-2]) session_id = int(path.split('/')[-2])
self._handle_session_files(session_id) self._handle_session_files(session_id)
@@ -475,6 +478,29 @@ class RequestHandler(BaseHTTPRequestHandler):
f'files={len(data["files"])} operations={data["timeline_total"]}') f'files={len(data["files"])} operations={data["timeline_total"]}')
self._send_json(data) self._send_json(data)
def _handle_session_files_image(self, session_id, params):
"""Before/after images of one change (the Files view's Details)."""
session = self.db.get_session(session_id)
if session is None:
self._send_error(404, 'Session not found')
return
key = (params.get('key', [''])[0]).strip()
try:
idx = int(params.get('change', ['0'])[0])
except ValueError:
self._send_error(400, 'change must be an index')
return
if not key or idx < 0:
self._send_error(400, 'key and change are required')
return
data = self.db.change_images(session_id, key, idx)
if data is None:
self._send_error(404, 'No such file or change')
return
_log(f'Session file change: session={session_id} key={key} '
f'change={idx} op={data["op"]} index={data["index"]}')
self._send_json(data)
def _handle_rename_session(self, session_id, name): def _handle_rename_session(self, session_id, name):
self.db.rename_session(session_id, name) self.db.rename_session(session_id, name)
self._send_json({'ok': True}) self._send_json({'ok': True})
+1 -1
View File
@@ -1,3 +1,3 @@
"""Single source of truth for the SIMtrace Analyser version.""" """Single source of truth for the SIMtrace Analyser version."""
__version__ = '2.3.1' __version__ = '2.4.0'
+200 -32
View File
@@ -39,16 +39,16 @@ def fcp(fid, sfi=None, structure=None, rec_len=None, n_rec=None, size=None):
return '62' + f'{len(body) // 2:02x}' + body return '62' + f'{len(body) // 2:02x}' + body
def read_binary(p1, p2, data_hex, sw='9000'): def read_binary(offset, data_hex, sw='9000'):
return f'00b0{p1:02x}{p2:02x}{len(data_hex) // 2:02x}{data_hex}{sw}' return f'00b0{offset >> 8:02x}{offset & 0xff:02x}{len(data_hex) // 2:02x}{data_hex}{sw}'
def read_binary_sfi(sfi, offset, data_hex, sw='9000'): def read_binary_sfi(sfi, offset, data_hex, sw='9000'):
return f'00b0{0x80 | sfi:02x}{offset:02x}{len(data_hex) // 2:02x}{data_hex}{sw}' return f'00b0{0x80 | sfi:02x}{offset:02x}{len(data_hex) // 2:02x}{data_hex}{sw}'
def update_binary(p1, p2, data_hex, sw='9000'): def update_binary(offset, data_hex, sw='9000'):
return f'00d6{p1:02x}{p2:02x}{len(data_hex) // 2:02x}{data_hex}{sw}' return f'00d6{offset >> 8:02x}{offset & 0xff:02x}{len(data_hex) // 2:02x}{data_hex}{sw}'
def update_binary_sfi(sfi, offset, data_hex, sw='9000'): def update_binary_sfi(sfi, offset, data_hex, sw='9000'):
@@ -89,8 +89,8 @@ class TestCountersAndRegistration(TrackerCase):
def test_select_read_update_counters(self): def test_select_read_update_counters(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '9807110900106014000000'[:22]), read_binary(0, '9807110900106014000000'[:22]),
update_binary(0, 0, '980711090010601495F9'), update_binary(0, '980711090010601495F9'),
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
@@ -107,7 +107,7 @@ class TestCountersAndRegistration(TrackerCase):
from simtrace2_pysniff.gsmtap import GSMTAP_SIM_ATR from simtrace2_pysniff.gsmtap import GSMTAP_SIM_ATR
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '98' + '07' * 5), read_binary(0, '98' + '07' * 5),
) )
gizmo = {'kind': 'read'} gizmo = {'kind': 'read'}
# An ATR (card reset) does not clear the per-file registry. # An ATR (card reset) does not clear the per-file registry.
@@ -119,11 +119,11 @@ class TestCountersAndRegistration(TrackerCase):
db.insert_messages(sid, [ db.insert_messages(sid, [
(0.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0), (0.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0),
(0.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0), (0.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0),
(1.0, 'tpdu', bytes.fromhex(read_binary(0, 0, '98' + '07' * 5)), 0), (1.0, 'tpdu', bytes.fromhex(read_binary(0, '98' + '07' * 5)), 0),
(2.0, 'atr', bytes.fromhex('3b9f96801fc78031e073fe2117574a3382033339001e'), 0), (2.0, 'atr', bytes.fromhex('3b9f96801fc78031e073fe2117574a3382033339001e'), 0),
(3.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0), (3.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0),
(3.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0), (3.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0),
(4.0, 'tpdu', bytes.fromhex(read_binary(0, 0, '98' + '07' * 5)), 0), (4.0, 'tpdu', bytes.fromhex(read_binary(0, '98' + '07' * 5)), 0),
]) ])
data = db.track_files(sid) data = db.track_files(sid)
self.assertEqual(data['files']['3f00/2fe2']['counters'], self.assertEqual(data['files']['3f00/2fe2']['counters'],
@@ -134,10 +134,10 @@ class TestCountersAndRegistration(TrackerCase):
def test_channels_merge_into_one_file(self): def test_channels_merge_into_one_file(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '9807110900'), read_binary(0, '9807110900'),
'01' + sel_fid('2fe2')[2:], # same file on channel 1 '01' + sel_fid('2fe2')[2:], # same file on channel 1
'01' + gr(fcp('2fe2', sfi=2, size=10))[2:], '01' + gr(fcp('2fe2', sfi=2, size=10))[2:],
'01' + read_binary(0, 0, '9807110900')[2:], '01' + read_binary(0, '9807110900')[2:],
) )
self.assertEqual(data['files']['3f00/2fe2']['counters']['read'], 2) self.assertEqual(data['files']['3f00/2fe2']['counters']['read'], 2)
self.assertEqual(len(data['files']), 1) self.assertEqual(len(data['files']), 1)
@@ -147,7 +147,7 @@ class TestImageAndChanges(TrackerCase):
def test_read_fills_unknown_bytes_without_a_change(self): def test_read_fills_unknown_bytes_without_a_change(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
self.assertEqual(entry['coverage']['known'], 10) self.assertEqual(entry['coverage']['known'], 10)
@@ -157,8 +157,8 @@ class TestImageAndChanges(TrackerCase):
def test_card_change_surfaced_by_a_read(self): def test_card_change_surfaced_by_a_read(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
read_binary(0, 0, '98071109001060140000'), read_binary(0, '98071109001060140000'),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
self.assertEqual(entry['changes_total'], 1) self.assertEqual(entry['changes_total'], 1)
@@ -178,8 +178,8 @@ class TestImageAndChanges(TrackerCase):
def test_repeating_a_read_is_not_a_change(self): def test_repeating_a_read_is_not_a_change(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
) )
self.assertEqual(data['files']['3f00/2fe2']['changes_total'], 0) self.assertEqual(data['files']['3f00/2fe2']['changes_total'], 0)
self.assertEqual([i['kind'] for i in data['timeline']], ['read', 'read']) self.assertEqual([i['kind'] for i in data['timeline']], ['read', 'read'])
@@ -187,8 +187,8 @@ class TestImageAndChanges(TrackerCase):
def test_device_change_from_an_update(self): def test_device_change_from_an_update(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
update_binary(0, 0, '98071109001060140000'), update_binary(0, '98071109001060140000'),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
self.assertEqual(entry['changes_total'], 1) self.assertEqual(entry['changes_total'], 1)
@@ -201,24 +201,39 @@ class TestImageAndChanges(TrackerCase):
def test_identical_update_is_flagged(self): def test_identical_update_is_flagged(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '980711090010601495F9'), read_binary(0, '980711090010601495F9'),
update_binary(0, 0, '980711090010601495F9'), update_binary(0, '980711090010601495F9'),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
self.assertEqual(entry['changes_total'], 0) self.assertEqual(entry['changes_total'], 0)
update_row = [i for i in data['timeline'] if i['kind'] == 'update'][0] update_row = [i for i in data['timeline'] if i['kind'] == 'update'][0]
self.assertTrue(update_row['no_change']) self.assertTrue(update_row['no_change'])
def test_update_of_unknown_bytes_learns_without_an_event(self): def test_update_of_unknown_bytes_is_an_observation_not_a_change(self):
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
update_binary(0, 0, '9807110900'), update_binary(0, '9807110900'),
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
self.assertEqual(entry['coverage']['known'], 5) self.assertEqual(entry['coverage']['known'], 5)
self.assertEqual(entry['changes_total'], 0) self.assertEqual(entry['changes_total'], 1)
change = entry['changes'][0]
self.assertEqual(change['kind'], 'device')
self.assertEqual(change['ranges'], []) # nothing changed
self.assertEqual(change['observed_ranges'], [[0, 5]]) # but it was new
update_row = [i for i in data['timeline'] if i['kind'] == 'update'][0] update_row = [i for i in data['timeline'] if i['kind'] == 'update'][0]
self.assertFalse(update_row['no_change']) # cannot prove a no-op self.assertFalse(update_row['no_change']) # cannot prove a no-op
self.assertEqual(update_row['change_ref'], {'file': '3f00/2fe2', 'idx': 0})
def test_read_records_the_newly_observed_bytes(self):
data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, '980711090010601495F9'),
read_binary(0, '98071109001060140000'),
)
change = data['files']['3f00/2fe2']['changes'][0]
self.assertEqual(change['ranges'], [[8, 2]])
self.assertEqual(change['observed_ranges'], []) # nothing was new
class TestRecordFiles(TrackerCase): class TestRecordFiles(TrackerCase):
@@ -235,7 +250,8 @@ class TestRecordFiles(TrackerCase):
self.assertEqual(entry['n_records'], 3) self.assertEqual(entry['n_records'], 3)
self.assertEqual(entry['coverage']['known'], 16) self.assertEqual(entry['coverage']['known'], 16)
self.assertEqual(entry['changes_total'], 1) self.assertEqual(entry['changes_total'], 1)
self.assertEqual(entry['changes'][0]['ranges'], [[1, 1]]) self.assertEqual(entry['changes'][0]['record'], 1)
self.assertEqual(entry['changes'][0]['ranges'], [[7, 1]])
def test_read_data_delivered_by_get_response(self): def test_read_data_delivered_by_get_response(self):
data = self.track( data = self.track(
@@ -258,7 +274,8 @@ class TestRecordFiles(TrackerCase):
entry = data['files']['7fff/6f07'] entry = data['files']['7fff/6f07']
self.assertEqual(entry['counters'], {'select': 1, 'read': 1, 'update': 1}) self.assertEqual(entry['counters'], {'select': 1, 'read': 1, 'update': 1})
self.assertEqual(entry['changes'][0]['kind'], 'device') self.assertEqual(entry['changes'][0]['kind'], 'device')
self.assertEqual(entry['changes'][0]['ranges'], [[2, 1]]) self.assertEqual(entry['changes'][0]['record'], 2)
self.assertEqual(entry['changes'][0]['ranges'], [[7, 1]])
class TestSfiResolution(TrackerCase): class TestSfiResolution(TrackerCase):
@@ -335,8 +352,8 @@ class TestCapsAndLabels(TrackerCase):
tracker = FileTracker(image_cap=8) tracker = FileTracker(image_cap=8)
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 0, '9807110900'), read_binary(0, '9807110900'),
read_binary(5, 0, '10601495F9'), read_binary(5, '10601495F9'),
tracker=tracker, tracker=tracker,
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
@@ -348,9 +365,9 @@ class TestCapsAndLabels(TrackerCase):
tracker = FileTracker(events_cap=1) tracker = FileTracker(events_cap=1)
data = self.track( data = self.track(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)), sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 0, 'AABBCCDD'), read_binary(0, 'AABBCCDD'),
read_binary(0, 0, 'AABBCC00'), read_binary(0, 'AABBCC00'),
read_binary(0, 0, 'AABB0000'), read_binary(0, 'AABB0000'),
tracker=tracker, tracker=tracker,
) )
entry = data['files']['3f00/2fe2'] entry = data['files']['3f00/2fe2']
@@ -371,11 +388,115 @@ class TestCapsAndLabels(TrackerCase):
def test_unattributed_operations_are_skipped(self): def test_unattributed_operations_are_skipped(self):
# A read with no selection and no SFI has no file to attribute. # A read with no selection and no SFI has no file to attribute.
data = self.track(read_binary(0, 0, 'AABBCCDD')) data = self.track(read_binary(0, 'AABBCCDD'))
self.assertEqual(data['files'], {}) self.assertEqual(data['files'], {})
self.assertEqual(data['timeline'], []) self.assertEqual(data['timeline'], [])
class TestChangeImages(TrackerCase):
"""The before/after views a change's Details modal shows."""
def _images(self, *tpdu, key='3f00/2fe2', idx=0):
from simtrace_analyser.filetrack import FileTracker
tmp = tempfile.NamedTemporaryFile(suffix='.db', delete=False)
tmp.close()
db = Database(tmp.name)
self.addCleanup(lambda: (db._conn.close(), os.remove(tmp.name)))
sid = db.create_session('test')
db.insert_messages(sid, [(float(i), 'tpdu', bytes.fromhex(h), 0)
for i, h in enumerate(tpdu)])
tracker = FileTracker()
db._decode_rows(db._conn.execute(
'SELECT id, session_id, elapsed, type, data, flags FROM messages '
'WHERE session_id=? ORDER BY elapsed, id', (sid,)).fetchall(),
on_message=tracker.observe)
return tracker.change_images(key, idx)
def test_device_change_before_and_after(self):
result = self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 'AABBCCDD'),
update_binary(0, 'AABB0000'),
)
self.assertEqual(result['kind'], 'device')
self.assertEqual(result['record'], None)
self.assertEqual(result['ranges'], [[2, 2]])
self.assertEqual(result['observed_ranges'], [])
# the whole file, not only the changed part
self.assertEqual(result['before'], [{'offset': 0, 'hex': 'aabbccdd'}])
self.assertEqual(result['after'], [{'offset': 0, 'hex': 'aabb0000'}])
def test_later_changes_are_undone_for_the_before_view(self):
# The first change's "after" must not include the second change,
# although the current image does.
first = self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 'AABBCCDD'),
update_binary(0, 'AABB0000'),
update_binary(2, 'CCDD'),
)
self.assertEqual(first['before'], [{'offset': 0, 'hex': 'aabbccdd'}])
self.assertEqual(first['after'], [{'offset': 0, 'hex': 'aabb0000'}])
def test_second_change_keeps_the_first_as_the_before_state(self):
result = self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 'AABBCCDD'),
update_binary(0, 'AABB0000'),
update_binary(2, 'CCDD'),
idx=1,
)
self.assertEqual(result['before'], [{'offset': 0, 'hex': 'aabb0000'}])
self.assertEqual(result['after'], [{'offset': 0, 'hex': 'aabbccdd'}])
self.assertEqual(result['ranges'], [[2, 2]])
def test_observation_only_event_shows_unknown_before(self):
result = self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
update_binary(0, 'AABBCCDD'),
)
self.assertEqual(result['ranges'], [])
self.assertEqual(result['observed_ranges'], [[0, 4]])
self.assertEqual(result['before'], []) # unknown before the write
self.assertEqual(result['after'], [{'offset': 0, 'hex': 'aabbccdd'}])
def test_record_change_shows_only_that_record(self):
result = self._images(
sel_fid('7fff'), sel_fid('6f07'),
gr(fcp('6f07', rec_len=4, n_rec=3)),
read_record(1, '11111111'),
read_record(2, '22222222'),
read_record(1, '11110011'),
key='7fff/6f07', idx=0,
)
self.assertEqual(result['record'], 1)
self.assertEqual(result['ranges'], [[2, 1]])
self.assertEqual(result['before'], [{'offset': 0, 'hex': '11111111'}])
self.assertEqual(result['after'], [{'offset': 0, 'hex': '11110011'}])
def test_unknown_key_or_index_returns_none(self):
self.assertIsNone(self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 'AABBCCDD'), key='3f00/9999'))
self.assertIsNone(self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=4)),
read_binary(0, 'AABBCCDD'), idx=5))
def test_segments_keep_unknown_gaps(self):
result = self._images(
sel_fid('2fe2'), gr(fcp('2fe2', sfi=2, size=10)),
read_binary(0, 'AABB'),
read_binary(6, 'CCDD'),
update_binary(6, 'CC00'),
)
self.assertEqual(result['before'],
[{'offset': 0, 'hex': 'aabb'},
{'offset': 6, 'hex': 'ccdd'}])
self.assertEqual(result['after'],
[{'offset': 0, 'hex': 'aabb'},
{'offset': 6, 'hex': 'cc00'}])
class TestEndpoint(unittest.TestCase): class TestEndpoint(unittest.TestCase):
def test_session_files_endpoint(self): def test_session_files_endpoint(self):
from simtrace_analyser.server import RequestHandler from simtrace_analyser.server import RequestHandler
@@ -402,7 +523,7 @@ class TestEndpoint(unittest.TestCase):
db.insert_messages(sid, [ db.insert_messages(sid, [
(0.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0), (0.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0),
(0.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0), (0.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=10))), 0),
(1.0, 'tpdu', bytes.fromhex(read_binary(0, 0, '980711090010601495F9')), 0), (1.0, 'tpdu', bytes.fromhex(read_binary(0, '980711090010601495F9')), 0),
]) ])
h = Fake(db) h = Fake(db)
h._handle_session_files(sid) h._handle_session_files(sid)
@@ -414,6 +535,53 @@ class TestEndpoint(unittest.TestCase):
h2._handle_session_files(99999) h2._handle_session_files(99999)
self.assertEqual(h2._status, 404) self.assertEqual(h2._status, 404)
def test_session_files_image_endpoint(self):
from simtrace_analyser.server import RequestHandler
class Fake(RequestHandler):
def __init__(self, db):
self.db = db
self._status = None
self._json = None
def _send_json(self, data, status=200):
self._status = status
self._json = data
def _send_error(self, status, message, code=None):
self._status = status
self._json = {'error': message}
tmp = tempfile.NamedTemporaryFile(suffix='.db', delete=False)
tmp.close()
db = Database(tmp.name)
self.addCleanup(lambda: (db._conn.close(), os.remove(tmp.name)))
sid = db.create_session('test')
db.insert_messages(sid, [
(0.0, 'tpdu', bytes.fromhex(sel_fid('2fe2')), 0),
(0.5, 'tpdu', bytes.fromhex(gr(fcp('2fe2', sfi=2, size=4))), 0),
(1.0, 'tpdu', bytes.fromhex(read_binary(0, 'AABBCCDD')), 0),
(2.0, 'tpdu', bytes.fromhex(update_binary(0, 'AABB0000')), 0),
])
h = Fake(db)
h._handle_session_files_image(sid, {'key': ['3f00/2fe2'], 'change': ['0']})
self.assertEqual(h._status, 200)
self.assertEqual(h._json['before'], [{'offset': 0, 'hex': 'aabbccdd'}])
self.assertEqual(h._json['after'], [{'offset': 0, 'hex': 'aabb0000'}])
self.assertEqual(h._json['ranges'], [[2, 2]])
self.assertEqual(h._json['op'], 'UPDATE BINARY')
# Unknown file / bad index → 404, bad parameters → 400.
for params in ({'key': ['3f00/9999'], 'change': ['0']},
{'key': ['3f00/2fe2'], 'change': ['9']}):
h2 = Fake(db)
h2._handle_session_files_image(sid, params)
self.assertEqual(h2._status, 404)
for params in ({'key': [''], 'change': ['0']},
{'key': ['3f00/2fe2'], 'change': ['x']}):
h3 = Fake(db)
h3._handle_session_files_image(sid, params)
self.assertEqual(h3._status, 400)
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
+1 -1
View File
@@ -224,7 +224,7 @@ class TestImportEndpoint(unittest.TestCase):
class TestVersion(unittest.TestCase): class TestVersion(unittest.TestCase):
def test_package_version(self): def test_package_version(self):
from simtrace2_pysniff import __version__ from simtrace2_pysniff import __version__
self.assertEqual(__version__, '2.3.1') self.assertEqual(__version__, '2.4.0')
def test_status_includes_version(self): def test_status_includes_version(self):
import os import os