WIP: saip.data_source

Change-Id: Ia6f70ff467ba58024d94742ba5cecd8141b93ad6

.checkpatch.conf

@@ -0,0 +1,2 @@
+--exclude ^pySim/esim/asn1/.*\.asn$
+--exclude ^smdpp-data/.*$

.gitignore

@@ -1,2 +1,5 @@
 *.pyc
 .*.swp
+
+/docs/_*
+/docs/generated

.gitreview

@@ -0,0 +1,3 @@
+[gerrit]
+host=gerrit.osmocom.org
+project=pysim

README

@@ -1,36 +0,0 @@
-This utility allows to :
-
- * Program customizable SIMs. Two modes are possible:
-
-  - one where you specify every parameter manually :
-
-./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -i <IMSI> -s <ICCID>
-
-
-  - one where they are generated from some minimal set :
-
-./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -z <random_string_of_choice> -j <card_num>
-
-    With <random_string_of_choice> and <card_num>, the soft will generate
-    'predictable' IMSI and ICCID, so make sure you choose them so as not to
-    conflict with anyone. (for eg. your name as <random_string_of_choice> and
-    0 1 2 ... for <card num>).
-
-  You also need to enter some parameters to select the device :
-   -t TYPE : type of card (supersim, magicsim, fakemagicsim or try 'auto')
-   -d DEV  : Serial port device (default /dev/ttyUSB0)
-   -b BAUD : Baudrate (default 9600)
-
- * Interact with SIMs from a python interactive shell (ipython for eg :)
-
-from pySim.transport.serial import SerialSimLink
-from pySim.commands import SimCardCommands
-
-sl = SerialSimLink(device='/dev/ttyUSB0', baudrate=9600)
-sc = SimCardCommands(sl)
-
-	# Print IMSI
-print sc.read_binary(['3f00', '7f20', '6f07'])
-
-	# Run A3/A8
-print sc.run_gsm('00112233445566778899aabbccddeeff')

README.md

@@ -0,0 +1,147 @@
+pySim - Read, Write and Browse Programmable SIM/USIM/ISIM/HPSIM Cards
+=====================================================================
+
+This repository contains a number of Python programs that can be used
+to read, program (write) and browse all fields/parameters/files on
+SIM/USIM/ISIM/HPSIM cards used in 3GPP cellular networks from 2G to 5G.
+
+Note that the access control configuration of normal production cards
+issue by operators will restrict significantly which files a normal
+user can read, and particularly write to.
+
+The full functionality of pySim hence can only be used with on so-called
+programmable SIM/USIM/ISIM/HPSIM cards.
+
+Such SIM/USIM/ISIM/HPSIM cards are special cards, which - unlike those
+issued by regular commercial operators - come with the kind of keys that
+allow you to write the files/fields that normally only an operator can
+program.
+
+This is useful particularly if you are running your own cellular
+network, and want to configure your own SIM/USIM/ISIM/HPSIM cards for
+that network.
+
+
+Homepage
+--------
+
+Please visit the [official homepage](https://osmocom.org/projects/pysim/wiki)
+for usage instructions, manual and examples.
+
+
+Documentation
+-------------
+
+The pySim user manual can be built from this very source code by means
+of sphinx (with sphinxcontrib-napoleon and sphinx-argparse).  See the
+Makefile in the 'docs' directory.
+
+A pre-rendered HTML user manual of the current pySim 'git master' is
+available from <https://downloads.osmocom.org/docs/latest/pysim/> and
+a downloadable PDF version is published at
+<https://downloads.osmocom.org/docs/latest/osmopysim-usermanual.pdf>.
+
+A slightly dated video presentation about pySim-shell can be found at
+<https://media.ccc.de/v/osmodevcall-20210409-laforge-pysim-shell>.
+
+
+pySim-shell vs. legacy tools
+----------------------------
+
+While you will find a lot of online resources still describing the use of
+pySim-prog.py and pySim-read.py, those tools are considered legacy by
+now and have by far been superseded by the much more capable
+pySim-shell.  We strongly encourage users to adopt pySim-shell, unless
+they have very specific requirements like batch programming of large
+quantities of cards, which is about the only remaining use case for the
+legacy tools.
+
+
+Git Repository
+--------------
+
+You can clone from the official Osmocom  git repository using
+```
+git clone https://gitea.osmocom.org/sim-card/pysim.git
+```
+
+There is a web interface at <https://gitea.osmocom.org/sim-card/pysim>.
+
+
+Installation
+------------
+
+Please install the following dependencies:
+
+ - bidict
+ - cmd2 >= 1.5.0
+ - colorlog
+ - construct >= 2.9.51
+ - gsm0338
+ - jsonpath-ng
+ - packaging
+ - pycryptodomex
+ - pyscard
+ - pyserial
+ - pytlv
+ - pyyaml >= 5.1
+ - smpp.pdu (from `github.com/hologram-io/smpp.pdu`)
+ - termcolor
+
+Example for Debian:
+```sh
+sudo apt-get install --no-install-recommends \
+	pcscd libpcsclite-dev \
+	python3 \
+	python3-setuptools \
+	python3-pycryptodome \
+	python3-pyscard \
+	python3-pip
+pip3 install --user -r requirements.txt
+```
+
+After installing all dependencies, the pySim applications ``pySim-read.py``, ``pySim-prog.py`` and ``pySim-shell.py`` may be started directly from the cloned repository.
+
+In addition to the dependencies above ``pySim-trace.py`` requires ``tshark`` and the python package ``pyshark`` to be installed. It is known that the ``tshark`` package
+in Debian versions before 11 may not work with pyshark.
+
+### Archlinux Package
+
+Archlinux users may install the package ``python-pysim-git``
+[![](https://img.shields.io/aur/version/python-pysim-git)](https://aur.archlinux.org/packages/python-pysim-git)
+from the [Arch User Repository (AUR)](https://aur.archlinux.org).
+The most convenient way is the use of an [AUR Helper](https://wiki.archlinux.org/index.php/AUR_helpers),
+e.g. [yay](https://aur.archlinux.org/packages/yay) or [pacaur](https://aur.archlinux.org/packages/pacaur).
+The following example shows the installation with ``yay``.
+
+```sh
+# Install
+yay -Sy python-pysim-git
+
+# Uninstall
+sudo pacman -Rs python-pysim-git
+```
+
+
+Mailing List
+------------
+
+There is no separate mailing list for this project. However,
+discussions related to pysim-prog are happening on the
+<openbsc@lists.osmocom.org> mailing list, please see
+<https://lists.osmocom.org/mailman/listinfo/openbsc> for subscription
+options and the list archive.
+
+Please observe the [Osmocom Mailing List
+Rules](https://osmocom.org/projects/cellular-infrastructure/wiki/Mailing_List_Rules)
+when posting.
+
+
+Contributing
+------------
+
+Our coding standards are described at
+<https://osmocom.org/projects/cellular-infrastructure/wiki/Coding_standards>
+
+We are using a gerrit-based patch review process explained at
+<https://osmocom.org/projects/cellular-infrastructure/wiki/Gerrit>

ccc-fix.py

@@ -1,153 +0,0 @@
-#!/usr/bin/env python
-
-#
-# Utility to write the cards
-#
-#
-# Copyright (C) 2009  Sylvain Munaut <tnt@246tNt.com>
-# Copyright (C) 2010  Harald Welte <laforge@gnumonks.org>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-
-from optparse import OptionParser
-
-from pySim.commands import SimCardCommands
-from pySim.cards import _cards_classes
-
-
-
-def card_detect(opts, scc):
-
-	# Detect type if needed
-	card = None
-	ctypes = dict([(kls.name, kls) for kls in _cards_classes])
-
-	if opts.type in ("auto", "auto_once"):
-		for kls in _cards_classes:
-			card = kls.autodetect(scc)
-			if card:
-				print "Autodetected card type %s" % card.name
-				card.reset()
-				break
-
-		if card is None:
-			print "Autodetection failed"
-			return
-
-		if opts.type == "auto_once":
-			opts.type = card.name
-
-	elif opts.type in ctypes:
-		card = ctypes[opts.type](scc)
-
-	else:
-		raise ValueError("Unknown card type %s" % opts.type)
-
-	return card
-
-
-#
-# Main
-#
-
-def parse_options():
-
-	parser = OptionParser(usage="usage: %prog [options]")
-
-	# Card interface
-	parser.add_option("-d", "--device", dest="device", metavar="DEV",
-			help="Serial Device for SIM access [default: %default]",
-			default="/dev/ttyUSB0",
-		)
-	parser.add_option("-b", "--baud", dest="baudrate", type="int", metavar="BAUD",
-			help="Baudrate used for SIM access [default: %default]",
-			default=9600,
-		)
-	parser.add_option("-p", "--pcsc-device", dest="pcsc_dev", type='int', metavar="PCSC",
-			help="Which PC/SC reader number for SIM access",
-			default=None,
-		)
-	parser.add_option("-t", "--type", dest="type",
-			help="Card type (user -t list to view) [default: %default]",
-			default="auto",
-		)
-
-	(options, args) = parser.parse_args()
-
-	if options.type == 'list':
-		for kls in _cards_classes:
-			print kls.name
-		sys.exit(0)
-
-	if args:
-		parser.error("Extraneous arguments")
-
-	return options
-
-
-def main():
-
-	# Parse options
-	opts = parse_options()
-
-	# Connect to the card
-	if opts.pcsc_dev is None:
-		from pySim.transport.serial import SerialSimLink
-		sl = SerialSimLink(device=opts.device, baudrate=opts.baudrate)
-	else:
-		from pySim.transport.pcsc import PcscSimLink
-		sl = PcscSimLink(opts.pcsc_dev)
-
-	# Create command layer
-	scc = SimCardCommands(transport=sl)
-
-	# Iterate
-	done = False
-	first = True
-	card = None
-
-	while not done:
-		# Connect transport
-		print "Insert card now (or CTRL-C to cancel)"
-		sl.wait_for_card(newcardonly=not first)
-
-		# Not the first anymore !
-		first = False
-
-		# Get card
-		card = card_detect(opts, scc)
-		if card is None:
-			if opts.batch_mode:
-				first = False
-				continue
-			else:
-				sys.exit(-1)
-
-		# Check type
-		if card.name != 'fakemagicsim':
-			print "Can't fix this type of card ..."
-			continue
-
-		# Fix record
-		data, sw = scc.read_record(['000c'], 1)
-		data_new = data[0:100] + 'fffffffffffffffffffffffffdffffffffffffffffffffffff0791947106004034ffffffffffffff'
-		scc.update_record(['000c'], 1, data_new)
-
-		# Done for this card and maybe for everything ?
-		print "Card should be fixed now !\n"
-
-
-if __name__ == '__main__':
-	main()

ccc-gen.py

@@ -1,230 +0,0 @@
-#!/usr/bin/env python
-
-#
-# Utility to generate the HLR
-#
-#
-# Copyright (C) 2010  Sylvain Munaut <tnt@246tNt.com>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-
-from optparse import OptionParser
-
-from ccc import StateManager, CardParametersGenerator, isnum
-from pySim.utils import h2b
-
-
-#
-# OpenBSC HLR Writing
-#
-
-def _dbi_binary_quote(s):
-	# Count usage of each char
-	cnt = {}
-	for c in s:
-		cnt[c] = cnt.get(c, 0) + 1
-
-	# Find best offset
-	e = 0
-	m = len(s)
-	for i in range(1, 256):
-		if i == 39:
-			continue
-		sum_ = cnt.get(i, 0) + cnt.get((i+1)&0xff, 0) + cnt.get((i+39)&0xff, 0)
-		if sum_ < m:
-			m = sum_
-			e = i
-			if m == 0:	# No overhead ? use this !
-				break;
-
-	# Generate output
-	out = []
-	out.append( chr(e) )	# Offset
-	for c in s:
-		x = (256 + ord(c) - e) % 256
-		if x in (0, 1, 39):
-			out.append('\x01')
-			out.append(chr(x+1))
-		else:
-			out.append(chr(x))
-
-	return ''.join(out)
-
-
-def hlr_write_cards(filename, network, cards):
-
-	import sqlite3
-
-	conn = sqlite3.connect(filename)
-
-	for card in cards:
-		c = conn.execute(
-			'INSERT INTO Subscriber ' +
-			'(imsi, name, extension, authorized, created, updated) ' +
-			'VALUES ' +
-			'(?,?,?,1,datetime(\'now\'),datetime(\'now\'));',
-			[
-				card.imsi,
-				'%s #%d' % (network.name, card.num),
-				'9%05d' % card.num,
-			],
-		)
-		sub_id = c.lastrowid
-		c.close()
-
-		c = conn.execute(
-			'INSERT INTO AuthKeys ' +
-			'(subscriber_id, algorithm_id, a3a8_ki)' +
-			'VALUES ' +
-			'(?,?,?)',
-			[ sub_id, 2, sqlite3.Binary(_dbi_binary_quote(h2b(card.ki))) ],
-		)
-		c.close()
-
-	conn.commit()
-	conn.close()
-
-
-#
-# CSV Writing
-#
-
-def csv_write_cards(filename, network, cards):
-	import csv
-	fh = open(filename, 'a')
-	cw = csv.writer(fh)
-	cw.writerows(cards)
-	fh.close()
-
-
-#
-# Main stuff
-#
-
-def parse_options():
-
-	parser = OptionParser(usage="usage: %prog [options]")
-
-	# Network parameters
-	parser.add_option("-n", "--name", dest="name",
-			help="Operator name [default: %default]",
-			default="CCC Event",
-		)
-	parser.add_option("-c", "--country", dest="country", type="int", metavar="CC",
-			help="Country code [default: %default]",
-			default=49,
-		)
-	parser.add_option("-x", "--mcc", dest="mcc", type="int",
-			help="Mobile Country Code [default: %default]",
-			default=262,
-		)
-	parser.add_option("-y", "--mnc", dest="mnc", type="int",
-			help="Mobile Network Code [default: %default]",
-			default=42,
-		)
-	parser.add_option("-m", "--smsp", dest="smsp",
-			help="SMSP [default: '00 + country code + 5555']",
-		)
-
-	# Autogen
-	parser.add_option("-z", "--secret", dest="secret", metavar="STR",
-			help="Secret used for ICCID/IMSI autogen",
-		)
-	parser.add_option("-k", "--count", dest="count", type="int", metavar="CNT",
-			help="Number of entried to generate [default: %default]",
-			default=1000,
-		)
-
-	# Output
-	parser.add_option("--state", dest="state_file", metavar="FILE",
-			help="Use this state file",
-		)
-	parser.add_option("--write-csv", dest="write_csv", metavar="FILE",
-			help="Append generated parameters in CSV file",
-		)
-	parser.add_option("--write-hlr", dest="write_hlr", metavar="FILE",
-			help="Append generated parameters to OpenBSC HLR sqlite3",
-		)
-
-	(options, args) = parser.parse_args()
-
-	if args:
-		parser.error("Extraneous arguments")
-
-	# Check everything
-	if 1 < len(options.name) > 16:
-		parser.error("Name must be between 1 and 16 characters")
-
-	if 0 < options.country > 999:
-		parser.error("Invalid country code")
-
-	if 0 < options.mcc > 999:
-		parser.error("Invalid Mobile Country Code (MCC)")
-	if 0 < options.mnc > 999:
-		parser.error("Invalid Mobile Network Code (MNC)")
-
-	if options.smsp is not None:
-		if not isnum(options.smsp):
-			parser.error("Invalid SMSP Number")
-	else:
-		options.smsp = '00%d' % options.country + '5555'
-
-	return options
-
-
-def main():
-
-	# Parse options
-	opts = parse_options()
-
-	# Load state
-	sm = StateManager(opts.state_file, opts)
-	sm.load()
-
-	# Instanciate generator
-	np = sm.network
-	cpg = CardParametersGenerator(np.cc, np.mcc, np.mnc, sm.get_secret())
-
-	# Generate cards
-	imsis = set()
-	cards = []
-	while len(cards) < opts.count:
-		# Next number
-		i = sm.next_gen_num()
-
-		# Generate card number
-		cp = cpg.generate(i)
-
-		# Check for dupes
-		if cp.imsi in imsis:
-			continue
-		imsis.add(cp.imsi)
-
-		# Collect
-		cards.append(cp)
-
-	# Save cards
-	if opts.write_hlr:
-		hlr_write_cards(opts.write_hlr, np, cards)
-
-	if opts.write_csv:
-		csv_write_cards(opts.write_csv, np, cards)
-
-	# Save state
-	sm.save()
-
-
-if __name__ == '__main__':
-	main()

ccc-prog.py

@@ -1,213 +0,0 @@
-#!/usr/bin/env python
-
-#
-# Utility to write the cards
-#
-#
-# Copyright (C) 2009  Sylvain Munaut <tnt@246tNt.com>
-# Copyright (C) 2010  Harald Welte <laforge@gnumonks.org>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-
-from optparse import OptionParser
-
-from pySim.commands import SimCardCommands
-from pySim.cards import _cards_classes
-
-from ccc import StateManager, CardParameters
-
-
-
-def csv_load_cards(filename):
-	import csv
-	fh = open(filename, 'r')
-	cr = csv.reader(fh)
-	cards = dict([(int(x[0]), CardParameters(int(x[0]), x[1], x[2], x[3])) for x in cr])
-	fh.close()
-	return cards
-
-
-def card_detect(opts, scc):
-
-	# Detect type if needed
-	card = None
-	ctypes = dict([(kls.name, kls) for kls in _cards_classes])
-
-	if opts.type in ("auto", "auto_once"):
-		for kls in _cards_classes:
-			card = kls.autodetect(scc)
-			if card:
-				print "Autodetected card type %s" % card.name
-				card.reset()
-				break
-
-		if card is None:
-			print "Autodetection failed"
-			return
-
-		if opts.type == "auto_once":
-			opts.type = card.name
-
-	elif opts.type in ctypes:
-		card = ctypes[opts.type](scc)
-
-	else:
-		raise ValueError("Unknown card type %s" % opts.type)
-
-	return card
-
-
-def print_parameters(params):
-
-	print """Generated card parameters :
- > Name    : %(name)s
- > SMSP    : %(smsp)s
- > ICCID   : %(iccid)s
- > MCC/MNC : %(mcc)d/%(mnc)d
- > IMSI    : %(imsi)s
- > Ki      : %(ki)s
-"""	% params
-
-
-#
-# Main
-#
-
-def parse_options():
-
-	parser = OptionParser(usage="usage: %prog [options]")
-
-	# Card interface
-	parser.add_option("-d", "--device", dest="device", metavar="DEV",
-			help="Serial Device for SIM access [default: %default]",
-			default="/dev/ttyUSB0",
-		)
-	parser.add_option("-b", "--baud", dest="baudrate", type="int", metavar="BAUD",
-			help="Baudrate used for SIM access [default: %default]",
-			default=9600,
-		)
-	parser.add_option("-p", "--pcsc-device", dest="pcsc_dev", type='int', metavar="PCSC",
-			help="Which PC/SC reader number for SIM access",
-			default=None,
-		)
-	parser.add_option("-t", "--type", dest="type",
-			help="Card type (user -t list to view) [default: %default]",
-			default="auto",
-		)
-	parser.add_option("-e", "--erase", dest="erase", action='store_true',
-			help="Erase beforehand [default: %default]",
-			default=False,
-		)
-
-	# Data source
-	parser.add_option("--state", dest="state_file", metavar="FILE",
-			help="Use this state file",
-			)
-	parser.add_option("--read-csv", dest="read_csv", metavar="FILE",
-			help="Read parameters from CSV file",
-		)
-
-	(options, args) = parser.parse_args()
-
-	if options.type == 'list':
-		for kls in _cards_classes:
-			print kls.name
-		sys.exit(0)
-
-	if args:
-		parser.error("Extraneous arguments")
-
-	return options
-
-
-def main():
-
-	# Parse options
-	opts = parse_options()
-
-	# Connect to the card
-	if opts.pcsc_dev is None:
-		from pySim.transport.serial import SerialSimLink
-		sl = SerialSimLink(device=opts.device, baudrate=opts.baudrate)
-	else:
-		from pySim.transport.pcsc import PcscSimLink
-		sl = PcscSimLink(opts.pcsc_dev)
-
-	# Create command layer
-	scc = SimCardCommands(transport=sl)
-
-	# Load state
-	sm = StateManager(opts.state_file)
-	sm.load()
-
-	np = sm.network
-
-	# Load cards
-	cards = csv_load_cards(opts.read_csv)
-
-	# Iterate
-	done = False
-	first = True
-	card = None
-
-	while not done:
-		# Connect transport
-		print "Insert card now (or CTRL-C to cancel)"
-		sl.wait_for_card(newcardonly=not first)
-
-		# Not the first anymore !
-		first = False
-
-		# Get card
-		card = card_detect(opts, scc)
-		if card is None:
-			if opts.batch_mode:
-				first = False
-				continue
-			else:
-				sys.exit(-1)
-
-		# Erase if requested
-		if opts.erase:
-			print "Formatting ..."
-			card.erase()
-			card.reset()
-
-		# Get parameters
-		cp = cards[sm.next_write_num()]
-		cpp = {
-			'name': np.name,
-			'smsp': np.smsp,
-			'iccid': cp.iccid,
-			'mcc': np.mcc,
-			'mnc': np.mnc,
-			'imsi': cp.imsi,
-			'ki': cp.ki,
-		}
-		print_parameters(cpp)
-
-		# Program the card
-		print "Programming ..."
-		card.program(cpp)
-
-		# Update state
-		sm.save()
-
-		# Done for this card and maybe for everything ?
-		print "Card written !\n"
-
-
-if __name__ == '__main__':
-	main()

ccc.py

@@ -1,193 +0,0 @@
-#!/usr/bin/env python
-
-#
-# CCC Event HLR management common stuff
-#
-#
-# Copyright (C) 2010  Sylvain Munaut <tnt@246tNt.com>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-#
-
-import hashlib
-import os
-import random
-
-from collections import namedtuple
-
-try:
-	import json
-except Importerror:
-	# Python < 2.5
-	import simplejson as json
-
-#
-# Various helpers
-#
-
-def isnum(s, l=-1):
-	return s.isdigit() and ((l== -1) or (len(s) == l))
-
-
-#
-# Storage tuples
-#
-
-CardParameters = namedtuple("CardParameters", "num iccid imsi ki")
-NetworkParameters = namedtuple("NetworkParameters", "name cc mcc mnc smsp")
-
-
-#
-# State management
-#
-
-class StateManager(object):
-
-	def __init__(self, filename=None, options=None):
-		# Filename for state storage
-		self._filename = filename
-
-		# Params from options
-		self._net_name = options.name		if options else None
-		self._net_cc   = options.country	if options else None
-		self._net_mcc  = options.mcc		if options else None
-		self._net_mnc  = options.mnc		if options else None
-		self._net_smsp = options.smsp		if options else None
-
-		self._secret   = options.secret		if options else None
-
-		# Default
-		self._num_gen = 0
-		self._num_write = 0
-
-	def load(self):
-		# Skip if no state file
-		if self._filename is None:
-			return
-
-		# Skip if doesn't exist yet
-		if not os.path.isfile(self._filename):
-			return
-
-		# Read
-		fh = open(self._filename, 'r')
-		data = fh.read()
-		fh.close()
-
-		# Decode json and merge
-		dd = json.loads(data)
-
-		self._net_name  = dd['name']
-		self._net_cc    = dd['cc']
-		self._net_mcc   = dd['mcc']
-		self._net_mnc   = dd['mnc']
-		self._net_smsp  = dd['smsp']
-		self._secret    = dd['secret']
-		self._num_gen   = dd['num_gen']
-		self._num_write = dd['num_write']
-
-	def save(self):
-		# Skip if no state file
-		if self._filename is None:
-			return
-
-		# Serialize
-		data = json.dumps({
-			'name':      self._net_name,
-			'cc':        self._net_cc,
-			'mcc':       self._net_mcc,
-			'mnc':       self._net_mnc,
-			'smsp':      self._net_smsp,
-			'secret':    self._secret,
-			'num_gen':   self._num_gen,
-			'num_write': self._num_write,
-		})
-
-		# Save in json
-		fh = open(self._filename, 'w')
-		fh.write(data)
-		fh.close()
-
-	@property
-	def network(self):
-		return NetworkParameters(
-			self._net_name,
-			self._net_cc,
-			self._net_mcc,
-			self._net_mnc,
-			self._net_smsp,
-		)
-
-	def get_secret(self):
-		return self._secret
-
-	def next_gen_num(self):
-		n = self._num_gen
-		self._num_gen += 1
-		return n
-
-	def next_write_num(self):
-		n = self._num_write
-		self._num_write += 1
-		return n
-
-#
-# Card parameters generation
-#
-
-class CardParametersGenerator(object):
-
-	def __init__(self, cc, mcc, mnc, secret):
-		# Digitize country code (2 or 3 digits)
-		self._cc_digits = ('%03d' if cc > 100 else '%02d') % cc
-
-		# Digitize MCC/MNC (5 or 6 digits)
-		self._plmn_digits = ('%03d%03d' if mnc > 100 else '%03d%02d') % (mcc, mnc)
-
-		# Store secret
-		self._secret = secret
-
-	def _digits(self, usage, len_, num):
-		s = hashlib.sha1(self._secret + usage + '%d' % num)
-		d = ''.join(['%02d'%ord(x) for x in s.digest()])
-		return d[0:len_]
-
-	def _gen_iccid(self, num):
-		iccid = (
-			'89' +				# Common prefix (telecom)
-			self._cc_digits +	# Country Code on 2/3 digits
-			self._plmn_digits 	# MCC/MNC on 5/6 digits
-		)
-		ml = 20 - len(iccid)
-		iccid += self._digits('ccid', ml, num)
-		return iccid
-
-	def _gen_imsi(self, num):
-		ml = 15 - len(self._plmn_digits)
-		msin = self._digits('imsi', ml, num)
-		return (
-			self._plmn_digits +	# MCC/MNC on 5/6 digits
-			msin				# MSIN
-		)
-
-	def _gen_ki(self):
-		return ''.join(['%02x' % random.randrange(0,256) for i in range(16)])
-
-	def generate(self, num):
-		return CardParameters(
-			num,
-			self._gen_iccid(num),
-			self._gen_imsi(num),
-			self._gen_ki(),
-		)

contrib/eidtool.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+
+# Command line tool to compute or verify EID (eUICC ID) values
+#
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+import argparse
+
+from pySim.euicc import compute_eid_checksum, verify_eid_checksum
+
+
+option_parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,
+                                        description="""pySim EID Tool
+This utility program can be used to compute or verify the checksum of an EID
+(eUICC Identifier).  See GSMA SGP.29 for the algorithm details.
+
+Example (verification):
+    $ eidtool.py --verify 89882119900000000000000000001654
+    EID checksum verified successfully
+
+Example (generation, passing first 30 digits):
+    $ eidtool.py --compute 898821199000000000000000000016
+    89882119900000000000000000001654
+
+Example (generation, passing all 32 digits):
+    $ eidtool.py --compute 89882119900000000000000000001600
+    89882119900000000000000000001654
+
+Example (generation, specifying base 30 digits and number to add):
+    $ eidtool.py --compute 898821199000000000000000000000 --add 16
+    89882119900000000000000000001654
+""")
+group = option_parser.add_mutually_exclusive_group(required=True)
+group.add_argument('--verify', help='Verify given EID csum')
+group.add_argument('--compute', help='Generate EID csum')
+option_parser.add_argument('--add', type=int, help='Add value to EID base before computing')
+
+
+if __name__ == '__main__':
+    opts = option_parser.parse_args()
+
+    if opts.verify:
+        res = verify_eid_checksum(opts.verify)
+        if res:
+            print("EID checksum verified successfully")
+            sys.exit(0)
+        else:
+            print("EID checksum invalid")
+            sys.exit(1)
+    elif opts.compute:
+        eid = opts.compute
+        if opts.add:
+            if len(eid) != 30:
+                print("EID base must be 30 digits when using --add")
+                sys.exit(2)
+            eid = str(int(eid) + int(opts.add))
+        res = compute_eid_checksum(eid)
+        print(res)
+

contrib/es2p_client.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import copy
+import argparse
+from pySim.esim import es2p
+
+EID_HELP='EID of the eUICC for which eSIM shall be made available'
+ICCID_HELP='The ICCID of the eSIM that shall be made available'
+MATCHID_HELP='MatchingID that shall be used by profile download'
+
+parser = argparse.ArgumentParser(description="""
+Utility to manuall issue requests against the ES2+ API of an SM-DP+ according to GSMA SGP.22.""")
+parser.add_argument('--url', required=True, help='Base URL of ES2+ API endpoint')
+parser.add_argument('--id', required=True, help='Entity identifier passed to SM-DP+')
+parser.add_argument('--client-cert', help='X.509 client certificate used to authenticate to server')
+parser.add_argument('--server-ca-cert', help="""X.509 CA certificates acceptable for the server side. In
+                    production use cases, this would be the GSMA Root CA (CI) certificate.""")
+subparsers = parser.add_subparsers(dest='command',help="The command (API function) to call")
+
+parser_dlo = subparsers.add_parser('download-order', help="ES2+ DownloadOrder function")
+parser_dlo.add_argument('--eid', help=EID_HELP)
+parser_dlo.add_argument('--iccid', help=ICCID_HELP)
+parser_dlo.add_argument('--profileType', help='The profile type of which one eSIM shall be made available')
+
+parser_cfo = subparsers.add_parser('confirm-order', help="ES2+ ConfirmOrder function")
+parser_cfo.add_argument('--iccid', required=True, help=ICCID_HELP)
+parser_cfo.add_argument('--eid', help=EID_HELP)
+parser_cfo.add_argument('--matchingId', help=MATCHID_HELP)
+parser_cfo.add_argument('--confirmationCode', help='Confirmation code that shall be used by profile download')
+parser_cfo.add_argument('--smdsAddress', help='SM-DS Address')
+parser_cfo.add_argument('--releaseFlag', action='store_true', help='Shall the profile be immediately released?')
+
+parser_co = subparsers.add_parser('cancel-order', help="ES2+ CancelOrder function")
+parser_co.add_argument('--iccid', required=True, help=ICCID_HELP)
+parser_co.add_argument('--eid', help=EID_HELP)
+parser_co.add_argument('--matchingId', help=MATCHID_HELP)
+parser_co.add_argument('--finalProfileStatusIndicator', required=True, choices=['Available','Unavailable'])
+
+parser_rp = subparsers.add_parser('release-profile', help='ES2+ ReleaseProfile function')
+parser_rp.add_argument('--iccid', required=True, help=ICCID_HELP)
+
+if __name__ == '__main__':
+    opts = parser.parse_args()
+    #print(opts)
+
+    peer = es2p.Es2pApiClient(opts.url, opts.id, server_cert_verify=opts.server_ca_cert, client_cert=opts.client_cert)
+
+    data = {}
+    for k, v in vars(opts).items():
+        if k in ['url', 'id', 'client_cert', 'server_ca_cert', 'command']:
+            # remove keys from dict that shold not end up in JSON...
+            continue
+        if v is not None:
+            data[k] = v
+
+    print(data)
+    if opts.command == 'download-order':
+        res = peer.call_downloadOrder(data)
+    elif opts.command == 'confirm-order':
+        res = peer.call_confirmOrder(data)
+    elif opts.command == 'cancel-order':
+        res = peer.call_cancelOrder(data)
+    elif opts.command == 'release-profile':
+        res = peer.call_releaseProfile(data)

contrib/jenkins.sh

@@ -0,0 +1,63 @@
+#!/bin/sh -xe
+# jenkins build helper script for pysim.  This is how we build on jenkins.osmocom.org
+#
+# environment variables:
+# * WITH_MANUALS: build manual PDFs if set to "1"
+# * PUBLISH: upload manuals after building if set to "1" (ignored without WITH_MANUALS = "1")
+# * JOB_TYPE: one of 'test', 'pylint', 'docs'
+#
+
+export PYTHONUNBUFFERED=1
+
+if [ ! -d "./pysim-testdata/" ] ; then
+	echo "###############################################"
+	echo "Please call from pySim-prog top directory"
+	echo "###############################################"
+	exit 1
+fi
+
+case "$JOB_TYPE" in
+"test")
+	virtualenv -p python3 venv --system-site-packages
+	. venv/bin/activate
+
+	pip install -r requirements.txt
+	pip install pyshark
+
+	# Execute automatically discovered unit tests first
+	python -m unittest discover -v -s tests/
+
+	# Run the test with physical cards
+	cd pysim-testdata
+	../tests/pySim-prog_test.sh
+	../tests/pySim-trace_test.sh
+	;;
+"pylint")
+	# Print pylint version
+	pip3 freeze | grep pylint
+	# Run pylint to find potential errors
+	# Ignore E1102: not-callable
+	#   pySim/filesystem.py: E1102: method is not callable (not-callable)
+	# Ignore E0401: import-error
+	#   pySim/utils.py:276: E0401: Unable to import 'Crypto.Cipher' (import-error)
+	#   pySim/utils.py:277: E0401: Unable to import 'Crypto.Util.strxor' (import-error)
+	python3 -m pylint -j0 --errors-only \
+		--disable E1102 \
+		--disable E0401 \
+		--enable W0301 \
+		pySim tests/*.py *.py \
+		contrib/es2p_client.py
+	;;
+"docs")
+	rm -rf docs/_build
+	make -C "docs" html latexpdf
+
+	if [ "$WITH_MANUALS" = "1" ] && [ "$PUBLISH" = "1" ]; then
+		make -C "docs" publish publish-html
+	fi
+	;;
+*)
+	set +x
+	echo "ERROR: JOB_TYPE has unexpected value '$JOB_TYPE'."
+	exit 1
+esac

contrib/sim-rest-client.py

@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+#
+# sim-rest-client.py: client program to test the sim-rest-server.py
+#
+# this will generate authentication tuples just like a HLR / HSS
+# and will then send the related challenge to the REST interface
+# of sim-rest-server.py
+#
+# sim-rest-server.py will then contact the SIM card to perform the
+# authentication (just like a 3GPP RAN), and return the results via
+# the REST to sim-rest-client.py.
+#
+# (C) 2021 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import Optional, Dict
+
+import sys
+import argparse
+import secrets
+import requests
+
+from CryptoMobile.Milenage import Milenage
+from CryptoMobile.utils import xor_buf
+
+def unpack48(x:bytes) -> int:
+    """Decode a big-endian 48bit number from binary to integer."""
+    return int.from_bytes(x, byteorder='big')
+
+def pack48(x:int) -> bytes:
+    """Encode a big-endian 48bit number from integer to binary."""
+    return x.to_bytes(48 // 8, byteorder='big')
+
+def milenage_generate(opc:bytes, amf:bytes, k:bytes, sqn:bytes, rand:bytes) -> Dict[str, bytes]:
+    """Generate an MILENAGE Authentication Tuple."""
+    m = Milenage(None)
+    m.set_opc(opc)
+    mac_a = m.f1(k, rand, sqn, amf)
+    res, ck, ik, ak = m.f2345(k, rand)
+
+    # AUTN = (SQN ^ AK) || AMF || MAC
+    sqn_ak = xor_buf(sqn, ak)
+    autn = b''.join([sqn_ak, amf, mac_a])
+
+    return {'res': res, 'ck': ck, 'ik': ik, 'autn': autn}
+
+def milenage_auts(opc:bytes, k:bytes, rand:bytes, auts:bytes) -> Optional[bytes]:
+    """Validate AUTS. If successful, returns SQN_MS"""
+    amf = b'\x00\x00' # TS 33.102 Section 6.3.3
+    m = Milenage(None)
+    m.set_opc(opc)
+    ak = m.f5star(k, rand)
+
+    sqn_ak = auts[:6]
+    sqn = xor_buf(sqn_ak, ak[:6])
+
+    mac_s = m.f1star(k, rand, sqn, amf)
+    if mac_s == auts[6:14]:
+        return sqn
+    else:
+        return False
+
+
+def build_url(suffix:str, base_path="/sim-auth-api/v1") -> str:
+    """Build an URL from global server_host, server_port, BASE_PATH and suffix."""
+    return "http://%s:%u%s%s" % (server_host, server_port, base_path, suffix)
+
+
+def rest_post(suffix:str, js:Optional[dict] = None):
+    """Perform a RESTful POST."""
+    url = build_url(suffix)
+    if verbose:
+        print("POST %s (%s)" % (url, str(js)))
+    resp = requests.post(url, json=js)
+    if verbose:
+        print("-> %s" % (resp))
+    if not resp.ok:
+        print("POST failed")
+    return resp
+
+def rest_get(suffix:str, base_path=None):
+    """Perform a RESTful GET."""
+    url = build_url(suffix, base_path)
+    if verbose:
+        print("GET %s" % url)
+    resp = requests.get(url)
+    if verbose:
+        print("-> %s" % (resp))
+    if not resp.ok:
+        print("GET failed")
+    return resp
+
+
+def main_info(args):
+    resp = rest_get('/slot/%u' % args.slot_nr, base_path="/sim-info-api/v1")
+    if not resp.ok:
+        print("<- ERROR %u: %s" % (resp.status_code, resp.text))
+        sys.exit(1)
+    resp_json = resp.json()
+    print("<- %s" % resp_json)
+
+
+def main_auth(args):
+    #opc = bytes.fromhex('767A662ACF4587EB0C450C6A95540A04')
+    #k = bytes.fromhex('876B2D8D403EE96755BEF3E0A1857EBE')
+    opc = bytes.fromhex(args.opc)
+    k = bytes.fromhex(args.key)
+    amf = bytes.fromhex(args.amf)
+    sqn = bytes.fromhex(args.sqn)
+
+    for i in range(args.count):
+        rand = secrets.token_bytes(16)
+        t = milenage_generate(opc=opc, amf=amf, k=k, sqn=sqn, rand=rand)
+
+        req_json = {'rand': rand.hex(), 'autn': t['autn'].hex()}
+        print("-> %s" % req_json)
+        resp = rest_post('/slot/%u' % args.slot_nr, req_json)
+        if not resp.ok:
+            print("<- ERROR %u: %s" % (resp.status_code, resp.text))
+            break
+        resp_json = resp.json()
+        print("<- %s" % resp_json)
+        if 'synchronisation_failure' in resp_json:
+            auts = bytes.fromhex(resp_json['synchronisation_failure']['auts'])
+            sqn_ms = milenage_auts(opc, k, rand, auts)
+            if sqn_ms is not False:
+                print("SQN_MS = %s" % sqn_ms.hex())
+                sqn_ms_int = unpack48(sqn_ms)
+                # we assume an IND bit-length of 5 here
+                sqn = pack48(sqn_ms_int + (1 << 5))
+            else:
+                raise RuntimeError("AUTS auth failure during re-sync?!?")
+        elif 'successful_3g_authentication' in resp_json:
+            auth_res = resp_json['successful_3g_authentication']
+            assert bytes.fromhex(auth_res['res']) == t['res']
+            assert bytes.fromhex(auth_res['ck']) == t['ck']
+            assert bytes.fromhex(auth_res['ik']) == t['ik']
+            # we assume an IND bit-length of 5 here
+            sqn = pack48(unpack48(sqn) + (1 << 5))
+        else:
+            raise RuntimeError("Auth failure")
+
+
+def main(argv):
+    global server_port, server_host, verbose
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-H", "--host", help="Host to connect to", default="localhost")
+    parser.add_argument("-p", "--port", help="TCP port to connect to", default=8000)
+    parser.add_argument("-v", "--verbose", help="increase output verbosity", action='count', default=0)
+    parser.add_argument("-n", "--slot-nr", help="SIM slot number", type=int, default=0)
+    subp = parser.add_subparsers()
+
+    auth_p = subp.add_parser('auth', help='UMTS AKA Authentication')
+    auth_p.add_argument("-c", "--count", help="Auth count", type=int, default=10)
+    auth_p.add_argument("-k", "--key", help="Secret key K (hex)", type=str, required=True)
+    auth_p.add_argument("-o", "--opc", help="Secret OPc (hex)", type=str, required=True)
+    auth_p.add_argument("-a", "--amf", help="AMF Field (hex)", type=str, default="0000")
+    auth_p.add_argument("-s", "--sqn", help="SQN Field (hex)", type=str, default="000000000000")
+    auth_p.set_defaults(func=main_auth)
+
+    info_p = subp.add_parser('info', help='Information about the Card')
+    info_p.set_defaults(func=main_info)
+
+    args = parser.parse_args()
+    server_host = args.host
+    server_port = args.port
+    verbose = args.verbose
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main(sys.argv)

contrib/sim-rest-server.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+
+# RESTful HTTP service for performing authentication against USIM cards
+#
+# (C) 2021-2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import sys
+import argparse
+
+from klein import Klein
+
+from pySim.transport import ApduTracer
+from pySim.transport.pcsc import PcscSimLink
+from pySim.commands import SimCardCommands
+from pySim.cards import UiccCardBase
+from pySim.utils import dec_iccid, dec_imsi
+from pySim.ts_51_011 import EF_IMSI
+from pySim.ts_102_221 import EF_ICCID
+from pySim.exceptions import *
+
+class ApduPrintTracer(ApduTracer):
+    def trace_response(self, cmd, sw, resp):
+        #print("CMD: %s -> RSP: %s %s" % (cmd, sw, resp))
+        pass
+
+def connect_to_card(slot_nr:int):
+    tp = PcscSimLink(argparse.Namespace(pcsc_dev=slot_nr), apdu_tracer=ApduPrintTracer())
+    tp.connect()
+
+    scc = SimCardCommands(tp)
+    card = UiccCardBase(scc)
+
+    # this should be part of UsimCard, but FairewavesSIM breaks with that :/
+    scc.cla_byte = "00"
+    scc.sel_ctrl = "0004"
+
+    card.read_aids()
+
+    # ensure that MF is selected when we are done.
+    card._scc.select_file('3f00')
+
+    return tp, scc, card
+
+class ApiError:
+    def __init__(self, msg:str, sw=None):
+        self.msg = msg
+        self.sw = sw
+
+    def __str__(self):
+        d = {'error': {'message':self.msg}}
+        if self.sw:
+            d['error']['status_word'] = self.sw
+        return json.dumps(d)
+
+
+def set_headers(request):
+    request.setHeader('Content-Type', 'application/json')
+
+class SimRestServer:
+    app = Klein()
+
+    @app.handle_errors(NoCardError)
+    def no_card_error(self, request, failure):
+        set_headers(request)
+        request.setResponseCode(410)
+        return str(ApiError("No SIM card inserted in slot"))
+
+    @app.handle_errors(ReaderError)
+    def reader_error(self, request, failure):
+        set_headers(request)
+        request.setResponseCode(404)
+        return str(ApiError("Reader Error: Specified SIM Slot doesn't exist"))
+
+    @app.handle_errors(ProtocolError)
+    def protocol_error(self, request, failure):
+        set_headers(request)
+        request.setResponseCode(500)
+        return str(ApiError("Protocol Error: %s" % failure.value))
+
+    @app.handle_errors(SwMatchError)
+    def sw_match_error(self, request, failure):
+        set_headers(request)
+        request.setResponseCode(500)
+        sw = failure.value.sw_actual
+        if sw == '9862':
+            return str(ApiError("Card Authentication Error - Incorrect MAC", sw))
+        elif sw == '6982':
+            return str(ApiError("Security Status not satisfied - Card PIN enabled?", sw))
+        else:
+            return str(ApiError("Card Communication Error %s" % failure.value, sw))
+
+
+    @app.route('/sim-auth-api/v1/slot/<int:slot>')
+    def auth(self, request, slot):
+        """REST API endpoint for performing authentication against a USIM.
+           Expects a JSON body containing RAND and AUTN.
+           Returns a JSON body containing RES, CK, IK and Kc."""
+        try:
+            # there are two hex-string JSON parameters in the body: rand and autn
+            content = json.loads(request.content.read())
+            rand = content['rand']
+            autn = content['autn']
+        except:
+            set_headers(request)
+            request.setResponseCode(400)
+            return str(ApiError("Malformed Request"))
+
+        tp, scc, card = connect_to_card(slot)
+
+        card.select_adf_by_aid(adf='usim')
+        res, sw = scc.authenticate(rand, autn)
+
+        tp.disconnect()
+
+        set_headers(request)
+        return json.dumps(res, indent=4)
+
+    @app.route('/sim-info-api/v1/slot/<int:slot>')
+    def info(self, request, slot):
+        """REST API endpoint for obtaining information about an USIM.
+        Expects empty body in request.
+        Returns a JSON body containing ICCID, IMSI."""
+
+        tp, scc, card = connect_to_card(slot)
+
+        ef_iccid = EF_ICCID()
+        (iccid, sw) = card._scc.read_binary(ef_iccid.fid)
+
+        card.select_adf_by_aid(adf='usim')
+        ef_imsi = EF_IMSI()
+        (imsi, sw) = card._scc.read_binary(ef_imsi.fid)
+
+        res = {"imsi": dec_imsi(imsi), "iccid": dec_iccid(iccid) }
+
+        tp.disconnect()
+
+        set_headers(request)
+        return json.dumps(res, indent=4)
+
+
+def main(argv):
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-H", "--host", help="Host/IP to bind HTTP to", default="localhost")
+    parser.add_argument("-p", "--port", help="TCP port to bind HTTP to", default=8000)
+    #parser.add_argument("-v", "--verbose", help="increase output verbosity", action='count', default=0)
+
+    args = parser.parse_args()
+
+    srr = SimRestServer()
+    srr.app.run(args.host, args.port)
+
+if __name__ == "__main__":
+    main(sys.argv)

contrib/sim-rest-server.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Osmocom SIM REST server
+
+[Service]
+Type=simple
+# we listen to 0.0.0.0, allowing remote, unauthenticated clients to connect from everywhere!
+ExecStart=/usr/local/src/pysim/contrib/sim-rest-server.py -H 0.0.0.0
+Restart=always
+RestartSec=2
+# this user must be created beforehand; it must have PC/SC access
+User=rest
+
+[Install]
+WantedBy=multi-user.target

contrib/unber.py

@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+
+# A more useful verion of the 'unber' tool provided with asn1c:
+# Give a hierarchical decode of BER/DER-encoded ASN.1 TLVs
+
+import sys
+import argparse
+
+from pySim.utils import bertlv_parse_one, bertlv_encode_tag, b2h, h2b
+
+def process_one_level(content: bytes, indent: int):
+    remainder = content
+    while len(remainder):
+        tdict, l, v, remainder = bertlv_parse_one(remainder)
+        #print(tdict)
+        rawtag = bertlv_encode_tag(tdict)
+        if tdict['constructed']:
+            print("%s%s l=%d" % (indent*"  ", b2h(rawtag), l))
+            process_one_level(v, indent + 1)
+        else:
+            print("%s%s l=%d %s" % (indent*"  ", b2h(rawtag), l, b2h(v)))
+
+
+option_parser = argparse.ArgumentParser(description='BER/DER data dumper')
+group = option_parser.add_mutually_exclusive_group(required=True)
+group.add_argument('--file', help='Input file')
+group.add_argument('--hex', help='Input hexstring')
+
+
+if __name__ == '__main__':
+    opts = option_parser.parse_args()
+
+    if opts.file:
+        with open(opts.file, 'rb') as f:
+            content = f.read()
+    elif opts.hex:
+        content = h2b(opts.hex)
+
+    process_one_level(content, 0)

csv-format

@@ -0,0 +1,16 @@
+This file aims to describe the format of the CSV file pySim uses.
+
+The first line contains the fieldnames which will be used by pySim. This
+avoids having a specific order.
+
+The field names are the following:
+
+iccid: ICCID of the card. Used to identify the cards (with --read-iccid)
+imsi: IMSI of the card
+mcc: Mobile Country Code (optional)
+mnc: Mobile Network Code (optional)
+smsp: MSISDN of the SMSC (optional)
+ki: Ki
+opc: OPc
+acc: Access class of the SIM (optional)
+pin_adm: Admin PIN of the SIM. Needed to reprogram various files

docs/Makefile

@@ -0,0 +1,52 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# for osmo-gsm-manuals
+OSMO_GSM_MANUALS_DIR ?= $(shell pkg-config osmo-gsm-manuals --variable=osmogsmmanualsdir 2>/dev/null)
+OSMO_REPOSITORY = "pysim"
+UPLOAD_FILES = $(BUILDDIR)/latex/osmopysim-usermanual.pdf
+CLEAN_FILES = $(UPLOAD_FILES)
+
+# Copy variables from Makefile.common.inc that are used in publish-html,
+# as Makefile.common.inc must be included after publish-html
+PUBLISH_REF   ?= master
+PUBLISH_TEMPDIR = _publish_tmpdir
+SSH_COMMAND = ssh -o 'UserKnownHostsFile=$(OSMO_GSM_MANUALS_DIR)/build/known_hosts' -p 48
+
+# Put it first so that "make" without argument is like "make help".
+.PHONY: help
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+$(BUILDDIR)/latex/pysim.pdf: latexpdf
+	@/bin/true
+
+publish-html: html
+	rm -rf "$(PUBLISH_TEMPDIR)"
+	mkdir -p "$(PUBLISH_TEMPDIR)/pysim/$(PUBLISH_REF)"
+	cp -r "$(BUILDDIR)"/html "$(PUBLISH_TEMPDIR)/pysim/$(PUBLISH_REF)"
+	cd "$(PUBLISH_TEMPDIR)" && \
+		rsync \
+			-avzR \
+			-e "$(SSH_COMMAND)" \
+			"pysim" \
+			docs@ftp.osmocom.org:web-files/
+	rm -rf "$(PUBLISH_TEMPDIR)"
+
+# put this before the catch-all below
+include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.common.inc
+
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%:
+	@if [ "$@" != "shrink" ]; then \
+		$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O); \
+	fi

docs/conf.py

@@ -0,0 +1,58 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('..'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'osmopysim-usermanual'
+copyright = '2009-2023 by Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta'
+author = 'Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+        "sphinx.ext.autodoc",
+        "sphinxarg.ext",
+        "sphinx.ext.autosectionlabel",
+        "sphinx.ext.napoleon"
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+autoclass_content = 'both'

docs/index.rst

@@ -0,0 +1,53 @@
+.. pysim documentation master file
+
+Welcome to Osmocom pySim
+========================
+
+Introduction
+------------
+
+pySim is a python implementation of various software that helps you with
+managing subscriber identity cards for cellular networks, so-called SIM
+cards.
+
+Many Osmocom (Open Source Mobile Communications) projects relate to operating
+private / custom cellular networks, and provisioning SIM cards for said networks
+is in many cases a requirement to operate such networks.
+
+To make use of most of pySim's features, you will need a `programmable` SIM card,
+i.e. a card where you are the owner/operator and have sufficient credentials (such
+as the `ADM PIN`) in order to write to many if not most of the files on the card.
+
+Such cards are, for example, available from sysmocom, a major contributor to pySim.
+See https://www.sysmocom.de/products/lab/sysmousim/ for more details.
+
+pySim supports classic GSM SIM cards as well as ETSI UICC with 3GPP USIM and ISIM
+applications.  It is easily extensible, so support for additional files, card
+applications, etc. can be added easily by any python developer.  We do encourage you
+to submit your contributions to help this collaborative development project.
+
+pySim consists of several parts:
+
+* a python :ref:`library<pySim library>` containing plenty of objects and methods that can be used for
+  writing custom programs interfacing with SIM cards.
+* the [new] :ref:`interactive pySim-shell command line program<pySim-shell>`
+* the [new] :ref:`pySim-trace APDU trace decoder<pySim-trace>`
+* the [legacy] :ref:`pySim-prog and pySim-read tools<Legacy tools>`
+
+.. toctree::
+   :maxdepth: 3
+   :caption: Contents:
+
+   shell
+   trace
+   legacy
+   library
+   osmo-smdpp
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/legacy.rst

@@ -0,0 +1,101 @@
+Legacy tools
+============
+
+*legacy tools* are the classic ``pySim-prog`` and ``pySim-read`` programs that
+existed long before ``pySim-shell``.
+
+These days, you should primarily use ``pySim-shell`` instead of these
+legacy tools.
+
+pySim-prog
+----------
+
+``pySim-prog`` was the first part of the pySim software suite.  It started as
+a tool to write ICCID, IMSI, MSISDN and Ki to very simplistic SIM cards, and
+was later extended to a variety of other cards.  As the number of features supported
+became no longer bearable to express with command-line arguments, `pySim-shell` was
+created.
+
+Basic use cases can still use `pySim-prog`.
+
+Program customizable SIMs
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Two modes are possible:
+
+  - one where you specify every parameter manually :
+
+``./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -i <IMSI> -s <ICCID>``
+
+
+  - one where they are generated from some minimal set :
+
+``./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -z <random_string_of_choice> -j <card_num>``
+
+    With <random_string_of_choice> and <card_num>, the soft will generate
+    'predictable' IMSI and ICCID, so make sure you choose them so as not to
+    conflict with anyone. (for eg. your name as <random_string_of_choice> and
+    0 1 2 ... for <card num>).
+
+  You also need to enter some parameters to select the device :
+   -t TYPE : type of card (supersim, magicsim, fakemagicsim or try 'auto')
+   -d DEV  : Serial port device (default /dev/ttyUSB0)
+   -b BAUD : Baudrate (default 9600)
+
+
+pySim-read
+----------
+
+``pySim-read`` allows you to read some data from a SIM card.  It will only some files
+of the card, and will only read files accessible to a normal user (without any special authentication)
+
+These days, you should use the ``export`` command of ``pySim-shell``
+instead. It performs a much more comprehensive export of all of the
+[standard] files that can be found on the card.  To get a human-readable
+decode instead of the raw hex export, you can use ``export --json``.
+
+Specifically, pySim-read will dump the following:
+
+* MF
+
+ * EF.ICCID
+
+* DF.GSM
+
+ * EF,IMSI
+ * EF.GID1
+ * EF.GID2
+ * EF.SMSP
+ * EF.SPN
+ * EF.PLMNsel
+ * EF.PLMNwAcT
+ * EF.OPLMNwAcT
+ * EF.HPLMNAcT
+ * EF.ACC
+ * EF.MSISDN
+ * EF.AD
+ * EF.SST
+
+* ADF.USIM
+
+ * EF.EHPLMN
+ * EF.UST
+ * EF.ePDGId
+ * EF.ePDGSelection
+
+* ADF.ISIM
+
+ * EF.PCSCF
+ * EF.DOMAIN
+ * EF.IMPI
+ * EF.IMPU
+ * EF.UICCIARI
+ * EF.IST
+
+
+pySim-read usage
+~~~~~~~~~~~~~~~~
+
+.. argparse::
+   :module: pySim-read
+   :func: option_parser
+   :prog: pySim-read.py

docs/library.rst

@@ -0,0 +1,111 @@
+pySim library
+=============
+
+pySim filesystem abstraction
+----------------------------
+
+.. automodule:: pySim.filesystem
+   :members:
+
+pySim commands abstraction
+--------------------------
+
+.. automodule:: pySim.commands
+   :members:
+
+pySim Transport
+---------------
+
+The pySim.transport classes implement specific ways how to
+communicate with a SIM card.  A "transport" provides ways
+to transceive APDUs with the card.
+
+The most commonly used transport uses the PC/SC interface to
+utilize a variety of smart card interfaces ("readers").
+
+Transport base class
+~~~~~~~~~~~~~~~~~~~~
+
+.. automodule:: pySim.transport
+   :members:
+
+
+calypso / OsmocomBB transport
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This allows the use of the SIM slot of an OsmocomBB compatible phone with the TI Calypso chipset,
+using the L1CTL interface to talk to the layer1.bin firmware on the phone.
+
+.. automodule:: pySim.transport.calypso
+   :members:
+
+
+AT-command Modem transport
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This transport uses AT commands of a cellular modem in order to get access to the SIM card inserted
+in such a modem.
+
+.. automodule:: pySim.transport.modem_atcmd
+   :members:
+
+
+PC/SC transport
+~~~~~~~~~~~~~~~
+
+PC/SC is the standard API for accessing smart card interfaces
+on all major operating systems, including the MS Windows Family,
+OS X as well as Linux / Unix OSs.
+
+.. automodule:: pySim.transport.pcsc
+   :members:
+
+
+Serial/UART transport
+~~~~~~~~~~~~~~~~~~~~~
+
+This transport implements interfacing smart cards via
+very simplistic UART readers.  These readers basically
+wire together the Rx+Tx pins of a RS232 UART, provide
+a fixed crystal oscillator for clock, and operate the UART
+at 9600 bps.  These readers are sometimes called `Phoenix`.
+
+.. automodule:: pySim.transport.serial
+   :members:
+
+
+pySim construct utilities
+-------------------------
+
+.. automodule:: pySim.construct
+   :members:
+
+pySim TLV utilities
+-------------------
+
+.. automodule:: pySim.tlv
+   :members:
+
+pySim utility functions
+-----------------------
+
+.. automodule:: pySim.utils
+   :members:
+
+pySim exceptions
+----------------
+
+.. automodule:: pySim.exceptions
+   :members:
+
+pySim card_handler
+------------------
+
+.. automodule:: pySim.card_handler
+   :members:
+
+pySim card_key_provider
+-----------------------
+
+.. automodule:: pySim.card_key_provider
+   :members:

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/osmo-smdpp.rst

@@ -0,0 +1,114 @@
+osmo-smdpp
+==========
+
+`osmo-smdpp` is a proof-of-concept implementation of a minimal **SM-DP+** as specified for the *GSMA
+Consumer eSIM Remote SIM provisioning*.
+
+At least at this point, it is intended to be used for research and development, and not as a
+production SM-DP+.
+
+Unless you are a GSMA SAS-SM accredited SM-DP+ operator and have related DPtls, DPauth and DPpb
+certificates signed by the GSMA CI, you **can not use osmo-smdpp with regular production eUICC**.
+This is due to how the GSMA eSIM security architecture works.  You can, however, use osmo-smdpp with
+so-called *test-eUICC*, which contain certificates/keys signed by GSMA test certificates as laid out
+in GSMA SGP.26.
+
+At this point, osmo-smdpp does not support anything beyond the bare minimum required to download
+eSIM profiles to an eUICC.  Specifically, there is no ES2+ interface, and there is no built-in
+support for profile personalization yet.
+
+osmo-smdpp currently
+
+* uses test certificates copied from GSMA SGP.26 into `./smdpp-data/certs`, assuming that your osmo-smdppp
+  would be running at the host name `testsmdpplus1.example.com`
+* doesn't understand profile state. Any profile can always be downloaded any number of times, irrespective
+  of the EID or whether it was donwloaded before
+* doesn't perform any personalization, so the IMSI/ICCID etc. are always identical
+* **is absolutely insecure**, as it
+
+ * does not perform any certificate verification
+ * does not evaluate/consider any *Matching ID* or *Confirmation Code*
+ * stores the sessions in an unencrypted _python shelve_ and is hence leaking one-time key materials
+   used for profile encryption and signing.
+
+
+Running osmo-smdpp
+------------------
+
+osmo-smdpp does not have built-in TLS support as the used *twisted* framework appears to have
+problems when using the example elliptic curve certificates (both NIST and Brainpool) from GSMA.
+
+So in order to use it, you have to put it behind a TLS reverse proxy, which terminates the ES9+
+HTTPS from the LPA, and then forwards it as plain HTTP to osmo-smdpp.
+
+nginx as TLS proxy
+~~~~~~~~~~~~~~~~~~
+
+If you use `nginx` as web server, you can use the following configuration snippet::
+
+  upstream smdpp {
+          server localhost:8000;
+  }
+
+  server {
+          listen 443 ssl;
+          server_name testsmdpplus1.example.com;
+
+          ssl_certificate /my/path/to/pysim/smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_NIST.pem;
+          ssl_certificate_key /my/path/to/pysim/smdpp-data/certs/DPtls/SK_S_SM_DP_TLS_NIST.pem;
+
+          location / {
+                  proxy_read_timeout 600s;
+
+                  proxy_hide_header X-Powered-By;
+                  proxy_set_header X-Real-IP $remote_addr;
+                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                  proxy_set_header X-Forwarded-Proto https;
+                  proxy_set_header X-Forwarded-Port $proxy_port;
+                  proxy_set_header Host $host;
+
+                  proxy_pass http://smdpp/;
+          }
+  }
+
+You can of course achieve a similar functionality with apache, lighttpd or many other web server
+software.
+
+
+osmo-smdpp
+~~~~~~~~~~
+
+osmo-smdpp currently doesn't have any configuration file or command line options.  You just run it,
+and it will bind its plain-HTTP ES9+ interface to local TCP port 8000.
+
+The `smdpp-data/certs`` directory contains the DPtls, DPauth and DPpb as well as CI certificates
+used; they are copied from GSMA SGP.26 v2.
+
+The `smdpp-data/upp` directory contains the UPP (Unprotected Profile Package) used.  The file names (without
+.der suffix) are looked up by the matchingID parameter from the activation code presented by the LPA.
+
+
+DNS setup for your LPA
+~~~~~~~~~~~~~~~~~~~~~~
+
+The LPA must resolve `testsmdpplus1.example.com` to the IP address of your TLS proxy.
+
+It must also accept the TLS certificates used by your TLS proxy.
+
+Supported eUICC
+~~~~~~~~~~~~~~~
+
+If you run osmo-smdpp with the included SGP.26 certificates, you must use an eUICC with matching SGP.26
+certificates, i.e. the EUM certificate must be signed by a SGP.26 test root CA and the eUICC certificate
+in turn must be signed by that SGP.26 EUM certificate.
+
+sysmocom (sponsoring development and maintenance of pySim and osmo-smdpp) is selling SGP.26 test eUICC
+as `sysmoEUICC1-C2T`.  They are publicly sold in the `sysmocom webshop <https://shop.sysmocom.de/eUICC-for-consumer-eSIM-RSP-with-SGP.26-Test-Certificates/sysmoEUICC1-C2T>`_.
+
+In general you can use osmo-smdpp also with certificates signed by any other certificate authority.  You
+just always must ensure that the certificates of the SM-DP+ are signed by the same root CA as those of your
+eUICCs.
+
+Hypothetically, osmo-smdpp could also be operated with GSMA production certificates, but it would require
+that somebody brings the code in-line with all the GSMA security requirements (HSM support, ...) and operate
+it in a GSMA SAS-SM accredited environment and pays for the related audits.

docs/suci-tutorial.rst

@@ -0,0 +1,195 @@
+
+Guide: Enabling 5G SUCI
+========================
+
+SUPI/SUCI Concealment is a feature of 5G-Standalone (SA) to encrypt the
+IMSI/SUPI with a network operator public key.  3GPP Specifies two different
+variants for this:
+
+* SUCI calculation *in the UE*, using data from the SIM
+* SUCI calculation *on the card itself*
+
+pySIM supports writing the 5G-specific files for *SUCI calculation in the UE* on USIM cards, assuming that
+your cards contain the required files, and you have the privileges/credentials to write to them.  This is
+the case using sysmocom sysmoISIM-SJA2 cards (or successor products).
+
+In short, you can enable SUCI with these steps:
+
+* activate USIM **Service 124**
+* make sure USIM **Service 125** is disabled
+* store the public keys in **SUCI_Calc_Info**
+* set the **Routing Indicator** (required)
+
+If you want to disable the feature, you can just disable USIM Service 124 (and 125).
+
+Technical References
+~~~~~~~~~~~~~~~~~~~~
+
+This guide covers the basic workflow of provisioning SIM cards with the 5G SUCI feature. For detailed information on the SUCI feature and file contents, the following documents are helpful:
+
+* USIM files and structure: `TS 31.102 <https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/16.06.00_60/ts_131102v160600p.pdf>`__
+* USIM tests (incl. file content examples) `TS 31.121 <https://www.etsi.org/deliver/etsi_ts/131100_131199/131121/16.01.00_60/ts_131121v160100p.pdf>`__
+
+For specific information on sysmocom SIM cards, refer to Section 9.1 of the `sysmoUSIM User
+Manual <https://www.sysmocom.de/manuals/sysmousim-manual.pdf>`__.
+
+--------------
+
+Admin PIN
+---------
+
+The usual way to authenticate yourself to the card as the cellular
+operator is to validate the so-called ADM1 (admin) PIN.  This may differ
+from card model/vendor to card model/vendor.
+
+Start pySIM-shell and enter the admin PIN for your card. If you bought
+the SIM card from your network operator and don’t have the admin PIN,
+you cannot change SIM contents!
+
+Launch pySIM:
+
+::
+
+    $ ./pySim-shell.py -p 0
+
+    Using PC/SC reader interface
+    Autodetected card type: sysmoISIM-SJA2
+    Welcome to pySim-shell!
+    pySIM-shell (00:MF)>
+
+Enter the ADM PIN:
+
+::
+
+   pySIM-shell (00:MF)> verify_adm XXXXXXXX
+
+Otherwise, write commands will fail with ``SW Mismatch: Expected 9000 and got 6982.``
+
+Key Provisioning
+----------------
+
+::
+
+   pySIM-shell (00:MF)> select MF
+   pySIM-shell (00:MF)> select ADF.USIM
+   pySIM-shell (00:MF/ADF.USIM)> select DF.5GS
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS)> select EF.SUCI_Calc_Info
+
+By default, the file is present but empty:
+
+::
+
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.SUCI_Calc_Info)> read_binary_decoded
+   missing Protection Scheme Identifier List data object tag
+   9000: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff -> {}
+
+The following JSON config defines the testfile from `TS 31.121 <https://www.etsi.org/deliver/etsi_ts/131100_131199/131121/16.01.00_60/ts_131121v160100p.pdf>`__ Section 4.9.4 with
+test keys from `TS 33.501 <hhttps://www.etsi.org/deliver/etsi_ts/133500_133599/133501/16.05.00_60/ts_133501v160500p.pdf>`__ Annex C.4. Highest priority (``0``) has a
+Profile-B (``identifier: 2``) key in key slot ``1``, which means the key
+with ``hnet_pubkey_identifier: 27``.
+
+.. code:: json
+
+   {
+        "prot_scheme_id_list": [
+           {"priority": 0, "identifier": 2, "key_index": 1},
+           {"priority": 1, "identifier": 1, "key_index": 2},
+           {"priority": 2, "identifier": 0, "key_index": 0}],
+        "hnet_pubkey_list": [
+           {"hnet_pubkey_identifier": 27,
+            "hnet_pubkey": "0272DA71976234CE833A6907425867B82E074D44EF907DFB4B3E21C1C2256EBCD1"},
+           {"hnet_pubkey_identifier": 30,
+            "hnet_pubkey": "5A8D38864820197C3394B92613B20B91633CBD897119273BF8E4A6F4EEC0A650"}]
+   }
+
+Write the config to file (must be single-line input as for now):
+
+::
+
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.SUCI_Calc_Info)> update_binary_decoded '{ "prot_scheme_id_list": [ {"priority": 0, "identifier": 2, "key_index": 1}, {"priority": 1, "identifier": 1, "key_index": 2}, {"priority": 2, "identifier": 0, "key_index": 0}], "hnet_pubkey_list": [ {"hnet_pubkey_identifier": 27, "hnet_pubkey": "0272DA71976234CE833A6907425867B82E074D44EF907DFB4B3E21C1C2256EBCD1"}, {"hnet_pubkey_identifier": 30, "hnet_pubkey": "5A8D38864820197C3394B92613B20B91633CBD897119273BF8E4A6F4EEC0A650"}]}'
+
+WARNING: These are TEST KEYS with publicly known/specified private keys, and hence unsafe for live/secure
+deployments! For use in production networks, you need to generate your own set[s] of keys.
+
+Routing Indicator
+-----------------
+
+The Routing Indicator must be present for the SUCI feature. By default,
+the contents of the file is **invalid** (ffffffff):
+
+::
+
+   pySIM-shell (00:MF)> select MF
+   pySIM-shell (00:MF)> select ADF.USIM
+   pySIM-shell (00:MF/ADF.USIM)> select DF.5GS
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS)> select EF.Routing_Indicator
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.Routing_Indicator)> read_binary_decoded
+   9000: ffffffff -> {'raw': 'ffffffff'}
+
+The Routing Indicator is a four-byte file but the actual Routing
+Indicator goes into bytes 0 and 1 (the other bytes are reserved). To set
+the Routing Indicator to 0x71:
+
+::
+
+   pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.Routing_Indicator)> update_binary 17ffffff
+
+You can also set the routing indicator to **0x0**, which is *valid* and
+means “routing indicator not specified”, leaving it to the modem.
+
+USIM Service Table
+------------------
+
+First, check out the USIM Service Table (UST):
+
+::
+
+   pySIM-shell (00:MF)> select MF
+   pySIM-shell (00:MF)> select ADF.USIM
+   pySIM-shell (00:MF/ADF.USIM)> select EF.UST
+   pySIM-shell (00:MF/ADF.USIM/EF.UST)> read_binary_decoded
+   9000: beff9f9de73e0408400170730000002e00000000 -> [2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 25, 27, 28, 29, 33, 34, 35, 38, 39, 42, 43, 44, 45, 46, 51, 60, 71, 73, 85, 86, 87, 89, 90, 93, 94, 95, 122, 123, 124, 126]
+
+.. list-table:: From TS31.102
+   :widths: 15 40
+   :header-rows: 1
+
+   * - Service No.
+     - Description
+   * - 122
+     - 5GS Mobility Management Information
+   * - 123
+     - 5G Security Parameters
+   * - 124
+     - Subscription identifier privacy support
+   * - 125
+     - SUCI calculation by the USIM
+   * - 126
+     - UAC Access Identities support
+   * - 129
+     - 5GS Operator PLMN List
+
+If you’d like to enable/disable any UST service:
+
+::
+
+   pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_deactivate 124
+   pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_activate 124
+   pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_deactivate 125
+
+In this case, UST Service 124 is already enabled and you’re good to go. The
+sysmoISIM-SJA2 does not support on-SIM calculation, so service 125 must
+be disabled.
+
+USIM Error with 5G and sysmoISIM
+--------------------------------
+
+sysmoISIMs come 5GS-enabled. By default however, the configuration stored
+in the card file-system is **not valid** for 5G networks: Service 124 is enabled,
+but EF.SUCI_Calc_Info and EF.Routing_Indicator are empty files (hence
+do not contain valid data).
+
+At least for Qualcomm’s X55 modem, this results in an USIM error and the
+whole modem shutting 5G down. If you don’t need SUCI concealment but the
+smartphone refuses to connect to any 5G network, try to disable the UST
+service 124.

docs/trace.rst

@@ -0,0 +1,64 @@
+pySim-trace
+===========
+
+pySim-trace is a utility for high-level decode of APDU protocol traces such as those obtained with
+`Osmocom SIMtrace2 <https://osmocom.org/projects/simtrace2/wiki>`_ or `osmo-qcdiag <https://osmocom.org/projects/osmo-qcdiag/wiki>`_.
+
+pySim-trace leverages the existing knowledge of pySim-shell on anything related to SIM cards,
+including the structure/encoding of the various files on SIM/USIM/ISIM/HPSIM cards, and applies this
+to decoding protocol traces.  This means that it shows not only the name of the command (like READ
+BINARY), but actually understands what the currently selected file is, and how to decode the
+contents of that file.
+
+pySim-trace also understands the parameters passed to commands and how to decode them, for example
+of the AUTHENTICATE command within the USIM/ISIM/HPSIM application.
+
+
+Demo
+----
+
+To get an idea how pySim-trace usage looks like, you can watch the relevant part of the 11/2022
+SIMtrace2 tutorial whose `recording is freely accessible <https://media.ccc.de/v/osmodevcall-20221019-laforge-simtrace2-tutorial#t=2134>`_.
+
+
+Running pySim-trace
+-------------------
+
+Running pySim-trace requires you to specify the *source* of the to-be-decoded APDUs.  There are several
+supported options, each with their own respective parameters (like a file name for PCAP decoding).
+
+See the detailed command line reference below for details.
+
+A typical execution of pySim-trace for doing live decodes of *GSMTAP (SIM APDU)* e.g. from SIMtrace2 or
+osmo-qcdiag would look like this:
+
+::
+
+  ./pySim-trace.py gsmtap-udp
+
+This binds to the default UDP port 4729 (GSMTAP) on localhost (127.0.0.1), and decodes any APDUs received
+there.
+
+
+
+pySim-trace command line reference
+----------------------------------
+
+.. argparse::
+   :module: pySim-trace
+   :func: option_parser
+   :prog: pySim-trace.py
+
+
+Constraints
+-----------
+
+* In order to properly track the current location in the filesystem tree and other state, it is
+  important that the trace you're decoding includes all of the communication with the SIM, ideally
+  from the very start (power up).
+
+* pySim-trace currently only supports ETSI UICC (USIM/ISIM/HPSIM) and doesn't yet support legacy GSM
+  SIM.  This is not a fundamental technical constraint, it's just simply that nobody got around
+  developing and testing that part. Contributions are most welcome.
+
+

osmo-smdpp.py

@@ -0,0 +1,566 @@
+#!/usr/bin/env python3
+
+# Early proof-of-concept towards a SM-DP+ HTTP service for GSMA consumer eSIM RSP
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import json
+import sys
+import argparse
+import uuid
+import os
+import functools
+from typing import Optional, Dict, List
+from pprint import pprint as pp
+
+import base64
+from base64 import b64decode
+from klein import Klein
+from twisted.web.iweb import IRequest
+import asn1tools
+
+from pySim.utils import h2b, b2h, swap_nibbles
+
+import pySim.esim.rsp as rsp
+from pySim.esim import saip
+from pySim.esim.es8p import *
+from pySim.esim.x509_cert import oid, cert_policy_has_oid, cert_get_auth_key_id
+from pySim.esim.x509_cert import CertAndPrivkey, CertificateSet, cert_get_subject_key_id, VerifyError
+
+# HACK: make this configurable
+DATA_DIR = './smdpp-data'
+HOSTNAME = 'testsmdpplus1.example.com' # must match certificates!
+
+
+def b64encode2str(req: bytes) -> str:
+    """Encode given input bytes as base64 and return result as string."""
+    return base64.b64encode(req).decode('ascii')
+
+def set_headers(request: IRequest):
+    """Set the request headers as mandatory by GSMA eSIM RSP."""
+    request.setHeader('Content-Type', 'application/json;charset=UTF-8')
+    request.setHeader('X-Admin-Protocol', 'gsma/rsp/v2.1.0')
+
+def build_status_code(subject_code: str, reason_code: str, subject_id: Optional[str], message: Optional[str]) -> Dict:
+    r = {'subjectCode': subject_code, 'reasonCode': reason_code }
+    if subject_id:
+        r['subjectIdentifier'] = subject_id
+    if message:
+        r['message'] = message
+    return r
+
+def build_resp_header(js: dict, status: str = 'Executed-Success', status_code_data = None) -> None:
+    # SGP.22 v3.0 6.5.1.4
+    js['header'] = {
+        'functionExecutionStatus': {
+            'status': status,
+        }
+    }
+    if status_code_data:
+        js['header']['functionExecutionStatus']['statusCodeData'] = status_code_data
+
+from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature, encode_dss_signature
+from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat, PrivateFormat, NoEncryption
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.hazmat.primitives import hashes
+from cryptography.exceptions import InvalidSignature
+from cryptography import x509
+
+def ecdsa_tr03111_to_dss(sig: bytes) -> bytes:
+    """convert an ECDSA signature from BSI TR-03111 format to DER: first get long integers; then encode those."""
+    assert len(sig) == 64
+    r = int.from_bytes(sig[0:32], 'big')
+    s = int.from_bytes(sig[32:32*2], 'big')
+    return encode_dss_signature(r, s)
+
+
+class ApiError(Exception):
+    def __init__(self, subject_code: str, reason_code: str, message: Optional[str] = None,
+                 subject_id: Optional[str] = None):
+        self.status_code = build_status_code(subject_code, reason_code, subject_id, message)
+
+    def encode(self) -> str:
+        """Encode the API Error into a responseHeader string."""
+        js = {}
+        build_resp_header(js, 'Failed', self.status_code)
+        return json.dumps(js)
+
+class SmDppHttpServer:
+    app = Klein()
+
+    @staticmethod
+    def load_certs_from_path(path: str) -> List[x509.Certificate]:
+        """Load all DER + PEM files from given directory path and return them as list of x509.Certificate
+        instances."""
+        certs = []
+        for dirpath, dirnames, filenames in os.walk(path):
+            for filename in filenames:
+                cert = None
+                if filename.endswith('.der'):
+                    with open(os.path.join(dirpath, filename), 'rb') as f:
+                        cert = x509.load_der_x509_certificate(f.read())
+                elif filename.endswith('.pem'):
+                    with open(os.path.join(dirpath, filename), 'rb') as f:
+                        cert = x509.load_pem_x509_certificate(f.read())
+                if cert:
+                    # verify it is a CI certificate (keyCertSign + i-rspRole-ci)
+                    if not cert_policy_has_oid(cert, oid.id_rspRole_ci):
+                        raise ValueError("alleged CI certificate %s doesn't have CI policy" % filename)
+                    certs.append(cert)
+        return certs
+
+    def ci_get_cert_for_pkid(self, ci_pkid: bytes) -> Optional[x509.Certificate]:
+        """Find CI certificate for given key identifier."""
+        for cert in self.ci_certs:
+            print("cert: %s" % cert)
+            subject_exts = list(filter(lambda x: isinstance(x.value, x509.SubjectKeyIdentifier), cert.extensions))
+            print(subject_exts)
+            subject_pkid = subject_exts[0].value
+            print(subject_pkid)
+            if subject_pkid and subject_pkid.key_identifier == ci_pkid:
+                return cert
+        return None
+
+    def __init__(self, server_hostname: str, ci_certs_path: str, use_brainpool: bool = False):
+        self.server_hostname = server_hostname
+        self.upp_dir = os.path.realpath(os.path.join(DATA_DIR, 'upp'))
+        self.ci_certs = self.load_certs_from_path(ci_certs_path)
+        # load DPauth cert + key
+        self.dp_auth = CertAndPrivkey(oid.id_rspRole_dp_auth_v2)
+        cert_dir = os.path.join(DATA_DIR, 'certs')
+        if use_brainpool:
+            self.dp_auth.cert_from_der_file(os.path.join(cert_dir, 'DPauth', 'CERT_S_SM_DPauth_ECDSA_BRP.der'))
+            self.dp_auth.privkey_from_pem_file(os.path.join(cert_dir, 'DPauth', 'SK_S_SM_DPauth_ECDSA_BRP.pem'))
+        else:
+            self.dp_auth.cert_from_der_file(os.path.join(cert_dir, 'DPauth', 'CERT_S_SM_DPauth_ECDSA_NIST.der'))
+            self.dp_auth.privkey_from_pem_file(os.path.join(cert_dir, 'DPauth', 'SK_S_SM_DPauth_ECDSA_NIST.pem'))
+        # load DPpb cert + key
+        self.dp_pb = CertAndPrivkey(oid.id_rspRole_dp_pb_v2)
+        if use_brainpool:
+            self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_BRP.der'))
+            self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_BRP.pem'))
+        else:
+            self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_NIST.der'))
+            self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_NIST.pem'))
+        self.rss = rsp.RspSessionStore(os.path.join(DATA_DIR, "sm-dp-sessions"))
+
+    @app.handle_errors(ApiError)
+    def handle_apierror(self, request: IRequest, failure):
+        request.setResponseCode(200)
+        pp(failure)
+        return failure.value.encode()
+
+    @staticmethod
+    def _ecdsa_verify(cert: x509.Certificate, signature: bytes, data: bytes) -> bool:
+        pubkey = cert.public_key()
+        dss_sig = ecdsa_tr03111_to_dss(signature)
+        try:
+            pubkey.verify(dss_sig, data, ec.ECDSA(hashes.SHA256()))
+            return True
+        except InvalidSignature:
+            return False
+
+    @staticmethod
+    def rsp_api_wrapper(func):
+        """Wrapper that can be used as decorator in order to perform common REST API endpoint entry/exit
+        functionality, such as JSON decoding/encoding and debug-printing."""
+        @functools.wraps(func)
+        def _api_wrapper(self, request: IRequest):
+            # TODO: evaluate User-Agent + X-Admin-Protocol header
+            # TODO: reject any non-JSON Content-type
+
+            content = json.loads(request.content.read())
+            print("Rx JSON: %s" % json.dumps(content))
+            set_headers(request)
+
+            output = func(self, request, content) or {}
+
+            build_resp_header(output)
+            print("Tx JSON: %s" % json.dumps(output))
+            return json.dumps(output)
+        return _api_wrapper
+
+    @app.route('/gsma/rsp2/es9plus/initiateAuthentication', methods=['POST'])
+    @rsp_api_wrapper
+    def initiateAutentication(self, request: IRequest, content: dict) -> dict:
+        """See ES9+ InitiateAuthentication SGP.22 Section 5.6.1"""
+        # Verify that the received address matches its own SM-DP+ address, where the comparison SHALL be
+        # case-insensitive. Otherwise, the SM-DP+ SHALL return a status code "SM-DP+ Address - Refused".
+        if content['smdpAddress'] != self.server_hostname:
+           raise ApiError('8.8.1', '3.8', 'Invalid SM-DP+ Address')
+
+        euiccChallenge = b64decode(content['euiccChallenge'])
+        if len(euiccChallenge) != 16:
+            raise ValueError
+
+        euiccInfo1_bin = b64decode(content['euiccInfo1'])
+        euiccInfo1 = rsp.asn1.decode('EUICCInfo1', euiccInfo1_bin)
+        print("Rx euiccInfo1: %s" % euiccInfo1)
+        #euiccInfo1['svn']
+
+        # TODO: If euiccCiPKIdListForSigningV3 is present ...
+
+        pkid_list = euiccInfo1['euiccCiPKIdListForSigning']
+        if 'euiccCiPKIdListForSigningV3' in euiccInfo1:
+            pkid_list = pkid_list + euiccInfo1['euiccCiPKIdListForSigningV3']
+        # verify it supports one of the keys indicated by euiccCiPKIdListForSigning
+        ci_cert = None
+        for x in pkid_list:
+            ci_cert = self.ci_get_cert_for_pkid(x)
+            # we already support multiple CI certificates but only one set of DPauth + DPpb keys. So we must
+            # make sure we choose a CI key-id which has issued both the eUICC as well as our own SM-DP side
+            # certs.
+            if ci_cert and cert_get_subject_key_id(ci_cert) == self.dp_auth.get_authority_key_identifier().key_identifier:
+                break
+            else:
+                ci_cert = None
+        if not ci_cert:
+           raise ApiError('8.8.2', '3.1', 'None of the proposed Public Key Identifiers is supported by the SM-DP+')
+
+        # TODO: Determine the set of CERT.DPauth.SIG that satisfy the following criteria:
+        # * Part of a certificate chain ending at one of the eSIM CA RootCA Certificate, whose Public Keys is
+        #   supported by the eUICC (indicated by euiccCiPKIdListForVerification).
+        # * Using a certificate chain that the eUICC and the LPA both support:
+        #euiccInfo1['euiccCiPKIdListForVerification']
+        #   raise ApiError('8.8.4', '3.7', 'The SM-DP+ has no CERT.DPauth.SIG which chains to one of the eSIM CA Root CA CErtificate with a Public Key supported by the eUICC')
+
+        # Generate a TransactionID which is used to identify the ongoing RSP session. The TransactionID
+        # SHALL be unique within the scope and lifetime of each SM-DP+.
+        transactionId = uuid.uuid4().hex
+        assert not transactionId in self.rss
+
+        # Generate a serverChallenge for eUICC authentication attached to the ongoing RSP session.
+        serverChallenge = os.urandom(16)
+
+        # Generate a serverSigned1 data object as expected by the eUICC and described in section 5.7.13 "ES10b.AuthenticateServer". If and only if both eUICC and LPA indicate crlStaplingV3Support, the SM-DP+ SHALL indicate crlStaplingV3Used in sessionContext.
+        serverSigned1 = {
+            'transactionId': h2b(transactionId),
+            'euiccChallenge': euiccChallenge,
+            'serverAddress': self.server_hostname,
+            'serverChallenge': serverChallenge,
+            }
+        print("Tx serverSigned1: %s" % serverSigned1)
+        serverSigned1_bin = rsp.asn1.encode('ServerSigned1', serverSigned1)
+        print("Tx serverSigned1: %s" % rsp.asn1.decode('ServerSigned1', serverSigned1_bin))
+        output = {}
+        output['serverSigned1'] = b64encode2str(serverSigned1_bin)
+
+        # Generate a signature (serverSignature1) as described in section 5.7.13 "ES10b.AuthenticateServer" using the SK related to the selected CERT.DPauth.SIG.
+        # serverSignature1 SHALL be created using the private key associated to the RSP Server Certificate for authentication, and verified by the eUICC using the contained public key as described in section 2.6.9. serverSignature1 SHALL apply on serverSigned1 data object.
+        output['serverSignature1'] = b64encode2str(b'\x5f\x37\x40' + self.dp_auth.ecdsa_sign(serverSigned1_bin))
+
+        output['transactionId'] = transactionId
+        server_cert_aki = self.dp_auth.get_authority_key_identifier()
+        output['euiccCiPKIdToBeUsed'] = b64encode2str(b'\x04\x14' + server_cert_aki.key_identifier)
+        output['serverCertificate'] = b64encode2str(self.dp_auth.get_cert_as_der()) # CERT.DPauth.SIG
+        # FIXME: add those certificate
+        #output['otherCertsInChain'] = b64encode2str()
+
+        # create SessionState and store it in rss
+        self.rss[transactionId] = rsp.RspSessionState(transactionId, serverChallenge,
+                                                      cert_get_subject_key_id(ci_cert))
+
+        return output
+
+    @app.route('/gsma/rsp2/es9plus/authenticateClient', methods=['POST'])
+    @rsp_api_wrapper
+    def authenticateClient(self, request: IRequest, content: dict) -> dict:
+        """See ES9+ AuthenticateClient in SGP.22 Section 5.6.3"""
+        transactionId = content['transactionId']
+
+        authenticateServerResp_bin = b64decode(content['authenticateServerResponse'])
+        authenticateServerResp = rsp.asn1.decode('AuthenticateServerResponse', authenticateServerResp_bin)
+        print("Rx %s: %s" % authenticateServerResp)
+        if authenticateServerResp[0] == 'authenticateResponseError':
+            r_err = authenticateServerResp[1]
+            #r_err['transactionId']
+            #r_err['authenticateErrorCode']
+            raise ValueError("authenticateResponseError %s" % r_err)
+
+        r_ok = authenticateServerResp[1]
+        euiccSigned1 = r_ok['euiccSigned1']
+        # TODO: use original data, don't re-encode?
+        euiccSigned1_bin = rsp.asn1.encode('EuiccSigned1', euiccSigned1)
+        euiccSignature1_bin = r_ok['euiccSignature1']
+        euiccCertificate_dec = r_ok['euiccCertificate']
+        # TODO: use original data, don't re-encode?
+        euiccCertificate_bin = rsp.asn1.encode('Certificate', euiccCertificate_dec)
+        eumCertificate_dec = r_ok['eumCertificate']
+        eumCertificate_bin = rsp.asn1.encode('Certificate', eumCertificate_dec)
+        # TODO v3: otherCertsInChain
+
+        # load certificate
+        euicc_cert = x509.load_der_x509_certificate(euiccCertificate_bin)
+        eum_cert = x509.load_der_x509_certificate(eumCertificate_bin)
+
+        # Verify that the transactionId is known and relates to an ongoing RSP session.  Otherwise, the SM-DP+
+        # SHALL return a status code "TransactionId - Unknown"
+        ss = self.rss.get(transactionId, None)
+        if ss is None:
+            raise ApiError('8.10.1', '3.9', 'Unknown')
+        ss.euicc_cert = euicc_cert
+        ss.eum_cert = eum_cert # TODO: do we need this in the state?
+
+        # Verify that the Root Certificate of the eUICC certificate chain corresponds to the
+        # euiccCiPKIdToBeUsed or TODO: euiccCiPKIdToBeUsedV3
+        if cert_get_auth_key_id(eum_cert) != ss.ci_cert_id:
+            raise ApiError('8.11.1', '3.9', 'Unknown')
+
+        # Verify the validity of the eUICC certificate chain
+        cs = CertificateSet(self.ci_get_cert_for_pkid(ss.ci_cert_id))
+        cs.add_intermediate_cert(eum_cert)
+        # TODO v3: otherCertsInChain
+        try:
+            cs.verify_cert_chain(euicc_cert)
+        except VerifyError:
+            raise ApiError('8.1.3', '6.1', 'Verification failed')
+        #   raise ApiError('8.1.3', '6.3', 'Expired')
+
+
+        # Verify euiccSignature1 over euiccSigned1 using pubkey from euiccCertificate.
+        # Otherwise, the SM-DP+ SHALL return a status code "eUICC - Verification failed"
+        if not self._ecdsa_verify(euicc_cert, euiccSignature1_bin, euiccSigned1_bin):
+            raise ApiError('8.1', '6.1', 'Verification failed')
+
+        # TODO: verify EID of eUICC cert is  within permitted range of EUM cert
+
+        ss.eid = ss.euicc_cert.subject.get_attributes_for_oid(x509.oid.NameOID.SERIAL_NUMBER)[0].value
+        print("EID (from eUICC cert): %s" % ss.eid)
+
+        # Verify that the serverChallenge attached to the ongoing RSP session matches the
+        # serverChallenge returned by the eUICC. Otherwise, the SM-DP+ SHALL return a status code "eUICC -
+        # Verification failed".
+        if euiccSigned1['serverChallenge'] != ss.serverChallenge:
+            raise ApiError('8.1', '6.1', 'Verification failed')
+
+        # If ctxParams1 contains a ctxParamsForCommonAuthentication data object, the SM-DP+ Shall [...]
+        # TODO: We really do a very simplistic job here, this needs to be properly implemented later,
+        # considering all the various cases, profile state, etc.
+        if euiccSigned1['ctxParams1'][0] == 'ctxParamsForCommonAuthentication':
+            cpca = euiccSigned1['ctxParams1'][1]
+            matchingId = cpca.get('matchingId', None)
+            if not matchingId:
+                # TODO: check if any pending profile downloads for the EID
+                raise ApiError('8.2.6', '3.8', 'Refused')
+            if matchingId:
+                # look up profile based on matchingID.  We simply check if a given file exists for now..
+                path = os.path.join(self.upp_dir, matchingId) + '.der'
+                # prevent directory traversal attack
+                if os.path.commonprefix((os.path.realpath(path),self.upp_dir)) != self.upp_dir:
+                    raise ApiError('8.2.6', '3.8', 'Refused')
+                if not os.path.isfile(path) or not os.access(path, os.R_OK):
+                    raise ApiError('8.2.6', '3.8', 'Refused')
+                ss.matchingId = matchingId
+                with open(path, 'rb') as f:
+                    pes = saip.ProfileElementSequence.from_der(f.read())
+                    iccid_str = b2h(pes.get_pe_for_type('header').decoded['iccid'])
+
+        # FIXME: we actually want to perform the profile binding herr, and read the profile metadat from the profile
+
+        # Put together profileMetadata + _bin
+        ss.profileMetadata = ProfileMetadata(iccid_bin=h2b(swap_nibbles(iccid_str)), spn="OsmocomSPN", profile_name=matchingId)
+        profileMetadata_bin = ss.profileMetadata.gen_store_metadata_request()
+
+        # Put together smdpSigned2 + _bin
+        smdpSigned2 = {
+            'transactionId': h2b(ss.transactionId),
+            'ccRequiredFlag': False,        # whether the Confirmation Code is required
+            #'bppEuiccOtpk': None,           # whether otPK.EUICC.ECKA already used for binding the BPP, tag '5F49'
+            }
+        smdpSigned2_bin = rsp.asn1.encode('SmdpSigned2', smdpSigned2)
+
+        ss.smdpSignature2_do = b'\x5f\x37\x40' + self.dp_pb.ecdsa_sign(smdpSigned2_bin + b'\x5f\x37\x40' + euiccSignature1_bin)
+
+        # update non-volatile state with updated ss object
+        self.rss[transactionId] = ss
+        return {
+            'transactionId': transactionId,
+            'profileMetadata': b64encode2str(profileMetadata_bin),
+            'smdpSigned2': b64encode2str(smdpSigned2_bin),
+            'smdpSignature2': b64encode2str(ss.smdpSignature2_do),
+            'smdpCertificate': b64encode2str(self.dp_pb.get_cert_as_der()), # CERT.DPpb.SIG
+        }
+
+    @app.route('/gsma/rsp2/es9plus/getBoundProfilePackage', methods=['POST'])
+    @rsp_api_wrapper
+    def getBoundProfilePackage(self, request: IRequest, content: dict) -> dict:
+        """See ES9+ GetBoundProfilePackage SGP.22 Section 5.6.2"""
+        transactionId = content['transactionId']
+
+        # Verify that the received transactionId is known and relates to an ongoing RSP session
+        ss = self.rss.get(transactionId, None)
+        if not ss:
+            raise ApiError('8.10.1', '3.9', 'The RSP session identified by the TransactionID is unknown')
+
+        prepDownloadResp_bin = b64decode(content['prepareDownloadResponse'])
+        prepDownloadResp = rsp.asn1.decode('PrepareDownloadResponse', prepDownloadResp_bin)
+        print("Rx %s: %s" % prepDownloadResp)
+
+        if prepDownloadResp[0] == 'downloadResponseError':
+            r_err = prepDownloadResp[1]
+            #r_err['transactionId']
+            #r_err['downloadErrorCode']
+            raise ValueError("downloadResponseError %s" % r_err)
+
+        r_ok = prepDownloadResp[1]
+
+        # Verify the euiccSignature2 computed over euiccSigned2 and smdpSignature2 using the PK.EUICC.SIG attached to the ongoing RSP session
+        euiccSigned2 = r_ok['euiccSigned2']
+        # TODO: use original data, don't re-encode?
+        euiccSigned2_bin = rsp.asn1.encode('EUICCSigned2', euiccSigned2)
+        if not self._ecdsa_verify(ss.euicc_cert, r_ok['euiccSignature2'], euiccSigned2_bin + ss.smdpSignature2_do):
+            raise ApiError('8.1', '6.1', 'eUICC signature is invalid')
+
+        # not in spec: Verify that signed TransactionID is outer transaction ID
+        if h2b(transactionId) != euiccSigned2['transactionId']:
+            raise ApiError('8.10.1', '3.9', 'The signed transactionId != outer transactionId')
+
+        # store otPK.EUICC.ECKA in session state
+        ss.euicc_otpk = euiccSigned2['euiccOtpk']
+        print("euiccOtpk: %s" % (b2h(ss.euicc_otpk)))
+
+        # Generate a one-time ECKA key pair (ot{PK,SK}.DP.ECKA) using the curve indicated by the Key Parameter
+        # Reference value of CERT.DPpb.ECDDSA
+        print("curve = %s" % self.dp_pb.get_curve())
+        ss.smdp_ot = ec.generate_private_key(self.dp_pb.get_curve())
+        # extract the public key in (hopefully) the right format for the ES8+ interface
+        ss.smdp_otpk = ss.smdp_ot.public_key().public_bytes(Encoding.X962, PublicFormat.UncompressedPoint)
+        print("smdpOtpk: %s" % b2h(ss.smdp_otpk))
+        print("smdpOtsk: %s" % b2h(ss.smdp_ot.private_bytes(Encoding.DER, PrivateFormat.PKCS8, NoEncryption())))
+
+        ss.host_id = b'mahlzeit'
+
+        # Generate Session Keys using the CRT, opPK.eUICC.ECKA and otSK.DP.ECKA according to annex G
+        euicc_public_key = ec.EllipticCurvePublicKey.from_encoded_point(ss.smdp_ot.curve, ss.euicc_otpk)
+        ss.shared_secret = ss.smdp_ot.exchange(ec.ECDH(), euicc_public_key)
+        print("shared_secret: %s" % b2h(ss.shared_secret))
+
+        # TODO: Check if this order requires a Confirmation Code verification
+
+        #  Perform actual protection + binding of profile package (or return  pre-bound one)
+        with open(os.path.join(self.upp_dir, ss.matchingId)+'.der', 'rb') as f:
+            upp = UnprotectedProfilePackage.from_der(f.read(), metadata=ss.profileMetadata)
+            # HACK: Use empty PPP as we're still debuggin the configureISDP step, and we want to avoid
+            # cluttering the log with stuff happening after the failure
+            #upp = UnprotectedProfilePackage.from_der(b'', metadata=ss.profileMetadata)
+        if False:
+            # Use random keys
+            bpp = BoundProfilePackage.from_upp(upp)
+        else:
+            # Use sesssion keys
+            ppp = ProtectedProfilePackage.from_upp(upp, BspInstance(b'\x00'*16, b'\x11'*16, b'\x22'*16))
+            bpp = BoundProfilePackage.from_ppp(ppp)
+
+        # update non-volatile state with updated ss object
+        self.rss[transactionId] = ss
+        return {
+            'transactionId': transactionId,
+            'boundProfilePackage': b64encode2str(bpp.encode(ss, self.dp_pb)),
+        }
+
+    @app.route('/gsma/rsp2/es9plus/handleNotification', methods=['POST'])
+    @rsp_api_wrapper
+    def handleNotification(self, request: IRequest, content: dict) -> dict:
+        """See ES9+ HandleNotification in SGP.22 Section 5.6.4"""
+        pendingNotification_bin = b64decode(content['pendingNotification'])
+        pendingNotification = rsp.asn1.decode('PendingNotification', pendingNotification_bin)
+        print("Rx %s: %s" % pendingNotification)
+        if pendingNotification[0] == 'profileInstallationResult':
+            profileInstallRes = pendingNotification[1]
+            pird = profileInstallRes['profileInstallationResultData']
+            transactionId = b2h(pird['transactionId'])
+            ss = self.rss.get(transactionId, None)
+            if ss is None:
+                print("Unable to find session for transactionId")
+                return
+            profileInstallRes['euiccSignPIR']
+            # TODO: use original data, don't re-encode?
+            pird_bin = rsp.asn1.encode('ProfileInstallationResultData', pird)
+            # verify eUICC signature
+            if not self._ecdsa_verify(ss.euicc_cert, profileInstallRes['euiccSignPIR'], pird_bin):
+                print("Unable to verify eUICC signature")
+            print("Profile Installation Final Result: ", pird['finalResult'])
+            # remove session state
+            del self.rss[transactionId]
+        elif pendingNotification[0] == 'otherSignedNotification':
+            # TODO
+            pass
+        else:
+            raise ValueError(pendingNotification)
+
+    #@app.route('/gsma/rsp3/es9plus/handleDeviceChangeRequest, methods=['POST']')
+    #@rsp_api_wrapper
+        #"""See ES9+ ConfirmDeviceChange in SGP.22 Section 5.6.6"""
+        # TODO: implement this
+
+    @app.route('/gsma/rsp2/es9plus/cancelSession', methods=['POST'])
+    @rsp_api_wrapper
+    def cancelSession(self, request: IRequest, content: dict) -> dict:
+        """See ES9+ CancelSession in SGP.22 Section 5.6.5"""
+        print("Rx JSON: %s" % content)
+        transactionId = content['transactionId']
+
+        # Verify that the received transactionId is known and relates to an ongoing RSP session
+        ss = self.rss.get(transactionId, None)
+        if ss is None:
+            raise ApiError('8.10.1', '3.9', 'The RSP session identified by the transactionId is unknown')
+
+        cancelSessionResponse_bin = b64decode(content['cancelSessionResponse'])
+        cancelSessionResponse = rsp.asn1.decode('CancelSessionResponse', cancelSessionResponse_bin)
+        print("Rx %s: %s" % cancelSessionResponse)
+
+        if cancelSessionResponse[0] == 'cancelSessionResponseError':
+            # FIXME: print some error
+            return
+        cancelSessionResponseOk = cancelSessionResponse[1]
+        # TODO: use original data, don't re-encode?
+        ecsr = cancelSessionResponseOk['euiccCancelSessionSigned']
+        ecsr_bin = rsp.asn1.encode('EuiccCancelSessionSigned', ecsr)
+        # Verify the eUICC signature (euiccCancelSessionSignature) using the PK.EUICC.SIG attached to the ongoing RSP session
+        if not self._ecdsa_verify(ss.euicc_cert, cancelSessionResponseOk['euiccCancelSessionSignature'], ecsr_bin):
+            raise ApiError('8.1', '6.1', 'eUICC signature is invalid')
+
+        # Verify that the received smdpOid corresponds to the one in SM-DP+ CERT.DPauth.SIG
+        subj_alt_name = self.dp_auth.get_subject_alt_name()
+        if x509.ObjectIdentifier(ecsr['smdpOid']) != subj_alt_name.oid:
+            raise ApiError('8.8', '3.10', 'The provided SM-DP+ OID is invalid.')
+
+        if ecsr['transactionId'] != h2b(transactionId):
+            raise ApiError('8.10.1', '3.9', 'The signed transactionId != outer transactionId')
+
+        # TODO: 1. Notify the Operator using the function "ES2+.HandleNotification" function
+        # TODO: 2. Terminate the corresponding pending download process.
+        # TODO: 3. If required, execute the SM-DS Event Deletion procedure described in section 3.6.3.
+
+        # delete actual session data
+        del self.rss[transactionId]
+        return { 'transactionId': transactionId }
+
+
+def main(argv):
+    parser = argparse.ArgumentParser()
+    #parser.add_argument("-H", "--host", help="Host/IP to bind HTTP to", default="localhost")
+    #parser.add_argument("-p", "--port", help="TCP port to bind HTTP to", default=8000)
+    #parser.add_argument("-v", "--verbose", help="increase output verbosity", action='count', default=0)
+
+    args = parser.parse_args()
+
+    hs = SmDppHttpServer(HOSTNAME, os.path.join(DATA_DIR, 'certs', 'CertificateIssuer'), use_brainpool=True)
+    #hs.app.run(endpoint_description="ssl:port=8000:dhParameters=dh_param_2048.pem")
+    hs.app.run("localhost", 8000)
+
+if __name__ == "__main__":
+    main(sys.argv)

pySim-read.py

@@ -0,0 +1,370 @@
+#!/usr/bin/env python3
+
+#
+# Utility to display some informations about a SIM card
+#
+#
+# Copyright (C) 2009  Sylvain Munaut <tnt@246tNt.com>
+# Copyright (C) 2010  Harald Welte <laforge@gnumonks.org>
+# Copyright (C) 2013  Alexander Chemeris <alexander.chemeris@gmail.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import hashlib
+import argparse
+import os
+import random
+import re
+import sys
+
+from pySim.ts_51_011 import EF_SST_map, EF_AD
+from pySim.legacy.ts_51_011 import EF, DF
+from pySim.ts_31_102 import EF_UST_map
+from pySim.legacy.ts_31_102 import EF_USIM_ADF_map
+from pySim.ts_31_103 import EF_IST_map
+from pySim.legacy.ts_31_103 import EF_ISIM_ADF_map
+
+from pySim.commands import SimCardCommands
+from pySim.transport import init_reader, argparse_add_reader_args
+from pySim.exceptions import SwMatchError
+from pySim.legacy.cards import card_detect, SimCard, UsimCard, IsimCard
+from pySim.utils import h2b, h2s, swap_nibbles, rpad, dec_imsi, dec_iccid, dec_msisdn
+from pySim.legacy.utils import format_xplmn_w_act, dec_st
+
+option_parser = argparse.ArgumentParser(description='Legacy tool for reading some parts of a SIM card',
+                                        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+argparse_add_reader_args(option_parser)
+
+
+def select_app(adf: str, card: SimCard):
+    """Select application by its AID"""
+    sw = 0
+    try:
+        if card._scc.cla_byte == "00":
+            data, sw = card.select_adf_by_aid(adf)
+    except SwMatchError as e:
+        if e.sw_actual == "6a82":
+            # If we can't select the file because it does not exist, we just remain silent since it means
+            # that this card just does not have an USIM application installed, which is not an error.
+            pass
+        else:
+            print("ADF." + adf + ": Can't select application -- " + str(e))
+    except Exception as e:
+        print("ADF." + adf + ": Can't select application -- " + str(e))
+
+    return sw
+
+
+if __name__ == '__main__':
+
+    # Parse options
+    opts = option_parser.parse_args()
+
+    # Init card reader driver
+    sl = init_reader(opts)
+
+    # Create command layer
+    scc = SimCardCommands(transport=sl)
+
+    # Wait for SIM card
+    sl.wait_for_card()
+
+    # Assuming UICC SIM
+    scc.cla_byte = "00"
+    scc.sel_ctrl = "0004"
+
+    # Testing for Classic SIM or UICC
+    (res, sw) = sl.send_apdu(scc.cla_byte + "a4" + scc.sel_ctrl + "02" + "3f00")
+    if sw == '6e00':
+        # Just a Classic SIM
+        scc.cla_byte = "a0"
+        scc.sel_ctrl = "0000"
+
+    # Read the card
+    print("Reading ...")
+
+    # Initialize Card object by auto detecting the card
+    card = card_detect("auto", scc) or SimCard(scc)
+
+    # Read all AIDs on the UICC
+    card.read_aids()
+
+    # EF.ICCID
+    (res, sw) = card.read_iccid()
+    if sw == '9000':
+        print("ICCID: %s" % (res,))
+    else:
+        print("ICCID: Can't read, response code = %s" % (sw,))
+
+    # EF.IMSI
+    (res, sw) = card.read_imsi()
+    if sw == '9000':
+        print("IMSI: %s" % (res,))
+    else:
+        print("IMSI: Can't read, response code = %s" % (sw,))
+
+    # EF.GID1
+    try:
+        (res, sw) = card.read_gid1()
+        if sw == '9000':
+            print("GID1: %s" % (res,))
+        else:
+            print("GID1: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("GID1: Can't read file -- %s" % (str(e),))
+
+    # EF.GID2
+    try:
+        (res, sw) = card.read_binary('GID2')
+        if sw == '9000':
+            print("GID2: %s" % (res,))
+        else:
+            print("GID2: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("GID2: Can't read file -- %s" % (str(e),))
+
+    # EF.SMSP
+    (res, sw) = card.read_record('SMSP', 1)
+    if sw == '9000':
+        print("SMSP: %s" % (res,))
+    else:
+        print("SMSP: Can't read, response code = %s" % (sw,))
+
+    # EF.SPN
+    try:
+        (res, sw) = card.read_spn()
+        if sw == '9000':
+            print("SPN: %s" % (res[0] or "Not available"))
+            print("Show in HPLMN: %s" % (res[1],))
+            print("Hide in OPLMN: %s" % (res[2],))
+        else:
+            print("SPN: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("SPN: Can't read file -- %s" % (str(e),))
+
+    # EF.PLMNsel
+    try:
+        (res, sw) = card.read_binary('PLMNsel')
+        if sw == '9000':
+            print("PLMNsel: %s" % (res))
+        else:
+            print("PLMNsel: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("PLMNsel: Can't read file -- " + str(e))
+
+    # EF.PLMNwAcT
+    try:
+        (res, sw) = card.read_plmn_act()
+        if sw == '9000':
+            print("PLMNwAcT:\n%s" % (res))
+        else:
+            print("PLMNwAcT: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("PLMNwAcT: Can't read file -- " + str(e))
+
+    # EF.OPLMNwAcT
+    try:
+        (res, sw) = card.read_oplmn_act()
+        if sw == '9000':
+            print("OPLMNwAcT:\n%s" % (res))
+        else:
+            print("OPLMNwAcT: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("OPLMNwAcT: Can't read file -- " + str(e))
+
+    # EF.HPLMNAcT
+    try:
+        (res, sw) = card.read_hplmn_act()
+        if sw == '9000':
+            print("HPLMNAcT:\n%s" % (res))
+        else:
+            print("HPLMNAcT: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("HPLMNAcT: Can't read file -- " + str(e))
+
+    # EF.ACC
+    (res, sw) = card.read_binary('ACC')
+    if sw == '9000':
+        print("ACC: %s" % (res,))
+    else:
+        print("ACC: Can't read, response code = %s" % (sw,))
+
+    # EF.MSISDN
+    try:
+        (res, sw) = card.read_msisdn()
+        if sw == '9000':
+            # (npi, ton, msisdn) = res
+            if res is not None:
+                print("MSISDN (NPI=%d ToN=%d): %s" % res)
+            else:
+                print("MSISDN: Not available")
+        else:
+            print("MSISDN: Can't read, response code = %s" % (sw,))
+    except Exception as e:
+        print("MSISDN: Can't read file -- " + str(e))
+
+    # EF.AD
+    (res, sw) = card.read_binary('AD')
+    if sw == '9000':
+        print("Administrative data: %s" % (res,))
+        ad = EF_AD()
+        decoded_data = ad.decode_hex(res)
+        print("\tMS operation mode: %s" % decoded_data['ms_operation_mode'])
+        if decoded_data['ofm']:
+            print("\tCiphering Indicator: enabled")
+        else:
+            print("\tCiphering Indicator: disabled")
+    else:
+        print("AD: Can't read, response code = %s" % (sw,))
+
+    # EF.SST
+    (res, sw) = card.read_binary('SST')
+    if sw == '9000':
+        print("SIM Service Table: %s" % res)
+        # Print those which are available
+        print("%s" % dec_st(res))
+    else:
+        print("SIM Service Table: Can't read, response code = %s" % (sw,))
+
+    # Check whether we have th AID of USIM, if so select it by its AID
+    # EF.UST - File Id in ADF USIM : 6f38
+    sw = select_app("USIM", card)
+    if sw == '9000':
+        # Select USIM profile
+        usim_card = UsimCard(scc)
+
+        # EF.EHPLMN
+        if usim_card.file_exists(EF_USIM_ADF_map['EHPLMN']):
+            (res, sw) = usim_card.read_ehplmn()
+            if sw == '9000':
+                print("EHPLMN:\n%s" % (res))
+            else:
+                print("EHPLMN: Can't read, response code = %s" % (sw,))
+
+        # EF.FPLMN
+        if usim_card.file_exists(EF_USIM_ADF_map['FPLMN']):
+            res, sw = usim_card.read_fplmn()
+            if sw == '9000':
+                print(f'FPLMN:\n{res}')
+            else:
+                print(f'FPLMN: Can\'t read, response code = {sw}')
+
+        # EF.UST
+        try:
+            if usim_card.file_exists(EF_USIM_ADF_map['UST']):
+                # res[0] - EF content of UST
+                # res[1] - Human readable format of services marked available in UST
+                (res, sw) = usim_card.read_ust()
+                if sw == '9000':
+                    print("USIM Service Table: %s" % res[0])
+                    print("%s" % res[1])
+                else:
+                    print("USIM Service Table: Can't read, response code = %s" % (sw,))
+        except Exception as e:
+            print("USIM Service Table: Can't read file -- " + str(e))
+
+        # EF.ePDGId - Home ePDG Identifier
+        try:
+            if usim_card.file_exists(EF_USIM_ADF_map['ePDGId']):
+                (res, sw) = usim_card.read_epdgid()
+                if sw == '9000':
+                    print("ePDGId:\n%s" %
+                          (len(res) and res or '\tNot available\n',))
+                else:
+                    print("ePDGId: Can't read, response code = %s" % (sw,))
+        except Exception as e:
+            print("ePDGId: Can't read file -- " + str(e))
+
+        # EF.ePDGSelection - ePDG Selection Information
+        try:
+            if usim_card.file_exists(EF_USIM_ADF_map['ePDGSelection']):
+                (res, sw) = usim_card.read_ePDGSelection()
+                if sw == '9000':
+                    print("ePDGSelection:\n%s" % (res,))
+                else:
+                    print("ePDGSelection: Can't read, response code = %s" % (sw,))
+        except Exception as e:
+            print("ePDGSelection: Can't read file -- " + str(e))
+
+    # Select ISIM application by its AID
+    sw = select_app("ISIM", card)
+    if sw == '9000':
+        # Select USIM profile
+        isim_card = IsimCard(scc)
+
+        # EF.P-CSCF - P-CSCF Address
+        try:
+            if isim_card.file_exists(EF_ISIM_ADF_map['PCSCF']):
+                res = isim_card.read_pcscf()
+                print("P-CSCF:\n%s" %
+                      (len(res) and res or '\tNot available\n',))
+        except Exception as e:
+            print("P-CSCF: Can't read file -- " + str(e))
+
+        # EF.DOMAIN - Home Network Domain Name e.g. ims.mncXXX.mccXXX.3gppnetwork.org
+        try:
+            if isim_card.file_exists(EF_ISIM_ADF_map['DOMAIN']):
+                (res, sw) = isim_card.read_domain()
+                if sw == '9000':
+                    print("Home Network Domain Name: %s" %
+                          (len(res) and res or 'Not available',))
+                else:
+                    print(
+                        "Home Network Domain Name: Can't read, response code = %s" % (sw,))
+        except Exception as e:
+            print("Home Network Domain Name: Can't read file -- " + str(e))
+
+        # EF.IMPI - IMS private user identity
+        try:
+            if isim_card.file_exists(EF_ISIM_ADF_map['IMPI']):
+                (res, sw) = isim_card.read_impi()
+                if sw == '9000':
+                    print("IMS private user identity: %s" %
+                          (len(res) and res or 'Not available',))
+                else:
+                    print(
+                        "IMS private user identity: Can't read, response code = %s" % (sw,))
+        except Exception as e:
+            print("IMS private user identity: Can't read file -- " + str(e))
+
+        # EF.IMPU - IMS public user identity
+        try:
+            if isim_card.file_exists(EF_ISIM_ADF_map['IMPU']):
+                res = isim_card.read_impu()
+                print("IMS public user identity:\n%s" %
+                      (len(res) and res or '\tNot available\n',))
+        except Exception as e:
+            print("IMS public user identity: Can't read file -- " + str(e))
+
+        # EF.UICCIARI - UICC IARI
+        try:
+            if isim_card.file_exists(EF_ISIM_ADF_map['UICCIARI']):
+                res = isim_card.read_iari()
+                print("UICC IARI:\n%s" %
+                      (len(res) and res or '\tNot available\n',))
+        except Exception as e:
+            print("UICC IARI: Can't read file -- " + str(e))
+
+        # EF.IST
+        (res, sw) = card.read_binary('6f07')
+        if sw == '9000':
+            print("ISIM Service Table: %s" % res)
+            # Print those which are available
+            print("%s" % dec_st(res, table="isim"))
+        else:
+            print("ISIM Service Table: Can't read, response code = %s" % (sw,))
+
+    # Done for this card and maybe for everything ?
+    print("Done !\n")

pySim-trace.py

@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+
+import sys
+import logging, colorlog
+import argparse
+from pprint import pprint as pp
+
+from pySim.apdu import *
+from pySim.runtime import RuntimeState
+
+from pySim.cards import UiccCardBase
+from pySim.commands import SimCardCommands
+from pySim.profile import CardProfile
+from pySim.ts_102_221 import CardProfileUICC
+from pySim.ts_31_102 import CardApplicationUSIM
+from pySim.ts_31_103 import CardApplicationISIM
+from pySim.transport import LinkBase
+
+from pySim.apdu_source.gsmtap import GsmtapApduSource
+from pySim.apdu_source.pyshark_rspro import PysharkRsproPcap, PysharkRsproLive
+from pySim.apdu_source.pyshark_gsmtap import PysharkGsmtapPcap
+
+from pySim.apdu.ts_102_221 import UiccSelect, UiccStatus
+
+log_format='%(log_color)s%(levelname)-8s%(reset)s %(name)s: %(message)s'
+colorlog.basicConfig(level=logging.INFO, format = log_format)
+logger = colorlog.getLogger()
+
+# merge all of the command sets into one global set. This will override instructions,
+# the one from the 'last' set in the addition below will prevail.
+from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
+from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
+from pySim.apdu.global_platform import ApduCommands as GpApduCommands
+ApduCommands = UiccApduCommands + UsimApduCommands #+ GpApduCommands
+
+
+class DummySimLink(LinkBase):
+    """A dummy implementation of the LinkBase abstract base class.  Currently required
+    as the UiccCardBase doesn't work without SimCardCommands, which in turn require
+    a LinkBase implementation talking to a card.
+
+    In the tracer, we don't actually talk to any card, so we simply drop everything
+    and claim it is successful.
+
+    The UiccCardBase / SimCardCommands should be refactored to make this obsolete later."""
+    def __init__(self, debug: bool = False, **kwargs):
+        super().__init__(**kwargs)
+        self._debug = debug
+        self._atr = h2i('3B9F96801F878031E073FE211B674A4C753034054BA9')
+
+    def __str__(self):
+        return "dummy"
+
+    def _send_apdu_raw(self, pdu):
+        #print("DummySimLink-apdu: %s" % pdu)
+        return [], '9000'
+
+    def connect(self):
+        pass
+
+    def disconnect(self):
+        pass
+
+    def reset_card(self):
+        return 1
+
+    def get_atr(self):
+        return self._atr
+
+    def wait_for_card(self):
+        pass
+
+
+class Tracer:
+    def __init__(self, **kwargs):
+        # we assume a generic UICC profile; as all APDUs return 9000 in DummySimLink above,
+        # all CardProfileAddon (including SIM) will probe successful.
+        profile = CardProfileUICC()
+        profile.add_application(CardApplicationUSIM())
+        profile.add_application(CardApplicationISIM())
+        scc = SimCardCommands(transport=DummySimLink())
+        card = UiccCardBase(scc)
+        self.rs = RuntimeState(card, profile)
+        # APDU Decoder
+        self.ad = ApduDecoder(ApduCommands)
+        # parameters
+        self.suppress_status = kwargs.get('suppress_status', True)
+        self.suppress_select = kwargs.get('suppress_select', True)
+        self.show_raw_apdu = kwargs.get('show_raw_apdu', False)
+        self.source = kwargs.get('source', None)
+
+    def format_capdu(self, apdu: Apdu, inst: ApduCommand):
+        """Output a single decoded + processed ApduCommand."""
+        if self.show_raw_apdu:
+            print(apdu)
+        print("%02u %-16s %-35s %-8s %s %s" % (inst.lchan_nr, inst._name, inst.path_str, inst.col_id, inst.col_sw, inst.processed))
+        print("===============================")
+
+    def format_reset(self, apdu: CardReset):
+        """Output a single decoded CardReset."""
+        print(apdu)
+        print("===============================")
+
+    def main(self):
+        """Main loop of tracer: Iterates over all Apdu received from source."""
+        apdu_counter = 0
+        while True:
+            # obtain the next APDU from the source (blocking read)
+            try:
+                apdu = self.source.read()
+                apdu_counter = apdu_counter + 1
+            except StopIteration:
+                print("%i APDUs parsed, stop iteration." % apdu_counter)
+                return 0
+
+            if isinstance(apdu, CardReset):
+                self.rs.reset()
+                self.format_reset(apdu)
+                continue
+
+            # ask ApduDecoder to look-up (INS,CLA) + instantiate an ApduCommand derived
+            # class like 'UiccSelect'
+            inst = self.ad.input(apdu)
+            # process the APDU (may modify the RuntimeState)
+            inst.process(self.rs)
+
+            # Avoid cluttering the log with too much verbosity
+            if self.suppress_select and isinstance(inst, UiccSelect):
+                continue
+            if self.suppress_status and isinstance(inst, UiccStatus):
+                continue
+
+            self.format_capdu(apdu, inst)
+
+option_parser = argparse.ArgumentParser(description='Osmocom pySim high-level SIM card trace decoder',
+                                        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+global_group = option_parser.add_argument_group('General Options')
+global_group.add_argument('--no-suppress-select', action='store_false', dest='suppress_select',
+                          help="""
+    Don't suppress displaying SELECT APDUs. We normally suppress them as they just clutter up
+    the output without giving any useful information.  Any subsequent READ/UPDATE/... operations
+    on the selected file will log the file name most recently SELECTed.""")
+global_group.add_argument('--no-suppress-status', action='store_false', dest='suppress_status',
+                          help="""
+    Don't suppress displaying STATUS APDUs. We normally suppress them as they don't provide any
+    information that was not already received in resposne to the most recent SEELCT.""")
+global_group.add_argument('--show-raw-apdu', action='store_true', dest='show_raw_apdu',
+                          help="""Show the raw APDU in addition to its parsed form.""")
+
+
+subparsers = option_parser.add_subparsers(help='APDU Source', dest='source', required=True)
+
+parser_gsmtap = subparsers.add_parser('gsmtap-udp', help="""
+    Read APDUs from live capture by receiving GSMTAP-SIM packets on specified UDP port.
+    Use this for live capture from SIMtrace2 or osmo-qcdiag.""")
+parser_gsmtap.add_argument('-i', '--bind-ip', default='127.0.0.1',
+                           help='Local IP address to which to bind the UDP port')
+parser_gsmtap.add_argument('-p', '--bind-port', default=4729,
+                           help='Local UDP port')
+
+parser_gsmtap_pyshark_pcap = subparsers.add_parser('gsmtap-pyshark-pcap', help="""
+    Read APDUs from PCAP file containing GSMTAP (SIM APDU) communication; processed via pyshark.
+    Use this if you have recorded a PCAP file containing GSMTAP (SIM APDU) e.g. via tcpdump or
+    wireshark/tshark.""")
+parser_gsmtap_pyshark_pcap.add_argument('-f', '--pcap-file', required=True,
+                                       help='Name of the PCAP[ng] file to be read')
+
+parser_rspro_pyshark_pcap = subparsers.add_parser('rspro-pyshark-pcap', help="""
+    Read APDUs from PCAP file containing RSPRO (osmo-remsim) communication; processed via pyshark.
+    REQUIRES OSMOCOM PATCHED WIRESHARK!""")
+parser_rspro_pyshark_pcap.add_argument('-f', '--pcap-file', required=True,
+                                       help='Name of the PCAP[ng] file to be read')
+
+parser_rspro_pyshark_live = subparsers.add_parser('rspro-pyshark-live', help="""
+    Read APDUs from live capture of RSPRO (osmo-remsim) communication; processed via pyshark.
+    REQUIRES OSMOCOM PATCHED WIRESHARK!""")
+parser_rspro_pyshark_live.add_argument('-i', '--interface', required=True,
+                                       help='Name of the network interface to capture on')
+
+if __name__ == '__main__':
+
+    opts = option_parser.parse_args()
+
+    logger.info('Opening source %s...', opts.source)
+    if opts.source == 'gsmtap-udp':
+        s = GsmtapApduSource(opts.bind_ip, opts.bind_port)
+    elif opts.source == 'rspro-pyshark-pcap':
+        s = PysharkRsproPcap(opts.pcap_file)
+    elif opts.source == 'rspro-pyshark-live':
+        s = PysharkRsproLive(opts.interface)
+    elif opts.source == 'gsmtap-pyshark-pcap':
+        s = PysharkGsmtapPcap(opts.pcap_file)
+
+    tracer = Tracer(source=s, suppress_status=opts.suppress_status, suppress_select=opts.suppress_select,
+                    show_raw_apdu=opts.show_raw_apdu)
+    logger.info('Entering main loop...')
+    tracer.main()
+

pySim/apdu/__init__.py

@@ -0,0 +1,451 @@
+"""APDU (and TPDU) parser for UICC/USIM/ISIM cards.
+
+The File (and its classes) represent the structure / hierarchy
+of the APDUs as seen in SIM/UICC/SIM/ISIM cards.  The primary use case
+is to perform a meaningful decode of protocol traces taken between card and UE.
+
+The ancient wirshark dissector developed for GSMTAP generated by SIMtrace
+is far too simplistic, while this decoder can utilize all of the information
+we already know in pySim about the filesystem structure, file encoding, etc.
+"""
+
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import abc
+import typing
+from typing import List, Dict, Optional
+from termcolor import colored
+
+from construct import Byte, GreedyBytes
+from construct import Optional as COptional
+
+from pySim.construct import *
+from pySim.utils import *
+from pySim.runtime import RuntimeLchan, RuntimeState, lchan_nr_from_cla
+from pySim.filesystem import CardADF, CardFile, TransparentEF, LinFixedEF
+
+"""There are multiple levels of decode:
+
+    1) pure TPDU / APDU level (no filesystem state required to decode)
+        1a) the raw C-TPDU + R-TPDU
+        1b) the raw C-APDU + R-APDU
+        1c) the C-APDU + R-APDU split in its portions (p1/p2/lc/le/cmd/rsp)
+        1d) the abstract C-APDU + R-APDU (mostly p1/p2 parsing; SELECT response)
+    2) the decoded DATA of command/response APDU
+        * READ/UPDATE: requires state/context: which file is selected? how to decode it?
+"""
+
+class ApduCommandMeta(abc.ABCMeta):
+    """A meta-class that we can use to set some class variables when declaring
+       a derived class of ApduCommand."""
+    def __new__(mcs, name, bases, namespace, **kwargs):
+        x = super().__new__(mcs, name, bases, namespace)
+        x._name = namespace.get('name', kwargs.get('n', None))
+        x._ins = namespace.get('ins', kwargs.get('ins', None))
+        x._cla = namespace.get('cla', kwargs.get('cla', None))
+        return x
+
+BytesOrHex = typing.Union[bytes, Hexstr]
+
+class Tpdu:
+    def __init__(self, cmd: BytesOrHex, rsp: Optional[BytesOrHex] = None):
+        if isinstance(cmd, str):
+            self.cmd = h2b(cmd)
+        else:
+            self.cmd = cmd
+        if  isinstance(rsp, str):
+            self.rsp = h2b(rsp)
+        else:
+            self.rsp = rsp
+
+    def __str__(self):
+        return '%s(%02X %02X %02X %02X %02X %s %s %s)' % (type(self).__name__, self.cla, self.ins, self.p1,
+                self.p2, self.p3, b2h(self.cmd_data), b2h(self.rsp_data), b2h(self.sw))
+
+    @property
+    def cla(self) -> int:
+        """Return CLA of the C-APDU Header."""
+        return self.cmd[0]
+
+    @property
+    def ins(self) -> int:
+        """Return INS of the C-APDU Header."""
+        return self.cmd[1]
+
+    @property
+    def p1(self) -> int:
+        """Return P1 of the C-APDU Header."""
+        return self.cmd[2]
+
+    @property
+    def p2(self) -> int:
+        """Return P2 of the C-APDU Header."""
+        return self.cmd[3]
+
+    @property
+    def p3(self) -> int:
+        """Return P3 of the C-APDU Header."""
+        return self.cmd[4]
+
+    @property
+    def cmd_data(self) -> int:
+        """Return the DATA portion of the C-APDU"""
+        return self.cmd[5:]
+
+    @property
+    def sw(self) -> Optional[bytes]:
+        """Return Status Word (SW) of the R-APDU"""
+        return self.rsp[-2:] if self.rsp else None
+
+    @property
+    def rsp_data(self) -> Optional[bytes]:
+        """Return the DATA portion of the R-APDU"""
+        return self.rsp[:-2] if self.rsp else None
+
+
+class Apdu(Tpdu):
+    @property
+    def lc(self) -> int:
+        """Return Lc; Length of C-APDU body."""
+        return len(self.cmd_data)
+
+    @property
+    def lr(self) -> int:
+        """Return Lr; Length of R-APDU body."""
+        return len(self.rsp_data)
+
+    @property
+    def successful(self) -> bool:
+        """Was the execution of this APDU successful?"""
+        method = getattr(self, '_is_success', None)
+        if callable(method):
+            return method()
+        # default case: only 9000 is success
+        if self.sw == b'\x90\x00':
+            return True
+        # This is not really a generic positive APDU SW but specific to UICC/SIM
+        if self.sw[0] == 0x91:
+            return True
+        return False
+
+
+class ApduCommand(Apdu, metaclass=ApduCommandMeta):
+    """Base class from which you would derive individual commands/instructions like SELECT.
+       A derived class represents a decoder for a specific instruction.
+       An instance of such a derived class is one concrete APDU."""
+    # fall-back constructs if the derived class provides no override
+    _construct_p1 = Byte
+    _construct_p2 = Byte
+    _construct = HexAdapter(GreedyBytes)
+    _construct_rsp = HexAdapter(GreedyBytes)
+
+    def __init__(self, cmd: BytesOrHex, rsp: Optional[BytesOrHex] = None):
+        """Instantiate a new ApduCommand from give cmd + resp."""
+        # store raw data
+        super().__init__(cmd, rsp)
+        # default to 'empty' ID column. To be set to useful values (like record number)
+        # by derived class {cmd_rsp}_to_dict() or process() methods
+        self.col_id = '-'
+        # fields only set by process_* methods
+        self.file = None
+        self.lchan = None
+        self.processed = None
+        # the methods below could raise exceptions and those handlers might assume cmd_{dict,resp}
+        self.cmd_dict = None
+        self.rsp_dict = None
+        # interpret the data
+        self.cmd_dict = self.cmd_to_dict()
+        self.rsp_dict = self.rsp_to_dict() if self.rsp else {}
+
+
+    @classmethod
+    def from_apdu(cls, apdu:Apdu, **kwargs) -> 'ApduCommand':
+        """Instantiate an ApduCommand from an existing APDU."""
+        return cls(cmd=apdu.cmd, rsp=apdu.rsp, **kwargs)
+
+    @classmethod
+    def from_bytes(cls, buffer:bytes) -> 'ApduCommand':
+        """Instantiate an ApduCommand from a linear byte buffer containing hdr,cmd,rsp,sw.
+        This is for example used when parsing GSMTAP traces that traditionally contain the
+        full command and response portion in one packet: "CLA INS P1 P2 P3 DATA SW" and we
+        now need to figure out whether the DATA part is part of the CMD or the RSP"""
+        apdu_case = cls.get_apdu_case(buffer)
+        if apdu_case in [1, 2]:
+            # data is part of response
+            return cls(buffer[:5], buffer[5:])
+        if apdu_case in [3, 4]:
+            # data is part of command
+            lc = buffer[4]
+            return cls(buffer[:5+lc], buffer[5+lc:])
+        raise ValueError('%s: Invalid APDU Case %u' % (cls.__name__, apdu_case))
+
+    @property
+    def path(self) -> List[str]:
+        """Return (if known) the path as list of files to the file on which this command operates."""
+        if self.file:
+            return self.file.fully_qualified_path()
+        return []
+
+    @property
+    def path_str(self) -> str:
+        """Return (if known) the path as string to the file on which this command operates."""
+        if self.file:
+            return self.file.fully_qualified_path_str()
+        return ''
+
+    @property
+    def col_sw(self) -> str:
+        """Return the ansi-colorized status word. Green==OK, Red==Error"""
+        if self.successful:
+            return colored(b2h(self.sw), 'green')
+        return colored(b2h(self.sw), 'red')
+
+    @property
+    def lchan_nr(self) -> int:
+        """Logical channel number over which this ApduCommand was transmitted."""
+        if self.lchan:
+            return self.lchan.lchan_nr
+        return lchan_nr_from_cla(self.cla)
+
+    def __str__(self) -> str:
+        return '%02u %s(%s): %s' % (self.lchan_nr, type(self).__name__, self.path_str, self.to_dict())
+
+    def __repr__(self) -> str:
+        return '%s(INS=%02x,CLA=%s)' % (self.__class__, self.ins, self.cla)
+
+    def _process_fallback(self, rs: RuntimeState):
+        """Fall-back function to be called if there is no derived-class-specific
+        process_global or process_on_lchan method. Uses information from APDU decode."""
+        self.processed = {}
+        if 'p1' not in self.cmd_dict:
+            self.processed = self.to_dict()
+        else:
+            self.processed['p1'] = self.cmd_dict['p1']
+            self.processed['p2'] = self.cmd_dict['p2']
+            if 'body' in self.cmd_dict and self.cmd_dict['body']:
+                self.processed['cmd'] = self.cmd_dict['body']
+            if 'body' in self.rsp_dict and self.rsp_dict['body']:
+                self.processed['rsp'] = self.rsp_dict['body']
+        return self.processed
+
+    def process(self, rs: RuntimeState):
+        # if there is a global method, use that; else use process_on_lchan
+        method = getattr(self, 'process_global', None)
+        if callable(method):
+            self.processed = method(rs)
+            return self.processed
+        method = getattr(self, 'process_on_lchan', None)
+        if callable(method):
+            self.lchan = rs.get_lchan_by_cla(self.cla)
+            self.processed = method(self.lchan)
+            return self.processed
+        # if none of the two methods exist:
+        return self._process_fallback(rs)
+
+    @classmethod
+    def get_apdu_case(cls, hdr:bytes) -> int:
+        if hasattr(cls, '_apdu_case'):
+            return cls._apdu_case
+        method = getattr(cls, '_get_apdu_case', None)
+        if callable(method):
+            return method(hdr)
+        raise ValueError('%s: Class definition missing _apdu_case attribute or _get_apdu_case method' % cls.__name__)
+
+    @classmethod
+    def match_cla(cls, cla) -> bool:
+        """Does the given CLA match the CLA list of the command?."""
+        if not isinstance(cla, str):
+            cla = '%02X' % cla
+        cla = cla.lower()
+        # see https://github.com/PyCQA/pylint/issues/7219
+        # pylint: disable=no-member
+        for cla_match in cls._cla:
+            cla_masked = ""
+            for i in range(0, 2):
+                if cla_match[i] == 'X':
+                    cla_masked += 'X'
+                else:
+                    cla_masked += cla[i]
+            if cla_masked == cla_match:
+                return True
+        return False
+
+    def cmd_to_dict(self) -> Dict:
+        """Convert the Command part of the APDU to a dict."""
+        method = getattr(self, '_decode_cmd', None)
+        if callable(method):
+            return method()
+        else:
+            r = {}
+            method = getattr(self, '_decode_p1p2', None)
+            if callable(method):
+                r = self._decode_p1p2()
+            else:
+                r['p1'] = parse_construct(self._construct_p1, self.p1.to_bytes(1, 'big'))
+                r['p2'] = parse_construct(self._construct_p2, self.p2.to_bytes(1, 'big'))
+            r['p3'] = self.p3
+            if self.cmd_data:
+                r['body'] = parse_construct(self._construct, self.cmd_data)
+            return r
+
+    def rsp_to_dict(self) -> Dict:
+        """Convert the Response part of the APDU to a dict."""
+        method = getattr(self, '_decode_rsp', None)
+        if callable(method):
+            return method()
+        else:
+            r = {}
+            if self.rsp_data:
+                r['body'] = parse_construct(self._construct_rsp, self.rsp_data)
+            r['sw'] = b2h(self.sw)
+            return r
+
+    def to_dict(self) -> Dict:
+        """Convert the entire APDU to a dict."""
+        return {'cmd': self.cmd_dict, 'rsp': self.rsp_dict}
+
+    def to_json(self) -> str:
+        """Convert the entire APDU to JSON."""
+        d = self.to_dict()
+        return json.dumps(d)
+
+    def _determine_file(self, lchan) -> CardFile:
+        """Helper function for read/update commands that might use SFI instead of selected file.
+        Expects that the self.cmd_dict has already been populated with the 'file' member."""
+        if self.cmd_dict['file'] == 'currently_selected_ef':
+            self.file = lchan.selected_file
+        elif self.cmd_dict['file'] == 'sfi':
+            cwd = lchan.get_cwd()
+            self.file = cwd.lookup_file_by_sfid(self.cmd_dict['sfi'])
+
+
+class ApduCommandSet:
+    """A set of card instructions, typically specified within one spec."""
+
+    def __init__(self, name: str, cmds: List[ApduCommand] =[]):
+        self.name = name
+        self.cmds = {c._ins: c for c in cmds}
+
+    def __str__(self) -> str:
+        return self.name
+
+    def __getitem__(self, idx) -> ApduCommand:
+        return self.cmds[idx]
+
+    def __add__(self, other) -> 'ApduCommandSet':
+        if isinstance(other, ApduCommand):
+            if other.ins in self.cmds:
+                raise ValueError('%s: INS 0x%02x already defined: %s' %
+                                 (self, other.ins, self.cmds[other.ins]))
+            self.cmds[other.ins] = other
+        elif isinstance(other, ApduCommandSet):
+            for c in other.cmds.keys():
+                self.cmds[c] = other.cmds[c]
+        else:
+            raise ValueError(
+                '%s: Unsupported type to add operator: %s' % (self, other))
+        return self
+
+    def lookup(self, ins, cla=None) -> Optional[ApduCommand]:
+        """look-up the command within the CommandSet."""
+        ins = int(ins)
+        if not ins in self.cmds:
+            return None
+        cmd = self.cmds[ins]
+        if cla and not cmd.match_cla(cla):
+            return None
+        return cmd
+
+    def parse_cmd_apdu(self, apdu: Apdu) -> ApduCommand:
+        """Parse a Command-APDU. Returns an instance of an ApduCommand derived class."""
+        # first look-up which of our member classes match CLA + INS
+        a_cls = self.lookup(apdu.ins, apdu.cla)
+        if not a_cls:
+            raise ValueError('Unknown CLA=%02X INS=%02X' % (apdu.cla, apdu.ins))
+        # then create an instance of that class and return it
+        return a_cls.from_apdu(apdu)
+
+    def parse_cmd_bytes(self, buf:bytes) -> ApduCommand:
+        """Parse from a buffer (simtrace style). Returns an instance of an ApduCommand derived class."""
+        # first look-up which of our member classes match CLA + INS
+        cla = buf[0]
+        ins = buf[1]
+        a_cls = self.lookup(ins, cla)
+        if not a_cls:
+            raise ValueError('Unknown CLA=%02X INS=%02X' % (cla, ins))
+        # then create an instance of that class and return it
+        return a_cls.from_bytes(buf)
+
+
+
+class ApduHandler(abc.ABC):
+    @abc.abstractmethod
+    def input(self, cmd: bytes, rsp: bytes):
+        pass
+
+
+class TpduFilter(ApduHandler):
+    """The TpduFilter removes the T=0 specific GET_RESPONSE from the TPDU stream and
+       calls the ApduHandler only with the actual APDU command and response parts."""
+    def __init__(self, apdu_handler: ApduHandler):
+        self.apdu_handler = apdu_handler
+        self.state = 'INIT'
+        self.last_cmd = None
+
+    def input_tpdu(self, tpdu:Tpdu):
+        # handle SW=61xx / 6Cxx
+        if tpdu.sw[0] == 0x61 or tpdu.sw[0] == 0x6C:
+            self.state = 'WAIT_GET_RESPONSE'
+            # handle successive 61/6c responses by stupid phone/modem OS
+            if tpdu.ins != 0xC0:
+                self.last_cmd = tpdu.cmd
+            return None
+        else:
+            if self.last_cmd:
+                icmd = self.last_cmd
+                self.last_cmd = None
+            else:
+                icmd = tpdu.cmd
+            apdu = Apdu(icmd, tpdu.rsp)
+            if self.apdu_handler:
+                return self.apdu_handler.input(apdu)
+            return Apdu(icmd, tpdu.rsp)
+
+    def input(self, cmd: bytes, rsp: bytes):
+        if isinstance(cmd, str):
+            cmd = bytes.fromhex(cmd)
+        if isinstance(rsp, str):
+            rsp = bytes.fromhex(rsp)
+        tpdu = Tpdu(cmd, rsp)
+        return self.input_tpdu(tpdu)
+
+class ApduDecoder(ApduHandler):
+    def __init__(self, cmd_set: ApduCommandSet):
+        self.cmd_set = cmd_set
+
+    def input(self, apdu: Apdu):
+        return self.cmd_set.parse_cmd_apdu(apdu)
+
+
+class CardReset:
+    def __init__(self, atr: bytes):
+        self.atr = atr
+
+    def __str__(self):
+        if self.atr:
+            return '%s(%s)' % (type(self).__name__, b2h(self.atr))
+        return '%s' % (type(self).__name__)

pySim/apdu/global_platform.py

@@ -0,0 +1,57 @@
+# coding=utf-8
+"""APDU definition/decoder of GlobalPLatform Card Spec (currently 2.1.1)
+
+(C) 2022 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from pySim.apdu import ApduCommand, ApduCommandSet
+
+class GpDelete(ApduCommand, n='DELETE', ins=0xE4, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpStoreData(ApduCommand, n='STORE DATA', ins=0xE2, cla=['8X', 'CX', 'EX']):
+    @classmethod
+    def _get_apdu_case(cls, hdr:bytes) -> int:
+        p1 = hdr[2]
+        if p1 & 0x01:
+            return 4
+        else:
+            return 3
+
+class GpGetDataCA(ApduCommand, n='GET DATA', ins=0xCA, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpGetDataCB(ApduCommand, n='GET DATA', ins=0xCB, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpGetStatus(ApduCommand, n='GET STATUS', ins=0xF2, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpInstall(ApduCommand, n='INSTALL', ins=0xE6, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpLoad(ApduCommand, n='LOAD', ins=0xE8, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpPutKey(ApduCommand, n='PUT KEY', ins=0xD8, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+class GpSetStatus(ApduCommand, n='SET STATUS', ins=0xF0, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 3
+
+ApduCommands = ApduCommandSet('GlobalPlatform v2.3.1', cmds=[GpDelete, GpStoreData,
+                              GpGetDataCA, GpGetDataCB, GpGetStatus, GpInstall,
+                              GpLoad, GpPutKey, GpSetStatus])

pySim/apdu/ts_102_221.py

@@ -0,0 +1,526 @@
+# coding=utf-8
+"""APDU definitions/decoders of ETSI TS 102 221, the core UICC spec.
+
+(C) 2022 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from typing import Optional, Dict
+import logging
+
+from construct import GreedyRange, Struct
+
+from pySim.construct import *
+from pySim.filesystem import *
+from pySim.runtime import RuntimeLchan
+from pySim.apdu import ApduCommand, ApduCommandSet
+from pySim.utils import i2h
+
+logger = logging.getLogger(__name__)
+
+# TS 102 221 Section 11.1.1
+class UiccSelect(ApduCommand, n='SELECT', ins=0xA4, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_p1 = Enum(Byte, df_ef_or_mf_by_file_id=0, child_df_of_current_df=1, parent_df_of_current_df=3,
+                         df_name=4, path_from_mf=8, path_from_current_df=9)
+    _construct_p2 = BitStruct(Flag,
+                              'app_session_control'/Enum(BitsInteger(2), activation_reset=0, termination=2),
+                              'return'/Enum(BitsInteger(3), fcp=1, no_data=3),
+                              'aid_control'/Enum(BitsInteger(2), first_or_only=0, last=1, next=2, previous=3))
+
+    @staticmethod
+    def _find_aid_substr(selectables, aid) -> Optional[CardADF]:
+        # full-length match
+        if aid in selectables:
+            return selectables[aid]
+        # sub-string match
+        for s in selectables.keys():
+            if aid[:len(s)] == s:
+                return selectables[s]
+        return None
+
+    def process_on_lchan(self, lchan: RuntimeLchan):
+        mode = self.cmd_dict['p1']
+        if mode in ['path_from_mf', 'path_from_current_df']:
+            # rewind to MF, if needed
+            if mode == 'path_from_mf':
+                lchan.selected_file = lchan.rs.mf
+            path = [self.cmd_data[i:i+2] for i in range(0, len(self.cmd_data), 2)]
+            for file in path:
+                file_hex = b2h(file)
+                if file_hex == '7fff': # current application
+                    if not lchan.selected_adf:
+                        sels = lchan.rs.mf.get_app_selectables(['ANAMES'])
+                        # HACK: Assume USIM
+                        logger.warning('SELECT relative to current ADF, but no ADF selected. Assuming ADF.USIM')
+                        lchan.selected_adf = sels['ADF.USIM']
+                    lchan.selected_file = lchan.selected_adf
+                    #print("\tSELECT CUR_ADF %s" % lchan.selected_file)
+                    # iterate to next element in path
+                    continue
+                else:
+                    sels = lchan.selected_file.get_selectables(['FIDS','MF','PARENT','SELF'])
+                    if file_hex in sels:
+                        if self.successful:
+                            #print("\tSELECT %s" % sels[file_hex])
+                            lchan.selected_file = sels[file_hex]
+                        else:
+                            #print("\tSELECT %s FAILED" % sels[file_hex])
+                            pass
+                        # iterate to next element in path
+                        continue
+                logger.warning('SELECT UNKNOWN FID %s (%s)', file_hex, '/'.join([b2h(x) for x in path]))
+        elif mode == 'df_ef_or_mf_by_file_id':
+            if len(self.cmd_data) != 2:
+                raise ValueError('Expecting a 2-byte FID')
+            sels = lchan.selected_file.get_selectables(['FIDS','MF','PARENT','SELF'])
+            file_hex = b2h(self.cmd_data)
+            if file_hex in sels:
+                if self.successful:
+                    #print("\tSELECT %s" % sels[file_hex])
+                    lchan.selected_file = sels[file_hex]
+                else:
+                    #print("\tSELECT %s FAILED" % sels[file_hex])
+                    pass
+            else:
+                logger.warning('SELECT UNKNOWN FID %s', file_hex)
+        elif mode == 'df_name':
+            # Select by AID (can be sub-string!)
+            aid = self.cmd_dict['body']
+            sels = lchan.rs.mf.get_app_selectables(['AIDS'])
+            adf = self._find_aid_substr(sels, aid)
+            if adf:
+                lchan.selected_adf = adf
+                lchan.selected_file = lchan.selected_adf
+                #print("\tSELECT AID %s" % adf)
+            else:
+                logger.warning('SELECT UNKNOWN AID %s', aid)
+        else:
+            raise ValueError('Select Mode %s not implemented' % mode)
+        # decode the SELECT response
+        if self.successful:
+            self.file = lchan.selected_file
+            if 'body' in self.rsp_dict:
+                # not every SELECT is asking for the FCP in response...
+                return lchan.selected_file.decode_select_response(self.rsp_dict['body'])
+        return None
+
+
+
+# TS 102 221 Section 11.1.2
+class UiccStatus(ApduCommand, n='STATUS', ins=0xF2, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 2
+    _construct_p1 = Enum(Byte, no_indication=0, current_app_is_initialized=1, terminal_will_terminate_current_app=2)
+    _construct_p2 = Enum(Byte, response_like_select=0, response_df_name_tlv=1, response_no_data=0x0c)
+
+    def process_on_lchan(self, lchan):
+        if self.cmd_dict['p2'] == 'response_like_select':
+            return lchan.selected_file.decode_select_response(self.rsp_dict['body'])
+
+def _decode_binary_p1p2(p1, p2) -> Dict:
+    ret = {}
+    if p1 & 0x80:
+        ret['file'] = 'sfi'
+        ret['sfi'] = p1 & 0x1f
+        ret['offset'] = p2
+    else:
+        ret['file'] = 'currently_selected_ef'
+        ret['offset'] = ((p1 & 0x7f) << 8) & p2
+    return ret
+
+# TS 102 221 Section 11.1.3
+class ReadBinary(ApduCommand, n='READ BINARY', ins=0xB0, cla=['0X', '4X', '6X']):
+    _apdu_case = 2
+    def _decode_p1p2(self):
+        return _decode_binary_p1p2(self.p1, self.p2)
+
+    def process_on_lchan(self, lchan):
+        self._determine_file(lchan)
+        if not isinstance(self.file, TransparentEF):
+            return b2h(self.rsp_data)
+        # our decoders don't work for non-zero offsets / short reads
+        if self.cmd_dict['offset'] != 0 or self.lr < self.file.size[0]:
+            return b2h(self.rsp_data)
+        method = getattr(self.file, 'decode_bin', None)
+        if self.successful and callable(method):
+            return method(self.rsp_data)
+
+# TS 102 221 Section 11.1.4
+class UpdateBinary(ApduCommand, n='UPDATE BINARY', ins=0xD6, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    def _decode_p1p2(self):
+        return _decode_binary_p1p2(self.p1, self.p2)
+
+    def process_on_lchan(self, lchan):
+        self._determine_file(lchan)
+        if not isinstance(self.file, TransparentEF):
+            return b2h(self.rsp_data)
+        # our decoders don't work for non-zero offsets / short writes
+        if self.cmd_dict['offset'] != 0 or self.lc < self.file.size[0]:
+            return b2h(self.cmd_data)
+        method = getattr(self.file, 'decode_bin', None)
+        if self.successful and callable(method):
+            return method(self.cmd_data)
+
+def _decode_record_p1p2(p1, p2):
+    ret = {}
+    ret['record_number'] = p1
+    if p2 >> 3 == 0:
+        ret['file'] = 'currently_selected_ef'
+    else:
+        ret['file'] = 'sfi'
+        ret['sfi'] = p2 >> 3
+    mode = p2 & 0x7
+    if mode == 2:
+        ret['mode'] = 'next_record'
+    elif mode == 3:
+        ret['mode'] = 'previous_record'
+    elif mode == 8:
+        ret['mode'] = 'absolute_current'
+    return ret
+
+# TS 102 221 Section 11.1.5
+class ReadRecord(ApduCommand, n='READ RECORD', ins=0xB2, cla=['0X', '4X', '6X']):
+    _apdu_case = 2
+    def _decode_p1p2(self):
+        r = _decode_record_p1p2(self.p1, self.p2)
+        self.col_id = '%02u' % r['record_number']
+        return r
+
+    def process_on_lchan(self, lchan):
+        self._determine_file(lchan)
+        if not isinstance(self.file, LinFixedEF):
+            return b2h(self.rsp_data)
+        method = getattr(self.file, 'decode_record_bin', None)
+        if self.successful and callable(method):
+            return method(self.rsp_data, self.cmd_dict['record_number'])
+
+# TS 102 221 Section 11.1.6
+class UpdateRecord(ApduCommand, n='UPDATE RECORD', ins=0xDC, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    def _decode_p1p2(self):
+        r = _decode_record_p1p2(self.p1, self.p2)
+        self.col_id = '%02u' % r['record_number']
+        return r
+
+    def process_on_lchan(self, lchan):
+        self._determine_file(lchan)
+        if not isinstance(self.file, LinFixedEF):
+            return b2h(self.cmd_data)
+        method = getattr(self.file, 'decode_record_bin', None)
+        if self.successful and callable(method):
+            return method(self.cmd_data, self.cmd_dict['record_number'])
+
+# TS 102 221 Section 11.1.7
+class SearchRecord(ApduCommand, n='SEARCH RECORD', ins=0xA2, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_rsp = GreedyRange(Int8ub)
+
+    def _decode_p1p2(self):
+        ret = {}
+        sfi = self.p2 >> 3
+        if sfi == 0:
+            ret['file'] = 'currently_selected_ef'
+        else:
+            ret['file'] = 'sfi'
+            ret['sfi'] = sfi
+        mode = self.p2 & 0x7
+        if mode in [0x4, 0x5]:
+            if mode == 0x4:
+                ret['mode'] = 'forward_search'
+            else:
+                ret['mode'] = 'backward_search'
+            ret['record_number'] = self.p1
+            self.col_id = '%02u' % ret['record_number']
+        elif mode == 6:
+            ret['mode'] = 'enhanced_search'
+            # TODO: further decode
+        elif mode == 7:
+            ret['mode'] = 'proprietary_search'
+        return ret
+
+    def _decode_cmd(self):
+        ret = self._decode_p1p2()
+        if self.cmd_data:
+            if ret['mode'] == 'enhanced_search':
+                ret['search_indication'] = b2h(self.cmd_data[:2])
+                ret['search_string'] = b2h(self.cmd_data[2:])
+            else:
+                ret['search_string'] = b2h(self.cmd_data)
+        return ret
+
+    def process_on_lchan(self, lchan):
+        self._determine_file(lchan)
+        return self.to_dict()
+
+# TS 102 221 Section 11.1.8
+class Increase(ApduCommand, n='INCREASE', ins=0x32, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+PinConstructP2 = BitStruct('scope'/Enum(Flag, global_mf=0, specific_df_adf=1),
+                           BitsInteger(2), 'reference_data_nr'/BitsInteger(5))
+# TS 102 221 Section 11.1.9
+class VerifyPin(ApduCommand, n='VERIFY PIN', ins=0x20, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    _construct_p2 = PinConstructP2
+
+    @staticmethod
+    def _pin_process(apdu):
+        processed = {
+            'scope': apdu.cmd_dict['p2']['scope'],
+            'referenced_data_nr': apdu.cmd_dict['p2']['reference_data_nr'],
+            }
+        if apdu.lc == 0:
+            # this is just a question on the counters remaining
+            processed['mode'] = 'check_remaining_attempts'
+        else:
+            processed['pin'] = b2h(apdu.cmd_data)
+        if apdu.sw[0] == 0x63:
+            processed['remaining_attempts'] = apdu.sw[1] & 0xf
+        return processed
+
+    @staticmethod
+    def _pin_is_success(sw):
+        return bool(sw[0] == 0x63)
+
+    def process_on_lchan(self, _lchan: RuntimeLchan):
+        return VerifyPin._pin_process(self)
+
+    def _is_success(self):
+        return VerifyPin._pin_is_success(self.sw)
+
+
+# TS 102 221 Section 11.1.10
+class ChangePin(ApduCommand, n='CHANGE PIN', ins=0x24, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    _construct_p2 = PinConstructP2
+
+    def process_on_lchan(self, _lchan: RuntimeLchan):
+        return VerifyPin._pin_process(self)
+
+    def _is_success(self):
+        return VerifyPin._pin_is_success(self.sw)
+
+
+# TS 102 221 Section 11.1.11
+class DisablePin(ApduCommand, n='DISABLE PIN', ins=0x26, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    _construct_p2 = PinConstructP2
+
+    def process_on_lchan(self, _lchan: RuntimeLchan):
+        return VerifyPin._pin_process(self)
+
+    def _is_success(self):
+        return VerifyPin._pin_is_success(self.sw)
+
+
+# TS 102 221 Section 11.1.12
+class EnablePin(ApduCommand, n='ENABLE PIN', ins=0x28, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    _construct_p2 = PinConstructP2
+    def process_on_lchan(self, _lchan: RuntimeLchan):
+        return VerifyPin._pin_process(self)
+
+    def _is_success(self):
+        return VerifyPin._pin_is_success(self.sw)
+
+
+# TS 102 221 Section 11.1.13
+class UnblockPin(ApduCommand, n='UNBLOCK PIN', ins=0x2C, cla=['0X', '4X', '6X']):
+    _apdu_case = 3
+    _construct_p2 = PinConstructP2
+
+    def process_on_lchan(self, _lchan: RuntimeLchan):
+        return VerifyPin._pin_process(self)
+
+    def _is_success(self):
+        return VerifyPin._pin_is_success(self.sw)
+
+
+# TS 102 221 Section 11.1.14
+class DeactivateFile(ApduCommand, n='DEACTIVATE FILE', ins=0x04, cla=['0X', '4X', '6X']):
+    _apdu_case = 1
+    _construct_p1 = BitStruct(BitsInteger(4),
+                              'select_mode'/Enum(BitsInteger(4), ef_by_file_id=0,
+                                                 path_from_mf=8, path_from_current_df=9))
+
+# TS 102 221 Section 11.1.15
+class ActivateFile(ApduCommand, n='ACTIVATE FILE', ins=0x44, cla=['0X', '4X', '6X']):
+    _apdu_case = 1
+    _construct_p1 = DeactivateFile._construct_p1
+
+# TS 102 221 Section 11.1.16
+auth_p2_construct = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
+                              BitsInteger(2),
+                              'reference_data_nr'/BitsInteger(5))
+class Authenticate88(ApduCommand, n='AUTHENTICATE', ins=0x88, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_p2 = auth_p2_construct
+
+# TS 102 221 Section 11.1.16
+class Authenticate89(ApduCommand, n='AUTHENTICATE', ins=0x89, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_p2 = auth_p2_construct
+
+# TS 102 221 Section 11.1.17
+class ManageChannel(ApduCommand, n='MANAGE CHANNEL', ins=0x70, cla=['0X', '4X', '6X']):
+    _apdu_case = 2
+    _construct_p1 = Enum(Flag, open_channel=0, close_channel=1)
+    _construct_p2 = Struct('logical_channel_number'/Int8ub)
+    _construct_rsp = Struct('logical_channel_number'/Int8ub)
+
+    def process_global(self, rs):
+        if not self.successful:
+            return
+        mode = self.cmd_dict['p1']
+        if mode == 'open_channel':
+            created_channel_nr = self.cmd_dict['p2']['logical_channel_number']
+            if created_channel_nr == 0:
+                # auto-assignment by UICC
+                # pylint: disable=unsubscriptable-object
+                created_channel_nr = self.rsp_data[0]
+            manage_channel = rs.get_lchan_by_cla(self.cla)
+            manage_channel.add_lchan(created_channel_nr)
+            self.col_id = '%02u' % created_channel_nr
+            return {'mode': mode, 'created_channel': created_channel_nr }
+        if mode == 'close_channel':
+            closed_channel_nr = self.cmd_dict['p2']['logical_channel_number']
+            rs.del_lchan(closed_channel_nr)
+            self.col_id = '%02u' % closed_channel_nr
+            return {'mode': mode, 'closed_channel': closed_channel_nr }
+        raise ValueError('Unsupported MANAGE CHANNEL P1=%02X' % self.p1)
+
+# TS 102 221 Section 11.1.18
+class GetChallenge(ApduCommand, n='GET CHALLENGE', ins=0x84, cla=['0X', '4X', '6X']):
+    _apdu_case = 2
+
+# TS 102 221 Section 11.1.19
+class TerminalCapability(ApduCommand, n='TERMINAL CAPABILITY', ins=0xAA, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 3
+
+# TS 102 221 Section 11.1.20
+class ManageSecureChannel(ApduCommand, n='MANAGE SECURE CHANNEL', ins=0x73, cla=['0X', '4X', '6X']):
+    @classmethod
+    def _get_apdu_case(cls, hdr:bytes) -> int:
+        p1 = hdr[2]
+        p2 = hdr[3]
+        if p1 & 0x7 == 0:   # retrieve UICC Endpoints
+            return 2
+        if p1 & 0xf in [1,2,3]:   # establish sa, start secure channel SA
+            p2_cmd = p2 >> 5
+            if p2_cmd in [0,2,4]:   # command data
+                return 3
+            if p2_cmd in [1,3,5]: # response data
+                return 2
+        if p1 & 0xf == 4:         # terminate secure channel SA
+            return 3
+        raise ValueError('%s: Unable to detect APDU case for %s' % (cls.__name__, b2h(hdr)))
+
+# TS 102 221 Section 11.1.21
+class TransactData(ApduCommand, n='TRANSACT DATA', ins=0x75, cla=['0X', '4X', '6X']):
+    @classmethod
+    def _get_apdu_case(cls, hdr:bytes) -> int:
+        p1 = hdr[2]
+        if p1 & 0x04:
+            return 3
+        return 2
+
+# TS 102 221 Section 11.1.22
+class SuspendUicc(ApduCommand, n='SUSPEND UICC', ins=0x76, cla=['80']):
+    _apdu_case = 4
+    _construct_p1 = BitStruct('rfu'/BitsInteger(7), 'mode'/Enum(Flag, suspend=0, resume=1))
+
+# TS 102 221 Section 11.1.23
+class GetIdentity(ApduCommand, n='GET IDENTITY', ins=0x78, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+    _construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1), BitsInteger(7))
+
+# TS 102 221 Section 11.1.24
+class ExchangeCapabilities(ApduCommand, n='EXCHANGE CAPABILITIES', ins=0x7A, cla=['80']):
+    _apdu_case = 4
+
+# TS 102 221 Section 11.2.1
+class TerminalProfile(ApduCommand, n='TERMINAL PROFILE', ins=0x10, cla=['80']):
+    _apdu_case = 3
+
+# TS 102 221 Section 11.2.2 / TS 102 223
+class Envelope(ApduCommand, n='ENVELOPE', ins=0xC2, cla=['80']):
+    _apdu_case = 4
+
+# TS 102 221 Section 11.2.3 / TS 102 223
+class Fetch(ApduCommand, n='FETCH', ins=0x12, cla=['80']):
+    _apdu_case = 2
+
+# TS 102 221 Section 11.2.3 / TS 102 223
+class TerminalResponse(ApduCommand, n='TERMINAL RESPONSE', ins=0x14, cla=['80']):
+    _apdu_case = 3
+
+# TS 102 221 Section 11.3.1
+class RetrieveData(ApduCommand, n='RETRIEVE DATA', ins=0xCB, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+
+    @staticmethod
+    def _tlv_decode_cmd(self : ApduCommand) -> Dict:
+        c = {}
+        if self.p2 & 0xc0 == 0x80:
+            c['mode'] = 'first_block'
+            sfi = self.p2 & 0x1f
+            if sfi == 0:
+                c['file'] = 'currently_selected_ef'
+            else:
+                c['file'] = 'sfi'
+                c['sfi'] = sfi
+            c['tag'] = i2h([self.cmd_data[0]])
+        elif self.p2 & 0xdf == 0x00:
+            c['mode'] = 'next_block'
+        elif self.p2 & 0xdf == 0x40:
+            c['mode'] = 'retransmit_previous_block'
+        else:
+            logger.warning('%s: invalid P2=%02x', self, self.p2)
+        return c
+
+    def _decode_cmd(self):
+        return RetrieveData._tlv_decode_cmd(self)
+
+    def _decode_rsp(self):
+        # TODO: parse tag/len/val?
+        return b2h(self.rsp_data)
+
+
+# TS 102 221 Section 11.3.2
+class SetData(ApduCommand, n='SET DATA', ins=0xDB, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 3
+
+    def _decode_cmd(self):
+        c = RetrieveData._tlv_decode_cmd(self)
+        if c['mode'] == 'first_block':
+            if len(self.cmd_data) == 0:
+                c['delete'] = True
+        # TODO: parse tag/len/val?
+        c['data'] = b2h(self.cmd_data)
+        return c
+
+
+# TS 102 221 Section 12.1.1
+class GetResponse(ApduCommand, n='GET RESPONSE', ins=0xC0, cla=['0X', '4X', '6X']):
+    _apdu_case = 2
+
+ApduCommands = ApduCommandSet('TS 102 221', cmds=[UiccSelect, UiccStatus, ReadBinary, UpdateBinary, ReadRecord,
+                              UpdateRecord, SearchRecord, Increase, VerifyPin, ChangePin, DisablePin,
+                              EnablePin, UnblockPin, DeactivateFile, ActivateFile, Authenticate88,
+                              Authenticate89, ManageChannel, GetChallenge, TerminalCapability,
+                              ManageSecureChannel, TransactData, SuspendUicc, GetIdentity,
+                              ExchangeCapabilities, TerminalProfile, Envelope, Fetch, TerminalResponse,
+                              RetrieveData, SetData, GetResponse])

pySim/apdu/ts_31_102.py

@@ -0,0 +1,113 @@
+# -*- coding: utf-8 -*-
+
+# without this, pylint will fail when inner classes are used
+# within the 'nested' kwarg of our TlvMeta metaclass on python 3.7 :(
+# pylint: disable=undefined-variable
+
+"""
+APDU commands of 3GPP TS 31.102 V16.6.0
+"""
+from typing import Dict
+
+from construct import BitStruct, Enum, BitsInteger, Int8ub, Bytes, this, Struct, If, Switch, Const
+from construct import Optional as COptional
+
+from pySim.filesystem import *
+from pySim.construct import *
+from pySim.ts_31_102 import SUCI_TlvDataObject
+from pySim.apdu import ApduCommand, ApduCommandSet
+
+# Copyright (C) 2022 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# Mapping between USIM Service Number and its description
+
+# TS 31.102 Section 7.1
+class UsimAuthenticateEven(ApduCommand, n='AUTHENTICATE', ins=0x88, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
+                              BitsInteger(4),
+                              'authentication_context'/Enum(BitsInteger(3), gsm=0, umts=1,
+                                                            vgcs_vbs=2, gba=4))
+    _cs_cmd_gsm_3g =  Struct('_rand_len'/Int8ub, 'rand'/HexAdapter(Bytes(this._rand_len)),
+                         '_autn_len'/COptional(Int8ub), 'autn'/If(this._autn_len, HexAdapter(Bytes(this._autn_len))))
+    _cs_cmd_vgcs = Struct('_vsid_len'/Int8ub, 'vservice_id'/HexAdapter(Bytes(this._vsid_len)),
+                          '_vkid_len'/Int8ub, 'vk_id'/HexAdapter(Bytes(this._vkid_len)),
+                          '_vstk_rand_len'/Int8ub, 'vstk_rand'/HexAdapter(Bytes(this._vstk_rand_len)))
+    _cmd_gba_bs = Struct('_rand_len'/Int8ub, 'rand'/HexAdapter(Bytes(this._rand_len)),
+                         '_autn_len'/Int8ub, 'autn'/HexAdapter(Bytes(this._autn_len)))
+    _cmd_gba_naf = Struct('_naf_id_len'/Int8ub, 'naf_id'/HexAdapter(Bytes(this._naf_id_len)),
+                          '_impi_len'/Int8ub, 'impi'/HexAdapter(Bytes(this._impi_len)))
+    _cs_cmd_gba = Struct('tag'/Int8ub, 'body'/Switch(this.tag, { 0xDD: 'bootstrap'/_cmd_gba_bs,
+                                                                 0xDE: 'naf_derivation'/_cmd_gba_naf }))
+    _cs_rsp_gsm = Struct('_len_sres'/Int8ub, 'sres'/HexAdapter(Bytes(this._len_sres)),
+                         '_len_kc'/Int8ub, 'kc'/HexAdapter(Bytes(this._len_kc)))
+    _rsp_3g_ok = Struct('_len_res'/Int8ub, 'res'/HexAdapter(Bytes(this._len_res)),
+                        '_len_ck'/Int8ub, 'ck'/HexAdapter(Bytes(this._len_ck)),
+                        '_len_ik'/Int8ub, 'ik'/HexAdapter(Bytes(this._len_ik)),
+                        '_len_kc'/COptional(Int8ub), 'kc'/If(this._len_kc, HexAdapter(Bytes(this._len_kc))))
+    _rsp_3g_sync = Struct('_len_auts'/Int8ub, 'auts'/HexAdapter(Bytes(this._len_auts)))
+    _cs_rsp_3g = Struct('tag'/Int8ub, 'body'/Switch(this.tag, { 0xDB: 'success'/_rsp_3g_ok,
+                                                                0xDC: 'sync_fail'/_rsp_3g_sync}))
+    _cs_rsp_vgcs = Struct(Const(b'\xDB'), '_vstk_len'/Int8ub, 'vstk'/HexAdapter(Bytes(this._vstk_len)))
+    _cs_rsp_gba_naf = Struct(Const(b'\xDB'), '_ks_ext_naf_len'/Int8ub, 'ks_ext_naf'/HexAdapter(Bytes(this._ks_ext_naf_len)))
+    def _decode_cmd(self) -> Dict:
+        r = {}
+        r['p1'] = parse_construct(self._construct_p1, self.p1.to_bytes(1, 'big'))
+        r['p2'] = parse_construct(self._construct_p2, self.p2.to_bytes(1, 'big'))
+        auth_ctx = r['p2']['authentication_context']
+        if auth_ctx in ['gsm', 'umts']:
+            r['body'] = parse_construct(self._cs_cmd_gsm_3g, self.cmd_data)
+        elif auth_ctx == 'vgcs_vbs':
+            r['body'] = parse_construct(self._cs_cmd_vgcs, self.cmd_data)
+        elif auth_ctx == 'gba':
+            r['body'] = parse_construct(self._cs_cmd_gba, self.cmd_data)
+        else:
+            raise ValueError('Unsupported authentication_context: %s' % auth_ctx)
+        return r
+
+    def _decode_rsp(self) -> Dict:
+        r = {}
+        auth_ctx = self.cmd_dict['p2']['authentication_context']
+        if auth_ctx == 'gsm':
+            r['body'] = parse_construct(self._cs_rsp_gsm, self.rsp_data)
+        elif auth_ctx == 'umts':
+            r['body'] = parse_construct(self._cs_rsp_3g, self.rsp_data)
+        elif auth_ctx == 'vgcs_vbs':
+            r['body'] = parse_construct(self._cs_rsp_vgcs, self.rsp_data)
+        elif auth_ctx == 'gba':
+            if self.cmd_dict['body']['tag'] == 0xDD:
+                r['body'] = parse_construct(self._cs_rsp_3g, self.rsp_data)
+            else:
+                r['body'] = parse_construct(self._cs_rsp_gba_naf, self.rsp_data)
+        else:
+            raise ValueError('Unsupported authentication_context: %s' % auth_ctx)
+        return r
+
+class UsimAuthenticateOdd(ApduCommand, n='AUTHENTICATE', ins=0x89, cla=['0X', '4X', '6X']):
+    _apdu_case = 4
+    _construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
+                              BitsInteger(4),
+                              'authentication_context'/Enum(BitsInteger(3), mbms=5, local_key=6))
+# TS 31.102 Section 7.5
+class UsimGetIdentity(ApduCommand, n='GET IDENTITY', ins=0x78, cla=['8X', 'CX', 'EX']):
+    _apdu_case = 4
+    _construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
+                              'identity_context'/Enum(BitsInteger(7), suci=1, suci_5g_nswo=2))
+    _tlv_rsp = SUCI_TlvDataObject
+
+ApduCommands = ApduCommandSet('TS 31.102', cmds=[UsimAuthenticateEven, UsimAuthenticateOdd,
+                                                 UsimGetIdentity])

pySim/apdu_source/__init__.py

@@ -0,0 +1,34 @@
+import abc
+import logging
+from typing import Union
+from pySim.apdu import Apdu, Tpdu, CardReset, TpduFilter
+
+PacketType = Union[Apdu, Tpdu, CardReset]
+
+logger = logging.getLogger(__name__)
+
+class ApduSource(abc.ABC):
+    def __init__(self):
+        self.apdu_filter = TpduFilter(None)
+
+    @abc.abstractmethod
+    def read_packet(self) -> PacketType:
+        """Read one packet from the source."""
+
+    def read(self) -> Union[Apdu, CardReset]:
+        """Main function to call by the user: Blocking read, returns Apdu or CardReset."""
+        apdu = None
+        # loop until we actually have an APDU to return
+        while not apdu:
+            r = self.read_packet()
+            if not r:
+                continue
+            if isinstance(r, Tpdu):
+                apdu = self.apdu_filter.input_tpdu(r)
+            elif isinstance(r, Apdu):
+                apdu = r
+            elif isinstance(r, CardReset):
+                apdu = r
+            else:
+                raise ValueError('Unknown read_packet() return %s' % r)
+        return apdu

pySim/apdu_source/gsmtap.py

@@ -0,0 +1,59 @@
+# coding=utf-8
+
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from pySim.gsmtap import GsmtapSource
+
+from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
+from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
+from pySim.apdu.global_platform import ApduCommands as GpApduCommands
+
+from . import ApduSource, PacketType, CardReset
+
+ApduCommands = UiccApduCommands + UsimApduCommands + GpApduCommands
+
+class GsmtapApduSource(ApduSource):
+    """ApduSource for handling GSMTAP-SIM messages received via UDP, such as
+    those generated by simtrace2-sniff.  Note that *if* you use IP loopback
+    and localhost addresses (which is the default), you will need to start
+    this source before starting simtrace2-sniff, as otherwise the latter will
+    claim the GSMTAP UDP port.
+    """
+    def __init__(self, bind_ip:str='127.0.0.1', bind_port:int=4729):
+        """Create a UDP socket for receiving GSMTAP-SIM messages.
+        Args:
+            bind_ip: IP address to which the socket should be bound (default: 127.0.0.1)
+            bind_port: UDP port number to which the socket should be bound (default: 4729)
+        """
+        super().__init__()
+        self.gsmtap = GsmtapSource(bind_ip, bind_port)
+
+    def read_packet(self) -> PacketType:
+        gsmtap_msg, _addr = self.gsmtap.read_packet()
+        if gsmtap_msg['type'] != 'sim':
+            raise ValueError('Unsupported GSMTAP type %s' % gsmtap_msg['type'])
+        sub_type = gsmtap_msg['sub_type']
+        if sub_type == 'apdu':
+            return ApduCommands.parse_cmd_bytes(gsmtap_msg['body'])
+        if sub_type == 'atr':
+            # card has been reset
+            return CardReset(gsmtap_msg['body'])
+        if sub_type in ['pps_req', 'pps_rsp']:
+            # simply ignore for now
+            pass
+        else:
+            raise ValueError('Unsupported GSMTAP-SIM sub-type %s' % sub_type)

pySim/apdu_source/pyshark_gsmtap.py

@@ -0,0 +1,88 @@
+# coding=utf-8
+
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import logging
+from typing import Tuple
+import pyshark
+
+from pySim.utils import h2b
+from pySim.gsmtap import GsmtapMessage
+
+from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
+from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
+from pySim.apdu.global_platform import ApduCommands as GpApduCommands
+
+from . import ApduSource, PacketType, CardReset
+
+ApduCommands = UiccApduCommands + UsimApduCommands + GpApduCommands
+
+logger = logging.getLogger(__name__)
+
+class _PysharkGsmtap(ApduSource):
+    """APDU Source [provider] base class for reading GSMTAP SIM APDU via tshark."""
+
+    def __init__(self, pyshark_inst):
+        self.pyshark = pyshark_inst
+        self.bank_id = None
+        self.bank_slot = None
+        self.cmd_tpdu = None
+        super().__init__()
+
+    def read_packet(self) -> PacketType:
+        p = self.pyshark.next()
+        return self._parse_packet(p)
+
+    def _set_or_verify_bank_slot(self, bsl: Tuple[int, int]):
+        """Keep track of the bank:slot to make sure we don't mix traces of multiple cards"""
+        if not self.bank_id:
+            self.bank_id = bsl[0]
+            self.bank_slot = bsl[1]
+        else:
+            if self.bank_id != bsl[0] or self.bank_slot != bsl[1]:
+                raise ValueError('Received data for unexpected B(%u:%u)' % (bsl[0], bsl[1]))
+
+    def _parse_packet(self, p) -> PacketType:
+        udp_layer = p['udp']
+        udp_payload_hex = udp_layer.get_field('payload').replace(':','')
+        gsmtap = GsmtapMessage(h2b(udp_payload_hex))
+        gsmtap_msg = gsmtap.decode()
+        if gsmtap_msg['type'] != 'sim':
+            raise ValueError('Unsupported GSMTAP type %s' % gsmtap_msg['type'])
+        sub_type = gsmtap_msg['sub_type']
+        if sub_type == 'apdu':
+            return ApduCommands.parse_cmd_bytes(gsmtap_msg['body'])
+        if sub_type == 'atr':
+            # card has been reset
+            return CardReset(gsmtap_msg['body'])
+        if sub_type in ['pps_req', 'pps_rsp']:
+            # simply ignore for now
+            pass
+        else:
+            raise ValueError('Unsupported GSMTAP-SIM sub-type %s' % sub_type)
+
+class PysharkGsmtapPcap(_PysharkGsmtap):
+    """APDU Source [provider] class for reading GSMTAP from a PCAP
+    file via pyshark, which in turn uses tshark (part of wireshark).
+    """
+    def __init__(self, pcap_filename):
+        """
+        Args:
+            pcap_filename: File name of the pcap file to be opened
+        """
+        pyshark_inst = pyshark.FileCapture(pcap_filename, display_filter='gsm_sim', use_json=True, keep_packets=False)
+        super().__init__(pyshark_inst)

pySim/apdu_source/pyshark_rspro.py

@@ -0,0 +1,158 @@
+# coding=utf-8
+
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import logging
+from typing import Tuple
+import pyshark
+
+from pySim.utils import h2b
+from pySim.apdu import Tpdu
+from . import ApduSource, PacketType, CardReset
+
+logger = logging.getLogger(__name__)
+
+class _PysharkRspro(ApduSource):
+    """APDU Source [provider] base class for reading RSPRO (osmo-remsim) via tshark."""
+
+    def __init__(self, pyshark_inst):
+        self.pyshark = pyshark_inst
+        self.bank_id = None
+        self.bank_slot = None
+        self.cmd_tpdu = None
+        super().__init__()
+
+    @staticmethod
+    def get_bank_slot(bank_slot) -> Tuple[int, int]:
+        """Convert a 'bankSlot_element' field into a tuple of bank_id, slot_nr"""
+        bank_id = bank_slot.get_field('bankId')
+        slot_nr = bank_slot.get_field('slotNr')
+        return int(bank_id), int(slot_nr)
+
+    @staticmethod
+    def get_client_slot(client_slot) -> Tuple[int, int]:
+        """Convert a 'clientSlot_element' field into a tuple of client_id, slot_nr"""
+        client_id = client_slot.get_field('clientId')
+        slot_nr = client_slot.get_field('slotNr')
+        return int(client_id), int(slot_nr)
+
+    @staticmethod
+    def get_pstatus(pstatus) -> Tuple[int, int, int]:
+        """Convert a 'slotPhysStatus_element' field into a tuple of vcc, reset, clk"""
+        vccPresent = int(pstatus.get_field('vccPresent'))
+        resetActive = int(pstatus.get_field('resetActive'))
+        clkActive = int(pstatus.get_field('clkActive'))
+        return vccPresent, resetActive, clkActive
+
+    def read_packet(self) -> PacketType:
+        p = self.pyshark.next()
+        return self._parse_packet(p)
+
+    def _set_or_verify_bank_slot(self, bsl: Tuple[int, int]):
+        """Keep track of the bank:slot to make sure we don't mix traces of multiple cards"""
+        if not self.bank_id:
+            self.bank_id = bsl[0]
+            self.bank_slot = bsl[1]
+        else:
+            if self.bank_id != bsl[0] or self.bank_slot != bsl[1]:
+                raise ValueError('Received data for unexpected B(%u:%u)' % (bsl[0], bsl[1]))
+
+    def _parse_packet(self, p) -> PacketType:
+        rspro_layer = p['rspro']
+        #print("Layer: %s" %  rspro_layer)
+        rspro_element = rspro_layer.get_field('RsproPDU_element')
+        #print("Element: %s" % rspro_element)
+        msg_type = rspro_element.get_field('msg')
+        rspro_msg = rspro_element.get_field('msg_tree')
+        if msg_type == '12': # tpduModemToCard
+            modem2card = rspro_msg.get_field('tpduModemToCard_element')
+            #print(modem2card)
+            client_slot = modem2card.get_field('fromClientSlot_element')
+            csl = self.get_client_slot(client_slot)
+            bank_slot = modem2card.get_field('toBankSlot_element')
+            bsl = self.get_bank_slot(bank_slot)
+            self._set_or_verify_bank_slot(bsl)
+            data = modem2card.get_field('data').replace(':','')
+            logger.debug("C(%u:%u) -> B(%u:%u): %s", csl[0], csl[1], bsl[0], bsl[1], data)
+            # store the CMD portion until the RSP portion arrives later
+            self.cmd_tpdu = h2b(data)
+        elif msg_type == '13': # tpduCardToModem
+            card2modem = rspro_msg.get_field('tpduCardToModem_element')
+            #print(card2modem)
+            client_slot = card2modem.get_field('toClientSlot_element')
+            csl = self.get_client_slot(client_slot)
+            bank_slot = card2modem.get_field('fromBankSlot_element')
+            bsl = self.get_bank_slot(bank_slot)
+            self._set_or_verify_bank_slot(bsl)
+            data = card2modem.get_field('data').replace(':','')
+            logger.debug("C(%u:%u) <- B(%u:%u): %s", csl[0], csl[1], bsl[0], bsl[1], data)
+            rsp_tpdu = h2b(data)
+            if self.cmd_tpdu:
+                # combine this R-TPDU with the C-TPDU we saw earlier
+                r = Tpdu(self.cmd_tpdu, rsp_tpdu)
+                self.cmd_tpdu = False
+                return r
+        elif msg_type == '14': # clientSlotStatus
+            cl_slotstatus = rspro_msg.get_field('clientSlotStatusInd_element')
+            #print(cl_slotstatus)
+            client_slot = cl_slotstatus.get_field('fromClientSlot_element')
+            bank_slot = cl_slotstatus.get_field('toBankSlot_element')
+            slot_pstatus = cl_slotstatus.get_field('slotPhysStatus_element')
+            vccPresent, resetActive, clkActive = self.get_pstatus(slot_pstatus)
+            if vccPresent and clkActive and not resetActive:
+                logger.debug("RESET")
+                #TODO: extract ATR from RSPRO message and use it here
+                return CardReset(None)
+        else:
+            print("Unhandled msg type %s: %s" % (msg_type, rspro_msg))
+
+
+class PysharkRsproPcap(_PysharkRspro):
+    """APDU Source [provider] class for reading RSPRO (osmo-remsim) from a PCAP
+    file via pyshark, which in turn uses tshark (part of wireshark).
+
+    In order to use this, you need a wireshark patched with RSPRO support,
+    such as can be found at https://gitea.osmocom.org/osmocom/wireshark/src/branch/laforge/rspro
+
+    A STANDARD UPSTREAM WIRESHARK *DOES NOT WORK*.
+    """
+    def __init__(self, pcap_filename):
+        """
+        Args:
+            pcap_filename: File name of the pcap file to be opened
+        """
+        pyshark_inst = pyshark.FileCapture(pcap_filename, display_filter='rspro', use_json=True, keep_packets=False)
+        super().__init__(pyshark_inst)
+
+class PysharkRsproLive(_PysharkRspro):
+    """APDU Source [provider] class for reading RSPRO (osmo-remsim) from a live capture
+    via pyshark, which in turn uses tshark (part of wireshark).
+
+    In order to use this, you need a wireshark patched with RSPRO support,
+    such as can be found at https://gitea.osmocom.org/osmocom/wireshark/src/branch/laforge/rspro
+
+    A STANDARD UPSTREAM WIRESHARK *DOES NOT WORK*.
+    """
+    def __init__(self, interface, bpf_filter='tcp port 9999 or tcp port 9998'):
+        """
+        Args:
+            interface: Network interface name to capture packets on (like "eth0")
+            bfp_filter: libpcap capture filter to use
+        """
+        pyshark_inst = pyshark.LiveCapture(interface=interface, display_filter='rspro', bpf_filter=bpf_filter,
+                                           use_json=True)
+        super().__init__(pyshark_inst)

pySim/app.py

@@ -0,0 +1,109 @@
+# (C) 2021-2023 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from typing import Tuple
+
+from pySim.transport import LinkBase
+from pySim.commands import SimCardCommands
+from pySim.filesystem import CardModel, CardApplication
+from pySim.cards import card_detect, SimCardBase, UiccCardBase
+from pySim.runtime import RuntimeState
+from pySim.profile import CardProfile
+from pySim.cdma_ruim import CardProfileRUIM
+from pySim.ts_102_221 import CardProfileUICC
+from pySim.utils import all_subclasses
+
+# we need to import this module so that the SysmocomSJA2 sub-class of
+# CardModel is created, which will add the ATR-based matching and
+# calling of SysmocomSJA2.add_files.  See  CardModel.apply_matching_models
+import pySim.sysmocom_sja2
+
+# we need to import these modules so that the various sub-classes of
+# CardProfile are created, which will be used in init_card() to iterate
+# over all known CardProfile sub-classes.
+import pySim.ts_31_102
+import pySim.ts_31_103
+import pySim.ts_31_104
+import pySim.ara_m
+import pySim.global_platform
+import pySim.euicc
+
+def init_card(sl: LinkBase) -> Tuple[RuntimeState, SimCardBase]:
+    """
+    Detect card in reader and setup card profile and runtime state. This
+    function must be called at least once on startup. The card and runtime
+    state object (rs) is required for all pySim-shell commands.
+    """
+
+    # Create command layer
+    scc = SimCardCommands(transport=sl)
+
+    # Wait up to three seconds for a card in reader and try to detect
+    # the card type.
+    print("Waiting for card...")
+    sl.wait_for_card(3)
+
+    generic_card = False
+    card = card_detect(scc)
+    if card is None:
+        print("Warning: Could not detect card type - assuming a generic card type...")
+        card = SimCardBase(scc)
+        generic_card = True
+
+    profile = CardProfile.pick(scc)
+    if profile is None:
+        # It is not an unrecoverable error in case profile detection fails. It
+        # just means that pySim was unable to recognize the card profile. This
+        # may happen in particular with unprovisioned cards that do not have
+        # any files on them yet.
+        print("Unsupported card type!")
+        return None, card
+
+    # ETSI TS 102 221, Table 9.3 specifies a default for the PIN key
+    # references, however card manufactures may still decide to pick an
+    # arbitrary key reference. In case we run on a generic card class that is
+    # detected as an UICC, we will pick the key reference that is officially
+    # specified.
+    if generic_card and isinstance(profile, CardProfileUICC):
+        card._adm_chv_num = 0x0A
+
+    print("Info: Card is of type: %s" % str(profile))
+
+    # FIXME: this shouldn't really be here but somewhere else/more generic.
+    # We cannot do it within pySim/profile.py as that would create circular
+    # dependencies between the individual profiles and profile.py.
+    if isinstance(profile, CardProfileUICC):
+        for app_cls in all_subclasses(CardApplication):
+            # skip any intermediary sub-classes such as CardApplicationSD
+            if hasattr(app_cls, '_' + app_cls.__name__ + '__intermediate'):
+                continue
+            profile.add_application(app_cls())
+        # We have chosen SimCard() above, but we now know it actually is an UICC
+        # so it's safe to assume it supports USIM application (which we're adding above).
+        # IF we don't do this, we will have a SimCard but try USIM specific commands like
+        # the update_ust method (see https://osmocom.org/issues/6055)
+        if generic_card:
+            card = UiccCardBase(scc)
+
+    # Create runtime state with card profile
+    rs = RuntimeState(card, profile)
+
+    CardModel.apply_matching_models(scc, rs)
+
+    # inform the transport that we can do context-specific SW interpretation
+    sl.set_sw_interpreter(rs)
+
+    return rs, card

pySim/ara_m.py

@@ -0,0 +1,412 @@
+# -*- coding: utf-8 -*-
+
+# without this, pylint will fail when inner classes are used
+# within the 'nested' kwarg of our TlvMeta metaclass on python 3.7 :(
+# pylint: disable=undefined-variable
+
+"""
+Support for the Secure Element Access Control, specifically the ARA-M inside an UICC.
+"""
+
+#
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+from construct import GreedyBytes, GreedyString, Struct, Enum, Int8ub, Int16ub
+from construct import Optional as COptional
+from pySim.construct import *
+from pySim.filesystem import *
+from pySim.tlv import *
+from pySim.utils import Hexstr
+import pySim.global_platform
+
+# various BER-TLV encoded Data Objects (DOs)
+
+
+class AidRefDO(BER_TLV_IE, tag=0x4f):
+    # SEID v1.1 Table 6-3
+    _construct = HexAdapter(GreedyBytes)
+
+
+class AidRefEmptyDO(BER_TLV_IE, tag=0xc0):
+    # SEID v1.1 Table 6-3
+    pass
+
+
+class DevAppIdRefDO(BER_TLV_IE, tag=0xc1):
+    # SEID v1.1 Table 6-4
+    _construct = HexAdapter(GreedyBytes)
+
+
+class PkgRefDO(BER_TLV_IE, tag=0xca):
+    # Android UICC Carrier Privileges specific extension, see https://source.android.com/devices/tech/config/uicc
+    _construct = Struct('package_name_string'/GreedyString("ascii"))
+
+
+class RefDO(BER_TLV_IE, tag=0xe1, nested=[AidRefDO, AidRefEmptyDO, DevAppIdRefDO, PkgRefDO]):
+    # SEID v1.1 Table 6-5
+    pass
+
+
+class ApduArDO(BER_TLV_IE, tag=0xd0):
+    # SEID v1.1 Table 6-8
+    def _from_bytes(self, do: bytes):
+        if len(do) == 1:
+            if do[0] == 0x00:
+                self.decoded = {'generic_access_rule': 'never'}
+                return self.decoded
+            if do[0] == 0x01:
+                self.decoded = {'generic_access_rule': 'always'}
+                return self.decoded
+            return ValueError('Invalid 1-byte generic APDU access rule')
+        else:
+            if len(do) % 8:
+                return ValueError('Invalid non-modulo-8 length of APDU filter: %d' % len(do))
+            self.decoded['apdu_filter'] = []
+            offset = 0
+            while offset < len(do):
+                self.decoded['apdu_filter'] += {'header': b2h(do[offset:offset+4]),
+                                                'mask': b2h(do[offset+4:offset+8])}
+            self.decoded = res
+            return res
+
+    def _to_bytes(self):
+        if 'generic_access_rule' in self.decoded:
+            if self.decoded['generic_access_rule'] == 'never':
+                return b'\x00'
+            if self.decoded['generic_access_rule'] == 'always':
+                return b'\x01'
+            return ValueError('Invalid 1-byte generic APDU access rule')
+        else:
+            if not 'apdu_filter' in self.decoded:
+                return ValueError('Invalid APDU AR DO')
+            filters = self.decoded['apdu_filter']
+            res = b''
+            for f in filters:
+                if not 'header' in f or not 'mask' in f:
+                    return ValueError('APDU filter must contain header and mask')
+                header_b = h2b(f['header'])
+                mask_b = h2b(f['mask'])
+                if len(header_b) != 4 or len(mask_b) != 4:
+                    return ValueError('APDU filter header and mask must each be 4 bytes')
+                res += header_b + mask_b
+            return res
+
+
+class NfcArDO(BER_TLV_IE, tag=0xd1):
+    # SEID v1.1 Table 6-9
+    _construct = Struct('nfc_event_access_rule' /
+                        Enum(Int8ub, never=0, always=1))
+
+
+class PermArDO(BER_TLV_IE, tag=0xdb):
+    # Android UICC Carrier Privileges specific extension, see https://source.android.com/devices/tech/config/uicc
+    # based on Table 6-8 of GlobalPlatform Device API Access Control v1.0
+    _construct = Struct('permissions'/HexAdapter(Bytes(8)))
+
+
+class ArDO(BER_TLV_IE, tag=0xe3, nested=[ApduArDO, NfcArDO, PermArDO]):
+    # SEID v1.1 Table 6-7
+    pass
+
+
+class RefArDO(BER_TLV_IE, tag=0xe2, nested=[RefDO, ArDO]):
+    # SEID v1.1 Table 6-6
+    pass
+
+
+class ResponseAllRefArDO(BER_TLV_IE, tag=0xff40, nested=[RefArDO]):
+    # SEID v1.1 Table 4-2
+    pass
+
+
+class ResponseArDO(BER_TLV_IE, tag=0xff50, nested=[ArDO]):
+    # SEID v1.1 Table 4-3
+    pass
+
+
+class ResponseRefreshTagDO(BER_TLV_IE, tag=0xdf20):
+    # SEID v1.1 Table 4-4
+    _construct = Struct('refresh_tag'/HexAdapter(Bytes(8)))
+
+
+class DeviceInterfaceVersionDO(BER_TLV_IE, tag=0xe6):
+    # SEID v1.1 Table 6-12
+    _construct = Struct('major'/Int8ub, 'minor'/Int8ub, 'patch'/Int8ub)
+
+
+class DeviceConfigDO(BER_TLV_IE, tag=0xe4, nested=[DeviceInterfaceVersionDO]):
+    # SEID v1.1 Table 6-10
+    pass
+
+
+class ResponseDeviceConfigDO(BER_TLV_IE, tag=0xff7f, nested=[DeviceConfigDO]):
+    # SEID v1.1 Table 5-14
+    pass
+
+
+class AramConfigDO(BER_TLV_IE, tag=0xe5, nested=[DeviceInterfaceVersionDO]):
+    # SEID v1.1 Table 6-11
+    pass
+
+
+class ResponseAramConfigDO(BER_TLV_IE, tag=0xdf21, nested=[AramConfigDO]):
+    # SEID v1.1 Table 4-5
+    pass
+
+
+class CommandStoreRefArDO(BER_TLV_IE, tag=0xf0, nested=[RefArDO]):
+    # SEID v1.1 Table 5-2
+    pass
+
+
+class CommandDelete(BER_TLV_IE, tag=0xf1, nested=[AidRefDO, AidRefEmptyDO, RefDO, RefArDO]):
+    # SEID v1.1 Table 5-4
+    pass
+
+
+class CommandUpdateRefreshTagDO(BER_TLV_IE, tag=0xf2):
+    # SEID V1.1 Table 5-6
+    pass
+
+
+class CommandRegisterClientAidsDO(BER_TLV_IE, tag=0xf7, nested=[AidRefDO, AidRefEmptyDO]):
+    # SEID v1.1 Table 5-7
+    pass
+
+
+class CommandGet(BER_TLV_IE, tag=0xf3, nested=[AidRefDO, AidRefEmptyDO]):
+    # SEID v1.1 Table 5-8
+    pass
+
+
+class CommandGetAll(BER_TLV_IE, tag=0xf4):
+    # SEID v1.1 Table 5-9
+    pass
+
+
+class CommandGetClientAidsDO(BER_TLV_IE, tag=0xf6):
+    # SEID v1.1 Table 5-10
+    pass
+
+
+class CommandGetNext(BER_TLV_IE, tag=0xf5):
+    # SEID v1.1 Table 5-11
+    pass
+
+
+class CommandGetDeviceConfigDO(BER_TLV_IE, tag=0xf8):
+    # SEID v1.1 Table 5-12
+    pass
+
+
+class ResponseAracAidDO(BER_TLV_IE, tag=0xff70, nested=[AidRefDO, AidRefEmptyDO]):
+    # SEID v1.1 Table 5-13
+    pass
+
+
+class BlockDO(BER_TLV_IE, tag=0xe7):
+    # SEID v1.1 Table 6-13
+    _construct = Struct('offset'/Int16ub, 'length'/Int8ub)
+
+
+# SEID v1.1 Table 4-1
+class GetCommandDoCollection(TLV_IE_Collection, nested=[RefDO, DeviceConfigDO]):
+    pass
+
+# SEID v1.1 Table 4-2
+
+
+class GetResponseDoCollection(TLV_IE_Collection, nested=[ResponseAllRefArDO, ResponseArDO,
+                                                         ResponseRefreshTagDO, ResponseAramConfigDO]):
+    pass
+
+# SEID v1.1 Table 5-1
+
+
+class StoreCommandDoCollection(TLV_IE_Collection,
+                               nested=[BlockDO, CommandStoreRefArDO, CommandDelete,
+                                       CommandUpdateRefreshTagDO, CommandRegisterClientAidsDO,
+                                       CommandGet, CommandGetAll, CommandGetClientAidsDO,
+                                       CommandGetNext, CommandGetDeviceConfigDO]):
+    pass
+
+
+# SEID v1.1 Section 5.1.2
+class StoreResponseDoCollection(TLV_IE_Collection,
+                                nested=[ResponseAllRefArDO, ResponseAracAidDO, ResponseDeviceConfigDO]):
+    pass
+
+
+class ADF_ARAM(CardADF):
+    def __init__(self, aid='a00000015141434c00', name='ADF.ARA-M', fid=None, sfid=None,
+                 desc='ARA-M Application'):
+        super().__init__(aid=aid, fid=fid, sfid=sfid, name=name, desc=desc)
+        self.shell_commands += [self.AddlShellCommands()]
+        files = []
+        self.add_files(files)
+
+    def decode_select_response(self, data_hex):
+        return pySim.global_platform.decode_select_response(data_hex)
+
+    @staticmethod
+    def xceive_apdu_tlv(tp, hdr: Hexstr, cmd_do, resp_cls, exp_sw='9000'):
+        """Transceive an APDU with the card, transparently encoding the command data from TLV
+        and decoding the response data tlv."""
+        if cmd_do:
+            cmd_do_enc = cmd_do.to_ie()
+            cmd_do_len = len(cmd_do_enc)
+            if cmd_do_len > 255:
+                return ValueError('DO > 255 bytes not supported yet')
+        else:
+            cmd_do_enc = b''
+            cmd_do_len = 0
+        c_apdu = hdr + ('%02x' % cmd_do_len) + b2h(cmd_do_enc)
+        (data, _sw) = tp.send_apdu_checksw(c_apdu, exp_sw)
+        if data:
+            if resp_cls:
+                resp_do = resp_cls()
+                resp_do.from_tlv(h2b(data))
+                return resp_do
+            return data
+        else:
+            return None
+
+    @staticmethod
+    def store_data(tp, do) -> bytes:
+        """Build the Command APDU for STORE DATA."""
+        return ADF_ARAM.xceive_apdu_tlv(tp, '80e29000', do, StoreResponseDoCollection)
+
+    @staticmethod
+    def get_all(tp):
+        return ADF_ARAM.xceive_apdu_tlv(tp, '80caff40', None, GetResponseDoCollection)
+
+    @staticmethod
+    def get_config(tp, v_major=0, v_minor=0, v_patch=1):
+        cmd_do = DeviceConfigDO()
+        cmd_do.from_dict([{'device_interface_version_do': {
+                         'major': v_major, 'minor': v_minor, 'patch': v_patch}}])
+        return ADF_ARAM.xceive_apdu_tlv(tp, '80cadf21', cmd_do, ResponseAramConfigDO)
+
+    @with_default_category('Application-Specific Commands')
+    class AddlShellCommands(CommandSet):
+        def do_aram_get_all(self, _opts):
+            """GET DATA [All] on the ARA-M Applet"""
+            res_do = ADF_ARAM.get_all(self._cmd.lchan.scc._tp)
+            if res_do:
+                self._cmd.poutput_json(res_do.to_dict())
+
+        def do_aram_get_config(self, _opts):
+            """Perform GET DATA [Config] on the ARA-M Applet: Tell it our version and retrieve its version."""
+            res_do = ADF_ARAM.get_config(self._cmd.lchan.scc._tp)
+            if res_do:
+                self._cmd.poutput_json(res_do.to_dict())
+
+        store_ref_ar_do_parse = argparse.ArgumentParser()
+        # REF-DO
+        store_ref_ar_do_parse.add_argument(
+            '--device-app-id', required=True, help='Identifies the specific device application that the rule appplies to. Hash of Certificate of Application Provider, or UUID. (20/32 hex bytes)')
+        aid_grp = store_ref_ar_do_parse.add_mutually_exclusive_group()
+        aid_grp.add_argument(
+            '--aid', help='Identifies the specific SE application for which rules are to be stored. Can be a partial AID, containing for example only the RID.  (5-16 hex bytes)')
+        aid_grp.add_argument('--aid-empty', action='store_true',
+                             help='No specific SE application, applies to all applications')
+        store_ref_ar_do_parse.add_argument(
+            '--pkg-ref', help='Full Android Java package name (up to 127 chars ASCII)')
+        # AR-DO
+        apdu_grp = store_ref_ar_do_parse.add_mutually_exclusive_group()
+        apdu_grp.add_argument(
+            '--apdu-never', action='store_true', help='APDU access is not allowed')
+        apdu_grp.add_argument(
+            '--apdu-always', action='store_true', help='APDU access is allowed')
+        apdu_grp.add_argument(
+            '--apdu-filter', help='APDU filter: 4 byte CLA/INS/P1/P2 followed by 4 byte mask (8 hex bytes)')
+        nfc_grp = store_ref_ar_do_parse.add_mutually_exclusive_group()
+        nfc_grp.add_argument('--nfc-always', action='store_true',
+                             help='NFC event access is allowed')
+        nfc_grp.add_argument('--nfc-never', action='store_true',
+                             help='NFC event access is not allowed')
+        store_ref_ar_do_parse.add_argument(
+            '--android-permissions', help='Android UICC Carrier Privilege Permissions (8 hex bytes)')
+
+        @cmd2.with_argparser(store_ref_ar_do_parse)
+        def do_aram_store_ref_ar_do(self, opts):
+            """Perform STORE DATA [Command-Store-REF-AR-DO] to store a (new) access rule."""
+            # REF
+            ref_do_content = []
+            if opts.aid is not None:
+                ref_do_content += [{'aid_ref_do': opts.aid}]
+            elif opts.aid_empty:
+                ref_do_content += [{'aid_ref_empty_do': None}]
+            ref_do_content += [{'dev_app_id_ref_do': opts.device_app_id}]
+            if opts.pkg_ref:
+                ref_do_content += [{'pkg_ref_do': {'package_name_string': opts.pkg_ref}}]
+            # AR
+            ar_do_content = []
+            if opts.apdu_never:
+                ar_do_content += [{'apdu_ar_od': {'generic_access_rule': 'never'}}]
+            elif opts.apdu_always:
+                ar_do_content += [{'apdu_ar_do': {'generic_access_rule': 'always'}}]
+            elif opts.apdu_filter:
+                # TODO: multiple filters
+                ar_do_content += [{'apdu_ar_do': {'apdu_filter': [opts.apdu_filter]}}]
+            if opts.nfc_always:
+                ar_do_content += [{'nfc_ar_do': {'nfc_event_access_rule': 'always'}}]
+            elif opts.nfc_never:
+                ar_do_content += [{'nfc_ar_do': {'nfc_event_access_rule': 'never'}}]
+            if opts.android_permissions:
+                ar_do_content += [{'perm_ar_do': {'permissions': opts.android_permissions}}]
+            d = [{'ref_ar_do': [{'ref_do': ref_do_content}, {'ar_do': ar_do_content}]}]
+            csrado = CommandStoreRefArDO()
+            csrado.from_dict(d)
+            res_do = ADF_ARAM.store_data(self._cmd.lchan.scc._tp, csrado)
+            if res_do:
+                self._cmd.poutput_json(res_do.to_dict())
+
+        def do_aram_delete_all(self, _opts):
+            """Perform STORE DATA [Command-Delete[all]] to delete all access rules."""
+            deldo = CommandDelete()
+            res_do = ADF_ARAM.store_data(self._cmd.lchan.scc._tp, deldo)
+            if res_do:
+                self._cmd.poutput_json(res_do.to_dict())
+
+
+# SEAC v1.1 Section 4.1.2.2 + 5.1.2.2
+sw_aram = {
+    'ARA-M': {
+        '6381': 'Rule successfully stored but an access rule already exists',
+        '6382': 'Rule successfully stored bu contained at least one unknown (discarded) BER-TLV',
+        '6581': 'Memory Problem',
+        '6700': 'Wrong Length in Lc',
+        '6981': 'DO is not supported by the ARA-M/ARA-C',
+        '6982': 'Security status not satisfied',
+        '6984': 'Rules have been updated and must be read again / logical channels in use',
+        '6985': 'Conditions not satisfied',
+        '6a80': 'Incorrect values in the command data',
+        '6a84': 'Rules have been updated and must be read again',
+        '6a86': 'Incorrect P1 P2',
+        '6a88': 'Referenced data not found',
+        '6a89': 'Conflicting access rule already exists in the Secure Element',
+        '6d00': 'Invalid instruction',
+        '6e00': 'Invalid class',
+    }
+}
+
+
+class CardApplicationARAM(CardApplication):
+    def __init__(self):
+        super().__init__('ARA-M', adf=ADF_ARAM(), sw=sw_aram)

pySim/card_handler.py

@@ -0,0 +1,146 @@
+# -*- coding: utf-8 -*-
+
+""" pySim: card handler utilities.  A 'card handler' is some method
+by which cards can be inserted/removed into the card reader.  For
+normal smart card readers, this has to be done manually.  However,
+there are also automatic card feeders.
+"""
+
+#
+# (C) 2019 by Sysmocom s.f.m.c. GmbH
+# All Rights Reserved
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import subprocess
+import sys
+import yaml
+
+from pySim.transport import LinkBase
+
+class CardHandlerBase:
+    """Abstract base class representing a mechanism for card insertion/removal."""
+
+    def __init__(self, sl: LinkBase):
+        self.sl = sl
+
+    def get(self, first: bool = False):
+        """Method called when pySim needs a new card to be inserted.
+
+        Args:
+                first : set to true when the get method is called the
+                        first time. This is required to prevent blocking
+                        when a card is already inserted into the reader.
+                        The reader API would not recognize that card as
+                        "new card" until it would be removed and re-inserted
+                        again.
+        """
+        print("Ready for Programming: ", end='')
+        self._get(first)
+
+    def error(self):
+        """Method called when pySim failed to program a card. Move card to 'bad' batch."""
+        print("Programming failed: ", end='')
+        self._error()
+
+    def done(self):
+        """Method called when pySim failed to program a card. Move card to 'good' batch."""
+        print("Programming successful: ", end='')
+        self._done()
+
+    def _get(self, first: bool = False):
+        pass
+
+    def _error(self):
+        pass
+
+    def _done(self):
+        pass
+
+
+class CardHandler(CardHandlerBase):
+    """Manual card handler: User is prompted to insert/remove card from the reader."""
+
+    def _get(self, first: bool = False):
+        print("Insert card now (or CTRL-C to cancel)")
+        self.sl.wait_for_card(newcardonly=not first)
+
+    def _error(self):
+        print("Remove card from reader")
+        print("")
+
+    def _done(self):
+        print("Remove card from reader")
+        print("")
+
+
+class CardHandlerAuto(CardHandlerBase):
+    """Automatic card handler: A machine is used to handle the cards."""
+
+    verbose = True
+
+    def __init__(self, sl: LinkBase, config_file: str):
+        super().__init__(sl)
+        print("Card handler Config-file: " + str(config_file))
+        with open(config_file) as cfg:
+            self.cmds = yaml.load(cfg, Loader=yaml.FullLoader)
+        self.verbose = self.cmds.get('verbose') is True
+
+    def __print_outout(self, out):
+        print("")
+        print("Card handler output:")
+        print("---------------------8<---------------------")
+        stdout = out[0].strip()
+        if len(stdout) > 0:
+            print("stdout:")
+            print(stdout)
+        stderr = out[1].strip()
+        if len(stderr) > 0:
+            print("stderr:")
+            print(stderr)
+        print("---------------------8<---------------------")
+        print("")
+
+    def __exec_cmd(self, command):
+        print("Card handler Commandline: " + str(command))
+
+        proc = subprocess.Popen(
+            [command], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+        out = proc.communicate()
+        rc = proc.returncode
+
+        if rc != 0 or self.verbose:
+            self.__print_outout(out)
+
+        if rc != 0:
+            print("")
+            print("Error: Card handler failure! (rc=" + str(rc) + ")")
+            sys.exit(rc)
+
+    def _get(self, first: bool = False):
+        print("Transporting card into the reader-bay...")
+        self.__exec_cmd(self.cmds['get'])
+        if self.sl:
+            self.sl.connect()
+
+    def _error(self):
+        print("Transporting card to the error-bin...")
+        self.__exec_cmd(self.cmds['error'])
+        print("")
+
+    def _done(self):
+        print("Transporting card into the collector bin...")
+        self.__exec_cmd(self.cmds['done'])
+        print("")

pySim/card_key_provider.py

@@ -0,0 +1,174 @@
+# coding=utf-8
+"""Obtaining card parameters (mostly key data) from external source.
+
+This module contains a base class and a concrete implementation of
+obtaining card key material (or other card-individual parameters) from
+an external data source.
+
+This is used e.g. to keep PIN/PUK data in some file on disk, avoiding
+the need of manually entering the related card-individual data on every
+operation with pySim-shell.
+"""
+
+# (C) 2021 by Sysmocom s.f.m.c. GmbH
+# All Rights Reserved
+#
+# Author: Philipp Maier
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import List, Dict, Optional
+
+import abc
+import csv
+
+card_key_providers = []  # type: List['CardKeyProvider']
+
+
+class CardKeyProvider(abc.ABC):
+    """Base class, not containing any concrete implementation."""
+
+    VALID_FIELD_NAMES = ['ICCID', 'ADM1',
+                         'IMSI', 'PIN1', 'PIN2', 'PUK1', 'PUK2']
+
+    # check input parameters, but do nothing concrete yet
+    def _verify_get_data(self, fields: List[str] = [], key: str = 'ICCID', value: str = "") -> Dict[str, str]:
+        """Verify multiple fields for identified card.
+
+        Args:
+                fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained
+                key : look-up key to identify card data, such as 'ICCID'
+                value : value for look-up key to identify card data
+        Returns:
+                dictionary of {field, value} strings for each requested field from 'fields'
+        """
+        for f in fields:
+            if f not in self.VALID_FIELD_NAMES:
+                raise ValueError("Requested field name '%s' is not a valid field name, valid field names are: %s" %
+                                 (f, str(self.VALID_FIELD_NAMES)))
+
+        if key not in self.VALID_FIELD_NAMES:
+            raise ValueError("Key field name '%s' is not a valid field name, valid field names are: %s" %
+                             (key, str(self.VALID_FIELD_NAMES)))
+
+        return {}
+
+    def get_field(self, field: str, key: str = 'ICCID', value: str = "") -> Optional[str]:
+        """get a single field from CSV file using a specified key/value pair"""
+        fields = [field]
+        result = self.get(fields, key, value)
+        return result.get(field)
+
+    @abc.abstractmethod
+    def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
+        """Get multiple card-individual fields for identified card.
+
+        Args:
+                fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained
+                key : look-up key to identify card data, such as 'ICCID'
+                value : value for look-up key to identify card data
+        Returns:
+                dictionary of {field, value} strings for each requested field from 'fields'
+        """
+
+
+class CardKeyProviderCsv(CardKeyProvider):
+    """Card key provider implementation that allows to query against a specified CSV file"""
+    csv_file = None
+    filename = None
+
+    def __init__(self, filename: str):
+        """
+        Args:
+                filename : file name (path) of CSV file containing card-individual key/data
+        """
+        self.csv_file = open(filename, 'r')
+        if not self.csv_file:
+            raise RuntimeError("Could not open CSV file '%s'" % filename)
+        self.filename = filename
+
+    def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
+        super()._verify_get_data(fields, key, value)
+
+        self.csv_file.seek(0)
+        cr = csv.DictReader(self.csv_file)
+        if not cr:
+            raise RuntimeError(
+                "Could not open DictReader for CSV-File '%s'" % self.filename)
+        cr.fieldnames = [field.upper() for field in cr.fieldnames]
+
+        rc = {}
+        for row in cr:
+            if row[key] == value:
+                for f in fields:
+                    if f in row:
+                        rc.update({f: row[f]})
+                    else:
+                        raise RuntimeError("CSV-File '%s' lacks column '%s'" %
+                                           (self.filename, f))
+        return rc
+
+
+def card_key_provider_register(provider: CardKeyProvider, provider_list=card_key_providers):
+    """Register a new card key provider.
+
+    Args:
+            provider : the to-be-registered provider
+            provider_list : override the list of providers from the global default
+    """
+    if not isinstance(provider, CardKeyProvider):
+        raise ValueError("provider is not a card data provier")
+    provider_list.append(provider)
+
+
+def card_key_provider_get(fields, key: str, value: str, provider_list=card_key_providers) -> Dict[str, str]:
+    """Query all registered card data providers for card-individual [key] data.
+
+    Args:
+            fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained
+            key : look-up key to identify card data, such as 'ICCID'
+            value : value for look-up key to identify card data
+            provider_list : override the list of providers from the global default
+    Returns:
+            dictionary of {field, value} strings for each requested field from 'fields'
+    """
+    for p in provider_list:
+        if not isinstance(p, CardKeyProvider):
+            raise ValueError(
+                "provider list contains element which is not a card data provier")
+        result = p.get(fields, key, value)
+        if result:
+            return result
+    return {}
+
+
+def card_key_provider_get_field(field: str, key: str, value: str, provider_list=card_key_providers) -> Optional[str]:
+    """Query all registered card data providers for a single field.
+
+    Args:
+            field : name valid field such as 'ADM1', 'PIN1', ... which is to be obtained
+            key : look-up key to identify card data, such as 'ICCID'
+            value : value for look-up key to identify card data
+            provider_list : override the list of providers from the global default
+    Returns:
+            dictionary of {field, value} strings for the requested field
+    """
+    for p in provider_list:
+        if not isinstance(p, CardKeyProvider):
+            raise ValueError(
+                "provider list contains element which is not a card data provier")
+        result = p.get_field(field, key, value)
+        if result:
+            return result
+    return None

pySim/cards.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """ pySim: Card programmation logic
@@ -6,6 +5,8 @@
 
 #
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
+# Copyright (C) 2011-2023  Harald Welte <laforge@gnumonks.org>
+# Copyright (C) 2017 Alexander.Chemeris <Alexander.Chemeris@gmail.com>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -21,239 +22,152 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from pySim.utils import b2h, swap_nibbles, rpad, lpad
+from typing import Optional, Tuple
+from pySim.ts_102_221 import EF_DIR
+from pySim.ts_51_011 import DF_GSM
+
+from pySim.utils import *
+from pySim.commands import Path, SimCardCommands
+
+class CardBase:
+    """General base class for some kind of telecommunications card."""
+    def __init__(self, scc: SimCardCommands):
+        self._scc = scc
+        self._aids = []
+
+    def reset(self) -> Optional[Hexstr]:
+        rc = self._scc.reset_card()
+        if rc == 1:
+            return self._scc.get_atr()
+        return None
+
+    def set_apdu_parameter(self, cla: Hexstr, sel_ctrl: Hexstr) -> None:
+        """Set apdu parameters (class byte and selection control bytes)"""
+        self._scc.cla_byte = cla
+        self._scc.sel_ctrl = sel_ctrl
+
+    def get_apdu_parameter(self) -> Tuple[Hexstr, Hexstr]:
+        """Get apdu parameters (class byte and selection control bytes)"""
+        return (self._scc.cla_byte, self._scc.sel_ctrl)
+
+    def erase(self):
+        print("warning: erasing is not supported for specified card type!")
+
+    def file_exists(self, fid: Path) -> bool:
+        res_arr = self._scc.try_select_path(fid)
+        for res in res_arr:
+            if res[1] != '9000':
+                return False
+        return True
+
+    def read_aids(self) -> List[Hexstr]:
+        # a non-UICC doesn't have any applications. Convenience helper to avoid
+        # callers having to do hasattr('read_aids') ahead of every call.
+        return []
 
 
-class Card(object):
+class SimCardBase(CardBase):
+    """Here we only add methods for commands specified in TS 51.011, without
+    any higher-layer processing."""
+    name = 'SIM'
 
-	def __init__(self, scc):
-		self._scc = scc
+    def __init__(self, scc: SimCardCommands):
+        super().__init__(scc)
+        self._scc.cla_byte = "A0"
+        self._scc.sel_ctrl = "0000"
 
-	def _e_iccid(self, iccid):
-		return swap_nibbles(iccid)
-
-	def _e_imsi(self, imsi):
-		"""Converts a string imsi into the value of the EF"""
-		l = (len(imsi) + 1) // 2	# Required bytes
-		oe = len(imsi) & 1			# Odd (1) / Even (0)
-		ei = '%02x' % l + swap_nibbles(lpad('%01x%s' % ((oe<<3)|1, imsi), 16))
-		return ei
-
-	def _e_plmn(self, mcc, mnc):
-		"""Converts integer MCC/MNC into 6 bytes for EF"""
-		return swap_nibbles(lpad('%d' % mcc, 3) + lpad('%d' % mnc, 3))
-
-	def reset(self):
-		self._scc.reset_card()
+    def probe(self) -> bool:
+        df_gsm = DF_GSM()
+        return self.file_exists(df_gsm.fid)
 
 
-class _MagicSimBase(Card):
-	"""
-	Theses cards uses several record based EFs to store the provider infos,
-	each possible provider uses a specific record number in each EF. The
-	indexes used are ( where N is the number of providers supported ) :
-	 - [2 .. N+1] for the operator name
-     - [1 .. N] for the programable EFs
+class UiccCardBase(SimCardBase):
+    name = 'UICC'
 
-	* 3f00/7f4d/8f0c : Operator Name
+    def __init__(self, scc: SimCardCommands):
+        super().__init__(scc)
+        self._scc.cla_byte = "00"
+        self._scc.sel_ctrl = "0004"  # request an FCP
+        # See also: ETSI TS 102 221, Table 9.3
+        self._adm_chv_num = 0x0A
 
-	bytes 0-15 : provider name, padded with 0xff
-	byte  16   : length of the provider name
-	byte  17   : 01 for valid records, 00 otherwise
+    def probe(self) -> bool:
+        # EF.DIR is a mandatory EF on all ICCIDs; however it *may* also exist on a TS 51.011 SIM
+        ef_dir = EF_DIR()
+        return self.file_exists(ef_dir.fid)
 
-	* 3f00/7f4d/8f0d : Programmable Binary EFs
+    def read_aids(self) -> List[Hexstr]:
+        """Fetch all the AIDs present on UICC"""
+        self._aids = []
+        try:
+            ef_dir = EF_DIR()
+            # Find out how many records the EF.DIR has
+            # and store all the AIDs in the UICC
+            rec_cnt = self._scc.record_count(ef_dir.fid)
+            for i in range(0, rec_cnt):
+                rec = self._scc.read_record(ef_dir.fid, i + 1)
+                if (rec[0][0:2], rec[0][4:6]) == ('61', '4f') and len(rec[0]) > 12 \
+                        and rec[0][8:8 + int(rec[0][6:8], 16) * 2] not in self._aids:
+                    self._aids.append(rec[0][8:8 + int(rec[0][6:8], 16) * 2])
+        except Exception as e:
+            print("Can't read AIDs from SIM -- %s" % (str(e),))
+            self._aids = []
+        return self._aids
 
-	* 3f00/7f4d/8f0e : Programmable Record EFs
+    @staticmethod
+    def _get_aid(adf="usim") -> Optional[Hexstr]:
+        aid_map = {}
+        # First (known) halves of the U/ISIM AID
+        aid_map["usim"] = "a0000000871002"
+        aid_map["isim"] = "a0000000871004"
+        adf = adf.lower()
+        if adf in aid_map:
+            return aid_map[adf]
+        return None
 
-	"""
+    def _complete_aid(self, aid: Hexstr) -> Optional[Hexstr]:
+        """find the complete version of an ADF.U/ISIM AID"""
+        # Find full AID by partial AID:
+        if is_hex(aid):
+            for aid_known in self._aids:
+                if len(aid_known) >= len(aid) and aid == aid_known[0:len(aid)]:
+                    return aid_known
+        return None
 
-	@classmethod
-	def autodetect(kls, scc):
-		try:
-			for p, l, t in kls._files.values():
-				if not t:
-					continue
-				if scc.record_size(['3f00', '7f4d', p]) != l:
-					return None
-		except:
-			return None
+    def adf_present(self, adf: str = "usim") -> bool:
+        """Check if the AID of the specified ADF is present in EF.DIR (call read_aids before use)"""
+        aid = self._get_aid(adf)
+        if aid:
+            aid_full = self._complete_aid(aid)
+            if aid_full:
+                return True
+        return False
 
-		return kls(scc)
+    def select_adf_by_aid(self, adf: str = "usim", scc: Optional[SimCardCommands] = None) -> Tuple[Optional[Hexstr], Optional[SwHexstr]]:
+        """Select ADF.U/ISIM in the Card using its full AID"""
+        # caller may pass a custom scc; we fall back to default
+        scc = scc or self._scc
+        if is_hex(adf):
+            aid = adf
+        else:
+            aid = self._get_aid(adf)
+        if aid:
+            aid_full = self._complete_aid(aid)
+            if aid_full:
+                return scc.select_adf(aid_full)
+            # If we cannot get the full AID, try with short AID
+            return scc.select_adf(aid)
+        return (None, None)
 
-	def _get_count(self):
-		"""
-		Selects the file and returns the total number of entries
-		and entry size
-		"""
-		f = self._files['name']
+def card_detect(scc: SimCardCommands) -> Optional[CardBase]:
+    # UICC always has higher preference, as a UICC might also contain a SIM application
+    uicc = UiccCardBase(scc)
+    if uicc.probe():
+        return uicc
 
-		r = self._scc.select_file(['3f00', '7f4d', f[0]])
-		rec_len = int(r[-1][28:30], 16)
-		tlen = int(r[-1][4:8],16)
-		rec_cnt = (tlen / rec_len) - 1;
+    # this is for detecting a real, classic TS 11.11 SIM card without any UICC support
+    sim = SimCardBase(scc)
+    if sim.probe():
+        return sim
 
-		if (rec_cnt < 1) or (rec_len != f[1]):
-			raise RuntimeError('Bad card type')
-
-		return rec_cnt
-
-	def program(self, p):
-		# Go to dir
-		self._scc.select_file(['3f00', '7f4d'])
-
-		# Home PLMN in PLMN_Sel format
-		hplmn = self._e_plmn(p['mcc'], p['mnc'])
-
-		# Operator name ( 3f00/7f4d/8f0c )
-		self._scc.update_record(self._files['name'][0], 2,
-			rpad(b2h(p['name']), 32)  + ('%02x' % len(p['name'])) + '01'
-		)
-
-		# ICCID/IMSI/Ki/HPLMN ( 3f00/7f4d/8f0d )
-		v = ''
-
-			# inline Ki
-		if self._ki_file is None:
-			v += p['ki']
-
-			# ICCID
-		v += '3f00' + '2fe2' + '0a' + self._e_iccid(p['iccid'])
-
-			# IMSI
-		v += '7f20' + '6f07' + '09' + self._e_imsi(p['imsi'])
-
-			# Ki
-		if self._ki_file:
-			v += self._ki_file + '10' + p['ki']
-
-			# PLMN_Sel
-		v+= '6f30' + '18' +  rpad(hplmn, 36)
-
-		self._scc.update_record(self._files['b_ef'][0], 1,
-			rpad(v, self._files['b_ef'][1]*2)
-		)
-
-		# SMSP ( 3f00/7f4d/8f0e )
-			# FIXME
-
-		# Write PLMN_Sel forcefully as well
-		r = self._scc.select_file(['3f00', '7f20', '6f30'])
-		tl = int(r[-1][4:8], 16)
-
-		hplmn = self._e_plmn(p['mcc'], p['mnc'])
-		self._scc.update_binary('6f30', hplmn + 'ff' * (tl-3))
-
-	def erase(self):
-		# Dummy
-		df = {}
-		for k, v in self._files.iteritems():
-			ofs = 1
-			fv = v[1] * 'ff'
-			if k == 'name':
-				ofs = 2
-				fv = fv[0:-4] + '0000'
-			df[v[0]] = (fv, ofs)
-
-		# Write
-		for n in range(0,self._get_count()):
-			for k, (msg, ofs) in df.iteritems():
-				self._scc.update_record(['3f00', '7f4d', k], n + ofs, msg)
-
-
-class SuperSim(_MagicSimBase):
-
-	name = 'supersim'
-
-	_files = {
-		'name' : ('8f0c', 18, True),
-		'b_ef' : ('8f0d', 74, True),
-		'r_ef' : ('8f0e', 50, True),
-	}
-
-	_ki_file = None
-
-
-class MagicSim(_MagicSimBase):
-
-	name = 'magicsim'
-
-	_files = {
-		'name' : ('8f0c', 18, True),
-		'b_ef' : ('8f0d', 130, True),
-		'r_ef' : ('8f0e', 102, False),
-	}
-
-	_ki_file = '6f1b'
-
-
-class FakeMagicSim(Card):
-	"""
-	Theses cards have a record based EF 3f00/000c that contains the provider
-	informations. See the program method for its format. The records go from
-	1 to N.
-	"""
-
-	name = 'fakemagicsim'
-
-	@classmethod
-	def autodetect(kls, scc):
-		try:
-			if scc.record_size(['3f00', '000c']) != 0x5a:
-				return None
-		except:
-			return None
-
-		return kls(scc)
-
-	def _get_infos(self):
-		"""
-		Selects the file and returns the total number of entries
-		and entry size
-		"""
-
-		r = self._scc.select_file(['3f00', '000c'])
-		rec_len = int(r[-1][28:30], 16)
-		tlen = int(r[-1][4:8],16)
-		rec_cnt = (tlen / rec_len) - 1;
-
-		if (rec_cnt < 1) or (rec_len != 0x5a):
-			raise RuntimeError('Bad card type')
-
-		return rec_cnt, rec_len
-
-	def program(self, p):
-		# Home PLMN
-		r = self._scc.select_file(['3f00', '7f20', '6f30'])
-		tl = int(r[-1][4:8], 16)
-
-		hplmn = self._e_plmn(p['mcc'], p['mnc'])
-		self._scc.update_binary('6f30', hplmn + 'ff' * (tl-3))
-
-		# Get total number of entries and entry size
-		rec_cnt, rec_len = self._get_infos()
-
-		# Set first entry
-		entry = (
-			'81' +								#  1b  Status: Valid & Active
-			rpad(b2h(p['name'][0:14]), 28) +	# 14b  Entry Name
-			self._e_iccid(p['iccid']) +			# 10b  ICCID
-			self._e_imsi(p['imsi']) +			#  9b  IMSI_len + id_type(9) + IMSI
-			p['ki'] +							# 16b  Ki
-			24*'f' + 'fd' + 24*'f' +			# 25b  (unknown ...)
-			rpad(p['smsp'], 20) +				# 10b  SMSP (padded with ff if needed)
-			10*'f'								#  5b  (unknown ...)
-		)
-		self._scc.update_record('000c', 1, entry)
-
-	def erase(self):
-		# Get total number of entries and entry size
-		rec_cnt, rec_len = self._get_infos()
-
-		# Erase all entries
-		entry = 'ff' * rec_len
-		for i in range(0, rec_cnt):
-			self._scc.update_record('000c', 1+i, entry)
-
-
-	# In order for autodetection ...
-_cards_classes = [ FakeMagicSim, SuperSim, MagicSim ]
+    return None

pySim/cdma_ruim.py

@@ -0,0 +1,206 @@
+# coding=utf-8
+"""R-UIM (Removable User Identity Module) card profile (see 3GPP2 C.S0023-D)
+
+(C) 2023 by Vadim Yanitskiy <fixeria@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import enum
+
+from construct import Bytewise, BitStruct, BitsInteger, Struct, FlagsEnum
+
+from pySim.utils import *
+from pySim.filesystem import *
+from pySim.profile import match_ruim
+from pySim.profile import CardProfile, CardProfileAddon
+from pySim.ts_51_011 import CardProfileSIM
+from pySim.ts_51_011 import DF_TELECOM, DF_GSM
+from pySim.ts_51_011 import EF_ServiceTable
+from pySim.construct import *
+
+
+# Mapping between CDMA Service Number and its description
+EF_CST_map = {
+    1 : 'CHV disable function',
+    2 : 'Abbreviated Dialing Numbers (ADN)',
+    3 : 'Fixed Dialing Numbers (FDN)',
+    4 : 'Short Message Storage (SMS)',
+    5 : 'HRPD',
+    6 : 'Enhanced Phone Book',
+    7 : 'Multi Media Domain (MMD)',
+    8 : 'SF_EUIMID-based EUIMID',
+    9 : 'MEID Support',
+    10 : 'Extension1',
+    11 : 'Extension2',
+    12 : 'SMS Parameters',
+    13 : 'Last Number Dialled (LND)',
+    14 : 'Service Category Program for BC-SMS',
+    15 : 'Messaging and 3GPD Extensions',
+    16 : 'Root Certificates',
+    17 : 'CDMA Home Service Provider Name',
+    18 : 'Service Dialing Numbers (SDN)',
+    19 : 'Extension3',
+    20 : '3GPD-SIP',
+    21 : 'WAP Browser',
+    22 : 'Java',
+    23 : 'Reserved for CDG',
+    24 : 'Reserved for CDG',
+    25 : 'Data Download via SMS Broadcast',
+    26 : 'Data Download via SMS-PP',
+    27 : 'Menu Selection',
+    28 : 'Call Control',
+    29 : 'Proactive R-UIM',
+    30 : 'AKA',
+    31 : 'IPv6',
+    32 : 'RFU',
+    33 : 'RFU',
+    34 : 'RFU',
+    35 : 'RFU',
+    36 : 'RFU',
+    37 : 'RFU',
+    38 : '3GPD-MIP',
+    39 : 'BCMCS',
+    40 : 'Multimedia Messaging Service (MMS)',
+    41 : 'Extension 8',
+    42 : 'MMS User Connectivity Parameters',
+    43 : 'Application Authentication',
+    44 : 'Group Identifier Level 1',
+    45 : 'Group Identifier Level 2',
+    46 : 'De-Personalization Control Keys',
+    47 : 'Cooperative Network List',
+}
+
+
+######################################################################
+# DF.CDMA
+######################################################################
+
+class EF_SPN(TransparentEF):
+    '''3.4.31 CDMA Home Service Provider Name'''
+
+    _test_de_encode = [
+        ( "010801536b796c696e6b204e57ffffffffffffffffffffffffffffffffffffffffffff",
+          { 'rfu1' : 0, 'show_in_hsa' : True, 'rfu2' : 0,
+            'char_encoding' : 8, 'lang_ind' : 1, 'spn' : 'Skylink NW' } ),
+    ]
+
+    def __init__(self, fid='6f41', sfid=None, name='EF.SPN',
+                 desc='Service Provider Name', size=(35, 35), **kwargs):
+        super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, **kwargs)
+        self._construct = BitStruct(
+            # Byte 1: Display Condition
+            'rfu1'/BitsRFU(7),
+            'show_in_hsa'/Flag,
+            # Byte 2: Character Encoding
+            'rfu2'/BitsRFU(3),
+            'char_encoding'/BitsInteger(5), # see C.R1001-G
+            # Byte 3: Language Indicator
+            'lang_ind'/BitsInteger(8), # see C.R1001-G
+            # Bytes 4-35: Service Provider Name
+            'spn'/Bytewise(GsmString(32))
+        )
+
+class EF_AD(TransparentEF):
+    '''3.4.33 Administrative Data'''
+
+    _test_de_encode = [
+        ( "000000", { 'ms_operation_mode' : 'normal', 'additional_info' : '0000', 'rfu' : '' } ),
+    ]
+    _test_no_pad = True
+
+    class OP_MODE(enum.IntEnum):
+        normal = 0x00
+        type_approval = 0x80
+        normal_and_specific_facilities = 0x01
+        type_approval_and_specific_facilities = 0x81
+        maintenance_off_line = 0x02
+        cell_test = 0x04
+
+    def __init__(self, fid='6f43', sfid=None, name='EF.AD',
+                 desc='Service Provider Name', size=(3, None), **kwargs):
+        super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, **kwargs)
+        self._construct = Struct(
+            # Byte 1: Display Condition
+            'ms_operation_mode'/Enum(Byte, self.OP_MODE),
+            # Bytes 2-3: Additional information
+            'additional_info'/HexAdapter(Bytes(2)),
+            # Bytes 4..: RFU
+            'rfu'/HexAdapter(GreedyBytesRFU),
+        )
+
+
+class EF_SMS(LinFixedEF):
+    '''3.4.27 Short Messages'''
+    def __init__(self, fid='6f3c', sfid=None, name='EF.SMS', desc='Short messages', **kwargs):
+        super().__init__(fid, sfid=sfid, name=name, desc=desc, rec_len=(2, 255), **kwargs)
+        self._construct = Struct(
+            # Byte 1: Status
+            'status'/BitStruct(
+                'rfu87'/BitsRFU(2),
+                'protection'/Flag,
+                'rfu54'/BitsRFU(2),
+                'status'/FlagsEnum(BitsInteger(2), read=0, to_be_read=1, sent=2, to_be_sent=3),
+                'used'/Flag,
+            ),
+            # Byte 2: Length
+            'length'/Int8ub,
+            # Bytes 3..: SMS Transport Layer Message
+            'tpdu'/Bytes(lambda ctx: ctx.length if ctx.status.used else 0),
+        )
+
+
+class DF_CDMA(CardDF):
+    def __init__(self):
+        super().__init__(fid='7f25', name='DF.CDMA',
+                         desc='CDMA related files (3GPP2 C.S0023-D)')
+        files = [
+            # TODO: lots of other files
+            EF_ServiceTable('6f32', None, 'EF.CST',
+                            'CDMA Service Table', table=EF_CST_map, size=(5, 16)),
+            EF_SPN(),
+            EF_AD(),
+            EF_SMS(),
+        ]
+        self.add_files(files)
+
+
+class CardProfileRUIM(CardProfile):
+    '''R-UIM card profile as per 3GPP2 C.S0023-D'''
+
+    ORDER = 2
+
+    def __init__(self):
+        super().__init__('R-UIM', desc='CDMA R-UIM Card', cla="a0",
+                         sel_ctrl="0000", files_in_mf=[DF_TELECOM(), DF_GSM(), DF_CDMA()])
+
+    @staticmethod
+    def decode_select_response(data_hex: str) -> object:
+        # TODO: Response parameters/data in case of DF_CDMA (section 2.6)
+        return CardProfileSIM.decode_select_response(data_hex)
+
+    @staticmethod
+    def match_with_card(scc: SimCardCommands) -> bool:
+        return match_ruim(scc)
+
+class AddonRUIM(CardProfileAddon):
+    """An Addon that can be found on on a combined SIM + RUIM or UICC + RUIM to support CDMA."""
+    def __init__(self):
+        files = [
+            DF_CDMA()
+        ]
+        super().__init__('RUIM', desc='CDMA RUIM', files_in_mf=files)
+
+    def probe(self, card: 'CardBase') -> bool:
+        return card.file_exists(self.files_in_mf[0].fid)

pySim/commands.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """ pySim: SIM Card commands according to ISO 7816-4 and TS 11.11
@@ -6,7 +5,7 @@
 
 #
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
-# Copyright (C) 2010       Harald Welte <laforge@gnumonks.org>
+# Copyright (C) 2010-2024  Harald Welte <laforge@gnumonks.org>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -22,74 +21,796 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from pySim.utils import rpad, b2h
+from typing import List, Tuple
+import typing # construct also has a Union, so we do typing.Union below
 
+from construct import Construct, Struct, Const, Select
+from construct import Optional as COptional
+from pySim.construct import LV, filter_dict
+from pySim.utils import rpad, lpad, b2h, h2b, sw_match, bertlv_encode_len, h2i, i2h, str_sanitize, expand_hex, SwMatchstr
+from pySim.utils import Hexstr, SwHexstr, ResTuple
+from pySim.exceptions import SwMatchError
+from pySim.transport import LinkBase
 
-class SimCardCommands(object):
-	def __init__(self, transport):
-		self._tp = transport;
+# A path can be either just a FID or a list of FID
+Path = typing.Union[Hexstr, List[Hexstr]]
 
-	def select_file(self, dir_list):
-		rv = []
-		for i in dir_list:
-			data, sw = self._tp.send_apdu_checksw("a0a4000002" + i)
-			rv.append(data)
-		return rv
+def lchan_nr_to_cla(cla: int, lchan_nr: int) -> int:
+    """Embed a logical channel number into the CLA byte."""
+    # TS 102 221 10.1.1 Coding of Class Byte
+    if lchan_nr < 4:
+        # standard logical channel number
+        if cla >> 4 in [0x0, 0xA, 0x8]:
+            return (cla & 0xFC) | (lchan_nr & 3)
+        else:
+            raise ValueError('Undefined how to use CLA %2X with logical channel %u' % (cla, lchan_nr))
+    elif lchan_nr < 16:
+        # extended logical channel number
+        if cla >> 6 in [1, 3]:
+            return (cla & 0xF0) | ((lchan_nr - 4) & 0x0F)
+        else:
+            raise ValueError('Undefined how to use CLA %2X with logical channel %u' % (cla, lchan_nr))
+    else:
+        raise ValueError('logical channel outside of range 0 .. 15')
 
-	def read_binary(self, ef, length=None, offset=0):
-		if not hasattr(type(ef), '__iter__'):
-			ef = [ef]
-		r = self.select_file(ef)
-		if length is None:
-			length = int(r[-1][4:8], 16) - offset
-		pdu = 'a0b0%04x%02x' % (offset, (min(256, length) & 0xff))
-		return self._tp.send_apdu(pdu)
+def cla_with_lchan(cla_byte: Hexstr, lchan_nr: int) -> Hexstr:
+    """Embed a logical channel number into the hex-string encoded CLA value."""
+    cla_int = h2i(cla_byte)[0]
+    return i2h([lchan_nr_to_cla(cla_int, lchan_nr)])
 
-	def update_binary(self, ef, data, offset=0):
-		if not hasattr(type(ef), '__iter__'):
-			ef = [ef]
-		self.select_file(ef)
-		pdu = 'a0d6%04x%02x' % (offset, len(data)/2) + data
-		return self._tp.send_apdu(pdu)
+class SimCardCommands:
+    """Class providing methods for various card-specific commands such as SELECT, READ BINARY, etc.
+    Historically one instance exists below CardBase, but with the introduction of multiple logical
+    channels there can be multiple instances.  The lchan number will then be patched into the CLA
+    byte by the respective instance. """
+    def __init__(self, transport: LinkBase, lchan_nr: int = 0):
+        self._tp = transport
+        self._cla_byte = None
+        self.sel_ctrl = "0000"
+        self.lchan_nr = lchan_nr
+        # invokes the setter below
+        self.cla_byte = "a0"
+        self.scp = None # Secure Channel Protocol
 
-	def read_record(self, ef, rec_no):
-		if not hasattr(type(ef), '__iter__'):
-			ef = [ef]
-		r = self.select_file(ef)
-		rec_length = int(r[-1][28:30], 16)
-		pdu = 'a0b2%02x04%02x' % (rec_no, rec_length)
-		return self._tp.send_apdu(pdu)
+    def fork_lchan(self, lchan_nr: int) -> 'SimCardCommands':
+        """Fork a per-lchan specific SimCardCommands instance off the current instance."""
+        ret = SimCardCommands(transport = self._tp, lchan_nr = lchan_nr)
+        ret.cla_byte = self._cla_byte
+        ret.sel_ctrl = self.sel_ctrl
+        return ret
 
-	def update_record(self, ef, rec_no, data, force_len=False):
-		if not hasattr(type(ef), '__iter__'):
-			ef = [ef]
-		r = self.select_file(ef)
-		if not force_len:
-			rec_length = int(r[-1][28:30], 16)
-			if (len(data)/2 != rec_length):
-				raise ValueError('Invalid data length (expected %d, got %d)' % (rec_length, len(data)/2))
-		else:
-			rec_length = len(data)/2
-		pdu = ('a0dc%02x04%02x' % (rec_no, rec_length)) + data
-		return self._tp.send_apdu(pdu)
+    @property
+    def cla_byte(self) -> Hexstr:
+        """Return the (cached) patched default CLA byte for this card."""
+        return self._cla4lchan
 
-	def record_size(self, ef):
-		r = self.select_file(ef)
-		return int(r[-1][28:30], 16)
+    @property
+    def max_cmd_len(self) -> int:
+        """Maximum length of the command apdu data section. Depends on secure channel protocol used."""
+        if self.scp:
+            return 255 - self.scp.overhead
+        else:
+            return 255
 
-	def record_count(self, ef):
-		r = self.select_file(ef)
-		return int(r[-1][4:8], 16) // int(r[-1][28:30], 16)
+    @cla_byte.setter
+    def cla_byte(self, new_val: Hexstr):
+        """Set the (raw, without lchan) default CLA value for this card."""
+        self._cla_byte = new_val
+        # compute cached result
+        self._cla4lchan = cla_with_lchan(self._cla_byte, self.lchan_nr)
 
-	def run_gsm(self, rand):
-		if len(rand) != 32:
-			raise ValueError('Invalid rand')
-		self.select_file(['3f00', '7f20'])
-		return self._tp.send_apdu('a088000010' + rand)
+    def cla4lchan(self, cla: Hexstr) -> Hexstr:
+        """Compute the lchan-patched value of the given CLA value. If no CLA
+        value is provided as argument, the lchan-patched version of the SimCardCommands._cla_byte
+        value is used. Most commands will use the latter, while some wish to override it and
+        can pass it as argument here."""
+        if not cla:
+            # return cached result to avoid re-computing this over and over again
+            return self._cla4lchan
+        else:
+            return cla_with_lchan(cla, self.lchan_nr)
 
-	def reset_card(self):
-		return self._tp.reset_card()
+    def send_apdu(self, pdu: Hexstr) -> ResTuple:
+        """Sends an APDU and auto fetch response data
 
-	def verify_chv(self, chv_no, code):
-		fc = rpad(b2h(code), 16)
-		return self._tp.send_apdu('a02000' + ('%02x' % chv_no) + '08' + fc)
+        Args:
+           pdu : string of hexadecimal characters (ex. "A0A40000023F00")
+        Returns:
+           tuple(data, sw), where
+                        data : string (in hex) of returned data (ex. "074F4EFFFF")
+                        sw   : string (in hex) of status word (ex. "9000")
+        """
+        if self.scp:
+            return self.scp.send_apdu_wrapper(self._tp.send_apdu, pdu)
+        else:
+            return self._tp.send_apdu(pdu)
+
+    def send_apdu_checksw(self, pdu: Hexstr, sw: SwMatchstr = "9000") -> ResTuple:
+        """Sends an APDU and check returned SW
+
+        Args:
+           pdu : string of hexadecimal characters (ex. "A0A40000023F00")
+           sw : string of 4 hexadecimal characters (ex. "9000"). The user may mask out certain
+                        digits using a '?' to add some ambiguity if needed.
+        Returns:
+                tuple(data, sw), where
+                        data : string (in hex) of returned data (ex. "074F4EFFFF")
+                        sw   : string (in hex) of status word (ex. "9000")
+        """
+        if self.scp:
+            return self.scp.send_apdu_wrapper(self._tp.send_apdu_checksw, pdu, sw)
+        else:
+            return self._tp.send_apdu_checksw(pdu, sw)
+
+    def send_apdu_constr(self, cla: Hexstr, ins: Hexstr, p1: Hexstr, p2: Hexstr, cmd_constr: Construct,
+                         cmd_data: Hexstr, resp_constr: Construct) -> Tuple[dict, SwHexstr]:
+        """Build and sends an APDU using a 'construct' definition; parses response.
+
+        Args:
+                cla : string (in hex) ISO 7816 class byte
+                ins : string (in hex) ISO 7816 instruction byte
+                p1 : string (in hex) ISO 7116 Parameter 1 byte
+                p2 : string (in hex) ISO 7116 Parameter 2 byte
+                cmd_cosntr : defining how to generate binary APDU command data
+                cmd_data : command data passed to cmd_constr
+                resp_cosntr : defining how to decode  binary APDU response data
+        Returns:
+                Tuple of (decoded_data, sw)
+        """
+        cmd = cmd_constr.build(cmd_data) if cmd_data else ''
+        p3 = i2h([len(cmd)])
+        pdu = ''.join([cla, ins, p1, p2, p3, b2h(cmd)])
+        (data, sw) = self.send_apdu(pdu)
+        if data:
+            # filter the resulting dict to avoid '_io' members inside
+            rsp = filter_dict(resp_constr.parse(h2b(data)))
+        else:
+            rsp = None
+        return (rsp, sw)
+
+    def send_apdu_constr_checksw(self, cla: Hexstr, ins: Hexstr, p1: Hexstr, p2: Hexstr,
+                                 cmd_constr: Construct, cmd_data: Hexstr, resp_constr: Construct,
+                                 sw_exp: SwMatchstr="9000") -> Tuple[dict, SwHexstr]:
+        """Build and sends an APDU using a 'construct' definition; parses response.
+
+        Args:
+                cla : string (in hex) ISO 7816 class byte
+                ins : string (in hex) ISO 7816 instruction byte
+                p1 : string (in hex) ISO 7116 Parameter 1 byte
+                p2 : string (in hex) ISO 7116 Parameter 2 byte
+                cmd_cosntr : defining how to generate binary APDU command data
+                cmd_data : command data passed to cmd_constr
+                resp_cosntr : defining how to decode  binary APDU response data
+                exp_sw : string (in hex) of status word (ex. "9000")
+        Returns:
+                Tuple of (decoded_data, sw)
+        """
+        (rsp, sw) = self.send_apdu_constr(cla, ins,
+                                          p1, p2, cmd_constr, cmd_data, resp_constr)
+        if not sw_match(sw, sw_exp):
+            raise SwMatchError(sw, sw_exp.lower(), self._tp.sw_interpreter)
+        return (rsp, sw)
+
+    # Extract a single FCP item from TLV
+    def __parse_fcp(self, fcp: Hexstr):
+        # see also: ETSI TS 102 221, chapter 11.1.1.3.1 Response for MF,
+        # DF or ADF
+        from pytlv.TLV import TLV
+        tlvparser = TLV(['82', '83', '84', 'a5', '8a', '8b',
+                        '8c', '80', 'ab', 'c6', '81', '88'])
+
+        # pytlv is case sensitive!
+        fcp = fcp.lower()
+
+        if fcp[0:2] != '62':
+            raise ValueError(
+                'Tag of the FCP template does not match, expected 62 but got %s' % fcp[0:2])
+
+        # Unfortunately the spec is not very clear if the FCP length is
+        # coded as one or two byte vale, so we have to try it out by
+        # checking if the length of the remaining TLV string matches
+        # what we get in the length field.
+        # See also ETSI TS 102 221, chapter 11.1.1.3.0 Base coding.
+        exp_tlv_len = int(fcp[2:4], 16)
+        if len(fcp[4:]) // 2 == exp_tlv_len:
+            skip = 4
+        else:
+            exp_tlv_len = int(fcp[2:6], 16)
+            if len(fcp[4:]) // 2 == exp_tlv_len:
+                skip = 6
+
+        # Skip FCP tag and length
+        tlv = fcp[skip:]
+        return tlvparser.parse(tlv)
+
+    # Tell the length of a record by the card response
+    # USIMs respond with an FCP template, which is different
+    # from what SIMs responds. See also:
+    # USIM: ETSI TS 102 221, chapter 11.1.1.3 Response Data
+    # SIM: GSM 11.11, chapter 9.2.1 SELECT
+    def __record_len(self, r) -> int:
+        if self.sel_ctrl == "0004":
+            tlv_parsed = self.__parse_fcp(r[-1])
+            file_descriptor = tlv_parsed['82']
+            # See also ETSI TS 102 221, chapter 11.1.1.4.3 File Descriptor
+            return int(file_descriptor[4:8], 16)
+        else:
+            return int(r[-1][28:30], 16)
+
+    # Tell the length of a binary file. See also comment
+    # above.
+    def __len(self, r) -> int:
+        if self.sel_ctrl == "0004":
+            tlv_parsed = self.__parse_fcp(r[-1])
+            return int(tlv_parsed['80'], 16)
+        else:
+            return int(r[-1][4:8], 16)
+
+    def get_atr(self) -> Hexstr:
+        """Return the ATR of the currently inserted card."""
+        return self._tp.get_atr()
+
+    def try_select_path(self, dir_list: List[Hexstr]) -> List[ResTuple]:
+        """ Try to select a specified path
+
+        Args:
+                dir_list : list of hex-string FIDs
+        """
+
+        rv = []
+        if not isinstance(dir_list, list):
+            dir_list = [dir_list]
+        for i in dir_list:
+            data, sw = self.send_apdu(self.cla_byte + "a4" + self.sel_ctrl + "02" + i)
+            rv.append((data, sw))
+            if sw != '9000':
+                return rv
+        return rv
+
+    def select_path(self, dir_list: Path) -> List[Hexstr]:
+        """Execute SELECT for an entire list/path of FIDs.
+
+        Args:
+                dir_list: list of FIDs representing the path to select
+
+        Returns:
+                list of return values (FCP in hex encoding) for each element of the path
+        """
+        rv = []
+        if not isinstance(dir_list, list):
+            dir_list = [dir_list]
+        for i in dir_list:
+            data, _sw = self.select_file(i)
+            rv.append(data)
+        return rv
+
+    def select_file(self, fid: Hexstr) -> ResTuple:
+        """Execute SELECT a given file by FID.
+
+        Args:
+                fid : file identifier as hex string
+        """
+
+        return self.send_apdu_checksw(self.cla_byte + "a4" + self.sel_ctrl + "02" + fid)
+
+    def select_parent_df(self) -> ResTuple:
+        """Execute SELECT to switch to the parent DF """
+        return self.send_apdu_checksw(self.cla_byte + "a4030400")
+
+    def select_adf(self, aid: Hexstr) -> ResTuple:
+        """Execute SELECT a given Applicaiton ADF.
+
+        Args:
+                aid : application identifier as hex string
+        """
+
+        aidlen = ("0" + format(len(aid) // 2, 'x'))[-2:]
+        return self.send_apdu_checksw(self.cla_byte + "a4" + "0404" + aidlen + aid)
+
+    def read_binary(self, ef: Path, length: int = None, offset: int = 0) -> ResTuple:
+        """Execute READD BINARY.
+
+        Args:
+                ef : string or list of strings indicating name or path of transparent EF
+                length : number of bytes to read
+                offset : byte offset in file from which to start reading
+        """
+        r = self.select_path(ef)
+        if len(r[-1]) == 0:
+            return (None, None)
+        if length is None:
+            length = self.__len(r) - offset
+        if length < 0:
+            return (None, None)
+
+        total_data = ''
+        chunk_offset = 0
+        while chunk_offset < length:
+            chunk_len = min(self.max_cmd_len, length-chunk_offset)
+            pdu = self.cla_byte + \
+                'b0%04x%02x' % (offset + chunk_offset, chunk_len)
+            try:
+                data, sw = self.send_apdu_checksw(pdu)
+            except Exception as e:
+                raise ValueError('%s, failed to read (offset %d)' %
+                                 (str_sanitize(str(e)), offset)) from e
+            total_data += data
+            chunk_offset += chunk_len
+        return total_data, sw
+
+    def __verify_binary(self, ef, data: str, offset: int = 0):
+        """Verify contents of transparent EF.
+
+        Args:
+                ef : string or list of strings indicating name or path of transparent EF
+                data : hex string of expected data
+                offset : byte offset in file from which to start verifying
+        """
+        res = self.read_binary(ef, len(data) // 2, offset)
+        if res[0].lower() != data.lower():
+            raise ValueError('Binary verification failed (expected %s, got %s)' % (
+                data.lower(), res[0].lower()))
+
+    def update_binary(self, ef: Path, data: Hexstr, offset: int = 0, verify: bool = False,
+                      conserve: bool = False) -> ResTuple:
+        """Execute UPDATE BINARY.
+
+        Args:
+                ef : string or list of strings indicating name or path of transparent EF
+                data : hex string of data to be written
+                offset : byte offset in file from which to start writing
+                verify : Whether or not to verify data after write
+        """
+
+        file_len = self.binary_size(ef)
+        data = expand_hex(data, file_len)
+
+        data_length = len(data) // 2
+
+        # Save write cycles by reading+comparing before write
+        if conserve:
+            try:
+                data_current, sw = self.read_binary(ef, data_length, offset)
+                if data_current == data:
+                    return None, sw
+            except Exception:
+                # cannot read data. This is not a fatal error, as reading is just done to
+                # conserve the amount of smart card writes.  The access conditions of the file
+                # may well permit us to UPDATE but not permit us to READ.  So let's ignore
+                # any such exception during READ.
+                pass
+
+        self.select_path(ef)
+        total_data = ''
+        chunk_offset = 0
+        while chunk_offset < data_length:
+            chunk_len = min(self.max_cmd_len, data_length - chunk_offset)
+            # chunk_offset is bytes, but data slicing is hex chars, so we need to multiply by 2
+            pdu = self.cla_byte + \
+                'd6%04x%02x' % (offset + chunk_offset, chunk_len) + \
+                data[chunk_offset*2: (chunk_offset+chunk_len)*2]
+            try:
+                chunk_data, chunk_sw = self.send_apdu_checksw(pdu)
+            except Exception as e:
+                raise ValueError('%s, failed to write chunk (chunk_offset %d, chunk_len %d)' %
+                                 (str_sanitize(str(e)), chunk_offset, chunk_len)) from e
+            total_data += data
+            chunk_offset += chunk_len
+        if verify:
+            self.__verify_binary(ef, data, offset)
+        return total_data, chunk_sw
+
+    def read_record(self, ef: Path, rec_no: int) -> ResTuple:
+        """Execute READ RECORD.
+
+        Args:
+                ef : string or list of strings indicating name or path of linear fixed EF
+                rec_no : record number to read
+        """
+        r = self.select_path(ef)
+        rec_length = self.__record_len(r)
+        pdu = self.cla_byte + 'b2%02x04%02x' % (rec_no, rec_length)
+        return self.send_apdu_checksw(pdu)
+
+    def __verify_record(self, ef: Path, rec_no: int, data: str):
+        """Verify record against given data
+
+        Args:
+                ef : string or list of strings indicating name or path of linear fixed EF
+                rec_no : record number to read
+                data : hex string of data to be verified
+        """
+        res = self.read_record(ef, rec_no)
+        if res[0].lower() != data.lower():
+            raise ValueError('Record verification failed (expected %s, got %s)' % (
+                data.lower(), res[0].lower()))
+
+    def update_record(self, ef: Path, rec_no: int, data: Hexstr, force_len: bool = False,
+                      verify: bool = False, conserve: bool = False, leftpad: bool = False) -> ResTuple:
+        """Execute UPDATE RECORD.
+
+        Args:
+                ef : string or list of strings indicating name or path of linear fixed EF
+                rec_no : record number to read
+                data : hex string of data to be written
+                force_len : enforce record length by using the actual data length
+                verify : verify data by re-reading the record
+                conserve : read record and compare it with data, skip write on match
+                leftpad : apply 0xff padding from the left instead from the right side.
+        """
+
+        res = self.select_path(ef)
+        rec_length = self.__record_len(res)
+        data = expand_hex(data, rec_length)
+
+        if force_len:
+            # enforce the record length by the actual length of the given data input
+            rec_length = len(data) // 2
+        else:
+            # make sure the input data is padded to the record length using 0xFF.
+            # In cases where the input data exceed we throw an exception.
+            if len(data) // 2 > rec_length:
+                raise ValueError('Data length exceeds record length (expected max %d, got %d)' % (
+                    rec_length, len(data) // 2))
+            elif len(data) // 2 < rec_length:
+                if leftpad:
+                    data = lpad(data, rec_length * 2)
+                else:
+                    data = rpad(data, rec_length * 2)
+
+        # Save write cycles by reading+comparing before write
+        if conserve:
+            try:
+                data_current, sw = self.read_record(ef, rec_no)
+                data_current = data_current[0:rec_length*2]
+                if data_current == data:
+                    return None, sw
+            except Exception:
+                # cannot read data. This is not a fatal error, as reading is just done to
+                # conserve the amount of smart card writes.  The access conditions of the file
+                # may well permit us to UPDATE but not permit us to READ.  So let's ignore
+                # any such exception during READ.
+                pass
+
+        pdu = (self.cla_byte + 'dc%02x04%02x' % (rec_no, rec_length)) + data
+        res = self.send_apdu_checksw(pdu)
+        if verify:
+            self.__verify_record(ef, rec_no, data)
+        return res
+
+    def record_size(self, ef: Path) -> int:
+        """Determine the record size of given file.
+
+        Args:
+                ef : string or list of strings indicating name or path of linear fixed EF
+        """
+        r = self.select_path(ef)
+        return self.__record_len(r)
+
+    def record_count(self, ef: Path) -> int:
+        """Determine the number of records in given file.
+
+        Args:
+                ef : string or list of strings indicating name or path of linear fixed EF
+        """
+        r = self.select_path(ef)
+        return self.__len(r) // self.__record_len(r)
+
+    def binary_size(self, ef: Path) -> int:
+        """Determine the size of given transparent file.
+
+        Args:
+                ef : string or list of strings indicating name or path of transparent EF
+        """
+        r = self.select_path(ef)
+        return self.__len(r)
+
+    # TS 102 221 Section 11.3.1 low-level helper
+    def _retrieve_data(self, tag: int, first: bool = True) -> ResTuple:
+        if first:
+            pdu = self.cla4lchan('80') + 'cb008001%02x' % (tag)
+        else:
+            pdu = self.cla4lchan('80') + 'cb000000'
+        return self.send_apdu_checksw(pdu)
+
+    def retrieve_data(self, ef: Path, tag: int) -> ResTuple:
+        """Execute RETRIEVE DATA, see also TS 102 221 Section 11.3.1.
+
+        Args
+                ef : string or list of strings indicating name or path of transparent EF
+                tag : BER-TLV Tag of value to be retrieved
+        """
+        r = self.select_path(ef)
+        if len(r[-1]) == 0:
+            return (None, None)
+        total_data = ''
+        # retrieve first block
+        data, sw = self._retrieve_data(tag, first=True)
+        total_data += data
+        while sw in ['62f1', '62f2']:
+            data, sw = self._retrieve_data(tag, first=False)
+            total_data += data
+        return total_data, sw
+
+    # TS 102 221 Section 11.3.2 low-level helper
+    def _set_data(self, data: Hexstr, first: bool = True) -> ResTuple:
+        if first:
+            p1 = 0x80
+        else:
+            p1 = 0x00
+        if isinstance(data, (bytes, bytearray)):
+            data = b2h(data)
+        pdu = self.cla4lchan('80') + 'db00%02x%02x%s' % (p1, len(data)//2, data)
+        return self.send_apdu_checksw(pdu)
+
+    def set_data(self, ef, tag: int, value: str, verify: bool = False, conserve: bool = False) -> ResTuple:
+        """Execute SET DATA.
+
+        Args
+                ef : string or list of strings indicating name or path of transparent EF
+                tag : BER-TLV Tag of value to be stored
+                value : BER-TLV value to be stored
+        """
+        r = self.select_path(ef)
+        if len(r[-1]) == 0:
+            return (None, None)
+
+        # in case of deleting the data, we only have 'tag' but no 'value'
+        if not value:
+            return self._set_data('%02x' % tag, first=True)
+
+        # FIXME: proper BER-TLV encode
+        tl = '%02x%s' % (tag, b2h(bertlv_encode_len(len(value)//2)))
+        tlv = tl + value
+        tlv_bin = h2b(tlv)
+
+        first = True
+        total_len = len(tlv_bin)
+        remaining = tlv_bin
+        while len(remaining) > 0:
+            fragment = remaining[:self.max_cmd_len]
+            rdata, sw = self._set_data(fragment, first=first)
+            first = False
+            remaining = remaining[self.max_cmd_len:]
+        return rdata, sw
+
+    def run_gsm(self, rand: Hexstr) -> ResTuple:
+        """Execute RUN GSM ALGORITHM.
+
+        Args:
+                rand : 16 byte random data as hex string (RAND)
+        """
+        if len(rand) != 32:
+            raise ValueError('Invalid rand')
+        self.select_path(['3f00', '7f20'])
+        return self.send_apdu_checksw(self.cla4lchan('a0') + '88000010' + rand, sw='9000')
+
+    def authenticate(self, rand: Hexstr, autn: Hexstr, context: str = '3g') -> ResTuple:
+        """Execute AUTHENTICATE (USIM/ISIM).
+
+        Args:
+                rand : 16 byte random data as hex string (RAND)
+                autn : 8 byte Autentication Token (AUTN)
+                context : 16 byte random data ('3g' or 'gsm')
+        """
+        # 3GPP TS 31.102 Section 7.1.2.1
+        AuthCmd3G = Struct('rand'/LV, 'autn'/COptional(LV))
+        AuthResp3GSyncFail = Struct(Const(b'\xDC'), 'auts'/LV)
+        AuthResp3GSuccess = Struct(Const(b'\xDB'), 'res'/LV, 'ck'/LV, 'ik'/LV, 'kc'/COptional(LV))
+        AuthResp3G = Select(AuthResp3GSyncFail, AuthResp3GSuccess)
+        # build parameters
+        cmd_data = {'rand': rand, 'autn': autn}
+        if context == '3g':
+            p2 = '81'
+        elif context == 'gsm':
+            p2 = '80'
+        (data, sw) = self.send_apdu_constr_checksw(
+            self.cla_byte, '88', '00', p2, AuthCmd3G, cmd_data, AuthResp3G)
+        if 'auts' in data:
+            ret = {'synchronisation_failure': data}
+        else:
+            ret = {'successful_3g_authentication': data}
+        return (ret, sw)
+
+    def status(self) -> ResTuple:
+        """Execute a STATUS command as per TS 102 221 Section 11.1.2."""
+        return self.send_apdu_checksw(self.cla4lchan('80') + 'F20000ff')
+
+    def deactivate_file(self) -> ResTuple:
+        """Execute DECATIVATE FILE command as per TS 102 221 Section 11.1.14."""
+        return self.send_apdu_constr_checksw(self.cla_byte, '04', '00', '00', None, None, None)
+
+    def activate_file(self, fid: Hexstr) -> ResTuple:
+        """Execute ACTIVATE FILE command as per TS 102 221 Section 11.1.15.
+
+        Args:
+                fid : file identifier as hex string
+        """
+        return self.send_apdu_checksw(self.cla_byte + '44000002' + fid)
+
+    def create_file(self, payload: Hexstr) -> ResTuple:
+        """Execute CREEATE FILE command as per TS 102 222 Section 6.3"""
+        return self.send_apdu_checksw(self.cla_byte + 'e00000%02x%s' % (len(payload)//2, payload))
+
+    def resize_file(self, payload: Hexstr) -> ResTuple:
+        """Execute RESIZE FILE command as per TS 102 222 Section 6.10"""
+        return self.send_apdu_checksw(self.cla4lchan('80') + 'd40000%02x%s' % (len(payload)//2, payload))
+
+    def delete_file(self, fid: Hexstr) -> ResTuple:
+        """Execute DELETE FILE command as per TS 102 222 Section 6.4"""
+        return self.send_apdu_checksw(self.cla_byte + 'e4000002' + fid)
+
+    def terminate_df(self, fid: Hexstr) -> ResTuple:
+        """Execute TERMINATE DF command as per TS 102 222 Section 6.7"""
+        return self.send_apdu_checksw(self.cla_byte + 'e6000002' + fid)
+
+    def terminate_ef(self, fid: Hexstr) -> ResTuple:
+        """Execute TERMINATE EF command as per TS 102 222 Section 6.8"""
+        return self.send_apdu_checksw(self.cla_byte + 'e8000002' + fid)
+
+    def terminate_card_usage(self) -> ResTuple:
+        """Execute TERMINATE CARD USAGE command as per TS 102 222 Section 6.9"""
+        return self.send_apdu_checksw(self.cla_byte + 'fe000000')
+
+    def manage_channel(self, mode: str = 'open', lchan_nr: int =0) -> ResTuple:
+        """Execute MANAGE CHANNEL command as per TS 102 221 Section 11.1.17.
+
+        Args:
+                mode : logical channel operation code ('open' or 'close')
+                lchan_nr : logical channel number (1-19, 0=assigned by UICC)
+        """
+        if mode == 'close':
+            p1 = 0x80
+        else:
+            p1 = 0x00
+        pdu = self.cla_byte + '70%02x%02x00' % (p1, lchan_nr)
+        return self.send_apdu_checksw(pdu)
+
+    def reset_card(self) -> Hexstr:
+        """Physically reset the card"""
+        return self._tp.reset_card()
+
+    def _chv_process_sw(self, op_name: str, chv_no: int, pin_code: Hexstr, sw: SwHexstr):
+        if sw_match(sw, '63cx'):
+            raise RuntimeError('Failed to %s chv_no 0x%02X with code 0x%s, %i tries left.' %
+                               (op_name, chv_no, b2h(pin_code).upper(), int(sw[3])))
+        if sw != '9000':
+            raise SwMatchError(sw, '9000')
+
+    def verify_chv(self, chv_no: int, code: Hexstr) -> ResTuple:
+        """Verify a given CHV (Card Holder Verification == PIN)
+
+        Args:
+                chv_no : chv number (1=CHV1, 2=CHV2, ...)
+                code : chv code as hex string
+        """
+        fc = rpad(b2h(code), 16)
+        data, sw = self.send_apdu(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
+        self._chv_process_sw('verify', chv_no, code, sw)
+        return (data, sw)
+
+    def unblock_chv(self, chv_no: int, puk_code: str, pin_code: str):
+        """Unblock a given CHV (Card Holder Verification == PIN)
+
+        Args:
+                chv_no : chv number (1=CHV1, 2=CHV2, ...)
+                puk_code : puk code as hex string
+                pin_code : new chv code as hex string
+        """
+        fc = rpad(b2h(puk_code), 16) + rpad(b2h(pin_code), 16)
+        data, sw = self.send_apdu(self.cla_byte + '2C00' + ('%02X' % chv_no) + '10' + fc)
+        self._chv_process_sw('unblock', chv_no, pin_code, sw)
+        return (data, sw)
+
+    def change_chv(self, chv_no: int, pin_code: Hexstr, new_pin_code: Hexstr) -> ResTuple:
+        """Change a given CHV (Card Holder Verification == PIN)
+
+        Args:
+                chv_no : chv number (1=CHV1, 2=CHV2, ...)
+                pin_code : current chv code as hex string
+                new_pin_code : new chv code as hex string
+        """
+        fc = rpad(b2h(pin_code), 16) + rpad(b2h(new_pin_code), 16)
+        data, sw = self.send_apdu(self.cla_byte + '2400' + ('%02X' % chv_no) + '10' + fc)
+        self._chv_process_sw('change', chv_no, pin_code, sw)
+        return (data, sw)
+
+    def disable_chv(self, chv_no: int, pin_code: Hexstr) -> ResTuple:
+        """Disable a given CHV (Card Holder Verification == PIN)
+
+        Args:
+                chv_no : chv number (1=CHV1, 2=CHV2, ...)
+                pin_code : current chv code as hex string
+                new_pin_code : new chv code as hex string
+        """
+        fc = rpad(b2h(pin_code), 16)
+        data, sw = self.send_apdu(self.cla_byte + '2600' + ('%02X' % chv_no) + '08' + fc)
+        self._chv_process_sw('disable', chv_no, pin_code, sw)
+        return (data, sw)
+
+    def enable_chv(self, chv_no: int, pin_code: Hexstr) -> ResTuple:
+        """Enable a given CHV (Card Holder Verification == PIN)
+
+        Args:
+                chv_no : chv number (1=CHV1, 2=CHV2, ...)
+                pin_code : chv code as hex string
+        """
+        fc = rpad(b2h(pin_code), 16)
+        data, sw = self.send_apdu(self.cla_byte + '2800' + ('%02X' % chv_no) + '08' + fc)
+        self._chv_process_sw('enable', chv_no, pin_code, sw)
+        return (data, sw)
+
+    def envelope(self, payload: Hexstr) -> ResTuple:
+        """Send one ENVELOPE command to the SIM
+
+        Args:
+                payload : payload as hex string
+        """
+        return self.send_apdu_checksw('80c20000%02x%s' % (len(payload)//2, payload))
+
+    def terminal_profile(self, payload: Hexstr) -> ResTuple:
+        """Send TERMINAL PROFILE to card
+
+        Args:
+                payload : payload as hex string
+        """
+        data_length = len(payload) // 2
+        data, sw = self.send_apdu(('80100000%02x' % data_length) + payload)
+        return (data, sw)
+
+    # ETSI TS 102 221 11.1.22
+    def suspend_uicc(self, min_len_secs: int = 60, max_len_secs: int = 43200) -> Tuple[int, Hexstr, SwHexstr]:
+        """Send SUSPEND UICC to the card.
+
+        Args:
+                 min_len_secs : mimumum suspend time seconds
+                 max_len_secs : maximum suspend time seconds
+        """
+        def encode_duration(secs: int) -> Hexstr:
+            if secs >= 10*24*60*60:
+                return '04%02x' % (secs // (10*24*60*60))
+            if secs >= 24*60*60:
+                return '03%02x' % (secs // (24*60*60))
+            if secs >= 60*60:
+                return '02%02x' % (secs // (60*60))
+            if secs >= 60:
+                return '01%02x' % (secs // 60)
+            return '00%02x' % secs
+
+        def decode_duration(enc: Hexstr) -> int:
+            time_unit = enc[:2]
+            length = h2i(enc[2:4])[0]
+            if time_unit == '04':
+                return length * 10*24*60*60
+            if time_unit == '03':
+                return length * 24*60*60
+            if time_unit == '02':
+                return length * 60*60
+            if time_unit == '01':
+                return length * 60
+            if time_unit == '00':
+                return length
+            raise ValueError('Time unit must be 0x00..0x04')
+        min_dur_enc = encode_duration(min_len_secs)
+        max_dur_enc = encode_duration(max_len_secs)
+        data, sw = self.send_apdu_checksw('8076000004' + min_dur_enc + max_dur_enc)
+        negotiated_duration_secs = decode_duration(data[:4])
+        resume_token = data[4:]
+        return (negotiated_duration_secs, resume_token, sw)
+
+    # ETSI TS 102 221 11.1.22
+    def resume_uicc(self, token: Hexstr) -> ResTuple:
+        """Send SUSPEND UICC (resume) to the card."""
+        if len(h2b(token)) != 8:
+            raise ValueError("Token must be 8 bytes long")
+        data, sw = self.send_apdu_checksw('8076010008' + token)
+        return (data, sw)
+
+    def get_data(self, tag: int, cla: int = 0x00):
+        data, sw = self.send_apdu('%02xca%04x00' % (cla, tag))
+        return (data, sw)
+
+    # TS 31.102 Section 7.5.2
+    def get_identity(self, context: int) -> Tuple[Hexstr, SwHexstr]:
+        data, sw = self.send_apdu_checksw('807800%02x00' % (context))
+        return (data, sw)

pySim/construct.py

@@ -0,0 +1,587 @@
+"""Utility code related to the integration of the 'construct' declarative parser."""
+
+import typing
+import codecs
+import ipaddress
+
+import gsm0338
+
+from construct.lib.containers import Container, ListContainer
+from construct.core import EnumIntegerString
+from construct import Adapter, Prefixed, Int8ub, GreedyBytes, Default, Flag, Byte, Construct, Enum
+from construct import BitsInteger, BitStruct, Bytes, StreamError, stream_read_entire, stream_write
+from construct import SizeofError, IntegerError, swapbytes
+from construct.core import evaluate
+from construct.lib import integertypes
+
+from pySim.utils import b2h, h2b, swap_nibbles
+
+# (C) 2021-2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+class HexAdapter(Adapter):
+    """convert a bytes() type to a string of hex nibbles."""
+
+    def _decode(self, obj, context, path):
+        return b2h(obj)
+
+    def _encode(self, obj, context, path):
+        return h2b(obj)
+
+class Utf8Adapter(Adapter):
+    """convert a bytes() type that contains utf8 encoded text to human readable text."""
+
+    def _decode(self, obj, context, path):
+        # In case the string contains only 0xff bytes we interpret it as an empty string
+        if obj == b'\xff' * len(obj):
+            return ""
+        return codecs.decode(obj, "utf-8")
+
+    def _encode(self, obj, context, path):
+        return codecs.encode(obj, "utf-8")
+
+class GsmOrUcs2Adapter(Adapter):
+    """Try to encode into a GSM 03.38 string; if that fails, fall back to UCS-2 as described
+    in TS 102 221 Annex A."""
+    def _decode(self, obj, context, path):
+        # In case the string contains only 0xff bytes we interpret it as an empty string
+        if obj == b'\xff' * len(obj):
+            return ""
+        # one of the magic bytes of TS 102 221 Annex A
+        if obj[0] in [0x80, 0x81, 0x82]:
+            ad = Ucs2Adapter(GreedyBytes)
+        else:
+            ad = GsmString(GreedyBytes)
+        return ad._decode(obj, context, path)
+
+    def _encode(self, obj, context, path):
+        # first try GSM 03.38; then fall back to TS 102 221 Annex A UCS-2
+        try:
+            ad = GsmString(GreedyBytes)
+            return ad._encode(obj, context, path)
+        except:
+            ad = Ucs2Adapter(GreedyBytes)
+            return ad._encode(obj, context, path)
+
+class Ucs2Adapter(Adapter):
+    """convert a bytes() type that contains UCS2 encoded characters encoded as defined in TS 102 221
+    Annex A to normal python string representation (and back)."""
+    def _decode(self, obj, context, path):
+        # In case the string contains only 0xff bytes we interpret it as an empty string
+        if obj == b'\xff' * len(obj):
+            return ""
+        if obj[0] == 0x80:
+            # TS 102 221 Annex A Variant 1
+            return codecs.decode(obj[1:], 'utf_16_be')
+        elif obj[0] == 0x81:
+            # TS 102 221 Annex A Variant 2
+            out = ""
+            # second byte contains a value indicating the number of characters
+            num_of_chars = obj[1]
+            # the third byte contains an 8 bit number which defines bits 15 to 8 of a 16 bit base
+            # pointer, where bit 16 is set to zero, and bits 7 to 1 are also set to zero. These
+            # sixteen bits constitute a base pointer to a "half-page" in the UCS2 code space
+            base_ptr = obj[2] << 7
+            for ch in obj[3:3+num_of_chars]:
+                # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a
+                # GSM Default Alphabet character, whereas if bit 8 of the byte is set to one, then
+                # the remaining seven bits are an offset value added to the 16 bit base pointer
+                # defined earlier, and the resultant 16 bit value is a UCS2 code point
+                if ch & 0x80:
+                    codepoint = (ch & 0x7f) + base_ptr
+                    out += codecs.decode(codepoint.to_bytes(2, byteorder='big'), 'utf_16_be')
+                else:
+                    out += codecs.decode(bytes([ch]), 'gsm03.38')
+            return out
+        elif obj[0] == 0x82:
+            # TS 102 221 Annex A Variant 3
+            out = ""
+            # second byte contains a value indicating the number of characters
+            num_of_chars = obj[1]
+            # third and fourth bytes contain a 16 bit number which defines the complete 16 bit base
+            # pointer to a half-page in the UCS2 code space, for use with some or all of the
+            # remaining bytes in the string
+            base_ptr = obj[2] << 8 | obj[3]
+            for ch in obj[4:4+num_of_chars]:
+                # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a
+                # GSM Default Alphabet character, whereas if bit 8 of the byte is set to one, the
+                # remaining seven bits are an offset value added to the base pointer defined in
+                # bytes three and four, and the resultant 16 bit value is a UCS2 code point, else: #
+                # GSM default alphabet
+                if ch & 0x80:
+                    codepoint = (ch & 0x7f) + base_ptr
+                    out += codecs.decode(codepoint.to_bytes(2, byteorder='big'), 'utf_16_be')
+                else:
+                    out += codecs.decode(bytes([ch]), 'gsm03.38')
+            return out
+        else:
+            raise ValueError('First byte of TS 102 221 UCS-2 must be 0x80, 0x81 or 0x82')
+
+    def _encode(self, obj, context, path):
+        def encodable_in_gsm338(instr: str) -> bool:
+            """Determine if given input string is encode-ale in gsm03.38."""
+            try:
+                # TODO: figure out if/how we can constrain to default alphabet.  The gsm0338
+                # library seems to include the spanish lock/shift table
+                codecs.encode(instr, 'gsm03.38')
+            except ValueError:
+                return False
+            return True
+
+        def codepoints_not_in_gsm338(instr: str) -> typing.List[int]:
+            """Return an integer list of UCS2 codepoints for all characters of 'inster'
+            which are not representable in the GSM 03.38 default alphabet."""
+            codepoint_list = []
+            for c in instr:
+                if encodable_in_gsm338(c):
+                    continue
+                c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+                codepoint_list.append(c_codepoint)
+            return codepoint_list
+
+        def diff_between_min_and_max_of_list(inlst: typing.List) -> int:
+            return max(inlst) - min(inlst)
+
+        def encodable_in_variant2(instr: str) -> bool:
+            codepoint_prefix = None
+            for c in instr:
+                if encodable_in_gsm338(c):
+                    continue
+                c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+                if c_codepoint >= 0x8000:
+                    return False
+                c_prefix = c_codepoint >> 7
+                if codepoint_prefix is None:
+                    codepoint_prefix = c_prefix
+                else:
+                    if c_prefix != codepoint_prefix:
+                        return False
+            return True
+
+        def encodable_in_variant3(instr: str) -> bool:
+            codepoint_list = codepoints_not_in_gsm338(instr)
+            # compute delta between max and min; check if it's encodable in 7 bits
+            if diff_between_min_and_max_of_list(codepoint_list) >= 0x80:
+                return False
+            return True
+
+        def _encode_variant1(instr: str) -> bytes:
+            """Encode according to TS 102 221 Annex A Variant 1"""
+            return b'\x80' + codecs.encode(instr, 'utf_16_be')
+
+        def _encode_variant2(instr: str) -> bytes:
+            """Encode according to TS 102 221 Annex A Variant 2"""
+            codepoint_prefix = None
+            # second byte contains a value indicating the number of characters
+            hdr = b'\x81' + len(instr).to_bytes(1, byteorder='big')
+            chars = b''
+            for c in instr:
+                try:
+                    enc = codecs.encode(c, 'gsm03.38')
+                except ValueError:
+                    c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+                    c_prefix = c_codepoint >> 7
+                    if codepoint_prefix is None:
+                        codepoint_prefix = c_prefix
+                    assert codepoint_prefix == c_prefix
+                    enc = (0x80 + (c_codepoint & 0x7f)).to_bytes(1, byteorder='big')
+                chars += enc
+            if codepoint_prefix is None:
+                codepoint_prefix = 0
+            return hdr + codepoint_prefix.to_bytes(1, byteorder='big') + chars
+
+        def _encode_variant3(instr: str) -> bytes:
+            """Encode according to TS 102 221 Annex A Variant 3"""
+            # second byte contains a value indicating the number of characters
+            hdr = b'\x82' + len(instr).to_bytes(1, byteorder='big')
+            chars = b''
+            codepoint_list = codepoints_not_in_gsm338(instr)
+            codepoint_base = min(codepoint_list)
+            for c in instr:
+                try:
+                    # if bit 8 of the byte is set to zero, the remaining 7 bits of the byte contain a GSM
+                    # Default # Alphabet character
+                    enc = codecs.encode(c, 'gsm03.38')
+                except ValueError:
+                    # if bit 8 of the byte is set to one, the remaining seven bits are an offset
+                    # value added to the base pointer defined in bytes three and four, and the
+                    # resultant 16 bit value is a UCS2 code point
+                    c_codepoint = int.from_bytes(codecs.encode(c, 'utf_16_be'), byteorder='big')
+                    c_codepoint_delta = c_codepoint - codepoint_base
+                    assert c_codepoint_delta < 0x80
+                    enc = (0x80 + c_codepoint_delta).to_bytes(1, byteorder='big')
+                chars += enc
+            # third and fourth bytes contain a 16 bit number which defines the complete 16 bit base
+            # pointer to a half-page in the UCS2 code space
+            return hdr + codepoint_base.to_bytes(2, byteorder='big') + chars
+
+        if encodable_in_variant2(obj):
+            return _encode_variant2(obj)
+        elif encodable_in_variant3(obj):
+            return _encode_variant3(obj)
+        else:
+            return _encode_variant1(obj)
+
+class BcdAdapter(Adapter):
+    """convert a bytes() type to a string of BCD nibbles."""
+
+    def _decode(self, obj, context, path):
+        return swap_nibbles(b2h(obj))
+
+    def _encode(self, obj, context, path):
+        return h2b(swap_nibbles(obj))
+
+class PlmnAdapter(BcdAdapter):
+    """convert a bytes(3) type to BCD string like 262-02 or 262-002."""
+    def _decode(self, obj, context, path):
+        bcd = super()._decode(obj, context, path)
+        if bcd[3] == 'f':
+            return '-'.join([bcd[:3], bcd[4:]])
+        else:
+            return '-'.join([bcd[:3], bcd[3:]])
+
+    def _encode(self, obj, context, path):
+        l = obj.split('-')
+        if len(l[1]) == 2:
+            bcd = l[0] + 'f' + l[1]
+        else:
+            bcd = l[0] + l[1]
+        return super()._encode(bcd, context, path)
+
+class InvertAdapter(Adapter):
+    """inverse logic (false->true, true->false)."""
+    @staticmethod
+    def _invert_bool_in_obj(obj):
+        for k,v in obj.items():
+            # skip all private entries
+            if k.startswith('_'):
+                continue
+            if v is False:
+                obj[k] = True
+            elif v is True:
+                obj[k] = False
+        return obj
+
+    def _decode(self, obj, context, path):
+        return self._invert_bool_in_obj(obj)
+
+    def _encode(self, obj, context, path):
+        return self._invert_bool_in_obj(obj)
+
+class Rpad(Adapter):
+    """
+    Encoder appends padding bytes (b'\\xff') or characters up to target size.
+    Decoder removes trailing padding bytes/characters.
+
+    Parameters:
+        subcon: Subconstruct as defined by construct library
+        pattern: set padding pattern (default: b'\\xff')
+        num_per_byte: number of 'elements' per byte. E.g. for hex nibbles: 2
+    """
+
+    def __init__(self, subcon, pattern=b'\xff', num_per_byte=1):
+        super().__init__(subcon)
+        self.pattern = pattern
+        self.num_per_byte = num_per_byte
+
+    def _decode(self, obj, context, path):
+        return obj.rstrip(self.pattern)
+
+    def _encode(self, obj, context, path):
+        target_size = self.sizeof() * self.num_per_byte
+        if len(obj) > target_size:
+            raise SizeofError("Input ({}) exceeds target size ({})".format(
+                len(obj), target_size))
+        return obj + self.pattern * (target_size - len(obj))
+
+class MultiplyAdapter(Adapter):
+    """
+    Decoder multiplies by multiplicator
+    Encoder divides by multiplicator
+
+    Parameters:
+        subcon: Subconstruct as defined by construct library
+        multiplier: Multiplier to apply to raw encoded value
+    """
+
+    def __init__(self, subcon, multiplicator):
+        super().__init__(subcon)
+        self.multiplicator = multiplicator
+
+    def _decode(self, obj, context, path):
+        return obj * 8
+
+    def _encode(self, obj, context, path):
+        return obj // 8
+
+
+class GsmStringAdapter(Adapter):
+    """Convert GSM 03.38 encoded bytes to a string."""
+
+    def __init__(self, subcon, codec='gsm03.38', err='strict'):
+        super().__init__(subcon)
+        self.codec = codec
+        self.err = err
+
+    def _decode(self, obj, context, path):
+        return obj.decode(self.codec)
+
+    def _encode(self, obj, context, path):
+        return obj.encode(self.codec, self.err)
+
+class Ipv4Adapter(Adapter):
+    """
+    Encoder converts from 4 bytes to string representation (A.B.C.D).
+    Decoder converts from string representation (A.B.C.D) to four bytes.
+    """
+    def _decode(self, obj, context, path):
+        ia = ipaddress.IPv4Address(obj)
+        return ia.compressed
+
+    def _encode(self, obj, context, path):
+        ia = ipaddress.IPv4Address(obj)
+        return ia.packed
+
+class Ipv6Adapter(Adapter):
+    """
+    Encoder converts from 16 bytes to string representation.
+    Decoder converts from string representation to 16 bytes.
+    """
+    def _decode(self, obj, context, path):
+        ia = ipaddress.IPv6Address(obj)
+        return ia.compressed
+
+    def _encode(self, obj, context, path):
+        ia = ipaddress.IPv6Address(obj)
+        return ia.packed
+
+class StripTrailerAdapter(Adapter):
+    """
+    Encoder removes all trailing bytes matching the default_value
+    Decoder pads input data up to total_length with default_value
+
+    This is used in constellations like "FlagsEnum(StripTrailerAdapter(GreedyBytes, 3), ..."
+    where you have a bit-mask that may have 1, 2 or 3 bytes, depending on whether or not any
+    of the LSBs are actually set.
+    """
+    def __init__(self, subcon, total_length:int, default_value=b'\x00', min_len=1):
+        super().__init__(subcon)
+        assert len(default_value) == 1
+        self.total_length = total_length
+        self.default_value = default_value
+        self.min_len = min_len
+
+    def _decode(self, obj, context, path):
+        assert isinstance(obj, bytes)
+        # pad with suppressed/missing bytes
+        if len(obj) < self.total_length:
+            obj += self.default_value * (self.total_length - len(obj))
+        return int.from_bytes(obj, 'big')
+
+    def _encode(self, obj, context, path):
+        assert isinstance(obj, int)
+        obj = obj.to_bytes(self.total_length, 'big')
+        # remove trailing bytes if they are zero
+        while len(obj) > self.min_len and obj[-1] == self.default_value[0]:
+            obj = obj[:-1]
+        return obj
+
+
+def filter_dict(d, exclude_prefix='_'):
+    """filter the input dict to ensure no keys starting with 'exclude_prefix' remain."""
+    if not isinstance(d, dict):
+        return d
+    res = {}
+    for (key, value) in d.items():
+        if key.startswith(exclude_prefix):
+            continue
+        if isinstance(value, dict):
+            res[key] = filter_dict(value)
+        else:
+            res[key] = value
+    return res
+
+
+def normalize_construct(c, exclude_prefix: str = '_'):
+    """Convert a construct specific type to a related base type, mostly useful
+    so we can serialize it."""
+    # we need to include the filter_dict as we otherwise get elements like this
+    # in the dict: '_io': <_io.BytesIO object at 0x7fdb64e05860> which we cannot json-serialize
+    c = filter_dict(c, exclude_prefix)
+    if isinstance(c, (Container, dict)):
+        r = {k: normalize_construct(v) for (k, v) in c.items()}
+    elif isinstance(c, ListContainer):
+        r = [normalize_construct(x) for x in c]
+    elif isinstance(c, list):
+        r = [normalize_construct(x) for x in c]
+    elif isinstance(c, EnumIntegerString):
+        r = str(c)
+    else:
+        r = c
+    return r
+
+
+def parse_construct(c, raw_bin_data: bytes, length: typing.Optional[int] = None, exclude_prefix: str = '_', context: dict = {}):
+    """Helper function to wrap around normalize_construct() and filter_dict()."""
+    if not length:
+        length = len(raw_bin_data)
+    try:
+        parsed = c.parse(raw_bin_data, total_len=length, **context)
+    except StreamError as e:
+        # if the input is all-ff, this means the content is undefined.  Let's avoid passing StreamError
+        # exceptions in those situations (which might occur if a length field 0xff is 255 but then there's
+        # actually less bytes in the remainder of the file.
+        if all(v == 0xff for v in raw_bin_data):
+            return None
+        else:
+            raise e
+    return normalize_construct(parsed, exclude_prefix)
+
+def build_construct(c, decoded_data, context: dict = {}):
+    """Helper function to handle total_len."""
+    return c.build(decoded_data, total_len=None, **context)
+
+# here we collect some shared / common definitions of data types
+LV = Prefixed(Int8ub, HexAdapter(GreedyBytes))
+
+# Default value for Reserved for Future Use (RFU) bits/bytes
+# See TS 31.101 Sec. "3.4 Coding Conventions"
+__RFU_VALUE = 0
+
+# Field that packs Reserved for Future Use (RFU) bit
+FlagRFU = Default(Flag, __RFU_VALUE)
+
+# Field that packs Reserved for Future Use (RFU) byte
+ByteRFU = Default(Byte, __RFU_VALUE)
+
+# Field that packs all remaining Reserved for Future Use (RFU) bytes
+GreedyBytesRFU = Default(GreedyBytes, b'')
+
+
+def BitsRFU(n=1):
+    '''
+    Field that packs Reserved for Future Use (RFU) bit(s)
+    as defined in TS 31.101 Sec. "3.4 Coding Conventions"
+
+    Use this for (currently) unused/reserved bits whose contents
+    should be initialized automatically but should not be cleared
+    in the future or when restoring read data (unlike padding).
+
+    Parameters:
+        n (Integer): Number of bits (default: 1)
+    '''
+    return Default(BitsInteger(n), __RFU_VALUE)
+
+
+def BytesRFU(n=1):
+    '''
+    Field that packs Reserved for Future Use (RFU) byte(s)
+    as defined in TS 31.101 Sec. "3.4 Coding Conventions"
+
+    Use this for (currently) unused/reserved bytes whose contents
+    should be initialized automatically but should not be cleared
+    in the future or when restoring read data (unlike padding).
+
+    Parameters:
+        n (Integer): Number of bytes (default: 1)
+    '''
+    return Default(Bytes(n), __RFU_VALUE)
+
+
+def GsmString(n):
+    '''
+    GSM 03.38 encoded byte string of fixed length n.
+    Encoder appends padding bytes (b'\\xff') to maintain
+    length. Decoder removes those trailing bytes.
+
+    Exceptions are raised for invalid characters
+    and length excess.
+
+    Parameters:
+        n (Integer): Fixed length of the encoded byte string
+    '''
+    return GsmStringAdapter(Rpad(Bytes(n), pattern=b'\xff'), codec='gsm03.38')
+
+def GsmOrUcs2String(n):
+    '''
+    GSM 03.38 or UCS-2 (TS 102 221 Annex A) encoded byte string of fixed length n.
+    Encoder appends padding bytes (b'\\xff') to maintain
+    length. Decoder removes those trailing bytes.
+
+    Exceptions are raised for invalid characters
+    and length excess.
+
+    Parameters:
+        n (Integer): Fixed length of the encoded byte string
+    '''
+    return GsmOrUcs2Adapter(Rpad(Bytes(n), pattern=b'\xff'))
+
+class GreedyInteger(Construct):
+    """A variable-length integer implementation, think of combining GrredyBytes with BytesInteger."""
+    def __init__(self, signed=False, swapped=False, minlen=0):
+        super().__init__()
+        self.signed = signed
+        self.swapped = swapped
+        self.minlen = minlen
+
+    def _parse(self, stream, context, path):
+        data = stream_read_entire(stream, path)
+        if evaluate(self.swapped, context):
+            data = swapbytes(data)
+        try:
+            return int.from_bytes(data, byteorder='big', signed=self.signed)
+        except ValueError as e:
+            raise IntegerError(str(e), path=path)
+
+    def __bytes_required(self, i, minlen=0):
+        if self.signed:
+            raise NotImplementedError("FIXME: Implement support for encoding signed integer")
+
+        # compute how many bytes we need
+        nbytes = 1
+        while True:
+            i = i >> 8
+            if i == 0:
+                break
+            else:
+                nbytes = nbytes + 1
+
+        # round up to the minimum number
+        # of bytes we anticipate
+        nbytes = max(nbytes, minlen)
+
+        return nbytes
+
+    def _build(self, obj, stream, context, path):
+        if not isinstance(obj, integertypes):
+            raise IntegerError(f"value {obj} is not an integer", path=path)
+        length = self.__bytes_required(obj, self.minlen)
+        try:
+            data = obj.to_bytes(length, byteorder='big', signed=self.signed)
+        except ValueError as e:
+            raise IntegerError(str(e), path=path) from e
+        if evaluate(self.swapped, context):
+            data = swapbytes(data)
+        stream_write(stream, data, length, path)
+        return obj
+
+# merged definitions of 24.008 + 23.040
+TypeOfNumber = Enum(BitsInteger(3), unknown=0, international=1, national=2, network_specific=3,
+                    short_code=4, alphanumeric=5, abbreviated=6, reserved_for_extension=7)
+NumberingPlan = Enum(BitsInteger(4), unknown=0, isdn_e164=1, data_x121=3, telex_f69=4,
+                     sc_specific_5=5, sc_specific_6=6, national=8, private=9,
+                     ermes=10, reserved_cts=11, reserved_for_extension=15)
+TonNpi = BitStruct('ext'/Flag, 'type_of_number'/TypeOfNumber, 'numbering_plan_id'/NumberingPlan)

pySim/esim/__init__.py

@@ -0,0 +1,93 @@
+import sys
+from typing import Optional
+from importlib import resources
+
+
+def compile_asn1_subdir(subdir_name:str):
+    """Helper function that compiles ASN.1 syntax from all files within given subdir"""
+    import asn1tools
+    asn_txt = ''
+    __ver = sys.version_info
+    if (__ver.major, __ver.minor) >= (3, 9):
+        for i in resources.files('pySim.esim').joinpath('asn1').joinpath(subdir_name).iterdir():
+            asn_txt += i.read_text()
+            asn_txt += "\n"
+    #else:
+        #print(resources.read_text(__name__, 'asn1/rsp.asn'))
+    return asn1tools.compile_string(asn_txt, codec='der')
+
+
+# SGP.22 section 4.1 Activation Code
+class ActivationCode:
+    def __init__(self, hostname:str, token:str, oid: Optional[str] = None, cc_required: Optional[bool] = False):
+        if '$' in hostname:
+            raise ValueError('$ sign not permitted in hostname')
+        self.hostname = hostname
+        if '$' in token:
+            raise ValueError('$ sign not permitted in token')
+        self.token = token
+        # TODO: validate OID
+        self.oid = oid
+        self.cc_required = cc_required
+        # only format 1 is specified and supported here
+        self.format = 1
+
+    @staticmethod
+    def decode_str(ac: str) -> dict:
+        if ac[0] != '1':
+            raise ValueError("Unsupported AC_Format '%s'!" % ac[0])
+        ac_elements = ac.split('$')
+        d = {
+            'oid': None,
+            'cc_required': False,
+          }
+        d['format'] = ac_elements.pop(0)
+        d['hostname'] = ac_elements.pop(0)
+        d['token'] = ac_elements.pop(0)
+        if len(ac_elements):
+            oid = ac_elements.pop(0)
+            if oid != '':
+                d['oid'] = oid
+        if len(ac_elements):
+            ccr = ac_elements.pop(0)
+            if ccr == '1':
+                d['cc_required'] = True
+        return d
+
+    @classmethod
+    def from_string(cls, ac: str) -> 'ActivationCode':
+        """Create new instance from SGP.22 section 4.1 string representation."""
+        d = cls.decode_str(ac)
+        return cls(d['hostname'], d['token'], d['oid'], d['cc_required'])
+
+    def to_string(self, for_qrcode:bool = False) -> str:
+        """Convert from internal representation to SGP.22 section 4.1 string representation."""
+        if for_qrcode:
+            ret = 'LPA:'
+        else:
+            ret = ''
+        ret += '%d$%s$%s' % (self.format, self.hostname, self.token)
+        if self.oid:
+            ret += '$%s' % (self.oid)
+        elif self.cc_required:
+            ret += '$'
+        if self.cc_required:
+            ret += '$1'
+        return ret
+
+    def __str__(self):
+        return self.to_string()
+
+    def to_qrcode(self):
+        """Encode internal representation to QR code."""
+        import qrcode
+        qr = qrcode.QRCode()
+        qr.add_data(self.to_string(for_qrcode=True))
+        return qr.make_image()
+
+    def __repr__(self):
+        return "ActivationCode(format=%u, hostname='%s', token='%s', oid=%s, cc_required=%s)" % (self.format,
+                                                                                                 self.hostname,
+                                                                                                 self.token,
+                                                                                                 self.oid,
+                                                                                                 self.cc_required)

pySim/esim/asn1/rsp/PKIX1Explicit88.asn

@@ -0,0 +1,657 @@
+PKIX1Explicit88 { iso(1) identified-organization(3) dod(6) internet(1)
+  security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-explicit(18) }
+
+DEFINITIONS EXPLICIT TAGS ::=
+
+BEGIN
+
+-- EXPORTS ALL --
+
+-- IMPORTS NONE --
+
+-- UNIVERSAL Types defined in 1993 and 1998 ASN.1
+-- and required by this specification
+
+-- pycrate: UniversalString, BMPString and UTF8String already in the builtin types
+
+--UniversalString ::= [UNIVERSAL 28] IMPLICIT OCTET STRING
+        -- UniversalString is defined in ASN.1:1993
+
+--BMPString ::= [UNIVERSAL 30] IMPLICIT OCTET STRING
+      -- BMPString is the subtype of UniversalString and models
+      -- the Basic Multilingual Plane of ISO/IEC 10646
+
+--UTF8String ::= [UNIVERSAL 12] IMPLICIT OCTET STRING
+      -- The content of this type conforms to RFC 3629.
+
+-- PKIX specific OIDs
+
+id-pkix  OBJECT IDENTIFIER  ::=
+         { iso(1) identified-organization(3) dod(6) internet(1)
+                    security(5) mechanisms(5) pkix(7) }
+
+-- PKIX arcs
+
+id-pe OBJECT IDENTIFIER ::= { id-pkix 1 }
+        -- arc for private certificate extensions
+id-qt OBJECT IDENTIFIER ::= { id-pkix 2 }
+        -- arc for policy qualifier types
+id-kp OBJECT IDENTIFIER ::= { id-pkix 3 }
+        -- arc for extended key purpose OIDS
+id-ad OBJECT IDENTIFIER ::= { id-pkix 48 }
+        -- arc for access descriptors
+
+-- policyQualifierIds for Internet policy qualifiers
+
+id-qt-cps      OBJECT IDENTIFIER ::=  { id-qt 1 }
+      -- OID for CPS qualifier
+id-qt-unotice  OBJECT IDENTIFIER ::=  { id-qt 2 }
+      -- OID for user notice qualifier
+
+-- access descriptor definitions
+
+id-ad-ocsp         OBJECT IDENTIFIER ::= { id-ad 1 }
+id-ad-caIssuers    OBJECT IDENTIFIER ::= { id-ad 2 }
+id-ad-timeStamping OBJECT IDENTIFIER ::= { id-ad 3 }
+id-ad-caRepository OBJECT IDENTIFIER ::= { id-ad 5 }
+
+-- attribute data types
+
+Attribute               ::= SEQUENCE {
+      type             AttributeType,
+      values    SET OF AttributeValue }
+            -- at least one value is required
+
+AttributeType           ::= OBJECT IDENTIFIER
+
+AttributeValue          ::= ANY -- DEFINED BY AttributeType
+
+AttributeTypeAndValue   ::= SEQUENCE {
+        type    AttributeType,
+        value   AttributeValue }
+
+-- suggested naming attributes: Definition of the following
+--   information object set may be augmented to meet local
+--   requirements.  Note that deleting members of the set may
+--   prevent interoperability with conforming implementations.
+-- presented in pairs: the AttributeType followed by the
+--   type definition for the corresponding AttributeValue
+
+-- Arc for standard naming attributes
+
+id-at OBJECT IDENTIFIER ::= { joint-iso-ccitt(2) ds(5) 4 }
+
+-- Naming attributes of type X520name
+
+id-at-name                AttributeType ::= { id-at 41 }
+id-at-surname             AttributeType ::= { id-at  4 }
+id-at-givenName           AttributeType ::= { id-at 42 }
+id-at-initials            AttributeType ::= { id-at 43 }
+id-at-generationQualifier AttributeType ::= { id-at 44 }
+
+-- Naming attributes of type X520Name:
+--   X520name ::= DirectoryString (SIZE (1..ub-name))
+--
+-- Expanded to avoid parameterized type:
+X520name ::= CHOICE {
+      teletexString     TeletexString   (SIZE (1..ub-name)),
+      printableString   PrintableString (SIZE (1..ub-name)),
+      universalString   UniversalString (SIZE (1..ub-name)),
+      utf8String        UTF8String      (SIZE (1..ub-name)),
+      bmpString         BMPString       (SIZE (1..ub-name)) }
+
+-- Naming attributes of type X520CommonName
+
+id-at-commonName        AttributeType ::= { id-at 3 }
+
+-- Naming attributes of type X520CommonName:
+--   X520CommonName ::= DirectoryName (SIZE (1..ub-common-name))
+--
+-- Expanded to avoid parameterized type:
+X520CommonName ::= CHOICE {
+      teletexString     TeletexString   (SIZE (1..ub-common-name)),
+      printableString   PrintableString (SIZE (1..ub-common-name)),
+      universalString   UniversalString (SIZE (1..ub-common-name)),
+      utf8String        UTF8String      (SIZE (1..ub-common-name)),
+      bmpString         BMPString       (SIZE (1..ub-common-name)) }
+
+-- Naming attributes of type X520LocalityName
+
+id-at-localityName      AttributeType ::= { id-at 7 }
+
+-- Naming attributes of type X520LocalityName:
+--   X520LocalityName ::= DirectoryName (SIZE (1..ub-locality-name))
+--
+-- Expanded to avoid parameterized type:
+X520LocalityName ::= CHOICE {
+      teletexString     TeletexString   (SIZE (1..ub-locality-name)),
+      printableString   PrintableString (SIZE (1..ub-locality-name)),
+      universalString   UniversalString (SIZE (1..ub-locality-name)),
+      utf8String        UTF8String      (SIZE (1..ub-locality-name)),
+      bmpString         BMPString       (SIZE (1..ub-locality-name)) }
+
+-- Naming attributes of type X520StateOrProvinceName
+
+id-at-stateOrProvinceName AttributeType ::= { id-at 8 }
+
+-- Naming attributes of type X520StateOrProvinceName:
+--   X520StateOrProvinceName ::= DirectoryName (SIZE (1..ub-state-name))
+--
+-- Expanded to avoid parameterized type:
+X520StateOrProvinceName ::= CHOICE {
+      teletexString     TeletexString   (SIZE (1..ub-state-name)),
+      printableString   PrintableString (SIZE (1..ub-state-name)),
+      universalString   UniversalString (SIZE (1..ub-state-name)),
+      utf8String        UTF8String      (SIZE (1..ub-state-name)),
+      bmpString         BMPString       (SIZE (1..ub-state-name)) }
+
+-- Naming attributes of type X520OrganizationName
+
+id-at-organizationName  AttributeType ::= { id-at 10 }
+
+-- Naming attributes of type X520OrganizationName:
+--   X520OrganizationName ::=
+--          DirectoryName (SIZE (1..ub-organization-name))
+--
+-- Expanded to avoid parameterized type:
+X520OrganizationName ::= CHOICE {
+      teletexString     TeletexString
+                          (SIZE (1..ub-organization-name)),
+      printableString   PrintableString
+                          (SIZE (1..ub-organization-name)),
+      universalString   UniversalString
+                          (SIZE (1..ub-organization-name)),
+      utf8String        UTF8String
+                          (SIZE (1..ub-organization-name)),
+      bmpString         BMPString
+                          (SIZE (1..ub-organization-name))  }
+
+-- Naming attributes of type X520OrganizationalUnitName
+
+id-at-organizationalUnitName AttributeType ::= { id-at 11 }
+
+-- Naming attributes of type X520OrganizationalUnitName:
+--   X520OrganizationalUnitName ::=
+--          DirectoryName (SIZE (1..ub-organizational-unit-name))
+--
+-- Expanded to avoid parameterized type:
+X520OrganizationalUnitName ::= CHOICE {
+      teletexString     TeletexString
+                          (SIZE (1..ub-organizational-unit-name)),
+      printableString   PrintableString
+                          (SIZE (1..ub-organizational-unit-name)),
+      universalString   UniversalString
+                          (SIZE (1..ub-organizational-unit-name)),
+      utf8String        UTF8String
+                          (SIZE (1..ub-organizational-unit-name)),
+      bmpString         BMPString
+                          (SIZE (1..ub-organizational-unit-name)) }
+
+-- Naming attributes of type X520Title
+
+id-at-title             AttributeType ::= { id-at 12 }
+
+-- Naming attributes of type X520Title:
+--   X520Title ::= DirectoryName (SIZE (1..ub-title))
+--
+-- Expanded to avoid parameterized type:
+X520Title ::= CHOICE {
+      teletexString     TeletexString   (SIZE (1..ub-title)),
+      printableString   PrintableString (SIZE (1..ub-title)),
+      universalString   UniversalString (SIZE (1..ub-title)),
+      utf8String        UTF8String      (SIZE (1..ub-title)),
+      bmpString         BMPString       (SIZE (1..ub-title)) }
+
+-- Naming attributes of type X520dnQualifier
+
+id-at-dnQualifier       AttributeType ::= { id-at 46 }
+
+X520dnQualifier ::=     PrintableString
+
+-- Naming attributes of type X520countryName (digraph from IS 3166)
+
+id-at-countryName       AttributeType ::= { id-at 6 }
+
+X520countryName ::=     PrintableString (SIZE (2))
+
+-- Naming attributes of type X520SerialNumber
+
+id-at-serialNumber      AttributeType ::= { id-at 5 }
+
+X520SerialNumber ::=    PrintableString (SIZE (1..ub-serial-number))
+
+-- Naming attributes of type X520Pseudonym
+
+id-at-pseudonym         AttributeType ::= { id-at 65 }
+
+-- Naming attributes of type X520Pseudonym:
+--   X520Pseudonym ::= DirectoryName (SIZE (1..ub-pseudonym))
+--
+-- Expanded to avoid parameterized type:
+X520Pseudonym ::= CHOICE {
+   teletexString     TeletexString   (SIZE (1..ub-pseudonym)),
+   printableString   PrintableString (SIZE (1..ub-pseudonym)),
+   universalString   UniversalString (SIZE (1..ub-pseudonym)),
+   utf8String        UTF8String      (SIZE (1..ub-pseudonym)),
+   bmpString         BMPString       (SIZE (1..ub-pseudonym)) }
+
+-- Naming attributes of type DomainComponent (from RFC 4519)
+
+id-domainComponent   AttributeType ::= { 0 9 2342 19200300 100 1 25 }
+
+DomainComponent ::=  IA5String
+
+-- Legacy attributes
+
+pkcs-9 OBJECT IDENTIFIER ::=
+       { iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) 9 }
+
+id-emailAddress      AttributeType ::= { pkcs-9 1 }
+
+EmailAddress ::=     IA5String (SIZE (1..ub-emailaddress-length))
+
+-- naming data types --
+
+Name ::= CHOICE { -- only one possibility for now --
+      rdnSequence  RDNSequence }
+
+RDNSequence ::= SEQUENCE OF RelativeDistinguishedName
+
+DistinguishedName ::=   RDNSequence
+
+RelativeDistinguishedName ::= SET SIZE (1..MAX) OF AttributeTypeAndValue
+
+-- Directory string type --
+
+DirectoryString ::= CHOICE {
+      teletexString       TeletexString   (SIZE (1..MAX)),
+      printableString     PrintableString (SIZE (1..MAX)),
+      universalString     UniversalString (SIZE (1..MAX)),
+      utf8String          UTF8String      (SIZE (1..MAX)),
+      bmpString           BMPString       (SIZE (1..MAX)) }
+
+-- certificate and CRL specific structures begin here
+
+Certificate  ::=  SEQUENCE  {
+     tbsCertificate       TBSCertificate,
+     signatureAlgorithm   AlgorithmIdentifier,
+     signature            BIT STRING  }
+
+TBSCertificate  ::=  SEQUENCE  {
+     version         [0]  Version DEFAULT v1,
+     serialNumber         CertificateSerialNumber,
+     signature            AlgorithmIdentifier,
+     issuer               Name,
+     validity             Validity,
+     subject              Name,
+     subjectPublicKeyInfo SubjectPublicKeyInfo,
+     issuerUniqueID  [1]  IMPLICIT UniqueIdentifier OPTIONAL,
+                          -- If present, version MUST be v2 or v3
+     subjectUniqueID [2]  IMPLICIT UniqueIdentifier OPTIONAL,
+                          -- If present, version MUST be v2 or v3
+     extensions      [3]  Extensions OPTIONAL
+                          -- If present, version MUST be v3 --  }
+
+Version  ::=  INTEGER  {  v1(0), v2(1), v3(2)  }
+
+CertificateSerialNumber  ::=  INTEGER
+
+Validity ::= SEQUENCE {
+     notBefore      Time,
+     notAfter       Time  }
+
+Time ::= CHOICE {
+     utcTime        UTCTime,
+     generalTime    GeneralizedTime }
+
+UniqueIdentifier  ::=  BIT STRING
+
+SubjectPublicKeyInfo  ::=  SEQUENCE  {
+     algorithm            AlgorithmIdentifier,
+     subjectPublicKey     BIT STRING  }
+
+Extensions  ::=  SEQUENCE SIZE (1..MAX) OF Extension
+
+Extension  ::=  SEQUENCE  {
+     extnID      OBJECT IDENTIFIER,
+     critical    BOOLEAN DEFAULT FALSE,
+     extnValue   OCTET STRING
+                 -- contains the DER encoding of an ASN.1 value
+                 -- corresponding to the extension type identified
+                 -- by extnID
+     }
+
+-- CRL structures
+
+CertificateList  ::=  SEQUENCE  {
+     tbsCertList          TBSCertList,
+     signatureAlgorithm   AlgorithmIdentifier,
+     signature            BIT STRING  }
+
+TBSCertList  ::=  SEQUENCE  {
+     version                 Version OPTIONAL,
+                                   -- if present, MUST be v2
+     signature               AlgorithmIdentifier,
+     issuer                  Name,
+     thisUpdate              Time,
+     nextUpdate              Time OPTIONAL,
+     revokedCertificates     SEQUENCE OF SEQUENCE  {
+          userCertificate         CertificateSerialNumber,
+          revocationDate          Time,
+          crlEntryExtensions      Extensions OPTIONAL
+                                   -- if present, version MUST be v2
+                               }  OPTIONAL,
+     crlExtensions           [0] Extensions OPTIONAL }
+                                   -- if present, version MUST be v2
+
+-- Version, Time, CertificateSerialNumber, and Extensions were
+-- defined earlier for use in the certificate structure
+
+AlgorithmIdentifier  ::=  SEQUENCE  {
+     algorithm               OBJECT IDENTIFIER,
+     parameters              ANY DEFINED BY algorithm OPTIONAL  }
+                                -- contains a value of the type
+                                -- registered for use with the
+                                -- algorithm object identifier value
+
+-- X.400 address syntax starts here
+
+ORAddress ::= SEQUENCE {
+   built-in-standard-attributes BuiltInStandardAttributes,
+   built-in-domain-defined-attributes
+                   BuiltInDomainDefinedAttributes OPTIONAL,
+   -- see also teletex-domain-defined-attributes
+   extension-attributes ExtensionAttributes OPTIONAL }
+
+-- Built-in Standard Attributes
+
+BuiltInStandardAttributes ::= SEQUENCE {
+   country-name                  CountryName OPTIONAL,
+   administration-domain-name    AdministrationDomainName OPTIONAL,
+   network-address           [0] IMPLICIT NetworkAddress OPTIONAL,
+     -- see also extended-network-address
+   terminal-identifier       [1] IMPLICIT TerminalIdentifier OPTIONAL,
+   private-domain-name       [2] PrivateDomainName OPTIONAL,
+   organization-name         [3] IMPLICIT OrganizationName OPTIONAL,
+     -- see also teletex-organization-name
+   numeric-user-identifier   [4] IMPLICIT NumericUserIdentifier
+                                 OPTIONAL,
+   personal-name             [5] IMPLICIT PersonalName OPTIONAL,
+     -- see also teletex-personal-name
+   organizational-unit-names [6] IMPLICIT OrganizationalUnitNames
+                                 OPTIONAL }
+     -- see also teletex-organizational-unit-names
+
+CountryName ::= [APPLICATION 1] CHOICE {
+   x121-dcc-code         NumericString
+                           (SIZE (ub-country-name-numeric-length)),
+   iso-3166-alpha2-code  PrintableString
+                           (SIZE (ub-country-name-alpha-length)) }
+
+AdministrationDomainName ::= [APPLICATION 2] CHOICE {
+   numeric   NumericString   (SIZE (0..ub-domain-name-length)),
+   printable PrintableString (SIZE (0..ub-domain-name-length)) }
+
+NetworkAddress ::= X121Address  -- see also extended-network-address
+
+X121Address ::= NumericString (SIZE (1..ub-x121-address-length))
+
+TerminalIdentifier ::= PrintableString (SIZE (1..ub-terminal-id-length))
+
+PrivateDomainName ::= CHOICE {
+   numeric   NumericString   (SIZE (1..ub-domain-name-length)),
+   printable PrintableString (SIZE (1..ub-domain-name-length)) }
+
+OrganizationName ::= PrintableString
+                            (SIZE (1..ub-organization-name-length))
+  -- see also teletex-organization-name
+
+NumericUserIdentifier ::= NumericString
+                            (SIZE (1..ub-numeric-user-id-length))
+
+PersonalName ::= SET {
+   surname     [0] IMPLICIT PrintableString
+                    (SIZE (1..ub-surname-length)),
+   given-name  [1] IMPLICIT PrintableString
+                    (SIZE (1..ub-given-name-length)) OPTIONAL,
+   initials    [2] IMPLICIT PrintableString
+                    (SIZE (1..ub-initials-length)) OPTIONAL,
+   generation-qualifier [3] IMPLICIT PrintableString
+                    (SIZE (1..ub-generation-qualifier-length))
+                    OPTIONAL }
+  -- see also teletex-personal-name
+
+OrganizationalUnitNames ::= SEQUENCE SIZE (1..ub-organizational-units)
+                             OF OrganizationalUnitName
+  -- see also teletex-organizational-unit-names
+
+OrganizationalUnitName ::= PrintableString (SIZE
+                    (1..ub-organizational-unit-name-length))
+
+-- Built-in Domain-defined Attributes
+
+BuiltInDomainDefinedAttributes ::= SEQUENCE SIZE
+                    (1..ub-domain-defined-attributes) OF
+                    BuiltInDomainDefinedAttribute
+
+BuiltInDomainDefinedAttribute ::= SEQUENCE {
+   type PrintableString (SIZE
+                   (1..ub-domain-defined-attribute-type-length)),
+   value PrintableString (SIZE
+                   (1..ub-domain-defined-attribute-value-length)) }
+
+-- Extension Attributes
+
+ExtensionAttributes ::= SET SIZE (1..ub-extension-attributes) OF
+               ExtensionAttribute
+
+ExtensionAttribute ::=  SEQUENCE {
+   extension-attribute-type [0] IMPLICIT INTEGER
+                   (0..ub-extension-attributes),
+   extension-attribute-value [1]
+                   ANY DEFINED BY extension-attribute-type }
+
+-- Extension types and attribute values
+
+common-name INTEGER ::= 1
+
+CommonName ::= PrintableString (SIZE (1..ub-common-name-length))
+
+teletex-common-name INTEGER ::= 2
+
+TeletexCommonName ::= TeletexString (SIZE (1..ub-common-name-length))
+
+teletex-organization-name INTEGER ::= 3
+
+TeletexOrganizationName ::=
+                TeletexString (SIZE (1..ub-organization-name-length))
+
+teletex-personal-name INTEGER ::= 4
+
+TeletexPersonalName ::= SET {
+   surname     [0] IMPLICIT TeletexString
+                    (SIZE (1..ub-surname-length)),
+   given-name  [1] IMPLICIT TeletexString
+                    (SIZE (1..ub-given-name-length)) OPTIONAL,
+   initials    [2] IMPLICIT TeletexString
+                    (SIZE (1..ub-initials-length)) OPTIONAL,
+   generation-qualifier [3] IMPLICIT TeletexString
+                    (SIZE (1..ub-generation-qualifier-length))
+                    OPTIONAL }
+
+teletex-organizational-unit-names INTEGER ::= 5
+
+TeletexOrganizationalUnitNames ::= SEQUENCE SIZE
+      (1..ub-organizational-units) OF TeletexOrganizationalUnitName
+
+TeletexOrganizationalUnitName ::= TeletexString
+                  (SIZE (1..ub-organizational-unit-name-length))
+
+pds-name INTEGER ::= 7
+
+PDSName ::= PrintableString (SIZE (1..ub-pds-name-length))
+
+physical-delivery-country-name INTEGER ::= 8
+
+PhysicalDeliveryCountryName ::= CHOICE {
+   x121-dcc-code NumericString (SIZE (ub-country-name-numeric-length)),
+   iso-3166-alpha2-code PrintableString
+                               (SIZE (ub-country-name-alpha-length)) }
+
+postal-code INTEGER ::= 9
+
+PostalCode ::= CHOICE {
+   numeric-code   NumericString (SIZE (1..ub-postal-code-length)),
+   printable-code PrintableString (SIZE (1..ub-postal-code-length)) }
+
+physical-delivery-office-name INTEGER ::= 10
+
+PhysicalDeliveryOfficeName ::= PDSParameter
+
+physical-delivery-office-number INTEGER ::= 11
+
+PhysicalDeliveryOfficeNumber ::= PDSParameter
+
+extension-OR-address-components INTEGER ::= 12
+
+ExtensionORAddressComponents ::= PDSParameter
+
+physical-delivery-personal-name INTEGER ::= 13
+
+PhysicalDeliveryPersonalName ::= PDSParameter
+
+physical-delivery-organization-name INTEGER ::= 14
+
+PhysicalDeliveryOrganizationName ::= PDSParameter
+
+extension-physical-delivery-address-components INTEGER ::= 15
+
+ExtensionPhysicalDeliveryAddressComponents ::= PDSParameter
+
+unformatted-postal-address INTEGER ::= 16
+
+UnformattedPostalAddress ::= SET {
+   printable-address SEQUENCE SIZE (1..ub-pds-physical-address-lines)
+        OF PrintableString (SIZE (1..ub-pds-parameter-length)) OPTIONAL,
+   teletex-string TeletexString
+        (SIZE (1..ub-unformatted-address-length)) OPTIONAL }
+
+street-address INTEGER ::= 17
+
+StreetAddress ::= PDSParameter
+
+post-office-box-address INTEGER ::= 18
+
+PostOfficeBoxAddress ::= PDSParameter
+
+poste-restante-address INTEGER ::= 19
+
+PosteRestanteAddress ::= PDSParameter
+
+unique-postal-name INTEGER ::= 20
+
+UniquePostalName ::= PDSParameter
+
+local-postal-attributes INTEGER ::= 21
+
+LocalPostalAttributes ::= PDSParameter
+
+PDSParameter ::= SET {
+   printable-string PrintableString
+                (SIZE(1..ub-pds-parameter-length)) OPTIONAL,
+   teletex-string TeletexString
+                (SIZE(1..ub-pds-parameter-length)) OPTIONAL }
+
+extended-network-address INTEGER ::= 22
+
+ExtendedNetworkAddress ::= CHOICE {
+   e163-4-address SEQUENCE {
+      number      [0] IMPLICIT NumericString
+                       (SIZE (1..ub-e163-4-number-length)),
+      sub-address [1] IMPLICIT NumericString
+                       (SIZE (1..ub-e163-4-sub-address-length))
+                       OPTIONAL },
+   psap-address   [0] IMPLICIT PresentationAddress }
+
+PresentationAddress ::= SEQUENCE {
+    pSelector     [0] EXPLICIT OCTET STRING OPTIONAL,
+    sSelector     [1] EXPLICIT OCTET STRING OPTIONAL,
+    tSelector     [2] EXPLICIT OCTET STRING OPTIONAL,
+    nAddresses    [3] EXPLICIT SET SIZE (1..MAX) OF OCTET STRING }
+
+terminal-type  INTEGER ::= 23
+
+TerminalType ::= INTEGER {
+   telex        (3),
+   teletex      (4),
+   g3-facsimile (5),
+   g4-facsimile (6),
+   ia5-terminal (7),
+   videotex     (8) } (0..ub-integer-options)
+
+-- Extension Domain-defined Attributes
+
+teletex-domain-defined-attributes INTEGER ::= 6
+
+TeletexDomainDefinedAttributes ::= SEQUENCE SIZE
+   (1..ub-domain-defined-attributes) OF TeletexDomainDefinedAttribute
+
+TeletexDomainDefinedAttribute ::= SEQUENCE {
+        type TeletexString
+               (SIZE (1..ub-domain-defined-attribute-type-length)),
+        value TeletexString
+               (SIZE (1..ub-domain-defined-attribute-value-length)) }
+
+--  specifications of Upper Bounds MUST be regarded as mandatory
+--  from Annex B of ITU-T X.411 Reference Definition of MTS Parameter
+--  Upper Bounds
+
+-- Upper Bounds
+ub-name INTEGER ::= 32768
+ub-common-name INTEGER ::= 64
+ub-locality-name INTEGER ::= 128
+ub-state-name INTEGER ::= 128
+ub-organization-name INTEGER ::= 64
+ub-organizational-unit-name INTEGER ::= 64
+ub-title INTEGER ::= 64
+ub-serial-number INTEGER ::= 64
+ub-match INTEGER ::= 128
+ub-emailaddress-length INTEGER ::= 255
+ub-common-name-length INTEGER ::= 64
+ub-country-name-alpha-length INTEGER ::= 2
+ub-country-name-numeric-length INTEGER ::= 3
+ub-domain-defined-attributes INTEGER ::= 4
+ub-domain-defined-attribute-type-length INTEGER ::= 8
+ub-domain-defined-attribute-value-length INTEGER ::= 128
+ub-domain-name-length INTEGER ::= 16
+ub-extension-attributes INTEGER ::= 256
+ub-e163-4-number-length INTEGER ::= 15
+ub-e163-4-sub-address-length INTEGER ::= 40
+ub-generation-qualifier-length INTEGER ::= 3
+ub-given-name-length INTEGER ::= 16
+ub-initials-length INTEGER ::= 5
+ub-integer-options INTEGER ::= 256
+ub-numeric-user-id-length INTEGER ::= 32
+ub-organization-name-length INTEGER ::= 64
+ub-organizational-unit-name-length INTEGER ::= 32
+ub-organizational-units INTEGER ::= 4
+ub-pds-name-length INTEGER ::= 16
+ub-pds-parameter-length INTEGER ::= 30
+ub-pds-physical-address-lines INTEGER ::= 6
+ub-postal-code-length INTEGER ::= 16
+ub-pseudonym INTEGER ::= 128
+ub-surname-length INTEGER ::= 40
+ub-terminal-id-length INTEGER ::= 24
+ub-unformatted-address-length INTEGER ::= 180
+ub-x121-address-length INTEGER ::= 16
+
+-- Note - upper bounds on string types, such as TeletexString, are
+-- measured in characters.  Excepting PrintableString or IA5String, a
+-- significantly greater number of octets will be required to hold
+-- such a value.  As a minimum, 16 octets, or twice the specified
+-- upper bound, whichever is the larger, should be allowed for
+-- TeletexString.  For UTF8String or UniversalString at least four
+-- times the upper bound should be allowed.
+
+END
+

pySim/esim/asn1/rsp/PKIX1Implicit88.asn

@@ -0,0 +1,343 @@
+PKIX1Implicit88 { iso(1) identified-organization(3) dod(6) internet(1)
+  security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-implicit(19) }
+
+DEFINITIONS IMPLICIT TAGS ::=
+
+BEGIN
+
+-- EXPORTS ALL --
+
+IMPORTS
+      id-pe, id-kp, id-qt-unotice, id-qt-cps,
+      ORAddress, Name, RelativeDistinguishedName,
+      CertificateSerialNumber, Attribute, DirectoryString
+      FROM PKIX1Explicit88 { iso(1) identified-organization(3)
+            dod(6) internet(1) security(5) mechanisms(5) pkix(7)
+            id-mod(0) id-pkix1-explicit(18) };
+
+-- ISO arc for standard certificate and CRL extensions
+
+id-ce OBJECT IDENTIFIER  ::=  {joint-iso-ccitt(2) ds(5) 29}
+
+-- authority key identifier OID and syntax
+
+id-ce-authorityKeyIdentifier OBJECT IDENTIFIER ::=  { id-ce 35 }
+
+AuthorityKeyIdentifier ::= SEQUENCE {
+    keyIdentifier             [0] KeyIdentifier            OPTIONAL,
+    authorityCertIssuer       [1] GeneralNames             OPTIONAL,
+    authorityCertSerialNumber [2] CertificateSerialNumber  OPTIONAL }
+    -- authorityCertIssuer and authorityCertSerialNumber MUST both
+    -- be present or both be absent
+
+KeyIdentifier ::= OCTET STRING
+
+-- subject key identifier OID and syntax
+
+id-ce-subjectKeyIdentifier OBJECT IDENTIFIER ::=  { id-ce 14 }
+
+SubjectKeyIdentifier ::= KeyIdentifier
+
+-- key usage extension OID and syntax
+
+id-ce-keyUsage OBJECT IDENTIFIER ::=  { id-ce 15 }
+
+KeyUsage ::= BIT STRING {
+     digitalSignature        (0),
+     nonRepudiation          (1),  -- recent editions of X.509 have
+                                -- renamed this bit to contentCommitment
+     keyEncipherment         (2),
+     dataEncipherment        (3),
+     keyAgreement            (4),
+     keyCertSign             (5),
+     cRLSign                 (6),
+     encipherOnly            (7),
+     decipherOnly            (8) }
+
+-- private key usage period extension OID and syntax
+
+id-ce-privateKeyUsagePeriod OBJECT IDENTIFIER ::=  { id-ce 16 }
+
+PrivateKeyUsagePeriod ::= SEQUENCE {
+     notBefore       [0]     GeneralizedTime OPTIONAL,
+     notAfter        [1]     GeneralizedTime OPTIONAL }
+     -- either notBefore or notAfter MUST be present
+
+-- certificate policies extension OID and syntax
+
+id-ce-certificatePolicies OBJECT IDENTIFIER ::=  { id-ce 32 }
+
+anyPolicy OBJECT IDENTIFIER ::= { id-ce-certificatePolicies 0 }
+
+CertificatePolicies ::= SEQUENCE SIZE (1..MAX) OF PolicyInformation
+
+PolicyInformation ::= SEQUENCE {
+     policyIdentifier   CertPolicyId,
+     policyQualifiers   SEQUENCE SIZE (1..MAX) OF
+             PolicyQualifierInfo OPTIONAL }
+
+CertPolicyId ::= OBJECT IDENTIFIER
+
+PolicyQualifierInfo ::= SEQUENCE {
+     policyQualifierId  PolicyQualifierId,
+     qualifier          ANY DEFINED BY policyQualifierId }
+
+-- Implementations that recognize additional policy qualifiers MUST
+-- augment the following definition for PolicyQualifierId
+
+PolicyQualifierId ::= OBJECT IDENTIFIER ( id-qt-cps | id-qt-unotice )
+
+-- CPS pointer qualifier
+
+CPSuri ::= IA5String
+
+-- user notice qualifier
+
+UserNotice ::= SEQUENCE {
+     noticeRef        NoticeReference OPTIONAL,
+     explicitText     DisplayText OPTIONAL }
+
+NoticeReference ::= SEQUENCE {
+     organization     DisplayText,
+     noticeNumbers    SEQUENCE OF INTEGER }
+
+DisplayText ::= CHOICE {
+     ia5String        IA5String      (SIZE (1..200)),
+     visibleString    VisibleString  (SIZE (1..200)),
+     bmpString        BMPString      (SIZE (1..200)),
+     utf8String       UTF8String     (SIZE (1..200)) }
+
+-- policy mapping extension OID and syntax
+
+id-ce-policyMappings OBJECT IDENTIFIER ::=  { id-ce 33 }
+
+PolicyMappings ::= SEQUENCE SIZE (1..MAX) OF SEQUENCE {
+     issuerDomainPolicy      CertPolicyId,
+     subjectDomainPolicy     CertPolicyId }
+
+-- subject alternative name extension OID and syntax
+
+id-ce-subjectAltName OBJECT IDENTIFIER ::=  { id-ce 17 }
+
+SubjectAltName ::= GeneralNames
+
+GeneralNames ::= SEQUENCE SIZE (1..MAX) OF GeneralName
+
+GeneralName ::= CHOICE {
+     otherName                 [0]  AnotherName,
+     rfc822Name                [1]  IA5String,
+     dNSName                   [2]  IA5String,
+     x400Address               [3]  ORAddress,
+     directoryName             [4]  Name,
+     ediPartyName              [5]  EDIPartyName,
+     uniformResourceIdentifier [6]  IA5String,
+     iPAddress                 [7]  OCTET STRING,
+     registeredID              [8]  OBJECT IDENTIFIER }
+
+-- AnotherName replaces OTHER-NAME ::= TYPE-IDENTIFIER, as
+-- TYPE-IDENTIFIER is not supported in the '88 ASN.1 syntax
+
+AnotherName ::= SEQUENCE {
+     type-id    OBJECT IDENTIFIER,
+     value      [0] EXPLICIT ANY DEFINED BY type-id }
+
+EDIPartyName ::= SEQUENCE {
+     nameAssigner              [0]  DirectoryString OPTIONAL,
+     partyName                 [1]  DirectoryString }
+
+-- issuer alternative name extension OID and syntax
+
+id-ce-issuerAltName OBJECT IDENTIFIER ::=  { id-ce 18 }
+
+IssuerAltName ::= GeneralNames
+
+id-ce-subjectDirectoryAttributes OBJECT IDENTIFIER ::=  { id-ce 9 }
+
+SubjectDirectoryAttributes ::= SEQUENCE SIZE (1..MAX) OF Attribute
+
+-- basic constraints extension OID and syntax
+
+id-ce-basicConstraints OBJECT IDENTIFIER ::=  { id-ce 19 }
+
+BasicConstraints ::= SEQUENCE {
+     cA                      BOOLEAN DEFAULT FALSE,
+     pathLenConstraint       INTEGER (0..MAX) OPTIONAL }
+
+-- name constraints extension OID and syntax
+
+id-ce-nameConstraints OBJECT IDENTIFIER ::=  { id-ce 30 }
+
+NameConstraints ::= SEQUENCE {
+     permittedSubtrees       [0]     GeneralSubtrees OPTIONAL,
+     excludedSubtrees        [1]     GeneralSubtrees OPTIONAL }
+
+GeneralSubtrees ::= SEQUENCE SIZE (1..MAX) OF GeneralSubtree
+
+GeneralSubtree ::= SEQUENCE {
+     base                    GeneralName,
+     minimum         [0]     BaseDistance DEFAULT 0,
+     maximum         [1]     BaseDistance OPTIONAL }
+
+BaseDistance ::= INTEGER (0..MAX)
+
+-- policy constraints extension OID and syntax
+
+id-ce-policyConstraints OBJECT IDENTIFIER ::=  { id-ce 36 }
+
+PolicyConstraints ::= SEQUENCE {
+     requireExplicitPolicy   [0]     SkipCerts OPTIONAL,
+     inhibitPolicyMapping    [1]     SkipCerts OPTIONAL }
+
+SkipCerts ::= INTEGER (0..MAX)
+
+-- CRL distribution points extension OID and syntax
+
+id-ce-cRLDistributionPoints     OBJECT IDENTIFIER  ::=  {id-ce 31}
+
+CRLDistributionPoints ::= SEQUENCE SIZE (1..MAX) OF DistributionPoint
+
+DistributionPoint ::= SEQUENCE {
+     distributionPoint       [0]     DistributionPointName OPTIONAL,
+     reasons                 [1]     ReasonFlags OPTIONAL,
+     cRLIssuer               [2]     GeneralNames OPTIONAL }
+
+DistributionPointName ::= CHOICE {
+     fullName                [0]     GeneralNames,
+     nameRelativeToCRLIssuer [1]     RelativeDistinguishedName }
+
+ReasonFlags ::= BIT STRING {
+     unused                  (0),
+     keyCompromise           (1),
+     cACompromise            (2),
+     affiliationChanged      (3),
+     superseded              (4),
+     cessationOfOperation    (5),
+     certificateHold         (6),
+     privilegeWithdrawn      (7),
+     aACompromise            (8) }
+
+-- extended key usage extension OID and syntax
+
+id-ce-extKeyUsage OBJECT IDENTIFIER ::= {id-ce 37}
+
+ExtKeyUsageSyntax ::= SEQUENCE SIZE (1..MAX) OF KeyPurposeId
+
+KeyPurposeId ::= OBJECT IDENTIFIER
+
+-- permit unspecified key uses
+
+anyExtendedKeyUsage OBJECT IDENTIFIER ::= { id-ce-extKeyUsage 0 }
+
+-- extended key purpose OIDs
+
+id-kp-serverAuth             OBJECT IDENTIFIER ::= { id-kp 1 }
+id-kp-clientAuth             OBJECT IDENTIFIER ::= { id-kp 2 }
+id-kp-codeSigning            OBJECT IDENTIFIER ::= { id-kp 3 }
+id-kp-emailProtection        OBJECT IDENTIFIER ::= { id-kp 4 }
+id-kp-timeStamping           OBJECT IDENTIFIER ::= { id-kp 8 }
+id-kp-OCSPSigning            OBJECT IDENTIFIER ::= { id-kp 9 }
+
+-- inhibit any policy OID and syntax
+
+id-ce-inhibitAnyPolicy OBJECT IDENTIFIER ::=  { id-ce 54 }
+
+InhibitAnyPolicy ::= SkipCerts
+
+-- freshest (delta)CRL extension OID and syntax
+
+id-ce-freshestCRL OBJECT IDENTIFIER ::=  { id-ce 46 }
+
+FreshestCRL ::= CRLDistributionPoints
+
+-- authority info access
+
+id-pe-authorityInfoAccess OBJECT IDENTIFIER ::= { id-pe 1 }
+
+AuthorityInfoAccessSyntax  ::=
+        SEQUENCE SIZE (1..MAX) OF AccessDescription
+
+AccessDescription  ::=  SEQUENCE {
+        accessMethod          OBJECT IDENTIFIER,
+        accessLocation        GeneralName  }
+
+-- subject info access
+
+id-pe-subjectInfoAccess OBJECT IDENTIFIER ::= { id-pe 11 }
+
+SubjectInfoAccessSyntax  ::=
+        SEQUENCE SIZE (1..MAX) OF AccessDescription
+
+-- CRL number extension OID and syntax
+
+id-ce-cRLNumber OBJECT IDENTIFIER ::= { id-ce 20 }
+
+CRLNumber ::= INTEGER (0..MAX)
+
+-- issuing distribution point extension OID and syntax
+
+id-ce-issuingDistributionPoint OBJECT IDENTIFIER ::= { id-ce 28 }
+
+IssuingDistributionPoint ::= SEQUENCE {
+     distributionPoint          [0] DistributionPointName OPTIONAL,
+     onlyContainsUserCerts      [1] BOOLEAN DEFAULT FALSE,
+     onlyContainsCACerts        [2] BOOLEAN DEFAULT FALSE,
+     onlySomeReasons            [3] ReasonFlags OPTIONAL,
+     indirectCRL                [4] BOOLEAN DEFAULT FALSE,
+     onlyContainsAttributeCerts [5] BOOLEAN DEFAULT FALSE }
+     -- at most one of onlyContainsUserCerts, onlyContainsCACerts,
+     -- and onlyContainsAttributeCerts may be set to TRUE.
+
+id-ce-deltaCRLIndicator OBJECT IDENTIFIER ::= { id-ce 27 }
+
+BaseCRLNumber ::= CRLNumber
+
+-- reason code extension OID and syntax
+
+id-ce-cRLReasons OBJECT IDENTIFIER ::= { id-ce 21 }
+
+CRLReason ::= ENUMERATED {
+     unspecified             (0),
+     keyCompromise           (1),
+     cACompromise            (2),
+     affiliationChanged      (3),
+     superseded              (4),
+     cessationOfOperation    (5),
+     certificateHold         (6),
+     removeFromCRL           (8),
+     privilegeWithdrawn      (9),
+     aACompromise           (10) }
+
+-- certificate issuer CRL entry extension OID and syntax
+
+id-ce-certificateIssuer OBJECT IDENTIFIER ::= { id-ce 29 }
+
+CertificateIssuer ::= GeneralNames
+
+-- hold instruction extension OID and syntax
+
+id-ce-holdInstructionCode OBJECT IDENTIFIER ::= { id-ce 23 }
+
+HoldInstructionCode ::= OBJECT IDENTIFIER
+
+-- ANSI x9 arc holdinstruction arc
+
+holdInstruction OBJECT IDENTIFIER ::=
+          {joint-iso-itu-t(2) member-body(2) us(840) x9cm(10040) 2}
+
+-- ANSI X9 holdinstructions
+
+id-holdinstruction-none OBJECT IDENTIFIER  ::=
+                                      {holdInstruction 1} -- deprecated
+
+id-holdinstruction-callissuer OBJECT IDENTIFIER ::= {holdInstruction 2}
+
+id-holdinstruction-reject OBJECT IDENTIFIER ::= {holdInstruction 3}
+
+-- invalidity date CRL entry extension OID and syntax
+
+id-ce-invalidityDate OBJECT IDENTIFIER ::= { id-ce 24 }
+
+InvalidityDate ::=  GeneralizedTime
+
+END
+

pySim/esim/asn1/rsp/rsp.asn

@@ -0,0 +1,785 @@
+RSPDefinitions {joint-iso-itu-t(2) international-organizations(23) gsma(146) rsp(1) spec-version(1) version-two(2)}
+DEFINITIONS
+AUTOMATIC TAGS
+EXTENSIBILITY IMPLIED ::=
+BEGIN
+ 
+IMPORTS Certificate, CertificateList, Time FROM PKIX1Explicit88 {iso(1) identified-organization(3) dod(6) internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-explicit(18)}
+SubjectKeyIdentifier FROM PKIX1Implicit88 {iso(1) identified-organization(3) dod(6) internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-implicit(19)};
+ 
+id-rsp OBJECT IDENTIFIER ::= {joint-iso-itu-t(2) international-organizations(23) gsma(146) rsp(1)}
+ 
+-- Basic types, for size constraints
+Octet8 ::= OCTET STRING (SIZE(8))
+Octet16 ::= OCTET STRING (SIZE(16))
+OctetTo16 ::= OCTET STRING (SIZE(1..16))
+Octet32 ::= OCTET STRING (SIZE(32))
+Octet1 ::= OCTET STRING(SIZE(1))
+Octet2 ::= OCTET STRING (SIZE(2))
+VersionType ::= OCTET STRING(SIZE(3)) -- major/minor/revision version are coded as binary value on byte 1/2/3, e.g. '02 00 0C' for v2.0.12.
+Iccid ::= [APPLICATION 26] OCTET STRING (SIZE(10)) -- ICCID as coded in EFiccid, corresponding tag is '5A'
+RemoteOpId ::= [2] INTEGER {installBoundProfilePackage(1)}
+TransactionId ::= OCTET STRING (SIZE(1..16))
+ 
+-- Definition of EUICCInfo1 --------------------------
+GetEuiccInfo1Request ::= [32] SEQUENCE { -- Tag 'BF20'
+}
+ 
+EUICCInfo1 ::= [32] SEQUENCE { -- Tag 'BF20'
+    svn [2] VersionType,    -- GSMA SGP.22 version supported (SVN)
+    euiccCiPKIdListForVerification [9] SEQUENCE OF SubjectKeyIdentifier, -- List of CI Public Key Identifiers supported on the eUICC for signature verification
+    euiccCiPKIdListForSigning [10] SEQUENCE OF SubjectKeyIdentifier -- List of CI Public Key Identifier supported on the eUICC for signature creation
+}
+ 
+-- Definition of EUICCInfo2 --------------------------
+GetEuiccInfo2Request ::= [34] SEQUENCE { -- Tag 'BF22'
+}
+ 
+EUICCInfo2 ::= [34] SEQUENCE { -- Tag 'BF22'
+    profileVersion [1] VersionType,     -- SIMAlliance Profile package version supported
+    svn [2] VersionType,    -- GSMA SGP.22 version supported (SVN)
+    euiccFirmwareVer [3] VersionType,   -- eUICC Firmware version
+    extCardResource [4] OCTET STRING,   -- Extended Card Resource Information according to ETSI TS 102 226
+    uiccCapability [5] UICCCapability,
+    javacardVersion [6] VersionType OPTIONAL,
+    globalplatformVersion [7] VersionType OPTIONAL,
+    rspCapability [8] RspCapability,
+    euiccCiPKIdListForVerification [9] SEQUENCE OF SubjectKeyIdentifier, -- List of CI Public Key Identifiers supported on the eUICC for signature verification
+    euiccCiPKIdListForSigning [10] SEQUENCE OF SubjectKeyIdentifier, -- List of CI Public Key Identifier supported on the eUICC for signature creation
+    euiccCategory [11] INTEGER {
+        other(0),
+        basicEuicc(1),
+        mediumEuicc(2),
+        contactlessEuicc(3)
+    } OPTIONAL,
+    forbiddenProfilePolicyRules [25] PprIds OPTIONAL, -- Tag '99'
+    ppVersion VersionType, -- Protection Profile version
+    sasAcreditationNumber UTF8String (SIZE(0..64)),
+    certificationDataObject [12] CertificationDataObject OPTIONAL
+}
+ 
+-- Definition of RspCapability
+RspCapability ::= BIT STRING {
+    additionalProfile(0), -- at least one more Profile can be installed
+    crlSupport(1), -- CRL
+    rpmSupport(2), -- Remote Profile Management
+    testProfileSupport (3) -- support for test profile
+}
+ 
+-- Definition of CertificationDataObject
+CertificationDataObject ::= SEQUENCE {
+    platformLabel UTF8String,   -- Platform_Label as defined in GlobalPlatform DLOA specification [57]
+    discoveryBaseURL UTF8String -- Discovery Base URL of the SE default DLOA Registrar as defined in GlobalPlatform DLOA specification [57]
+}
+ 
+CertificateInfo ::= BIT STRING {
+ 
+    reserved(0),    -- eUICC has a CERT.EUICC.ECDSA in GlobalPlatform format. The use of this bit is deprecated.
+    certSigningX509(1),     -- eUICC has a CERT.EUICC.ECDSA in X.509 format
+    rfu2(2),
+    rfu3(3),
+    reserved2(4), -- Handling of Certificate in GlobalPlatform format. The use of this bit is deprecated.
+    certVerificationX509(5)-- Handling of Certificate in X.509 format
+}
+ 
+-- Definition of UICCCapability
+UICCCapability ::= BIT STRING {
+/* Sequence is derived from ServicesList[] defined in SIMalliance PEDefinitions*/
+    contactlessSupport(0),  -- Contactless (SWP, HCI and associated APIs)
+    usimSupport(1),         -- USIM as defined by 3GPP
+    isimSupport(2),         -- ISIM as defined by 3GPP
+    csimSupport(3),         -- CSIM as defined by 3GPP2
+ 
+    akaMilenage(4),         -- Milenage as AKA algorithm
+    akaCave(5),             -- CAVE as authentication algorithm
+    akaTuak128(6),          -- TUAK as AKA algorithm with 128 bit key length
+    akaTuak256(7),          -- TUAK as AKA algorithm with 256 bit key length
+    rfu1(8),                    -- reserved for further algorithms
+    rfu2(9),                    -- reserved for further algorithms
+ 
+    gbaAuthenUsim(10),  -- GBA authentication in the context of USIM
+    gbaAuthenISim(11),  -- GBA authentication in the context of ISIM
+    mbmsAuthenUsim(12),     -- MBMS authentication in the context of USIM
+    eapClient(13),          -- EAP client
+ 
+    javacard(14),               -- Javacard support
+    multos(15),             -- Multos support
+ 
+    multipleUsimSupport(16),     -- Multiple USIM applications are supported within the same Profile
+    multipleIsimSupport(17),      -- Multiple ISIM applications are supported within the same Profile
+    multipleCsimSupport(18)   -- Multiple CSIM applications are supported within the same Profile
+}
+ 
+-- Definition of DeviceInfo
+DeviceInfo ::= SEQUENCE {
+    tac Octet8,
+    deviceCapabilities DeviceCapabilities,
+    imei Octet8 OPTIONAL
+}
+ 
+DeviceCapabilities ::= SEQUENCE { -- Highest fully supported release for each definition
+  -- The device SHALL set all the capabilities it supports
+    gsmSupportedRelease VersionType OPTIONAL,
+    utranSupportedRelease VersionType OPTIONAL,
+    cdma2000onexSupportedRelease VersionType OPTIONAL,
+    cdma2000hrpdSupportedRelease VersionType OPTIONAL,
+    cdma2000ehrpdSupportedRelease VersionType OPTIONAL,
+    eutranSupportedRelease VersionType OPTIONAL,
+    contactlessSupportedRelease VersionType OPTIONAL,
+    rspCrlSupportedVersion VersionType OPTIONAL,
+    rspRpmSupportedVersion VersionType OPTIONAL
+}
+ 
+ProfileInfoListRequest ::= [45] SEQUENCE { -- Tag 'BF2D'
+    searchCriteria [0] CHOICE {
+        isdpAid [APPLICATION 15] OctetTo16, -- AID of the ISD-P, tag '4F'
+        iccid Iccid, -- ICCID, tag '5A'
+        profileClass [21] ProfileClass -- Tag '95'
+    } OPTIONAL,
+    tagList [APPLICATION 28] OCTET STRING OPTIONAL -- tag '5C'
+}
+ 
+-- Definition of ProfileInfoList
+ProfileInfoListResponse ::= [45] CHOICE { -- Tag 'BF2D'
+    profileInfoListOk SEQUENCE OF ProfileInfo,
+    profileInfoListError ProfileInfoListError
+}
+ 
+ProfileInfo ::= [PRIVATE 3] SEQUENCE { -- Tag 'E3'
+    iccid Iccid OPTIONAL,
+    isdpAid [APPLICATION 15] OctetTo16 OPTIONAL, -- AID of the ISD-P containing the Profile, tag '4F'
+    profileState [112] ProfileState OPTIONAL, -- Tag '9F70'
+    profileNickname [16] UTF8String (SIZE(0..64)) OPTIONAL, -- Tag '90'
+    serviceProviderName [17] UTF8String (SIZE(0..32)) OPTIONAL, -- Tag '91'
+    profileName [18] UTF8String (SIZE(0..64)) OPTIONAL, -- Tag '92'
+    iconType [19] IconType OPTIONAL, -- Tag '93'
+    icon [20] OCTET STRING (SIZE(0..1024)) OPTIONAL, -- Tag '94', see condition in ES10c:GetProfilesInfo
+    profileClass [21] ProfileClass DEFAULT operational, -- Tag '95'
+    notificationConfigurationInfo [22] SEQUENCE OF NotificationConfigurationInformation OPTIONAL, -- Tag 'B6'
+    profileOwner [23] OperatorID OPTIONAL, -- Tag 'B7'
+    dpProprietaryData [24] DpProprietaryData OPTIONAL, -- Tag 'B8'
+    profilePolicyRules [25] PprIds OPTIONAL -- Tag '99'
+}
+ 
+PprIds ::= BIT STRING {-- Definition of Profile Policy Rules identifiers
+    pprUpdateControl(0), -- defines how to update PPRs via ES6
+    ppr1(1), -- Indicator for PPR1 'Disabling of this Profile is not allowed'
+    ppr2(2), -- Indicator for PPR2 'Deletion of this Profile is not allowed'
+    ppr3(3)  -- Indicator for PPR3 'Deletion of this Profile is required upon its successful disabling'
+}
+ 
+OperatorID ::= SEQUENCE {
+    mccMnc OCTET STRING (SIZE(3)), -- MCC and MNC coded as defined in 3GPP TS 24.008 [32]
+    gid1 OCTET STRING OPTIONAL, -- referring to content of EF GID1 (file identifier '6F3E') as defined in 3GPP TS 31.102 [54]
+    gid2 OCTET STRING OPTIONAL -- referring to content of EF GID2 (file identifier '6F3F') as defined in 3GPP TS 31.102 [54]
+}
+ 
+ProfileInfoListError ::= INTEGER {incorrectInputValues(1), undefinedError(127)}
+ 
+-- Definition of StoreMetadata request
+ 
+StoreMetadataRequest ::= [37] SEQUENCE { -- Tag 'BF25'
+    iccid Iccid,
+    serviceProviderName [17] UTF8String (SIZE(0..32)), -- Tag '91'
+    profileName [18] UTF8String (SIZE(0..64)), -- Tag '92' (corresponds to 'Short Description' defined in SGP.21 [2])
+    iconType [19] IconType OPTIONAL, -- Tag '93' (JPG or PNG)
+    icon [20] OCTET STRING (SIZE(0..1024)) OPTIONAL, -- Tag '94'(Data of the icon. Size 64 x 64 pixel. This field SHALL only be present if iconType is present)
+    profileClass [21] ProfileClass OPTIONAL, -- Tag '95' (default if absent: 'operational')
+    notificationConfigurationInfo [22] SEQUENCE OF NotificationConfigurationInformation OPTIONAL,
+    profileOwner [23] OperatorID OPTIONAL, -- Tag 'B7'
+    profilePolicyRules [25] PprIds OPTIONAL -- Tag '99'
+}
+ 
+NotificationEvent ::= BIT STRING {
+    notificationInstall (0),
+    notificationEnable(1),
+    notificationDisable(2),
+    notificationDelete(3)
+}
+ 
+NotificationConfigurationInformation ::= SEQUENCE {
+    profileManagementOperation NotificationEvent,
+    notificationAddress UTF8String -- FQDN to forward the notification
+}
+ 
+IconType ::= INTEGER {jpg(0), png(1)}
+ProfileState ::= INTEGER {disabled(0), enabled(1)}
+ProfileClass ::= INTEGER {test(0), provisioning(1), operational(2)}
+ 
+-- Definition of UpdateMetadata request
+UpdateMetadataRequest ::= [42] SEQUENCE {  -- Tag 'BF2A'
+    serviceProviderName [17] UTF8String (SIZE(0..32)) OPTIONAL, -- Tag '91'
+    profileName [18] UTF8String (SIZE(0..64)) OPTIONAL, -- Tag '92'
+    iconType [19] IconType OPTIONAL, -- Tag '93'
+    icon [20] OCTET STRING (SIZE(0..1024)) OPTIONAL, -- Tag '94'
+    profilePolicyRules [25] PprIds OPTIONAL -- Tag '99'
+}
+ 
+-- Definition of data objects for command PrepareDownload -------------------------
+PrepareDownloadRequest ::= [33] SEQUENCE {  -- Tag 'BF21'
+    smdpSigned2 SmdpSigned2,            -- Signed information
+    smdpSignature2 [APPLICATION 55] OCTET STRING,   -- DP_Sign1, tag '5F37'
+    hashCc Octet32 OPTIONAL, -- Hash of confirmation code
+    smdpCertificate Certificate -- CERT.DPpb.ECDSA
+}
+ 
+SmdpSigned2 ::= SEQUENCE {
+    transactionId [0] TransactionId,    -- The TransactionID generated by the SM DP+
+    ccRequiredFlag BOOLEAN, --Indicates if the Confirmation Code is required
+    bppEuiccOtpk [APPLICATION 73] OCTET STRING OPTIONAL     -- otPK.EUICC.ECKA already used for binding the BPP, tag '5F49'
+}
+ 
+PrepareDownloadResponse ::= [33] CHOICE {  -- Tag 'BF21'
+    downloadResponseOk PrepareDownloadResponseOk,
+    downloadResponseError PrepareDownloadResponseError
+}
+ 
+PrepareDownloadResponseOk ::= SEQUENCE {
+    euiccSigned2 EUICCSigned2,  -- Signed information
+    euiccSignature2 [APPLICATION 55] OCTET STRING   -- tag '5F37'
+}
+ 
+EUICCSigned2 ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    euiccOtpk [APPLICATION 73] OCTET STRING,        -- otPK.EUICC.ECKA, tag '5F49'
+    hashCc Octet32 OPTIONAL         -- Hash of confirmation code
+}
+ 
+PrepareDownloadResponseError ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    downloadErrorCode DownloadErrorCode
+}
+ 
+DownloadErrorCode ::= INTEGER {invalidCertificate(1), invalidSignature(2), unsupportedCurve(3), noSessionContext(4), invalidTransactionId(5), undefinedError(127)}
+ 
+-- Definition of data objects for command AuthenticateServer--------------------
+AuthenticateServerRequest ::= [56] SEQUENCE { -- Tag 'BF38'
+    serverSigned1 ServerSigned1,                -- Signed information
+    serverSignature1 [APPLICATION 55] OCTET STRING,     -- tag ?5F37?
+    euiccCiPKIdToBeUsed SubjectKeyIdentifier,       -- CI Public Key Identifier to be used
+    serverCertificate Certificate, -- RSP Server Certificate CERT.XXauth.ECDSA
+    ctxParams1 CtxParams1
+}
+ 
+ServerSigned1 ::= SEQUENCE {
+    transactionId [0] TransactionId,        -- The Transaction ID generated by the RSP Server
+    euiccChallenge [1] Octet16,     -- The eUICC Challenge
+    serverAddress [3] UTF8String,   -- The RSP Server address
+    serverChallenge [4] Octet16     -- The RSP Server Challenge
+}
+ 
+CtxParams1 ::= CHOICE {
+    ctxParamsForCommonAuthentication CtxParamsForCommonAuthentication -- New contextual data objects may be defined for extensibility
+}
+ 
+CtxParamsForCommonAuthentication ::= SEQUENCE {
+    matchingId UTF8String OPTIONAL,-- The MatchingId could be the Activation code token or EventID or empty
+    deviceInfo DeviceInfo -- The Device information
+}
+ 
+AuthenticateServerResponse ::= [56] CHOICE { -- Tag 'BF38'
+    authenticateResponseOk AuthenticateResponseOk,
+    authenticateResponseError AuthenticateResponseError
+}
+ 
+AuthenticateResponseOk ::= SEQUENCE {
+    euiccSigned1 EuiccSigned1,      -- Signed information
+    euiccSignature1 [APPLICATION 55] OCTET STRING,  --EUICC_Sign1, tag 5F37
+    euiccCertificate Certificate,   -- eUICC Certificate (CERT.EUICC.ECDSA) signed by the EUM
+    eumCertificate Certificate  -- EUM Certificate (CERT.EUM.ECDSA) signed by the requested CI
+}
+ 
+EuiccSigned1 ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    serverAddress [3] UTF8String,
+    serverChallenge [4] Octet16,    -- The RSP Server Challenge
+    euiccInfo2 [34] EUICCInfo2,
+    ctxParams1 CtxParams1
+}
+ 
+AuthenticateResponseError ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    authenticateErrorCode AuthenticateErrorCode
+}
+ 
+AuthenticateErrorCode ::= INTEGER {invalidCertificate(1), invalidSignature(2), unsupportedCurve(3), noSessionContext(4), invalidOid(5), euiccChallengeMismatch(6), ciPKUnknown(7), undefinedError(127)}
+ 
+-- Definition of Cancel Session------------------------------
+CancelSessionRequest ::= [65] SEQUENCE { -- Tag 'BF41'
+    transactionId TransactionId,     -- The TransactionID generated by the RSP Server
+    reason CancelSessionReason
+}
+ 
+CancelSessionReason ::= INTEGER {endUserRejection(0), postponed(1), timeout(2), pprNotAllowed(3)}
+ 
+CancelSessionResponse ::= [65] CHOICE { -- Tag 'BF41'
+    cancelSessionResponseOk CancelSessionResponseOk,
+    cancelSessionResponseError INTEGER {invalidTransactionId(5), undefinedError(127)}
+}
+ 
+CancelSessionResponseOk ::= SEQUENCE {
+    euiccCancelSessionSigned EuiccCancelSessionSigned,  -- Signed information
+    euiccCancelSessionSignature [APPLICATION 55] OCTET STRING   -- tag '5F37
+}
+ 
+EuiccCancelSessionSigned ::= SEQUENCE {
+    transactionId TransactionId,
+    smdpOid OBJECT IDENTIFIER, -- SM-DP+ OID as contained in CERT.DPauth.ECDSA
+    reason CancelSessionReason
+}
+ 
+-- Definition of Bound Profile Package --------------------------
+BoundProfilePackage ::= [54] SEQUENCE { -- Tag 'BF36'
+    initialiseSecureChannelRequest [35] InitialiseSecureChannelRequest, -- Tag 'BF23'
+    firstSequenceOf87 [0] SEQUENCE OF [7] OCTET STRING, -- sequence of '87' TLVs
+    sequenceOf88 [1] SEQUENCE OF [8] OCTET STRING, -- sequence of '88' TLVs
+    secondSequenceOf87 [2] SEQUENCE OF [7] OCTET STRING OPTIONAL, -- sequence of '87' TLVs
+    sequenceOf86 [3] SEQUENCE OF [6] OCTET STRING -- sequence of '86' TLVs
+}
+ 
+-- Definition of Get eUICC Challenge --------------------------
+GetEuiccChallengeRequest ::= [46] SEQUENCE { -- Tag 'BF2E'
+}
+ 
+GetEuiccChallengeResponse ::= [46] SEQUENCE { -- Tag 'BF2E'
+    euiccChallenge Octet16  -- random eUICC challenge
+}
+ 
+-- Definition of Profile Installation Resulceipt
+ProfileInstallationResult ::= [55] SEQUENCE { -- Tag 'BF37'
+    profileInstallationResultData [39] ProfileInstallationResultData,
+    euiccSignPIR EuiccSignPIR
+}
+ 
+ProfileInstallationResultData ::= [39] SEQUENCE { -- Tag 'BF27'
+    transactionId[0] TransactionId, -- The TransactionID generated by the SM-DP+
+    notificationMetadata[47] NotificationMetadata,
+    smdpOid OBJECT IDENTIFIER OPTIONAL, -- SM-DP+ OID (same value as in CERT.DPpb.ECDSA)
+    finalResult [2] CHOICE {
+        successResult SuccessResult,
+        errorResult ErrorResult
+    }
+}
+ 
+EuiccSignPIR ::= [APPLICATION 55] OCTET STRING -- Tag '5F37', eUICC?s signature
+ 
+SuccessResult ::= SEQUENCE {
+    aid [APPLICATION 15] OCTET STRING (SIZE (5..16)), -- AID of ISD-P
+    simaResponse OCTET STRING -- contains (multiple) 'EUICCResponse' as defined in [5]
+}
+ 
+ErrorResult ::= SEQUENCE {
+    bppCommandId BppCommandId,
+    errorReason ErrorReason,
+    simaResponse OCTET STRING OPTIONAL -- contains (multiple) 'EUICCResponse' as defined in [5]
+}
+ 
+BppCommandId ::= INTEGER {initialiseSecureChannel(0), configureISDP(1), storeMetadata(2), storeMetadata2(3), replaceSessionKeys(4), loadProfileElements(5)}
+ 
+ErrorReason ::= INTEGER {
+    incorrectInputValues(1),
+    invalidSignature(2),
+    invalidTransactionId(3),
+    unsupportedCrtValues(4),
+    unsupportedRemoteOperationType(5),
+    unsupportedProfileClass(6),
+    scp03tStructureError(7),
+    scp03tSecurityError(8),
+    installFailedDueToIccidAlreadyExistsOnEuicc(9), installFailedDueToInsufficientMemoryForProfile(10),
+    installFailedDueToInterruption(11),
+    installFailedDueToPEProcessingError (12),
+    installFailedDueToIccidMismatch(13),
+    testProfileInstallFailedDueToInvalidNaaKey(14),
+    pprNotAllowed(15),
+    installFailedDueToUnknownError(127)
+}
+ 
+ListNotificationRequest ::= [40] SEQUENCE { -- Tag 'BF28'
+    profileManagementOperation [1] NotificationEvent OPTIONAL
+}
+ 
+ListNotificationResponse ::= [40] CHOICE { -- Tag 'BF28'
+    notificationMetadataList SEQUENCE OF NotificationMetadata,
+    listNotificationsResultError INTEGER {undefinedError(127)}
+}
+ 
+NotificationMetadata ::= [47] SEQUENCE { -- Tag 'BF2F'
+    seqNumber [0] INTEGER,
+    profileManagementOperation [1] NotificationEvent, --Only one bit set to 1
+    notificationAddress UTF8String, -- FQDN to forward the notification
+    iccid Iccid OPTIONAL
+}
+ 
+-- Definition of Profile Nickname Information
+SetNicknameRequest ::= [41] SEQUENCE {  -- Tag 'BF29'
+    iccid Iccid,
+    profileNickname [16] UTF8String (SIZE(0..64))
+}
+ 
+SetNicknameResponse ::= [41] SEQUENCE { -- Tag 'BF29'
+    setNicknameResult INTEGER {ok(0), iccidNotFound (1), undefinedError(127)}
+}
+ 
+id-rsp-cert-objects OBJECT IDENTIFIER ::= {  id-rsp cert-objects(2)}
+ 
+id-rspExt OBJECT IDENTIFIER ::= {id-rsp-cert-objects 0}
+ 
+id-rspRole OBJECT IDENTIFIER ::= {id-rsp-cert-objects 1}
+ 
+-- Definition of OIDs for role identification
+id-rspRole-ci OBJECT IDENTIFIER ::= {id-rspRole 0}
+id-rspRole-euicc OBJECT IDENTIFIER ::= {id-rspRole 1}
+id-rspRole-eum OBJECT IDENTIFIER ::= {id-rspRole 2}
+id-rspRole-dp-tls OBJECT IDENTIFIER ::= {id-rspRole 3}
+id-rspRole-dp-auth OBJECT IDENTIFIER ::= {id-rspRole 4}
+id-rspRole-dp-pb OBJECT IDENTIFIER ::= {id-rspRole 5}
+id-rspRole-ds-tls OBJECT IDENTIFIER ::= {id-rspRole 6}
+id-rspRole-ds-auth OBJECT IDENTIFIER ::= {id-rspRole 7}
+ 
+--Definition of data objects for InitialiseSecureChannel Request
+InitialiseSecureChannelRequest ::= [35] SEQUENCE { -- Tag 'BF23'
+    remoteOpId RemoteOpId, -- Remote Operation Type Identifier (value SHALL be set to installBoundProfilePackage)
+    transactionId [0] TransactionId, -- The TransactionID generated by the SM-DP+
+    controlRefTemplate[6] IMPLICIT ControlRefTemplate, -- Control Reference Template (Key Agreement). Current specification considers a subset of CRT specified in GlobalPlatform Card Specification [8], section 6.4.2.3 for the Mutual Authentication Data Field
+    smdpOtpk [APPLICATION 73] OCTET STRING, ---otPK.DP.ECKA as specified in GlobalPlatform Card Specification [8] section 6.4.2.3 for ePK.OCE.ECKA, tag '5F49'
+    smdpSign [APPLICATION 55] OCTET STRING -- SM-DP's signature, tag '5F37'
+}
+ 
+ControlRefTemplate ::= SEQUENCE {
+keyType[0] Octet1, -- Key type according to GlobalPlatform Card Specification [8] Table 11-16, AES= '88', Tag '80'
+keyLen[1] Octet1, --Key length in number of bytes. For current specification key length SHALL by 0x10 bytes, Tag '81'
+hostId[4] OctetTo16 -- Host ID value , Tag '84'
+}
+ 
+--Definition of data objects for ConfigureISDPRequest
+ConfigureISDPRequest ::= [36] SEQUENCE { -- Tag 'BF24'
+    dpProprietaryData [24] DpProprietaryData OPTIONAL -- Tag 'B8'
+}
+ 
+DpProprietaryData ::= SEQUENCE { --  maximum size including tag and length field: 128 bytes
+    dpOid OBJECT IDENTIFIER -- OID in the tree of the SM-DP+ that created the Profile
+    -- additional data objects defined by the SM-DP+ MAY follow
+}
+ 
+-- Definition of request message for command ReplaceSessionKeys
+ReplaceSessionKeysRequest ::= [38] SEQUENCE { -- tag 'BF26'
+/*The new initial MAC chaining value*/
+    initialMacChainingValue OCTET STRING,
+/*New session key value for encryption/decryption (PPK-ENC)*/
+    ppkEnc OCTET STRING,
+/*New session key value of the session key C-MAC computation/verification (PPK-MAC)*/
+    ppkCmac OCTET STRING
+}
+ 
+-- Definition of data objects for RetrieveNotificationsList
+RetrieveNotificationsListRequest ::= [43] SEQUENCE { -- Tag 'BF2B'
+    searchCriteria CHOICE {
+        seqNumber [0] INTEGER,
+        profileManagementOperation [1] NotificationEvent
+    } OPTIONAL
+}
+ 
+RetrieveNotificationsListResponse ::= [43] CHOICE { -- Tag 'BF2B'
+    notificationList SEQUENCE OF PendingNotification,
+    notificationsListResultError INTEGER {noResultAvailable(1), undefinedError(127)}
+}
+ 
+PendingNotification ::= CHOICE {
+    profileInstallationResult [55] ProfileInstallationResult, -- tag 'BF37'
+    otherSignedNotification OtherSignedNotification
+}
+ 
+OtherSignedNotification ::= SEQUENCE {
+    tbsOtherNotification NotificationMetadata,
+    euiccNotificationSignature [APPLICATION 55] OCTET STRING,   -- eUICC signature of tbsOtherNotification, Tag '5F37'
+    euiccCertificate Certificate,   -- eUICC Certificate (CERT.EUICC.ECDSA) signed by the EUM
+    eumCertificate Certificate  -- EUM Certificate (CERT.EUM.ECDSA) signed by the requested CI
+}
+ 
+-- Definition of notificationSent
+NotificationSentRequest ::= [48] SEQUENCE { -- Tag 'BF30'
+    seqNumber [0] INTEGER
+}
+ 
+NotificationSentResponse ::= [48] SEQUENCE { -- Tag 'BF30'
+    deleteNotificationStatus INTEGER {ok(0), nothingToDelete(1), undefinedError(127)}
+}
+ 
+-- Definition of Enable Profile --------------------------
+EnableProfileRequest ::= [49] SEQUENCE { -- Tag 'BF31'
+    profileIdentifier CHOICE {
+        isdpAid [APPLICATION 15] OctetTo16, -- AID, tag '4F'
+        iccid Iccid -- ICCID, tag '5A'
+    },
+    refreshFlag BOOLEAN -- indicating whether REFRESH is required
+}
+ 
+EnableProfileResponse ::= [49] SEQUENCE { -- Tag 'BF31'
+    enableResult INTEGER {ok(0), iccidOrAidNotFound (1), profileNotInDisabledState(2), disallowedByPolicy(3), wrongProfileReenabling(4), undefinedError(127)}
+}
+ 
+-- Definition of Disable Profile --------------------------
+DisableProfileRequest ::= [50] SEQUENCE { -- Tag 'BF32'
+    profileIdentifier CHOICE {
+        isdpAid [APPLICATION 15] OctetTo16, -- AID, tag '4F'
+        iccid Iccid -- ICCID, tag '5A'
+    },
+    refreshFlag BOOLEAN -- indicating whether REFRESH is required
+}
+ 
+DisableProfileResponse ::= [50] SEQUENCE { -- Tag 'BF32'
+    disableResult INTEGER {ok(0), iccidOrAidNotFound (1), profileNotInEnabledState(2), disallowedByPolicy(3), undefinedError(127)}
+}
+ 
+-- Definition of Delete Profile --------------------------
+DeleteProfileRequest ::= [51] CHOICE { -- Tag 'BF33'
+    isdpAid [APPLICATION 15] OctetTo16, -- AID, tag '4F'
+    iccid Iccid -- ICCID, tag '5A'
+}
+ 
+DeleteProfileResponse ::= [51] SEQUENCE { -- Tag 'BF33'
+    deleteResult INTEGER {ok(0), iccidOrAidNotFound (1), profileNotInDisabledState(2), disallowedByPolicy(3), undefinedError(127)}
+}
+ 
+-- Definition of Memory Reset --------------------------
+EuiccMemoryResetRequest ::= [52] SEQUENCE { -- Tag 'BF34'
+    resetOptions [2] BIT STRING {
+        deleteOperationalProfiles(0),
+        deleteFieldLoadedTestProfiles(1),
+        resetDefaultSmdpAddress(2)}
+}
+ 
+EuiccMemoryResetResponse ::= [52] SEQUENCE { -- Tag 'BF34'
+    resetResult INTEGER {ok(0), nothingToDelete(1), undefinedError(127)}
+}
+ 
+-- Definition of Get EID --------------------------
+GetEuiccDataRequest ::= [62] SEQUENCE { -- Tag 'BF3E'
+    tagList [APPLICATION 28] Octet1  -- tag '5C', the value SHALL be set to '5A'
+}
+ 
+GetEuiccDataResponse ::= [62] SEQUENCE { -- Tag 'BF3E'
+    eidValue [APPLICATION 26] Octet16  -- tag '5A'
+}
+ 
+-- Definition of Get Rat
+ 
+GetRatRequest ::= [67] SEQUENCE { -- Tag ' BF43'
+    -- No input data
+}
+ 
+ 
+GetRatResponse ::= [67] SEQUENCE { -- Tag 'BF43'
+    rat RulesAuthorisationTable
+}
+ 
+RulesAuthorisationTable ::= SEQUENCE OF ProfilePolicyAuthorisationRule
+ProfilePolicyAuthorisationRule ::= SEQUENCE {
+    pprIds PprIds,
+    allowedOperators SEQUENCE OF OperatorID,
+    pprFlags BIT STRING {consentRequired(0)}
+}
+ 
+-- Definition of data structure command for loading a CRL
+LoadCRLRequest ::= [53] SEQUENCE { -- Tag 'BF35'
+    -- A CRL-A
+    crl CertificateList
+}
+ 
+-- Definition of data structure response for loading a CRL
+LoadCRLResponse ::= [53] CHOICE {  -- Tag 'BF35'
+loadCRLResponseOk LoadCRLResponseOk,
+loadCRLResponseError LoadCRLResponseError
+}
+ 
+LoadCRLResponseOk ::= SEQUENCE {
+    missingParts SEQUENCE OF SEQUENCE {
+        number INTEGER (0..MAX)
+    } OPTIONAL
+}
+LoadCRLResponseError ::= INTEGER {invalidSignature(1), invalidCRLFormat(2), notEnoughMemorySpace(3), verificationKeyNotFound(4), undefinedError(127)}
+ 
+-- Definition of the extension for Certificate Expiration Date
+id-rsp-expDate OBJECT IDENTIFIER ::= {id-rspExt 1}
+ExpirationDate ::= Time
+ 
+-- Definition of the extension id for total partial-CRL number
+id-rsp-totalPartialCrlNumber OBJECT IDENTIFIER ::= {id-rspExt 2}
+TotalPartialCrlNumber ::= INTEGER
+ 
+ 
+-- Definition of the extension id for the partial-CRL number
+id-rsp-partialCrlNumber OBJECT IDENTIFIER ::= {id-rspExt 3}
+PartialCrlNumber ::= INTEGER
+ 
+-- Definition for ES9+ ASN.1 Binding --------------------------
+RemoteProfileProvisioningRequest ::= [2] CHOICE { -- Tag 'A2'
+    initiateAuthenticationRequest [57] InitiateAuthenticationRequest,  -- Tag 'BF39'
+    authenticateClientRequest [59] AuthenticateClientRequest, -- Tag 'BF3B'
+    getBoundProfilePackageRequest [58] GetBoundProfilePackageRequest,  -- Tag 'BF3A'
+    cancelSessionRequestEs9 [65] CancelSessionRequestEs9, -- Tag 'BF41'
+    handleNotification [61] HandleNotification -- tag 'BF3D'
+}
+ 
+RemoteProfileProvisioningResponse ::= [2] CHOICE { -- Tag 'A2'
+    initiateAuthenticationResponse [57] InitiateAuthenticationResponse, -- Tag 'BF39'
+    authenticateClientResponseEs9 [59] AuthenticateClientResponseEs9, -- Tag 'BF3B'
+    getBoundProfilePackageResponse [58] GetBoundProfilePackageResponse, -- Tag 'BF3A'
+    cancelSessionResponseEs9 [65] CancelSessionResponseEs9, -- Tag 'BF41'
+    authenticateClientResponseEs11 [64] AuthenticateClientResponseEs11 -- Tag 'BF40'
+}
+ 
+InitiateAuthenticationRequest ::= [57] SEQUENCE { -- Tag 'BF39'
+    euiccChallenge [1] Octet16, -- random eUICC challenge
+    smdpAddress [3] UTF8String,
+    euiccInfo1 EUICCInfo1
+}
+ 
+InitiateAuthenticationResponse ::= [57] CHOICE { -- Tag 'BF39'
+    initiateAuthenticationOk InitiateAuthenticationOkEs9,
+    initiateAuthenticationError INTEGER {
+        invalidDpAddress(1),
+        euiccVersionNotSupportedByDp(2),
+        ciPKNotSupported(3)
+    }
+}
+ 
+InitiateAuthenticationOkEs9 ::= SEQUENCE {
+    transactionId [0] TransactionId, -- The TransactionID generated by the SM-DP+
+    serverSigned1 ServerSigned1, -- Signed information
+    serverSignature1 [APPLICATION 55] OCTET STRING, -- Server_Sign1, tag '5F37'
+    euiccCiPKIdToBeUsed SubjectKeyIdentifier, -- The curve CI Public Key to be used as required by ES10b.AuthenticateServer
+    serverCertificate Certificate
+}
+ 
+AuthenticateClientRequest ::= [59] SEQUENCE { -- Tag 'BF3B'
+    transactionId [0] TransactionId,
+    authenticateServerResponse [56] AuthenticateServerResponse -- This is the response from ES10b.AuthenticateServer
+}
+ 
+AuthenticateClientResponseEs9 ::= [59] CHOICE { -- Tag 'BF3B'
+    authenticateClientOk AuthenticateClientOk,
+    authenticateClientError INTEGER {
+        eumCertificateInvalid(1),
+        eumCertificateExpired(2),
+        euiccCertificateInvalid(3),
+        euiccCertificateExpired(4),
+        euiccSignatureInvalid(5),
+        matchingIdRefused(6),
+        eidMismatch(7),
+        noEligibleProfile(8),
+        ciPKUnknown(9),
+        invalidTransactionId(10),
+        undefinedError(127)
+    }
+}
+ 
+AuthenticateClientOk ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    profileMetaData [37] StoreMetadataRequest,
+    prepareDownloadRequest [33] PrepareDownloadRequest
+}
+ 
+GetBoundProfilePackageRequest ::= [58] SEQUENCE { -- Tag 'BF3A'
+    transactionId [0] TransactionId,
+    prepareDownloadResponse [33] PrepareDownloadResponse
+}
+ 
+GetBoundProfilePackageResponse ::= [58] CHOICE { -- Tag 'BF3A'
+    getBoundProfilePackageOk GetBoundProfilePackageOk,
+    getBoundProfilePackageError INTEGER {
+        euiccSignatureInvalid(1),
+        confirmationCodeMissing(2),
+        confirmationCodeRefused(3),
+        confirmationCodeRetriesExceeded(4),
+        invalidTransactionId(95),
+        undefinedError(127)
+    }
+}
+ 
+GetBoundProfilePackageOk ::= SEQUENCE {
+    transactionId [0] TransactionId,
+    boundProfilePackage [54] BoundProfilePackage
+}
+ 
+HandleNotification ::= [61] SEQUENCE {  -- Tag 'BF3D'
+    pendingNotification PendingNotification
+}
+ 
+CancelSessionRequestEs9 ::= [65] SEQUENCE { -- Tag 'BF41'
+    transactionId TransactionId,
+    cancelSessionResponse  CancelSessionResponse -- data structure defined for ES10b.CancelSession function
+}
+ 
+CancelSessionResponseEs9 ::= [65] CHOICE { -- Tag 'BF41'
+    cancelSessionOk CancelSessionOk,
+    cancelSessionError INTEGER {
+        invalidTransactionId(1),
+        euiccSignatureInvalid(2),
+        undefinedError(127)
+    }
+}
+ 
+CancelSessionOk ::= SEQUENCE { -- This function has no output data
+}
+ 
+EuiccConfiguredAddressesRequest ::= [60] SEQUENCE { -- Tag 'BF3C'
+}
+ 
+EuiccConfiguredAddressesResponse ::= [60] SEQUENCE {  -- Tag 'BF3C'
+    defaultDpAddress UTF8String OPTIONAL,  -- Default SM-DP+ address as an FQDN
+    rootDsAddress UTF8String  -- Root SM-DS address as an FQDN
+}
+ 
+ISDRProprietaryApplicationTemplate ::= [PRIVATE 0] SEQUENCE { -- Tag 'E0'  
+    svn [2] VersionType,    -- GSMA SGP.22 version supported (SVN)
+    lpaeSupport BIT STRING {
+        lpaeUsingCat(0), -- LPA in the eUICC using Card Application Toolkit
+        lpaeUsingScws(1) -- LPA in the eUICC using Smartcard Web Server
+    } OPTIONAL
+}
+ 
+LpaeActivationRequest ::= [66] SEQUENCE { -- Tag 'BF42'
+    lpaeOption BIT STRING {
+        activateCatBasedLpae(0), -- LPAe with LUIe based on CAT
+        activateScwsBasedLpae(1) -- LPAe with LUIe based on SCWS
+    }
+}
+ 
+LpaeActivationResponse ::= [66] SEQUENCE { -- Tag 'BF42'
+    lpaeActivationResult INTEGER {ok(0), notSupported(1)}
+}
+ 
+SetDefaultDpAddressRequest ::= [63] SEQUENCE { -- Tag 'BF3F'
+    defaultDpAddress UTF8String -- Default SM-DP+ address as an FQDN
+}
+ 
+SetDefaultDpAddressResponse ::= [63] SEQUENCE { -- Tag 'BF3F'
+    setDefaultDpAddressResult INTEGER { ok (0), undefinedError (127)}
+}
+ 
+AuthenticateClientResponseEs11 ::= [64] CHOICE {  -- Tag 'BF40'
+    authenticateClientOk AuthenticateClientOkEs11,
+    authenticateClientError INTEGER {
+        eumCertificateInvalid(1),
+        eumCertificateExpired(2),
+        euiccCertificateInvalid(3),
+        euiccCertificateExpired(4),
+        euiccSignatureInvalid(5),
+        eventIdUnknown(6),
+        invalidTransactionId(7),
+        undefinedError(127)
+    }
+}
+ 
+AuthenticateClientOkEs11 ::= SEQUENCE {
+    transactionId TransactionId,
+    eventEntries SEQUENCE OF EventEntries
+}
+ 
+EventEntries ::= SEQUENCE {
+    eventId UTF8String,
+    rspServerAddress UTF8String
+}
+ 
+END

pySim/esim/bsp.py

@@ -0,0 +1,294 @@
+# Early proof-of-concept implementation of
+# GSMA eSIM RSP (Remote SIM Provisioning BSP (BPP Protection Protocol),
+# where BPP is the Bound  Profile Package.  So the full expansion is the
+# "GSMA eSIM Remote SIM Provisioning Bound Profile Packate Protection Protocol"
+#
+# Originally (SGP.22 v2.x) this was called SCP03t, but it has since been
+# renamed to BSP.
+#
+# (C) 2023 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# SGP.22 v3.0 Section 2.5.3:
+# That block of data is split into segments of a maximum size of 1020 bytes (including the tag, length field and MAC).
+
+import abc
+from typing import List
+import logging
+
+# for BSP key derivation
+from cryptography.hazmat.primitives import hashes
+from cryptography.hazmat.primitives.kdf.x963kdf import X963KDF
+
+from Cryptodome.Cipher import AES
+from Cryptodome.Hash import CMAC
+
+from pySim.utils import bertlv_encode_len, bertlv_parse_one, b2h
+
+# don't log by default
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.NullHandler())
+
+MAX_SEGMENT_SIZE = 1020
+
+class BspAlgo(abc.ABC):
+    blocksize: int
+
+    def _get_padding(self, in_len: int, multiple: int, padding: int = 0) -> bytes:
+        """Return padding bytes towards multiple of N."""
+        if in_len % multiple == 0:
+            return b''
+        pad_cnt = multiple - (in_len % multiple)
+        return bytes([padding]) * pad_cnt
+
+    def _pad_to_multiple(self, indat: bytes, multiple: int, padding: int = 0) -> bytes:
+        """Pad the input data to multiples of 'multiple'."""
+        return indat + self._get_padding(len(indat), multiple, padding)
+
+    def __str__(self):
+        return self.__class__.__name__
+
+class BspAlgoCrypt(BspAlgo, abc.ABC):
+
+    def __init__(self, s_enc: bytes):
+        self.s_enc = s_enc
+        self.block_nr = 1
+
+    def encrypt(self, data:bytes) -> bytes:
+        """Encrypt given input bytes using the key material given in constructor."""
+        padded_data = self._pad_to_multiple(data, self.blocksize)
+        block_nr = self.block_nr
+        ciphertext = self._encrypt(padded_data)
+        logger.debug("encrypt(block_nr=%u, s_enc=%s, plaintext=%s, padded=%s) -> %s",
+                     block_nr, b2h(self.s_enc), b2h(data), b2h(padded_data), b2h(ciphertext))
+        return ciphertext
+
+    def decrypt(self, data:bytes) -> bytes:
+        """Decrypt given input bytes using the key material given in constructor."""
+        return self._unpad(self._decrypt(data))
+
+    @abc.abstractmethod
+    def _unpad(self, padded: bytes) -> bytes:
+        """Remove the padding from padded data."""
+
+    @abc.abstractmethod
+    def _encrypt(self, data:bytes) -> bytes:
+        """Actual implementation, to be implemented by derived class."""
+
+    @abc.abstractmethod
+    def _decrypt(self, data:bytes) -> bytes:
+        """Actual implementation, to be implemented by derived class."""
+
+class BspAlgoCryptAES128(BspAlgoCrypt):
+    name = 'AES-CBC-128'
+    blocksize = 16
+
+    def _get_padding(self, in_len: int, multiple: int, padding: int = 0):
+        # SGP.22 section 2.6.4.4
+        # Append a byte with value '80' to the right of the data block;
+        # Append 0 to 15 bytes with value '00' so that the length of the padded data block
+        # is a multiple of 16 bytes.
+        return b'\x80' + super()._get_padding(in_len + 1, multiple, padding)
+
+    def _unpad(self, padded: bytes) -> bytes:
+        """Remove the customary 80 00 00 ... padding used for AES."""
+        # first remove any trailing zero bytes
+        stripped = padded.rstrip(b'\0')
+        # then remove the final 80
+        assert stripped[-1] == 0x80
+        return stripped[:-1]
+
+    def _get_icv(self):
+        # The binary value of this number SHALL be left padded with zeroes to form a full block.
+        data = self.block_nr.to_bytes(self.blocksize, "big")
+        #iv = bytes([0] * (self.blocksize-1)) + b'\x01'
+        iv = bytes([0] * self.blocksize)
+        # This block SHALL be encrypted with S-ENC to produce the ICV for command encryption.
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, iv)
+        icv = cipher.encrypt(data)
+        logger.debug("_get_icv(block_nr=%u, data=%s) -> icv=%s", self.block_nr, b2h(data), b2h(icv))
+        self.block_nr = self.block_nr + 1
+        return icv
+
+    def _encrypt(self, data: bytes) -> bytes:
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, self._get_icv())
+        return cipher.encrypt(data)
+
+    def _decrypt(self, data: bytes) -> bytes:
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, self._get_icv())
+        return cipher.decrypt(data)
+
+
+class BspAlgoMac(BspAlgo, abc.ABC):
+    l_mac = 0 # must be overridden by derived class
+
+    def __init__(self, s_mac: bytes, initial_mac_chaining_value: bytes):
+        self.s_mac = s_mac
+        self.mac_chain = initial_mac_chaining_value
+
+    def auth(self, tag: int, data: bytes) -> bytes:
+        assert tag in range (256)
+        # The input data used for C-MAC computation comprises the MAC Chaining value, the tag, the final length and the result of step 2
+        lcc = len(data) + self.l_mac
+        tag_and_length = bytes([tag]) + bertlv_encode_len(lcc)
+        temp_data = self.mac_chain + tag_and_length + data
+        old_mcv = self.mac_chain
+        c_mac = self._auth(temp_data)
+        # The output data is computed by concatenating the following data: the tag, the final length, the result of step 2 and the C-MAC value.
+        ret = tag_and_length + data + c_mac
+        logger.debug("auth(tag=0x%x, mcv=%s, s_mac=%s, plaintext=%s, temp=%s) -> %s",
+                     tag, b2h(old_mcv), b2h(self.s_mac), b2h(data), b2h(temp_data), b2h(ret))
+        return ret
+
+    def verify(self, ciphertext: bytes) -> bool:
+        mac_stripped = ciphertext[0:-self.l_mac]
+        mac_received = ciphertext[-self.l_mac:]
+        temp_data = self.mac_chain + mac_stripped
+        mac_computed = self._auth(temp_data)
+        if mac_received != mac_computed:
+            raise ValueError("MAC value not matching: received: %s, computed: %s" % (mac_received, mac_computed))
+        return mac_stripped
+
+    @abc.abstractmethod
+    def _auth(self, temp_data: bytes) -> bytes:
+        """To be implemented by algorithm specific derived class."""
+
+class BspAlgoMacAES128(BspAlgoMac):
+    name = 'AES-CMAC-128'
+    l_mac = 8
+
+    def _auth(self, temp_data: bytes) -> bytes:
+        # The full MAC value is computed using the MACing algorithm as defined in table 4c.
+        cmac = CMAC.new(self.s_mac, ciphermod=AES)
+        cmac.update(temp_data)
+        full_c_mac = cmac.digest()
+        # Subsequent MAC chaining values are the full result of step 4 of the previous data block
+        self.mac_chain = full_c_mac
+        # If the algorithm is AES-CBC-128 or SM4-CBC, the C-MAC value is the 8 most significant bytes of the result of step 4
+        return full_c_mac[0:8]
+
+
+
+def bsp_key_derivation(shared_secret: bytes, key_type: int, key_length: int, host_id: bytes, eid, l : int = 16):
+    """BSP protocol key derivation as per SGP.22 v3.0 Section 2.6.4.2"""
+    assert key_type <= 255
+    assert key_length <= 255
+
+    host_id_lv = bertlv_encode_len(len(host_id)) + host_id
+    eid_lv = bertlv_encode_len(len(eid)) + eid
+    shared_info = bytes([key_type, key_length]) + host_id_lv + eid_lv
+    logger.debug("kdf_shared_info: %s", b2h(shared_info))
+
+    # X9.63 Key Derivation Function with SHA256
+    xkdf = X963KDF(algorithm=hashes.SHA256(), length=l*3, sharedinfo=shared_info)
+    out = xkdf.derive(shared_secret)
+    logger.debug("kdf_out: %s", b2h(out))
+
+    initial_mac_chaining_value = out[0:l]
+    s_enc = out[l:2*l]
+    s_mac = out[l*2:3*l]
+
+    return s_enc, s_mac, initial_mac_chaining_value
+
+
+
+class BspInstance:
+    """An instance of the BSP crypto.  Initialized once with the key material via constructor,
+    then the user can call any number of encrypt_and_mac cycles to protect plaintext and
+    generate the respective ciphertext."""
+    def __init__(self, s_enc: bytes, s_mac: bytes, initial_mcv: bytes):
+        logger.debug("%s(s_enc=%s, s_mac=%s, initial_mcv=%s)", self.__class__.__name__, b2h(s_enc), b2h(s_mac), b2h(initial_mcv))
+        self.c_algo = BspAlgoCryptAES128(s_enc)
+        self.m_algo = BspAlgoMacAES128(s_mac, initial_mcv)
+
+        TAG_LEN = 1
+        length_len = len(bertlv_encode_len(MAX_SEGMENT_SIZE))
+        self.max_payload_size = MAX_SEGMENT_SIZE - TAG_LEN - length_len - self.m_algo.l_mac
+
+    @classmethod
+    def from_kdf(cls, shared_secret: bytes, key_type: int, key_length: int, host_id: bytes, eid: bytes):
+        """Convenience constructor for constructing an instance with keys from KDF."""
+        s_enc, s_mac, initial_mcv = bsp_key_derivation(shared_secret, key_type, key_length, host_id, eid)
+        return cls(s_enc, s_mac, initial_mcv)
+
+    def encrypt_and_mac_one(self, tag: int, plaintext:bytes) -> bytes:
+        """Encrypt + MAC a single plaintext TLV. Returns the protected ciphertex."""
+        assert tag <= 255
+        assert len(plaintext) <= self.max_payload_size
+        logger.debug("encrypt_and_mac_one(tag=0x%x, plaintext=%s)", tag, b2h(plaintext))
+        ciphered = self.c_algo.encrypt(plaintext)
+        maced = self.m_algo.auth(tag, ciphered)
+        return maced
+
+    def encrypt_and_mac(self, tag: int, plaintext:bytes) -> List[bytes]:
+        remainder = plaintext
+        result = []
+        while len(remainder):
+            remaining_len = len(remainder)
+            if remaining_len < self.max_payload_size:
+                segment_len = remaining_len
+                segment = remainder
+                remainder = b''
+            else:
+                segment_len = self.max_payload_size
+                segment = remainder[0:segment_len]
+                remainder = remainder[segment_len:]
+            result.append(self.encrypt_and_mac_one(tag, segment))
+        return result
+
+    def mac_only_one(self, tag: int, plaintext: bytes) -> bytes:
+        """MAC a single plaintext TLV. Returns the protected ciphertex."""
+        assert tag <= 255
+        assert len(plaintext) < self.max_payload_size
+        maced = self.m_algo.auth(tag, plaintext)
+        # The data block counter for ICV caluclation is incremented also for each segment with C-MAC only.
+        self.c_algo.block_nr += 1
+        return maced
+
+    def mac_only(self, tag: int, plaintext:bytes) -> List[bytes]:
+        remainder = plaintext
+        result = []
+        while len(remainder):
+            remaining_len = len(remainder)
+            if remaining_len < self.max_payload_size:
+                segment_len = remaining_len
+                segment = remainder
+                remainder = b''
+            else:
+                segment_len = self.max_payload_size
+                segment = remainder[0:segment_len]
+                remainder = remainder[segment_len:]
+            result.append(self.mac_only_one(tag, segment))
+        return result
+
+    def demac_and_decrypt_one(self, ciphertext: bytes) -> bytes:
+        payload = self.m_algo.verify(ciphertext)
+        tdict, l, val, remain = bertlv_parse_one(payload)
+        logger.debug("tag=%s, l=%u, val=%s, remain=%s", tdict, l, b2h(val), b2h(remain))
+        plaintext = self.c_algo.decrypt(val)
+        return plaintext
+
+    def demac_and_decrypt(self, ciphertext_list: List[bytes]) -> bytes:
+        plaintext_list = [self.demac_and_decrypt_one(x) for x in ciphertext_list]
+        return b''.join(plaintext_list)
+
+    def demac_only_one(self, ciphertext: bytes) -> bytes:
+        payload = self.m_algo.verify(ciphertext)
+        _tdict, _l, val, _remain = bertlv_parse_one(payload)
+        return val
+
+    def demac_only(self, ciphertext_list: List[bytes]) -> bytes:
+        plaintext_list = [self.demac_only_one(x) for x in ciphertext_list]
+        return b''.join(plaintext_list)

pySim/esim/es2p.py

@@ -0,0 +1,458 @@
+"""GSMA eSIM RSP ES2+ interface according to SGP.22 v2.5"""
+
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+import requests
+import logging
+import json
+from datetime import datetime
+import time
+import base64
+
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)
+
+class ApiParam(abc.ABC):
+    """A class reprsenting a single parameter in the ES2+ API."""
+    @classmethod
+    def verify_decoded(cls, data):
+        """Verify the decoded reprsentation of a value. Should raise an exception if somthing is odd."""
+        pass
+
+    @classmethod
+    def verify_encoded(cls, data):
+        """Verify the encoded reprsentation of a value. Should raise an exception if somthing is odd."""
+        pass
+
+    @classmethod
+    def encode(cls, data):
+        """[Validate and] Encode the given value."""
+        cls.verify_decoded(data)
+        encoded = cls._encode(data)
+        cls.verify_decoded(encoded)
+        return encoded
+
+    @classmethod
+    def _encode(cls, data):
+        """encoder function, typically [but not always] overridden by derived class."""
+        return data
+
+    @classmethod
+    def decode(cls, data):
+        """[Validate and] Decode the given value."""
+        cls.verify_encoded(data)
+        decoded = cls._decode(data)
+        cls.verify_decoded(decoded)
+        return decoded
+
+    @classmethod
+    def _decode(cls, data):
+        """decoder function, typically [but not always] overridden by derived class."""
+        return data
+
+class ApiParamString(ApiParam):
+    """Base class representing an API parameter of 'string' type."""
+    pass
+
+
+class ApiParamInteger(ApiParam):
+    """Base class representing an API parameter of 'integer' type."""
+    @classmethod
+    def _decode(cls, data):
+        return int(data)
+
+    @classmethod
+    def _encode(cls, data):
+        return str(data)
+
+    @classmethod
+    def verify_decoded(cls, data):
+        if not isinstance(data, int):
+            raise TypeError('Expected an integer input data type')
+
+    @classmethod
+    def verify_encoded(cls, data):
+        if not data.isdecimal():
+            raise ValueError('integer (%s) contains non-decimal characters' % data)
+        assert str(int(data)) == data
+
+class ApiParamBoolean(ApiParam):
+    """Base class representing an API parameter of 'boolean' type."""
+    @classmethod
+    def _encode(cls, data):
+        return bool(data)
+
+class ApiParamFqdn(ApiParam):
+    """String, as a list of domain labels concatenated using the full stop (dot, period) character as
+    separator between labels. Labels are restricted to the Alphanumeric mode character set defined in table 5
+    of ISO/IEC 18004"""
+    @classmethod
+    def verify_encoded(cls, data):
+        # FIXME
+        pass
+
+class param:
+    class Iccid(ApiParamString):
+        """String representation of 19 or 20 digits, where the 20th digit MAY optionally be the padding
+        character F."""
+        @classmethod
+        def _encode(cls, data):
+            data = str(data)
+            # SGP.22 version prior to 2.2 do not require support for 19-digit ICCIDs, so let's always
+            # encode it with padding F at the end.
+            if len(data) == 19:
+                data += 'F'
+            return data
+
+        @classmethod
+        def verify_encoded(cls, data):
+            if len(data) not in [19, 20]:
+                raise ValueError('ICCID (%s) length (%u) invalid' % (data, len(data)))
+
+        @classmethod
+        def _decode(cls, data):
+            # strip trailing padding (if it's 20 digits)
+            if len(data) == 20 and data[-1] in ['F', 'f']:
+                data = data[:-1]
+            return data
+
+        @classmethod
+        def verify_decoded(cls, data):
+            data = str(data)
+            if len(data) not in [19, 20]:
+                raise ValueError('ICCID (%s) length (%u) invalid' % (data, len(data)))
+            if len(data) == 19:
+                decimal_part = data
+            else:
+                decimal_part = data[:-1]
+                final_part = data[-1:]
+                if final_part not in ['F', 'f'] and not final_part.isdecimal():
+                    raise ValueError('ICCID (%s) contains non-decimal characters' % data)
+            if not decimal_part.isdecimal():
+                raise ValueError('ICCID (%s) contains non-decimal characters' % data)
+
+
+    class Eid(ApiParamString):
+        """String of 32 decimal characters"""
+        @classmethod
+        def verify_encoded(cls, data):
+            if len(data) != 32:
+                raise ValueError('EID length invalid: "%s" (%u)' % (data, len(data)))
+
+        @classmethod
+        def verify_decoded(cls, data):
+            if not data.isdecimal():
+                raise ValueError('EID (%s) contains non-decimal characters' % data)
+
+    class ProfileType(ApiParamString):
+        pass
+
+    class MatchingId(ApiParamString):
+        pass
+
+    class ConfirmationCode(ApiParamString):
+        pass
+
+    class SmdsAddress(ApiParamFqdn):
+        pass
+
+    class SmdpAddress(ApiParamFqdn):
+        pass
+
+    class ReleaseFlag(ApiParamBoolean):
+        pass
+
+    class FinalProfileStatusIndicator(ApiParamString):
+        pass
+
+    class Timestamp(ApiParamString):
+        """String format as specified by W3C: YYYY-MM-DDThh:mm:ssTZD"""
+        @classmethod
+        def _decode(cls, data):
+            return datetime.fromisoformat(data)
+
+        @classmethod
+        def _encode(cls, data):
+            return datetime.toisoformat(data)
+
+    class NotificationPointId(ApiParamInteger):
+        pass
+
+    class NotificationPointStatus(ApiParam):
+        pass
+
+    class ResultData(ApiParam):
+        @classmethod
+        def _decode(cls, data):
+            return base64.b64decode(data)
+
+        @classmethod
+        def _encode(cls, data):
+            return base64.b64encode(data)
+
+    class JsonResponseHeader(ApiParam):
+        """SGP.22 section 6.5.1.4."""
+        @classmethod
+        def verify_decoded(cls, data):
+            fe_status = data.get('functionExecutionStatus')
+            if not fe_status:
+                raise ValueError('Missing mandatory functionExecutionStatus in header')
+            status = fe_status.get('status')
+            if not status:
+                raise ValueError('Missing mandatory status in header functionExecutionStatus')
+            if status not in ['Executed-Success', 'Executed-WithWarning', 'Failed', 'Expired']:
+                raise ValueError('Unknown/unspecified status "%s"' % status)
+
+
+class HttpStatusError(Exception):
+    pass
+
+class HttpHeaderError(Exception):
+    pass
+
+class Es2PlusApiError(Exception):
+    """Exception representing an error at the ES2+ API level (status != Executed)."""
+    def __init__(self, func_ex_status: dict):
+        self.status = func_ex_status['status']
+        sec = {
+            'subjectCode': None,
+            'reasonCode': None,
+            'subjectIdentifier': None,
+            'message': None,
+        }
+        actual_sec = func_ex_status.get('statusCodeData', None)
+        sec.update(actual_sec)
+        self.subject_code = sec['subjectCode']
+        self.reason_code = sec['reasonCode']
+        self.subject_id = sec['subjectIdentifier']
+        self.message = sec['message']
+
+    def __str__(self):
+        return f'{self.status}("{self.subject_code}","{self.reason_code}","{self.subject_id}","{self.message}")'
+
+class Es2PlusApiFunction(abc.ABC):
+    """Base classs for representing an ES2+ API Function."""
+    # the below class variables are expected to be overridden in derived classes
+
+    path = None
+    # dictionary of input parameters. key is parameter name, value is ApiParam class
+    input_params = {}
+    # list of mandatory input parameters
+    input_mandatory = []
+    # dictionary of output parameters. key is parameter name, value is ApiParam class
+    output_params = {}
+    # list of mandatory output parameters (for successful response)
+    output_mandatory = []
+    # expected HTTP status code of the response
+    expected_http_status = 200
+
+    def __init__(self, url_prefix: str, func_req_id: str, session):
+        self.url_prefix = url_prefix
+        self.func_req_id = func_req_id
+        self.session = session
+
+    def encode(self, data: dict, func_call_id: str) -> dict:
+        """Validate an encode input dict into JSON-serializable dict for request body."""
+        output = {
+            'header': {
+                'functionRequesterIdentifier': self.func_req_id,
+                'functionCallIdentifier': func_call_id
+            }
+        }
+        for p in self.input_mandatory:
+            if not p in data:
+                raise ValueError('Mandatory input parameter %s missing' % p)
+        for p, v in data.items():
+            p_class = self.input_params.get(p)
+            if not p_class:
+                logger.warning('Unexpected/unsupported input parameter %s=%s', p, v)
+                output[p] = v
+            else:
+                output[p] = p_class.encode(v)
+        return output
+
+
+    def decode(self, data: dict) -> dict:
+        """[further] Decode and validate the JSON-Dict of the respnse body."""
+        output = {}
+        # let's first do the header, it's special
+        if not 'header' in data:
+            raise ValueError('Mandatory output parameter "header" missing')
+        hdr_class = self.output_params.get('header')
+        output['header'] = hdr_class.decode(data['header'])
+
+        if output['header']['functionExecutionStatus']['status'] not in ['Executed-Success','Executed-WithWarning']:
+            raise Es2PlusApiError(output['header']['functionExecutionStatus'])
+        # we can only expect mandatory parameters to be present in case of successful execution
+        for p in self.output_mandatory:
+            if p == 'header':
+                continue
+            if not p in data:
+                raise ValueError('Mandatory output parameter "%s" missing' % p)
+        for p, v in data.items():
+            p_class = self.output_params.get(p)
+            if not p_class:
+                logger.warning('Unexpected/unsupported output parameter "%s"="%s"', p, v)
+                output[p] = v
+            else:
+                output[p] = p_class.decode(v)
+        return output
+
+    def call(self, data: dict, func_call_id:str, timeout=10) -> dict:
+        """Make an API call to the ES2+ API endpoint represented by this object.
+        Input data is passed in `data` as json-serializable dict.  Output data
+        is returned as json-deserialized dict."""
+        url = self.url_prefix + self.path
+        encoded = json.dumps(self.encode(data, func_call_id))
+        headers = {
+            'Content-Type': 'application/json',
+            'X-Admin-Protocol': 'gsma/rsp/v2.5.0',
+        }
+
+        logger.debug("HTTP REQ %s - '%s'" % (url, encoded))
+        response = self.session.post(url, data=encoded, headers=headers, timeout=timeout)
+        logger.debug("HTTP RSP-STS: [%u] hdr: %s" % (response.status_code, response.headers))
+        logger.debug("HTTP RSP: %s" % (response.content))
+
+        if response.status_code != self.expected_http_status:
+            raise HttpStatusError(response)
+        if not response.headers.get('Content-Type').startswith(headers['Content-Type']):
+            raise HttpHeaderError(response)
+        if not response.headers.get('X-Admin-Protocol', 'gsma/rsp/v2.unknown').startswith('gsma/rsp/v2.'):
+            raise HttpHeaderError(response)
+
+        return self.decode(response.json())
+
+
+# ES2+ DownloadOrder function (SGP.22 section 5.3.1)
+class DownloadOrder(Es2PlusApiFunction):
+    path = '/gsma/rsp2/es2plus/downloadOrder'
+    input_params = {
+        'eid': param.Eid,
+        'iccid': param.Iccid,
+        'profileType': param.ProfileType
+      }
+    output_params = {
+        'header': param.JsonResponseHeader,
+        'iccid': param.Iccid,
+      }
+    output_mandatory = ['header', 'iccid']
+
+# ES2+ ConfirmOrder function (SGP.22 section 5.3.2)
+class ConfirmOrder(Es2PlusApiFunction):
+    path = '/gsma/rsp2/es2plus/confirmOrder'
+    input_params = {
+        'iccid': param.Iccid,
+        'eid': param.Eid,
+        'matchingId': param.MatchingId,
+        'confirmationCode': param.ConfirmationCode,
+        'smdsAddress': param.SmdsAddress,
+        'releaseFlag': param.ReleaseFlag,
+      }
+    input_mandatory = ['iccid', 'releaseFlag']
+    output_params = {
+        'header': param.JsonResponseHeader,
+        'eid': param.Eid,
+        'matchingId': param.MatchingId,
+        'smdpAddress': param.SmdpAddress,
+      }
+    output_mandatory = ['header', 'matchingId']
+
+# ES2+ CancelOrder function (SGP.22 section 5.3.3)
+class CancelOrder(Es2PlusApiFunction):
+    path = '/gsma/rsp2/es2plus/cancelOrder'
+    input_params = {
+        'iccid': param.Iccid,
+        'eid': param.Eid,
+        'matchingId': param.MatchingId,
+        'finalProfileStatusIndicator': param.FinalProfileStatusIndicator,
+      }
+    input_mandatory = ['finalProfileStatusIndicator', 'iccid']
+    output_params = {
+        'header': param.JsonResponseHeader,
+      }
+    output_mandatory = ['header']
+
+# ES2+ ReleaseProfile function (SGP.22 section 5.3.4)
+class ReleaseProfile(Es2PlusApiFunction):
+    path = '/gsma/rsp2/es2plus/releaseProfile'
+    input_params = {
+        'iccid': param.Iccid,
+      }
+    input_mandatory = ['iccid']
+    output_params = {
+        'header': param.JsonResponseHeader,
+      }
+    output_mandatory = ['header']
+
+# ES2+ HandleDownloadProgress function (SGP.22 section 5.3.5)
+class HandleDownloadProgressInfo(Es2PlusApiFunction):
+    path = '/gsma/rsp2/es2plus/handleDownloadProgressInfo'
+    input_params = {
+        'eid': param.Eid,
+        'iccid': param.Iccid,
+        'profileType': param.ProfileType,
+        'timestamp': param.Timestamp,
+        'notificationPointId': param.NotificationPointId,
+        'notificationPointStatus': param.NotificationPointStatus,
+        'resultData': param.ResultData,
+    }
+    input_mandatory = ['iccid', 'profileType', 'timestamp', 'notificationPointId', 'notificationPointStatus']
+    expected_http_status = 204
+
+
+class Es2pApiClient:
+    """Main class representing a full ES2+ API client. Has one method for each API function."""
+    def __init__(self, url_prefix:str, func_req_id:str, server_cert_verify: str = None, client_cert: str = None):
+        self.func_id = 0
+        self.session = requests.Session()
+        if server_cert_verify:
+            self.session.verify = server_cert_verify
+        if client_cert:
+            self.session.cert = client_cert
+
+        self.downloadOrder = DownloadOrder(url_prefix, func_req_id, self.session)
+        self.confirmOrder = ConfirmOrder(url_prefix, func_req_id, self.session)
+        self.cancelOrder = CancelOrder(url_prefix, func_req_id, self.session)
+        self.releaseProfile = ReleaseProfile(url_prefix, func_req_id, self.session)
+        self.handleDownloadProgressInfo = HandleDownloadProgressInfo(url_prefix, func_req_id, self.session)
+
+    def _gen_func_id(self) -> str:
+        """Generate the next function call id."""
+        self.func_id += 1
+        return 'FCI-%u-%u' % (time.time(), self.func_id)
+
+
+    def call_downloadOrder(self, data: dict) -> dict:
+        """Perform ES2+ DownloadOrder function (SGP.22 section 5.3.1)."""
+        return self.downloadOrder.call(data, self._gen_func_id())
+
+    def call_confirmOrder(self, data: dict) -> dict:
+        """Perform ES2+ ConfirmOrder function (SGP.22 section 5.3.2)."""
+        return self.confirmOrder.call(data, self._gen_func_id())
+
+    def call_cancelOrder(self, data: dict) -> dict:
+        """Perform ES2+ CancelOrder function (SGP.22 section 5.3.3)."""
+        return self.cancelOrder.call(data, self._gen_func_id())
+
+    def call_releaseProfile(self, data: dict) -> dict:
+        """Perform ES2+ CancelOrder function (SGP.22 section 5.3.4)."""
+        return self.releaseProfile.call(data, self._gen_func_id())
+
+    def call_handleDownloadProgressInfo(self, data: dict) -> dict:
+        """Perform ES2+ HandleDownloadProgressInfo function (SGP.22 section 5.3.5)."""
+        return self.handleDownloadProgressInfo.call(data, self._gen_func_id())

pySim/esim/es8p.py

@@ -0,0 +1,185 @@
+# Implementation of GSMA eSIM RSP (Remote SIM Provisioning) ES8+
+# as per SGP22 v3.0 Section 5.5
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import Dict, List, Optional
+from pySim.utils import b2h, h2b, bertlv_encode_tag, bertlv_encode_len
+
+import pySim.esim.rsp as rsp
+from pySim.esim.bsp import BspInstance
+
+# Given that GSMA RSP uses ASN.1 in a very weird way, we actually cannot encode the full data type before
+# signing, but we have to build parts of it separately first, then sign that, so we can put the signature
+# into the same sequence as the signed data.  We use the existing pySim TLV code for this.
+
+def wrap_as_der_tlv(tag: int, val: bytes) -> bytes:
+    """Wrap the 'value' into a DER-encoded TLV."""
+    return bertlv_encode_tag(tag) + bertlv_encode_len(len(val)) + val
+
+def gen_init_sec_chan_signed_part(iscsp: Dict) -> bytes:
+    """Generate the concatenated remoteOpId, transactionId, controlRefTemplate and smdpOtpk data objects
+    without the outer SEQUENCE tag / length or the remainder of initialiseSecureChannel, as is required
+    for signing purpose."""
+    out = b''
+    out += wrap_as_der_tlv(0x82, bytes([iscsp['remoteOpId']]))
+    out += wrap_as_der_tlv(0x80, iscsp['transactionId'])
+
+    crt = iscsp['controlRefTemplate']
+    out_crt = wrap_as_der_tlv(0x80, crt['keyType'])
+    out_crt += wrap_as_der_tlv(0x81, crt['keyLen'])
+    out_crt += wrap_as_der_tlv(0x84, crt['hostId'])
+    out += wrap_as_der_tlv(0xA6, out_crt)
+
+    out += wrap_as_der_tlv(0x5F49, iscsp['smdpOtpk'])
+    return out
+
+
+# SGP.22 Section 5.5.1
+def gen_initialiseSecureChannel(transactionId: str, host_id: bytes, smdp_otpk: bytes, euicc_otpk: bytes, dp_pb):
+    """Generate decoded representation of (signed) initialiseSecureChannel (SGP.22 5.5.2)"""
+    init_scr = { 'remoteOpId': 1, # installBoundProfilePackage
+                 'transactionId': h2b(transactionId),
+                 # GlobalPlatform Card Specification Amendment F [13] section 6.5.2.3 for the Mutual Authentication Data Field
+                 'controlRefTemplate': { 'keyType': bytes([0x88]), 'keyLen': bytes([16]), 'hostId': host_id },
+                 'smdpOtpk': smdp_otpk, # otPK.DP.KA
+                }
+    to_sign = gen_init_sec_chan_signed_part(init_scr) + wrap_as_der_tlv(0x5f49, euicc_otpk)
+    init_scr['smdpSign'] = dp_pb.ecdsa_sign(to_sign)
+    return init_scr
+
+def gen_replace_session_keys(ppk_enc: bytes, ppk_cmac: bytes, initial_mcv: bytes) -> bytes:
+    """Generate encoded (but unsigned) ReplaceSessionKeysReqest DO (SGP.22 5.5.4)"""
+    rsk = { 'ppkEnc': ppk_enc, 'ppkCmac': ppk_cmac, 'initialMacChainingValue': initial_mcv }
+    return rsp.asn1.encode('ReplaceSessionKeysRequest', rsk)
+
+
+class ProfileMetadata:
+    """Representation of Profile metadata. Right now only the mandatory bits are
+    supported, but in general this should follow the StoreMetadataRequest of SGP.22 5.5.3"""
+    def __init__(self, iccid_bin: bytes, spn: str, profile_name: str):
+        self.iccid_bin = iccid_bin
+        self.spn = spn
+        self.profile_name = profile_name
+
+    def gen_store_metadata_request(self) -> bytes:
+        """Generate encoded (but unsigned) StoreMetadataReqest DO (SGP.22 5.5.3)"""
+        smr = {
+            'iccid': self.iccid_bin,
+            'serviceProviderName': self.spn,
+            'profileName': self.profile_name,
+        }
+        return rsp.asn1.encode('StoreMetadataRequest', smr)
+
+
+class ProfilePackage:
+    def __init__(self, metadata: Optional[ProfileMetadata] = None):
+        self.metadata = metadata
+
+class UnprotectedProfilePackage(ProfilePackage):
+    """Representing an unprotected profile package (UPP) as defined in SGP.22 Section 2.5.2"""
+
+    @classmethod
+    def from_der(cls, der: bytes, metadata: Optional[ProfileMetadata] = None) -> 'UnprotectedProfilePackage':
+        """Load an UPP from its DER representation."""
+        inst = cls(metadata=metadata)
+        cls.der = der
+        # TODO: we later certainly want to parse it so we can perform modification (IMSI, key material, ...)
+        # just like in the traditional SIM/USIM dynamic data phase at the end of personalization
+        return inst
+
+    def to_der(self):
+        """Return the DER representation of the UPP."""
+        # TODO: once we work on decoded structures, we may want to re-encode here
+        return self.der
+
+class ProtectedProfilePackage(ProfilePackage):
+    """Representing a protected profile package (PPP) as defined in SGP.22 Section 2.5.3"""
+
+    @classmethod
+    def from_upp(cls, upp: UnprotectedProfilePackage, bsp: BspInstance) -> 'ProtectedProfilePackage':
+        """Generate the PPP as a sequence of encrypted and MACed Command TLVs representing the UPP"""
+        inst = cls(metadata=upp.metadata)
+        inst.upp = upp
+        # store ppk-enc, ppc-mac
+        inst.ppk_enc = bsp.c_algo.s_enc
+        inst.ppk_mac = bsp.m_algo.s_mac
+        inst.initial_mcv = bsp.m_algo.mac_chain
+        inst.encoded = bsp.encrypt_and_mac(0x86, upp.to_der())
+        return inst
+
+    #def __val__(self):
+        #return self.encoded
+
+class BoundProfilePackage(ProfilePackage):
+    """Representing a bound profile package (BPP) as defined in SGP.22 Section 2.5.4"""
+
+    @classmethod
+    def from_ppp(cls, ppp: ProtectedProfilePackage):
+        inst = cls()
+        inst.upp = None
+        inst.ppp = ppp
+        return inst
+
+    @classmethod
+    def from_upp(cls, upp: UnprotectedProfilePackage):
+        inst = cls()
+        inst.upp = upp
+        inst.ppp = None
+        return inst
+
+    def encode(self, ss: 'RspSessionState', dp_pb: 'CertAndPrivkey') -> bytes:
+        """Generate a bound profile package (SGP.22 2.5.4)."""
+
+        def encode_seq(tag: int, sequence: List[bytes]) -> bytes:
+            """Encode a "sequenceOfXX" as specified in SGP.22 specifying the raw SEQUENCE OF tag,
+            and assuming the caller provides the fully-encoded (with TAG + LEN) member TLVs."""
+            payload = b''.join(sequence)
+            return bertlv_encode_tag(tag) + bertlv_encode_len(len(payload)) + payload
+
+        bsp = BspInstance.from_kdf(ss.shared_secret, 0x88, 16, ss.host_id, h2b(ss.eid))
+
+        iscr = gen_initialiseSecureChannel(ss.transactionId, ss.host_id, ss.smdp_otpk, ss.euicc_otpk, dp_pb)
+        # generate unprotected input data
+        conf_idsp_bin = rsp.asn1.encode('ConfigureISDPRequest', {})
+        if self.upp:
+            smr_bin = self.upp.metadata.gen_store_metadata_request()
+        else:
+            smr_bin = self.ppp.metadata.gen_store_metadata_request()
+
+        # we don't use rsp.asn1.encode('boundProfilePackage') here, as the BSP already provides
+        # fully encoded + MACed TLVs including their tag + length values.  We cannot put those as
+        # 'value' input into an ASN.1 encoder, as that would double the TAG + LENGTH :(
+
+        # 'initialiseSecureChannelRequest'
+        bpp_seq = rsp.asn1.encode('InitialiseSecureChannelRequest', iscr)
+        # firstSequenceOf87
+        bpp_seq += encode_seq(0xa0, bsp.encrypt_and_mac(0x87, conf_idsp_bin))
+        # sequenceOF88
+        bpp_seq += encode_seq(0xa1, bsp.mac_only(0x88, smr_bin))
+
+        if self.ppp: # we have to use session keys
+            rsk_bin = gen_replace_session_keys(self.ppp.ppk_enc, self.ppp.ppk_mac, self.ppp.initial_mcv)
+            # secondSequenceOf87
+            bpp_seq += encode_seq(0xa2, bsp.encrypt_and_mac(0x87, rsk_bin))
+        else:
+            self.ppp = ProtectedProfilePackage.from_upp(self.upp, bsp)
+
+        # 'sequenceOf86'
+        bpp_seq += encode_seq(0xa3, self.ppp.encoded)
+
+        # manual DER encode: wrap in outer SEQUENCE
+        return bertlv_encode_tag(0xbf36) + bertlv_encode_len(len(bpp_seq)) + bpp_seq

pySim/esim/rsp.py

@@ -0,0 +1,98 @@
+# Implementation of GSMA eSIM RSP (Remote SIM Provisioning)
+# as per SGP22 v3.0
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from typing import Optional
+import shelve
+
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.hazmat.primitives.serialization import Encoding
+from cryptography import x509
+
+from pySim.esim import compile_asn1_subdir
+
+asn1 = compile_asn1_subdir('rsp')
+
+class RspSessionState:
+    """Encapsulates the state of a RSP session.  It is created during the initiateAuthentication
+    and subsequently used by further API calls using the same transactionId.  The session state
+    is removed either after cancelSession or after notification.
+    TODO: add some kind of time based expiration / garbage collection."""
+    def __init__(self, transactionId: str, serverChallenge: bytes, ci_cert_id: bytes):
+        self.transactionId = transactionId
+        self.serverChallenge = serverChallenge
+        #  used at a later point between API calsl
+        self.ci_cert_id = ci_cert_id
+        self.euicc_cert: Optional[x509.Certificate] = None
+        self.eum_cert: Optional[x509.Certificate] = None
+        self.eid: Optional[bytes] = None
+        self.profileMetadata: Optional['ProfileMetadata'] = None
+        self.smdpSignature2_do = None
+        # really only needed while processing getBoundProfilePackage request?
+        self.euicc_otpk: Optional[bytes] = None
+        self.smdp_ot: Optional[ec.EllipticCurvePrivateKey] = None
+        self.smdp_otpk: Optional[bytes] = None
+        self.host_id: Optional[bytes] = None
+        self.shared_secret: Optional[bytes] = None
+
+
+    def __getstate__(self):
+        """helper function called when pickling the object to persistent storage.  We must pickel all
+        members that are not pickle-able."""
+        state = self.__dict__.copy()
+        # serialize eUICC certificate as DER
+        if state.get('euicc_cert', None):
+            state['_euicc_cert'] = self.euicc_cert.public_bytes(Encoding.DER)
+            del state['euicc_cert']
+        # serialize EUM certificate as DER
+        if state.get('eum_cert', None):
+            state['_eum_cert'] = self.eum_cert.public_bytes(Encoding.DER)
+            del state['eum_cert']
+        # serialize one-time SMDP private key to integer + curve
+        if state.get('smdp_ot', None):
+            state['_smdp_otsk'] = self.smdp_ot.private_numbers().private_value
+            state['_smdp_ot_curve'] = self.smdp_ot.curve
+            del state['smdp_ot']
+        return state
+
+    def __setstate__(self, state):
+        """helper function called when unpickling the object from persistent storage. We must recreate all
+        members from the state generated in __getstate__ above."""
+        # restore eUICC certificate from DER
+        if '_euicc_cert' in state:
+            self.euicc_cert = x509.load_der_x509_certificate(state['_euicc_cert'])
+            del state['_euicc_cert']
+        else:
+            self.euicc_cert = None
+        # restore EUM certificate from DER
+        if '_eum_cert' in state:
+            self.eum_cert = x509.load_der_x509_certificate(state['_eum_cert'])
+            del state['_eum_cert']
+        # restore one-time SMDP private key from integer + curve
+        if state.get('_smdp_otsk', None):
+            self.smdp_ot = ec.derive_private_key(state['_smdp_otsk'], state['_smdp_ot_curve'])
+            # FIXME: how to add the public key from smdp_otpk to an instance of EllipticCurvePrivateKey?
+            del state['_smdp_otsk']
+            del state['_smdp_ot_curve']
+        # automatically recover all the remainig state
+        self.__dict__.update(state)
+
+
+class RspSessionStore(shelve.DbfilenameShelf):
+    """A derived class as wrapper around the database-backed non-volatile storage 'shelve', in case we might
+    need to extend it in the future. We use it to store RspSessionState objects indexed by transactionId."""

pySim/esim/saip/__init__.py

@@ -0,0 +1,308 @@
+# Implementation of SimAlliance/TCA Interoperable Profile handling
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+import io
+from typing import Tuple, List, Optional, Dict, Union
+
+import asn1tools
+
+from pySim.utils import bertlv_parse_tag, bertlv_parse_len
+from pySim.ts_102_221 import FileDescriptor
+from pySim.construct import build_construct
+from pySim.esim import compile_asn1_subdir
+from pySim.esim.saip import templates
+
+asn1 = compile_asn1_subdir('saip')
+
+class File:
+    """Internal representation of a file in a profile filesystem.
+
+    Parameters:
+        pename: Name string of the profile element
+        l: List of tuples [fileDescriptor, fillFileContent, fillFileOffset profile elements]
+        template: Applicable FileTemplate describing defaults as per SAIP spec
+    """
+    def __init__(self, pename: str, l: Optional[List[Tuple]] = None, template: Optional[templates.FileTemplate] = None):
+        self.pe_name = pename
+        self.template = template
+        self.fileDescriptor = {}
+        self.stream = None
+        # apply some defaults from profile
+        if self.template:
+            self.from_template(self.template)
+        print("after template: %s" % repr(self))
+        if l:
+            self.from_tuples(l)
+
+    def from_template(self, template: templates.FileTemplate):
+        """Determine defaults for file based on given FileTemplate."""
+        fdb_dec = {}
+        self.rec_len = None
+        if template.fid:
+            self.fileDescriptor['fileID'] = template.fid.to_bytes(2, 'big')
+        if template.sfi:
+            self.fileDescriptor['shortEFID'] = bytes([template.sfi])
+        if template.arr:
+            self.fileDescriptor['securityAttributesReferenced'] = bytes([template.arr])
+        # All the files defined in the templates shall have, by default, shareable/not-shareable bit in the file descriptor set to "shareable".
+        fdb_dec['shareable'] = True
+        if template.file_type in ['LF', 'CY']:
+            fdb_dec['file_type'] = 'working_ef'
+            if template.rec_len:
+                self.record_len = template.rec_len
+            if template.nb_rec and template.rec_len:
+                self.fileDescriptor['efFileSize'] = (template.nb_rec * template.rec_len).to_bytes(2, 'big') # FIXME
+            if template.file_type == 'LF':
+                fdb_dec['structure'] = 'linear_fixed'
+            elif template.file_type == 'CY':
+                fdb_dec['structure'] = 'cyclic'
+        elif template.file_type in ['TR', 'BT']:
+            fdb_dec['file_type'] = 'working_ef'
+            if template.file_size:
+                self.fileDescriptor['efFileSize'] = template.file_size.to_bytes(2, 'big') # FIXME
+            if template.file_type == 'BT':
+                fdb_dec['structure'] = 'ber_tlv'
+            elif template.file_type == 'TR':
+                fdb_dec['structure'] = 'transparent'
+        elif template.file_type in ['MF', 'DF', 'ADF']:
+            fdb_dec['file_type'] = 'df'
+            fdb_dec['structure'] = 'no_info_given'
+        # build file descriptor based on above input data
+        fd_dict = {'file_descriptor_byte': fdb_dec}
+        if self.rec_len:
+            fd_dict['record_len'] = self.rec_len
+        self.fileDescriptor['fileDescriptor'] = build_construct(FileDescriptor._construct, fd_dict)
+        # FIXME: default_val
+        # FIXME: high_update
+        # FIXME: params?
+
+    def from_tuples(self, l:List[Tuple]):
+        """Parse a list of fileDescriptor, fillFileContent, fillFileOffset tuples into this instance."""
+        def get_fileDescriptor(l:List[Tuple]):
+            for k, v in l:
+                if k == 'fileDescriptor':
+                    return v
+        fd = get_fileDescriptor(l)
+        if not fd:
+            raise ValueError("No fileDescriptor found")
+        self.fileDescriptor.update(dict(fd))
+        self.stream = self.linearize_file_content(l)
+
+    def to_tuples(self) -> List[Tuple]:
+        """Generate a list of fileDescriptor, fillFileContent, fillFileOffset tuples into this instance."""
+        raise NotImplementedError
+
+    @staticmethod
+    def linearize_file_content(l: List[Tuple]) -> Optional[io.BytesIO]:
+        """linearize a list of fillFileContent / fillFileOffset tuples into a stream of bytes."""
+        stream = io.BytesIO()
+        for k, v in l:
+            if k == 'doNotCreate':
+                return None
+            if k == 'fileDescriptor':
+                pass
+            elif k == 'fillFileOffset':
+                stream.write(b'\xff' * v)
+            elif k == 'fillFileContent':
+                stream.write(v)
+            else:
+                return ValueError("Unknown key '%s' in tuple list" % k)
+        return stream
+
+    def __str__(self) -> str:
+        return "File(%s)" % self.pe_name
+
+    def __repr__(self) -> str:
+        return "File(%s): %s" % (self.pe_name, self.fileDescriptor)
+
+class ProfileElement:
+    """Class representing a Profile Element (PE) within a SAIP Profile."""
+    FILE_BEARING = ['mf', 'cd', 'telecom', 'usim', 'opt-usim', 'isim', 'opt-isim', 'phonebook', 'gsm-access',
+                    'csim', 'opt-csim', 'eap', 'df-5gs', 'df-saip', 'df-snpn', 'df-5gprose', 'iot', 'opt-iot']
+    def _fixup_sqnInit_dec(self) -> None:
+        """asn1tools has a bug when working with SEQUENCE OF that have DEFAULT values. Let's work around
+        this."""
+        if self.type != 'akaParameter':
+            return
+        sqn_init = self.decoded.get('sqnInit', None)
+        if not sqn_init:
+            return
+        # this weird '0x' value in a string is what we get from our (slightly hacked) ASN.1 syntax
+        if sqn_init == '0x000000000000':
+            # SEQUENCE (SIZE (32)) OF OCTET STRING (SIZE (6))
+            self.decoded['sqnInit'] = [b'\x00'*6] * 32
+
+    def _fixup_sqnInit_enc(self) -> None:
+        """asn1tools has a bug when working with SEQUENCE OF that have DEFAULT values. Let's work around
+        this."""
+        if self.type != 'akaParameter':
+            return
+        sqn_init = self.decoded.get('sqnInit', None)
+        if not sqn_init:
+            return
+        for s in sqn_init:
+            if any(s):
+                return
+        # none of the fields were initialized with a non-default (non-zero) value, so we can skip it
+        del self.decoded['sqnInit']
+
+    def parse_der(self, der: bytes) -> None:
+        """Parse a sequence of PE and store the result in instance attributes."""
+        self.type, self.decoded = asn1.decode('ProfileElement', der)
+        # work around asn1tools bug regarding DEFAULT for a SEQUENCE OF
+        self._fixup_sqnInit_dec()
+
+    @property
+    def header_name(self) -> str:
+        """Return the name of the header field within the profile element."""
+        # unneccessarry compliaction by inconsistent naming :(
+        if self.type.startswith('opt-'):
+            return self.type.replace('-','') + '-header'
+        return self.type + '-header'
+
+    @property
+    def header(self):
+        """Return the decoded ProfileHeader."""
+        return self.decoded.get(self.header_name, None)
+
+    @property
+    def templateID(self):
+        """Return the decoded templateID used by this profile element (if any)."""
+        return self.decoded.get('templateID', None)
+
+    @property
+    def files(self):
+        """Return dict of decoded 'File' ASN.1 items."""
+        if not self.type in self.FILE_BEARING:
+            return {}
+        return {k:v for (k,v) in self.decoded.items() if k not in ['templateID', self.header_name]}
+
+    @classmethod
+    def from_der(cls, der: bytes) -> 'ProfileElement':
+        """Construct an instance from given raw, DER encoded bytes."""
+        inst = cls()
+        inst.parse_der(der)
+        return inst
+
+    def to_der(self) -> bytes:
+        """Build an encoded DER representation of the instance."""
+        # work around asn1tools bug regarding DEFAULT for a SEQUENCE OF
+        self._fixup_sqnInit_enc()
+        return asn1.encode('ProfileElement', (self.type, self.decoded))
+
+    def __str__(self) -> str:
+        return self.type
+
+
+def bertlv_first_segment(binary: bytes) -> Tuple[bytes, bytes]:
+    """obtain the first segment of a binary concatenation of BER-TLV objects.
+        Returns: tuple of first TLV and remainder."""
+    _tagdict, remainder = bertlv_parse_tag(binary)
+    length, remainder = bertlv_parse_len(remainder)
+    tl_length = len(binary) - len(remainder)
+    tlv_length = tl_length + length
+    return binary[:tlv_length], binary[tlv_length:]
+
+class ProfileElementSequence:
+    """A sequence of ProfileElement objects, which is the overall representation of an eSIM profile."""
+    def __init__(self):
+        self.pe_list: List[ProfileElement] = None
+        self.pe_by_type: Dict = {}
+        self.pes_by_naa: Dict = {}
+
+    def get_pes_for_type(self, tname: str) -> List[ProfileElement]:
+        """Return list of profile elements present for given profile element type."""
+        return self.pe_by_type.get(tname, [])
+
+    def get_pe_for_type(self, tname: str) -> Optional[ProfileElement]:
+        """Return a single profile element for given profile element type. Works only for
+        types of which there is only a signle instance in the PE Sequence!"""
+        l = self.get_pes_for_type(tname)
+        if len(l) == 0:
+            return None
+        assert len(l) == 1
+        return l[0]
+
+    def parse_der(self, der: bytes) -> None:
+        """Parse a sequence of PE and store the result in self.pe_list."""
+        self.pe_list = []
+        remainder = der
+        while len(remainder):
+            first_tlv, remainder = bertlv_first_segment(remainder)
+            self.pe_list.append(ProfileElement.from_der(first_tlv))
+        self._process_pelist()
+
+    def _process_pelist(self) -> None:
+        self._rebuild_pe_by_type()
+        self._rebuild_pes_by_naa()
+
+    def _rebuild_pe_by_type(self) -> None:
+        self.pe_by_type = {}
+        # build a dict {pe_type: [pe, pe, pe]}
+        for pe in self.pe_list:
+            if pe.type in self.pe_by_type:
+                self.pe_by_type[pe.type].append(pe)
+            else:
+                self.pe_by_type[pe.type] = [pe]
+
+    def _rebuild_pes_by_naa(self) -> None:
+        """rebuild the self.pes_by_naa dict {naa: [ [pe, pe, pe], [pe, pe] ]} form,
+        which basically means for every NAA there's a lsit of instances, and each consists
+        of a list of a list of PEs."""
+        self.pres_by_naa = {}
+        petype_not_naa_related = ['securityDomain', 'rfm', 'application', 'end']
+        naa = ['mf', 'usim', 'isim', 'csim']
+        cur_naa = None
+        cur_naa_list = []
+        for pe in self.pe_list:
+            # skip all PE that are not related to NAA
+            if pe.type in petype_not_naa_related:
+                continue
+            if pe.type in naa:
+                if cur_naa:
+                    if not cur_naa in self.pes_by_naa:
+                        self.pes_by_naa[cur_naa] = []
+                    self.pes_by_naa[cur_naa].append(cur_naa_list)
+                cur_naa = pe.type
+                cur_naa_list = []
+            cur_naa_list.append(pe)
+        # append the final one
+        if cur_naa and len(cur_naa_list) > 0:
+            if not cur_naa in self.pes_by_naa:
+                self.pes_by_naa[cur_naa] = []
+            self.pes_by_naa[cur_naa].append(cur_naa_list)
+
+    @classmethod
+    def from_der(cls, der: bytes) -> 'ProfileElementSequence':
+        """Construct an instance from given raw, DER encoded bytes."""
+        inst = cls()
+        inst.parse_der(der)
+        return inst
+
+    def to_der(self) -> bytes:
+        """Build an encoded DER representation of the instance."""
+        out = b''
+        for pe in self.pe_list:
+            out += pe.to_der()
+        return out
+
+    def __repr__(self) -> str:
+        return "PESequence(%s)" % ', '.join([str(x) for x in self.pe_list])
+
+    def __iter__(self) -> str:
+        yield from self.pe_list

pySim/esim/saip/data_source.py

@@ -0,0 +1,90 @@
+# Data sources: Provding data for profile personalization
+#
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+import secrets
+
+from Cryptodome.Random import get_random_bytes
+
+class DataSource(abc.ABC):
+    """Base class for something that can provide data during a personalization process."""
+
+    @abc.abstractmethod
+    def generate_one(self):
+        pass
+
+
+class DataSourceFixed(DataSource):
+    """A data source that provides a fixed value (of any type).
+
+    Parameters:
+        fixed_value: The fixed value that shall be used during each data generation
+    """
+    def __init__(self, fixed_value, **kwargs):
+        self.fixed_value = fixed_value
+        super().__init__(**kwargs)
+
+    def generate_one(self):
+        return self.fixed_value
+
+
+class DataSourceIncrementing(DataSource):
+    """A data source that provides incrementing integer numbers.
+
+    Parameters:
+        base_value: The start value (value returned during first data generation)
+        step_size: Increment step size (Default: 1)
+    """
+    def __init__(self, base_value: int, **kwargs):
+        self.base_value = int(base_value)
+        self.step_size = kwargs.pop('step_size', 1)
+        self.i = 0
+        super().__init__(**kwargs)
+
+    def generate_one(self):
+        val = self.base_value + self.i
+        self.i += self.step_size
+        return val
+
+
+class DataSourceRandomBytes(DataSource):
+    """A data source that provides a configurable number of random bytes.
+
+    Parameters:
+        size: Number of bytes to generate each turn
+    """
+    def __init__(self, size: int, **kwargs):
+        self.size = size
+        super().__init__(**kwargs)
+
+    def generate_one(self):
+        return get_random_bytes(self.size)
+
+
+class DataSourceRandomUInt(DataSource):
+    """A data source that provides a configurable unsigned integer value.
+
+    Parameters:
+        below: Number one greater than the maximum permitted random unsigned integer
+    """
+    def __init__(self, below: int, **kwargs):
+        self.below = below
+        super().__init__(**kwargs)
+
+    def generate_one(self):
+        return secrets.randbelow(self.below)
+

pySim/esim/saip/oid.py

@@ -0,0 +1,77 @@
+# Implementation of SimAlliance/TCA Interoperable Profile OIDs
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import List, Union
+
+class OID:
+    @staticmethod
+    def intlist_from_str(instr: str) -> List[int]:
+        return [int(x) for x in instr.split('.')]
+
+    @staticmethod
+    def str_from_intlist(intlist: List[int]) -> str:
+        return '.'.join([str(x) for x in intlist])
+
+    def __init__(self, initializer: Union[List[int], str]):
+        if isinstance(initializer, str):
+            self.intlist = self.intlist_from_str(initializer)
+        else:
+            self.intlist = initializer
+
+    def __str__(self) -> str:
+        return self.str_from_intlist(self.intlist)
+
+    def __repr__(self) -> str:
+        return 'OID(%s)' % (str(self))
+
+
+class eOID(OID):
+    """OID helper for TCA eUICC prefix"""
+    __prefix = [2,23,143,1]
+    def __init__(self, initializer):
+        if isinstance(initializer, str):
+            initializer = self.intlist_from_str(initializer)
+        super().__init__(self.__prefix + initializer)
+
+MF = eOID("2.1")
+DF_CD = eOID("2.2")
+DF_TELECOM = eOID("2.3")
+DF_TELECOM_v2 = eOID("2.3.2")
+ADF_USIM_by_default = eOID("2.4")
+ADF_USIM_by_default_v2 = eOID("2.4.2")
+ADF_USIM_not_by_default = eOID("2.5")
+ADF_USIM_not_by_default_v2 = eOID("2.5.2")
+ADF_USIM_not_by_default_v3 = eOID("2.5.3")
+DF_PHONEBOOK_ADF_USIM = eOID("2.6")
+DF_GSM_ACCESS_ADF_USIM = eOID("2.7")
+ADF_ISIM_by_default = eOID("2.8")
+ADF_ISIM_not_by_default = eOID("2.9")
+ADF_ISIM_not_by_default_v2 = eOID("2.9.2")
+ADF_CSIM_by_default = eOID("2.10")
+ADF_CSIM_by_default_v2 = eOID("2.10.2")
+ADF_CSIM_not_by_default = eOID("2.11")
+ADF_CSIM_not_by_default_v2 = eOID("2.11.2")
+DF_EAP = eOID("2.12")
+DF_5GS = eOID("2.13")
+DF_5GS_v2 = eOID("2.13.2")
+DF_5GS_v3 = eOID("2.13.3")
+DF_5GS_v4 = eOID("2.13.4")
+DF_SAIP = eOID("2.14")
+DF_SNPN = eOID("2.15")
+DF_5GProSe = eOID("2.16")
+IoT_default = eOID("2.17")
+IoT_default = eOID("2.18")

pySim/esim/saip/personalization.py

@@ -0,0 +1,324 @@
+# Implementation of SimAlliance/TCA Interoperable Profile handling
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+import io
+from typing import List, Tuple
+
+from pySim.tlv import camel_to_snake
+from pySim.utils import enc_iccid, enc_imsi, h2b, rpad, sanitize_iccid
+from pySim.esim.saip import ProfileElement, ProfileElementSequence
+
+def remove_unwanted_tuples_from_list(l: List[Tuple], unwanted_keys: List[str]) -> List[Tuple]:
+    """In a list of tuples, remove all tuples whose first part equals 'unwanted_key'."""
+    return list(filter(lambda x: x[0] not in unwanted_keys, l))
+
+def file_replace_content(file: List[Tuple], new_content: bytes):
+    """Completely replace all fillFileContent of a decoded 'File' with the new_content."""
+    # use [:] to avoid making a copy, as we're doing in-place modification of the list here
+    file[:] = remove_unwanted_tuples_from_list(file, ['fillFileContent', 'fillFileOffset'])
+    file.append(('fillFileContent', new_content))
+    return file
+
+class ClassVarMeta(abc.ABCMeta):
+    """Metaclass that puts all additional keyword-args into the class. We use this to have one
+    class definition for something like a PIN, and then have derived classes for PIN1, PIN2, ..."""
+    def __new__(metacls, name, bases, namespace, **kwargs):
+        #print("Meta_new_(metacls=%s, name=%s, bases=%s, namespace=%s, kwargs=%s)" % (metacls, name, bases, namespace, kwargs))
+        x = super().__new__(metacls, name, bases, namespace)
+        for k, v in kwargs.items():
+            setattr(x, k, v)
+        setattr(x, 'name', camel_to_snake(name))
+        return x
+
+class ConfigurableParameter(abc.ABC, metaclass=ClassVarMeta):
+    """Base class representing a part of the eSIM profile that is configurable during the
+    personalization process (with dynamic data from elsewhere)."""
+    def __init__(self, input_value):
+        self.input_value = input_value # the raw input value as given by caller
+        self.value = None # the processed input value (e.g. with check digit) as produced by validate()
+
+    def validate(self):
+        """Optional validation method. Can be used by derived classes to perform validation
+        of the input value (self.value).  Will raise an exception if validation fails."""
+        # default implementation: simply copy input_value over to value
+        self.value = self.input_value
+
+    @abc.abstractmethod
+    def apply(self, pes: ProfileElementSequence):
+        pass
+
+class Iccid(ConfigurableParameter):
+    """Configurable ICCID.  Expects the value to be a string of decimal digits.
+    If the string of digits is only 18 digits long, a Luhn check digit will be added."""
+
+    def validate(self):
+        # convert to string as it migt be an integer
+        iccid_str = str(self.input_value)
+        if len(iccid_str) < 18 or len(iccid_str) > 20:
+            raise ValueError('ICCID must be 18, 19 or 20 digits long')
+        if not iccid_str.isdecimal():
+            raise ValueError('ICCID must only contain decimal digits')
+        self.value = sanitize_iccid(iccid_str)
+
+    def apply(self, pes: ProfileElementSequence):
+        # patch the header
+        pes.get_pe_for_type('header').decoded['iccid'] = h2b(rpad(self.value, 20))
+        # patch MF/EF.ICCID
+        file_replace_content(pes.get_pe_for_type('mf').decoded['ef-iccid'], h2b(enc_iccid(self.value)))
+
+class Imsi(ConfigurableParameter):
+    """Configurable IMSI. Expects value to be a string of digits. Automatically sets the ACC to
+    the last digit of the IMSI."""
+
+    def validate(self):
+        # convert to string as it migt be an integer
+        imsi_str = str(self.input_value)
+        if len(imsi_str) < 6 or len(imsi_str) > 15:
+            raise ValueError('IMSI must be 6..15 digits long')
+        if not imsi_str.isdecimal():
+            raise ValueError('IMSI must only contain decimal digits')
+        self.value = imsi_str
+
+    def apply(self, pes: ProfileElementSequence):
+        imsi_str = self.value
+        # we always use the least significant byte of the IMSI as ACC
+        acc = (1 << int(imsi_str[-1]))
+        # patch ADF.USIM/EF.IMSI
+        for pe in pes.get_pes_for_type('usim'):
+            file_replace_content(pe.decoded['ef-imsi'], h2b(enc_imsi(imsi_str)))
+            file_replace_content(pe.decoded['ef-acc'], acc.to_bytes(2, 'big'))
+        # TODO: DF.GSM_ACCESS if not linked?
+
+
+class SdKey(ConfigurableParameter, metaclass=ClassVarMeta):
+    """Configurable Security Domain (SD) Key.  Value is presented as bytes."""
+    # these will be set by derived classes
+    key_type = None
+    key_id = None
+    kvn = None
+    key_usage_qual = None
+    permitted_len = None
+
+    def validate(self):
+        if not isinstance(self.input_value, (io.BytesIO, bytes, bytearray)):
+            raise ValueError('Value must be of bytes-like type')
+        if self.permitted_len:
+            if len(self.input_value) not in self.permitted_len:
+                raise ValueError('Value length must be %s' % self.permitted_len)
+        self.value = self.input_value
+
+    def _apply_sd(self, pe: ProfileElement):
+        assert pe.type == 'securityDomain'
+        for key in pe.decoded['keyList']:
+            if key['keyIdentifier'][0] == self.key_id and key['keyVersionNumber'][0] == self.kvn:
+                assert len(key['keyComponents']) == 1
+                key['keyComponents'][0]['keyData'] = self.value
+                return
+        # Could not find matching key to patch, create a new one
+        key = {
+            'keyUsageQualifier': bytes([self.key_usage_qual]),
+            'keyIdentifier': bytes([self.key_id]),
+            'keyVersionNumber': bytes([self.kvn]),
+            'keyComponents': [
+                { 'keyType': bytes([self.key_type]), 'keyData': self.value },
+            ]
+        }
+        pe.decoded['keyList'].append(key)
+
+    def apply(self, pes: ProfileElementSequence):
+        for pe in pes.get_pes_for_type('securityDomain'):
+            self._apply_sd(pe)
+
+class SdKeyScp80_01(SdKey, kvn=0x01, key_type=0x88, permitted_len=[16,24,32]): # AES key type
+    pass
+class SdKeyScp80_01Kic(SdKeyScp80_01, key_id=0x01, key_usage_qual=0x18): # FIXME: ordering?
+    pass
+class SdKeyScp80_01Kid(SdKeyScp80_01, key_id=0x02, key_usage_qual=0x14):
+    pass
+class SdKeyScp80_01Kik(SdKeyScp80_01, key_id=0x03, key_usage_qual=0x48):
+    pass
+
+class SdKeyScp81_01(SdKey, kvn=0x81): # FIXME
+    pass
+class SdKeyScp81_01Psk(SdKeyScp81_01, key_id=0x01, key_type=0x85, key_usage_qual=0x3C):
+    pass
+class SdKeyScp81_01Dek(SdKeyScp81_01, key_id=0x02, key_type=0x88, key_usage_qual=0x48):
+    pass
+
+class SdKeyScp02_20(SdKey, kvn=0x20, key_type=0x88, permitted_len=[16,24,32]): # AES key type
+    pass
+class SdKeyScp02_20Enc(SdKeyScp02_20, key_id=0x01, key_usage_qual=0x18):
+    pass
+class SdKeyScp02_20Mac(SdKeyScp02_20, key_id=0x02, key_usage_qual=0x14):
+    pass
+class SdKeyScp02_20Dek(SdKeyScp02_20, key_id=0x03, key_usage_qual=0x48):
+    pass
+
+class SdKeyScp03_30(SdKey, kvn=0x30, key_type=0x88, permitted_len=[16,24,32]): # AES key type
+    pass
+class SdKeyScp03_30Enc(SdKeyScp03_30, key_id=0x01, key_usage_qual=0x18):
+    pass
+class SdKeyScp03_30Mac(SdKeyScp03_30, key_id=0x02, key_usage_qual=0x14):
+    pass
+class SdKeyScp03_30Dek(SdKeyScp03_30, key_id=0x03, key_usage_qual=0x48):
+    pass
+
+class SdKeyScp03_31(SdKey, kvn=0x31, key_type=0x88, permitted_len=[16,24,32]): # AES key type
+    pass
+class SdKeyScp03_31Enc(SdKeyScp03_31, key_id=0x01, key_usage_qual=0x18):
+    pass
+class SdKeyScp03_31Mac(SdKeyScp03_31, key_id=0x02, key_usage_qual=0x14):
+    pass
+class SdKeyScp03_31Dek(SdKeyScp03_31, key_id=0x03, key_usage_qual=0x48):
+    pass
+
+class SdKeyScp03_32(SdKey, kvn=0x32, key_type=0x88, permitted_len=[16,24,32]): # AES key type
+    pass
+class SdKeyScp03_32Enc(SdKeyScp03_32, key_id=0x01, key_usage_qual=0x18):
+    pass
+class SdKeyScp03_32Mac(SdKeyScp03_32, key_id=0x02, key_usage_qual=0x14):
+    pass
+class SdKeyScp03_32Dek(SdKeyScp03_32, key_id=0x03, key_usage_qual=0x48):
+    pass
+
+
+
+
+def obtain_singleton_pe_from_pelist(l: List[ProfileElement], wanted_type: str) -> ProfileElement:
+    filtered = list(filter(lambda x: x.type == wanted_type, l))
+    assert len(filtered) == 1
+    return filtered[0]
+
+def obtain_first_pe_from_pelist(l: List[ProfileElement], wanted_type: str) -> ProfileElement:
+    filtered = list(filter(lambda x: x.type == wanted_type, l))
+    return filtered[0]
+
+class Puk(ConfigurableParameter, metaclass=ClassVarMeta):
+    """Configurable PUK (Pin Unblock Code). String ASCII-encoded digits."""
+    keyReference = None
+    def validate(self):
+        if isinstance(self.input_value, int):
+            self.value = '%08d' % self.input_value
+        else:
+            self.value = self.input_value
+        # FIXME: valid length?
+        if not self.value.isdecimal():
+            raise ValueError('PUK must only contain decimal digits')
+
+    def apply(self, pes: ProfileElementSequence):
+        puk = ''.join(['%02x' % (ord(x)) for x in self.value])
+        padded_puk = rpad(puk, 16)
+        mf_pes = pes.pes_by_naa['mf'][0]
+        pukCodes = obtain_singleton_pe_from_pelist(mf_pes, 'pukCodes')
+        for pukCode in pukCodes.decoded['pukCodes']:
+            if pukCode['keyReference'] == self.keyReference:
+                pukCode['pukValue'] = h2b(padded_puk)
+                return
+        raise ValueError('cannot find pukCode')
+class Puk1(Puk, keyReference=0x01):
+    pass
+class Puk2(Puk, keyReference=0x81):
+    pass
+
+class Pin(ConfigurableParameter, metaclass=ClassVarMeta):
+    """Configurable PIN (Personal Identification Number).  String of digits."""
+    keyReference = None
+    def validate(self):
+        if isinstance(self.input_value, int):
+            self.value = '%04d' % self.input_value
+        else:
+            self.value = self.input_value
+        if len(self.value) < 4 or len(self.value) > 8:
+            raise ValueError('PIN mus be 4..8 digits long')
+        if not self.value.isdecimal():
+            raise ValueError('PIN must only contain decimal digits')
+    def apply(self, pes: ProfileElementSequence):
+        pin = ''.join(['%02x' % (ord(x)) for x in self.value])
+        padded_pin = rpad(pin, 16)
+        mf_pes = pes.pes_by_naa['mf'][0]
+        pinCodes = obtain_first_pe_from_pelist(mf_pes, 'pinCodes')
+        if pinCodes.decoded['pinCodes'][0] != 'pinconfig':
+            return
+        for pinCode in pinCodes.decoded['pinCodes'][1]:
+            if pinCode['keyReference'] == self.keyReference:
+                 pinCode['pinValue'] = h2b(padded_pin)
+                 return
+        raise ValueError('cannot find pinCode')
+class AppPin(ConfigurableParameter, metaclass=ClassVarMeta):
+    """Configurable PIN (Personal Identification Number).  String of digits."""
+    keyReference = None
+    def validate(self):
+        if isinstance(self.input_value, int):
+            self.value = '%04d' % self.input_value
+        else:
+            self.value = self.input_value
+        if len(self.value) < 4 or len(self.value) > 8:
+            raise ValueError('PIN mus be 4..8 digits long')
+        if not self.value.isdecimal():
+            raise ValueError('PIN must only contain decimal digits')
+    def _apply_one(self, pe: ProfileElement):
+        pin = ''.join(['%02x' % (ord(x)) for x in self.value])
+        padded_pin = rpad(pin, 16)
+        pinCodes = obtain_first_pe_from_pelist(pe, 'pinCodes')
+        if pinCodes.decoded['pinCodes'][0] != 'pinconfig':
+            return
+        for pinCode in pinCodes.decoded['pinCodes'][1]:
+            if pinCode['keyReference'] == self.keyReference:
+                pinCode['pinValue'] = h2b(padded_pin)
+                return
+        raise ValueError('cannot find pinCode')
+    def apply(self, pes: ProfileElementSequence):
+        for naa in pes.pes_by_naa:
+            if naa not in ['usim','isim','csim','telecom']:
+                continue
+            for instance in pes.pes_by_naa[naa]:
+                self._apply_one(instance)
+class Pin1(Pin, keyReference=0x01):
+    pass
+# PIN2 is special: telecom + usim + isim + csim
+class Pin2(AppPin, keyReference=0x81):
+    pass
+class Adm1(Pin, keyReference=0x0A):
+    pass
+class Adm2(Pin, keyReference=0x0B):
+    pass
+
+
+class AlgoConfig(ConfigurableParameter, metaclass=ClassVarMeta):
+    """Configurable Algorithm parameter.  bytes."""
+    key = None
+    def validate(self):
+        if not isinstance(self.input_value, (io.BytesIO, bytes, bytearray)):
+            raise ValueError('Value must be of bytes-like type')
+        self.value = self.input_value
+    def apply(self, pes: ProfileElementSequence):
+        for pe in pes.get_pes_for_type('akaParameter'):
+            algoConfiguration = pe.decoded['algoConfiguration']
+            if algoConfiguration[0] != 'algoParameter':
+                continue
+            algoConfiguration[1][self.key] = self.value
+
+class K(AlgoConfig, key='key'):
+    pass
+class Opc(AlgoConfig, key='opc'):
+    pass
+class AlgorithmID(AlgoConfig, key='algorithmID'):
+    def validate(self):
+        if self.input_value not in [1, 2, 3]:
+            raise ValueError('Invalid algorithmID %s' % (self.input_value))
+        self.value = self.input_value

pySim/esim/saip/templates.py

@@ -0,0 +1,675 @@
+# Implementation of SimAlliance/TCA Interoperable Profile Template handling
+#
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import *
+from copy import deepcopy
+import pySim.esim.saip.oid as OID
+
+class FileTemplate:
+    """Representation of a single file in a SimAlliance/TCA Profile Template."""
+    def __init__(self, fid:int, name:str, ftype, nb_rec: Optional[int], size:Optional[int], arr:int,
+                 sfi:Optional[int] = None, default_val:Optional[str] = None, content_rqd:bool = True,
+                 params:Optional[List] = None, ass_serv:Optional[List[int]]=None, high_update:bool = False,
+                 pe_name:Optional[str] = None):
+        # initialize from arguments
+        self.fid = fid
+        self.name = name
+        if pe_name:
+            self.pe_name = pe_name
+        else:
+            self.pe_name = self.name.replace('.','-').replace('_','-').lower()
+        self.file_type = ftype
+        if ftype in ['LF', 'CY']:
+            self.nb_rec = nb_rec
+            self.rec_len = size
+        elif ftype in ['TR']:
+            self.file_size = size
+        self.arr = arr
+        self.sfi = sfi
+        self.default_val = default_val
+        self.content_rqd = content_rqd
+        self.params = params
+        self.ass_serv = ass_serv
+        self.high_update = high_update
+        # initialize empty
+        self.parent = None
+        self.children = []
+
+    def __str__(self) -> str:
+        return "FileTemplate(%s)" % (self.name)
+
+    def __repr__(self) -> str:
+        s_fid = "%04x" % self.fid if self.fid is not None else 'None'
+        s_arr = self.arr if self.arr is not None else 'None'
+        s_sfi = "%02x" % self.sfi if self.sfi is not None else 'None'
+        return "FileTemplate(%s/%s, %s, %s, arr=%s, sfi=%s)" % (self.name, self.pe_name, s_fid,
+                                                                self.file_type, s_arr, s_sfi)
+
+class ProfileTemplate:
+    """Representation of a SimAlliance/TCA Profile Template.  Each Template is identified by its OID and
+    consists of a number of file definitions.  We implement each profile template as a class derived from this
+    base class.  Each such derived class is a singleton and has no instances."""
+    created_by_default: bool = False
+    oid: Optional[OID.eOID] = None
+    files: List[FileTemplate] = []
+    files_by_pename: dict[str,FileTemplate] = {}
+
+    def __init_subclass__(cls, **kwargs):
+        """This classmethod is called automatically after executing the subclass body. We use it to
+        initialize the cls.files_by_pename from the cls.files"""
+        super().__init_subclass__(**kwargs)
+        for f in cls.files:
+            cls.files_by_pename[f.pe_name] = f
+        ProfileTemplateRegistry.add(cls)
+
+class ProfileTemplateRegistry:
+    """A registry of profile templates.  Exists as a singleton class with no instances and only
+    classmethods."""
+    by_oid = {}
+
+    @classmethod
+    def add(cls, tpl: ProfileTemplate):
+        """Add a ProfileTemplate to the registry.  There can only be one Template per OID."""
+        oid_str = str(tpl.oid)
+        if oid_str in cls.by_oid:
+            raise ValueError("We already have a template for OID %s" % oid_str)
+        cls.by_oid[oid_str] = tpl
+
+    @classmethod
+    def get_by_oid(cls, oid: Union[List[int], str]) -> Optional[ProfileTemplate]:
+        """Look-up the ProfileTemplate based on its OID.  The OID can be given either in dotted-string format,
+        or as a list of integers."""
+        if not isinstance(oid, str):
+            oid = OID.OID.str_from_intlist(oid)
+        return cls.by_oid.get(oid, None)
+
+# below are transcribed template definitions from "ANNEX A (Normative): File Structure Templates Definition"
+# of "Profile interoperability specification V3.1 Final" (unless other version explicitly specified).
+
+# Section 9.2
+class FilesAtMF(ProfileTemplate):
+    created_by_default = True
+    oid = OID.MF
+    files = [
+        FileTemplate(0x3f00, 'MF',           'MF', None, None,  14, None, None, None, params=['pinStatusTemplateDO']),
+        FileTemplate(0x2f05, 'EF.PL',        'TR', None,    2,   1, 0x05, 'FF...FF', None),
+        FileTemplate(0x2f02, 'EF.ICCID',     'TR', None,   10,  11, None, None, True),
+        FileTemplate(0x2f00, 'EF.DIR',       'LF', None, None,  10, 0x1e, None, True, params=['nb_rec', 'size']),
+        FileTemplate(0x2f06, 'EF.ARR',       'LF', None, None,  10, None, None, True, params=['nb_rec', 'size']),
+        FileTemplate(0x2f08, 'EF.UMPC',      'TR', None,    5,  10, 0x08, None, False),
+    ]
+
+
+# Section 9.3
+class FilesCD(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_CD
+    files = [
+        FileTemplate(0x7f11, 'DF.CD',        'DF', None, None,  14, None, None, False, params=['pinStatusTemplateDO']),
+        FileTemplate(0x6f01, 'EF.LAUNCHPAD', 'TR', None, None,   2, None, None, True, params=['size']),
+    ]
+    for i in range(0x40, 0x7f):
+        files.append(FileTemplate(0x6f00+i, 'EF.ICON',      'TR', None, None,   2, None, None, True, params=['size']))
+
+
+# Section 9.4: Do this separately, so we can use them also from 9.5.3
+df_pb_files = [
+    FileTemplate(0x5f3a, 'DF.PHONEBOOK', 'DF', None, None,  14, None, None, True, ['pinStatusTemplateDO']),
+    FileTemplate(0x4f30, 'EF.PBR',       'LF', None, None,   2, None, None, True, ['nb_rec', 'size']),
+]
+for i in range(0x38, 0x40):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.EXT1', 'LF', None,   13,  5, None, '00FF...FF', False, ['size','sfi']))
+for i in range(0x40, 0x48):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.AAS', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size']))
+for i in range(0x48, 0x50):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.GAS', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size']))
+df_pb_files += [
+    FileTemplate(0x4f22, 'EF.PSC',       'TR', None,    4,   5, None, '00000000', False, ['sfi']),
+    FileTemplate(0x4f23, 'EF.CC',        'TR', None,    2,   5, None, '0000', False, ['sfi'], high_update=True),
+    FileTemplate(0x4f24, 'EF.PUID',      'TR', None,    2,   5, None, '0000', False, ['sfi'], high_update=True),
+]
+for i in range(0x50, 0x58):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.IAP', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+for i in range(0x58, 0x60):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.ADN', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+for i in range(0x60, 0x68):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.ADN', 'LF', None,    2,  5, None, '00...00', False, ['nb_rec','sfi']))
+for i in range(0x68, 0x70):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.ANR', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+for i in range(0x70, 0x78):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.PURI', 'LF', None, None,  5, None, None, True, ['nb_rec','size','sfi']))
+for i in range(0x78, 0x80):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.EMAIL', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+for i in range(0x80, 0x88):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.SNE', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+for i in range(0x88, 0x90):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.UID', 'LF', None,    2,  5, None, '0000', False, ['nb_rec','sfi']))
+for i in range(0x90, 0x98):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.GRP', 'LF', None, None,  5, None, '00...00', False, ['nb_rec','size','sfi']))
+for i in range(0x98, 0xa0):
+    df_pb_files.append(FileTemplate(0x4f00+i, 'EF.CCP1', 'LF', None, None,  5, None, 'FF...FF', False, ['nb_rec','size','sfi']))
+
+# Section 9.4 v2.3.1
+class FilesTelecom(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_TELECOM
+    files = [
+        FileTemplate(0x7f11, 'DF.TELECOM',   'DF', None, None,  14, None, None, False, params=['pinStatusTemplateDO']),
+        FileTemplate(0x6f06, 'EF.ARR',       'LF', None, None,  10, None, None, True, ['nb_rec', 'size']),
+        FileTemplate(0x6f53, 'EF.RMA',       'LF', None, None,   3, None, None, True, ['nb_rec', 'size']),
+        FileTemplate(0x6f54, 'EF.SUME',      'TR', None,   22,   3, None, None, True),
+        FileTemplate(0x6fe0, 'EF.ICE_DN',    'LF',   50,   24,   9, None, 'FF...FF', False),
+        FileTemplate(0x6fe1, 'EF.ICE_FF',    'LF', None, None,   9, None, 'FF...FF', False, ['nb_rec', 'size']),
+        FileTemplate(0x6fe5, 'EF.PSISMSC',   'LF', None, None,   5, None, None, True, ['nb_rec', 'size'], ass_serv=[12,91]),
+        FileTemplate(0x5f50, 'DF.GRAPHICS',  'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO']),
+        FileTemplate(0x4f20, 'EF.IMG',       'LF', None, None,   2, None, '00FF...FF', False, ['nb_rec', 'size']),
+        # EF.IIDF below
+        FileTemplate(0x4f21, 'EF.ICE_GRAPHICS','BT',None,None,   9, None, None, False, ['size']),
+        FileTemplate(0x4f01, 'EF.LAUNCH_SCWS','TR',None, None,  10, None, None, True, ['size']),
+        # EF.ICON below
+    ]
+    for i in range(0x40, 0x80):
+        files.append(FileTemplate(0x4f00+i, 'EF.IIDF', 'TR', None, None, 2, None, 'FF...FF', False, ['size']))
+    for i in range(0x80, 0xC0):
+        files.append(FileTemplate(0x4f00+i, 'EF.ICON', 'TR', None, None, 10, None, None, True, ['size']))
+
+    # we copy the objects (instances) here as we also use them below from FilesUsimDfPhonebook
+    files += [deepcopy(x) for x in df_pb_files]
+
+    files += [
+        FileTemplate(0x5f3b, 'DF.MULTIMEDIA','DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[67]),
+        FileTemplate(0x4f47, 'EF.MML',       'BT', None, None,   5, None, None, False, ['size'], ass_serv=[67]),
+        FileTemplate(0x4f48, 'EF.MMDF',      'BT', None, None,   5, None, None, False, ['size'], ass_serv=[67]),
+
+        FileTemplate(0x5f3c, 'DF.MMSS',      'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO']),
+        FileTemplate(0x4f20, 'EF.MLPL',      'TR', None, None,   2, 0x01, None, True, ['size']),
+        FileTemplate(0x4f21, 'EF.MSPL',      'TR', None, None,   2, 0x02, None, True, ['size']),
+        FileTemplate(0x4f21, 'EF.MMSSMODE',  'TR', None,    1,   2, 0x03, None, True),
+    ]
+
+
+# Section 9.4
+class FilesTelecomV2(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_TELECOM_v2
+    files = [
+        FileTemplate(0x7f11, 'DF.TELECOM',   'DF', None, None,  14, None, None, False, params=['pinStatusTemplateDO']),
+        FileTemplate(0x6f06, 'EF.ARR',       'LF', None, None,  10, None, None, True, ['nb_rec', 'size']),
+        FileTemplate(0x6f53, 'EF.RMA',       'LF', None, None,   3, None, None, True, ['nb_rec', 'size']),
+        FileTemplate(0x6f54, 'EF.SUME',      'TR', None,   22,   3, None, None, True),
+        FileTemplate(0x6fe0, 'EF.ICE_DN',    'LF',   50,   24,   9, None, 'FF...FF', False),
+        FileTemplate(0x6fe1, 'EF.ICE_FF',    'LF', None, None,   9, None, 'FF...FF', False, ['nb_rec', 'size']),
+        FileTemplate(0x6fe5, 'EF.PSISMSC',   'LF', None, None,   5, None, None, True, ['nb_rec', 'size'], ass_serv=[12,91]),
+        FileTemplate(0x5f50, 'DF.GRAPHICS',  'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO']),
+        FileTemplate(0x4f20, 'EF.IMG',       'LF', None, None,   2, None, '00FF...FF', False, ['nb_rec', 'size']),
+        # EF.IIDF below
+        FileTemplate(0x4f21, 'EF.ICE_GRRAPHICS','BT',None,None,   9, None, None, False, ['size']),
+        FileTemplate(0x4f01, 'EF.LAUNCH_SCWS','TR',None, None,  10, None, None, True, ['size']),
+        # EF.ICON below
+    ]
+    for i in range(0x40, 0x80):
+        files.append(FileTemplate(0x4f00+i, 'EF.IIDF', 'TR', None, None, 2, None, 'FF...FF', False, ['size']))
+    for i in range(0x80, 0xC0):
+        files.append(FileTemplate(0x4f00+i, 'EF.ICON', 'TR', None, None, 10, None, None, True, ['size']))
+
+    # we copy the objects (instances) here as we also use them below from FilesUsimDfPhonebook
+    files += [deepcopy(x) for x in df_pb_files]
+
+    files += [
+        FileTemplate(0x5f3b, 'DF.MULTIMEDIA','DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[67]),
+        FileTemplate(0x4f47, 'EF.MML',       'BT', None, None,   5, None, None, False, ['size'], ass_serv=[67]),
+        FileTemplate(0x4f48, 'EF.MMDF',      'BT', None, None,   5, None, None, False, ['size'], ass_serv=[67]),
+
+        FileTemplate(0x5f3c, 'DF.MMSS',      'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO']),
+        FileTemplate(0x4f20, 'EF.MLPL',      'TR', None, None,   2, 0x01, None, True, ['size']),
+        FileTemplate(0x4f21, 'EF.MSPL',      'TR', None, None,   2, 0x02, None, True, ['size']),
+        FileTemplate(0x4f21, 'EF.MMSSMODE',  'TR', None,    1,   2, 0x03, None, True),
+
+
+        FileTemplate(0x5f3d, 'DF.MCS',       'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv={'usim':109, 'isim': 15}),
+        FileTemplate(0x4f01, 'EF.MST',       'TR', None, None,   2, 0x01, None, True, ['size'], ass_serv={'usim':109, 'isim': 15}),
+        FileTemplate(0x4f02, 'EF.MCSCONFIG', 'BT', None, None,   2, 0x02, None, True, ['size'], ass_serv={'usim':109, 'isim': 15}),
+
+        FileTemplate(0x5f3e, 'DF.V2X',       'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[119]),
+        FileTemplate(0x4f01, 'EF.VST',       'TR', None, None,   2, 0x01, None, True, ['size'], ass_serv=[119]),
+        FileTemplate(0x4f02, 'EF.V2X_CONFIG','BT', None, None,   2, 0x02, None, True, ['size'], ass_serv=[119]),
+        FileTemplate(0x4f03, 'EF.V2XP_PC5',  'TR', None, None,   2, None, None, True, ['size'], ass_serv=[119]), # VST: 2
+        FileTemplate(0x4f04, 'EF.V2XP_Uu',   'TR', None, None,   2, None, None, True, ['size'], ass_serv=[119]), # VST: 3
+    ]
+
+
+# Section 9.5.1 v2.3.1
+class FilesUsimMandatory(ProfileTemplate):
+    created_by_default = True
+    oid = OID.ADF_USIM_by_default
+    files = [
+        FileTemplate(  None, 'ADF.USIM',    'ADF', None, None,  14, None, None, False, ['aid', 'temp_fid', 'pinStatusTemplateDO']),
+        FileTemplate(0x6f07, 'EF.IMSI',      'TR', None,    9,   2, 0x07, None, True, ['size']),
+        FileTemplate(0x6f06, 'EF.ARR',       'LF', None, None,  10, 0x17, None, True, ['nb_rec','size']),
+        FileTemplate(0x6f08, 'EF.Keys',      'TR', None,   33,   5, 0x08, '07FF...FF', False, high_update=True),
+        FileTemplate(0x6f09, 'EF.KeysPS',    'TR', None,   33,   5, 0x09, '07FF...FF', False, high_update=True, pe_name = 'ef-keysPS'),
+        FileTemplate(0x6f31, 'EF.HPPLMN',    'TR', None,    1,   2, 0x12, '0A', False),
+        FileTemplate(0x6f38, 'EF.UST',       'TR', None,   14,   2, 0x04, None, True),
+        FileTemplate(0x6f3b, 'EF.FDN',       'LF',   20,   26,   8, None, 'FF...FF', False, ass_serv=[2, 89]),
+        FileTemplate(0x6f3c, 'EF.SMS',       'LF',   10,  176,   5, None, '00FF...FF', False, ass_serv=[10]),
+        FileTemplate(0x6f42, 'EF.SMSP',      'LF',    1,   38,   5, None, 'FF...FF', False, ass_serv=[12]),
+        FileTemplate(0x6f43, 'EF.SMSS',      'TR', None,    2,   5, None, 'FFFF', False, ass_serv=[10]),
+        FileTemplate(0x6f46, 'EF.SPN',       'TR', None,   17,  10, None, None, True, ass_serv=[19]),
+        FileTemplate(0x6f56, 'EF.EST',       'TR', None,    1,   8, 0x05, None, True, ass_serv=[2,6,34,35]),
+        FileTemplate(0x6f5b, 'EF.START-HFN', 'TR', None,    6,   5, 0x0f, 'F00000F00000', False, high_update=True),
+        FileTemplate(0x6f5c, 'EF.THRESHOLD', 'TR', None,    3,   2, 0x10, 'FFFFFF', False),
+        FileTemplate(0x6f73, 'EF.PSLOCI',    'TR', None,   14,   5, 0x0c, 'FFFFFFFFFFFFFFFFFFFF0000FF01', False, high_update=True),
+        FileTemplate(0x6f78, 'EF.ACC',       'TR', None,    2,   2, 0x06, None, True),
+        FileTemplate(0x6f7b, 'EF.FPLMN',     'TR', None,   12,   5, 0x0d, 'FF...FF', False),
+        FileTemplate(0x6f7e, 'EF.LOCI',      'TR', None,   11,   5, 0x0b, 'FFFFFFFFFFFFFF0000FF01', False, high_update=True),
+        FileTemplate(0x6fad, 'EF.AD',        'TR', None,    4,  10, 0x03, '00000002', False),
+        FileTemplate(0x6fb7, 'EF.ECC',       'LF',    1,    4,  10, 0x01, None, True),
+        FileTemplate(0x6fc4, 'EF.NETPAR',    'TR', None,  128,   5, None, 'FF...FF', False, high_update=True),
+        FileTemplate(0x6fe3, 'EF.EPSLOCI',   'TR', None,   18,   5, 0x1e, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000001', False, ass_serv=[85], high_update=True),
+        FileTemplate(0x6fe4, 'EF.EPSNSC',    'LF',    1,   80,   5, 0x18, 'FF...FF', False, ass_serv=[85], high_update=True),
+    ]
+
+# Section 9.5.1
+class FilesUsimMandatoryV2(ProfileTemplate):
+    created_by_default = True
+    oid = OID.ADF_USIM_by_default_v2
+    files = [
+        FileTemplate(  None, 'ADF.USIM',    'ADF', None, None,  14, None, None, False, ['aid', 'temp_fid', 'pinStatusTemplateDO']),
+        FileTemplate(0x6f07, 'EF.IMSI',      'TR', None,    9,   2, 0x07, None, True, ['size']),
+        FileTemplate(0x6f06, 'EF.ARR',       'LF', None, None,  10, 0x17, None, True, ['nb_rec','size']),
+        FileTemplate(0x6f08, 'EF.Keys',      'TR', None,   33,   5, 0x08, '07FF...FF', False, high_update=True),
+        FileTemplate(0x6f09, 'EF.KeysPS',    'TR', None,   33,   5, 0x09, '07FF...FF', False, high_update=True, pe_name='ef-keysPS'),
+        FileTemplate(0x6f31, 'EF.HPPLMN',    'TR', None,    1,   2, 0x12, '0A', False),
+        FileTemplate(0x6f38, 'EF.UST',       'TR', None,   17,   2, 0x04, None, True),
+        FileTemplate(0x6f3b, 'EF.FDN',       'LF',   20,   26,   8, None, 'FF...FF', False, ass_serv=[2, 89]),
+        FileTemplate(0x6f3c, 'EF.SMS',       'LF',   10,  176,   5, None, '00FF...FF', False, ass_serv=[10]),
+        FileTemplate(0x6f42, 'EF.SMSP',      'LF',    1,   38,   5, None, 'FF...FF', False, ass_serv=[12]),
+        FileTemplate(0x6f43, 'EF.SMSS',      'TR', None,    2,   5, None, 'FFFF', False, ass_serv=[10]),
+        FileTemplate(0x6f46, 'EF.SPN',       'TR', None,   17,  10, None, None, True, ass_serv=[19]),
+        FileTemplate(0x6f56, 'EF.EST',       'TR', None,    1,   8, 0x05, None, True, ass_serv=[2,6,34,35]),
+        FileTemplate(0x6f5b, 'EF.START-HFN', 'TR', None,    6,   5, 0x0f, 'F00000F00000', False, high_update=True),
+        FileTemplate(0x6f5c, 'EF.THRESHOLD', 'TR', None,    3,   2, 0x10, 'FFFFFF', False),
+        FileTemplate(0x6f73, 'EF.PSLOCI',    'TR', None,   14,   5, 0x0c, 'FFFFFFFFFFFFFFFFFFFF0000FF01', False, high_update=True),
+        FileTemplate(0x6f78, 'EF.ACC',       'TR', None,    2,   2, 0x06, None, True),
+        FileTemplate(0x6f7b, 'EF.FPLMN',     'TR', None,   12,   5, 0x0d, 'FF...FF', False),
+        FileTemplate(0x6f7e, 'EF.LOCI',      'TR', None,   11,   5, 0x0b, 'FFFFFFFFFFFFFF0000FF01', False, high_update=True),
+        FileTemplate(0x6fad, 'EF.AD',        'TR', None,    4,  10, 0x03, '00000002', False),
+        FileTemplate(0x6fb7, 'EF.ECC',       'LF',    1,    4,  10, 0x01, None, True),
+        FileTemplate(0x6fc4, 'EF.NETPAR',    'TR', None,  128,   5, None, 'FF...FF', False, high_update=True),
+        FileTemplate(0x6fe3, 'EF.EPSLOCI',   'TR', None,   18,   5, 0x1e, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF000001', False, ass_serv=[85], high_update=True),
+        FileTemplate(0x6fe4, 'EF.EPSNSC',    'LF',    1,   80,   5, 0x18, 'FF...FF', False, ass_serv=[85], high_update=True),
+    ]
+
+
+# Section 9.5.2 v2.3.1
+class FilesUsimOptional(ProfileTemplate):
+    created_by_default = False
+    oid = OID.ADF_USIM_not_by_default
+    files = [
+        FileTemplate(0x6f05, 'EF.LI',        'TR', None,    6,   1, 0x02, 'FF...FF', False),
+        FileTemplate(0x6f37, 'EF.ACMmax',    'TR', None,    3,   5, None, '000000', False, ass_serv=[13], pe_name='ef-acmax'),
+        FileTemplate(0x6f39, 'EF.ACM',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[13], high_update=True),
+        FileTemplate(0x6f3e, 'EF.GID1',      'TR', None,    8,   2, None, None, True, ass_serv=[17]),
+        FileTemplate(0x6f3f, 'EF.GID2',      'TR', None,    8,   2, None, None, True, ass_serv=[18]),
+        FileTemplate(0x6f40, 'EF.MSISDN',    'LF',    1,   24,   2, None, 'FF...FF', False, ass_serv=[21]),
+        FileTemplate(0x6f41, 'EF.PUCT',      'TR', None,    5,   5, None, 'FFFFFF0000', False, ass_serv=[13]),
+        FileTemplate(0x6f45, 'EF.CBMI',      'TR', None,   10,   5, None, 'FF...FF', False, ass_serv=[15]),
+        FileTemplate(0x6f48, 'EF.CBMID',     'TR', None,   10,   2, 0x0e, 'FF...FF', False, ass_serv=[19]),
+        FileTemplate(0x6f49, 'EF.SDN',       'LF',   10,   24,   2, None, 'FF...FF', False, ass_serv=[4,89]),
+        FileTemplate(0x6f4b, 'EF.EXT2',      'LF',   10,   13,   8, None, '00FF...FF', False, ass_serv=[3]),
+        FileTemplate(0x6f4c, 'EF.EXT3',      'LF',   10,   13,   2, None, '00FF...FF', False, ass_serv=[5]),
+        FileTemplate(0x6f50, 'EF.CBMIR',     'TR', None,   20,   5, None, 'FF...FF', False, ass_serv=[16]),
+        FileTemplate(0x6f60, 'EF.PLMNwAcT',  'TR', None,   40,   5, 0x0a, 'FFFFFF0000'*8, False, ass_serv=[20]),
+        FileTemplate(0x6f61, 'EF.OPLMNwAcT', 'TR', None,   40,   2, 0x11, 'FFFFFF0000'*8, False, ass_serv=[42]),
+        FileTemplate(0x6f62, 'EF.HPLMNwAcT', 'TR', None,    5,   2, 0x13, 'FFFFFF0000', False, ass_serv=[43]),
+        FileTemplate(0x6f2c, 'EF.DCK',       'TR', None,   16,   5, None, 'FF...FF', False, ass_serv=[36]),
+        FileTemplate(0x6f32, 'EF.CNL',       'TR', None,   30,   2, None, 'FF...FF', False, ass_serv=[37]),
+        FileTemplate(0x6f47, 'EF.SMSR',      'LF',   10,   30,   5, None, '00FF...FF', False, ass_serv=[11]),
+        FileTemplate(0x6f4d, 'EF.BDN',       'LF',   10,   25,   8, None, 'FF...FF', False, ass_serv=[6]),
+        FileTemplate(0x6f4e, 'EF.EXT5',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[44]),
+        FileTemplate(0x6f4f, 'EF.CCP2',      'LF',    5,   15,   5, 0x16, 'FF...FF', False, ass_serv=[14]),
+        FileTemplate(0x6f55, 'EF.EXT4',      'LF',   10,   13,   8, None, '00FF...FF', False, ass_serv=[7]),
+        FileTemplate(0x6f57, 'EF.ACL',       'TR', None,  101,   8, None, '00FF...FF', False, ass_serv=[35]),
+        FileTemplate(0x6f58, 'EF.CMI',       'LF',   10,   11,   2, None, 'FF...FF', False, ass_serv=[6]),
+        FileTemplate(0x6f80, 'EF.ICI',       'CY',   20,   38,   5, 0x14, 'FF...FF0000000001FFFF', False, ass_serv=[9], high_update=True),
+        FileTemplate(0x6f81, 'EF.OCI',       'CY',   20,   37,   5, 0x15, 'FF...FF00000001FFFF', False, ass_serv=[8], high_update=True),
+        FileTemplate(0x6f82, 'EF.ICT',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[9], high_update=True),
+        FileTemplate(0x6f83, 'EF.OCT',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[8], high_update=True),
+        FileTemplate(0x6fb1, 'EF.VGCS',      'TR', None,   20,   2, None, None, True, ass_serv=[57]),
+        FileTemplate(0x6fb2, 'EF.VGCSS',     'TR', None,    7,   5, None, None, True, ass_serv=[57]),
+        FileTemplate(0x6fb3, 'EF.VBS',       'TR', None,   20,   2, None, None, True, ass_serv=[58]),
+        FileTemplate(0x6fb4, 'EF.VBSS',      'TR', None,    7,   5, None, None, True, ass_serv=[58]), # ARR 2!??
+        FileTemplate(0x6fb5, 'EF.eMLPP',     'TR', None,    2,   2, None, None, True, ass_serv=[24]),
+        FileTemplate(0x6fb6, 'EF.AaeM',      'TR', None,    1,   5, None, '00', False, ass_serv=[25]),
+        FileTemplate(0x6fc3, 'EF.HiddenKey', 'TR', None,    4,   5, None, 'FF...FF', False),
+        FileTemplate(0x6fc5, 'EF.PNN',       'LF',   10,   16,  10, 0x19, None, True, ass_serv=[45]),
+        FileTemplate(0x6fc6, 'EF.OPL',       'LF',    5,    8,  10, 0x1a, None, True, ass_serv=[46]),
+        FileTemplate(0x6fc7, 'EF.MBDN',      'LF',    3,   24,   5, None, None, True, ass_serv=[47]),
+        FileTemplate(0x6fc8, 'EF.EXT6',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[47]),
+        FileTemplate(0x6fc9, 'EF.MBI',       'LF',   10,    5,   5, None, None, True, ass_serv=[47]),
+        FileTemplate(0x6fca, 'EF.MWIS',      'LF',   10,    6,   5, None, '00...00', False, ass_serv=[48], high_update=True),
+        FileTemplate(0x6fcb, 'EF.CFIS',      'LF',   10,   16,   5, None, '0100FF...FF', False, ass_serv=[49]),
+        FileTemplate(0x6fcb, 'EF.EXT7',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[49]),
+        FileTemplate(0x6fcd, 'EF.SPDI',      'TR', None,   17,   2, 0x1b, None, True, ass_serv=[51]),
+        FileTemplate(0x6fce, 'EF.MMSN',      'LF',   10,    6,   5, None, '000000FF...FF', False, ass_serv=[52]),
+        FileTemplate(0x6fcf, 'EF.EXT8',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[53]),
+        FileTemplate(0x6fd0, 'EF.MMSICP',    'TR', None,  100,   2, None, 'FF...FF', False, ass_serv=[52]),
+        FileTemplate(0x6fd1, 'EF.MMSUP',     'LF', None, None,   5, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[52]),
+        FileTemplate(0x6fd2, 'EF.MMSUCP',    'TR', None,  100,   5, None, 'FF...FF', False, ass_serv=[52,55]),
+        FileTemplate(0x6fd3, 'EF.NIA',       'LF',    5,   11,   2, None, 'FF...FF', False, ass_serv=[56]),
+        FileTemplate(0x6fd4, 'EF.VGCSCA',    'TR', None, None,   2, None, '00...00', False, ['size'], ass_serv=[64]),
+        FileTemplate(0x6fd5, 'EF.VBSCA',     'TR', None, None,   2, None, '00...00', False, ['size'], ass_serv=[65]),
+        FileTemplate(0x6fd6, 'EF.GBABP',     'TR', None, None,   5, None, 'FF...FF', False, ['size'], ass_serv=[68]),
+        FileTemplate(0x6fd7, 'EF.MSK',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[69], high_update=True),
+        FileTemplate(0x6fd8, 'EF.MUK',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[69]),
+        FileTemplate(0x6fd9, 'EF.EHPLMN',    'TR', None,   15,   2, 0x1d, 'FF...FF', False, ass_serv=[71]),
+        FileTemplate(0x6fda, 'EF.GBANL',     'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[68]),
+        FileTemplate(0x6fdb, 'EF.EHPLMNPI',  'TR', None,    1,   2, None, '00', False, ass_serv=[71,73]),
+        FileTemplate(0x6fdc, 'EF.LRPLMNSI',  'TR', None,    1,   2, None, '00', False, ass_serv=[74]),
+        FileTemplate(0x6fdd, 'EF.NAFKCA',    'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[68,76]),
+        FileTemplate(0x6fde, 'EF.SPNI',      'TR', None, None,  10, None, '00FF...FF', False, ['size'], ass_serv=[78]),
+        FileTemplate(0x6fdf, 'EF.PNNI',      'LF', None, None,  10, None, '00FF...FF', False, ['nb_rec','size'], ass_serv=[79]),
+        FileTemplate(0x6fe2, 'EF.NCP-IP',    'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[80]),
+        FileTemplate(0x6fe6, 'EF.UFC',       'TR', None,   30,  10, None, '801E60C01E900080040000000000000000F0000000004000000000000080', False),
+        FileTemplate(0x6fe8, 'EF.NASCONFIG', 'TR', None,   18,   2, None, None, True, ass_serv=[96]),
+        FileTemplate(0x6fe7, 'EF.UICCIARI',  'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[95]),
+        FileTemplate(0x6fec, 'EF.PWS',       'TR', None, None,  10, None, None, True, ['size'], ass_serv=[97]),
+        FileTemplate(0x6fed, 'EF.FDNURI',    'LF', None, None,   8, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2,99]),
+        FileTemplate(0x6fee, 'EF.BDNURI',    'LF', None, None,   8, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[6,99]),
+        FileTemplate(0x6fef, 'EF.SDNURI',    'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[4,99]),
+        FileTemplate(0x6ff0, 'EF.IWL',       'LF', None, None,   3, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[102]),
+        FileTemplate(0x6ff1, 'EF.IPS',       'CY', None,    4,  10, None, 'FF...FF', False, ['size'], ass_serv=[102], high_update=True),
+        FileTemplate(0x6ff2, 'EF.IPD',       'LF', None, None,   3, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[102], high_update=True),
+    ]
+
+
+# Section 9.5.2
+class FilesUsimOptionalV2(ProfileTemplate):
+    created_by_default = False
+    oid = OID.ADF_USIM_not_by_default_v2
+    files = [
+        FileTemplate(0x6f05, 'EF.LI',        'TR', None,    6,   1, 0x02, 'FF...FF', False),
+        FileTemplate(0x6f37, 'EF.ACMmax',    'TR', None,    3,   5, None, '000000', False, ass_serv=[13]),
+        FileTemplate(0x6f39, 'EF.ACM',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[13], high_update=True),
+        FileTemplate(0x6f3e, 'EF.GID1',      'TR', None,    8,   2, None, None, True, ass_serv=[17]),
+        FileTemplate(0x6f3f, 'EF.GID2',      'TR', None,    8,   2, None, None, True, ass_serv=[18]),
+        FileTemplate(0x6f40, 'EF.MSISDN',    'LF',    1,   24,   2, None, 'FF...FF', False, ass_serv=[21]),
+        FileTemplate(0x6f41, 'EF.PUCT',      'TR', None,    5,   5, None, 'FFFFFF0000', False, ass_serv=[13]),
+        FileTemplate(0x6f45, 'EF.CBMI',      'TR', None,   10,   5, None, 'FF...FF', False, ass_serv=[15]),
+        FileTemplate(0x6f48, 'EF.CBMID',     'TR', None,   10,   2, 0x0e, 'FF...FF', False, ass_serv=[19]),
+        FileTemplate(0x6f49, 'EF.SDN',       'LF',   10,   24,   2, None, 'FF...FF', False, ass_serv=[4,89]),
+        FileTemplate(0x6f4b, 'EF.EXT2',      'LF',   10,   13,   8, None, '00FF...FF', False, ass_serv=[3]),
+        FileTemplate(0x6f4c, 'EF.EXT3',      'LF',   10,   13,   2, None, '00FF...FF', False, ass_serv=[5]),
+        FileTemplate(0x6f50, 'EF.CBMIR',     'TR', None,   20,   5, None, 'FF...FF', False, ass_serv=[16]),
+        FileTemplate(0x6f60, 'EF.PLMNwAcT',  'TR', None,   40,   5, 0x0a, 'FFFFFF0000'*8, False, ass_serv=[20]),
+        FileTemplate(0x6f61, 'EF.OPLMNwAcT', 'TR', None,   40,   2, 0x11, 'FFFFFF0000'*8, False, ass_serv=[42]),
+        FileTemplate(0x6f62, 'EF.HPLMNwAcT', 'TR', None,    5,   2, 0x13, 'FFFFFF0000', False, ass_serv=[43]),
+        FileTemplate(0x6f2c, 'EF.DCK',       'TR', None,   16,   5, None, 'FF...FF', False, ass_serv=[36]),
+        FileTemplate(0x6f32, 'EF.CNL',       'TR', None,   30,   2, None, 'FF...FF', False, ass_serv=[37]),
+        FileTemplate(0x6f47, 'EF.SMSR',      'LF',   10,   30,   5, None, '00FF...FF', False, ass_serv=[11]),
+        FileTemplate(0x6f4d, 'EF.BDN',       'LF',   10,   25,   8, None, 'FF...FF', False, ass_serv=[6]),
+        FileTemplate(0x6f4e, 'EF.EXT5',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[44]),
+        FileTemplate(0x6f4f, 'EF.CCP2',      'LF',    5,   15,   5, 0x16, 'FF...FF', False, ass_serv=[14]),
+        FileTemplate(0x6f55, 'EF.EXT4',      'LF',   10,   13,   8, None, '00FF...FF', False, ass_serv=[7]),
+        FileTemplate(0x6f57, 'EF.ACL',       'TR', None,  101,   8, None, '00FF...FF', False, ass_serv=[35]),
+        FileTemplate(0x6f58, 'EF.CMI',       'LF',   10,   11,   2, None, 'FF...FF', False, ass_serv=[6]),
+        FileTemplate(0x6f80, 'EF.ICI',       'CY',   20,   38,   5, 0x14, 'FF...FF0000000001FFFF', False, ass_serv=[9], high_update=True),
+        FileTemplate(0x6f81, 'EF.OCI',       'CY',   20,   37,   5, 0x15, 'FF...FF00000001FFFF', False, ass_serv=[8], high_update=True),
+        FileTemplate(0x6f82, 'EF.ICT',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[9], high_update=True),
+        FileTemplate(0x6f83, 'EF.OCT',       'CY',    1,    3,   7, None, '000000', False, ass_serv=[8], high_update=True),
+        FileTemplate(0x6fb1, 'EF.VGCS',      'TR', None,   20,   2, None, None, True, ass_serv=[57]),
+        FileTemplate(0x6fb2, 'EF.VGCSS',     'TR', None,    7,   5, None, None, True, ass_serv=[57]),
+        FileTemplate(0x6fb3, 'EF.VBS',       'TR', None,   20,   2, None, None, True, ass_serv=[58]),
+        FileTemplate(0x6fb4, 'EF.VBSS',      'TR', None,    7,   5, None, None, True, ass_serv=[58]), # ARR 2!??
+        FileTemplate(0x6fb5, 'EF.eMLPP',     'TR', None,    2,   2, None, None, True, ass_serv=[24]),
+        FileTemplate(0x6fb6, 'EF.AaeM',      'TR', None,    1,   5, None, '00', False, ass_serv=[25]),
+        FileTemplate(0x6fc3, 'EF.HiddenKey', 'TR', None,    4,   5, None, 'FF...FF', False),
+        FileTemplate(0x6fc5, 'EF.PNN',       'LF',   10,   16,  10, 0x19, None, True, ass_serv=[45]),
+        FileTemplate(0x6fc6, 'EF.OPL',       'LF',    5,    8,  10, 0x1a, None, True, ass_serv=[46]),
+        FileTemplate(0x6fc7, 'EF.MBDN',      'LF',    3,   24,   5, None, None, True, ass_serv=[47]),
+        FileTemplate(0x6fc8, 'EF.EXT6',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[47]),
+        FileTemplate(0x6fc9, 'EF.MBI',       'LF',   10,    5,   5, None, None, True, ass_serv=[47]),
+        FileTemplate(0x6fca, 'EF.MWIS',      'LF',   10,    6,   5, None, '00...00', False, ass_serv=[48], high_update=True),
+        FileTemplate(0x6fcb, 'EF.CFIS',      'LF',   10,   16,   5, None, '0100FF...FF', False, ass_serv=[49]),
+        FileTemplate(0x6fcb, 'EF.EXT7',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[49]),
+        FileTemplate(0x6fcd, 'EF.SPDI',      'TR', None,   17,   2, 0x1b, None, True, ass_serv=[51]),
+        FileTemplate(0x6fce, 'EF.MMSN',      'LF',   10,    6,   5, None, '000000FF...FF', False, ass_serv=[52]),
+        FileTemplate(0x6fcf, 'EF.EXT8',      'LF',   10,   13,   5, None, '00FF...FF', False, ass_serv=[53]),
+        FileTemplate(0x6fd0, 'EF.MMSICP',    'TR', None,  100,   2, None, 'FF...FF', False, ass_serv=[52]),
+        FileTemplate(0x6fd1, 'EF.MMSUP',     'LF', None, None,   5, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[52]),
+        FileTemplate(0x6fd2, 'EF.MMSUCP',    'TR', None,  100,   5, None, 'FF...FF', False, ass_serv=[52,55]),
+        FileTemplate(0x6fd3, 'EF.NIA',       'LF',    5,   11,   2, None, 'FF...FF', False, ass_serv=[56]),
+        FileTemplate(0x6fd4, 'EF.VGCSCA',    'TR', None, None,   2, None, '00...00', False, ['size'], ass_serv=[64]),
+        FileTemplate(0x6fd5, 'EF.VBSCA',     'TR', None, None,   2, None, '00...00', False, ['size'], ass_serv=[65]),
+        FileTemplate(0x6fd6, 'EF.GBABP',     'TR', None, None,   5, None, 'FF...FF', False, ['size'], ass_serv=[68]),
+        FileTemplate(0x6fd7, 'EF.MSK',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[69], high_update=True),
+        FileTemplate(0x6fd8, 'EF.MUK',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[69]),
+        FileTemplate(0x6fd9, 'EF.EHPLMN',    'TR', None,   15,   2, 0x1d, 'FF...FF', False, ass_serv=[71]),
+        FileTemplate(0x6fda, 'EF.GBANL',     'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[68]),
+        FileTemplate(0x6fdb, 'EF.EHPLMNPI',  'TR', None,    1,   2, None, '00', False, ass_serv=[71,73]),
+        FileTemplate(0x6fdc, 'EF.LRPLMNSI',  'TR', None,    1,   2, None, '00', False, ass_serv=[74]),
+        FileTemplate(0x6fdd, 'EF.NAFKCA',    'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[68,76]),
+        FileTemplate(0x6fde, 'EF.SPNI',      'TR', None, None,  10, None, '00FF...FF', False, ['size'], ass_serv=[78]),
+        FileTemplate(0x6fdf, 'EF.PNNI',      'LF', None, None,  10, None, '00FF...FF', False, ['nb_rec','size'], ass_serv=[79]),
+        FileTemplate(0x6fe2, 'EF.NCP-IP',    'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[80]),
+        FileTemplate(0x6fe6, 'EF.UFC',       'TR', None,   30,  10, None, '801E60C01E900080040000000000000000F0000000004000000000000080', False),
+        FileTemplate(0x6fe8, 'EF.NASCONFIG', 'TR', None,   18,   2, None, None, True, ass_serv=[96]),
+        FileTemplate(0x6fe7, 'EF.UICCIARI',  'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[95]),
+        FileTemplate(0x6fec, 'EF.PWS',       'TR', None, None,  10, None, None, True, ['size'], ass_serv=[97]),
+        FileTemplate(0x6fed, 'EF.FDNURI',    'LF', None, None,   8, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2,99]),
+        FileTemplate(0x6fee, 'EF.BDNURI',    'LF', None, None,   8, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[6,99]),
+        FileTemplate(0x6fef, 'EF.SDNURI',    'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[4,99]),
+        FileTemplate(0x6ff0, 'EF.IWL',       'LF', None, None,   3, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[102]),
+        FileTemplate(0x6ff1, 'EF.IPS',       'CY', None,    4,  10, None, 'FF...FF', False, ['size'], ass_serv=[102], high_update=True),
+        FileTemplate(0x6ff2, 'EF.IPD',       'LF', None, None,   3, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[102], high_update=True),
+        FileTemplate(0x6ff3, 'EF.EPDGID',    'TR', None, None,   2, None, None, True, ['size'], ass_serv=[(106, 107)]),
+        FileTemplate(0x6ff4, 'EF.EPDGSELECTION','TR',None,None,  2, None, None, True, ['size'], ass_serv=[(106, 107)]),
+        FileTemplate(0x6ff5, 'EF.EPDGIDEM',  'TR', None, None,   2, None, None, True, ['size'], ass_serv=[(110, 111)]),
+        FileTemplate(0x6ff6, 'EF.EPDGIDEMSEL','TR',None, None,   2, None, None, True, ['size'], ass_serv=[(110, 111)]),
+        FileTemplate(0x6ff7, 'EF.FromPreferred','TR',None,  1,   2, None, '00', False, ass_serv=[114]),
+        FileTemplate(0x6ff8, 'EF.IMSConfigData','BT',None,None,  2, None, None, True, ['size'], ass_serv=[115]),
+        FileTemplate(0x6ff9, 'EF.3GPPPSDataOff','TR',None,  4,   2, None, None, True, ass_serv=[117]),
+        FileTemplate(0x6ffa, 'EF.3GPPPSDOSLIST','LF',None, None, 2, None, None, True, ['nb_rec','size'], ass_serv=[118]),
+        FileTemplate(0x6ffc, 'EF.XCAPConfigData','BT',None,None, 2, None, None, True, ['size'], ass_serv=[120]),
+        FileTemplate(0x6ffd, 'EF.EARFCNLIST','TR', None, None,  10, None, None, True, ['size'], ass_serv=[121]),
+        FileTemplate(0x6ffd, 'EF.MudMidCfgdata','BT', None, None,2, None, None, True, ['size'], ass_serv=[134]),
+    ]
+
+
+# Section 9.5.3
+class FilesUsimDfPhonebook(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_PHONEBOOK_ADF_USIM
+    files = df_pb_files
+
+
+# Section 9.5.4
+class FilesUsimDfGsmAccess(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_GSM_ACCESS_ADF_USIM
+    files = [
+        FileTemplate(0x5f3b, 'DF.GSM-ACCESS','DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[27]),
+        FileTemplate(0x4f20, 'EF.Kc',        'TR', None,    9,   5, 0x01, 'FF...FF07', False, ass_serv=[27], high_update=True),
+        FileTemplate(0x4f52, 'EF.KcGPRS',    'TR', None,    9,   5, 0x02, 'FF...FF07', False, ass_serv=[27], high_update=True),
+        FileTemplate(0x4f63, 'EF.CPBCCH',    'TR', None,   10,   5, None, 'FF...FF', False, ass_serv=[39], high_update=True),
+        FileTemplate(0x4f64, 'EF.InvScan',   'TR', None,    1,   2, None, '00', False, ass_serv=[40]),
+    ]
+
+
+# Section 9.5.11 v2.3.1
+class FilesUsimDf5GS(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_5GS
+    files = [
+        FileTemplate(0x6fc0, 'DF.5GS',               'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[122,126,127,128,129,130], pe_name='df-df-5gs'),
+        FileTemplate(0x4f01, 'EF.5GS3GPPLOCI',       'TR', None,   20,   5, 0x01, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f02, 'EF.5GSN3GPPLOCI',      'TR', None,   20,   5, 0x02, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f03, 'EF.5GS3GPPNSC',        'LF',    1,   57,   5, 0x03, 'FF...FF', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f04, 'EF.5GSN3GPPNSC',       'LF',    1,   57,   5, 0x04, 'FF...FF', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f05, 'EF.5GAUTHKEYS',        'TR', None,  110,   5, 0x05, None, True, ass_serv=[123], high_update=True),
+        FileTemplate(0x4f06, 'EF.UAC_AIC',           'TR', None,    4,   2, 0x06, None, True, ass_serv=[126]),
+        FileTemplate(0x4f07, 'EF.SUCI_Calc_Info',    'TR', None, None,   2, 0x07, 'FF...FF', False, ass_serv=[124]),
+        FileTemplate(0x4f08, 'EF.OPL5G',             'LF', None,   10,  10, 0x08, 'FF...FF', False, ['nb_rec'], ass_serv=[129]),
+        FileTemplate(0x4f09, 'EF.SUPI_NAI',          'TR', None, None,   2, 0x09, None, True, ['size'], ass_serv=[130]),
+        FileTemplate(0x4f0a, 'EF.Routing_Indicator', 'TR', None,    4,   2, 0x0a, 'F0FFFFFF', False, ass_serv=[124]),
+    ]
+
+
+# Section 9.5.11.2
+class FilesUsimDf5GSv2(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_5GS_v2
+    files = [
+        FileTemplate(0x6fc0, 'DF.5GS',               'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[122,126,127,128,129,130], pe_name='df-df-5gs'),
+        FileTemplate(0x4f01, 'EF.5GS3GPPLOCI',       'TR', None,   20,   5, 0x01, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f02, 'EF.5GSN3GPPLOCI',      'TR', None,   20,   5, 0x02, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f03, 'EF.5GS3GPPNSC',        'LF',    1,   57,   5, 0x03, 'FF...FF', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f04, 'EF.5GSN3GPPNSC',       'LF',    1,   57,   5, 0x04, 'FF...FF', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f05, 'EF.5GAUTHKEYS',        'TR', None,  110,   5, 0x05, None, True, ass_serv=[123], high_update=True),
+        FileTemplate(0x4f06, 'EF.UAC_AIC',           'TR', None,    4,   2, 0x06, None, True, ass_serv=[126]),
+        FileTemplate(0x4f07, 'EF.SUCI_Calc_Info',    'TR', None, None,   2, 0x07, 'FF...FF', False, ass_serv=[124]),
+        FileTemplate(0x4f08, 'EF.OPL5G',             'LF', None,   10,  10, 0x08, 'FF...FF', False, ['nb_rec'], ass_serv=[129]),
+        FileTemplate(0x4f09, 'EF.SUPI_NAI',          'TR', None, None,   2, 0x09, None, True, ['size'], ass_serv=[130]),
+        FileTemplate(0x4f0a, 'EF.Routing_Indicator', 'TR', None,    4,   2, 0x0a, 'F0FFFFFF', False, ass_serv=[124]),
+        FileTemplate(0x4f0b, 'EF.URSP',              'BT', None, None,   2, None, None, False, ass_serv=[132]),
+        FileTemplate(0x4f0c, 'EF.TN3GPPSNN',         'TR', None,    1,   2, 0x0c, '00', False, ass_serv=[135]),
+    ]
+
+
+# Section 9.5.11.3
+class FilesUsimDf5GSv3(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_5GS_v3
+    files = [
+        FileTemplate(0x6fc0, 'DF.5GS',               'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[122,126,127,128,129,130], pe_name='df-df-5gs'),
+        FileTemplate(0x4f01, 'EF.5GS3GPPLOCI',       'TR', None,   20,   5, 0x01, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f02, 'EF.5GSN3GPPLOCI',      'TR', None,   20,   5, 0x02, 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00000001', False, ass_serv=[122], high_update=True),
+        FileTemplate(0x4f03, 'EF.5GS3GPPNSC',        'LF',    2,   62,   5, 0x03, 'FF...FF', False, ass_serv=[122,136], high_update=True),
+        # ^ If Service n°136 is not "available" in EF UST, the Profile Creator shall ensure that these files shall contain one record; otherwise, they shall contain 2 records.
+        FileTemplate(0x4f04, 'EF.5GSN3GPPNSC',       'LF',    2,   62,   5, 0x04, 'FF...FF', False, ass_serv=[122,136], high_update=True),
+        # ^ If Service n°136 is not "available" in EF UST, the Profile Creator shall ensure that these files shall contain one record; otherwise, they shall contain 2 records.
+        FileTemplate(0x4f05, 'EF.5GAUTHKEYS',        'TR', None,  110,   5, 0x05, None, True, ass_serv=[123], high_update=True),
+        FileTemplate(0x4f06, 'EF.UAC_AIC',           'TR', None,    4,   2, 0x06, None, True, ass_serv=[126]),
+        FileTemplate(0x4f07, 'EF.SUCI_Calc_Info',    'TR', None, None,   2, 0x07, 'FF...FF', False, ass_serv=[124]),
+        FileTemplate(0x4f08, 'EF.OPL5G',             'LF', None,   10,  10, 0x08, 'FF...FF', False, ['nb_rec'], ass_serv=[129]),
+        FileTemplate(0x4f09, 'EF.SUPI_NAI',          'TR', None, None,   2, 0x09, None, True, ['size'], ass_serv=[130], pe_name='ef-supinai'),
+        FileTemplate(0x4f0a, 'EF.Routing_Indicator', 'TR', None,    4,   2, 0x0a, 'F0FFFFFF', False, ass_serv=[124]),
+        FileTemplate(0x4f0b, 'EF.URSP',              'BT', None, None,   2, None, None, False, ass_serv=[132]),
+        FileTemplate(0x4f0c, 'EF.TN3GPPSNN',         'TR', None,    1,   2, 0x0c, '00', False, ass_serv=[135]),
+    ]
+
+
+# Section 9.5.12
+class FilesUsimDfSaip(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_SAIP
+    files = [
+        FileTemplate(0x6fd0, 'DF.SAIP',        'DF', None, None,  14, None, None, False, ['pinStatusTemplateDO'], ass_serv=[(124, 125)], pe_name='df-df-saip'),
+        FileTemplate(0x4f01, 'EF.SUCICalcInfo','TR', None, None, 3, None, 'FF..FF', False, ['size'], ass_serv=[125], pe_name='ef-suci-calc-info-usim'),
+    ]
+
+
+# Section 9.6.1
+class FilesIsimMandatory(ProfileTemplate):
+    created_by_default = True
+    oid = OID.ADF_ISIM_by_default
+    files = [
+        FileTemplate(  None, 'ADF.ISIM',      'ADF', None, None,  14, None, None, False, ['aid','temporary_fid','pinStatusTemplateDO']),
+        FileTemplate(0x6f02, 'EF.IMPI',        'TR', None, None,   2, 0x02, None, True, ['size']),
+        FileTemplate(0x6f04, 'EF.IMPU',        'LF',    1, None,   2, 0x04, None, True, ['size']),
+        FileTemplate(0x6f03, 'EF.Domain',      'TR', None, None,   2, 0x05, None, True, ['size']),
+        FileTemplate(0x6f07, 'EF.IST',         'TR', None,   14,   2, 0x07, None, True),
+        FileTemplate(0x6fad, 'EF.AD',          'TR', None,    3,  10, 0x03, '000000', False),
+        FileTemplate(0x6f06, 'EF.ARR',         'LF', None, None,  10, 0x06, None, True, ['nb_rec','size']),
+    ]
+
+
+# Section 9.6.2 v2.3.1
+class FilesIsimOptional(ProfileTemplate):
+    created_by_default = False
+    oid = OID.ADF_ISIM_not_by_default
+    files = [
+        FileTemplate(0x6f09, 'EF.P-CSCF',      'LF',    1, None,   2, None, None, True, ['size'], ass_serv=[1,5]),
+        FileTemplate(0x6f3c, 'EF.SMS',         'LF',   10,  176,   5, None, '00FF...FF', False, ass_serv=[6,8]),
+        FileTemplate(0x6f42, 'EF.SMSP',        'LF',    1,   38,   5, None, 'FF...FF', False, ass_serv=[8]),
+        FileTemplate(0x6f43, 'EF.SMSS',        'TR', None,    2,   5, None, 'FFFF', False, ass_serv=[6,8]),
+        FileTemplate(0x6f47, 'EF.SMSR',        'LF',   10,   30,   5, None, '00FF...FF', False, ass_serv=[7,8]),
+        FileTemplate(0x6fd5, 'EF.GBABP',       'TR', None, None,   5, None, 'FF...FF', False, ['size'], ass_serv=[2]),
+        FileTemplate(0x6fd7, 'EF.GBANL',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2]),
+        FileTemplate(0x6fdd, 'EF.NAFKCA',      'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2,4]),
+        FileTemplate(0x6fe7, 'EF.UICCIARI',    'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[10]),
+    ]
+
+
+# Section 9.6.2
+class FilesIsimOptionalv2(ProfileTemplate):
+    created_by_default = False
+    oid = OID.ADF_ISIM_not_by_default_v2
+    files = [
+        FileTemplate(0x6f09, 'EF.PCSCF',       'LF',    1, None,   2, None, None, True, ['size'], ass_serv=[1,5]),
+        FileTemplate(0x6f3c, 'EF.SMS',         'LF',   10,  176,   5, None, '00FF...FF', False, ass_serv=[6,8]),
+        FileTemplate(0x6f42, 'EF.SMSP',        'LF',    1,   38,   5, None, 'FF...FF', False, ass_serv=[8]),
+        FileTemplate(0x6f43, 'EF.SMSS',        'TR', None,    2,   5, None, 'FFFF', False, ass_serv=[6,8]),
+        FileTemplate(0x6f47, 'EF.SMSR',        'LF',   10,   30,   5, None, '00FF...FF', False, ass_serv=[7,8]),
+        FileTemplate(0x6fd5, 'EF.GBABP',       'TR', None, None,   5, None, 'FF...FF', False, ['size'], ass_serv=[2]),
+        FileTemplate(0x6fd7, 'EF.GBANL',       'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2]),
+        FileTemplate(0x6fdd, 'EF.NAFKCA',      'LF', None, None,   2, None, 'FF...FF', False, ['nb_rec','size'], ass_serv=[2,4]),
+        FileTemplate(0x6fe7, 'EF.UICCIARI',    'LF', None, None,   2, None, None, True, ['nb_rec','size'], ass_serv=[10]),
+        FileTemplate(0x6ff7, 'EF.FromPreferred','TR', None,   1,   2, None, '00', False, ass_serv=[17]),
+        FileTemplate(0x6ff8, 'EF.ImsConfigData','BT', None,None,   2, None, None, True, ['size'], ass_serv=[18]),
+        FileTemplate(0x6ffc, 'EF.XcapconfigData','BT',None,None,   2, None, None, True, ['size'], ass_serv=[19]),
+        FileTemplate(0x6ffa, 'EF.WebRTCURI',   'LF', None, None,   2, None, None, True, ['nb_rec', 'size'], ass_serv=[20]),
+        FileTemplate(0x6ffa, 'EF.MudMidCfgData','BT',None, None,   2, None, None, True, ['size'], ass_serv=[21]),
+    ]
+
+
+# TODO: CSIM
+
+
+# Section 9.8
+class FilesEap(ProfileTemplate):
+    created_by_default = False
+    oid = OID.DF_EAP
+    files = [
+        FileTemplate(  None, 'DF.EAP',         'DF', None, None,  14, None, None, False, ['fid','pinStatusTemplateDO'], ass_serv=[(124, 125)]),
+        FileTemplate(0x4f01, 'EF.EAPKEYS',     'TR', None, None,   2, None, None, True, ['size'], high_update=True),
+        FileTemplate(0x4f02, 'EF.EAPSTATUS',   'TR', None,    1,   2, None, '00', False, high_update=True),
+        FileTemplate(0x4f03, 'EF.PUId',        'TR', None, None,   2, None, None, True, ['size']),
+        FileTemplate(0x4f04, 'EF.Ps',          'TR', None, None,   5, None, 'FF..FF', False, ['size'], high_update=True),
+        FileTemplate(0x4f20, 'EF.CurID',       'TR', None, None,   5, None, 'FF..FF', False, ['size'], high_update=True),
+        FileTemplate(0x4f21, 'EF.RelID',       'TR', None, None,   5, None, None, True, ['size']),
+        FileTemplate(0x4f22, 'EF.Realm',       'TR', None, None,   5, None, None, True, ['size']),
+    ]
+
+
+# Section 9.9 Access Rules Definition
+ARR_DEFINITION = {
+     1: ['8001019000', '800102A406830101950108', '800158A40683010A950108'],
+     2: ['800101A406830101950108', '80015AA40683010A950108'],
+     3: ['80015BA40683010A950108'],
+     4: ['8001019000', '80011A9700', '800140A40683010A950108'],
+     5: ['800103A406830101950108', '800158A40683010A950108'],
+     6: ['800111A406830101950108', '80014AA40683010A950108'],
+     7: ['800103A406830101950108', '800158A40683010A950108', '840132A406830101950108'],
+     8: ['800101A406830101950108', '800102A406830181950108', '800158A40683010A950108'],
+     9: ['8001019000', '80011AA406830101950108', '800140A40683010A950108'],
+    10: ['8001019000', '80015AA40683010A950108'],
+    11: ['8001019000', '800118A40683010A950108', '8001429700'],
+    12: ['800101A406830101950108', '80015A9700'],
+    13: ['800113A406830101950108', '800148A40683010A950108'],
+    14: ['80015EA40683010A950108'],
+}

pySim/esim/saip/validation.py

@@ -0,0 +1,128 @@
+# Implementation of SimAlliance/TCA Interoperable Profile handling
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from pySim.esim.saip import *
+
+class ProfileError(Exception):
+    pass
+
+class ProfileConstraintChecker:
+    def check(self, pes: ProfileElementSequence):
+        for name in dir(self):
+            if name.startswith('check_'):
+                method = getattr(self, name)
+                method(pes)
+
+class CheckBasicStructure(ProfileConstraintChecker):
+    def _is_after_if_exists(self, pes: ProfileElementSequence, opt:str, after:str):
+        opt_pe = pes.get_pe_for_type(opt)
+        if opt_pe:
+            after_pe = pes.get_pe_for_type(after)
+            if not after_pe:
+                raise ProfileError('PE-%s without PE-%s' % (opt.upper(), after.upper()))
+            # FIXME: check order
+
+    def check_start_and_end(self, pes: ProfileElementSequence):
+        if pes.pe_list[0].type != 'header':
+            raise ProfileError('first element is not header')
+        if pes.pe_list[1].type != 'mf':
+            # strictly speaking: permitted, but we don't support MF via GenericFileManagement
+            raise ProfileError('second  element is not mf')
+        if pes.pe_list[-1].type != 'end':
+            raise ProfileError('last element is not end')
+
+    def check_number_of_occurrence(self, pes: ProfileElementSequence):
+        # check for invalid number of occurrences
+        if len(pes.get_pes_for_type('header')) != 1:
+            raise ProfileError('multiple ProfileHeader')
+        if len(pes.get_pes_for_type('mf')) != 1:
+            # strictly speaking: 0 permitted, but we don't support MF via GenericFileManagement
+            raise ProfileError('multiple PE-MF')
+        for tn in ['end', 'cd', 'telecom',
+                   'usim', 'isim', 'csim', 'opt-usim','opt-isim','opt-csim',
+                   'df-saip', 'df-5gs']:
+            if len(pes.get_pes_for_type(tn)) > 1:
+                raise ProfileError('multiple PE-%s' % tn.upper())
+
+    def check_optional_ordering(self, pes: ProfileElementSequence):
+        # ordering and required depenencies
+        self._is_after_if_exists(pes,'opt-usim', 'usim')
+        self._is_after_if_exists(pes,'opt-isim', 'isim')
+        self._is_after_if_exists(pes,'gsm-access', 'usim')
+        self._is_after_if_exists(pes,'phonebook', 'usim')
+        self._is_after_if_exists(pes,'df-5gs', 'usim')
+        self._is_after_if_exists(pes,'df-saip', 'usim')
+        self._is_after_if_exists(pes,'opt-csim', 'csim')
+
+    def check_mandatory_services(self, pes: ProfileElementSequence):
+        """Ensure that the PE for the mandatory services exist."""
+        m_svcs = pes.get_pe_for_type('header').decoded['eUICC-Mandatory-services']
+        if 'usim' in m_svcs and not pes.get_pe_for_type('usim'):
+            raise ProfileError('no PE-USIM for mandatory usim service')
+        if 'isim' in m_svcs and not pes.get_pe_for_type('isim'):
+            raise ProfileError('no PE-ISIM for mandatory isim service')
+        if 'csim' in m_svcs and not pes.get_pe_for_type('csim'):
+            raise ProfileError('no PE-ISIM for mandatory csim service')
+        if 'gba-usim' in m_svcs and not 'usim' in m_svcs:
+            raise ProfileError('gba-usim mandatory, but no usim')
+        if 'gba-isim' in m_svcs and not 'isim' in m_svcs:
+            raise ProfileError('gba-isim mandatory, but no isim')
+        if 'multiple-usim' in m_svcs and not 'usim' in m_svcs:
+            raise ProfileError('multiple-usim mandatory, but no usim')
+        if 'multiple-isim' in m_svcs and not 'isim' in m_svcs:
+            raise ProfileError('multiple-isim mandatory, but no isim')
+        if 'multiple-csim' in m_svcs and not 'csim' in m_svcs:
+            raise ProfileError('multiple-csim mandatory, but no csim')
+        if 'get-identity' in m_svcs and not ('usim' in m_svcs or 'isim' in m_svcs):
+            raise ProfileError('get-identity mandatory, but no usim or isim')
+        if 'profile-a-x25519' in m_svcs and not ('usim' in m_svcs or 'isim' in m_svcs):
+            raise ProfileError('profile-a-x25519 mandatory, but no usim or isim')
+        if 'profile-a-p256' in m_svcs and not ('usim' in m_svcs or 'isim' in m_svcs):
+            raise ProfileError('profile-a-p256 mandatory, but no usim or isim')
+
+FileChoiceList = List[Tuple]
+
+class FileError(ProfileError):
+    pass
+
+class FileConstraintChecker:
+    def check(self, l: FileChoiceList):
+        for name in dir(self):
+            if name.startswith('check_'):
+                method = getattr(self, name)
+                method(l)
+
+class FileCheckBasicStructure(FileConstraintChecker):
+    def check_seqence(self, l: FileChoiceList):
+        by_type = {}
+        for k, v in l:
+            if k in by_type:
+                by_type[k].append(v)
+            else:
+                by_type[k] = [v]
+        if 'doNotCreate' in by_type:
+            if len(l) != 1:
+                raise FileError("doNotCreate must be the only element")
+        if 'fileDescriptor' in by_type:
+            if len(by_type['fileDescriptor']) != 1:
+                raise FileError("fileDescriptor must be the only element")
+            if l[0][0] != 'fileDescriptor':
+                raise FileError("fileDescriptor must be the first element")
+
+    def check_forbidden(self, l: FileChoiceList):
+        """Perform checks for forbidden parameters as described in Section 8.3.3."""

pySim/esim/x509_cert.py

@@ -0,0 +1,210 @@
+# Implementation of X.509 certificate handling in GSMA eSIM
+# as per SGP22 v3.0
+#
+# (C) 2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import requests
+from typing import Optional, List
+
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.hazmat.primitives import hashes
+from cryptography import x509
+from cryptography.hazmat.primitives.serialization import load_pem_private_key, Encoding
+from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature
+
+from pySim.utils import b2h
+
+def check_signed(signed: x509.Certificate, signer: x509.Certificate) -> bool:
+    """Verify if 'signed' certificate was signed using 'signer'."""
+    # this code only works for ECDSA, but this is all we need for GSMA eSIM
+    pkey = signer.public_key()
+    # this 'signed.signature_algorithm_parameters' below requires cryptopgraphy 41.0.0 :(
+    pkey.verify(signed.signature, signed.tbs_certificate_bytes, signed.signature_algorithm_parameters)
+
+def cert_get_subject_key_id(cert: x509.Certificate) -> bytes:
+    """Obtain the subject key identifier of the given cert object (as raw bytes)."""
+    ski_ext = cert.extensions.get_extension_for_class(x509.SubjectKeyIdentifier).value
+    return ski_ext.key_identifier
+
+def cert_get_auth_key_id(cert: x509.Certificate) -> bytes:
+    """Obtain the authority key identifier of the given cert object (as raw bytes)."""
+    aki_ext = cert.extensions.get_extension_for_class(x509.AuthorityKeyIdentifier).value
+    return aki_ext.key_identifier
+
+def cert_policy_has_oid(cert: x509.Certificate, match_oid: x509.ObjectIdentifier) -> bool:
+    """Determine if given certificate has a certificatePolicy extension of matching OID."""
+    for policy_ext in filter(lambda x: isinstance(x.value, x509.CertificatePolicies), cert.extensions):
+        if any(policy.policy_identifier == match_oid for policy in policy_ext.value._policies):
+            return True
+    return False
+
+ID_RSP = "2.23.146.1"
+ID_RSP_CERT_OBJECTS = '.'.join([ID_RSP, '2'])
+ID_RSP_ROLE = '.'.join([ID_RSP_CERT_OBJECTS, '1'])
+
+class oid:
+    id_rspRole_ci = x509.ObjectIdentifier(ID_RSP_ROLE + '.0')
+    id_rspRole_euicc_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.1')
+    id_rspRole_eum_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.2')
+    id_rspRole_dp_tls_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.3')
+    id_rspRole_dp_auth_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.4')
+    id_rspRole_dp_pb_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.5')
+    id_rspRole_ds_tls_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.6')
+    id_rspRole_ds_auth_v2 = x509.ObjectIdentifier(ID_RSP_ROLE + '.7')
+
+class VerifyError(Exception):
+    """An error during certificate verification,"""
+
+class CertificateSet:
+    """A set of certificates consisting of a trusted [self-signed] CA root certificate,
+    and an optional number of intermediate certificates.  Can be used to verify the certificate chain
+    of any given other certificate."""
+    def __init__(self, root_cert: x509.Certificate):
+        check_signed(root_cert, root_cert)
+        # TODO: check other mandatory attributes for CA Cert
+        if not cert_policy_has_oid(root_cert, oid.id_rspRole_ci):
+            raise ValueError("Given root certificate doesn't have rspRole_ci OID")
+        usage_ext = root_cert.extensions.get_extension_for_class(x509.KeyUsage).value
+        if not usage_ext.key_cert_sign:
+            raise ValueError('Given root certificate key usage does not permit signing of certificates')
+        if not usage_ext.crl_sign:
+            raise ValueError('Given root certificate key usage does not permit signing of CRLs')
+        self.root_cert = root_cert
+        self.intermediate_certs = {}
+        self.crl = None
+
+    def load_crl(self, urls: Optional[List[str]] = None):
+        if urls and isinstance(urls, str):
+            urls = [urls]
+        if not urls:
+            # generate list of CRL URLs from root CA certificate
+            crl_ext = self.root_cert.extensions.get_extension_for_class(x509.CRLDistributionPoints).value
+            name_list = [x.full_name for x in crl_ext]
+            merged_list = []
+            for n in name_list:
+                merged_list += n
+            uri_list = filter(lambda x: isinstance(x, x509.UniformResourceIdentifier), merged_list)
+            urls = [x.value for x in uri_list]
+
+        for url in urls:
+            try:
+                crl_bytes = requests.get(url, timeout=10)
+            except requests.exceptions.ConnectionError:
+                continue
+            crl = x509.load_der_x509_crl(crl_bytes)
+            if not crl.is_signature_valid(self.root_cert.public_key()):
+                raise ValueError('Given CRL has incorrect signature and cannot be trusted')
+            # FIXME: various other checks
+            self.crl = crl
+            # FIXME: should we support multiple CRLs? we only support a single CRL right now
+            return
+        # FIXME: report on success/failure
+
+    @property
+    def root_cert_id(self) -> bytes:
+        return cert_get_subject_key_id(self.root_cert)
+
+    def add_intermediate_cert(self, cert: x509.Certificate):
+        """Add a potential intermediate certificate to the CertificateSet."""
+        # TODO: check mandatory attributes for intermediate cert
+        usage_ext = cert.extensions.get_extension_for_class(x509.KeyUsage).value
+        if not usage_ext.key_cert_sign:
+            raise ValueError('Given intermediate certificate key usage does not permit signing of certificates')
+        aki = cert_get_auth_key_id(cert)
+        ski = cert_get_subject_key_id(cert)
+        if aki == ski:
+            raise ValueError('Cannot add self-signed cert as intermediate cert')
+        self.intermediate_certs[ski] = cert
+        # TODO: we could test if this cert verifies against the root, and mark it as pre-verified
+        # so we don't need to verify again and again the chain of intermediate certificates
+
+    def verify_cert_crl(self, cert: x509.Certificate):
+        if not self.crl:
+            # we cannot check if there's no CRL
+            return
+        if self.crl.get_revoked_certificate_by_serial_number(cert.serial_nr):
+            raise VerifyError('Certificate is present in CRL, verification failed')
+
+    def verify_cert_chain(self, cert: x509.Certificate, max_depth: int = 100):
+        """Verify if a given certificate's signature chain can be traced back to the root CA of this
+        CertificateSet."""
+        depth = 1
+        c = cert
+        while True:
+            aki = cert_get_auth_key_id(c)
+            if aki == self.root_cert_id:
+                # last step:
+                check_signed(c, self.root_cert)
+                return
+            parent_cert = self.intermediate_certs.get(aki, None)
+            if not aki:
+                raise VerifyError('Could not find intermediate certificate for AuthKeyId %s' % b2h(aki))
+            check_signed(c, parent_cert)
+            # if we reach here, we passed (no exception raised)
+            c = parent_cert
+            depth += 1
+            if depth > max_depth:
+                raise VerifyError('Maximum depth %u exceeded while verifying certificate chain' % max_depth)
+
+
+def ecdsa_dss_to_tr03111(sig: bytes) -> bytes:
+    """convert from DER format to BSI TR-03111; first get long integers; then convert those to bytes."""
+    r, s = decode_dss_signature(sig)
+    return r.to_bytes(32, 'big') + s.to_bytes(32, 'big')
+
+
+class CertAndPrivkey:
+    """A pair of certificate and private key, as used for ECDSA signing."""
+    def __init__(self, required_policy_oid: Optional[x509.ObjectIdentifier] = None,
+                 cert: Optional[x509.Certificate] = None, priv_key = None):
+        self.required_policy_oid = required_policy_oid
+        self.cert = cert
+        self.priv_key = priv_key
+
+    def cert_from_der_file(self, path: str):
+        with open(path, 'rb') as f:
+            cert = x509.load_der_x509_certificate(f.read())
+        if self.required_policy_oid:
+            # verify it is the right type of certificate (id-rspRole-dp-auth, id-rspRole-dp-auth-v2, etc.)
+            assert cert_policy_has_oid(cert, self.required_policy_oid)
+        self.cert = cert
+
+    def privkey_from_pem_file(self, path: str, password: Optional[str] = None):
+        with open(path, 'rb') as f:
+            self.priv_key = load_pem_private_key(f.read(), password)
+
+    def ecdsa_sign(self, plaintext: bytes) -> bytes:
+        """Sign some input-data using an ECDSA signature compliant with SGP.22,
+        which internally refers to Global Platform 2.2 Annex E, which in turn points
+        to BSI TS-03111 which states "concatengated raw R + S values". """
+        sig = self.priv_key.sign(plaintext, ec.ECDSA(hashes.SHA256()))
+        # convert from DER format to BSI TR-03111; first get long integers; then convert those to bytes
+        return ecdsa_dss_to_tr03111(sig)
+
+    def get_authority_key_identifier(self) -> x509.AuthorityKeyIdentifier:
+        """Return the AuthorityKeyIdentifier X.509 extension of the certificate."""
+        return list(filter(lambda x: isinstance(x.value, x509.AuthorityKeyIdentifier), self.cert.extensions))[0].value
+
+    def get_subject_alt_name(self) -> x509.SubjectAlternativeName:
+        """Return the SubjectAlternativeName X.509 extension of the certificate."""
+        return list(filter(lambda x: isinstance(x.value, x509.SubjectAlternativeName), self.cert.extensions))[0].value
+
+    def get_cert_as_der(self) -> bytes:
+        """Return certificate encoded as DER."""
+        return self.cert.public_bytes(Encoding.DER)
+
+    def get_curve(self) -> ec.EllipticCurve:
+        return self.cert.public_key().public_numbers().curve

pySim/euicc.py

@@ -0,0 +1,528 @@
+# -*- coding: utf-8 -*-
+
+"""
+Various definitions related to GSMA eSIM / eUICC
+
+Related Specs: GSMA SGP.22, GSMA SGP.02, etc.
+"""
+
+# Copyright (C) 2023 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import argparse
+
+from construct import Array, Struct, FlagsEnum, GreedyRange
+from cmd2 import cmd2, CommandSet, with_default_category
+
+from pySim.tlv import *
+from pySim.construct import *
+from pySim.commands import SimCardCommands
+from pySim.utils import Hexstr, SwHexstr, SwMatchstr
+import pySim.global_platform
+
+def compute_eid_checksum(eid) -> str:
+    """Compute and add/replace check digits of an EID value according to GSMA SGP.29 Section 10."""
+    if isinstance(eid, str):
+        if len(eid) == 30:
+            # first pad by 2 digits
+            eid += "00"
+        elif len(eid) == 32:
+            # zero the last two digits
+            eid = eid[:-2] + "00"
+        else:
+            raise ValueError("and EID must be 30 or 32 digits")
+        eid_int = int(eid)
+    elif isinstance(eid, int):
+        eid_int = eid
+        if eid_int % 100:
+            # zero the last two digits
+            eid_int -= eid_int % 100
+    # Using the resulting 32 digits as a decimal integer, compute the remainder of that number on division by
+    # 97, Subtract the remainder from 98, and use the decimal result for the two check digits, if the result
+    # is one digit long, its value SHALL be prefixed by one digit of 0.
+    csum = 98 - (eid_int % 97)
+    eid_int += csum
+    return str(eid_int)
+
+def verify_eid_checksum(eid) -> bool:
+    """Verify the check digits of an EID value according to GSMA SGP.29 Section 10."""
+    # Using the 32 digits as a decimal integer, compute the remainder of that number on division by 97. If the
+    # remainder of the division is 1, the verification is successful; otherwise the EID is invalid.
+    return int(eid) % 97 == 1
+
+class VersionAdapter(Adapter):
+    """convert an EUICC Version (3-int array) to a textual representation."""
+
+    def _decode(self, obj, context, path):
+        return "%u.%u.%u" % (obj[0], obj[1], obj[2])
+
+    def _encode(self, obj, context, path):
+        return [int(x) for x in obj.split('.')]
+
+VersionType = VersionAdapter(Array(3, Int8ub))
+
+# Application Identifiers as defined in GSMA SGP.02 Annex H
+AID_ISD_R           = "A0000005591010FFFFFFFF8900000100"
+AID_ECASD           = "A0000005591010FFFFFFFF8900000200"
+AID_ISD_P_FILE      = "A0000005591010FFFFFFFF8900000D00"
+AID_ISD_P_MODULE    = "A0000005591010FFFFFFFF8900000E00"
+
+class SupportedVersionNumber(BER_TLV_IE, tag=0x82):
+    _construct = GreedyBytes
+
+class IsdrProprietaryApplicationTemplate(BER_TLV_IE, tag=0xe0, nested=[SupportedVersionNumber]):
+    # FIXME: lpaeSupport - what kind of tag  would it have?
+    pass
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1 from pySim/global_platform.py extended with E0
+class FciTemplate(BER_TLV_IE, tag=0x6f, nested=pySim.global_platform.FciTemplateNestedList +
+                                               [IsdrProprietaryApplicationTemplate]):
+    pass
+
+
+# SGP.22 Section 5.7.3: GetEuiccConfiguredAddresses
+class DefaultDpAddress(BER_TLV_IE, tag=0x80):
+    _construct = Utf8Adapter(GreedyBytes)
+class RootDsAddress(BER_TLV_IE, tag=0x81):
+    _construct = Utf8Adapter(GreedyBytes)
+class EuiccConfiguredAddresses(BER_TLV_IE, tag=0xbf3c, nested=[DefaultDpAddress, RootDsAddress]):
+    pass
+
+# SGP.22 Section 5.7.4: SetDefaultDpAddress
+class SetDefaultDpAddrRes(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, undefinedError=127)
+class SetDefaultDpAddress(BER_TLV_IE, tag=0xbf3f, nested=[DefaultDpAddress, SetDefaultDpAddrRes]):
+    pass
+
+# SGP.22 Section 5.7.7: GetEUICCChallenge
+class EuiccChallenge(BER_TLV_IE, tag=0x80):
+    _construct = HexAdapter(Bytes(16))
+class GetEuiccChallenge(BER_TLV_IE, tag=0xbf2e, nested=[EuiccChallenge]):
+    pass
+
+# SGP.22 Section 5.7.8: GetEUICCInfo
+class SVN(BER_TLV_IE, tag=0x82):
+    _construct = VersionType
+class SubjectKeyIdentifier(BER_TLV_IE, tag=0x04):
+    _construct = HexAdapter(GreedyBytes)
+class EuiccCiPkiListForVerification(BER_TLV_IE, tag=0xa9, nested=[SubjectKeyIdentifier]):
+    pass
+class EuiccCiPkiListForSigning(BER_TLV_IE, tag=0xaa, nested=[SubjectKeyIdentifier]):
+    pass
+class EuiccInfo1(BER_TLV_IE, tag=0xbf20, nested=[SVN, EuiccCiPkiListForVerification, EuiccCiPkiListForSigning]):
+    pass
+class ProfileVersion(BER_TLV_IE, tag=0x81):
+    _construct = VersionType
+class EuiccFirmwareVer(BER_TLV_IE, tag=0x83):
+    _construct = VersionType
+class ExtCardResource(BER_TLV_IE, tag=0x84):
+    _construct = HexAdapter(GreedyBytes)
+class UiccCapability(BER_TLV_IE, tag=0x85):
+    _construct = HexAdapter(GreedyBytes) # FIXME
+class TS102241Version(BER_TLV_IE, tag=0x86):
+    _construct = VersionType
+class GlobalPlatformVersion(BER_TLV_IE, tag=0x87):
+    _construct = VersionType
+class RspCapability(BER_TLV_IE, tag=0x88):
+    _construct = HexAdapter(GreedyBytes) # FIXME
+class EuiccCategory(BER_TLV_IE, tag=0x8b):
+    _construct = Enum(Int8ub, other=0, basicEuicc=1, mediumEuicc=2, contactlessEuicc=3)
+class PpVersion(BER_TLV_IE, tag=0x04):
+    _construct = VersionType
+class SsAcreditationNumber(BER_TLV_IE, tag=0x0c):
+    _construct = Utf8Adapter(GreedyBytes)
+class IpaMode(BER_TLV_IE, tag=0x90):    # see SGP.32 v1.0
+    _construct = Enum(Int8ub, ipad=0, ipea=1)
+class IotVersion(BER_TLV_IE, tag=0x80): # see SGP.32 v1.0
+    _construct = VersionType
+class IotVersionSeq(BER_TLV_IE, tag=0xa0, nested=[IotVersion]): # see SGP.32 v1.0
+    pass
+class IotSpecificInfo(BER_TLV_IE, tag=0x94, nested=[IotVersionSeq]): # see SGP.32 v1.0
+    pass
+class EuiccInfo2(BER_TLV_IE, tag=0xbf22, nested=[ProfileVersion, SVN, EuiccFirmwareVer, ExtCardResource,
+                                                 UiccCapability, TS102241Version, GlobalPlatformVersion,
+                                                 RspCapability, EuiccCiPkiListForVerification,
+                                                 EuiccCiPkiListForSigning, EuiccCategory, PpVersion,
+                                                 SsAcreditationNumber, IpaMode, IotSpecificInfo]):
+    pass
+
+# SGP.22 Section 5.7.9: ListNotification
+class ProfileMgmtOperation(BER_TLV_IE, tag=0x81):
+    # we have to ignore the first byte which tells us how many padding bits are used in the last octet
+    _construct = Struct(Byte, "pmo"/FlagsEnum(Byte, install=0x80, enable=0x40, disable=0x20, delete=0x10))
+class ListNotificationReq(BER_TLV_IE, tag=0xbf28, nested=[ProfileMgmtOperation]):
+    pass
+class SeqNumber(BER_TLV_IE, tag=0x80):
+    _construct = GreedyInteger()
+class NotificationAddress(BER_TLV_IE, tag=0x0c):
+    _construct = Utf8Adapter(GreedyBytes)
+class Iccid(BER_TLV_IE, tag=0x5a):
+    _construct = BcdAdapter(GreedyBytes)
+class NotificationMetadata(BER_TLV_IE, tag=0xbf2f, nested=[SeqNumber, ProfileMgmtOperation,
+                                                           NotificationAddress, Iccid]):
+    pass
+class NotificationMetadataList(BER_TLV_IE, tag=0xa0, nested=[NotificationMetadata]):
+    pass
+class ListNotificationsResultError(BER_TLV_IE, tag=0x81):
+    _construct = Enum(Int8ub, undefinedError=127)
+class ListNotificationResp(BER_TLV_IE, tag=0xbf28, nested=[NotificationMetadataList,
+                                                           ListNotificationsResultError]):
+    pass
+
+# SGP.22 Section 5.7.11: RemoveNotificationFromList
+class DeleteNotificationStatus(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, nothingToDelete=1, undefinedError=127)
+class NotificationSentReq(BER_TLV_IE, tag=0xbf30, nested=[SeqNumber]):
+    pass
+class NotificationSentResp(BER_TLV_IE, tag=0xbf30, nested=[DeleteNotificationStatus]):
+    pass
+
+# SGP.22 Section 5.7.12: LoadCRL: FIXME
+class LoadCRL(BER_TLV_IE, tag=0xbf35, nested=[]): # FIXME
+    pass
+
+# SGP.22 Section 5.7.15: GetProfilesInfo
+class TagList(BER_TLV_IE, tag=0x5c):
+    _construct = GreedyRange(Int8ub) # FIXME: tags could be multi-byte
+class ProfileInfoListReq(BER_TLV_IE, tag=0xbf2d, nested=[TagList]): # FIXME: SearchCriteria
+    pass
+class IsdpAid(BER_TLV_IE, tag=0x4f):
+    _construct = HexAdapter(GreedyBytes)
+class ProfileState(BER_TLV_IE, tag=0x9f70):
+    _construct = Enum(Int8ub, disabled=0, enabled=1)
+class ProfileNickname(BER_TLV_IE, tag=0x90):
+    _construct = Utf8Adapter(GreedyBytes)
+class ServiceProviderName(BER_TLV_IE, tag=0x91):
+    _construct = Utf8Adapter(GreedyBytes)
+class ProfileName(BER_TLV_IE, tag=0x92):
+    _construct = Utf8Adapter(GreedyBytes)
+class IconType(BER_TLV_IE, tag=0x93):
+    _construct = Enum(Int8ub, jpg=0, png=1)
+class Icon(BER_TLV_IE, tag=0x94):
+    _construct = GreedyBytes
+class ProfileClass(BER_TLV_IE, tag=0x95):
+    _construct = Enum(Int8ub, test=0, provisioning=1, operational=2)
+class ProfileInfo(BER_TLV_IE, tag=0xe3, nested=[Iccid, IsdpAid, ProfileState, ProfileNickname,
+                                                ServiceProviderName, ProfileName, IconType, Icon,
+                                                ProfileClass]): # FIXME: more IEs
+    pass
+class ProfileInfoSeq(BER_TLV_IE, tag=0xa0, nested=[ProfileInfo]):
+    pass
+class ProfileInfoListError(BER_TLV_IE, tag=0x81):
+    _construct = Enum(Int8ub, incorrectInputValues=1, undefinedError=2)
+class ProfileInfoListResp(BER_TLV_IE, tag=0xbf2d, nested=[ProfileInfoSeq, ProfileInfoListError]):
+    pass
+
+# SGP.22 Section 5.7.16:: EnableProfile
+class RefreshFlag(BER_TLV_IE, tag=0x81): # FIXME
+    _construct = Int8ub # FIXME
+class EnableResult(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, iccidOrAidNotFound=1, profileNotInDisabledState=2,
+                      disallowedByPolicy=3, wrongProfileReenabling=4, catBusy=5, undefinedError=127)
+class ProfileIdentifier(BER_TLV_IE, tag=0xa0, nested=[IsdpAid, Iccid]):
+    pass
+class EnableProfileReq(BER_TLV_IE, tag=0xbf31, nested=[ProfileIdentifier, RefreshFlag]):
+    pass
+class EnableProfileResp(BER_TLV_IE, tag=0xbf31, nested=[EnableResult]):
+    pass
+
+# SGP.22 Section 5.7.17 DisableProfile
+class DisableResult(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, iccidOrAidNotFound=1, profileNotInEnabledState=2,
+                      disallowedByPolicy=3, catBusy=5, undefinedError=127)
+class DisableProfileReq(BER_TLV_IE, tag=0xbf32, nested=[ProfileIdentifier, RefreshFlag]):
+    pass
+class DisableProfileResp(BER_TLV_IE, tag=0xbf32, nested=[DisableResult]):
+    pass
+
+# SGP.22 Section 5.7.18: DeleteProfile
+class DeleteResult(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, iccidOrAidNotFound=1, profileNotInDisabledState=2,
+                      disallowedByPolicy=3, undefinedError=127)
+class DeleteProfileReq(BER_TLV_IE, tag=0xbf33, nested=[IsdpAid, Iccid]):
+    pass
+class DeleteProfileResp(BER_TLV_IE, tag=0xbf33, nested=[DeleteResult]):
+    pass
+
+# SGP.22 Section 5.7.20 GetEID
+class EidValue(BER_TLV_IE, tag=0x5a):
+    _construct = HexAdapter(GreedyBytes)
+class GetEuiccData(BER_TLV_IE, tag=0xbf3e, nested=[TagList, EidValue]):
+    pass
+
+# SGP.22 Section 5.7.21: ES10c SetNickname
+class SnrProfileNickname(BER_TLV_IE, tag=0x8f):
+    _construct = Utf8Adapter(GreedyBytes)
+class SetNicknameReq(BER_TLV_IE, tag=0xbf29, nested=[Iccid, SnrProfileNickname]):
+    pass
+class SetNicknameResult(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, ok=0, iccidNotFound=1, undefinedError=127)
+class SetNicknameResp(BER_TLV_IE, tag=0xbf29, nested=[SetNicknameResult]):
+    pass
+
+# SGP.32 Section 5.9.10: ES10b: GetCerts
+class GetCertsReq(BER_TLV_IE, tag=0xbf56):
+    pass
+class EumCertificate(BER_TLV_IE, tag=0xa5):
+    _construct = GreedyBytes
+class EuiccCertificate(BER_TLV_IE, tag=0xa6):
+    _construct = GreedyBytes
+class GetCertsError(BER_TLV_IE, tag=0x80):
+    _construct = Enum(Int8ub, invalidCiPKId=1, undefinedError=127)
+class GetCertsResp(BER_TLV_IE, tag=0xbf56, nested=[EumCertificate, EuiccCertificate, GetCertsError]):
+    pass
+
+# SGP.32 Section 5.9.18: ES10b: GetEimConfigurationData
+class EimId(BER_TLV_IE, tag=0x80):
+    _construct = Utf8Adapter(GreedyBytes)
+class EimFqdn(BER_TLV_IE, tag=0x81):
+    _construct = Utf8Adapter(GreedyBytes)
+class EimIdType(BER_TLV_IE, tag=0x82):
+    _construct = Enum(Int8ub, eimIdTypeOid=1, eimIdTypeFqdn=2, eimIdTypeProprietary=3)
+class CounterValue(BER_TLV_IE, tag=0x83):
+    _construct = GreedyInteger
+class AssociationToken(BER_TLV_IE, tag=0x84):
+    _construct = GreedyInteger
+class EimSupportedProtocol(BER_TLV_IE, tag=0x87):
+    _construct = Enum(Int8ub, eimRetrieveHttps=0, eimRetrieveCoaps=1, eimInjectHttps=2, eimInjectCoaps=3,
+                      eimProprietary=4)
+# FIXME: eimPublicKeyData, trustedPublicKeyDataTls, euiccCiPKId
+class EimConfigurationData(BER_TLV_IE, tag=0x80, nested=[EimId, EimFqdn, EimIdType, CounterValue,
+                                                         AssociationToken, EimSupportedProtocol]):
+    pass
+class EimConfigurationDataSeq(BER_TLV_IE, tag=0xa0, nested=[EimConfigurationData]):
+    pass
+class GetEimConfigurationData(BER_TLV_IE, tag=0xbf55, nested=[EimConfigurationDataSeq]):
+    pass
+
+class CardApplicationISDR(pySim.global_platform.CardApplicationSD):
+    def __init__(self):
+        super().__init__(name='ADF.ISD-R', aid=AID_ISD_R,
+                         desc='ISD-R (Issuer Security Domain Root) Application')
+        self.adf.decode_select_response = self.decode_select_response
+        self.adf.shell_commands += [self.AddlShellCommands()]
+
+    @staticmethod
+    def store_data(scc: SimCardCommands, tx_do: Hexstr, exp_sw: SwMatchstr ="9000") -> Tuple[Hexstr, SwHexstr]:
+        """Perform STORE DATA according to Table 47+48 in Section 5.7.2 of SGP.22.
+        Only single-block store supported for now."""
+        capdu = '%sE29100%02x%s' % (scc.cla4lchan('80'), len(tx_do)//2, tx_do)
+        return scc.send_apdu_checksw(capdu, exp_sw)
+
+    @staticmethod
+    def store_data_tlv(scc: SimCardCommands, cmd_do, resp_cls, exp_sw: SwMatchstr = '9000'):
+        """Transceive STORE DATA APDU with the card, transparently encoding the command data from TLV
+        and decoding the response data tlv."""
+        if cmd_do:
+            cmd_do_enc = cmd_do.to_tlv()
+            cmd_do_len = len(cmd_do_enc)
+            if cmd_do_len > 255:
+                return ValueError('DO > 255 bytes not supported yet')
+        else:
+            cmd_do_enc = b''
+        (data, _sw) = CardApplicationISDR.store_data(scc, b2h(cmd_do_enc), exp_sw=exp_sw)
+        if data:
+            if resp_cls:
+                resp_do = resp_cls()
+                resp_do.from_tlv(h2b(data))
+                return resp_do
+            else:
+                return data
+        else:
+            return None
+
+    def decode_select_response(self, data_hex: Hexstr) -> object:
+        t = FciTemplate()
+        t.from_tlv(h2b(data_hex))
+        d = t.to_dict()
+        return flatten_dict_lists(d['fci_template'])
+
+    @with_default_category('Application-Specific Commands')
+    class AddlShellCommands(CommandSet):
+
+        es10x_store_data_parser = argparse.ArgumentParser()
+        es10x_store_data_parser.add_argument('TX_DO', help='Hexstring of encoded to-be-transmitted DO')
+
+        @cmd2.with_argparser(es10x_store_data_parser)
+        def do_es10x_store_data(self, opts):
+            """Perform a raw STORE DATA command as defined for the ES10x eUICC interface."""
+            (_data, _sw) = CardApplicationISDR.store_data(self._cmd.lchan.scc, opts.TX_DO)
+
+        def do_get_euicc_configured_addresses(self, _opts):
+            """Perform an ES10a GetEuiccConfiguredAddresses function."""
+            eca = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, EuiccConfiguredAddresses(), EuiccConfiguredAddresses)
+            d = eca.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['euicc_configured_addresses']))
+
+        set_def_dp_addr_parser = argparse.ArgumentParser()
+        set_def_dp_addr_parser.add_argument('DP_ADDRESS', help='Default SM-DP+ address as UTF-8 string')
+
+        @cmd2.with_argparser(set_def_dp_addr_parser)
+        def do_set_default_dp_address(self, opts):
+            """Perform an ES10a SetDefaultDpAddress function."""
+            sdda_cmd = SetDefaultDpAddress(children=[DefaultDpAddress(decoded=opts.DP_ADDRESS)])
+            sdda = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, sdda_cmd, SetDefaultDpAddress)
+            d = sdda.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['set_default_dp_address']))
+
+        def do_get_euicc_challenge(self, _opts):
+            """Perform an ES10b GetEUICCChallenge function."""
+            gec = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, GetEuiccChallenge(), GetEuiccChallenge)
+            d = gec.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['get_euicc_challenge']))
+
+        def do_get_euicc_info1(self, _opts):
+            """Perform an ES10b GetEUICCInfo (1) function."""
+            ei1 = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, EuiccInfo1(), EuiccInfo1)
+            d = ei1.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['euicc_info1']))
+
+        def do_get_euicc_info2(self, _opts):
+            """Perform an ES10b GetEUICCInfo (2) function."""
+            ei2 = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, EuiccInfo2(), EuiccInfo2)
+            d = ei2.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['euicc_info2']))
+
+        def do_list_notification(self, _opts):
+            """Perform an ES10b ListNotification function."""
+            ln = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, ListNotificationReq(), ListNotificationResp)
+            d = ln.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['list_notification_resp']))
+
+        rem_notif_parser = argparse.ArgumentParser()
+        rem_notif_parser.add_argument('SEQ_NR', type=int, help='Sequence Number of the to-be-removed notification')
+
+        @cmd2.with_argparser(rem_notif_parser)
+        def do_remove_notification_from_list(self, opts):
+            """Perform an ES10b RemoveNotificationFromList function."""
+            rn_cmd = NotificationSentReq(children=[SeqNumber(decoded=opts.SEQ_NR)])
+            rn = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, rn_cmd, NotificationSentResp)
+            d = rn.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['notification_sent_resp']))
+
+        def do_get_profiles_info(self, _opts):
+            """Perform an ES10c GetProfilesInfo function."""
+            pi = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, ProfileInfoListReq(), ProfileInfoListResp)
+            d = pi.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['profile_info_list_resp']))
+
+        en_prof_parser = argparse.ArgumentParser()
+        en_prof_grp = en_prof_parser.add_mutually_exclusive_group()
+        en_prof_grp.add_argument('--isdp-aid', help='Profile identified by its ISD-P AID')
+        en_prof_grp.add_argument('--iccid', help='Profile identified by its ICCID')
+        en_prof_parser.add_argument('--refresh-required', action='store_true', help='whether a REFRESH is required')
+
+        @cmd2.with_argparser(en_prof_parser)
+        def do_enable_profile(self, opts):
+            """Perform an ES10c EnableProfile function."""
+            if opts.isdp_aid:
+                p_id = ProfileIdentifier(children=[IsdpAid(decoded=opts.isdp_aid)])
+            if opts.iccid:
+                p_id = ProfileIdentifier(children=[Iccid(decoded=opts.iccid)])
+            ep_cmd_contents = [p_id, RefreshFlag(decoded=opts.refresh_required)]
+            ep_cmd = EnableProfileReq(children=ep_cmd_contents)
+            ep = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, ep_cmd, EnableProfileResp)
+            d = ep.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['enable_profile_resp']))
+
+        dis_prof_parser = argparse.ArgumentParser()
+        dis_prof_grp = dis_prof_parser.add_mutually_exclusive_group()
+        dis_prof_grp.add_argument('--isdp-aid', help='Profile identified by its ISD-P AID')
+        dis_prof_grp.add_argument('--iccid', help='Profile identified by its ICCID')
+        dis_prof_parser.add_argument('--refresh-required', action='store_true', help='whether a REFRESH is required')
+
+        @cmd2.with_argparser(dis_prof_parser)
+        def do_disable_profile(self, opts):
+            """Perform an ES10c DisableProfile function."""
+            if opts.isdp_aid:
+                p_id = ProfileIdentifier(children=[IsdpAid(decoded=opts.isdp_aid)])
+            if opts.iccid:
+                p_id = ProfileIdentifier(children=[Iccid(decoded=opts.iccid)])
+            dp_cmd_contents = [p_id, RefreshFlag(decoded=opts.refresh_required)]
+            dp_cmd = DisableProfileReq(children=dp_cmd_contents)
+            dp = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, dp_cmd, DisableProfileResp)
+            d = dp.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['disable_profile_resp']))
+
+        del_prof_parser = argparse.ArgumentParser()
+        del_prof_grp = del_prof_parser.add_mutually_exclusive_group()
+        del_prof_grp.add_argument('--isdp-aid', help='Profile identified by its ISD-P AID')
+        del_prof_grp.add_argument('--iccid', help='Profile identified by its ICCID')
+
+        @cmd2.with_argparser(del_prof_parser)
+        def do_delete_profile(self, opts):
+            """Perform an ES10c DeleteProfile function."""
+            if opts.isdp_aid:
+                p_id = IsdpAid(decoded=opts.isdp_aid)
+            if opts.iccid:
+                p_id = Iccid(decoded=opts.iccid)
+            dp_cmd_contents = [p_id]
+            dp_cmd = DeleteProfileReq(children=dp_cmd_contents)
+            dp = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, dp_cmd, DeleteProfileResp)
+            d = dp.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['delete_profile_resp']))
+
+
+        def do_get_eid(self, _opts):
+            """Perform an ES10c GetEID function."""
+            (_data, _sw) = CardApplicationISDR.store_data(self._cmd.lchan.scc, 'BF3E035C015A')
+            ged_cmd = GetEuiccData(children=[TagList(decoded=[0x5A])])
+            ged = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, ged_cmd, GetEuiccData)
+            d = ged.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['get_euicc_data']))
+
+        set_nickname_parser = argparse.ArgumentParser()
+        set_nickname_parser.add_argument('ICCID', help='ICCID of the profile whose nickname to set')
+        set_nickname_parser.add_argument('--profile-nickname', help='Nickname of the profile')
+
+        @cmd2.with_argparser(set_nickname_parser)
+        def do_set_nickname(self, opts):
+            """Perform an ES10c SetNickname function."""
+            nickname = opts.profile_nickname or ''
+            sn_cmd_contents = [Iccid(decoded=opts.ICCID), ProfileNickname(decoded=nickname)]
+            sn_cmd = SetNicknameReq(children=sn_cmd_contents)
+            sn = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, sn_cmd, SetNicknameResp)
+            d = sn.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['set_nickname_resp']))
+
+        def do_get_certs(self, _opts):
+            """Perform an ES10c GetCerts() function on an IoT eUICC."""
+            gc = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, GetCertsReq(), GetCertsResp)
+            d = gc.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['get_certficiates_resp']))
+
+        def do_get_eim_configuration_data(self, _opts):
+            """Perform an ES10b GetEimConfigurationData function on an Iot eUICC."""
+            gec = CardApplicationISDR.store_data_tlv(self._cmd.lchan.scc, GetEimConfigurationData(),
+                                                     GetEimConfigurationData)
+            d = gec.to_dict()
+            self._cmd.poutput_json(flatten_dict_lists(d['get_eim_configuration_data']))
+
+class CardApplicationECASD(pySim.global_platform.CardApplicationSD):
+    def decode_select_response(self, data_hex: Hexstr) -> object:
+        t = FciTemplate()
+        t.from_tlv(h2b(data_hex))
+        d = t.to_dict()
+        return flatten_dict_lists(d['fci_template'])
+
+    def __init__(self):
+        super().__init__(name='ADF.ECASD', aid=AID_ECASD,
+                         desc='ECASD (eUICC Controlling Authority Security Domain) Application')
+        self.adf.decode_select_response = self.decode_select_response
+        self.adf.shell_commands += [self.AddlShellCommands()]
+
+    @with_default_category('Application-Specific Commands')
+    class AddlShellCommands(CommandSet):
+        pass

pySim/exceptions.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """ pySim: Exceptions
@@ -6,6 +5,7 @@
 
 #
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -21,13 +21,37 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from __future__ import absolute_import
 
-import exceptions
+class NoCardError(Exception):
+    """No card was found in the reader."""
 
 
-class NoCardError(exceptions.Exception):
-	pass
+class ProtocolError(Exception):
+    """Some kind of protocol level error interfacing with the card."""
 
-class ProtocolError(exceptions.Exception):
-	pass
+
+class ReaderError(Exception):
+    """Some kind of general error with the card reader."""
+
+
+class SwMatchError(Exception):
+    """Raised when an operation specifies an expected SW but the actual SW from
+       the card doesn't match."""
+
+    def __init__(self, sw_actual: str, sw_expected: str, rs=None):
+        """
+        Args:
+                sw_actual : the SW we actually received from the card (4 hex digits)
+                sw_expected : the SW we expected to receive from the card (4 hex digits)
+                rs : interpreter class to convert SW to string
+        """
+        self.sw_actual = sw_actual
+        self.sw_expected = sw_expected
+        self.rs = rs
+
+    def __str__(self):
+        if self.rs and self.rs.lchan[0]:
+            r = self.rs.lchan[0].interpret_sw(self.sw_actual)
+            if r:
+                return "SW match failed! Expected %s and got %s: %s - %s" % (self.sw_expected, self.sw_actual, r[0], r[1])
+        return "SW match failed! Expected %s and got %s." % (self.sw_expected, self.sw_actual)

pySim/global_platform/__init__.py

@@ -0,0 +1,926 @@
+# coding=utf-8
+"""Partial Support for GlobalPLatform Card Spec (currently 2.1.1)
+
+(C) 2022-2024 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import io
+from copy import deepcopy
+from typing import Optional, List, Dict, Tuple
+from construct import Optional as COptional
+from construct import Struct, GreedyRange, FlagsEnum, Int16ub, Int24ub, Padding, Bit, Const
+from Cryptodome.Random import get_random_bytes
+from Cryptodome.Cipher import DES, DES3, AES
+from pySim.global_platform.scp import SCP02, SCP03
+from pySim.construct import *
+from pySim.utils import *
+from pySim.filesystem import *
+from pySim.tlv import *
+from pySim.profile import CardProfile
+
+sw_table = {
+    'Warnings': {
+        '6200': 'Logical Channel already closed',
+        '6283': 'Card Life Cycle State is CARD_LOCKED',
+        '6310': 'More data available',
+    },
+    'Execution errors': {
+        '6400': 'No specific diagnosis',
+        '6581': 'Memory failure',
+    },
+    'Checking errors': {
+        '6700': 'Wrong length in Lc',
+    },
+    'Functions in CLA not supported': {
+        '6881': 'Logical channel not supported or active',
+        '6882': 'Secure messaging not supported',
+    },
+    'Command not allowed': {
+        '6982': 'Security Status not satisfied',
+        '6985': 'Conditions of use not satisfied',
+    },
+    'Wrong parameters': {
+        '6a80': 'Incorrect values in command data',
+        '6a81': 'Function not supported e.g. card Life Cycle State is CARD_LOCKED',
+        '6a82': 'Application not found',
+        '6a84': 'Not enough memory space',
+        '6a86': 'Incorrect P1 P2',
+        '6a88': 'Referenced data not found',
+    },
+    'GlobalPlatform': {
+        '6d00': 'Invalid instruction',
+        '6e00': 'Invalid class',
+    },
+    'Application errors': {
+        '9484': 'Algorithm not supported',
+        '9485': 'Invalid key check value',
+    },
+}
+
+# GlobalPlatform 2.1.1 Section 9.1.6
+KeyType = Enum(Byte,    des=0x80,
+                        tls_psk=0x85,                           # v2.3.1 Section 11.1.8
+                        aes=0x88,                               # v2.3.1 Section 11.1.8
+                        hmac_sha1=0x90,                         # v2.3.1 Section 11.1.8
+                        hmac_sha1_160=0x91,                     # v2.3.1 Section 11.1.8
+                        rsa_public_exponent_e_cleartex=0xA0,
+                        rsa_modulus_n_cleartext=0xA1,
+                        rsa_modulus_n=0xA2,
+                        rsa_private_exponent_d=0xA3,
+                        rsa_chines_remainder_p=0xA4,
+                        rsa_chines_remainder_q=0xA5,
+                        rsa_chines_remainder_pq=0xA6,
+                        rsa_chines_remainder_dpi=0xA7,
+                        rsa_chines_remainder_dqi=0xA8,
+                        ecc_public_key=0xB0,                    # v2.3.1 Section 11.1.8
+                        ecc_private_key=0xB1,                   # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_p=0xB2,             # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_a=0xB3,             # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_b=0xB4,             # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_g=0xB5,             # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_n=0xB6,             # v2.3.1 Section 11.1.8
+                        ecc_field_parameter_k=0xB7,             # v2.3.1 Section 11.1.8
+                        ecc_key_parameters_reference=0xF0,      # v2.3.1 Section 11.1.8
+                        not_available=0xff)
+
+# GlobalPlatform 2.3 Section 11.10.2.1 Table 11-86
+SetStatusScope = Enum(Byte, isd=0x80, app_or_ssd=0x40, isd_and_assoc_apps=0xc0)
+
+# GlobalPlatform 2.3 section 11.1.1
+CLifeCycleState = Enum(Byte, loaded=0x01, installed=0x03, selectable=0x07, personalized=0x0f, locked=0x83)
+
+# GlobalPlatform 2.1.1 Section 9.3.3.1
+class KeyInformationData(BER_TLV_IE, tag=0xc0):
+    _test_de_encode = [
+        ( 'c00401708010', {"key_identifier": 1, "key_version_number": 112, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00402708010', {"key_identifier": 2, "key_version_number": 112, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00403708010', {"key_identifier": 3, "key_version_number": 112, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00401018010', {"key_identifier": 1, "key_version_number": 1, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00402018010', {"key_identifier": 2, "key_version_number": 1, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00403018010', {"key_identifier": 3, "key_version_number": 1, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00401028010', {"key_identifier": 1, "key_version_number": 2, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00402028010', {"key_identifier": 2, "key_version_number": 2, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00403038010', {"key_identifier": 3, "key_version_number": 3, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00401038010', {"key_identifier": 1, "key_version_number": 3, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00402038010', {"key_identifier": 2, "key_version_number": 3, "key_types": [ {"length": 16, "type": "des"} ]} ),
+        ( 'c00402038810', {"key_identifier": 2, "key_version_number": 3, "key_types": [ {"length": 16, "type": "aes"} ]} ),
+    ]
+    KeyTypeLen = Struct('type'/KeyType, 'length'/Int8ub)
+    _construct = Struct('key_identifier'/Byte, 'key_version_number'/Byte,
+                        'key_types'/GreedyRange(KeyTypeLen))
+class KeyInformation(BER_TLV_IE, tag=0xe0, nested=[KeyInformationData]):
+    pass
+
+# GP v2.3 11.1.9
+KeyUsageQualifier = FlagsEnum(StripTrailerAdapter(GreedyBytes, 2),
+                              verification_encryption=0x8000,
+                              computation_decipherment=0x4000,
+                              sm_response=0x2000,
+                              sm_command=0x1000,
+                              confidentiality=0x0800,
+                              crypto_checksum=0x0400,
+                              digital_signature=0x0200,
+                              crypto_authorization=0x0100,
+                              key_agreement=0x0080)
+
+# GP v2.3 11.1.10
+KeyAccess = Enum(Byte, sd_and_any_assoc_app=0x00, sd_only=0x01, any_assoc_app_but_not_sd=0x02,
+                 not_available=0xff)
+
+class KeyLoading:
+    # Global Platform Specification v2.3 Section 11.11.4.2.2.3 DGIs for the CC Private Key
+
+    class KeyUsageQualifier(BER_TLV_IE, tag=0x95):
+        _construct = KeyUsageQualifier
+
+    class KeyAccess(BER_TLV_IE, tag=0x96):
+        _construct = KeyAccess
+
+    class KeyType(BER_TLV_IE, tag=0x80):
+        _construct = KeyType
+
+    class KeyLength(BER_TLV_IE, tag=0x81):
+        _construct = GreedyInteger()
+
+    class KeyIdentifier(BER_TLV_IE, tag=0x82):
+        _construct = Int8ub
+
+    class KeyVersionNumber(BER_TLV_IE, tag=0x83):
+        _construct = Int8ub
+
+    class KeyParameterReferenceValue(BER_TLV_IE, tag=0x85):
+        _construct = Enum(Byte, secp256r1=0x00, secp384r1=0x01, secp521r1=0x02, brainpoolP256r1=0x03,
+                          brainpoolP256t1=0x04, brainpoolP384r1=0x05, brainpoolP384t1=0x06,
+                          brainpoolP512r1=0x07, brainpoolP512t1=0x08)
+
+    # pylint: disable=undefined-variable
+    class ControlReferenceTemplate(BER_TLV_IE, tag=0xb9,
+                                   nested=[KeyUsageQualifier,
+                                           KeyAccess,
+                                           KeyType,
+                                           KeyLength,
+                                           KeyIdentifier,
+                                           KeyVersionNumber,
+                                           KeyParameterReferenceValue]):
+        pass
+
+    # Table 11-103
+    class EccPublicKey(DGI_TLV_IE, tag=0x0036):
+        _construct = GreedyBytes
+
+    # Table 11-105
+    class EccPrivateKey(DGI_TLV_IE, tag=0x8137):
+        _construct = GreedyBytes
+
+    # Global Platform Specification v2.3 Section 11.11.4 / Table 11-91
+    class KeyControlReferenceTemplate(DGI_TLV_IE, tag=0x00b9, nested=[ControlReferenceTemplate]):
+        pass
+
+
+# GlobalPlatform v2.3.1 Section H.4 / Table H-6
+class ScpType(BER_TLV_IE, tag=0x80):
+    _construct = HexAdapter(Byte)
+class ListOfSupportedOptions(BER_TLV_IE, tag=0x81):
+    _construct = GreedyBytes
+class SupportedKeysForScp03(BER_TLV_IE, tag=0x82):
+    _construct = FlagsEnum(Byte, aes128=0x01, aes192=0x02, aes256=0x04)
+class SupportedTlsCipherSuitesForScp81(BER_TLV_IE, tag=0x83):
+    _consuruct = GreedyRange(Int16ub)
+class ScpInformation(BER_TLV_IE, tag=0xa0, nested=[ScpType, ListOfSupportedOptions, SupportedKeysForScp03,
+                                                   SupportedTlsCipherSuitesForScp81]):
+    pass
+class PrivilegesAvailableSSD(BER_TLV_IE, tag=0x81):
+    pass
+class PrivilegesAvailableApplication(BER_TLV_IE, tag=0x82):
+    pass
+class SupportedLFDBHAlgorithms(BER_TLV_IE, tag=0x83):
+    pass
+# GlobalPlatform Card Specification v2.3 / Table H-8
+class CiphersForLFDBEncryption(BER_TLV_IE, tag=0x84):
+    _construct = Enum(Byte, tripledes16=0x01, aes128=0x02, aes192=0x04, aes256=0x08,
+                      icv_supported_for_lfdb=0x80)
+CipherSuitesForSignatures = FlagsEnum(StripTrailerAdapter(GreedyBytes, 2),
+                                      rsa1024_pkcsv15_sha1=0x0100,
+                                      rsa_gt1024_pss_sha256=0x0200,
+                                      single_des_plus_final_triple_des_mac_16b=0x0400,
+                                      cmac_aes128=0x0800, cmac_aes192=0x1000, cmac_aes256=0x2000,
+                                      ecdsa_ecc256_sha256=0x4000, ecdsa_ecc384_sha384=0x8000,
+                                      ecdsa_ecc512_sha512=0x0001, ecdsa_ecc_521_sha512=0x0002)
+class CiphersForTokens(BER_TLV_IE, tag=0x85):
+    _construct = CipherSuitesForSignatures
+class CiphersForReceipts(BER_TLV_IE, tag=0x86):
+    _construct = CipherSuitesForSignatures
+class CiphersForDAPs(BER_TLV_IE, tag=0x87):
+    _construct = CipherSuitesForSignatures
+class KeyParameterReferenceList(BER_TLV_IE, tag=0x88, nested=[KeyLoading.KeyParameterReferenceValue]):
+    pass
+class CardCapabilityInformation(BER_TLV_IE, tag=0x67, nested=[ScpInformation, PrivilegesAvailableSSD,
+                                                              PrivilegesAvailableApplication,
+                                                              SupportedLFDBHAlgorithms,
+                                                              CiphersForLFDBEncryption, CiphersForTokens,
+                                                              CiphersForReceipts, CiphersForDAPs,
+                                                              KeyParameterReferenceList]):
+    pass
+
+class CurrentSecurityLevel(BER_TLV_IE, tag=0xd3):
+    _construct = Int8ub
+
+# GlobalPlatform v2.3.1 Section 11.3.3.1.3
+class ApplicationAID(BER_TLV_IE, tag=0x4f):
+    _construct = HexAdapter(GreedyBytes)
+class ApplicationTemplate(BER_TLV_IE, tag=0x61, ntested=[ApplicationAID]):
+    pass
+class ListOfApplications(BER_TLV_IE, tag=0x2f00, nested=[ApplicationTemplate]):
+    pass
+
+# GlobalPlatform v2.3.1 Section 11.3.3.1.2 + TS 102 226
+class NumberOFInstalledApp(BER_TLV_IE, tag=0x81):
+    _construct = GreedyInteger()
+class FreeNonVolatileMemory(BER_TLV_IE, tag=0x82):
+    _construct = GreedyInteger()
+class FreeVolatileMemory(BER_TLV_IE, tag=0x83):
+    _construct = GreedyInteger()
+class ExtendedCardResourcesInfo(BER_TLV_IE, tag=0xff21, nested=[NumberOFInstalledApp, FreeNonVolatileMemory,
+                                                                FreeVolatileMemory]):
+    pass
+
+# GlobalPlatform v2.3.1 Section 7.4.2.4 + GP SPDM
+class SecurityDomainManagerURL(BER_TLV_IE, tag=0x5f50):
+    pass
+
+
+# card data sample, returned in response to GET DATA (80ca006600):
+# 66 31
+#    73 2f
+#        06 07
+#            2a864886fc6b01
+#        60 0c
+#            06 0a
+#                2a864886fc6b02020101
+#        63 09
+#            06 07
+#                2a864886fc6b03
+#        64 0b
+#            06 09
+#                2a864886fc6b040215
+
+# GlobalPlatform 2.1.1 Table F-1
+class ObjectIdentifier(BER_TLV_IE, tag=0x06):
+    _construct = GreedyBytes
+class CardManagementTypeAndVersion(BER_TLV_IE, tag=0x60, nested=[ObjectIdentifier]):
+    pass
+class CardIdentificationScheme(BER_TLV_IE, tag=0x63, nested=[ObjectIdentifier]):
+    pass
+class SecureChannelProtocolOfISD(BER_TLV_IE, tag=0x64, nested=[ObjectIdentifier]):
+    pass
+class CardConfigurationDetails(BER_TLV_IE, tag=0x65):
+    _construct = GreedyBytes
+class CardChipDetails(BER_TLV_IE, tag=0x66):
+    _construct = GreedyBytes
+class CardRecognitionData(BER_TLV_IE, tag=0x73, nested=[ObjectIdentifier,
+                                                        CardManagementTypeAndVersion,
+                                                        CardIdentificationScheme,
+                                                        SecureChannelProtocolOfISD,
+                                                        CardConfigurationDetails,
+                                                        CardChipDetails]):
+    pass
+class CardData(BER_TLV_IE, tag=0x66, nested=[CardRecognitionData]):
+    pass
+
+# GlobalPlatform 2.1.1 Table F-2
+class SecureChannelProtocolOfSelectedSD(BER_TLV_IE, tag=0x64, nested=[ObjectIdentifier]):
+    pass
+class SecurityDomainMgmtData(BER_TLV_IE, tag=0x73, nested=[CardManagementTypeAndVersion,
+                                                           CardIdentificationScheme,
+                                                           SecureChannelProtocolOfSelectedSD,
+                                                           CardConfigurationDetails,
+                                                           CardChipDetails]):
+    pass
+
+# GlobalPlatform 2.1.1 Section 9.1.1
+IsdLifeCycleState = Enum(Byte, op_ready=0x01, initialized=0x07, secured=0x0f,
+                         card_locked = 0x7f, terminated=0xff)
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class ApplicationID(BER_TLV_IE, tag=0x84):
+    _construct = GreedyBytes
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class SecurityDomainManagementData(BER_TLV_IE, tag=0x73):
+    _construct = GreedyBytes
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class ApplicationProductionLifeCycleData(BER_TLV_IE, tag=0x9f6e):
+    _construct = GreedyBytes
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class MaximumLengthOfDataFieldInCommandMessage(BER_TLV_IE, tag=0x9f65):
+    _construct = GreedyInteger()
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class ProprietaryData(BER_TLV_IE, tag=0xA5, nested=[SecurityDomainManagementData,
+                                                    ApplicationProductionLifeCycleData,
+                                                    MaximumLengthOfDataFieldInCommandMessage]):
+    pass
+
+# explicitly define this list and give it a name so pySim.euicc can reference it
+FciTemplateNestedList = [ApplicationID, SecurityDomainManagementData,
+                         ApplicationProductionLifeCycleData,
+                         MaximumLengthOfDataFieldInCommandMessage,
+                         ProprietaryData]
+
+# GlobalPlatform 2.1.1 Section 9.9.3.1
+class FciTemplate(BER_TLV_IE, tag=0x6f, nested=FciTemplateNestedList):
+    pass
+
+class IssuerIdentificationNumber(BER_TLV_IE, tag=0x42):
+    _construct = HexAdapter(GreedyBytes)
+
+class CardImageNumber(BER_TLV_IE, tag=0x45):
+    _construct = HexAdapter(GreedyBytes)
+
+class SequenceCounterOfDefaultKvn(BER_TLV_IE, tag=0xc1):
+    _construct = GreedyInteger()
+
+class ConfirmationCounter(BER_TLV_IE, tag=0xc2):
+    _construct = GreedyInteger()
+
+# Collection of all the data objects we can get from GET DATA
+class DataCollection(TLV_IE_Collection, nested=[IssuerIdentificationNumber,
+                                                CardImageNumber,
+                                                CardData,
+                                                KeyInformation,
+                                                SequenceCounterOfDefaultKvn,
+                                                ConfirmationCounter,
+                                                # v2.3.1
+                                                CardCapabilityInformation,
+                                                CurrentSecurityLevel,
+                                                ListOfApplications,
+                                                ExtendedCardResourcesInfo,
+                                                SecurityDomainManagerURL]):
+    pass
+
+def decode_select_response(resp_hex: str) -> object:
+    t = FciTemplate()
+    t.from_tlv(h2b(resp_hex))
+    d = t.to_dict()
+    return flatten_dict_lists(d['fci_template'])
+
+# 11.4.2.1
+StatusSubset = Enum(Byte, isd=0x80, applications=0x40, files=0x20, files_and_modules=0x10)
+
+
+# Section 11.4.3.1 Table 11-36
+class LifeCycleState(BER_TLV_IE, tag=0x9f70):
+    _construct = CLifeCycleState
+
+# Section 11.4.3.1 Table 11-36 + Section 11.1.2
+class Privileges(BER_TLV_IE, tag=0xc5):
+    # we only support 3-byte encoding. Can't use StripTrailerAdapter as length==2 is not permitted. sigh.
+    _construct = FlagsEnum(Int24ub,
+                           security_domain=0x800000, dap_verification=0x400000,
+                           delegated_management=0x200000, card_lock=0x100000, card_terminate=0x080000,
+                           card_reset=0x040000, cvm_management=0x020000,
+                           mandated_dap_verification=0x010000,
+                           trusted_path=0x8000, authorized_management=0x4000,
+                           token_management=0x2000, global_delete=0x1000, global_lock=0x0800,
+                           global_registry=0x0400, final_application=0x0200, global_service=0x0100,
+                           receipt_generation=0x80, ciphered_load_file_data_block=0x40,
+                           contactless_activation=0x20, contactless_self_activation=0x10)
+
+# Section 11.4.3.1 Table 11-36 + Section 11.1.7
+class ImplicitSelectionParameter(BER_TLV_IE, tag=0xcf):
+    _construct = BitStruct('contactless_io'/Flag,
+                           'contact_io'/Flag,
+                           '_rfu'/Flag,
+                           'logical_channel_number'/BitsInteger(5))
+
+# Section 11.4.3.1 Table 11-36
+class ExecutableLoadFileAID(BER_TLV_IE, tag=0xc4):
+    _construct = HexAdapter(GreedyBytes)
+
+# Section 11.4.3.1 Table 11-36
+class ExecutableLoadFileVersionNumber(BER_TLV_IE, tag=0xce):
+    # Note: the Executable Load File Version Number format and contents are beyond the scope of this
+    # specification. It shall consist of the version information contained in the original Load File: on a
+    # Java Card based card, this version number represents the major and minor version attributes of the
+    # original Load File Data Block.
+    _construct = HexAdapter(GreedyBytes)
+
+# Section 11.4.3.1 Table 11-36
+class ExecutableModuleAID(BER_TLV_IE, tag=0x84):
+    _construct = HexAdapter(GreedyBytes)
+
+# Section 11.4.3.1 Table 11-36
+class AssociatedSecurityDomainAID(BER_TLV_IE, tag=0xcc):
+    _construct = HexAdapter(GreedyBytes)
+
+# Section 11.4.3.1 Table 11-36
+class GpRegistryRelatedData(BER_TLV_IE, tag=0xe3, nested=[ApplicationAID, LifeCycleState, Privileges,
+                                                          ImplicitSelectionParameter, ExecutableLoadFileAID,
+                                                          ExecutableLoadFileVersionNumber,
+                                                          ExecutableModuleAID, AssociatedSecurityDomainAID]):
+    pass
+
+
+# Section 11.6.2.3 / Table 11-58
+class SecurityDomainAid(BER_TLV_IE, tag=0x4f):
+    _construct = GreedyBytes
+class LoadFileDataBlockSignature(BER_TLV_IE, tag=0xc3):
+    _construct = GreedyBytes
+class DapBlock(BER_TLV_IE, tag=0xe2, nested=[SecurityDomainAid, LoadFileDataBlockSignature]):
+    pass
+class LoadFileDataBlock(BER_TLV_IE, tag=0xc4):
+    _construct = GreedyBytes
+class Icv(BER_TLV_IE, tag=0xd3):
+    _construct = GreedyBytes
+class CipheredLoadFileDataBlock(BER_TLV_IE, tag=0xd4):
+    _construct = GreedyBytes
+class LoadFile(TLV_IE_Collection, nested=[DapBlock, LoadFileDataBlock, Icv, CipheredLoadFileDataBlock]):
+    pass
+
+# Application Dedicated File of a Security Domain
+class ADF_SD(CardADF):
+    StoreData = BitStruct('last_block'/Flag,
+                          'encryption'/Enum(BitsInteger(2), none=0, application_dependent=1, rfu=2, encrypted=3),
+                          'structure'/Enum(BitsInteger(2), none=0, dgi=1, ber_tlv=2, rfu=3),
+                          '_pad'/Padding(2),
+                          'response'/Enum(Bit, not_expected=0, may_be_returned=1))
+
+    def __init__(self, aid: str, name: str, desc: str):
+        super().__init__(aid=aid, fid=None, sfid=None, name=name, desc=desc)
+        self.shell_commands += [self.AddlShellCommands()]
+
+    def decode_select_response(self, data_hex: str) -> object:
+        return decode_select_response(data_hex)
+
+    @with_default_category('Application-Specific Commands')
+    class AddlShellCommands(CommandSet):
+        get_data_parser = argparse.ArgumentParser()
+        get_data_parser.add_argument('data_object_name', type=str,
+            help='Name of the data object to be retrieved from the card')
+
+        @cmd2.with_argparser(get_data_parser)
+        def do_get_data(self, opts):
+            """Perform the GlobalPlatform GET DATA command in order to obtain some card-specific data."""
+            tlv_cls_name = opts.data_object_name
+            try:
+                tlv_cls = DataCollection().members_by_name[tlv_cls_name]
+            except KeyError:
+                do_names = [camel_to_snake(str(x.__name__)) for x in DataCollection.possible_nested]
+                self._cmd.poutput('Unknown data object "%s", available options: %s' % (tlv_cls_name,
+                                                                                       do_names))
+                return
+            (data, _sw) = self._cmd.lchan.scc.get_data(cla=0x80, tag=tlv_cls.tag)
+            ie = tlv_cls()
+            ie.from_tlv(h2b(data))
+            self._cmd.poutput_json(ie.to_dict())
+
+        def complete_get_data(self, text, line, begidx, endidx) -> List[str]:
+            data_dict = {camel_to_snake(str(x.__name__)): x for x in DataCollection.possible_nested}
+            index_dict = {1: data_dict}
+            return self._cmd.index_based_complete(text, line, begidx, endidx, index_dict=index_dict)
+
+        store_data_parser = argparse.ArgumentParser()
+        store_data_parser.add_argument('--data-structure', type=str, choices=['none','dgi','ber_tlv','rfu'], default='none')
+        store_data_parser.add_argument('--encryption', type=str, choices=['none','application_dependent', 'rfu', 'encrypted'], default='none')
+        store_data_parser.add_argument('--response', type=str, choices=['not_expected','may_be_returned'], default='not_expected')
+        store_data_parser.add_argument('DATA', type=is_hexstr)
+
+        @cmd2.with_argparser(store_data_parser)
+        def do_store_data(self, opts):
+            """Perform the GlobalPlatform GET DATA command in order to store some card-specific data.
+            See GlobalPlatform CardSpecification v2.3Section 11.11 for details."""
+            response_permitted = opts.response == 'may_be_returned'
+            self.store_data(h2b(opts.DATA), opts.data_structure, opts.encryption, response_permitted)
+
+        def store_data(self, data: bytes, structure:str = 'none', encryption:str = 'none', response_permitted: bool = False) -> bytes:
+            """Perform the GlobalPlatform GET DATA command in order to store some card-specific data.
+            See GlobalPlatform CardSpecification v2.3Section 11.11 for details."""
+            max_cmd_len = self._cmd.lchan.scc.max_cmd_len
+            # Table 11-89 of GP Card Specification v2.3
+            remainder = data
+            block_nr = 0
+            response = ''
+            while len(remainder):
+                chunk = remainder[:max_cmd_len]
+                remainder = remainder[max_cmd_len:]
+                p1b = build_construct(ADF_SD.StoreData,
+                                      {'last_block': len(remainder) == 0, 'encryption': encryption,
+                                       'structure': structure, 'response': response_permitted})
+                hdr = "80E2%02x%02x%02x" % (p1b[0], block_nr, len(chunk))
+                data, _sw = self._cmd.lchan.scc.send_apdu_checksw(hdr + b2h(chunk))
+                block_nr += 1
+                response += data
+            return data
+
+        put_key_parser = argparse.ArgumentParser()
+        put_key_parser.add_argument('--old-key-version-nr', type=auto_uint8, default=0, help='Old Key Version Number')
+        put_key_parser.add_argument('--key-version-nr', type=auto_uint8, required=True, help='Key Version Number')
+        put_key_parser.add_argument('--key-id', type=auto_uint7, required=True, help='Key Identifier (base)')
+        put_key_parser.add_argument('--key-type', choices=KeyType.ksymapping.values(), action='append', required=True, help='Key Type')
+        put_key_parser.add_argument('--key-data', type=is_hexstr, action='append', required=True, help='Key Data Block')
+        put_key_parser.add_argument('--key-check', type=is_hexstr, action='append', help='Key Check Value')
+        put_key_parser.add_argument('--suppress-key-check', action='store_true', help='Suppress generation of Key Check Values')
+
+        @cmd2.with_argparser(put_key_parser)
+        def do_put_key(self, opts):
+            """Perform the GlobalPlatform PUT KEY command in order to store a new key on the card.
+            See GlobalPlatform CardSpecification v2.3 Section 11.8 for details.
+
+            The KCV (Key Check Values) can either be explicitly specified using `--key-check`, or will
+            otherwise be automatically generated for DES and AES keys.  You can suppress the latter using
+            `--suppress-key-check`.
+
+            Example (SCP80 KIC/KID/KIK):
+                put_key --key-version-nr 1 --key-id 0x01    --key-type aes --key-data 000102030405060708090a0b0c0d0e0f
+                                                            --key-type aes --key-data 101112131415161718191a1b1c1d1e1f
+                                                            --key-type aes --key-data 202122232425262728292a2b2c2d2e2f
+
+            Example (SCP81 TLS-PSK/KEK):
+                put_key --key-version-nr 0x40 --key-id 0x01 --key-type tls_psk --key-data 303132333435363738393a3b3c3d3e3f
+                                                            --key-type des --key-data 404142434445464748494a4b4c4d4e4f
+
+            """
+            if len(opts.key_type) != len(opts.key_data):
+                raise ValueError('There must be an equal number of key-type and key-data arguments')
+            kdb = []
+            for i in range(0, len(opts.key_type)):
+                if opts.key_check and len(opts.key_check) > i:
+                    kcv = opts.key_check[i]
+                elif opts.suppress_key_check:
+                    kcv = ''
+                else:
+                    kcv_bin = compute_kcv(opts.key_type[i], h2b(opts.key_data[i])) or b''
+                    kcv = b2h(kcv_bin)
+                if self._cmd.lchan.scc.scp:
+                    # encrypte key data with DEK of current SCP
+                    kcb = b2h(self._cmd.lchan.scc.scp.card_keys.encrypt_key(h2b(opts.key_data[i])))
+                else:
+                    # (for example) during personalization, DEK might not be required)
+                    kcb = opts.key_data[i]
+                kdb.append({'key_type': opts.key_type[i], 'kcb': kcb, 'kcv': kcv})
+            p2 = opts.key_id
+            if len(opts.key_type) > 1:
+                p2 |= 0x80
+            self.put_key(opts.old_key_version_nr, opts.key_version_nr, p2, kdb)
+
+        # Table 11-68: Key Data Field - Format 1 (Basic Format)
+        KeyDataBasic = GreedyRange(Struct('key_type'/KeyType,
+                                          'kcb'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                          'kcv'/HexAdapter(Prefixed(Int8ub, GreedyBytes))))
+
+        def put_key(self, old_kvn:int, kvn: int, kid: int, key_dict: dict) -> bytes:
+            """Perform the GlobalPlatform PUT KEY command in order to store a new key on the card.
+            See GlobalPlatform CardSpecification v2.3 Section 11.8 for details."""
+            key_data = kvn.to_bytes(1, 'big') + build_construct(ADF_SD.AddlShellCommands.KeyDataBasic, key_dict)
+            hdr = "80D8%02x%02x%02x" % (old_kvn, kid, len(key_data))
+            data, _sw = self._cmd.lchan.scc.send_apdu_checksw(hdr + b2h(key_data))
+            return data
+
+        get_status_parser = argparse.ArgumentParser()
+        get_status_parser.add_argument('subset', choices=StatusSubset.ksymapping.values(),
+                                       help='Subset of statuses to be included in the response')
+        get_status_parser.add_argument('--aid', type=is_hexstr, default='',
+                                       help='AID Search Qualifier (search only for given AID)')
+
+        @cmd2.with_argparser(get_status_parser)
+        def do_get_status(self, opts):
+            """Perform GlobalPlatform GET STATUS command in order to retrieve status information
+            on Issuer Security Domain, Executable Load File, Executable Module or Applications."""
+            grd_list = self.get_status(opts.subset, opts.aid)
+            for grd in grd_list:
+                self._cmd.poutput_json(grd.to_dict())
+
+        def get_status(self, subset:str, aid_search_qualifier:Hexstr = '') -> List[GpRegistryRelatedData]:
+            subset_hex = b2h(build_construct(StatusSubset, subset))
+            aid = ApplicationAID(decoded=aid_search_qualifier)
+            cmd_data = aid.to_tlv() + h2b('5c054f9f70c5cc')
+            p2 = 0x02 # TLV format according to Table 11-36
+            grd_list = []
+            while True:
+                hdr = "80F2%s%02x%02x" % (subset_hex, p2, len(cmd_data))
+                data, sw = self._cmd.lchan.scc.send_apdu(hdr + b2h(cmd_data))
+                remainder = h2b(data)
+                while len(remainder):
+                    # tlv sequence, each element is one GpRegistryRelatedData()
+                    grd = GpRegistryRelatedData()
+                    _dec, remainder = grd.from_tlv(remainder)
+                    grd_list.append(grd)
+                if sw != '6310':
+                    return grd_list
+                else:
+                    p2 |= 0x01
+            return grd_list
+
+        set_status_parser = argparse.ArgumentParser()
+        set_status_parser.add_argument('scope', choices=SetStatusScope.ksymapping.values(),
+                                       help='Defines the scope of the requested status change')
+        set_status_parser.add_argument('status', choices=CLifeCycleState.ksymapping.values(),
+                                       help='Specify the new intended status')
+        set_status_parser.add_argument('--aid', type=is_hexstr,
+                                       help='AID of the target Application or Security Domain')
+
+        @cmd2.with_argparser(set_status_parser)
+        def do_set_status(self, opts):
+            """Perform GlobalPlatform SET STATUS command in order to change the life cycle state of the
+            Issuer Security Domain, Supplementary Security Domain or Application.  This normally requires
+            prior authentication with a Secure Channel Protocol."""
+            self.set_status(opts.scope, opts.status, opts.aid)
+
+        def set_status(self, scope:str, status:str, aid:Hexstr = ''):
+            SetStatus = Struct(Const(0x80, Byte), Const(0xF0, Byte),
+                               'scope'/SetStatusScope, 'status'/CLifeCycleState,
+                               'aid'/HexAdapter(Prefixed(Int8ub, COptional(GreedyBytes))))
+            apdu = build_construct(SetStatus, {'scope':scope, 'status':status, 'aid':aid})
+            _data, _sw = self._cmd.lchan.scc.send_apdu_checksw(b2h(apdu))
+
+        inst_perso_parser = argparse.ArgumentParser()
+        inst_perso_parser.add_argument('application-aid', type=is_hexstr, help='Application AID')
+
+        @cmd2.with_argparser(inst_perso_parser)
+        def do_install_for_personalization(self, opts):
+            """Perform GlobalPlatform INSTALL [for personalization] command in order to inform a Security
+            Domain that the following STORE DATA commands are meant for a specific AID (specified here)."""
+            # Section 11.5.2.3.6 / Table 11-47
+            self.install(0x20, 0x00, "0000%02x%s000000" % (len(opts.application_aid)//2, opts.application_aid))
+
+        inst_inst_parser = argparse.ArgumentParser()
+        inst_inst_parser.add_argument('--load-file-aid', type=is_hexstr, default='',
+                                      help='Executable Load File AID')
+        inst_inst_parser.add_argument('--module-aid', type=is_hexstr, default='',
+                                      help='Executable Module AID')
+        inst_inst_parser.add_argument('--application-aid', type=is_hexstr, required=True,
+                                      help='Application AID')
+        inst_inst_parser.add_argument('--install-parameters', type=is_hexstr, default='',
+                                      help='Install Parameters')
+        inst_inst_parser.add_argument('--privilege', action='append', dest='privileges', default=[],
+                                      choices=Privileges._construct.flags.keys(),
+                                      help='Privilege granted to newly installed Application')
+        inst_inst_parser.add_argument('--install-token', type=is_hexstr, default='',
+                                      help='Install Token (Section GPCS C.4.2/C.4.7)')
+        inst_inst_parser.add_argument('--make-selectable', action='store_true',
+                                      help='Install and make selectable')
+
+        @cmd2.with_argparser(inst_inst_parser)
+        def do_install_for_install(self, opts):
+            """Perform GlobalPlatform INSTALL [for install] command in order to install an application."""
+            InstallForInstallCD = Struct('load_file_aid'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                         'module_aid'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                         'application_aid'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                         'privileges'/Prefixed(Int8ub, Privileges._construct),
+                                         'install_parameters'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                         'install_token'/HexAdapter(Prefixed(Int8ub, GreedyBytes)))
+            p1 = 0x04
+            if opts.make_selectable:
+                p1 |= 0x08
+            decoded = vars(opts)
+            # convert from list to "true-dict" as required by construct.FlagsEnum
+            decoded['privileges'] = {x: True for x in decoded['privileges']}
+            ifi_bytes = build_construct(InstallForInstallCD, decoded)
+            self.install(p1, 0x00, b2h(ifi_bytes))
+
+        inst_load_parser = argparse.ArgumentParser()
+        inst_load_parser.add_argument('--load-file-aid', type=is_hexstr, required=True,
+                                      help='AID of the loded file')
+        inst_load_parser.add_argument('--security-domain-aid', type=is_hexstr, default='',
+                                      help='AID of the Security Domain into which the file shalle be added')
+        inst_load_parser.add_argument('--load-file-hash', type=is_hexstr, default='',
+                                      help='Load File Data Block Hash')
+        inst_inst_parser.add_argument('--load-parameters', type=is_hexstr, default='',
+                                      help='Load Token (Section GPCS C.4.1)')
+        inst_inst_parser.add_argument('--load-token', type=is_hexstr, default='',
+                                      help='Load Token (Section GPCS C.4.1)')
+
+        @cmd2.with_argparser(inst_load_parser)
+        def do_install_for_load(self, opts):
+            """Perform GlobalPlatform INSTALL [for load] command."""
+            if opts.load_token != '' and opts.load_file_hash == '':
+                raise ValueError('Load File Data Block Hash is mandatory if a Load Token is present')
+            InstallForLoadCD = Struct('load_file_aid'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                      'security_domain_aid'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                      'load_file_hash'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                      'load_parameters'/HexAdapter(Prefixed(Int8ub, GreedyBytes)),
+                                      'load_token'/HexAdapter(Prefixed(Int8ub, GreedyBytes)))
+            ifl_bytes = build_construct(InstallForLoadCD, vars(opts))
+            self.install(0x02, 0x00, b2h(ifl_bytes))
+
+        def install(self, p1:int, p2:int, data:Hexstr) -> ResTuple:
+            cmd_hex = "80E6%02x%02x%02x%s" % (p1, p2, len(data)//2, data)
+            return self._cmd.lchan.scc.send_apdu_checksw(cmd_hex)
+
+        del_cc_parser = argparse.ArgumentParser()
+        del_cc_parser.add_argument('aid', type=is_hexstr,
+                                   help='Executable Load File or Application AID')
+        del_cc_parser.add_argument('--delete-related-objects', action='store_true',
+                                   help='Delete not only the object but also its related objects')
+
+        @cmd2.with_argparser(del_cc_parser)
+        def do_delete_card_content(self, opts):
+            """Perform a GlobalPlatform DELETE [card content] command in order to delete an Executable Load
+            File, an Application or an Executable Load File and its related Applications."""
+            p2 = 0x80 if opts.delete_related_objects else 0x00
+            aid = ApplicationAID(decoded=opts.aid)
+            self.delete(0x00, p2, b2h(aid.to_tlv()))
+
+        del_key_parser = argparse.ArgumentParser()
+        del_key_parser.add_argument('--key-id', type=auto_uint7, help='Key Identifier (KID)')
+        del_key_parser.add_argument('--key-ver', type=auto_uint8, help='Key Version Number (KVN)')
+        del_key_parser.add_argument('--delete-related-objects', action='store_true',
+                                   help='Delete not only the object but also its related objects')
+
+        @cmd2.with_argparser(del_key_parser)
+        def do_delete_key(self, opts):
+            """Perform GlobalPlaform DELETE (Key) command.
+            If both KID and KVN are specified, exactly one key is deleted. If only either of the two is
+            specified, multiple matching keys may be deleted."""
+            if opts.key_id is None and opts.key_ver is None:
+                raise ValueError('At least one of KID or KVN must be specified')
+            p2 = 0x80 if opts.delete_related_objects else 0x00
+            cmd = ""
+            if opts.key_id is not None:
+                cmd += "d001%02x" % opts.key_id
+            if opts.key_ver is not None:
+                cmd += "d201%02x" % opts.key_ver
+            self.delete(0x00, p2, cmd)
+
+        def delete(self, p1:int, p2:int, data:Hexstr) -> ResTuple:
+            cmd_hex = "80E4%02x%02x%02x%s" % (p1, p2, len(data)//2, data)
+            return self._cmd.lchan.scc.send_apdu_checksw(cmd_hex)
+
+        load_parser = argparse.ArgumentParser()
+        # we make this a required --optional argument now, so we can later have other sources for load data
+        load_parser.add_argument('--from-file', required=True)
+
+        @cmd2.with_argparser(load_parser)
+        def do_load(self, opts):
+            """Perform a GlobalPlatform LOAD command. We currently only support loading without DAP and
+            without ciphering."""
+            with open(opts.from_file, 'rb') as f:
+                self.load(f)
+
+        def load(self, stream: io.RawIOBase, chunk_len:int = 240):
+            # we might want to tune chunk_len based on the overhead of the used SCP?
+            contents = stream.readall()
+            # build TLV according to 11.6.2.3 / Table 11-58 for unencrypted case
+            remainder = b'\xC4' + bertlv_encode_len(len(contents)) + contents
+            # transfer this in vaious chunks to the card
+            total_size = len(remainder)
+            block_nr = 0
+            while len(remainder):
+                block = remainder[:chunk_len]
+                remainder = remainder[chunk_len:]
+                # build LOAD command APDU according to 11.6.2 / Table 11-56
+                p1 = 0x80 if len(remainder) else 0x00
+                p2 = block_nr % 256
+                block_nr += 1
+                cmd_hex = "80E8%02x%02x%02x%s" % (p1, p2, len(block), b2h(block))
+                _rsp_hex, _sw = self._cmd.lchan.scc.send_apdu_checksw(cmd_hex)
+            self._cmd.poutput("Loaded a total of %u bytes in %u blocks. Don't forget install_for_load now!" % (total_size, block_nr))
+
+
+        est_scp02_parser = argparse.ArgumentParser()
+        est_scp02_parser.add_argument('--key-ver', type=auto_uint8, required=True,
+                                      help='Key Version Number (KVN)')
+        est_scp02_parser.add_argument('--key-enc', type=is_hexstr, required=True,
+                                      help='Secure Channel Encryption Key')
+        est_scp02_parser.add_argument('--key-mac', type=is_hexstr, required=True,
+                                      help='Secure Channel MAC Key')
+        est_scp02_parser.add_argument('--key-dek', type=is_hexstr, required=True,
+                                      help='Data Encryption Key')
+        est_scp02_parser.add_argument('--host-challenge', type=is_hexstr,
+                                      help='Hard-code the host challenge; default: random')
+        est_scp02_parser.add_argument('--security-level', type=auto_uint8, default=0x01,
+                                      help='Security Level. Default: 0x01 (C-MAC only)')
+
+        @cmd2.with_argparser(est_scp02_parser)
+        def do_establish_scp02(self, opts):
+            """Establish a secure channel using the GlobalPlatform SCP02 protocol.  It can be released
+            again by using `release_scp`."""
+            if self._cmd.lchan.scc.scp:
+                self._cmd.poutput("Cannot establish SCP02 as this lchan already has a SCP instance!")
+                return
+            host_challenge = h2b(opts.host_challenge) if opts.host_challenge else get_random_bytes(8)
+            kset = GpCardKeyset(opts.key_ver, h2b(opts.key_enc), h2b(opts.key_mac), h2b(opts.key_dek))
+            scp02 = SCP02(card_keys=kset)
+            self._establish_scp(scp02, host_challenge, opts.security_level)
+
+        est_scp03_parser = deepcopy(est_scp02_parser)
+        est_scp03_parser.add_argument('--s16-mode', action='store_true', help='S16 mode (S8 is default)')
+
+        @cmd2.with_argparser(est_scp03_parser)
+        def do_establish_scp03(self, opts):
+            """Establish a secure channel using the GlobalPlatform SCP03 protocol.  It can be released
+            again by using `release_scp`."""
+            if self._cmd.lchan.scc.scp:
+                self._cmd.poutput("Cannot establish SCP03 as this lchan already has a SCP instance!")
+                return
+            s_mode = 16 if opts.s16_mode else 8
+            host_challenge = h2b(opts.host_challenge) if opts.host_challenge else get_random_bytes(s_mode)
+            kset = GpCardKeyset(opts.key_ver, h2b(opts.key_enc), h2b(opts.key_mac), h2b(opts.key_dek))
+            scp03 = SCP03(card_keys=kset, s_mode = s_mode)
+            self._establish_scp(scp03, host_challenge, opts.security_level)
+
+        def _establish_scp(self, scp, host_challenge, security_level):
+            # perform the common functionality shared by SCP02 and SCP03 establishment
+            init_update_apdu = scp.gen_init_update_apdu(host_challenge=host_challenge)
+            init_update_resp, _sw = self._cmd.lchan.scc.send_apdu_checksw(b2h(init_update_apdu))
+            scp.parse_init_update_resp(h2b(init_update_resp))
+            ext_auth_apdu = scp.gen_ext_auth_apdu(security_level)
+            _ext_auth_resp, _sw = self._cmd.lchan.scc.send_apdu_checksw(b2h(ext_auth_apdu))
+            self._cmd.poutput("Successfully established a %s secure channel" % str(scp))
+            # store a reference to the SCP instance
+            self._cmd.lchan.scc.scp = scp
+            self._cmd.update_prompt()
+
+
+        def do_release_scp(self, _opts):
+            """Release a previously establiehed secure channel."""
+            if not self._cmd.lchan.scc.scp:
+                self._cmd.poutput("Cannot release SCP as none is established")
+                return
+            self._cmd.lchan.scc.scp = None
+            self._cmd.update_prompt()
+
+
+# Card Application of a Security Domain
+class CardApplicationSD(CardApplication):
+    __intermediate = True
+    def __init__(self, aid: str, name: str, desc: str):
+        super().__init__(name, adf=ADF_SD(aid, name, desc), sw=sw_table)
+
+# Card Application of Issuer Security Domain
+class CardApplicationISD(CardApplicationSD):
+    # FIXME: ISD AID is not static, but could be different. One can select the empty
+    # application using '00a4040000' and then parse the response FCI to get the ISD AID
+    def __init__(self, aid='a000000003000000'):
+        super().__init__(aid=aid, name='ADF.ISD', desc='Issuer Security Domain')
+
+#class CardProfileGlobalPlatform(CardProfile):
+#    ORDER = 23
+#
+#    def __init__(self, name='GlobalPlatform'):
+#        super().__init__(name, desc='GlobalPlatfomr 2.1.1', cla=['00','80','84'], sw=sw_table)
+
+
+class GpCardKeyset:
+    """A single set of GlobalPlatform card keys and the associated KVN."""
+    def __init__(self, kvn: int, enc: bytes, mac: bytes, dek: bytes):
+        assert 0 < kvn < 256
+        assert len(enc) == len(mac) == len(dek)
+        self.kvn = kvn
+        self.enc = enc
+        self.mac = mac
+        self.dek = dek
+
+    @classmethod
+    def from_single_key(cls, kvn: int, base_key: bytes) -> 'GpCardKeyset':
+        return cls(kvn, base_key, base_key, base_key)
+
+    def __str__(self):
+        return "%s(KVN=%u, ENC=%s, MAC=%s, DEK=%s)" % (self.__class__.__name__,
+                self.kvn, b2h(self.enc), b2h(self.mac), b2h(self.dek))
+
+
+def compute_kcv_des(key:bytes) -> bytes:
+    # GP Card Spec B.6: For a DES key, the key check value is computed by encrypting 8 bytes, each with
+    # value '00', with the key to be checked and retaining the 3 highest-order bytes of the encrypted
+    # result.
+    plaintext = b'\x00' * 8
+    cipher = DES3.new(key, DES.MODE_ECB)
+    return cipher.encrypt(plaintext)
+
+def compute_kcv_aes(key:bytes) -> bytes:
+    # GP Card Spec B.6: For a AES key, the key check value is computed by encrypting 16 bytes, each with
+    # value '01', with the key to be checked and retaining the 3 highest-order bytes of the encrypted
+    # result.
+    plaintext = b'\x01' * 16
+    cipher = AES.new(key, AES.MODE_ECB)
+    return cipher.encrypt(plaintext)
+
+# dict is keyed by the string name of the KeyType enum above in this file
+KCV_CALCULATOR = {
+        'aes': compute_kcv_aes,
+        'des': compute_kcv_des,
+    }
+
+def compute_kcv(key_type: str, key: bytes) -> Optional[bytes]:
+    """Compute the KCV (Key Check Value) for given key type and key."""
+    kcv_calculator = KCV_CALCULATOR.get(key_type)
+    if not kcv_calculator:
+        return None
+    else:
+        return kcv_calculator(key)[:3]

pySim/global_platform/scp.py

@@ -0,0 +1,534 @@
+# Global Platform SCP02 + SCP03 (Secure Channel Protocol) implementation
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+import logging
+from typing import Optional
+from Cryptodome.Cipher import DES3, DES
+from Cryptodome.Util.strxor import strxor
+from construct import Struct, Bytes, Int8ub, Int16ub, Const
+from construct import Optional as COptional
+from pySim.utils import b2h, bertlv_parse_len, bertlv_encode_len
+from pySim.secure_channel import SecureChannel
+
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)
+
+def scp02_key_derivation(constant: bytes, counter: int, base_key: bytes) -> bytes:
+    assert len(constant) == 2
+    assert(counter >= 0 and counter <= 65535)
+    assert len(base_key) == 16
+
+    derivation_data = constant + counter.to_bytes(2, 'big') + b'\x00' * 12
+    cipher = DES3.new(base_key, DES.MODE_CBC, b'\x00' * 8)
+    return cipher.encrypt(derivation_data)
+
+# TODO: resolve duplication with BspAlgoCryptAES128
+def pad80(s: bytes, BS=8) -> bytes:
+    """ Pad bytestring s: add '\x80' and '\0'* so the result to be multiple of BS."""
+    l = BS-1 - len(s) % BS
+    return s + b'\x80' + b'\0'*l
+
+# TODO: resolve duplication with BspAlgoCryptAES128
+def unpad80(padded: bytes) -> bytes:
+    """Remove the customary 80 00 00 ... padding used for AES."""
+    # first remove any trailing zero bytes
+    stripped = padded.rstrip(b'\0')
+    # then remove the final 80
+    assert stripped[-1] == 0x80
+    return stripped[:-1]
+
+class Scp02SessionKeys:
+    """A single set of GlobalPlatform session keys."""
+    DERIV_CONST_CMAC = b'\x01\x01'
+    DERIV_CONST_RMAC = b'\x01\x02'
+    DERIV_CONST_ENC = b'\x01\x82'
+    DERIV_CONST_DENC = b'\x01\x81'
+    blocksize = 8
+
+    def calc_mac_1des(self, data: bytes, reset_icv: bool = False) -> bytes:
+        """Pad and calculate MAC according to B.1.2.2 - Single DES plus final 3DES"""
+        e = DES.new(self.c_mac[:8], DES.MODE_ECB)
+        d = DES.new(self.c_mac[8:], DES.MODE_ECB)
+        padded_data = pad80(data, 8)
+        q = len(padded_data) // 8
+        icv = b'\x00' * 8 if reset_icv else self.icv
+        h = icv
+        for i in range(q):
+            h = e.encrypt(strxor(h, bytes(padded_data[8*i:8*(i+1)])))
+        h = d.decrypt(h)
+        h = e.encrypt(h)
+        logger.debug("mac_1des(%s,icv=%s) -> %s", b2h(data), b2h(icv), b2h(h))
+        if self.des_icv_enc:
+            self.icv = self.des_icv_enc.encrypt(h)
+        else:
+            self.icv = h
+        return h
+
+    def calc_mac_3des(self, data: bytes) -> bytes:
+        e = DES3.new(self.enc, DES.MODE_ECB)
+        padded_data = pad80(data, 8)
+        q = len(padded_data) // 8
+        h = b'\x00' * 8
+        for i in range(q):
+            h = e.encrypt(strxor(h, bytes(padded_data[8*i:8*(i+1)])))
+        logger.debug("mac_3des(%s) -> %s", b2h(data), b2h(h))
+        return h
+
+    def __init__(self, counter: int, card_keys: 'GpCardKeyset', icv_encrypt=True):
+        self.icv = None
+        self.counter = counter
+        self.card_keys = card_keys
+        self.c_mac = scp02_key_derivation(self.DERIV_CONST_CMAC, self.counter, card_keys.mac)
+        self.r_mac = scp02_key_derivation(self.DERIV_CONST_RMAC, self.counter, card_keys.mac)
+        self.enc = scp02_key_derivation(self.DERIV_CONST_ENC, self.counter, card_keys.enc)
+        self.data_enc = scp02_key_derivation(self.DERIV_CONST_DENC, self.counter, card_keys.dek)
+        self.des_icv_enc = DES.new(self.c_mac[:8], DES.MODE_ECB) if icv_encrypt else None
+
+    def __str__(self) -> str:
+        return "%s(CTR=%u, ICV=%s, ENC=%s, D-ENC=%s, MAC-C=%s, MAC-R=%s)" % (
+                self.__class__.__name__, self.counter, b2h(self.icv) if self.icv else "None",
+                b2h(self.enc), b2h(self.data_enc), b2h(self.c_mac), b2h(self.r_mac))
+
+INS_INIT_UPDATE = 0x50
+INS_EXT_AUTH = 0x82
+CLA_SM = 0x04
+
+class SCP(SecureChannel, abc.ABC):
+    """Abstract base class containing some common interface + functionality for SCP protocols."""
+    def __init__(self, card_keys: 'GpCardKeyset', lchan_nr: int = 0):
+        if hasattr(self, 'kvn_range'):
+            if not card_keys.kvn in range(self.kvn_range[0], self.kvn_range[1]+1):
+                raise ValueError('%s cannot be used with KVN outside range 0x%02x..0x%02x' %
+                                 (self.__class__.__name__, self.kvn_range[0], self.kvn_range[1]))
+        self.lchan_nr = lchan_nr
+        self.card_keys = card_keys
+        self.sk = None
+        self.mac_on_unmodified = False
+        self.security_level = 0x00
+
+    @property
+    def do_cmac(self) -> bool:
+        """Should we perform C-MAC?"""
+        return self.security_level & 0x01
+
+    @property
+    def do_rmac(self) -> bool:
+        """Should we perform R-MAC?"""
+        return self.security_level & 0x10
+
+    @property
+    def do_cenc(self) -> bool:
+        """Should we perform C-ENC?"""
+        return self.security_level & 0x02
+
+    @property
+    def do_renc(self) -> bool:
+        """Should we perform R-ENC?"""
+        return self.security_level & 0x20
+
+    def __str__(self) -> str:
+        return "%s[%02x]" % (self.__class__.__name__, self.security_level)
+
+    def _cla(self, sm: bool = False, b8: bool = True) -> int:
+        ret = 0x80 if b8 else 0x00
+        if sm:
+            ret = ret | CLA_SM
+        return ret + self.lchan_nr
+
+    def wrap_cmd_apdu(self, apdu: bytes, *args, **kwargs) -> bytes:
+        # Generic handling of GlobalPlatform SCP, implements SecureChannel.wrap_cmd_apdu
+        # only protect those APDUs that actually are global platform commands
+        if apdu[0] & 0x80:
+            return self._wrap_cmd_apdu(apdu, *args, **kwargs)
+        return apdu
+
+    @abc.abstractmethod
+    def _wrap_cmd_apdu(self, apdu: bytes, *args, **kwargs) -> bytes:
+        """Method implementation to be provided by derived class."""
+        pass
+
+    @abc.abstractmethod
+    def gen_init_update_apdu(self, host_challenge: Optional[bytes]) -> bytes:
+        pass
+
+    @abc.abstractmethod
+    def parse_init_update_resp(self, resp_bin: bytes):
+        pass
+
+    @abc.abstractmethod
+    def gen_ext_auth_apdu(self, security_level: int = 0x01) -> bytes:
+        pass
+
+    def encrypt_key(self, key: bytes) -> bytes:
+        """Encrypt a key with the DEK."""
+        num_pad = len(key) % self.sk.blocksize
+        if num_pad:
+            return bertlv_encode_len(len(key)) + self.dek_encrypt(key + b'\x00'*num_pad)
+        return self.dek_encrypt(key)
+
+    def decrypt_key(self, encrypted_key:bytes) -> bytes:
+        """Decrypt a key with the DEK."""
+        if len(encrypted_key) % self.sk.blocksize:
+            # If the length of the Key Component Block is not a multiple of the block size of the encryption #
+            # algorithm (i.e. 8 bytes for DES, 16 bytes for AES), then it shall be assumed that the key
+            # component value was right-padded prior to encryption and that the Key Component Block was
+            # formatted as described in Table 11-70. In this case, the first byte(s) of the Key Component
+            # Block provides the actual length of the key component value, which allows recovering the
+            # clear-text key component value after decryption of the encrypted key component value and removal
+            # of padding bytes.
+            decrypted = self.dek_decrypt(encrypted_key)
+            key_len, remainder = bertlv_parse_len(decrypted)
+            return remainder[:key_len]
+        else:
+            # If the length of the Key Component Block is a multiple of the block size of the encryption
+            # algorithm (i.e.  8 bytes for DES, 16 bytes for AES), then it shall be assumed that no padding
+            # bytes were added before encrypting the key component value and that the Key Component Block is
+            # only composed of the encrypted key component value (as shown in Table 11-71). In this case, the
+            # clear-text key component value is simply recovered by decrypting the Key Component Block.
+            return self.dek_decrypt(encrypted_key)
+
+    @abc.abstractmethod
+    def dek_encrypt(self, plaintext:bytes) -> bytes:
+        pass
+
+    @abc.abstractmethod
+    def dek_decrypt(self, ciphertext:bytes) -> bytes:
+        pass
+
+
+class SCP02(SCP):
+    """An instance of the GlobalPlatform SCP02 secure channel protocol."""
+
+    constr_iur = Struct('key_div_data'/Bytes(10), 'key_ver'/Int8ub, Const(b'\x02'),
+                        'seq_counter'/Int16ub, 'card_challenge'/Bytes(6), 'card_cryptogram'/Bytes(8))
+    kvn_range = [0x20, 0x2f]
+
+    def __init__(self, *args, **kwargs):
+        self.overhead = 8
+        super().__init__(*args, **kwargs)
+
+    def dek_encrypt(self, plaintext:bytes) -> bytes:
+        cipher = DES.new(self.card_keys.dek, DES.MODE_ECB)
+        return cipher.encrypt(plaintext)
+
+    def dek_decrypt(self, ciphertext:bytes) -> bytes:
+        cipher = DES.new(self.card_keys.dek, DES.MODE_ECB)
+        return cipher.decrypt(ciphertext)
+
+    def _compute_cryptograms(self, card_challenge: bytes, host_challenge: bytes):
+        logger.debug("host_challenge(%s), card_challenge(%s)", b2h(host_challenge), b2h(card_challenge))
+        self.host_cryptogram = self.sk.calc_mac_3des(self.sk.counter.to_bytes(2, 'big') + card_challenge + host_challenge)
+        self.card_cryptogram = self.sk.calc_mac_3des(self.host_challenge + self.sk.counter.to_bytes(2, 'big') + card_challenge)
+        logger.debug("host_cryptogram(%s), card_cryptogram(%s)", b2h(self.host_cryptogram), b2h(self.card_cryptogram))
+
+    def gen_init_update_apdu(self, host_challenge: bytes = b'\x00'*8) -> bytes:
+        """Generate INITIALIZE UPDATE APDU."""
+        self.host_challenge = host_challenge
+        return bytes([self._cla(), INS_INIT_UPDATE, self.card_keys.kvn, 0, 8]) + self.host_challenge
+
+    def parse_init_update_resp(self, resp_bin: bytes):
+        """Parse response to INITIALZIE UPDATE."""
+        resp = self.constr_iur.parse(resp_bin)
+        self.card_challenge = resp['card_challenge']
+        self.sk = Scp02SessionKeys(resp['seq_counter'], self.card_keys)
+        logger.debug(self.sk)
+        self._compute_cryptograms(self.card_challenge, self.host_challenge)
+        if self.card_cryptogram != resp['card_cryptogram']:
+            raise ValueError("card cryptogram doesn't match")
+
+    def gen_ext_auth_apdu(self, security_level: int = 0x01) -> bytes:
+        """Generate EXTERNAL AUTHENTICATE APDU."""
+        if security_level & 0xf0:
+            raise NotImplementedError('R-MAC/R-ENC for SCP02 not implemented yet.')
+        self.security_level = security_level
+        if self.mac_on_unmodified:
+            header = bytes([self._cla(), INS_EXT_AUTH, self.security_level, 0, 8])
+        else:
+            header = bytes([self._cla(True), INS_EXT_AUTH, self.security_level, 0, 16])
+        #return self.wrap_cmd_apdu(header + self.host_cryptogram)
+        mac = self.sk.calc_mac_1des(header + self.host_cryptogram, True)
+        return bytes([self._cla(True), INS_EXT_AUTH, self.security_level, 0, 16]) + self.host_cryptogram + mac
+
+    def _wrap_cmd_apdu(self, apdu: bytes, *args, **kwargs) -> bytes:
+        """Wrap Command APDU for SCP02: calculate MAC and encrypt."""
+        lc = len(apdu) - 5
+        assert len(apdu) >= 5, "Wrong APDU length: %d" % len(apdu)
+        assert len(apdu) == 5 or apdu[4] == lc, "Lc differs from length of data: %d vs %d" % (apdu[4], lc)
+
+        logger.debug("wrap_cmd_apdu(%s)", b2h(apdu))
+
+        cla = apdu[0]
+        b8 = cla & 0x80
+        if cla & 0x03 or cla & CLA_SM:
+            # nonzero logical channel in APDU, check that are the same
+            assert cla == self._cla(False, b8), "CLA mismatch"
+        # CLA without log. channel can be 80 or 00 only
+        if self.do_cmac:
+            if self.mac_on_unmodified:
+                mlc = lc
+                clac = cla
+            else:                      # CMAC on modified APDU
+                mlc = lc + 8
+                clac = cla | CLA_SM
+            mac = self.sk.calc_mac_1des(bytes([clac]) + apdu[1:4] + bytes([mlc]) + apdu[5:])
+            if self.do_cenc:
+                k = DES3.new(self.sk.enc, DES.MODE_CBC, b'\x00'*8)
+                data = k.encrypt(pad80(apdu[5:], 8))
+                lc = len(data)
+            else:
+                data = apdu[5:]
+            lc += 8
+            apdu = bytes([self._cla(True, b8)]) + apdu[1:4] + bytes([lc]) + data + mac
+        return apdu
+
+    def unwrap_rsp_apdu(self, sw: bytes, rsp_apdu: bytes) -> bytes:
+        # TODO: Implement R-MAC / R-ENC
+        return rsp_apdu
+
+
+
+from Cryptodome.Cipher import AES
+from Cryptodome.Hash import CMAC
+
+def scp03_key_derivation(constant: bytes, context: bytes, base_key: bytes, l: Optional[int] = None) -> bytes:
+    """SCP03 Key Derivation Function as specified in Annex D 4.1.5."""
+    # Data derivation shall use KDF in counter mode as specified in NIST SP 800-108 ([NIST 800-108]). The PRF
+    # used in the KDF shall be CMAC as specified in [NIST 800-38B], used with full 16-byte output length.
+    def prf(key: bytes, data:bytes):
+        return CMAC.new(key, data, AES).digest()
+
+    if l is None:
+        l = len(base_key) * 8
+
+    logger.debug("scp03_kdf(constant=%s, context=%s, base_key=%s, l=%u)", b2h(constant), b2h(context), b2h(base_key), l)
+    output_len = l // 8
+    # SCP03 Section 4.1.5 defines a different parameter order than NIST SP 800-108, so we cannot use the
+    # existing Cryptodome.Protocol.KDF.SP800_108_Counter function :(
+    # A 12-byte “label” consisting of 11 bytes with value '00' followed by a 1-byte derivation constant
+    assert len(constant) == 1
+    label = b'\x00' *11 + constant
+    i = 1
+    dk = b''
+    while len(dk) < output_len:
+        # 12B label, 1B separation, 2B L, 1B i, Context
+        info = label + b'\x00' + l.to_bytes(2, 'big') + bytes([i]) + context
+        dk += prf(base_key, info)
+        i += 1
+        if i > 0xffff:
+            raise ValueError("Overflow in SP800 108 counter")
+    return dk[:output_len]
+
+
+class Scp03SessionKeys:
+    # GPC 2.3 Amendment D v1.2 Section 4.1.5 Table 4-1
+    DERIV_CONST_AUTH_CGRAM_CARD = b'\x00'
+    DERIV_CONST_AUTH_CGRAM_HOST = b'\x01'
+    DERIV_CONST_CARD_CHLG_GEN = b'\x02'
+    DERIV_CONST_KDERIV_S_ENC = b'\x04'
+    DERIV_CONST_KDERIV_S_MAC = b'\x06'
+    DERIV_CONST_KDERIV_S_RMAC = b'\x07'
+    blocksize = 16
+
+    def __init__(self, card_keys: 'GpCardKeyset', host_challenge: bytes, card_challenge: bytes):
+        # GPC 2.3 Amendment D v1.2 Section 6.2.1
+        context = host_challenge + card_challenge
+        self.s_enc = scp03_key_derivation(self.DERIV_CONST_KDERIV_S_ENC, context, card_keys.enc)
+        self.s_mac = scp03_key_derivation(self.DERIV_CONST_KDERIV_S_MAC, context, card_keys.mac)
+        self.s_rmac = scp03_key_derivation(self.DERIV_CONST_KDERIV_S_RMAC, context, card_keys.mac)
+
+
+        # The first MAC chaining value is set to 16 bytes '00'
+        self.mac_chaining_value = b'\x00' * 16
+        # The encryption counter’s start value shall be set to 1 (we set it immediately before generating ICV)
+        self.block_nr = 0
+
+    def calc_cmac(self, apdu: bytes):
+        """Compute C-MAC for given to-be-transmitted APDU.
+        Returns the full 16-byte MAC, caller must truncate it if needed for S8 mode."""
+        cmac_input = self.mac_chaining_value + apdu
+        cmac_val = CMAC.new(self.s_mac, cmac_input, ciphermod=AES).digest()
+        self.mac_chaining_value = cmac_val
+        return cmac_val
+
+    def calc_rmac(self, rdata_and_sw: bytes):
+        """Compute R-MAC for given received R-APDU data section.
+        Returns the full 16-byte MAC, caller must truncate it if needed for S8 mode."""
+        rmac_input = self.mac_chaining_value + rdata_and_sw
+        return CMAC.new(self.s_rmac, rmac_input, ciphermod=AES).digest()
+
+    def _get_icv(self, is_response: bool = False):
+        """Obtain the ICV value computed as described in 6.2.6.
+        This method has two modes:
+            * is_response=False for computing the ICV for C-ENC. Will pre-increment the counter.
+            * is_response=False for computing the ICV for R-DEC."""
+        if not is_response:
+            self.block_nr += 1
+        # The binary value of this number SHALL be left padded with zeroes to form a full block.
+        data = self.block_nr.to_bytes(self.blocksize, "big")
+        if is_response:
+            # Section 6.2.7: additional intermediate step: Before encryption, the most significant byte of
+            # this block shall be set to '80'.
+            data = b'\x80' + data[1:]
+        iv = bytes([0] * self.blocksize)
+        # This block SHALL be encrypted with S-ENC to produce the ICV for command encryption.
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, iv)
+        icv = cipher.encrypt(data)
+        logger.debug("_get_icv(data=%s, is_resp=%s) -> icv=%s", b2h(data), is_response, b2h(icv))
+        return icv
+
+    # TODO: Resolve duplication with pySim.esim.bsp.BspAlgoCryptAES128 which provides pad80-wrapping
+    def _encrypt(self, data: bytes, is_response: bool = False) -> bytes:
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, self._get_icv(is_response))
+        return cipher.encrypt(data)
+
+    # TODO: Resolve duplication with pySim.esim.bsp.BspAlgoCryptAES128 which provides pad80-unwrapping
+    def _decrypt(self, data: bytes, is_response: bool = True) -> bytes:
+        cipher = AES.new(self.s_enc, AES.MODE_CBC, self._get_icv(is_response))
+        return cipher.decrypt(data)
+
+
+class SCP03(SCP):
+    """Secure Channel Protocol (SCP) 03 as specified in GlobalPlatform v2.3 Amendment D."""
+
+    # Section 7.1.1.6 / Table 7-3
+    constr_iur = Struct('key_div_data'/Bytes(10), 'key_ver'/Int8ub, Const(b'\x03'), 'i_param'/Int8ub,
+                        'card_challenge'/Bytes(lambda ctx: ctx._.s_mode),
+                        'card_cryptogram'/Bytes(lambda ctx: ctx._.s_mode),
+                        'sequence_counter'/COptional(Bytes(3)))
+    kvn_range = [0x30, 0x3f]
+
+    def __init__(self, *args, **kwargs):
+        self.s_mode = kwargs.pop('s_mode', 8)
+        self.overhead = self.s_mode
+        super().__init__(*args, **kwargs)
+
+    def dek_encrypt(self, plaintext:bytes) -> bytes:
+        cipher = AES.new(self.card_keys.dek, AES.MODE_CBC, b'\x00'*16)
+        return cipher.encrypt(plaintext)
+
+    def dek_decrypt(self, ciphertext:bytes) -> bytes:
+        cipher = AES.new(self.card_keys.dek, AES.MODE_CBC, b'\x00'*16)
+        return cipher.decrypt(ciphertext)
+
+    def _compute_cryptograms(self):
+        logger.debug("host_challenge(%s), card_challenge(%s)", b2h(self.host_challenge), b2h(self.card_challenge))
+        # Card + Host Authentication Cryptogram: Section 6.2.2.2 + 6.2.2.3
+        context = self.host_challenge + self.card_challenge
+        self.card_cryptogram = scp03_key_derivation(self.sk.DERIV_CONST_AUTH_CGRAM_CARD, context, self.sk.s_mac, l=self.s_mode*8)
+        self.host_cryptogram = scp03_key_derivation(self.sk.DERIV_CONST_AUTH_CGRAM_HOST, context, self.sk.s_mac, l=self.s_mode*8)
+        logger.debug("host_cryptogram(%s), card_cryptogram(%s)", b2h(self.host_cryptogram), b2h(self.card_cryptogram))
+
+    def gen_init_update_apdu(self, host_challenge: Optional[bytes] = None) -> bytes:
+        """Generate INITIALIZE UPDATE APDU."""
+        if host_challenge is None:
+            host_challenge = b'\x00' * self.s_mode
+        if len(host_challenge) != self.s_mode:
+            raise ValueError('Host Challenge must be %u bytes long' % self.s_mode)
+        self.host_challenge = host_challenge
+        return bytes([self._cla(), INS_INIT_UPDATE, self.card_keys.kvn, 0, len(host_challenge)]) + host_challenge
+
+    def parse_init_update_resp(self, resp_bin: bytes):
+        """Parse response to INITIALIZE UPDATE."""
+        if len(resp_bin) not in [10+3+8+8, 10+3+16+16, 10+3+8+8+3, 10+3+16+16+3]:
+            raise ValueError('Invalid length of Initialize Update Response')
+        resp = self.constr_iur.parse(resp_bin, s_mode=self.s_mode)
+        self.card_challenge = resp['card_challenge']
+        self.i_param = resp['i_param']
+        # derive session keys and compute cryptograms
+        self.sk = Scp03SessionKeys(self.card_keys, self.host_challenge, self.card_challenge)
+        logger.debug(self.sk)
+        self._compute_cryptograms()
+        # verify computed cryptogram matches received cryptogram
+        if self.card_cryptogram != resp['card_cryptogram']:
+            raise ValueError("card cryptogram doesn't match")
+
+    def gen_ext_auth_apdu(self, security_level: int = 0x01) -> bytes:
+        """Generate EXTERNAL AUTHENTICATE APDU."""
+        self.security_level = security_level
+        header = bytes([self._cla(), INS_EXT_AUTH, self.security_level, 0, self.s_mode])
+        # bypass encryption for EXTERNAL AUTHENTICATE
+        return self.wrap_cmd_apdu(header + self.host_cryptogram, skip_cenc=True)
+
+    def _wrap_cmd_apdu(self, apdu: bytes, skip_cenc: bool = False) -> bytes:
+        """Wrap Command APDU for SCP02: calculate MAC and encrypt."""
+        cla = apdu[0]
+        ins = apdu[1]
+        p1 = apdu[2]
+        p2 = apdu[3]
+        lc = apdu[4]
+        assert lc == len(apdu) - 5
+        cmd_data = apdu[5:]
+
+        if self.do_cenc and not skip_cenc:
+            assert self.do_cmac
+            if lc == 0:
+                # No encryption shall be applied to a command where there is no command data field. In this
+                # case, the encryption counter shall still be incremented
+                self.sk.block_nr += 1
+            else:
+                # data shall be padded as defined in [GPCS] section B.2.3
+                padded_data = pad80(cmd_data, 16)
+                lc = len(padded_data)
+                if lc >= 256:
+                    raise ValueError('Modified Lc (%u) would exceed maximum when appending padding' % (lc))
+                # perform AES-CBC with ICV + S_ENC
+                cmd_data = self.sk._encrypt(padded_data)
+
+        if self.do_cmac:
+            # The length of the command message (Lc) shall be incremented by 8 (in S8 mode) or 16 (in S16
+            # mode) to indicate the inclusion of the C-MAC in the data field of the command message.
+            mlc = lc + self.s_mode
+            if mlc >= 256:
+                raise ValueError('Modified Lc (%u) would exceed maximum when appending %u bytes of mac' % (mlc, self.s_mode))
+            # The class byte shall be modified for the generation or verification of the C-MAC: The logical
+            # channel number shall be set to zero, bit 4 shall be set to 0 and bit 3 shall be set to 1 to indicate
+            # GlobalPlatform proprietary secure messaging.
+            mcla = (cla & 0xF0) | CLA_SM
+            mapdu = bytes([mcla, ins, p1, p2, mlc]) + cmd_data
+            cmac = self.sk.calc_cmac(mapdu)
+            mapdu += cmac[:self.s_mode]
+
+        return mapdu
+
+    def unwrap_rsp_apdu(self, sw: bytes, rsp_apdu: bytes) -> bytes:
+        # No R-MAC shall be generated and no protection shall be applied to a response that includes an error
+        # status word: in this case only the status word shall be returned in the response. All status words
+        # except '9000' and warning status words (i.e. '62xx' and '63xx') shall be interpreted as error status
+        # words.
+        logger.debug("unwrap_rsp_apdu(sw=%s, rsp_apdu=%s)", sw, rsp_apdu)
+        if not self.do_rmac:
+            assert not self.do_renc
+            return rsp_apdu
+
+        if sw != b'\x90\x00' and sw[0] not in [0x62, 0x63]:
+            return rsp_apdu
+        response_data = rsp_apdu[:-self.s_mode]
+        rmac = rsp_apdu[-self.s_mode:]
+        rmac_exp = self.sk.calc_rmac(response_data + sw)[:self.s_mode]
+        if rmac != rmac_exp:
+            raise ValueError("R-MAC value not matching: received: %s, computed: %s" % (rmac, rmac_exp))
+
+        if self.do_renc:
+            # decrypt response data
+            decrypted = self.sk._decrypt(response_data)
+            logger.debug("decrypted: %s", b2h(decrypted))
+            # remove padding
+            response_data = unpad80(decrypted)
+            logger.debug("response_data: %s", b2h(response_data))
+
+        return response_data

pySim/gsm_r.py

@@ -0,0 +1,370 @@
+"""
+The File (and its derived classes) uses the classes of pySim.filesystem in
+order to describe the files specified in UIC Reference P38 T 9001 5.0 "FFFIS for GSM-R SIM Cards"
+"""
+# without this, pylint will fail when inner classes are used
+# within the 'nested' kwarg of our TlvMeta metaclass on python 3.7 :(
+# pylint: disable=undefined-variable
+
+#
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+from pySim.utils import *
+from struct import pack, unpack
+from construct import Struct, Bytes, Int8ub, Int16ub, Int24ub, Int32ub, FlagsEnum
+from construct import Optional as COptional
+
+from pySim.construct import *
+from pySim.profile import CardProfileAddon
+from pySim.filesystem import *
+
+######################################################################
+# DF.EIRENE (FFFIS for GSM-R SIM Cards)
+######################################################################
+
+
+class FuncNTypeAdapter(Adapter):
+    def _decode(self, obj, context, path):
+        bcd = swap_nibbles(b2h(obj))
+        last_digit = int(bcd[-1], 16)
+        return {'functional_number': bcd[:-1],
+                'presentation_of_only_this_fn': bool(last_digit & 4),
+                'permanent_fn': bool(last_digit & 8)}
+
+    def _encode(self, obj, context, path):
+        return 'FIXME'
+
+
+class EF_FN(LinFixedEF):
+    """Section 7.2"""
+    _test_decode = [
+        ( "40315801000010ff01",
+          { "functional_number_and_type": { "functional_number": "04138510000001f",
+                "presentation_of_only_this_fn": True, "permanent_fn": True }, "list_number": 1 } ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6ff1', sfid=None, name='EF.FN',
+                         desc='Functional numbers', rec_len=(9, 9))
+        self._construct = Struct('functional_number_and_type'/FuncNTypeAdapter(Bytes(8)),
+                                 'list_number'/Int8ub)
+
+
+class PlConfAdapter(Adapter):
+    """Section 7.4.3"""
+
+    def _decode(self, obj, context, path):
+        num = int(obj) & 0x7
+        if num == 0:
+            return 'None'
+        if num == 1:
+            return 4
+        if num == 2:
+            return 3
+        if num == 3:
+            return 2
+        if num == 4:
+            return 1
+        if num == 5:
+            return 0
+
+    def _encode(self, obj, context, path):
+        if obj == 'None':
+            return 0
+        obj = int(obj)
+        if obj == 4:
+            return 1
+        if obj == 3:
+            return 2
+        if obj == 2:
+            return 3
+        if obj == 1:
+            return 4
+        if obj == 0:
+            return 5
+
+
+class PlCallAdapter(Adapter):
+    """Section 7.4.12"""
+
+    def _decode(self, obj, context, path):
+        num = int(obj) & 0x7
+        if num == 0:
+            return 'None'
+        if num == 1:
+            return 4
+        if num == 2:
+            return 3
+        if num == 3:
+            return 2
+        if num == 4:
+            return 1
+        if num == 5:
+            return 0
+        if num == 6:
+            return 'B'
+        if num == 7:
+            return 'A'
+
+    def _encode(self, obj, context, path):
+        if obj == 'None':
+            return 0
+        if obj == 4:
+            return 1
+        if obj == 3:
+            return 2
+        if obj == 2:
+            return 3
+        if obj == 1:
+            return 4
+        if obj == 0:
+            return 5
+        if obj == 'B':
+            return 6
+        if obj == 'A':
+            return 7
+
+
+NextTableType = Enum(Byte, decision=0xf0, predefined=0xf1,
+                     num_dial_digits=0xf2, ic=0xf3, empty=0xff)
+
+
+class EF_CallconfC(TransparentEF):
+    """Section 7.3"""
+    _test_de_encode = [
+        ( "026121ffffffffffff1e000a040a010253600795792426f0",
+          { "pl_conf": 3, "conf_nr": "1612ffffffffffff", "max_rand": 30, "n_ack_max": 10,
+            "pl_ack": 1, "n_nested_max": 10, "train_emergency_gid": 1, "shunting_emergency_gid": 2,
+           "imei": "350670599742620f" } ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6ff2', sfid=None, name='EF.CallconfC', size=(24, 24),
+                         desc='Call Configuration of emergency calls Configuration')
+        self._construct = Struct('pl_conf'/PlConfAdapter(Int8ub),
+                                 'conf_nr'/BcdAdapter(Bytes(8)),
+                                 'max_rand'/Int8ub,
+                                 'n_ack_max'/Int16ub,
+                                 'pl_ack'/PlCallAdapter(Int8ub),
+                                 'n_nested_max'/Int8ub,
+                                 'train_emergency_gid'/Int8ub,
+                                 'shunting_emergency_gid'/Int8ub,
+                                 'imei'/BcdAdapter(Bytes(8)))
+
+
+class EF_CallconfI(LinFixedEF):
+    """Section 7.5"""
+
+    def __init__(self):
+        super().__init__(fid='6ff3', sfid=None, name='EF.CallconfI', rec_len=(21, 21),
+                         desc='Call Configuration of emergency calls Information')
+        self._construct = Struct('t_dur'/Int24ub,
+                                 't_relcalc'/Int32ub,
+                                 'pl_call'/PlCallAdapter(Int8ub),
+                                 'cause' /
+                                 FlagsEnum(Int8ub, powered_off=1,
+                                           radio_link_error=2, user_command=5),
+                                 'gcr'/BcdAdapter(Bytes(4)),
+                                 'fnr'/BcdAdapter(Bytes(8)))
+
+
+class EF_Shunting(TransparentEF):
+    """Section 7.6"""
+    _test_de_encode = [
+        ( "03f8ffffff000000", { "common_gid": 3, "shunting_gid": "f8ffffff000000" } ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6ff4', sfid=None,
+                         name='EF.Shunting', desc='Shunting', size=(8, 8))
+        self._construct = Struct('common_gid'/Int8ub,
+                                 'shunting_gid'/HexAdapter(Bytes(7)))
+
+
+class EF_GsmrPLMN(LinFixedEF):
+    """Section 7.7"""
+    _test_de_encode = [
+        ( "22f860f86f8d6f8e01", { "plmn": "228-06", "class_of_network": {
+                                    "supported": { "vbs": True, "vgcs": True, "emlpp": True,
+                                    "fn": True, "eirene": True }, "preference": 0 },
+                                  "ic_incoming_ref_tbl": "6f8d", "outgoing_ref_tbl": "6f8e",
+                                  "ic_table_ref": "01" } ),
+        ( "22f810416f8d6f8e02", { "plmn": "228-01", "class_of_network": {
+                                    "supported": { "vbs": False, "vgcs": False, "emlpp": False,
+                                    "fn": True, "eirene": False }, "preference": 1 },
+                                  "ic_incoming_ref_tbl": "6f8d", "outgoing_ref_tbl": "6f8e",
+                                  "ic_table_ref": "02" } ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6ff5', sfid=None, name='EF.GsmrPLMN',
+                         desc='GSM-R network selection', rec_len=(9, 9))
+        self._construct = Struct('plmn'/PlmnAdapter(Bytes(3)),
+                                 'class_of_network'/BitStruct('supported'/FlagsEnum(BitsInteger(5), vbs=1, vgcs=2, emlpp=4, fn=8, eirene=16),
+                                                              'preference'/BitsInteger(3)),
+                                 'ic_incoming_ref_tbl'/HexAdapter(Bytes(2)),
+                                 'outgoing_ref_tbl'/HexAdapter(Bytes(2)),
+                                 'ic_table_ref'/HexAdapter(Bytes(1)))
+
+
+class EF_IC(LinFixedEF):
+    """Section 7.8"""
+    _test_de_encode = [
+        ( "f06f8e40f10001", { "next_table_type": "decision", "id_of_next_table": "6f8e",
+                              "ic_decision_value": "041f", "network_string_table_index": 1 } ),
+        ( "ffffffffffffff", { "next_table_type": "empty", "id_of_next_table": "ffff",
+                              "ic_decision_value": "ffff", "network_string_table_index": 65535 } ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6f8d', sfid=None, name='EF.IC',
+                         desc='International Code', rec_len=(7, 7))
+        self._construct = Struct('next_table_type'/NextTableType,
+                                 'id_of_next_table'/HexAdapter(Bytes(2)),
+                                 'ic_decision_value'/BcdAdapter(Bytes(2)),
+                                 'network_string_table_index'/Int16ub)
+
+
+class EF_NW(LinFixedEF):
+    """Section 7.9"""
+    _test_de_encode = [
+        ( "47534d2d52204348", "GSM-R CH" ),
+        ( "537769737347534d", "SwissGSM" ),
+        ( "47534d2d52204442", "GSM-R DB" ),
+        ( "47534d2d52524649", "GSM-RRFI" ),
+    ]
+    def __init__(self):
+        super().__init__(fid='6f80', sfid=None, name='EF.NW',
+                         desc='Network Name', rec_len=(8, 8))
+        self._construct = GsmString(8)
+
+
+class EF_Switching(LinFixedEF):
+    """Section 8.4"""
+    _test_de_encode = [
+        ( "f26f87f0ff00", { "next_table_type": "num_dial_digits", "id_of_next_table": "6f87",
+                            "decision_value": "0fff", "string_table_index": 0 } ),
+        ( "f06f8ff1ff01", { "next_table_type": "decision", "id_of_next_table": "6f8f",
+                            "decision_value": "1fff", "string_table_index": 1 } ),
+        ( "f16f89f5ff05", { "next_table_type": "predefined", "id_of_next_table": "6f89",
+                            "decision_value": "5fff", "string_table_index": 5 } ),
+    ]
+    def __init__(self, fid='1234', name='Switching', desc=None):
+        super().__init__(fid=fid, sfid=None,
+                         name=name, desc=desc, rec_len=(6, 6))
+        self._construct = Struct('next_table_type'/NextTableType,
+                                 'id_of_next_table'/HexAdapter(Bytes(2)),
+                                 'decision_value'/BcdAdapter(Bytes(2)),
+                                 'string_table_index'/Int8ub)
+
+
+class EF_Predefined(LinFixedEF):
+    """Section 8.5"""
+    _test_de_encode = [
+        ( "f26f85", 1, { "next_table_type": "num_dial_digits", "id_of_next_table": "6f85" } ),
+        ( "f0ffc8", 2, { "predefined_value1": "0fff", "string_table_index1": 200 } ),
+    ]
+    # header and other records have different structure. WTF !?!
+    construct_first = Struct('next_table_type'/NextTableType,
+                             'id_of_next_table'/HexAdapter(Bytes(2)))
+    construct_others = Struct('predefined_value1'/BcdAdapter(Bytes(2)),
+                              'string_table_index1'/Int8ub)
+
+    def __init__(self, fid='1234', name='Predefined', desc=None):
+        super().__init__(fid=fid, sfid=None,
+                         name=name, desc=desc, rec_len=(3, 3))
+
+    def _decode_record_bin(self, raw_bin_data : bytes, record_nr : int) -> dict:
+        if record_nr == 1:
+            return parse_construct(self.construct_first, raw_bin_data)
+        else:
+            return parse_construct(self.construct_others, raw_bin_data)
+
+    def _encode_record_bin(self, abstract_data : dict, record_nr : int) -> bytearray:
+        r = None
+        if record_nr == 1:
+            r = self.construct_first.build(abstract_data)
+        else:
+            r = self.construct_others.build(abstract_data)
+        return filter_dict(r)
+
+class EF_DialledVals(TransparentEF):
+    """Section 8.6"""
+    _test_de_encode = [
+        ( "ffffff22", { "next_table_type": "empty", "id_of_next_table": "ffff", "dialed_digits": "22" } ),
+        ( "f16f8885", { "next_table_type": "predefined", "id_of_next_table": "6f88", "dialed_digits": "58" }),
+    ]
+    def __init__(self, fid='1234', name='DialledVals', desc=None):
+        super().__init__(fid=fid, sfid=None, name=name, desc=desc, size=(4, 4))
+        self._construct = Struct('next_table_type'/NextTableType,
+                                 'id_of_next_table'/HexAdapter(Bytes(2)),
+                                 'dialed_digits'/BcdAdapter(Bytes(1)))
+
+
+class DF_EIRENE(CardDF):
+    def __init__(self, fid='7fe0', name='DF.EIRENE', desc='GSM-R EIRENE'):
+        super().__init__(fid=fid, name=name, desc=desc)
+        files = [
+            # Section 7.1.6 / Table 10 EIRENE GSM EFs
+            EF_FN(),
+            EF_CallconfC(),
+            EF_CallconfI(),
+            EF_Shunting(),
+            EF_GsmrPLMN(),
+            EF_IC(),
+            EF_NW(),
+
+            # support of the numbering plan
+            EF_Switching(fid='6f8e', name='EF.CT', desc='Call Type'),
+            EF_Switching(fid='6f8f', name='EF.SC', desc='Short Code'),
+            EF_Predefined(fid='6f88', name='EF.FC', desc='Function Code'),
+            EF_Predefined(fid='6f89', name='EF.Service',
+                          desc='VGCS/VBS Service Code'),
+            EF_Predefined(fid='6f8a', name='EF.Call',
+                          desc='First digit of the group ID'),
+            EF_Predefined(fid='6f8b', name='EF.FctTeam',
+                          desc='Call Type 6 Team Type + Team member function'),
+            EF_Predefined(fid='6f92', name='EF.Controller',
+                          desc='Call Type 7 Controller function code'),
+            EF_Predefined(fid='6f8c', name='EF.Gateway',
+                          desc='Access to external networks'),
+            EF_DialledVals(fid='6f81', name='EF.5to8digits',
+                           desc='Call Type 2 User Identity Number length'),
+            EF_DialledVals(fid='6f82', name='EF.2digits',
+                           desc='2 digits input'),
+            EF_DialledVals(fid='6f83', name='EF.8digits',
+                           desc='8 digits input'),
+            EF_DialledVals(fid='6f84', name='EF.9digits',
+                           desc='9 digits input'),
+            EF_DialledVals(fid='6f85', name='EF.SSSSS',
+                           desc='Group call area input'),
+            EF_DialledVals(fid='6f86', name='EF.LLLLL',
+                           desc='Location number Call Type 6'),
+            EF_DialledVals(fid='6f91', name='EF.Location',
+                           desc='Location number Call Type 7'),
+            EF_DialledVals(fid='6f87', name='EF.FreeNumber',
+                           desc='Free Number Call Type 0 and 8'),
+        ]
+        self.add_files(files)
+
+
+class AddonGSMR(CardProfileAddon):
+    """An Addon that can be found on either classic GSM SIM or on UICC to support GSM-R."""
+    def __init__(self):
+        files = [
+            DF_EIRENE()
+        ]
+        super().__init__('GSM-R', desc='Railway GSM', files_in_mf=files)
+
+    def probe(self, card: 'CardBase') -> bool:
+        return card.file_exists(self.files_in_mf[0].fid)

pySim/gsmtap.py

@@ -0,0 +1,214 @@
+# -*- coding: utf-8 -*-
+
+""" Osmocom GSMTAP python implementation.
+GSMTAP is a packet format used for conveying a number of different
+telecom-related protocol traces over UDP.
+"""
+
+#
+# Copyright (C) 2022  Harald Welte <laforge@gnumonks.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import socket
+from construct import Optional as COptional
+from construct import Int8ub, Int8sb, Int32ub, BitStruct, Enum, GreedyBytes, Struct, Switch
+from construct import this, PaddedString
+from pySim.construct import *
+
+# The root definition of GSMTAP can be found at
+# https://cgit.osmocom.org/cgit/libosmocore/tree/include/osmocom/core/gsmtap.h
+
+GSMTAP_UDP_PORT = 4729
+
+# GSMTAP_TYPE_*
+gsmtap_type_construct = Enum(Int8ub,
+                             gsm_um = 0x01,
+                             gsm_abis = 0x02,
+                             gsm_um_burst = 0x03,
+                             sim = 0x04,
+                             tetra_i1 = 0x05,
+                             tetra_i1_burst = 0x06,
+                             wimax_burst = 0x07,
+                             gprs_gb_llc = 0x08,
+                             gprs_gb_sndcp = 0x09,
+                             gmr1_um = 0x0a,
+                             umts_rlc_mac = 0x0b,
+                             umts_rrc = 0x0c,
+                             lte_rrc = 0x0d,
+                             lte_mac = 0x0e,
+                             lte_mac_framed = 0x0f,
+                             osmocore_log = 0x10,
+                             qc_diag = 0x11,
+                             lte_nas = 0x12,
+                             e1_t1 = 0x13)
+
+
+# TYPE_UM_BURST
+gsmtap_subtype_burst_construct = Enum(Int8ub,
+                                      unknown = 0x00,
+                                      fcch = 0x01,
+                                      partial_sch = 0x02,
+                                      sch = 0x03,
+                                      cts_sch = 0x04,
+                                      compact_sch = 0x05,
+                                      normal = 0x06,
+                                      dummy = 0x07,
+                                      access = 0x08,
+                                      none = 0x09)
+
+gsmtap_subtype_wimax_burst_construct = Enum(Int8ub,
+                                            cdma_code = 0x10,
+                                            fch = 0x11,
+                                            ffb = 0x12,
+                                            pdu = 0x13,
+                                            hack = 0x14,
+                                            phy_attributes = 0x15)
+
+# GSMTAP_CHANNEL_*
+gsmtap_subtype_um_construct = Enum(Int8ub,
+                                   unknown = 0x00,
+                                   bcch = 0x01,
+                                   ccch = 0x02,
+                                   rach = 0x03,
+                                   agch = 0x04,
+                                   pch = 0x05,
+                                   sdcch = 0x06,
+                                   sdcch4 = 0x07,
+                                   sdcch8 = 0x08,
+                                   facch_f = 0x09,
+                                   facch_h = 0x0a,
+                                   pacch = 0x0b,
+                                   cbch52 = 0x0c,
+                                   pdtch = 0x0d,
+                                   ptcch = 0x0e,
+                                   cbch51 = 0x0f,
+                                   voice_f = 0x10,
+                                   voice_h = 0x11)
+
+
+# GSMTAP_SIM_*
+gsmtap_subtype_sim_construct = Enum(Int8ub,
+                                    apdu = 0x00,
+                                    atr = 0x01,
+                                    pps_req = 0x02,
+                                    pps_rsp = 0x03,
+                                    tpdu_hdr = 0x04,
+                                    tpdu_cmd = 0x05,
+                                    tpdu_rsp = 0x06,
+                                    tpdu_sw = 0x07)
+
+gsmtap_subtype_tetra_construct = Enum(Int8ub,
+                                      bsch = 0x01,
+                                      aach = 0x02,
+                                      sch_hu = 0x03,
+                                      sch_hd = 0x04,
+                                      sch_f = 0x05,
+                                      bnch = 0x06,
+                                      stch = 0x07,
+                                      tch_f = 0x08,
+                                      dmo_sch_s = 0x09,
+                                      dmo_sch_h = 0x0a,
+                                      dmo_sch_f = 0x0b,
+                                      dmo_stch = 0x0c,
+                                      dmo_tch = 0x0d)
+
+gsmtap_subtype_gmr1_construct = Enum(Int8ub,
+                                     unknown = 0x00,
+                                     bcch = 0x01,
+                                     ccch = 0x02,
+                                     pch = 0x03,
+                                     agch = 0x04,
+                                     bach = 0x05,
+                                     rach = 0x06,
+                                     cbch = 0x07,
+                                     sdcch = 0x08,
+                                     tachh = 0x09,
+                                     gbch = 0x0a,
+                                     tch3 = 0x10,
+                                     tch6 = 0x14,
+                                     tch9 = 0x18)
+
+gsmtap_subtype_e1t1_construct = Enum(Int8ub,
+                                     lapd = 0x01,
+                                     fr = 0x02,
+                                     raw = 0x03,
+                                     trau16 = 0x04,
+                                     trau8 = 0x05)
+
+gsmtap_arfcn_construct = BitStruct('pcs'/Flag, 'uplink'/Flag, 'arfcn'/BitsInteger(14))
+
+gsmtap_hdr_construct = Struct('version'/Int8ub,
+                              'hdr_len'/Int8ub,
+                              'type'/gsmtap_type_construct,
+                              'timeslot'/Int8ub,
+                              'arfcn'/gsmtap_arfcn_construct,
+                              'signal_dbm'/Int8sb,
+                              'snr_db'/Int8sb,
+                              'frame_nr'/Int32ub,
+                              'sub_type'/Switch(this.type, {
+                                                'gsm_um': gsmtap_subtype_um_construct,
+                                                'gsm_um_burst': gsmtap_subtype_burst_construct,
+                                                'sim': gsmtap_subtype_sim_construct,
+                                                'tetra_i1': gsmtap_subtype_tetra_construct,
+                                                'tetra_i1_burst': gsmtap_subtype_tetra_construct,
+                                                'wimax_burst': gsmtap_subtype_wimax_burst_construct,
+                                                'gmr1_um': gsmtap_subtype_gmr1_construct,
+                                                'e1_t1': gsmtap_subtype_e1t1_construct,
+                                                }),
+                              'antenna_nr'/Int8ub,
+                              'sub_slot'/Int8ub,
+                              'res'/Int8ub,
+                              'body'/GreedyBytes)
+
+osmocore_log_ts_construct = Struct('sec'/Int32ub, 'usec'/Int32ub)
+osmocore_log_level_construct = Enum(Int8ub, debug=1, info=3, notice=5, error=7, fatal=8)
+gsmtap_osmocore_log_hdr_construct = Struct('ts'/osmocore_log_ts_construct,
+                                           'proc_name'/PaddedString(16, 'ascii'),
+                                           'pid'/Int32ub,
+                                           'level'/osmocore_log_level_construct,
+                                           Bytes(3),
+                                           'subsys'/PaddedString(16, 'ascii'),
+                                           'src_file'/Struct('name'/PaddedString(32, 'ascii'), 'line_nr'/Int32ub))
+
+
+class GsmtapMessage:
+    """Class whose objects represent a single GSMTAP message. Can encode and decode messages."""
+    def __init__(self, encoded = None):
+        self.encoded = encoded
+        self.decoded = None
+
+    def decode(self):
+        self.decoded = parse_construct(gsmtap_hdr_construct, self.encoded)
+        return self.decoded
+
+    def encode(self, decoded):
+        self.encoded = gsmtap_hdr_construct.build(decoded)
+        return self.encoded
+
+class GsmtapSource:
+    def __init__(self, bind_ip:str='127.0.0.1', bind_port:int=4729):
+        self.bind_ip = bind_ip
+        self.bind_port = bind_port
+        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        self.sock.bind((self.bind_ip, self.bind_port))
+
+    def read_packet(self) -> GsmtapMessage:
+        data, addr = self.sock.recvfrom(1024)
+        gsmtap_msg = GsmtapMessage(data)
+        gsmtap_msg.decode()
+        if gsmtap_msg.decoded['version'] != 0x02:
+            raise ValueError('Unknown GSMTAP version 0x%02x' % gsmtap_msg.decoded['version'])
+        return gsmtap_msg.decoded, addr

pySim/iso7816_4.py

@@ -0,0 +1,62 @@
+# coding=utf-8
+"""Utilities / Functions related to ISO 7816-4
+
+(C) 2022 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from construct import GreedyBytes, GreedyString
+from pySim.construct import *
+from pySim.utils import *
+from pySim.filesystem import *
+from pySim.tlv import *
+
+# Table 91 + Section 8.2.1.2
+class ApplicationId(BER_TLV_IE, tag=0x4f):
+    _construct = GreedyBytes
+
+# Table 91
+class ApplicationLabel(BER_TLV_IE, tag=0x50):
+    _construct = GreedyBytes
+
+# Table 91 + Section 5.3.1.2
+class FileReference(BER_TLV_IE, tag=0x51):
+    _construct = GreedyBytes
+
+# Table 91
+class CommandApdu(BER_TLV_IE, tag=0x52):
+    _construct = GreedyBytes
+
+# Table 91
+class DiscretionaryData(BER_TLV_IE, tag=0x53):
+    _construct = GreedyBytes
+
+# Table 91
+class DiscretionaryTemplate(BER_TLV_IE, tag=0x73):
+    _construct = GreedyBytes
+
+# Table 91 + RFC1738 / RFC2396
+class URL(BER_TLV_IE, tag=0x5f50):
+    _construct = GreedyString('ascii')
+
+# Table 91
+class ApplicationRelatedDOSet(BER_TLV_IE, tag=0x61):
+    _construct = GreedyBytes
+
+# Section 8.2.1.3 Application Template
+class ApplicationTemplate(BER_TLV_IE, tag=0x61, nested=[ApplicationId, ApplicationLabel, FileReference,
+                          CommandApdu, DiscretionaryData, DiscretionaryTemplate, URL,
+                          ApplicationRelatedDOSet]):
+    pass

pySim/jsonpath.py

@@ -0,0 +1,46 @@
+"""JSONpath utility functions as needed within pysim.
+
+As pySim-sell has the ability to represent SIM files as JSON strings,
+adding JSONpath allows us to conveniently modify individual sub-fields
+of a file or record in its JSON representation.
+"""
+
+import jsonpath_ng
+
+# (C) 2021 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+def js_path_find(js_dict, js_path):
+    """Find/Match a JSON path within a given JSON-serializable dict.
+    Args:
+        js_dict : JSON-serializable dict to operate on
+        js_path : JSONpath string
+    Returns: Result of the JSONpath expression
+    """
+    jsonpath_expr = jsonpath_ng.parse(js_path)
+    return jsonpath_expr.find(js_dict)
+
+
+def js_path_modify(js_dict, js_path, new_val):
+    """Find/Match a JSON path within a given JSON-serializable dict.
+    Args:
+        js_dict : JSON-serializable dict to operate on
+        js_path : JSONpath string
+        new_val : New value for field in js_dict at js_path
+    """
+    jsonpath_expr = jsonpath_ng.parse(js_path)
+    jsonpath_expr.find(js_dict)
+    jsonpath_expr.update(js_dict, new_val)

pySim/legacy/ts_31_102.py

@@ -0,0 +1,134 @@
+# -*- coding: utf-8 -*-
+
+"""
+Various constants from 3GPP TS 31.102 V17.9.0 usd by *legacy* code
+"""
+
+#
+# Copyright (C) 2020 Supreeth Herle <herlesupreeth@gmail.com>
+# Copyright (C) 2021-2023 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+EF_USIM_ADF_map = {
+    'LI': '6F05',
+    'ARR': '6F06',
+    'IMSI': '6F07',
+    'Keys': '6F08',
+    'KeysPS': '6F09',
+    'DCK': '6F2C',
+    'HPPLMN': '6F31',
+    'CNL': '6F32',
+    'ACMmax': '6F37',
+    'UST': '6F38',
+    'ACM': '6F39',
+    'FDN': '6F3B',
+    'SMS': '6F3C',
+    'GID1': '6F3E',
+    'GID2': '6F3F',
+    'MSISDN': '6F40',
+    'PUCT': '6F41',
+    'SMSP': '6F42',
+    'SMSS': '6F42',
+    'CBMI': '6F45',
+    'SPN': '6F46',
+    'SMSR': '6F47',
+    'CBMID': '6F48',
+    'SDN': '6F49',
+    'EXT2': '6F4B',
+    'EXT3': '6F4C',
+    'BDN': '6F4D',
+    'EXT5': '6F4E',
+    'CCP2': '6F4F',
+    'CBMIR': '6F50',
+    'EXT4': '6F55',
+    'EST': '6F56',
+    'ACL': '6F57',
+    'CMI': '6F58',
+    'START-HFN': '6F5B',
+    'THRESHOLD': '6F5C',
+    'PLMNwAcT': '6F60',
+    'OPLMNwAcT': '6F61',
+    'HPLMNwAcT': '6F62',
+    'PSLOCI': '6F73',
+    'ACC': '6F78',
+    'FPLMN': '6F7B',
+    'LOCI': '6F7E',
+    'ICI': '6F80',
+    'OCI': '6F81',
+    'ICT': '6F82',
+    'OCT': '6F83',
+    'AD': '6FAD',
+    'VGCS': '6FB1',
+    'VGCSS': '6FB2',
+    'VBS': '6FB3',
+    'VBSS': '6FB4',
+    'eMLPP': '6FB5',
+    'AAeM': '6FB6',
+    'ECC': '6FB7',
+    'Hiddenkey': '6FC3',
+    'NETPAR': '6FC4',
+    'PNN': '6FC5',
+    'OPL': '6FC6',
+    'MBDN': '6FC7',
+    'EXT6': '6FC8',
+    'MBI': '6FC9',
+    'MWIS': '6FCA',
+    'CFIS': '6FCB',
+    'EXT7': '6FCC',
+    'SPDI': '6FCD',
+    'MMSN': '6FCE',
+    'EXT8': '6FCF',
+    'MMSICP': '6FD0',
+    'MMSUP': '6FD1',
+    'MMSUCP': '6FD2',
+    'NIA': '6FD3',
+    'VGCSCA': '6FD4',
+    'VBSCA': '6FD5',
+    'GBAP': '6FD6',
+    'MSK': '6FD7',
+    'MUK': '6FD8',
+    'EHPLMN': '6FD9',
+    'GBANL': '6FDA',
+    'EHPLMNPI': '6FDB',
+    'LRPLMNSI': '6FDC',
+    'NAFKCA': '6FDD',
+    'SPNI': '6FDE',
+    'PNNI': '6FDF',
+    'NCP-IP': '6FE2',
+    'EPSLOCI': '6FE3',
+    'EPSNSC': '6FE4',
+    'UFC': '6FE6',
+    'UICCIARI': '6FE7',
+    'NASCONFIG': '6FE8',
+    'PWC': '6FEC',
+    'FDNURI': '6FED',
+    'BDNURI': '6FEE',
+    'SDNURI': '6FEF',
+    'IWL': '6FF0',
+    'IPS': '6FF1',
+    'IPD': '6FF2',
+    'ePDGId': '6FF3',
+    'ePDGSelection': '6FF4',
+    'ePDGIdEm': '6FF5',
+    'ePDGSelectionEm': '6FF6',
+}
+
+LOCI_STATUS_map = {
+    0:	'updated',
+    1:	'not updated',
+    2:	'plmn not allowed',
+    3:	'locatation area not allowed'
+}

pySim/legacy/ts_31_103.py

@@ -0,0 +1,43 @@
+# -*- coding: utf-8 -*-
+
+"""
+Various constants from 3GPP TS 31.103 V16.1.0 used by *legacy* code only
+"""
+
+# Copyright (C) 2020 Supreeth Herle <herlesupreeth@gmail.com>
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+EF_ISIM_ADF_map = {
+    'IST': '6F07',
+    'IMPI': '6F02',
+    'DOMAIN': '6F03',
+    'IMPU': '6F04',
+    'AD': '6FAD',
+    'ARR': '6F06',
+    'PCSCF': '6F09',
+    'GBAP': '6FD5',
+    'GBANL': '6FD7',
+    'NAFKCA': '6FDD',
+    'UICCIARI': '6FE7',
+    'SMS': '6F3C',
+    'SMSS': '6F43',
+    'SMSR': '6F47',
+    'SMSP': '6F42',
+    'FromPreferred': '6FF7',
+    'IMSConfigData': '6FF8',
+    'XCAPConfigData': '6FFC',
+    'WebRTCURI': '6FFA'
+}

pySim/legacy/ts_51_011.py

@@ -0,0 +1,257 @@
+# -*- coding: utf-8 -*-
+
+""" Various constants from 3GPP TS 51.011 used by *legacy* code only.
+
+This will likely be removed in future versions of pySim.  Please instead
+use the pySim.filesystem class model with the various profile/application
+specific extension modules.
+"""
+
+# Copyright (C) 2017 Alexander.Chemeris <Alexander.Chemeris@gmail.com>
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+MF_num = '3F00'
+
+DF_num = {
+    'TELECOM': '7F10',
+
+    'GSM': '7F20',
+    'IS-41': '7F22',
+    'FP-CTS': '7F23',
+
+    'GRAPHICS': '5F50',
+
+    'IRIDIUM': '5F30',
+    'GLOBST': '5F31',
+    'ICO': '5F32',
+    'ACeS': '5F33',
+
+    'EIA/TIA-553': '5F40',
+    'CTS': '5F60',
+    'SOLSA': '5F70',
+
+    'MExE': '5F3C',
+}
+
+EF_num = {
+    # MF
+    'ICCID': '2FE2',
+    'ELP': '2F05',
+    'DIR': '2F00',
+
+    # DF_TELECOM
+    'ADN': '6F3A',
+    'FDN': '6F3B',
+    'SMS': '6F3C',
+    'CCP': '6F3D',
+    'MSISDN': '6F40',
+    'SMSP': '6F42',
+    'SMSS': '6F43',
+    'LND': '6F44',
+    'SMSR': '6F47',
+    'SDN': '6F49',
+    'EXT1': '6F4A',
+    'EXT2': '6F4B',
+    'EXT3': '6F4C',
+    'BDN': '6F4D',
+    'EXT4': '6F4E',
+    'CMI': '6F58',
+    'ECCP': '6F4F',
+
+    # DF_GRAPHICS
+    'IMG': '4F20',
+
+    # DF_SoLSA
+    'SAI': '4F30',
+    'SLL': '4F31',
+
+    # DF_MExE
+    'MExE-ST': '4F40',
+    'ORPK': '4F41',
+    'ARPK': '4F42',
+    'TPRPK': '4F43',
+
+    # DF_GSM
+    'LP': '6F05',
+    'IMSI': '6F07',
+    'Kc': '6F20',
+    'DCK': '6F2C',
+    'PLMNsel': '6F30',
+    'HPPLMN': '6F31',
+    'CNL': '6F32',
+    'ACMmax': '6F37',
+    'SST': '6F38',
+    'ACM': '6F39',
+    'GID1': '6F3E',
+    'GID2': '6F3F',
+    'PUCT': '6F41',
+    'CBMI': '6F45',
+    'SPN': '6F46',
+    'CBMID': '6F48',
+    'BCCH': '6F74',
+    'ACC': '6F78',
+    'FPLMN': '6F7B',
+    'LOCI': '6F7E',
+    'AD': '6FAD',
+    'PHASE': '6FAE',
+    'VGCS': '6FB1',
+    'VGCSS': '6FB2',
+    'VBS': '6FB3',
+    'VBSS': '6FB4',
+    'eMLPP': '6FB5',
+    'AAeM': '6FB6',
+    'ECC': '6FB7',
+    'CBMIR': '6F50',
+    'NIA': '6F51',
+    'KcGPRS': '6F52',
+    'LOCIGPRS': '6F53',
+    'SUME': '6F54',
+    'PLMNwAcT': '6F60',
+    'OPLMNwAcT': '6F61',
+    # Figure 8 names it HPLMNAcT, but in the text it's names it HPLMNwAcT
+    'HPLMNAcT': '6F62',
+    'HPLMNwAcT': '6F62',
+    'CPBCCH': '6F63',
+    'INVSCAN': '6F64',
+    'PNN': '6FC5',
+    'OPL': '6FC6',
+    'MBDN': '6FC7',
+    'EXT6': '6FC8',
+    'MBI': '6FC9',
+    'MWIS': '6FCA',
+    'CFIS': '6FCB',
+    'EXT7': '6FCC',
+    'SPDI': '6FCD',
+    'MMSN': '6FCE',
+    'EXT8': '6FCF',
+    'MMSICP': '6FD0',
+    'MMSUP': '6FD1',
+    'MMSUCP': '6FD2',
+}
+
+DF = {
+    'TELECOM':  [MF_num, DF_num['TELECOM']],
+
+    'GSM':      [MF_num, DF_num['GSM']],
+    'IS-41':    [MF_num, DF_num['IS-41']],
+    'FP-CTS':   [MF_num, DF_num['FP-CTS']],
+
+    'GRAPHICS': [MF_num, DF_num['GRAPHICS']],
+
+    'IRIDIUM':  [MF_num, DF_num['IRIDIUM']],
+    'GLOBST':   [MF_num, DF_num['GLOBST']],
+    'ICO':      [MF_num, DF_num['ICO']],
+    'ACeS':     [MF_num, DF_num['ACeS']],
+
+    'EIA/TIA-553': [MF_num, DF_num['EIA/TIA-553']],
+    'CTS':      [MF_num, DF_num['CTS']],
+    'SoLSA':    [MF_num, DF_num['SOLSA']],
+
+    'MExE':     [MF_num, DF_num['MExE']],
+}
+
+
+EF = {
+    'ICCID':  [MF_num, EF_num['ICCID']],
+    'ELP':    [MF_num, EF_num['ELP']],
+    'DIR':    [MF_num, EF_num['DIR']],
+
+    'ADN':    DF['TELECOM']+[EF_num['ADN']],
+    'FDN':    DF['TELECOM']+[EF_num['FDN']],
+    'SMS':    DF['TELECOM']+[EF_num['SMS']],
+    'CCP':    DF['TELECOM']+[EF_num['CCP']],
+    'MSISDN': DF['TELECOM']+[EF_num['MSISDN']],
+    'SMSP':   DF['TELECOM']+[EF_num['SMSP']],
+    'SMSS':   DF['TELECOM']+[EF_num['SMSS']],
+    'LND':    DF['TELECOM']+[EF_num['LND']],
+    'SMSR':   DF['TELECOM']+[EF_num['SMSR']],
+    'SDN':    DF['TELECOM']+[EF_num['SDN']],
+    'EXT1':   DF['TELECOM']+[EF_num['EXT1']],
+    'EXT2':   DF['TELECOM']+[EF_num['EXT2']],
+    'EXT3':   DF['TELECOM']+[EF_num['EXT3']],
+    'BDN':    DF['TELECOM']+[EF_num['BDN']],
+    'EXT4':   DF['TELECOM']+[EF_num['EXT4']],
+    'CMI':    DF['TELECOM']+[EF_num['CMI']],
+    'ECCP':   DF['TELECOM']+[EF_num['ECCP']],
+
+    'IMG':    DF['GRAPHICS']+[EF_num['IMG']],
+
+    'SAI':    DF['SoLSA']+[EF_num['SAI']],
+    'SLL':    DF['SoLSA']+[EF_num['SLL']],
+
+    'MExE-ST': DF['MExE']+[EF_num['MExE-ST']],
+    'ORPK':   DF['MExE']+[EF_num['ORPK']],
+    'ARPK':   DF['MExE']+[EF_num['ARPK']],
+    'TPRPK':  DF['MExE']+[EF_num['TPRPK']],
+
+    'LP':     DF['GSM']+[EF_num['LP']],
+    'IMSI':   DF['GSM']+[EF_num['IMSI']],
+    'Kc':     DF['GSM']+[EF_num['Kc']],
+    'DCK':    DF['GSM']+[EF_num['DCK']],
+    'PLMNsel': DF['GSM']+[EF_num['PLMNsel']],
+    'HPPLMN': DF['GSM']+[EF_num['HPPLMN']],
+    'CNL':    DF['GSM']+[EF_num['CNL']],
+    'ACMmax': DF['GSM']+[EF_num['ACMmax']],
+    'SST':    DF['GSM']+[EF_num['SST']],
+    'ACM':    DF['GSM']+[EF_num['ACM']],
+    'GID1':   DF['GSM']+[EF_num['GID1']],
+    'GID2':   DF['GSM']+[EF_num['GID2']],
+    'PUCT':   DF['GSM']+[EF_num['PUCT']],
+    'CBMI':   DF['GSM']+[EF_num['CBMI']],
+    'SPN':    DF['GSM']+[EF_num['SPN']],
+    'CBMID':  DF['GSM']+[EF_num['CBMID']],
+    'BCCH':   DF['GSM']+[EF_num['BCCH']],
+    'ACC':    DF['GSM']+[EF_num['ACC']],
+    'FPLMN':  DF['GSM']+[EF_num['FPLMN']],
+    'LOCI':   DF['GSM']+[EF_num['LOCI']],
+    'AD':     DF['GSM']+[EF_num['AD']],
+    'PHASE':  DF['GSM']+[EF_num['PHASE']],
+    'VGCS':   DF['GSM']+[EF_num['VGCS']],
+    'VGCSS':  DF['GSM']+[EF_num['VGCSS']],
+    'VBS':    DF['GSM']+[EF_num['VBS']],
+    'VBSS':   DF['GSM']+[EF_num['VBSS']],
+    'eMLPP':  DF['GSM']+[EF_num['eMLPP']],
+    'AAeM':   DF['GSM']+[EF_num['AAeM']],
+    'ECC':    DF['GSM']+[EF_num['ECC']],
+    'CBMIR':  DF['GSM']+[EF_num['CBMIR']],
+    'NIA':    DF['GSM']+[EF_num['NIA']],
+    'KcGPRS': DF['GSM']+[EF_num['KcGPRS']],
+    'LOCIGPRS': DF['GSM']+[EF_num['LOCIGPRS']],
+    'SUME':   DF['GSM']+[EF_num['SUME']],
+    'PLMNwAcT': DF['GSM']+[EF_num['PLMNwAcT']],
+    'OPLMNwAcT': DF['GSM']+[EF_num['OPLMNwAcT']],
+    # Figure 8 names it HPLMNAcT, but in the text it's names it HPLMNwAcT
+    'HPLMNAcT': DF['GSM']+[EF_num['HPLMNAcT']],
+    'HPLMNwAcT': DF['GSM']+[EF_num['HPLMNAcT']],
+    'CPBCCH': DF['GSM']+[EF_num['CPBCCH']],
+    'INVSCAN': DF['GSM']+[EF_num['INVSCAN']],
+    'PNN':    DF['GSM']+[EF_num['PNN']],
+    'OPL':    DF['GSM']+[EF_num['OPL']],
+    'MBDN':   DF['GSM']+[EF_num['MBDN']],
+    'EXT6':   DF['GSM']+[EF_num['EXT6']],
+    'MBI':    DF['GSM']+[EF_num['MBI']],
+    'MWIS':   DF['GSM']+[EF_num['MWIS']],
+    'CFIS':   DF['GSM']+[EF_num['CFIS']],
+    'EXT7':   DF['GSM']+[EF_num['EXT7']],
+    'SPDI':   DF['GSM']+[EF_num['SPDI']],
+    'MMSN':   DF['GSM']+[EF_num['MMSN']],
+    'EXT8':   DF['GSM']+[EF_num['EXT8']],
+    'MMSICP': DF['GSM']+[EF_num['MMSICP']],
+    'MMSUP':  DF['GSM']+[EF_num['MMSUP']],
+    'MMSUCP': DF['GSM']+[EF_num['MMSUCP']],
+}
+

pySim/legacy/utils.py

@@ -0,0 +1,332 @@
+# -*- coding: utf-8 -*-
+
+""" pySim: various utilities only used by legacy tools (pySim-{prog,read})
+"""
+
+# Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from pySim.utils import Hexstr, rpad, enc_plmn, h2i, i2s, s2h
+from pySim.utils import dec_xplmn_w_act, dec_xplmn, dec_mcc_from_plmn, dec_mnc_from_plmn
+
+def hexstr_to_Nbytearr(s, nbytes):
+    return [s[i:i+(nbytes*2)] for i in range(0, len(s), (nbytes*2))]
+
+def format_xplmn_w_act(hexstr):
+    s = ""
+    for rec_data in hexstr_to_Nbytearr(hexstr, 5):
+        rec_info = dec_xplmn_w_act(rec_data)
+        if rec_info['mcc'] == "" and rec_info['mnc'] == "":
+            rec_str = "unused"
+        else:
+            rec_str = "MCC: %s MNC: %s AcT: %s" % (
+                rec_info['mcc'], rec_info['mnc'], ", ".join(rec_info['act']))
+        s += "\t%s # %s\n" % (rec_data, rec_str)
+    return s
+
+
+def format_xplmn(hexstr: Hexstr) -> str:
+    s = ""
+    for rec_data in hexstr_to_Nbytearr(hexstr, 3):
+        rec_info = dec_xplmn(rec_data)
+        if not rec_info['mcc'] and not rec_info['mnc']:
+            rec_str = "unused"
+        else:
+            rec_str = "MCC: %s MNC: %s" % (rec_info['mcc'], rec_info['mnc'])
+        s += "\t%s # %s\n" % (rec_data, rec_str)
+    return s
+
+
+def format_ePDGSelection(hexstr):
+    ePDGSelection_info_tag_chars = 2
+    ePDGSelection_info_tag_str = hexstr[:2]
+    s = ""
+    # Minimum length
+    len_chars = 2
+    # TODO: Need to determine length properly - definite length support only
+    # Inconsistency in spec: 3GPP TS 31.102 version 15.2.0 Release 15, 4.2.104
+    # As per spec, length is 5n, n - number of PLMNs
+    # But, each PLMN entry is made of PLMN (3 Bytes) + ePDG Priority (2 Bytes) + ePDG FQDN format (1 Byte)
+    # Totalling to 6 Bytes, maybe length should be 6n
+    len_str = hexstr[ePDGSelection_info_tag_chars:ePDGSelection_info_tag_chars+len_chars]
+
+    # Not programmed scenario
+    if int(len_str, 16) == 255 or int(ePDGSelection_info_tag_str, 16) == 255:
+        len_chars = 0
+        ePDGSelection_info_tag_chars = 0
+    if len_str[0] == '8':
+        # The bits 7 to 1 denotes the number of length octets if length > 127
+        if int(len_str[1]) > 0:
+            # Update number of length octets
+            len_chars = len_chars * int(len_str[1])
+            len_str = hexstr[ePDGSelection_info_tag_chars:len_chars]
+
+    content_str = hexstr[ePDGSelection_info_tag_chars+len_chars:]
+    # Right pad to prevent index out of range - multiple of 6 bytes
+    content_str = rpad(content_str, len(content_str) +
+                       (12 - (len(content_str) % 12)))
+    for rec_data in hexstr_to_Nbytearr(content_str, 6):
+        rec_info = dec_ePDGSelection(rec_data)
+        if rec_info['mcc'] == 0xFFF and rec_info['mnc'] == 0xFFF:
+            rec_str = "unused"
+        else:
+            rec_str = "MCC: %03d MNC: %03d ePDG Priority: %s ePDG FQDN format: %s" % \
+                (rec_info['mcc'], rec_info['mnc'],
+                 rec_info['epdg_priority'], rec_info['epdg_fqdn_format'])
+        s += "\t%s # %s\n" % (rec_data, rec_str)
+    return s
+
+def enc_st(st, service, state=1):
+    """
+    Encodes the EF S/U/IST/EST and returns the updated Service Table
+
+    Parameters:
+            st - Current value of SIM/USIM/ISIM Service Table
+            service - Service Number to encode as activated/de-activated
+            state - 1 mean activate, 0 means de-activate
+
+    Returns:
+            s - Modified value of SIM/USIM/ISIM Service Table
+
+    Default values:
+            - state: 1 - Sets the particular Service bit to 1
+    """
+    st_bytes = [st[i:i+2] for i in range(0, len(st), 2)]
+
+    s = ""
+    # Check whether the requested service is present in each byte
+    for i in range(0, len(st_bytes)):
+        # Byte i contains info about Services num (8i+1) to num (8i+8)
+        if service in range((8*i) + 1, (8*i) + 9):
+            byte = int(st_bytes[i], 16)
+            # Services in each byte are in order MSB to LSB
+            # MSB - Service (8i+8)
+            # LSB - Service (8i+1)
+            mod_byte = 0x00
+            # Copy bit by bit contents of byte to mod_byte with modified bit
+            # for requested service
+            for j in range(1, 9):
+                mod_byte = mod_byte >> 1
+                if service == (8*i) + j:
+                    mod_byte = state == 1 and mod_byte | 0x80 or mod_byte & 0x7f
+                else:
+                    mod_byte = byte & 0x01 == 0x01 and mod_byte | 0x80 or mod_byte & 0x7f
+                byte = byte >> 1
+
+            s += ('%02x' % (mod_byte))
+        else:
+            s += st_bytes[i]
+
+    return s
+
+
+def dec_st(st, table="sim") -> str:
+    """
+    Parses the EF S/U/IST and prints the list of available services in EF S/U/IST
+    """
+
+    if table == "isim":
+        from pySim.ts_31_103 import EF_IST_map
+        lookup_map = EF_IST_map
+    elif table == "usim":
+        from pySim.ts_31_102 import EF_UST_map
+        lookup_map = EF_UST_map
+    else:
+        from pySim.ts_51_011 import EF_SST_map
+        lookup_map = EF_SST_map
+
+    st_bytes = [st[i:i+2] for i in range(0, len(st), 2)]
+
+    avail_st = ""
+    # Get each byte and check for available services
+    for i in range(0, len(st_bytes)):
+        # Byte i contains info about Services num (8i+1) to num (8i+8)
+        byte = int(st_bytes[i], 16)
+        # Services in each byte are in order MSB to LSB
+        # MSB - Service (8i+8)
+        # LSB - Service (8i+1)
+        for j in range(1, 9):
+            if byte & 0x01 == 0x01 and ((8*i) + j in lookup_map):
+                # Byte X contains info about Services num (8X-7) to num (8X)
+                # bit = 1: service available
+                # bit = 0: service not available
+                avail_st += '\tService %d - %s\n' % (
+                    (8*i) + j, lookup_map[(8*i) + j])
+            byte = byte >> 1
+    return avail_st
+
+
+def enc_ePDGSelection(hexstr, mcc, mnc, epdg_priority='0001', epdg_fqdn_format='00'):
+    """
+    Encode ePDGSelection so it can be stored at EF.ePDGSelection or EF.ePDGSelectionEm.
+    See 3GPP TS 31.102 version 15.2.0 Release 15, section 4.2.104 and 4.2.106.
+
+    Default values:
+            - epdg_priority: '0001' - 1st Priority
+            - epdg_fqdn_format: '00' - Operator Identifier FQDN
+    """
+
+    plmn1 = enc_plmn(mcc, mnc) + epdg_priority + epdg_fqdn_format
+    # TODO: Handle encoding of Length field for length more than 127 Bytes
+    content = '80' + ('%02x' % (len(plmn1)//2)) + plmn1
+    content = rpad(content, len(hexstr))
+    return content
+
+
+def dec_ePDGSelection(sixhexbytes):
+    """
+    Decode ePDGSelection to get EF.ePDGSelection or EF.ePDGSelectionEm.
+    See 3GPP TS 31.102 version 15.2.0 Release 15, section 4.2.104 and 4.2.106.
+    """
+
+    res = {'mcc': 0, 'mnc': 0, 'epdg_priority': 0, 'epdg_fqdn_format': ''}
+    plmn_chars = 6
+    epdg_priority_chars = 4
+    epdg_fqdn_format_chars = 2
+    # first three bytes (six ascii hex chars)
+    plmn_str = sixhexbytes[:plmn_chars]
+    # two bytes after first three bytes
+    epdg_priority_str = sixhexbytes[plmn_chars:plmn_chars +
+                                    epdg_priority_chars]
+    # one byte after first five bytes
+    epdg_fqdn_format_str = sixhexbytes[plmn_chars +
+                                       epdg_priority_chars:plmn_chars + epdg_priority_chars + epdg_fqdn_format_chars]
+    res['mcc'] = dec_mcc_from_plmn(plmn_str)
+    res['mnc'] = dec_mnc_from_plmn(plmn_str)
+    res['epdg_priority'] = epdg_priority_str
+    res['epdg_fqdn_format'] = epdg_fqdn_format_str == '00' and 'Operator Identifier FQDN' or 'Location based FQDN'
+    return res
+
+
+def first_TLV_parser(bytelist):
+    '''
+    first_TLV_parser([0xAA, 0x02, 0xAB, 0xCD, 0xFF, 0x00]) -> (170, 2, [171, 205])
+
+    parses first TLV format record in a list of bytelist
+    returns a 3-Tuple: Tag, Length, Value
+    Value is a list of bytes
+    parsing of length is ETSI'style 101.220
+    '''
+    Tag = bytelist[0]
+    if bytelist[1] == 0xFF:
+        Len = bytelist[2]*256 + bytelist[3]
+        Val = bytelist[4:4+Len]
+    else:
+        Len = bytelist[1]
+        Val = bytelist[2:2+Len]
+    return (Tag, Len, Val)
+
+
+def TLV_parser(bytelist):
+    '''
+    TLV_parser([0xAA, ..., 0xFF]) -> [(T, L, [V]), (T, L, [V]), ...]
+
+    loops on the input list of bytes with the "first_TLV_parser()" function
+    returns a list of 3-Tuples
+    '''
+    ret = []
+    while len(bytelist) > 0:
+        T, L, V = first_TLV_parser(bytelist)
+        if T == 0xFF:
+            # padding bytes
+            break
+        ret.append((T, L, V))
+        # need to manage length of L
+        if L > 0xFE:
+            bytelist = bytelist[L+4:]
+        else:
+            bytelist = bytelist[L+2:]
+    return ret
+
+
+def dec_addr_tlv(hexstr):
+    """
+    Decode hex string to get EF.P-CSCF Address or EF.ePDGId or EF.ePDGIdEm.
+    See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.8, 4.2.102 and 4.2.104.
+    """
+
+    # Convert from hex str to int bytes list
+    addr_tlv_bytes = h2i(hexstr)
+
+    # Get list of tuples containing parsed TLVs
+    tlvs = TLV_parser(addr_tlv_bytes)
+
+    for tlv in tlvs:
+        # tlv = (T, L, [V])
+        # T = Tag
+        # L = Length
+        # [V] = List of value
+
+        # Invalid Tag value scenario
+        if tlv[0] != 0x80:
+            continue
+
+        # Empty field - Zero length
+        if tlv[1] == 0:
+            continue
+
+        # Uninitialized field
+        if all([v == 0xff for v in tlv[2]]):
+            continue
+
+        # First byte in the value has the address type
+        addr_type = tlv[2][0]
+        # TODO: Support parsing of IPv6
+        # Address Type: 0x00 (FQDN), 0x01 (IPv4), 0x02 (IPv6), other (Reserved)
+        if addr_type == 0x00:  # FQDN
+            # Skip address tye byte i.e. first byte in value list
+            content = tlv[2][1:]
+            return (i2s(content), '00')
+
+        elif addr_type == 0x01:  # IPv4
+            # Skip address tye byte i.e. first byte in value list
+            # Skip the unused byte in Octect 4 after address type byte as per 3GPP TS 31.102
+            ipv4 = tlv[2][2:]
+            content = '.'.join(str(x) for x in ipv4)
+            return (content, '01')
+        else:
+            raise ValueError("Invalid address type")
+
+    return (None, None)
+
+
+def enc_addr_tlv(addr, addr_type='00'):
+    """
+    Encode address TLV object used in EF.P-CSCF Address, EF.ePDGId and EF.ePDGIdEm.
+    See 3GPP TS 31.102 version 13.4.0 Release 13, section 4.2.8, 4.2.102 and 4.2.104.
+
+    Default values:
+      - addr_type: 00 - FQDN format of Address
+    """
+
+    s = ""
+
+    # TODO: Encoding of IPv6 address
+    if addr_type == '00':  # FQDN
+        hex_str = s2h(addr)
+        s += '80' + ('%02x' % ((len(hex_str)//2)+1)) + '00' + hex_str
+    elif addr_type == '01':  # IPv4
+        ipv4_list = addr.split('.')
+        ipv4_str = ""
+        for i in ipv4_list:
+            ipv4_str += ('%02x' % (int(i)))
+
+        # Unused bytes shall be set to 'ff'. i.e 4th Octet after Address Type is not used
+        # IPv4 Address is in octet 5 to octet 8 of the TLV data object
+        s += '80' + ('%02x' % ((len(ipv4_str)//2)+2)) + '01' + 'ff' + ipv4_str
+
+    return s

pySim/ota.py

@@ -0,0 +1,441 @@
+"""Code related to SIM/UICC OTA according to TS 102 225 + TS 31.115."""
+
+# (C) 2021-2023 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import zlib
+import abc
+import struct
+from typing import Optional
+from construct import Enum, Int8ub, Int16ub, Struct, Bytes, GreedyBytes, BitsInteger, BitStruct
+from construct import Flag, Padding, Switch, this
+
+from pySim.construct import *
+from pySim.utils import b2h
+from pySim.sms import UserDataHeader
+
+# ETS TS 102 225 gives the general command structure and the dialects for CAT_TP, TCP/IP and HTTPS
+# 3GPP TS 31.115 gives the dialects for SMS-PP, SMS-CB, USSD and HTTP
+
+# CPI CPL CHI CHL SPI KIc KID TAR CNTR PCNTR RC/CC/DS data
+
+#                                           CAT_TP  TCP/IP          SMS
+# CPI                                       0x01    0x01            =IEIa=70,len=0
+# CHI                                       NULL    NULL            NULL
+# CPI, CPL and CHL included in RC/CC/DS     true    true
+# RPI                                       0x02    0x02            =IEIa=71,len=0
+# RHI                                       NULL    NULL
+# RPI, RPL and RHL included in RC/CC/DS     true    true
+# packet-id                             0-bf,ff     0-bf,ff
+# identification packet                     false   102 225 tbl 6
+
+# KVN 1..f; KI1=KIc, KI2=KID, KI3=DEK
+
+# ETSI TS 102 225 Table 5 + 3GPP TS 31.115 Section 7
+ResponseStatus = Enum(Int8ub, por_ok=0, rc_cc_ds_failed=1, cntr_low=2, cntr_high=3,
+                      cntr_blocked=4, ciphering_error=5, undefined_security_error=6,
+                      insufficient_memory=7, more_time_needed=8, tar_unknown=9,
+                      insufficient_security_level=0x0A,
+                      actual_response_sms_submit=0x0B,
+                      actual_response_ussd=0x0C)
+
+# ETSI TS 102 226 Section 5.1.2
+CompactRemoteResp = Struct('number_of_commands'/Int8ub,
+                           'last_status_word'/HexAdapter(Bytes(2)),
+                           'last_response_data'/HexAdapter(GreedyBytes))
+
+RC_CC_DS = Enum(BitsInteger(2), no_rc_cc_ds=0, rc=1, cc=2, ds=3)
+
+# TS 102 225 Section 5.1.1 + TS 31.115 Section 4.2
+SPI = BitStruct(  # first octet
+    Padding(3),
+    'counter'/Enum(BitsInteger(2), no_counter=0, counter_no_replay_or_seq=1,
+                   counter_must_be_higher=2, counter_must_be_lower=3),
+    'ciphering'/Flag,
+    'rc_cc_ds'/RC_CC_DS,
+    # second octet
+    Padding(2),
+    'por_in_submit'/Flag,
+    'por_shall_be_ciphered'/Flag,
+    'por_rc_cc_ds'/RC_CC_DS,
+    'por'/Enum(BitsInteger(2), no_por=0,
+               por_required=1, por_only_when_error=2)
+)
+
+# TS 102 225 Section 5.1.2
+KIC = BitStruct('key'/BitsInteger(4),
+                'algo'/Enum(BitsInteger(4), implicit=0, single_des=1, triple_des_cbc2=5, triple_des_cbc3=9,
+                            aes_cbc=2)
+                )
+
+# TS 102 225 Section 5.1.3.1
+KID_CC = BitStruct('key'/BitsInteger(4),
+                   'algo'/Enum(BitsInteger(4), implicit=0, single_des=1, triple_des_cbc2=5, triple_des_cbc3=9,
+                               aes_cmac=2)
+                   )
+
+# TS 102 225 Section 5.1.3.2
+KID_RC = BitStruct('key'/BitsInteger(4),
+                   'algo'/Enum(BitsInteger(4), implicit=0, crc16=1, crc32=5, proprietary=3)
+                   )
+
+SmsCommandPacket = Struct('cmd_pkt_len'/Int16ub,
+                          'cmd_hdr_len'/Int8ub,
+                          'spi'/SPI,
+                          'kic'/KIC,
+                          'kid'/Switch(this.spi.rc_cc_ds, {'cc': KID_CC, 'rc': KID_RC }),
+                          'tar'/Bytes(3),
+                          'secured_data'/GreedyBytes)
+
+class OtaKeyset:
+    """The OTA related data (key material, counter) to be used in encrypt/decrypt."""
+    def __init__(self, algo_crypt: str, kic_idx: int, kic: bytes,
+                 algo_auth: str, kid_idx: int, kid: bytes, cntr: int = 0):
+        self.algo_crypt = algo_crypt
+        self.kic = bytes(kic)
+        self.kic_idx = kic_idx
+        self.algo_auth = algo_auth
+        self.kid = bytes(kid)
+        self.kid_idx = kid_idx
+        self.cntr = cntr
+
+    @property
+    def auth(self):
+        """Return an instance of the matching OtaAlgoAuth."""
+        return OtaAlgoAuth.from_keyset(self)
+
+    @property
+    def crypt(self):
+        """Return an instance of the matching OtaAlgoCrypt."""
+        return OtaAlgoCrypt.from_keyset(self)
+
+class OtaCheckError(Exception):
+    pass
+
+class OtaDialect(abc.ABC):
+    """Base Class for OTA dialects such as SMS, BIP, ..."""
+
+    def _compute_sig_len(self, spi:SPI):
+        if spi['rc_cc_ds'] == 'no_rc_cc_ds':
+            return 0
+        if spi['rc_cc_ds'] == 'rc': # CRC-32
+            return 4
+        if spi['rc_cc_ds'] == 'cc': # Cryptographic Checksum (CC)
+            # TODO: this is not entirely correct, as in AES case it could be 4 or 8
+            return 8
+        raise ValueError("Invalid rc_cc_ds: %s" % spi['rc_cc_ds'])
+
+    @abc.abstractmethod
+    def encode_cmd(self, otak: OtaKeyset, tar: bytes, spi: dict, apdu: bytes) -> bytes:
+        pass
+
+    @abc.abstractmethod
+    def decode_resp(self, otak: OtaKeyset, spi: dict, apdu: bytes) -> (object, Optional["CompactRemoteResp"]):
+        """Decode a response into a response packet and, if indicted (by a
+        response status of `"por_ok"`) a decoded response.
+
+        The response packet's common characteristics are not fully determined,
+        and (so far) completely proprietary per dialect."""
+
+
+from Cryptodome.Cipher import DES, DES3, AES
+from Cryptodome.Hash import CMAC
+
+class OtaAlgo(abc.ABC):
+    iv = property(lambda self: bytes([0] * self.blocksize))
+    blocksize = None
+    enum_name = None
+
+    @staticmethod
+    def _get_padding(in_len: int, multiple: int, padding: int = 0):
+        """Return padding bytes towards multiple of N."""
+        if in_len % multiple == 0:
+            return b''
+        pad_cnt = multiple - (in_len % multiple)
+        return b'\x00' * pad_cnt
+
+    @staticmethod
+    def _pad_to_multiple(indat: bytes, multiple: int, padding: int = 0):
+        """Pad input bytes to multiple of N."""
+        return indat + OtaAlgo._get_padding(len(indat), multiple, padding)
+
+    def pad_to_blocksize(self, indat: bytes, padding: int = 0):
+        """Pad the given input data to multiple of the cipher block size."""
+        return self._pad_to_multiple(indat, self.blocksize, padding)
+
+    def __init__(self, otak: OtaKeyset):
+        self.otak = otak
+
+    def __str__(self):
+        return self.__class__.__name__
+
+class OtaAlgoCrypt(OtaAlgo, abc.ABC):
+    def __init__(self, otak: OtaKeyset):
+        if self.enum_name != otak.algo_crypt:
+            raise ValueError('Cannot use algorithm %s with key for %s' % (self.enum_name, otak.algo_crypt))
+        super().__init__(otak)
+
+    def encrypt(self, data:bytes) -> bytes:
+        """Encrypt given input bytes using the key material given in constructor."""
+        padded_data = self.pad_to_blocksize(data)
+        return self._encrypt(padded_data)
+
+    def decrypt(self, data:bytes) -> bytes:
+        """Decrypt given input bytes using the key material given in constructor."""
+        return self._decrypt(data)
+
+    @abc.abstractmethod
+    def _encrypt(self, data:bytes) -> bytes:
+        """Actual implementation, to be implemented by derived class."""
+
+    @abc.abstractmethod
+    def _decrypt(self, data:bytes) -> bytes:
+        """Actual implementation, to be implemented by derived class."""
+
+    @classmethod
+    def from_keyset(cls, otak: OtaKeyset) -> 'OtaAlgoCrypt':
+        """Resolve the class for the encryption algorithm of otak and instantiate it."""
+        for subc in cls.__subclasses__():
+            if subc.enum_name == otak.algo_crypt:
+                return subc(otak)
+        raise ValueError('No implementation for crypt algorithm %s' % otak.algo_auth)
+
+class OtaAlgoAuth(OtaAlgo, abc.ABC):
+    def __init__(self, otak: OtaKeyset):
+        if self.enum_name != otak.algo_auth:
+            raise ValueError('Cannot use algorithm %s with key for %s' % (self.enum_name, otak.algo_crypt))
+        super().__init__(otak)
+
+    def sign(self, data:bytes) -> bytes:
+        """Compute the CC/CR check bytes for the input data using key material
+        given in constructor."""
+        padded_data = self.pad_to_blocksize(data)
+        sig = self._sign(padded_data)
+        return sig
+
+    def check_sig(self, data:bytes, cc_received:bytes):
+        """Compute the CC/CR check bytes for the input data and compare against cc_received."""
+        cc = self.sign(data)
+        if cc_received != cc:
+            raise OtaCheckError('Received CC (%s) != Computed CC (%s)' % (b2h(cc_received), b2h(cc)))
+
+    @abc.abstractmethod
+    def _sign(self, data:bytes) -> bytes:
+        """Actual implementation, to be implemented by derived class."""
+        pass
+
+    @classmethod
+    def from_keyset(cls, otak: OtaKeyset) -> 'OtaAlgoAuth':
+        """Resolve the class for the authentication algorithm of otak and instantiate it."""
+        for subc in cls.__subclasses__():
+            if subc.enum_name == otak.algo_auth:
+                return subc(otak)
+        raise ValueError('No implementation for auth algorithm %s' % otak.algo_auth)
+
+class OtaAlgoCryptDES(OtaAlgoCrypt):
+    """DES is insecure.  For backwards compatibility with pre-Rel8"""
+    name = 'DES'
+    enum_name = 'single_des'
+    blocksize = 8
+    def _encrypt(self, data:bytes) -> bytes:
+        cipher = DES.new(self.otak.kic, DES.MODE_CBC, self.iv)
+        return cipher.encrypt(data)
+
+    def _decrypt(self, data:bytes) -> bytes:
+        cipher = DES.new(self.otak.kic, DES.MODE_CBC, self.iv)
+        return cipher.decrypt(data)
+
+class OtaAlgoAuthDES(OtaAlgoAuth):
+    """DES is insecure.  For backwards compatibility with pre-Rel8"""
+    name = 'DES'
+    enum_name = 'single_des'
+    blocksize = 8
+    def _sign(self, data:bytes) -> bytes:
+        cipher = DES.new(self.otak.kid, DES.MODE_CBC, self.iv)
+        ciph = cipher.encrypt(data)
+        return ciph[len(ciph) - 8:]
+
+class OtaAlgoCryptDES3(OtaAlgoCrypt):
+    name = '3DES'
+    enum_name = 'triple_des_cbc2'
+    blocksize = 8
+    def _encrypt(self, data:bytes) -> bytes:
+        cipher = DES3.new(self.otak.kic, DES3.MODE_CBC, self.iv)
+        return cipher.encrypt(data)
+
+    def _decrypt(self, data:bytes) -> bytes:
+        cipher = DES3.new(self.otak.kic, DES3.MODE_CBC, self.iv)
+        return cipher.decrypt(data)
+
+class OtaAlgoAuthDES3(OtaAlgoAuth):
+    name = '3DES'
+    enum_name = 'triple_des_cbc2'
+    blocksize = 8
+    def _sign(self, data:bytes) -> bytes:
+        cipher = DES3.new(self.otak.kid, DES3.MODE_CBC, self.iv)
+        ciph = cipher.encrypt(data)
+        return ciph[len(ciph) - 8:]
+
+class OtaAlgoCryptAES(OtaAlgoCrypt):
+    name = 'AES'
+    enum_name = 'aes_cbc'
+    blocksize = 16 # TODO: is this needed?
+    def _encrypt(self, data:bytes) -> bytes:
+        cipher = AES.new(self.otak.kic, AES.MODE_CBC, self.iv)
+        return cipher.encrypt(data)
+
+    def _decrypt(self, data:bytes) -> bytes:
+        cipher = AES.new(self.otak.kic, AES.MODE_CBC, self.iv)
+        return cipher.decrypt(data)
+
+class OtaAlgoAuthAES(OtaAlgoAuth):
+    name = 'AES'
+    enum_name = 'aes_cmac'
+    blocksize = 1 # AES CMAC doesn't need any padding by us
+    def _sign(self, data:bytes) -> bytes:
+        cmac = CMAC.new(self.otak.kid, ciphermod=AES, mac_len=8)
+        cmac.update(data)
+        ciph = cmac.digest()
+        return ciph[len(ciph) - 8:]
+
+
+
+class OtaDialectSms(OtaDialect):
+    """OTA dialect for SMS based transport, as described in 3GPP TS 31.115."""
+    SmsResponsePacket = Struct('rpl'/Int16ub,
+                               'rhl'/Int8ub,
+                               'tar'/Bytes(3),
+                               'cntr'/Bytes(5),
+                               'pcntr'/Int8ub,
+                               'response_status'/ResponseStatus,
+                               'cc_rc'/Bytes(this.rhl-10),
+                               'secured_data'/GreedyBytes)
+
+    def encode_cmd(self, otak: OtaKeyset, tar: bytes, spi: dict, apdu: bytes) -> bytes:
+        # length of signature in octets
+        len_sig = self._compute_sig_len(spi)
+        pad_cnt = 0
+        if spi['ciphering']: # ciphering is requested
+            # append padding bytes to end up with blocksize
+            len_cipher = 6 + len_sig + len(apdu)
+            padding = otak.crypt._get_padding(len_cipher, otak.crypt.blocksize)
+            pad_cnt = len(padding)
+            apdu += padding
+
+        kic = {'key': otak.kic_idx, 'algo': otak.algo_crypt}
+        kid = {'key': otak.kid_idx, 'algo': otak.algo_auth}
+
+        # CHL = number of octets from (and including) SPI to the end of RC/CC/DS
+        # 13 == SPI(2) + KIc(1) + KId(1) + TAR(3) + CNTR(5) + PCNTR(1)
+        chl = 13 + len_sig
+
+        # CHL + SPI (+ KIC + KID)
+        c = Struct('chl'/Int8ub, 'spi'/SPI, 'kic'/KIC, 'kid'/KID_CC, 'tar'/Bytes(3))
+        part_head = c.build({'chl': chl, 'spi':spi, 'kic':kic, 'kid':kid, 'tar':tar})
+        #print("part_head: %s" % b2h(part_head))
+
+        # CNTR + PCNTR (CNTR not used)
+        part_cnt = otak.cntr.to_bytes(5, 'big') + pad_cnt.to_bytes(1, 'big')
+        #print("part_cnt: %s" % b2h(part_cnt))
+
+        envelope_data = part_head + part_cnt + apdu
+        #print("envelope_data: %s" % b2h(envelope_data))
+
+        # 2-byte CPL. CPL is part of RC/CC/CPI to end of secured data, including any padding for ciphering
+        # CPL from and including CPI to end of secured data, including any padding for ciphering
+        cpl = len(envelope_data) + len_sig
+        envelope_data = cpl.to_bytes(2, 'big') + envelope_data
+        #print("envelope_data with cpl: %s" % b2h(envelope_data))
+
+        if spi['rc_cc_ds'] == 'cc':
+            cc = otak.auth.sign(envelope_data)
+            envelope_data = part_cnt + cc + apdu
+        elif spi['rc_cc_ds'] == 'rc':
+            # CRC32
+            crc32 = zlib.crc32(envelope_data) & 0xffffffff
+            envelope_data = part_cnt + crc32.to_bytes(4, 'big') + apdu
+        elif spi['rc_cc_ds'] == 'no_rc_cc_ds':
+            envelope_data = part_cnt + apdu
+        else:
+            raise ValueError("Invalid rc_cc_ds: %s" % spi['rc_cc_ds'])
+
+        #print("envelope_data with sig: %s" % b2h(envelope_data))
+
+        # encrypt as needed
+        if spi['ciphering']: # ciphering is requested
+            ciph = otak.crypt.encrypt(envelope_data)
+            envelope_data = part_head + ciph
+            # prefix with another CPL
+            cpl = len(envelope_data)
+            envelope_data = cpl.to_bytes(2, 'big') + envelope_data
+        else:
+            envelope_data = part_head + envelope_data
+
+        #print("envelope_data: %s" % b2h(envelope_data))
+
+        return envelope_data
+
+    def decode_resp(self, otak: OtaKeyset, spi: dict, data: bytes) -> ("OtaDialectSms.SmsResponsePacket", Optional["CompactRemoteResp"]):
+        if isinstance(data, str):
+            data = h2b(data)
+        # plain-text POR:   027100000e0ab000110000000000000001612f
+        # UDHL RPI IEDLa  RPL  RHL TAR    CNTR       PCNTR STS
+        # 02   71  00     000e 0a  b00011 0000000000 00    00  01 612f
+        # POR with CC:      027100001612b000110000000000000055f47118381175fb01612f
+        # POR with CC+CIPH: 027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
+        if data[0] != 0x02:
+            raise ValueError('Unexpected UDL=0x%02x' % data[0])
+        udhd, remainder = UserDataHeader.from_bytes(data)
+        if not udhd.has_ie(0x71):
+            raise ValueError('RPI 0x71 not found in UDH')
+        rph_rhl_tar = remainder[:6] # RPH+RHL+TAR; not ciphered
+        res = self.SmsResponsePacket.parse(remainder)
+
+        if spi['por_shall_be_ciphered']:
+            # decrypt
+            ciphered_part = remainder[6:]
+            deciph = otak.crypt.decrypt(ciphered_part)
+            temp_data = rph_rhl_tar + deciph
+            res = self.SmsResponsePacket.parse(temp_data)
+            # remove specified number of padding bytes, if any
+            if res['pcntr'] != 0:
+                # this conditional is needed as python [:-0] renders an empty return!
+                res['secured_data'] = res['secured_data'][:-res['pcntr']]
+            remainder = temp_data
+
+        # is there a CC/RC present?
+        len_sig = res['rhl'] - 10
+        if spi['por_rc_cc_ds'] == 'no_rc_cc_ds':
+            if len_sig:
+                raise OtaCheckError('No RC/CC/DS requested, but len_sig=%u' % len_sig)
+        elif spi['por_rc_cc_ds'] == 'cc':
+            # verify signature
+            # UDH is part of CC/RC!
+            udh = data[:3]
+            # RPL, RHL, TAR, CNTR, PCNTR and STSare part of CC/RC
+            rpl_rhl_tar_cntr_pcntr_sts = remainder[:13]
+            # remove the CC/RC bytes
+            temp_data = udh + rpl_rhl_tar_cntr_pcntr_sts + remainder[13+len_sig:]
+            otak.auth.check_sig(temp_data, res['cc_rc'])
+        # TODO: CRC
+        else:
+            raise OtaCheckError('Unknown por_rc_cc_ds: %s' % spi['por_rc_cc_ds'])
+
+        # TODO: ExpandedRemoteResponse according to TS 102 226 5.2.2
+        if res.response_status == 'por_ok':
+            dec = CompactRemoteResp.parse(res['secured_data'])
+        else:
+            dec = None
+        return (res, dec)

pySim/profile.py

@@ -0,0 +1,195 @@
+# -*- coding: utf-8 -*-
+
+""" pySim: tell old 2G SIMs apart from UICC
+"""
+
+#
+# (C) 2021 by Sysmocom s.f.m.c. GmbH
+# All Rights Reserved
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import abc
+import operator
+from typing import List
+
+from pySim.commands import SimCardCommands
+from pySim.filesystem import CardApplication, interpret_sw
+from pySim.utils import all_subclasses
+
+def _mf_select_test(scc: SimCardCommands,
+                    cla_byte: str, sel_ctrl: str,
+                    fids: List[str]) -> bool:
+    cla_byte_bak = scc.cla_byte
+    sel_ctrl_bak = scc.sel_ctrl
+    scc.reset_card()
+
+    scc.cla_byte = cla_byte
+    scc.sel_ctrl = sel_ctrl
+    rc = True
+    try:
+        for fid in fids:
+            scc.select_file(fid)
+    except:
+        rc = False
+
+    scc.reset_card()
+    scc.cla_byte = cla_byte_bak
+    scc.sel_ctrl = sel_ctrl_bak
+    return rc
+
+
+def match_uicc(scc: SimCardCommands) -> bool:
+    """ Try to access MF via UICC APDUs (3GPP TS 102.221), if this works, the
+    card is considered a UICC card.
+    """
+    return _mf_select_test(scc, "00", "0004", ["3f00"])
+
+
+def match_sim(scc: SimCardCommands) -> bool:
+    """ Try to access MF via 2G APDUs (3GPP TS 11.11), if this works, the card
+    is also a simcard. This will be the case for most UICC cards, but there may
+    also be plain UICC cards without 2G support as well.
+    """
+    return _mf_select_test(scc, "a0", "0000", ["3f00"])
+
+
+def match_ruim(scc: SimCardCommands) -> bool:
+    """ Try to access MF/DF.CDMA via 2G APDUs (3GPP TS 11.11), if this works,
+    the card is considered an R-UIM card for CDMA.
+    """
+    return _mf_select_test(scc, "a0", "0000", ["3f00", "7f25"])
+
+
+class CardProfile:
+    """A Card Profile describes a card, it's filesystem hierarchy, an [initial] list of
+    applications as well as profile-specific SW and shell commands.  Every card has
+    one card profile, but there may be multiple applications within that profile."""
+
+    def __init__(self, name, **kw):
+        """
+        Args:
+                desc (str) : Description
+                files_in_mf : List of CardEF instances present in MF
+                applications : List of CardApplications present on card
+                sw : List of status word definitions
+                shell_cmdsets : List of cmd2 shell command sets of profile-specific commands
+                cla : class byte that should be used with cards of this profile
+                sel_ctrl : selection control bytes class byte that should be used with cards of this profile
+                addons: List of optional CardAddons that a card of this profile might have
+        """
+        self.name = name
+        self.desc = kw.get("desc", None)
+        self.files_in_mf = kw.get("files_in_mf", [])
+        self.sw = kw.get("sw", {})
+        self.applications = kw.get("applications", [])
+        self.shell_cmdsets = kw.get("shell_cmdsets", [])
+        self.cla = kw.get("cla", "00")
+        self.sel_ctrl = kw.get("sel_ctrl", "0004")
+        # list of optional addons that a card of this profile might have
+        self.addons = kw.get("addons", [])
+
+    def __str__(self):
+        return self.name
+
+    def add_application(self, app: CardApplication):
+        """Add an application to a card profile.
+
+        Args:
+                app : CardApplication instance to be added to profile
+        """
+        self.applications.append(app)
+
+    def interpret_sw(self, sw: str):
+        """Interpret a given status word within the profile.
+
+        Args:
+                sw : Status word as string of 4 hex digits
+
+        Returns:
+                Tuple of two strings
+        """
+        return interpret_sw(self.sw, sw)
+
+    @staticmethod
+    def decode_select_response(data_hex: str) -> object:
+        """Decode the response to a SELECT command.
+
+        This is the fall-back method which doesn't perform any decoding. It mostly
+        exists so specific derived classes can overload it for actual decoding.
+        This method is implemented in the profile and is only used when application
+        specific decoding cannot be performed (no ADF is selected).
+
+        Args:
+                data_hex: Hex string of the select response
+        """
+        return data_hex
+
+    @staticmethod
+    @abc.abstractmethod
+    def match_with_card(scc: SimCardCommands) -> bool:
+        """Check if the specific profile matches the card. This method is a
+        placeholder that is overloaded by specific dirived classes. The method
+        actively probes the card to make sure the profile class matches the
+        physical card. This usually also means that the card is reset during
+        the process, so this method must not be called at random times. It may
+        only be called on startup.
+
+        Args:
+                scc: SimCardCommands class
+        Returns:
+                match = True, no match = False
+        """
+        return False
+
+    @staticmethod
+    def pick(scc: SimCardCommands):
+        profiles = list(all_subclasses(CardProfile))
+        profiles.sort(key=operator.attrgetter('ORDER'))
+
+        for p in profiles:
+            if p.match_with_card(scc):
+                return p()
+
+        return None
+
+    def add_addon(self, addon: 'CardProfileAddon'):
+        assert addon not in self.addons
+        # we don't install any additional files, as that is happening in the RuntimeState.
+        self.addons.append(addon)
+
+class CardProfileAddon(abc.ABC):
+    """A Card Profile Add-on is something that is not a card application or a full stand-alone
+    card profile, but an add-on to an existing profile.  Think of GSM-R specific files existing
+    on what is otherwise a SIM or USIM+SIM card."""
+
+    def __init__(self, name: str, **kw):
+        """
+        Args:
+                desc (str) : Description
+                files_in_mf : List of CardEF instances present in MF
+                shell_cmdsets : List of cmd2 shell command sets of profile-specific commands
+        """
+        self.name = name
+        self.desc = kw.get("desc", None)
+        self.files_in_mf = kw.get("files_in_mf", [])
+        self.shell_cmdsets = kw.get("shell_cmdsets", [])
+
+    def __str__(self):
+        return self.name
+
+    @abc.abstractmethod
+    def probe(self, card: 'CardBase') -> bool:
+        """Probe a given card to determine whether or not this add-on is present/supported."""

pySim/runtime.py

@@ -0,0 +1,545 @@
+# coding=utf-8
+"""Representation of the runtime state of an application like pySim-shell.
+"""
+
+# (C) 2021 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import Optional, Tuple
+
+from pySim.utils import h2b, i2h, is_hex, bertlv_parse_one, Hexstr
+from pySim.exceptions import *
+from pySim.filesystem import *
+
+def lchan_nr_from_cla(cla: int) -> int:
+    """Resolve the logical channel number from the CLA byte."""
+    # TS 102 221 10.1.1 Coding of Class Byte
+    if cla >> 4 in [0x0, 0xA, 0x8]:
+        # Table 10.3
+        return cla & 0x03
+    if cla & 0xD0 in [0x40, 0xC0]:
+        # Table 10.4a
+        return 4 + (cla & 0x0F)
+    raise ValueError('Could not determine logical channel for CLA=%2X' % cla)
+
+class RuntimeState:
+    """Represent the runtime state of a session with a card."""
+
+    def __init__(self, card: 'CardBase', profile: 'CardProfile'):
+        """
+        Args:
+            card : pysim.cards.Card instance
+            profile : CardProfile instance
+        """
+        self.mf = CardMF(profile=profile)
+        self.card = card
+        self.profile = profile
+        self.lchan = {}
+        # the basic logical channel always exists
+        self.lchan[0] = RuntimeLchan(0, self)
+
+        # make sure the class and selection control bytes, which are specified
+        # by the card profile are used
+        self.card.set_apdu_parameter(
+            cla=self.profile.cla, sel_ctrl=self.profile.sel_ctrl)
+
+        for addon_cls in self.profile.addons:
+            addon = addon_cls()
+            if addon.probe(self.card):
+                print("Detected %s Add-on \"%s\"" % (self.profile, addon))
+                for f in addon.files_in_mf:
+                    self.mf.add_file(f)
+
+        # go back to MF before the next steps (addon probing might have changed DF)
+        self.lchan[0].select('MF')
+
+        # add application ADFs + MF-files from profile
+        apps = self._match_applications()
+        for a in apps:
+            if a.adf:
+                self.mf.add_application_df(a.adf)
+        for f in self.profile.files_in_mf:
+            self.mf.add_file(f)
+        self.conserve_write = True
+
+        # make sure that when the runtime state is created, the card is also
+        # in a defined state.
+        self.reset()
+
+    def _match_applications(self):
+        """match the applications from the profile with applications on the card"""
+        apps_profile = self.profile.applications
+
+        # When the profile does not feature any applications, then we are done already
+        if not apps_profile:
+            return []
+
+        # Read AIDs from card and match them against the applications defined by the
+        # card profile
+        aids_card = self.card.read_aids()
+        apps_taken = []
+        if aids_card:
+            aids_taken = []
+            print("AIDs on card:")
+            for a in aids_card:
+                for f in apps_profile:
+                    if f.aid in a:
+                        print(" %s: %s (EF.DIR)" % (f.name, a))
+                        aids_taken.append(a)
+                        apps_taken.append(f)
+            aids_unknown = set(aids_card) - set(aids_taken)
+            for a in aids_unknown:
+                print(" unknown: %s (EF.DIR)" % a)
+        else:
+            print("warning: EF.DIR seems to be empty!")
+
+        # Some card applications may not be registered in EF.DIR, we will actively
+        # probe for those applications
+        for f in sorted(set(apps_profile) - set(apps_taken), key=str):
+            try:
+                # we can not use the lchan provided methods select, or select_file
+                # since those method work on an already finished file model. At
+                # this point we are still in the initialization process, so it is
+                # no problem when we access the card object directly without caring
+                # about updating other states. For normal selects at runtime, the
+                # caller must use the lchan provided methods select or select_file!
+                _data, sw = self.card.select_adf_by_aid(f.aid)
+                self.selected_adf = f
+                if sw == "9000":
+                    print(" %s: %s" % (f.name, f.aid))
+                    apps_taken.append(f)
+            except (SwMatchError, ProtocolError):
+                pass
+        return apps_taken
+
+    def reset(self, cmd_app=None) -> Hexstr:
+        """Perform physical card reset and obtain ATR.
+        Args:
+            cmd_app : Command Application State (for unregistering old file commands)
+        """
+        # delete all lchan != 0 (basic lchan)
+        for lchan_nr in list(self.lchan.keys()):
+            if lchan_nr == 0:
+                continue
+            del self.lchan[lchan_nr]
+        atr = i2h(self.card.reset())
+        # select MF to reset internal state and to verify card really works
+        self.lchan[0].select('MF', cmd_app)
+        self.lchan[0].selected_adf = None
+        return atr
+
+    def add_lchan(self, lchan_nr: int) -> 'RuntimeLchan':
+        """Add a logical channel to the runtime state.  You shouldn't call this
+        directly but always go through RuntimeLchan.add_lchan()."""
+        if lchan_nr in self.lchan.keys():
+            raise ValueError('Cannot create already-existing lchan %d' % lchan_nr)
+        self.lchan[lchan_nr] = RuntimeLchan(lchan_nr, self)
+        return self.lchan[lchan_nr]
+
+    def del_lchan(self, lchan_nr: int):
+        if lchan_nr in self.lchan.keys():
+            del self.lchan[lchan_nr]
+            return True
+        else:
+            return False
+
+    def get_lchan_by_cla(self, cla) -> Optional['RuntimeLchan']:
+        lchan_nr = lchan_nr_from_cla(cla)
+        if lchan_nr in self.lchan.keys():
+            return self.lchan[lchan_nr]
+        else:
+            return None
+
+
+class RuntimeLchan:
+    """Represent the runtime state of a logical channel with a card."""
+
+    def __init__(self, lchan_nr: int, rs: RuntimeState):
+        self.lchan_nr = lchan_nr
+        self.rs = rs
+        self.scc = self.rs.card._scc.fork_lchan(lchan_nr)
+
+        # File reference data
+        self.selected_file = self.rs.mf
+        self.selected_adf = None
+        self.selected_file_fcp = None
+        self.selected_file_fcp_hex = None
+
+    def add_lchan(self, lchan_nr: int) -> 'RuntimeLchan':
+        """Add a new logical channel from the current logical channel. Just affects
+        internal state, doesn't actually open a channel with the UICC."""
+        new_lchan = self.rs.add_lchan(lchan_nr)
+        # See TS 102 221 Table 8.3
+        if self.lchan_nr != 0:
+            new_lchan.selected_file = self.get_cwd()
+            new_lchan.selected_adf = self.selected_adf
+        return new_lchan
+
+    def selected_file_descriptor_byte(self) -> dict:
+        return self.selected_file_fcp['file_descriptor']['file_descriptor_byte']
+
+    def selected_file_shareable(self) -> bool:
+        return self.selected_file_descriptor_byte()['shareable']
+
+    def selected_file_structure(self) -> str:
+        return self.selected_file_descriptor_byte()['structure']
+
+    def selected_file_type(self) -> str:
+        return self.selected_file_descriptor_byte()['file_type']
+
+    def selected_file_num_of_rec(self) -> Optional[int]:
+        return self.selected_file_fcp['file_descriptor'].get('num_of_rec')
+
+    def get_cwd(self) -> CardDF:
+        """Obtain the current working directory.
+
+        Returns:
+            CardDF instance
+        """
+        if isinstance(self.selected_file, CardDF):
+            return self.selected_file
+        else:
+            return self.selected_file.parent
+
+    def get_application_df(self) -> Optional[CardADF]:
+        """Obtain the currently selected application DF (if any).
+
+        Returns:
+            CardADF() instance or None"""
+        # iterate upwards from selected file; check if any is an ADF
+        node = self.selected_file
+        while node.parent != node:
+            if isinstance(node, CardADF):
+                return node
+            node = node.parent
+        return None
+
+    def interpret_sw(self, sw: str):
+        """Interpret a given status word relative to the currently selected application
+        or the underlying card profile.
+
+        Args:
+            sw : Status word as string of 4 hex digits
+
+        Returns:
+            Tuple of two strings
+        """
+        res = None
+        adf = self.get_application_df()
+        if adf:
+            app = adf.application
+            # The application either comes with its own interpret_sw
+            # method or we will use the interpret_sw method from the
+            # card profile.
+            if app and hasattr(app, "interpret_sw"):
+                res = app.interpret_sw(sw)
+        return res or self.rs.profile.interpret_sw(sw)
+
+    def probe_file(self, fid: str, cmd_app=None):
+        """Blindly try to select a file and automatically add a matching file
+           object if the file actually exists."""
+        if not is_hex(fid, 4, 4):
+            raise ValueError(
+                "Cannot select unknown file by name %s, only hexadecimal 4 digit FID is allowed" % fid)
+
+        self._select_pre(cmd_app)
+
+        try:
+            # We access the card through the select_file method of the scc object.
+            # If we succeed, we know that the file exists on the card and we may
+            # proceed with creating a new CardEF object in the local file model at
+            # run time. In case the file does not exist on the card, we just abort.
+            # The state on the card (selected file/application) wont't be changed,
+            # so we do not have to update any state in that case.
+            (data, _sw) = self.scc.select_file(fid)
+        except SwMatchError as swm:
+            self._select_post(cmd_app)
+            k = self.interpret_sw(swm.sw_actual)
+            if not k:
+                raise swm
+            raise RuntimeError("%s: %s - %s" % (swm.sw_actual, k[0], k[1])) from swm
+
+        select_resp = self.selected_file.decode_select_response(data)
+        if select_resp['file_descriptor']['file_descriptor_byte']['file_type'] == 'df':
+            f = CardDF(fid=fid, sfid=None, name="DF." + str(fid).upper(),
+                       desc="dedicated file, manually added at runtime")
+        else:
+            if select_resp['file_descriptor']['file_descriptor_byte']['structure'] == 'transparent':
+                f = TransparentEF(fid=fid, sfid=None, name="EF." + str(fid).upper(),
+                                  desc="elementary file, manually added at runtime")
+            else:
+                f = LinFixedEF(fid=fid, sfid=None, name="EF." + str(fid).upper(),
+                               desc="elementary file, manually added at runtime")
+
+        self.selected_file.add_files([f])
+
+        self._select_post(cmd_app, f, data)
+
+    def _select_pre(self, cmd_app):
+        # unregister commands of old file
+        if cmd_app and self.selected_file.shell_commands:
+            for c in self.selected_file.shell_commands:
+                cmd_app.unregister_command_set(c)
+
+    def _select_post(self, cmd_app, file:Optional[CardFile] = None, select_resp_data = None):
+        # we store some reference data (see above) about the currently selected file.
+        # This data must be updated after every select.
+        if file:
+            self.selected_file = file
+            if isinstance(file, CardADF):
+                self.selected_adf = file
+            if select_resp_data:
+                self.selected_file_fcp_hex = select_resp_data
+                self.selected_file_fcp = self.selected_file.decode_select_response(select_resp_data)
+            else:
+                self.selected_file_fcp_hex = None
+                self.selected_file_fcp = None
+
+        # register commands of new file
+        if cmd_app and self.selected_file.shell_commands:
+            for c in self.selected_file.shell_commands:
+                cmd_app.register_command_set(c)
+
+    def select_file(self, file: CardFile, cmd_app=None):
+        """Select a file (EF, DF, ADF, MF, ...).
+
+        Args:
+            file : CardFile [or derived class] instance
+            cmd_app : Command Application State (for unregistering old file commands)
+        """
+        # we need to find a path from our self.selected_file to the destination
+        inter_path = self.selected_file.build_select_path_to(file)
+        if not inter_path:
+            raise RuntimeError('Cannot determine path from %s to %s' % (self.selected_file, file))
+        self._select_pre(cmd_app)
+
+        # be sure the variables that we pass to _select_post contain valid values.
+        selected_file = self.selected_file
+        data = self.selected_file_fcp_hex
+
+        for f in inter_path:
+            try:
+                # We now directly accessing the card to perform the selection. This
+                # will change the state of the card, so we must take care to update
+                # the local state (lchan) as well. This is done in the method
+                # _select_post. It should be noted that the caller must always use
+                # the methods select_file or select. The caller must not access the
+                # card directly since this would lead into an incoherence of the
+                # card state and the state of the lchan.
+                if isinstance(f, CardADF):
+                    (data, _sw) = self.rs.card.select_adf_by_aid(f.aid, scc=self.scc)
+                else:
+                    (data, _sw) = self.scc.select_file(f.fid)
+                selected_file = f
+            except SwMatchError as swm:
+                self._select_post(cmd_app, selected_file, data)
+                raise swm
+
+        self._select_post(cmd_app, f, data)
+
+    def select(self, name: str, cmd_app=None):
+        """Select a file (EF, DF, ADF, MF, ...).
+
+        Args:
+            name : Name of file to select
+            cmd_app : Command Application State (for unregistering old file commands)
+        """
+        # if any intermediate step fails, we must be able to go back where we were
+        prev_sel_file = self.selected_file
+
+        # handling of entire paths with multiple directories/elements
+        if '/' in name:
+            pathlist = name.split('/')
+            # treat /DF.GSM/foo like MF/DF.GSM/foo
+            if pathlist[0] == '':
+                pathlist[0] = 'MF'
+            try:
+                for p in pathlist:
+                    self.select(p, cmd_app)
+                return self.selected_file_fcp
+            except Exception as e:
+                self.select_file(prev_sel_file, cmd_app)
+                raise e
+
+        # we are now in the directory where the target file is located
+        # so we can now refer to the get_selectables() method to get the
+        # file object and select it using select_file()
+        sels = self.selected_file.get_selectables()
+        if is_hex(name):
+            name = name.lower()
+
+        try:
+            if name in sels:
+                self.select_file(sels[name], cmd_app)
+            else:
+                self.probe_file(name, cmd_app)
+        except Exception as e:
+            self.select_file(prev_sel_file, cmd_app)
+            raise e
+
+        return self.selected_file_fcp
+
+    def status(self):
+        """Request STATUS (current selected file FCP) from card."""
+        (data, _sw) = self.scc.status()
+        return self.selected_file.decode_select_response(data)
+
+    def get_file_for_selectable(self, name: str):
+        sels = self.selected_file.get_selectables()
+        return sels[name]
+
+    def activate_file(self, name: str):
+        """Request ACTIVATE FILE of specified file."""
+        sels = self.selected_file.get_selectables()
+        f = sels[name]
+        data, sw = self.scc.activate_file(f.fid)
+        return data, sw
+
+    def read_binary(self, length: int = None, offset: int = 0):
+        """Read [part of] a transparent EF binary data.
+
+        Args:
+            length : Amount of data to read (None: as much as possible)
+            offset : Offset into the file from which to read 'length' bytes
+        Returns:
+            binary data read from the file
+        """
+        if not isinstance(self.selected_file, TransparentEF):
+            raise TypeError("Only works with TransparentEF")
+        return self.scc.read_binary(self.selected_file.fid, length, offset)
+
+    def read_binary_dec(self) -> Tuple[dict, str]:
+        """Read [part of] a transparent EF binary data and decode it.
+
+        Args:
+            length : Amount of data to read (None: as much as possible)
+            offset : Offset into the file from which to read 'length' bytes
+        Returns:
+            abstract decode data read from the file
+        """
+        (data, sw) = self.read_binary()
+        dec_data = self.selected_file.decode_hex(data)
+        return (dec_data, sw)
+
+    def update_binary(self, data_hex: str, offset: int = 0):
+        """Update transparent EF binary data.
+
+        Args:
+            data_hex : hex string of data to be written
+            offset : Offset into the file from which to write 'data_hex'
+        """
+        if not isinstance(self.selected_file, TransparentEF):
+            raise TypeError("Only works with TransparentEF")
+        return self.scc.update_binary(self.selected_file.fid, data_hex, offset, conserve=self.rs.conserve_write)
+
+    def update_binary_dec(self, data: dict):
+        """Update transparent EF from abstract data. Encodes the data to binary and
+        then updates the EF with it.
+
+        Args:
+            data : abstract data which is to be encoded and written
+        """
+        data_hex = self.selected_file.encode_hex(data)
+        return self.update_binary(data_hex)
+
+    def read_record(self, rec_nr: int = 0):
+        """Read a record as binary data.
+
+        Args:
+            rec_nr : Record number to read
+        Returns:
+            hex string of binary data contained in record
+        """
+        if not isinstance(self.selected_file, LinFixedEF):
+            raise TypeError("Only works with Linear Fixed EF")
+        # returns a string of hex nibbles
+        return self.scc.read_record(self.selected_file.fid, rec_nr)
+
+    def read_record_dec(self, rec_nr: int = 0) -> Tuple[dict, str]:
+        """Read a record and decode it to abstract data.
+
+        Args:
+            rec_nr : Record number to read
+        Returns:
+            abstract data contained in record
+        """
+        (data, sw) = self.read_record(rec_nr)
+        return (self.selected_file.decode_record_hex(data, rec_nr), sw)
+
+    def update_record(self, rec_nr: int, data_hex: str):
+        """Update a record with given binary data
+
+        Args:
+            rec_nr : Record number to read
+            data_hex : Hex string binary data to be written
+        """
+        if not isinstance(self.selected_file, LinFixedEF):
+            raise TypeError("Only works with Linear Fixed EF")
+        return self.scc.update_record(self.selected_file.fid, rec_nr, data_hex,
+					       conserve=self.rs.conserve_write,
+					       leftpad=self.selected_file.leftpad)
+
+    def update_record_dec(self, rec_nr: int, data: dict):
+        """Update a record with given abstract data.  Will encode abstract to binary data
+        and then write it to the given record on the card.
+
+        Args:
+            rec_nr : Record number to read
+            data_hex : Abstract data to be written
+        """
+        data_hex = self.selected_file.encode_record_hex(data, rec_nr)
+        return self.update_record(rec_nr, data_hex)
+
+    def retrieve_data(self, tag: int = 0):
+        """Read a DO/TLV as binary data.
+
+        Args:
+            tag : Tag of TLV/DO to read
+        Returns:
+            hex string of full BER-TLV DO including Tag and Length
+        """
+        if not isinstance(self.selected_file, BerTlvEF):
+            raise TypeError("Only works with BER-TLV EF")
+        # returns a string of hex nibbles
+        return self.scc.retrieve_data(self.selected_file.fid, tag)
+
+    def retrieve_tags(self):
+        """Retrieve tags available on BER-TLV EF.
+
+        Returns:
+            list of integer tags contained in EF
+        """
+        if not isinstance(self.selected_file, BerTlvEF):
+            raise TypeError("Only works with BER-TLV EF")
+        data, _sw = self.scc.retrieve_data(self.selected_file.fid, 0x5c)
+        _tag, _length, value, _remainder = bertlv_parse_one(h2b(data))
+        return list(value)
+
+    def set_data(self, tag: int, data_hex: str):
+        """Update a TLV/DO with given binary data
+
+        Args:
+            tag : Tag of TLV/DO to be written
+            data_hex : Hex string binary data to be written (value portion)
+        """
+        if not isinstance(self.selected_file, BerTlvEF):
+            raise TypeError("Only works with BER-TLV EF")
+        return self.scc.set_data(self.selected_file.fid, tag, data_hex, conserve=self.rs.conserve_write)
+
+    def unregister_cmds(self, cmd_app=None):
+        """Unregister all file specific commands."""
+        if cmd_app and self.selected_file.shell_commands:
+            for c in self.selected_file.shell_commands:
+                cmd_app.unregister_command_set(c)

pySim/secure_channel.py

@@ -0,0 +1,37 @@
+# Generic code related to Secure Channel processing
+#
+# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import abc
+from pySim.utils import b2h, h2b, ResTuple, Hexstr
+
+class SecureChannel(abc.ABC):
+    @abc.abstractmethod
+    def wrap_cmd_apdu(self, apdu: bytes) -> bytes:
+        """Wrap Command APDU according to specific Secure Channel Protocol."""
+        pass
+
+    @abc.abstractmethod
+    def unwrap_rsp_apdu(self, sw: bytes, rsp_apdu: bytes) -> bytes:
+        """UnWrap Response-APDU according to specific Secure Channel Protocol."""
+        pass
+
+    def send_apdu_wrapper(self, send_fn: callable, pdu: Hexstr, *args, **kwargs) -> ResTuple:
+        """Wrapper function to wrap command APDU and unwrap repsonse APDU around send_apdu callable."""
+        pdu_wrapped = b2h(self.wrap_cmd_apdu(h2b(pdu)))
+        res, sw = send_fn(pdu_wrapped, *args, **kwargs)
+        res_unwrapped = b2h(self.unwrap_rsp_apdu(h2b(sw), h2b(res)))
+        return res_unwrapped, sw

pySim/sms.py

@@ -0,0 +1,399 @@
+"""Code related to SMS Encoding/Decoding"""
+# simplistic SMS T-PDU code, as unfortunately nobody bothered to port the python smspdu
+# module to python3, and I gave up after >= 3 hours of trying and failing to do so
+
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import typing
+import abc
+from bidict import bidict
+from construct import Int8ub, Byte, Bytes, Bit, Flag, BitsInteger
+from construct import Struct, Enum, Tell, BitStruct, this, Padding
+from construct import Prefixed, GreedyRange, GreedyBytes
+
+from pySim.construct import HexAdapter, BcdAdapter, TonNpi
+from pySim.utils import Hexstr, h2b, b2h
+
+from smpp.pdu import pdu_types, operations
+
+BytesOrHex = typing.Union[Hexstr, bytes]
+
+class UserDataHeader:
+    # a single IE in the user data header
+    ie_c = Struct('iei'/Int8ub, 'length'/Int8ub, 'value'/Bytes(this.length))
+    # parser for the full UDH: Length octet followed by sequence of IEs
+    _construct = Struct('ies'/Prefixed(Int8ub, GreedyRange(ie_c)),
+                        'data'/GreedyBytes)
+
+    def __init__(self, ies=[]):
+        self.ies = ies
+
+    def __repr__(self) -> str:
+        return 'UDH(%r)' % self.ies
+
+    def has_ie(self, iei:int) -> bool:
+        for ie in self.ies:
+            if ie['iei'] == iei:
+                return True
+        return False
+
+    @classmethod
+    def from_bytes(cls, inb: BytesOrHex) -> typing.Tuple['UserDataHeader', bytes]:
+        if isinstance(inb, str):
+            inb = h2b(inb)
+        res = cls._construct.parse(inb)
+        return cls(res['ies']), res['data']
+
+    def to_bytes(self) -> bytes:
+        return self._construct.build({'ies':self.ies, 'data':b''})
+
+
+def smpp_dcs_is_8bit(dcs: pdu_types.DataCoding) -> bool:
+    """Determine if the given SMPP data coding scheme is 8-bit or not."""
+    if dcs == pdu_types.DataCoding(pdu_types.DataCodingScheme.DEFAULT,
+                                   pdu_types.DataCodingDefault.OCTET_UNSPECIFIED):
+        return True
+    if dcs == pdu_types.DataCoding(pdu_types.DataCodingScheme.DEFAULT,
+                                   pdu_types.DataCodingDefault.OCTET_UNSPECIFIED_COMMON):
+        return True
+    # pySim/sms.py:72:21: E1101: Instance of 'DataCodingScheme' has no 'GSM_MESSAGE_CLASS' member (no-member)
+    # pylint: disable=no-member
+    if dcs.scheme == pdu_types.DataCodingScheme.GSM_MESSAGE_CLASS and dcs.schemeData['msgCoding'] == pdu_types.DataCodingGsmMsgCoding.DATA_8BIT:
+        return True
+    else:
+        return False
+
+def ensure_smpp_is_8bit(dcs: pdu_types.DataCoding):
+    """Assert if given SMPP data coding scheme is not 8-bit."""
+    if not smpp_dcs_is_8bit(dcs):
+        raise ValueError('We only support 8bit coded SMS for now')
+
+class AddressField:
+    """Representation of an address field as used in SMS T-PDU."""
+    _construct = Struct('addr_len'/Int8ub,
+                        'type_of_addr'/TonNpi,
+                        'digits'/BcdAdapter(Bytes(this.addr_len//2 + this.addr_len%2)),
+                        'tell'/Tell)
+    smpp_map_npi = bidict({
+        'UNKNOWN': 'unknown',
+        'ISDN': 'isdn_e164',
+        'DATA': 'data_x121',
+        'TELEX': 'telex_f69',
+        'LAND_MOBILE': 'sc_specific6',
+        'NATIONAL': 'national',
+        'PRIVATE': 'private',
+        'ERMES': 'ermes',
+        })
+    smpp_map_ton = bidict({
+        'UNKNOWN': 'unknown',
+        'INTERNATIONAL': 'international',
+        'NATIONAL': 'national',
+        'NETWORK_SPECIFIC': 'network_specific',
+        'SUBSCRIBER_NUMBER': 'short_code',
+        'ALPHANUMERIC': 'alphanumeric',
+        'ABBREVIATED': 'abbreviated',
+        })
+
+
+    def __init__(self, digits, ton='unknown', npi='unknown'):
+        self.ton = ton
+        self.npi = npi
+        self.digits = digits
+
+    def __str__(self):
+        return 'AddressField(TON=%s, NPI=%s, %s)' % (self.ton, self.npi, self.digits)
+
+    @classmethod
+    def from_bytes(cls, inb: BytesOrHex) -> typing.Tuple['AddressField', bytes]:
+        """Construct an AddressField instance from the binary T-PDU address format."""
+        if isinstance(inb, str):
+            inb = h2b(inb)
+        res = cls._construct.parse(inb)
+        #print("size: %s" % cls._construct.sizeof())
+        ton = res['type_of_addr']['type_of_number']
+        npi = res['type_of_addr']['numbering_plan_id']
+        # return resulting instance + remainder bytes
+        return cls(res['digits'][:res['addr_len']], ton, npi), inb[res['tell']:]
+
+    @classmethod
+    def from_smpp(cls, addr, ton, npi) -> 'AddressField':
+        """Construct an AddressField from {source,dest}_addr_{,ton,npi} attributes of smpp.pdu."""
+        # return the resulting instance
+        return cls(addr.decode('ascii'), AddressField.smpp_map_ton[ton.name], AddressField.smpp_map_npi[npi.name])
+
+    def to_smpp(self):
+        """Return smpp.pdo.*.source,dest}_addr_{,ton,npi} attributes for given AddressField."""
+        return (self.digits, self.smpp_map_ton.inverse[self.ton], self.smpp_map_npi.inverse[self.npi])
+
+    def to_bytes(self) -> bytes:
+        """Encode the AddressField into the binary representation as used in T-PDU."""
+        num_digits = len(self.digits)
+        if num_digits % 2:
+            self.digits += 'f'
+        d = {
+            'addr_len': num_digits,
+            'type_of_addr': {
+                'ext': True,
+                'type_of_number': self.ton,
+                'numbering_plan_id': self.npi,
+                },
+            'digits': self.digits,
+            }
+        return self._construct.build(d)
+
+
+class SMS_TPDU(abc.ABC):
+    """Base class for a SMS T-PDU."""
+    def __init__(self, **kwargs):
+        self.tp_mti = kwargs.get('tp_mti', None)
+        self.tp_rp = kwargs.get('tp_rp', False)
+        self.tp_udhi = kwargs.get('tp_udhi', False)
+        self.tp_pid = kwargs.get('tp_pid', None)
+        self.tp_dcs = kwargs.get('tp_dcs', None)
+        self.tp_udl = kwargs.get('tp_udl', None)
+        self.tp_ud = kwargs.get('tp_ud', None)
+
+
+
+class SMS_DELIVER(SMS_TPDU):
+    """Representation of a SMS-DELIVER T-PDU. This is the Network to MS/UE (downlink) direction."""
+    flags_construct = BitStruct('tp_rp'/Flag, 'tp_udhi'/Flag, 'tp_rp'/Flag, 'tp_sri'/Flag,
+                                Padding(1), 'tp_mms'/Flag, 'tp_mti'/BitsInteger(2))
+    def __init__(self, **kwargs):
+        kwargs['tp_mti'] = 0
+        super().__init__(**kwargs)
+        self.tp_lp = kwargs.get('tp_lp', False)
+        self.tp_mms = kwargs.get('tp_mms', False)
+        self.tp_oa = kwargs.get('tp_oa', None)
+        self.tp_scts = kwargs.get('tp_scts', None)
+        self.tp_sri = kwargs.get('tp_sri', False)
+
+    def __repr__(self):
+        return '%s(MTI=%s, MMS=%s, LP=%s, RP=%s, UDHI=%s, SRI=%s, OA=%s, PID=%2x, DCS=%x, SCTS=%s, UDL=%u, UD=%s)' % (self.__class__.__name__, self.tp_mti, self.tp_mms, self.tp_lp, self.tp_rp, self.tp_udhi, self.tp_sri, self.tp_oa, self.tp_pid, self.tp_dcs, self.tp_scts, self.tp_udl, self.tp_ud)
+
+    @classmethod
+    def from_bytes(cls, inb: BytesOrHex) -> 'SMS_DELIVER':
+        """Construct a SMS_DELIVER instance from the binary encoded format as used in T-PDU."""
+        if isinstance(inb, str):
+            inb = h2b(inb)
+        d = SMS_DELIVER.flags_construct.parse(inb)
+        oa, remainder = AddressField.from_bytes(inb[1:])
+        d['tp_oa'] = oa
+        offset = 0
+        d['tp_pid'] = remainder[offset]
+        offset += 1
+        d['tp_dcs'] = remainder[offset]
+        offset += 1
+        # TODO: further decode
+        d['tp_scts'] = remainder[offset:offset+7]
+        offset += 7
+        d['tp_udl'] = remainder[offset]
+        offset += 1
+        d['tp_ud'] = remainder[offset:]
+        return cls(**d)
+
+    def to_bytes(self) -> bytes:
+        """Encode a SMS_DELIVER instance to the binary encoded format as used in T-PDU."""
+        outb = bytearray()
+        d = {
+            'tp_mti': self.tp_mti, 'tp_mms': self.tp_mms, 'tp_lp': self.tp_lp,
+            'tp_rp': self.tp_rp, 'tp_udhi': self.tp_udhi, 'tp_sri': self.tp_sri,
+            }
+        flags = SMS_DELIVER.flags_construct.build(d)
+        outb.extend(flags)
+        outb.extend(self.tp_oa.to_bytes())
+        outb.append(self.tp_pid)
+        outb.append(self.tp_dcs)
+        outb.extend(self.tp_scts)
+        outb.append(self.tp_udl)
+        outb.extend(self.tp_ud)
+
+        return outb
+
+    @classmethod
+    def from_smpp(cls, smpp_pdu) -> 'SMS_DELIVER':
+        """Construct a SMS_DELIVER instance from the deliver format used by smpp.pdu."""
+        if smpp_pdu.id == pdu_types.CommandId.submit_sm:
+            return cls.from_smpp_submit(smpp_pdu)
+        else:
+            raise ValueError('Unsupported SMPP commandId %s' % smpp_pdu.id)
+
+    @classmethod
+    def from_smpp_submit(cls, smpp_pdu) -> 'SMS_DELIVER':
+        """Construct a SMS_DELIVER instance from the submit format used by smpp.pdu."""
+        ensure_smpp_is_8bit(smpp_pdu.params['data_coding'])
+        tp_oa = AddressField.from_smpp(smpp_pdu.params['source_addr'],
+                                      smpp_pdu.params['source_addr_ton'],
+                                      smpp_pdu.params['source_addr_npi'])
+        tp_ud = smpp_pdu.params['short_message']
+        d = {
+            'tp_lp': False,
+            'tp_mms': False,
+            'tp_oa': tp_oa,
+            'tp_scts': h2b('22705200000000'), # FIXME
+            'tp_sri': False,
+            'tp_rp': False,
+            'tp_udhi': pdu_types.EsmClassGsmFeatures.UDHI_INDICATOR_SET in smpp_pdu.params['esm_class'].gsmFeatures,
+            'tp_pid': smpp_pdu.params['protocol_id'],
+            'tp_dcs': 0xF6, # we only deal with binary SMS here
+            'tp_udl': len(tp_ud),
+            'tp_ud': tp_ud,
+            }
+        return cls(**d)
+
+
+
+class SMS_SUBMIT(SMS_TPDU):
+    """Representation of a SMS-SUBMIT T-PDU. This is the MS/UE -> network (uplink) direction."""
+    flags_construct = BitStruct('tp_srr'/Flag, 'tp_udhi'/Flag, 'tp_rp'/Flag,
+                                'tp_vpf'/Enum(BitsInteger(2), none=0, relative=2, enhanced=1, absolute=3),
+                                'tp_rd'/Flag, 'tp_mti'/BitsInteger(2))
+    def __init__(self, **kwargs):
+        kwargs['tp_mti'] = 1
+        super().__init__(**kwargs)
+        self.tp_rd = kwargs.get('tp_rd', False)
+        self.tp_vpf = kwargs.get('tp_vpf', 'none')
+        self.tp_srr = kwargs.get('tp_srr', False)
+        self.tp_mr = kwargs.get('tp_mr', None)
+        self.tp_da = kwargs.get('tp_da', None)
+        self.tp_vp = kwargs.get('tp_vp', None)
+
+    def __repr__(self):
+        return '%s(MTI=%s, RD=%s, VPF=%u, RP=%s, UDHI=%s, SRR=%s, DA=%s, PID=%2x, DCS=%x, VP=%s, UDL=%u, UD=%s)' % (self.__class__.__name__, self.tp_mti, self.tp_rd, self.tp_vpf, self.tp_rp, self.tp_udhi, self.tp_srr, self.tp_da, self.tp_pid, self.tp_dcs, self.tp_vp, self.tp_udl, self.tp_ud)
+
+    @classmethod
+    def from_bytes(cls, inb:BytesOrHex) -> 'SMS_SUBMIT':
+        """Construct a SMS_SUBMIT instance from the binary encoded format as used in T-PDU."""
+        offset = 0
+        if isinstance(inb, str):
+            inb = h2b(inb)
+        d = SMS_SUBMIT.flags_construct.parse(inb)
+        offset += 1
+        d['tp_mr']= inb[offset]
+        offset += 1
+        da, remainder = AddressField.from_bytes(inb[2:])
+        d['tp_da'] = da
+
+        offset = 0
+        d['tp_pid'] = remainder[offset]
+        offset += 1
+        d['tp_dcs'] = remainder[offset]
+        offset += 1
+        if d['tp_vpf'] == 'none':
+            pass
+        elif d['tp_vpf'] == 'relative':
+            # TODO: further decode
+            d['tp_vp'] = remainder[offset:offset+1]
+            offset += 1
+        elif d['tp_vpf'] == 'enhanced':
+            # TODO: further decode
+            d['tp_vp'] = remainder[offset:offset+7]
+            offset += 7
+        elif d['tp_vpf'] == 'absolute':
+            # TODO: further decode
+            d['tp_vp'] = remainder[offset:offset+7]
+            offset += 7
+        else:
+            raise ValueError('Invalid VPF: %s' % d['tp_vpf'])
+        d['tp_udl'] = remainder[offset]
+        offset += 1
+        d['tp_ud'] = remainder[offset:]
+        return cls(**d)
+
+    def to_bytes(self) -> bytes:
+        """Encode a SMS_SUBMIT instance to the binary encoded format as used in T-PDU."""
+        outb = bytearray()
+        d = {
+            'tp_mti': self.tp_mti, 'tp_rd': self.tp_rd, 'tp_vpf': self.tp_vpf,
+            'tp_rp': self.tp_rp, 'tp_udhi': self.tp_udhi, 'tp_srr': self.tp_srr,
+            }
+        flags = SMS_SUBMIT.flags_construct.build(d)
+        outb.extend(flags)
+        outb.append(self.tp_mr)
+        outb.extend(self.tp_da.to_bytes())
+        outb.append(self.tp_pid)
+        outb.append(self.tp_dcs)
+        if self.tp_vpf != 'none':
+            outb.extend(self.tp_vp)
+        outb.append(self.tp_udl)
+        outb.extend(self.tp_ud)
+        return outb
+
+    @classmethod
+    def from_smpp(cls, smpp_pdu) -> 'SMS_SUBMIT':
+        """Construct a SMS_SUBMIT instance from the format used by smpp.pdu."""
+        if smpp_pdu.id == pdu_types.CommandId.submit_sm:
+            return cls.from_smpp_submit(smpp_pdu)
+        else:
+            raise ValueError('Unsupported SMPP commandId %s' % smpp_pdu.id)
+
+    @classmethod
+    def from_smpp_submit(cls, smpp_pdu) -> 'SMS_SUBMIT':
+        """Construct a SMS_SUBMIT instance from the submit format used by smpp.pdu."""
+        ensure_smpp_is_8bit(smpp_pdu.params['data_coding'])
+        tp_da = AddressField.from_smpp(smpp_pdu.params['destination_addr'],
+                                       smpp_pdu.params['dest_addr_ton'],
+                                       smpp_pdu.params['dest_addr_npi'])
+        tp_ud = smpp_pdu.params['short_message']
+        #vp_smpp = smpp_pdu.params['validity_period']
+        #if not vp_smpp:
+        #    vpf = 'none'
+        d = {
+            'tp_rd': True if smpp_pdu.params['replace_if_present_flag'].name == 'REPLACE' else False,
+            'tp_vpf': None, # vpf,
+            'tp_rp': False, # related to ['registered_delivery'] ?
+            'tp_udhi': pdu_types.EsmClassGsmFeatures.UDHI_INDICATOR_SET in smpp_pdu.params['esm_class'].gsmFeatures,
+            'tp_srr': True if smpp_pdu.params['registered_delivery'] else False,
+            'tp_mr': 0, # FIXME: sm_default_msg_id ?
+            'tp_da': tp_da,
+            'tp_pid': smpp_pdu.params['protocol_id'],
+            'tp_dcs': 0xF6, # FIXME: we only deal with binary SMS here
+            'tp_vp': None, # FIXME: implement VPF conversion
+            'tp_udl': len(tp_ud),
+            'tp_ud': tp_ud,
+            }
+        return cls(**d)
+
+    def to_smpp(self) -> pdu_types.PDU:
+        """Translate a SMS_SUBMIT instance to a smpp.pdu.operations.SubmitSM instance."""
+        esm_class = pdu_types.EsmClass(pdu_types.EsmClassMode.DEFAULT, pdu_types.EsmClassType.DEFAULT)
+        reg_del = pdu_types.RegisteredDelivery(pdu_types.RegisteredDeliveryReceipt.NO_SMSC_DELIVERY_RECEIPT_REQUESTED)
+        if self.tp_rp:
+            repl_if = pdu_types.ReplaceIfPresentFlag.REPLACE
+        else:
+            repl_if = pdu_types.ReplaceIfPresentFlag.DO_NOT_REPLACE
+        # we only deal with binary SMS here:
+        if self.tp_dcs != 0xF6:
+            raise ValueError('Unsupported DCS: We only support DCS=0xF6 for now')
+        dc = pdu_types.DataCoding(pdu_types.DataCodingScheme.DEFAULT, pdu_types.DataCodingDefault.OCTET_UNSPECIFIED)
+        (daddr, ton, npi) = self.tp_da.to_smpp()
+        return operations.SubmitSM(service_type='',
+                                   source_addr_ton=pdu_types.AddrTon.ALPHANUMERIC,
+                                   source_addr_npi=pdu_types.AddrNpi.UNKNOWN,
+                                   source_addr='simcard',
+                                   dest_addr_ton=ton,
+                                   dest_addr_npi=npi,
+                                   destination_addr=daddr,
+                                   esm_class=esm_class,
+                                   protocol_id=self.tp_pid,
+                                   priority_flag=pdu_types.PriorityFlag.LEVEL_0,
+                                   #schedule_delivery_time,
+                                   #validity_period,
+                                   registered_delivery=reg_del,
+                                   replace_if_present_flag=repl_if,
+                                   data_coding=dc,
+                                   #sm_default_msg_id,
+                                   short_message=self.tp_ud)

pySim/sysmocom_sja2.py

@@ -0,0 +1,348 @@
+# coding=utf-8
+"""Utilities / Functions related to sysmocom SJA2/SJA5 cards
+
+(C) 2021-2023 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from struct import unpack
+from construct import FlagsEnum, Byte, Struct, Int8ub, Bytes, Mapping, Enum, Padding, BitsInteger
+from construct import Bit, this, Int32ub, Int16ub, Nibble, BytesInteger, GreedyRange
+from construct import Optional as COptional
+
+from pySim.utils import *
+from pySim.filesystem import *
+from pySim.runtime import RuntimeState
+from pySim.construct import *
+import pySim
+
+key_type2str = {
+    0: 'kic',
+    1: 'kid',
+    2: 'kik',
+    3: 'any',
+}
+
+key_algo2str = {
+    0: 'des',
+    1: 'aes'
+}
+
+mac_length = {
+    0: 8,
+    1: 4
+}
+
+
+class EF_PIN(TransparentEF):
+    _test_de_encode = [
+        ( 'f1030331323334ffffffff0a0a3132333435363738',
+          { 'state': { 'valid': True, 'change_able': True, 'unblock_able': True, 'disable_able': True,
+                       'not_initialized': False, 'disabled': True },
+           'attempts_remaining': 3, 'maximum_attempts': 3, 'pin': '31323334',
+           'puk': { 'attempts_remaining': 10, 'maximum_attempts': 10, 'puk': '3132333435363738' }
+          } ),
+        ( 'f003039999999999999999',
+          { 'state': { 'valid': True, 'change_able': True, 'unblock_able': True, 'disable_able': True,
+                       'not_initialized': False, 'disabled': False },
+           'attempts_remaining': 3, 'maximum_attempts': 3, 'pin': '9999999999999999',
+           'puk': None } ),
+    ]
+    def __init__(self, fid='6f01', name='EF.CHV1'):
+        super().__init__(fid, name=name, desc='%s PIN file' % name)
+        StateByte = FlagsEnum(Byte, disabled=1, not_initialized=2, disable_able=0x10, unblock_able=0x20,
+                                    change_able=0x40, valid=0x80)
+        PukStruct = Struct('attempts_remaining'/Int8ub,
+                           'maximum_attempts'/Int8ub,
+                           'puk'/HexAdapter(Rpad(Bytes(8))))
+        self._construct = Struct('state'/StateByte,
+                                 'attempts_remaining'/Int8ub,
+                                 'maximum_attempts'/Int8ub,
+                                 'pin'/HexAdapter(Rpad(Bytes(8))),
+                                 'puk'/COptional(PukStruct))
+
+
+class EF_MILENAGE_CFG(TransparentEF):
+    _test_de_encode = [
+        ( '40002040600000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000020000000000000000000000000000000400000000000000000000000000000008',
+          {"r1": 64, "r2": 0, "r3": 32, "r4": 64, "r5": 96, "c1": "00000000000000000000000000000000", "c2":
+           "00000000000000000000000000000001", "c3": "00000000000000000000000000000002", "c4":
+           "00000000000000000000000000000004", "c5": "00000000000000000000000000000008"} ),
+      ]
+    def __init__(self, fid='6f21', name='EF.MILENAGE_CFG', desc='Milenage connfiguration'):
+        super().__init__(fid, name=name, desc=desc)
+        self._construct = Struct('r1'/Int8ub, 'r2'/Int8ub, 'r3'/Int8ub, 'r4'/Int8ub, 'r5'/Int8ub,
+                                 'c1'/HexAdapter(Bytes(16)),
+                                 'c2'/HexAdapter(Bytes(16)),
+                                 'c3'/HexAdapter(Bytes(16)),
+                                 'c4'/HexAdapter(Bytes(16)),
+                                 'c5'/HexAdapter(Bytes(16)))
+
+
+class EF_0348_KEY(LinFixedEF):
+    def __init__(self, fid='6f22', name='EF.0348_KEY', desc='TS 03.48 OTA Keys'):
+        super().__init__(fid, name=name, desc=desc, rec_len=(27, 35))
+        KeyLenAndType = BitStruct('mac_length'/Mapping(Bit, {8:0, 4:1}),
+                                  'algorithm'/Enum(Bit, des=0, aes=1),
+                                  'key_length'/MultiplyAdapter(BitsInteger(3), 8),
+                                  '_rfu'/BitsRFU(1),
+                                  'key_type'/Enum(BitsInteger(2), kic=0, kid=1, kik=2, any=3))
+        self._construct = Struct('security_domain'/Int8ub,
+                                 'key_set_version'/Int8ub,
+                                 'key_len_and_type'/KeyLenAndType,
+                                 'key'/HexAdapter(Bytes(this.key_len_and_type.key_length)))
+
+
+class EF_0348_COUNT(LinFixedEF):
+    _test_de_encode = [
+        ( 'fe010000000000', {"sec_domain": 254, "key_set_version": 1, "counter": "0000000000"} ),
+      ]
+    def __init__(self, fid='6f23', name='EF.0348_COUNT', desc='TS 03.48 OTA Counters'):
+        super().__init__(fid, name=name, desc=desc, rec_len=(7, 7))
+        self._construct = Struct('sec_domain'/Int8ub,
+                                 'key_set_version'/Int8ub,
+                                 'counter'/HexAdapter(Bytes(5)))
+
+
+class EF_SIM_AUTH_COUNTER(TransparentEF):
+    def __init__(self, fid='af24', name='EF.SIM_AUTH_COUNTER'):
+        super().__init__(fid, name=name, desc='Number of remaining RUN GSM ALGORITHM executions')
+        self._construct = Struct('num_run_gsm_algo_remain'/Int32ub)
+
+
+class EF_GP_COUNT(LinFixedEF):
+    _test_de_encode = [
+        ( '0070000000', {"sec_domain": 0, "key_set_version": 112, "counter": 0, "rfu": 0} ),
+      ]
+    def __init__(self, fid='6f26', name='EF.GP_COUNT', desc='GP SCP02 Counters'):
+        super().__init__(fid, name=name, desc=desc, rec_len=(5, 5))
+        self._construct = Struct('sec_domain'/Int8ub,
+                                 'key_set_version'/Int8ub,
+                                 'counter'/Int16ub,
+                                 'rfu'/Int8ub)
+
+class EF_GP_DIV_DATA(LinFixedEF):
+    def __init__(self, fid='6f27', name='EF.GP_DIV_DATA', desc='GP SCP02 key diversification data'):
+        super().__init__(fid, name=name, desc=desc, rec_len=(12, 12))
+
+    def _decode_record_bin(self, raw_bin_data, **kwargs):
+        u = unpack('!BB8s', raw_bin_data)
+        return {'sec_domain': u[0], 'key_set_version': u[1], 'key_div_data': u[2].hex()}
+
+
+class EF_SIM_AUTH_KEY(TransparentEF):
+    _test_de_encode = [
+        ( '14000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f',
+          {"cfg": {"sres_deriv_func": 1, "use_opc_instead_of_op": True, "algorithm": "milenage"}, "key":
+           "000102030405060708090a0b0c0d0e0f", "op_opc": "101112131415161718191a1b1c1d1e1f"} ),
+      ]
+    def __init__(self, fid='6f20', name='EF.SIM_AUTH_KEY'):
+        super().__init__(fid, name=name, desc='USIM authentication key')
+        CfgByte = BitStruct(Padding(2),
+                            'sres_deriv_func'/Mapping(Bit, {1:0, 2:1}),
+                            'use_opc_instead_of_op'/Flag,
+                            'algorithm'/Enum(Nibble, milenage=4, comp128v1=1, comp128v2=2, comp128v3=3))
+        self._construct = Struct('cfg'/CfgByte,
+                                 'key'/HexAdapter(Bytes(16)),
+                                 'op_opc' /HexAdapter(Bytes(16)))
+
+
+class DF_SYSTEM(CardDF):
+    def __init__(self):
+        super().__init__(fid='a515', name='DF.SYSTEM', desc='CardOS specifics')
+        files = [
+            EF_PIN('6f01', 'EF.CHV1'),
+            EF_PIN('6f81', 'EF.CHV2'),
+            EF_PIN('6f0a', 'EF.ADM1'),
+            EF_PIN('6f0b', 'EF.ADM2'),
+            EF_PIN('6f0c', 'EF.ADM3'),
+            EF_PIN('6f0d', 'EF.ADM4'),
+            EF_MILENAGE_CFG(),
+            EF_0348_KEY(),
+            EF_SIM_AUTH_COUNTER(),
+            EF_SIM_AUTH_KEY(),
+            EF_0348_COUNT(),
+            EF_GP_COUNT(),
+            EF_GP_DIV_DATA(),
+        ]
+        self.add_files(files)
+
+    def decode_select_response(self, resp_hex):
+        return pySim.ts_102_221.CardProfileUICC.decode_select_response(resp_hex)
+
+
+class EF_USIM_SQN(TransparentEF):
+    _test_de_encode = [
+        ( 'd503000200000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+          {"flag1": {"skip_next_sqn_check": True, "delta_max_check": True, "age_limit_check": False, "sqn_check": True,
+                     "ind_len": 5}, "flag2": {"rfu": 0, "dont_clear_amf_for_macs": False, "aus_concealed": True,
+                                              "autn_concealed": True}, "delta_max": 8589934592, "age_limit":
+           8589934592, "freshness": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+                                     0, 0, 0, 0, 0, 0, 0, 0]} ),
+      ]
+    def __init__(self, fid='af30', name='EF.USIM_SQN'):
+        super().__init__(fid, name=name, desc='SQN parameters for AKA')
+        Flag1 = BitStruct('skip_next_sqn_check'/Flag, 'delta_max_check'/Flag,
+                          'age_limit_check'/Flag, 'sqn_check'/Flag,
+                          'ind_len'/BitsInteger(4))
+        Flag2 = BitStruct('rfu'/BitsRFU(5), 'dont_clear_amf_for_macs'/Flag,
+                          'aus_concealed'/Flag, 'autn_concealed'/Flag)
+        self._construct = Struct('flag1'/Flag1, 'flag2'/Flag2,
+                                 'delta_max' /
+                                 BytesInteger(6), 'age_limit'/BytesInteger(6),
+                                 'freshness'/GreedyRange(BytesInteger(6)))
+
+
+class EF_USIM_AUTH_KEY(TransparentEF):
+    def __init__(self, fid='af20', name='EF.USIM_AUTH_KEY'):
+        super().__init__(fid, name=name, desc='USIM authentication key')
+        Algorithm = Enum(Nibble, milenage=4, sha1_aka=5, tuak=6, xor=15)
+        CfgByte = BitStruct(Padding(1), 'only_4bytes_res_in_3g'/Flag,
+                            'sres_deriv_func_in_2g'/Mapping(Bit, {1:0, 2:1}),
+                            'use_opc_instead_of_op'/Mapping(Bit, {False:0, True:1}),
+                            'algorithm'/Algorithm)
+        self._construct = Struct('cfg'/CfgByte,
+                                 'key'/HexAdapter(Bytes(16)),
+                                 'op_opc' /HexAdapter(Bytes(16)))
+        # TUAK has a rather different layout for the data, so we define a different
+        # construct below and use explicit _{decode,encode}_bin() methods for separating
+        # the TUAK and non-TUAK situation
+        CfgByteTuak = BitStruct(Padding(1),
+                                'key_length'/Mapping(Bit, {128:0, 256:1}),
+                                'sres_deriv_func_in_2g'/Mapping(Bit, {1:0, 2:1}),
+                                'use_opc_instead_of_op'/Mapping(Bit, {False:0, True:1}),
+                                'algorithm'/Algorithm)
+        TuakCfgByte = BitStruct(Padding(1),
+                                'ck_and_ik_size'/Mapping(Bit, {128:0, 256:1}),
+                                'mac_size'/Mapping(BitsInteger(3), {64:0, 128:1, 256:2}),
+                                'res_size'/Mapping(BitsInteger(3), {32:0, 64:1, 128:2, 256:3}))
+        self._constr_tuak = Struct('cfg'/CfgByteTuak,
+                                   'tuak_cfg'/TuakCfgByte,
+                                   'num_of_keccak_iterations'/Int8ub,
+                                   'op_opc'/HexAdapter(Bytes(32)),
+                                   'k'/HexAdapter(Bytes(this.cfg.key_length//8)))
+
+    def _decode_bin(self, raw_bin_data: bytearray) -> dict:
+        if raw_bin_data[0] & 0x0F == 0x06:
+            return parse_construct(self._constr_tuak, raw_bin_data)
+        else:
+            return parse_construct(self._construct, raw_bin_data)
+
+    def _encode_bin(self, abstract_data: dict) -> bytearray:
+        if abstract_data['cfg']['algorithm'] == 'tuak':
+            return build_construct(self._constr_tuak, abstract_data)
+        else:
+            return build_construct(self._construct, abstract_data)
+
+
+class EF_USIM_AUTH_KEY_2G(TransparentEF):
+    _test_de_encode = [
+        ( '14000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f',
+          {"cfg": {"only_4bytes_res_in_3g": False, "sres_deriv_func_in_2g": 1, "use_opc_instead_of_op": True,
+                   "algorithm": "milenage"}, "key": "000102030405060708090a0b0c0d0e0f", "op_opc":
+           "101112131415161718191a1b1c1d1e1f"} ),
+      ]
+    def __init__(self, fid='af22', name='EF.USIM_AUTH_KEY_2G'):
+        super().__init__(fid, name=name, desc='USIM authentication key in 2G context')
+        CfgByte = BitStruct(Padding(1), 'only_4bytes_res_in_3g'/Flag,
+                            'sres_deriv_func_in_2g'/Mapping(Bit, {1:0, 2:1}),
+                            'use_opc_instead_of_op'/Flag,
+                            'algorithm'/Enum(Nibble, milenage=4, comp128v1=1, comp128v2=2, comp128v3=3, xor=14))
+        self._construct = Struct('cfg'/CfgByte,
+                                 'key'/HexAdapter(Bytes(16)),
+                                 'op_opc' /HexAdapter(Bytes(16)))
+
+
+class EF_GBA_SK(TransparentEF):
+    def __init__(self, fid='af31', name='EF.GBA_SK'):
+        super().__init__(fid, name=name, desc='Secret key for GBA key derivation')
+        self._construct = GreedyBytes
+
+
+class EF_GBA_REC_LIST(TransparentEF):
+    def __init__(self, fid='af32', name='EF.GBA_REC_LIST'):
+        super().__init__(fid, name=name, desc='Secret key for GBA key derivation')
+        # integers representing record numbers in EF-GBANL
+        self._construct = GreedyRange(Int8ub)
+
+
+class EF_GBA_INT_KEY(LinFixedEF):
+    def __init__(self, fid='af33', name='EF.GBA_INT_KEY'):
+        super().__init__(fid, name=name,
+                         desc='Secret key for GBA key derivation', rec_len=(32, 32))
+        self._construct = GreedyBytes
+
+
+class SysmocomSJA2(CardModel):
+    _atrs = ["3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 4C 75 30 34 05 4B A9",
+             "3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 4C 75 31 33 02 51 B2",
+             "3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 4C 52 75 31 04 51 D5"]
+
+    @classmethod
+    def add_files(cls, rs: RuntimeState):
+        """Add sysmocom SJA2 specific files to given RuntimeState."""
+        rs.mf.add_file(DF_SYSTEM())
+        # optional USIM application
+        if 'a0000000871002' in rs.mf.applications:
+            usim_adf = rs.mf.applications['a0000000871002']
+            files_adf_usim = [
+                EF_USIM_AUTH_KEY(),
+                EF_USIM_AUTH_KEY_2G(),
+                EF_GBA_SK(),
+                EF_GBA_REC_LIST(),
+                EF_GBA_INT_KEY(),
+                EF_USIM_SQN(),
+            ]
+            usim_adf.add_files(files_adf_usim)
+        # optional ISIM application
+        if 'a0000000871004' in rs.mf.applications:
+            isim_adf = rs.mf.applications['a0000000871004']
+            files_adf_isim = [
+                EF_USIM_AUTH_KEY(name='EF.ISIM_AUTH_KEY'),
+                EF_USIM_AUTH_KEY_2G(name='EF.ISIM_AUTH_KEY_2G'),
+                EF_USIM_SQN(name='EF.ISIM_SQN'),
+            ]
+            isim_adf.add_files(files_adf_isim)
+
+class SysmocomSJA5(CardModel):
+    _atrs = ["3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 35 75 30 35 02 51 CC",
+             "3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 35 75 30 35 02 65 F8",
+             "3B 9F 96 80 1F 87 80 31 E0 73 FE 21 1B 67 4A 35 75 30 35 02 59 C4"]
+
+    @classmethod
+    def add_files(cls, rs: RuntimeState):
+        """Add sysmocom SJA2 specific files to given RuntimeState."""
+        rs.mf.add_file(DF_SYSTEM())
+        # optional USIM application
+        if 'a0000000871002' in rs.mf.applications:
+            usim_adf = rs.mf.applications['a0000000871002']
+            files_adf_usim = [
+                EF_USIM_AUTH_KEY(),
+                EF_USIM_AUTH_KEY_2G(),
+                EF_GBA_SK(),
+                EF_GBA_REC_LIST(),
+                EF_GBA_INT_KEY(),
+                EF_USIM_SQN(),
+            ]
+            usim_adf.add_files(files_adf_usim)
+        # optional ISIM application
+        if 'a0000000871004' in rs.mf.applications:
+            isim_adf = rs.mf.applications['a0000000871004']
+            files_adf_isim = [
+                EF_USIM_AUTH_KEY(name='EF.ISIM_AUTH_KEY'),
+                EF_USIM_AUTH_KEY_2G(name='EF.ISIM_AUTH_KEY_2G'),
+                EF_USIM_SQN(name='EF.ISIM_SQN'),
+            ]
+            isim_adf.add_files(files_adf_isim)

pySim/tlv.py

@@ -0,0 +1,464 @@
+"""object-oriented TLV parser/encoder library."""
+
+# (C) 2021 by Harald Welte <laforge@osmocom.org>
+# All Rights Reserved
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import inspect
+import abc
+import re
+from typing import List, Tuple
+
+from pySim.utils import bertlv_encode_len, bertlv_parse_len, bertlv_encode_tag, bertlv_parse_tag
+from pySim.utils import comprehensiontlv_encode_tag, comprehensiontlv_parse_tag
+from pySim.utils import bertlv_parse_tag_raw, comprehensiontlv_parse_tag_raw
+from pySim.utils import dgi_parse_tag_raw, dgi_parse_len, dgi_encode_tag, dgi_encode_len
+
+from pySim.construct import build_construct, parse_construct
+
+
+def camel_to_snake(name):
+    name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
+    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()
+
+class TlvMeta(abc.ABCMeta):
+    """Metaclass which we use to set some class variables at the time of defining a subclass.
+    This allows us to create subclasses for each TLV/IE type, where the class represents fixed
+    parameters like the tag/type and instances of it represent the actual TLV data."""
+    def __new__(mcs, name, bases, namespace, **kwargs):
+        #print("TlvMeta_new_(mcs=%s, name=%s, bases=%s, namespace=%s, kwargs=%s)" % (mcs, name, bases, namespace, kwargs))
+        x = super().__new__(mcs, name, bases, namespace)
+        # this becomes a _class_ variable, not an instance variable
+        x.tag = namespace.get('tag', kwargs.get('tag', None))
+        x.desc = namespace.get('desc', kwargs.get('desc', None))
+        nested = namespace.get('nested', kwargs.get('nested', None))
+        if nested is None or inspect.isclass(nested) and issubclass(nested, TLV_IE_Collection):
+            # caller has specified TLV_IE_Collection sub-class, we can directly reference it
+            x.nested_collection_cls = nested
+        else:
+            # caller passed list of other TLV classes that might possibly appear within us,
+            # build a dynamically-created TLV_IE_Collection sub-class and reference it
+            name = 'auto_collection_%s' % (name)
+            cls = type(name, (TLV_IE_Collection,), {'nested': nested})
+            x.nested_collection_cls = cls
+        return x
+
+
+class TlvCollectionMeta(abc.ABCMeta):
+    """Metaclass which we use to set some class variables at the time of defining a subclass.
+    This allows us to create subclasses for each Collection type, where the class represents fixed
+    parameters like the nested IE classes and instances of it represent the actual TLV data."""
+    def __new__(mcs, name, bases, namespace, **kwargs):
+        #print("TlvCollectionMeta_new_(mcs=%s, name=%s, bases=%s, namespace=%s, kwargs=%s)" % (mcs, name, bases, namespace, kwargs))
+        x = super().__new__(mcs, name, bases, namespace)
+        # this becomes a _class_ variable, not an instance variable
+        x.possible_nested = namespace.get('nested', kwargs.get('nested', None))
+        return x
+
+
+class Transcodable(abc.ABC):
+    _construct = None
+    """Base class for something that can be encoded + encoded.  Decoding and Encoding happens either
+     * via a 'construct' object stored in a derived class' _construct variable, or
+     * via a 'construct' object stored in an instance _construct variable, or
+     * via a derived class' _{to,from}_bytes() methods."""
+
+    def __init__(self):
+        self.encoded = None
+        self.decoded = None
+        self._construct = None
+
+    def to_bytes(self, context: dict = {}) -> bytes:
+        """Convert from internal representation to binary bytes.  Store the binary result
+        in the internal state and return it."""
+        if self.decoded is None:
+            do = b''
+        elif self._construct:
+            do = build_construct(self._construct, self.decoded, context)
+        elif self.__class__._construct:
+            do = build_construct(self.__class__._construct, self.decoded, context)
+        else:
+            do = self._to_bytes()
+        self.encoded = do
+        return do
+
+    # not an abstractmethod, as it is only required if no _construct exists
+    def _to_bytes(self):
+        raise NotImplementedError('%s._to_bytes' % type(self).__name__)
+
+    def from_bytes(self, do: bytes, context: dict = {}):
+        """Convert from binary bytes to internal representation. Store the decoded result
+        in the internal state and return it."""
+        self.encoded = do
+        if self.encoded == b'':
+            self.decoded = None
+        elif self._construct:
+            self.decoded = parse_construct(self._construct, do, context=context)
+        elif self.__class__._construct:
+            self.decoded = parse_construct(self.__class__._construct, do, context=context)
+        else:
+            self.decoded = self._from_bytes(do)
+        return self.decoded
+
+    # not an abstractmethod, as it is only required if no _construct exists
+    def _from_bytes(self, do: bytes):
+        raise NotImplementedError('%s._from_bytes' % type(self).__name__)
+
+
+class IE(Transcodable, metaclass=TlvMeta):
+    # we specify the metaclass so any downstream subclasses will automatically use it
+    """Base class for various Information Elements. We understand the notion of a hierarchy
+    of IEs on top of the Transcodable class."""
+    # this is overridden by the TlvMeta metaclass, if it is used to create subclasses
+    nested_collection_cls = None
+    tag = None
+
+    def __init__(self, **kwargs):
+        super().__init__()
+        self.nested_collection = None
+        if self.nested_collection_cls:
+            self.nested_collection = self.nested_collection_cls()
+        # if we are a constructed IE, [ordered] list of actual child-IE instances
+        self.children = kwargs.get('children', [])
+        self.decoded = kwargs.get('decoded', None)
+
+    def __repr__(self):
+        """Return a string representing the [nested] IE data (for print)."""
+        if len(self.children):
+            member_strs = [repr(x) for x in self.children]
+            return '%s(%s)' % (type(self).__name__, ','.join(member_strs))
+        else:
+            return '%s(%s)' % (type(self).__name__, self.decoded)
+
+    def to_dict(self):
+        """Return a JSON-serializable dict representing the [nested] IE data."""
+        if len(self.children):
+            v = [x.to_dict() for x in self.children]
+        else:
+            v = self.decoded
+        return {camel_to_snake(type(self).__name__): v}
+
+    def from_dict(self, decoded: dict):
+        """Set the IE internal decoded representation to data from the argument.
+        If this is a nested IE, the child IE instance list is re-created."""
+        expected_key_name = camel_to_snake(type(self).__name__)
+        if not expected_key_name in decoded:
+            raise ValueError("Dict %s doesn't contain expected key %s" % (decoded, expected_key_name))
+        if self.nested_collection:
+            self.children = self.nested_collection.from_dict(decoded[expected_key_name])
+        else:
+            self.children = []
+            self.decoded = decoded[expected_key_name]
+
+    def is_constructed(self):
+        """Is this IE constructed by further nested IEs?"""
+        return bool(len(self.children) > 0)
+
+    @abc.abstractmethod
+    def to_ie(self, context: dict = {}) -> bytes:
+        """Convert the internal representation to entire IE including IE header."""
+
+    def to_bytes(self, context: dict = {}) -> bytes:
+        """Convert the internal representation *of the value part* to binary bytes."""
+        if self.is_constructed():
+            # concatenate the encoded IE of all children to form the value part
+            out = b''
+            for c in self.children:
+                out += c.to_ie(context=context)
+            return out
+        else:
+            return super().to_bytes(context=context)
+
+    def from_bytes(self, do: bytes, context: dict = {}):
+        """Parse *the value part* from binary bytes to internal representation."""
+        if self.nested_collection:
+            self.children = self.nested_collection.from_bytes(do, context=context)
+        else:
+            self.children = []
+            return super().from_bytes(do, context=context)
+
+
+class TLV_IE(IE):
+    """Abstract base class for various TLV type Information Elements."""
+
+    def _compute_tag(self) -> int:
+        """Compute the tag (sometimes the tag encodes part of the value)."""
+        return self.tag
+
+    @classmethod
+    @abc.abstractmethod
+    def _parse_tag_raw(cls, do: bytes) -> Tuple[int, bytes]:
+        """Obtain the raw TAG at the start of the bytes provided by the user."""
+
+    @classmethod
+    @abc.abstractmethod
+    def _parse_len(cls, do: bytes) -> Tuple[int, bytes]:
+        """Obtain the length encoded at the start of the bytes provided by the user."""
+
+    @abc.abstractmethod
+    def _encode_tag(self) -> bytes:
+        """Encode the tag part. Must be provided by derived (TLV format specific) class."""
+
+    @abc.abstractmethod
+    def _encode_len(self, val: bytes) -> bytes:
+        """Encode the length part assuming a certain binary value. Must be provided by
+        derived (TLV format specific) class."""
+
+    def to_ie(self, context: dict = {}):
+        return self.to_tlv(context=context)
+
+    def to_tlv(self, context: dict = {}):
+        """Convert the internal representation to binary TLV bytes."""
+        val = self.to_bytes(context=context)
+        return self._encode_tag() + self._encode_len(val) + val
+
+    def from_tlv(self, do: bytes, context: dict = {}):
+        if len(do) == 0:
+            return {}, b''
+        (rawtag, remainder) = self.__class__._parse_tag_raw(do)
+        if rawtag:
+            if rawtag != self._compute_tag():
+                raise ValueError("%s: Encountered tag %s doesn't match our supported tag %s" %
+                                 (self, rawtag, self.tag))
+            (length, remainder) = self.__class__._parse_len(remainder)
+            value = remainder[:length]
+            remainder = remainder[length:]
+        else:
+            value = do
+            remainder = b''
+        dec = self.from_bytes(value, context=context)
+        return dec, remainder
+
+
+class BER_TLV_IE(TLV_IE):
+    """TLV_IE formatted as ASN.1 BER described in ITU-T X.690 8.1.2."""
+
+    @classmethod
+    def _decode_tag(cls, do: bytes) -> Tuple[dict, bytes]:
+        return bertlv_parse_tag(do)
+
+    @classmethod
+    def _parse_tag_raw(cls, do: bytes) -> Tuple[int, bytes]:
+        return bertlv_parse_tag_raw(do)
+
+    @classmethod
+    def _parse_len(cls, do: bytes) -> Tuple[int, bytes]:
+        return bertlv_parse_len(do)
+
+    def _encode_tag(self) -> bytes:
+        return bertlv_encode_tag(self._compute_tag())
+
+    def _encode_len(self, val: bytes) -> bytes:
+        return bertlv_encode_len(len(val))
+
+
+class COMPR_TLV_IE(TLV_IE):
+    """TLV_IE formated as COMPREHENSION-TLV as described in ETSI TS 101 220."""
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self.comprehension = False
+
+    @classmethod
+    def _decode_tag(cls, do: bytes) -> Tuple[dict, bytes]:
+        return comprehensiontlv_parse_tag(do)
+
+    @classmethod
+    def _parse_tag_raw(cls, do: bytes) -> Tuple[int, bytes]:
+        return comprehensiontlv_parse_tag_raw(do)
+
+    @classmethod
+    def _parse_len(cls, do: bytes) -> Tuple[int, bytes]:
+        return bertlv_parse_len(do)
+
+    def _encode_tag(self) -> bytes:
+        return comprehensiontlv_encode_tag(self._compute_tag())
+
+    def _encode_len(self, val: bytes) -> bytes:
+        return bertlv_encode_len(len(val))
+
+
+class DGI_TLV_IE(TLV_IE):
+    """TLV_IE formated as  GlobalPlatform Systems Scripting Language Specification v1.1.0 Annex B."""
+
+    @classmethod
+    def _parse_tag_raw(cls, do: bytes) -> Tuple[int, bytes]:
+        return dgi_parse_tag_raw(do)
+
+    @classmethod
+    def _parse_len(cls, do: bytes) -> Tuple[int, bytes]:
+        return dgi_parse_len(do)
+
+    def _encode_tag(self) -> bytes:
+        return dgi_encode_tag(self._compute_tag())
+
+    def _encode_len(self, val: bytes) -> bytes:
+        return dgi_encode_len(len(val))
+
+
+class TLV_IE_Collection(metaclass=TlvCollectionMeta):
+    # we specify the metaclass so any downstream subclasses will automatically use it
+    """A TLV_IE_Collection consists of multiple TLV_IE classes identified by their tags.
+    A given encoded DO may contain any of them in any order, and may contain multiple instances
+    of each DO."""
+    # this is overridden by the TlvCollectionMeta metaclass, if it is used to create subclasses
+    possible_nested = []
+
+    def __init__(self, desc=None, **kwargs):
+        self.desc = desc
+        #print("possible_nested: ", self.possible_nested)
+        self.members = kwargs.get('nested', self.possible_nested)
+        self.members_by_tag = {}
+        self.members_by_name = {}
+        self.members_by_tag = {m.tag: m for m in self.members}
+        self.members_by_name = {camel_to_snake(m.__name__): m for m in self.members}
+        # if we are a constructed IE, [ordered] list of actual child-IE instances
+        self.children = kwargs.get('children', [])
+        self.encoded = None
+
+    def __str__(self):
+        member_strs = [str(x) for x in self.members]
+        return '%s(%s)' % (type(self).__name__, ','.join(member_strs))
+
+    def __repr__(self):
+        member_strs = [repr(x) for x in self.members]
+        return '%s(%s)' % (self.__class__, ','.join(member_strs))
+
+    def __add__(self, other):
+        """Extending TLV_IE_Collections with other TLV_IE_Collections or TLV_IEs."""
+        if isinstance(other, TLV_IE_Collection):
+            # adding one collection to another
+            members = self.members + other.members
+            return TLV_IE_Collection(self.desc, nested=members)
+        elif inspect.isclass(other) and issubclass(other, TLV_IE):
+            # adding a member to a collection
+            return TLV_IE_Collection(self.desc, nested=self.members + [other])
+        else:
+            raise TypeError
+
+    def from_bytes(self, binary: bytes, context: dict = {}) -> List[TLV_IE]:
+        """Create a list of TLV_IEs from the collection based on binary input data.
+        Args:
+            binary : binary bytes of encoded data
+        Returns:
+            list of instances of TLV_IE sub-classes containing parsed data
+        """
+        self.encoded = binary
+        # list of instances of TLV_IE collection member classes appearing in the data
+        res = []
+        remainder = binary
+        first = next(iter(self.members_by_tag.values()))
+        # iterate until no binary trailer is left
+        while len(remainder):
+            context['siblings'] = res
+            # obtain the tag at the start of the remainder
+            tag, _r = first._parse_tag_raw(remainder)
+            if tag is None:
+                break
+            if tag in self.members_by_tag:
+                cls = self.members_by_tag[tag]
+                # create an instance and parse accordingly
+                inst = cls()
+                _dec, remainder = inst.from_tlv(remainder, context=context)
+                res.append(inst)
+            else:
+                # unknown tag; create the related class on-the-fly using the same base class
+                name = 'unknown_%s_%X' % (first.__base__.__name__, tag)
+                cls = type(name, (first.__base__,), {'tag': tag, 'possible_nested': [],
+                                                     'nested_collection_cls': None})
+                cls._from_bytes = lambda s, a: {'raw': a.hex()}
+                cls._to_bytes = lambda s: bytes.fromhex(s.decoded['raw'])
+                # create an instance and parse accordingly
+                inst = cls()
+                _dec, remainder = inst.from_tlv(remainder, context=context)
+                res.append(inst)
+        self.children = res
+        return res
+
+    def from_dict(self, decoded: List[dict]) -> List[TLV_IE]:
+        """Create a list of TLV_IE instances from the collection based on an array
+        of dicts, where they key indicates the name of the TLV_IE subclass to use."""
+        # list of instances of TLV_IE collection member classes appearing in the data
+        res = []
+        # iterate over members of the list passed into "decoded"
+        for i in decoded:
+            # iterate over all the keys (typically one!) within the current list item dict
+            for k in i.keys():
+                # check if we have a member identified by the dict key
+                if k in self.members_by_name:
+                    # resolve the class for that name; create an instance of it
+                    cls = self.members_by_name[k]
+                    inst = cls()
+                    if cls.nested_collection_cls:
+                        # in case of collections, we want to pass the raw "value" portion to from_dict,
+                        # as to_dict() below intentionally omits the collection-class-name as key
+                        inst.from_dict(i[k])
+                    else:
+                        inst.from_dict({k: i[k]})
+                    res.append(inst)
+                else:
+                    raise ValueError('%s: Unknown TLV Class %s in %s; expected %s' %
+                                     (self, k, decoded, self.members_by_name.keys()))
+        self.children = res
+        return res
+
+    def to_dict(self):
+        # we intentionally return not a dict, but a list of dicts.  We could prefix by
+        # self.__class__.__name__, but that is usually some meaningless auto-generated  collection name.
+        return [x.to_dict() for x in self.children]
+
+    def to_bytes(self, context: dict = {}):
+        out = b''
+        context['siblings'] = self.children
+        for c in self.children:
+            out += c.to_tlv(context=context)
+        return out
+
+    def from_tlv(self, do, context: dict = {}):
+        return self.from_bytes(do, context=context)
+
+    def to_tlv(self, context: dict = {}):
+        return self.to_bytes(context=context)
+
+
+def flatten_dict_lists(inp):
+    """hierarchically flatten each list-of-dicts into a single dict. This is useful to
+       make the output of hierarchical TLV decoder structures flatter and more easy to read."""
+    def are_all_elements_dict(l):
+        for e in l:
+            if not isinstance(e, dict):
+                return False
+        return True
+
+    def are_elements_unique(lod):
+        set_of_keys = {list(x.keys())[0] for x in lod}
+        return len(lod) == len(set_of_keys)
+
+    if isinstance(inp, list):
+        if are_all_elements_dict(inp) and are_elements_unique(inp):
+            # flatten into one shared dict
+            newdict = {}
+            for e in inp:
+                key = list(e.keys())[0]
+                newdict[key] = e[key]
+            inp = newdict
+            # process result as any native dict
+            return {k:flatten_dict_lists(v) for k,v in inp.items()}
+        else:
+            return [flatten_dict_lists(x) for x in inp]
+    elif isinstance(inp, dict):
+        return {k:flatten_dict_lists(v) for k,v in inp.items()}
+    else:
+        return inp

pySim/transport/__init__.py

@@ -1,11 +1,21 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
 """ pySim: PCSC reader transport link base
 """
 
+import os
+import abc
+import argparse
+from typing import Optional, Tuple
+from construct import Construct
+
+from pySim.exceptions import *
+from pySim.utils import sw_match, b2h, h2b, i2h, Hexstr, SwHexstr, SwMatchstr, ResTuple
+from pySim.cat import ProactiveCommand, CommandDetails, DeviceIdentities, Result
+
 #
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
+# Copyright (C) 2021-2023 Harald Welte <laforge@osmocom.org>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -21,68 +31,229 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-class LinkBase(object):
 
-	def wait_for_card(self, timeout=None, newcardonly=False):
-		"""wait_for_card(): Wait for a card and connect to it
+class ApduTracer:
+    def trace_command(self, cmd):
+        pass
 
-		   timeout     : Maximum wait time (None=no timeout)
-		   newcardonly : Should we wait for a new card, or an already
-		                 inserted one ?
-		"""
-		pass
+    def trace_response(self, cmd, sw, resp):
+        pass
 
-	def connect(self):
-		"""connect(): Connect to a card immediately
-		"""
-		pass
+class ProactiveHandler(abc.ABC):
+    """Abstract base class representing the interface of some code that handles
+    the proactive commands, as returned by the card in responses to the FETCH
+    command."""
+    def receive_fetch_raw(self, pcmd: ProactiveCommand, parsed: Hexstr):
+        # try to find a generic handler like handle_SendShortMessage
+        handle_name = 'handle_%s' % type(parsed).__name__
+        if hasattr(self, handle_name):
+            handler = getattr(self, handle_name)
+            return handler(pcmd.decoded)
+        # fall back to common handler
+        return self.receive_fetch(pcmd)
 
-	def disconnect(self):
-		"""disconnect(): Disconnect from card
-		"""
-		pass
+    def receive_fetch(self, pcmd: ProactiveCommand):
+        """Default handler for not otherwise handled proactive commands."""
+        raise NotImplementedError('No handler method for %s' % pcmd.decoded)
 
-	def reset_card(self):
-		"""reset_card(): Resets the card (power down/up)
-		"""
-		pass
 
-	def send_apdu_raw(self, pdu):
-		"""send_apdu_raw(pdu): Sends an APDU with minimal processing
 
-		   pdu    : string of hexadecimal characters (ex. "A0A40000023F00")
-		   return : tuple(data, sw), where
-		            data : string (in hex) of returned data (ex. "074F4EFFFF")
-		            sw   : string (in hex) of status word (ex. "9000")
-		"""
-		pass
+class LinkBase(abc.ABC):
+    """Base class for link/transport to card."""
 
-	def send_apdu(self, pdu):
-		"""send_apdu(pdu): Sends an APDU and auto fetch response data
+    def __init__(self, sw_interpreter=None, apdu_tracer: Optional[ApduTracer]=None,
+                 proactive_handler: Optional[ProactiveHandler]=None):
+        self.sw_interpreter = sw_interpreter
+        self.apdu_tracer = apdu_tracer
+        self.proactive_handler = proactive_handler
 
-		   pdu    : string of hexadecimal characters (ex. "A0A40000023F00")
-		   return : tuple(data, sw), where
-		            data : string (in hex) of returned data (ex. "074F4EFFFF")
-		            sw   : string (in hex) of status word (ex. "9000")
-		"""
-		data, sw = self.send_apdu_raw(pdu)
+    @abc.abstractmethod
+    def __str__(self) -> str:
+        """Implementation specific method for printing an information to identify the device."""
 
-		if (sw is not None) and (sw[0:2] == '9f'):
-			pdu_gr = pdu[0:2] + 'c00000' + sw[2:4]
-			data, sw = self.send_apdu_raw(pdu_gr)
+    @abc.abstractmethod
+    def _send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
+        """Implementation specific method for sending the PDU."""
 
-		return data, sw
+    def set_sw_interpreter(self, interp):
+        """Set an (optional) status word interpreter."""
+        self.sw_interpreter = interp
 
-	def send_apdu_checksw(self, pdu, sw="9000"):
-		"""send_apdu_checksw(pdu,sw): Sends an APDU and check returned SW
+    @abc.abstractmethod
+    def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
+        """Wait for a card and connect to it
 
-		   pdu    : string of hexadecimal characters (ex. "A0A40000023F00")
-		   sw     : string of 4 hexadecimal characters (ex. "9000")
-		   return : tuple(data, sw), where
-		            data : string (in hex) of returned data (ex. "074F4EFFFF")
-		            sw   : string (in hex) of status word (ex. "9000")
-		"""
-		rv = self.send_apdu(pdu)
-		if sw.lower() != rv[1]:
-			raise RuntimeError("SW match failed ! Expected %s and got %s." % (sw.lower(), rv[1]))
-		return rv
+        Args:
+           timeout : Maximum wait time in seconds (None=no timeout)
+           newcardonly : Should we wait for a new card, or an already inserted one ?
+        """
+
+    @abc.abstractmethod
+    def connect(self):
+        """Connect to a card immediately
+        """
+
+    @abc.abstractmethod
+    def disconnect(self):
+        """Disconnect from card
+        """
+
+    @abc.abstractmethod
+    def reset_card(self):
+        """Resets the card (power down/up)
+        """
+
+    def send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
+        """Sends an APDU with minimal processing
+
+        Args:
+           pdu : string of hexadecimal characters (ex. "A0A40000023F00")
+        Returns:
+           tuple(data, sw), where
+                        data : string (in hex) of returned data (ex. "074F4EFFFF")
+                        sw   : string (in hex) of status word (ex. "9000")
+        """
+        if self.apdu_tracer:
+            self.apdu_tracer.trace_command(pdu)
+        (data, sw) = self._send_apdu_raw(pdu)
+        if self.apdu_tracer:
+            self.apdu_tracer.trace_response(pdu, sw, data)
+        return (data, sw)
+
+    def send_apdu(self, pdu: Hexstr) -> ResTuple:
+        """Sends an APDU and auto fetch response data
+
+        Args:
+           pdu : string of hexadecimal characters (ex. "A0A40000023F00")
+        Returns:
+           tuple(data, sw), where
+                        data : string (in hex) of returned data (ex. "074F4EFFFF")
+                        sw   : string (in hex) of status word (ex. "9000")
+        """
+        data, sw = self.send_apdu_raw(pdu)
+
+        # When we have sent the first APDU, the SW may indicate that there are response bytes
+        # available. There are two SWs commonly used for this 9fxx (sim) and 61xx (usim), where
+        # xx is the number of response bytes available.
+        # See also:
+        if sw is not None:
+            while ((sw[0:2] == '9f') or (sw[0:2] == '61')):
+                # SW1=9F: 3GPP TS 51.011 9.4.1, Responses to commands which are correctly executed
+                # SW1=61: ISO/IEC 7816-4, Table 5 — General meaning of the interindustry values of SW1-SW2
+                pdu_gr = pdu[0:2] + 'c00000' + sw[2:4]
+                d, sw = self.send_apdu_raw(pdu_gr)
+                data += d
+            if sw[0:2] == '6c':
+                # SW1=6C: ETSI TS 102 221 Table 7.1: Procedure byte coding
+                pdu_gr = pdu[0:8] + sw[2:4]
+                data, sw = self.send_apdu_raw(pdu_gr)
+
+        return data, sw
+
+    def send_apdu_checksw(self, pdu: Hexstr, sw: SwMatchstr = "9000") -> ResTuple:
+        """Sends an APDU and check returned SW
+
+        Args:
+           pdu : string of hexadecimal characters (ex. "A0A40000023F00")
+           sw : string of 4 hexadecimal characters (ex. "9000"). The user may mask out certain
+                        digits using a '?' to add some ambiguity if needed.
+        Returns:
+                tuple(data, sw), where
+                        data : string (in hex) of returned data (ex. "074F4EFFFF")
+                        sw   : string (in hex) of status word (ex. "9000")
+        """
+        rv = self.send_apdu(pdu)
+        last_sw = rv[1]
+
+        while sw == '9000' and sw_match(last_sw, '91xx'):
+            # It *was* successful after all -- the extra pieces FETCH handled
+            # need not concern the caller.
+            rv = (rv[0], '9000')
+            # proactive sim as per TS 102 221 Setion 7.4.2
+            # TODO: Check SW manually to avoid recursing on the stack (provided this piece of code stays in this place)
+            fetch_rv = self.send_apdu_checksw('80120000' + last_sw[2:], sw)
+            # Setting this in case we later decide not to send a terminal
+            # response immediately unconditionally -- the card may still have
+            # something pending even though the last command was not processed
+            # yet.
+            last_sw = fetch_rv[1]
+            # parse the proactive command
+            pcmd = ProactiveCommand()
+            parsed = pcmd.from_tlv(h2b(fetch_rv[0]))
+            print("FETCH: %s (%s)" % (fetch_rv[0], type(parsed).__name__))
+            result = Result()
+            if self.proactive_handler:
+                # Extension point: If this does return a list of TLV objects,
+                # they could be appended after the Result; if the first is a
+                # Result, that cuold replace the one built here.
+                self.proactive_handler.receive_fetch_raw(pcmd, parsed)
+                result.from_dict({'general_result': 'performed_successfully', 'additional_information': ''})
+            else:
+                result.from_dict({'general_result': 'command_beyond_terminal_capability', 'additional_information': ''})
+
+            # Send response immediately, thus also flushing out any further
+            # proactive commands that the card already wants to send
+            #
+            # Structure as per TS 102 223 V4.4.0 Section 6.8
+
+            # The Command Details are echoed from the command that has been processed.
+            (command_details,) = [c for c in pcmd.decoded.children if isinstance(c, CommandDetails)]
+            # The Device Identities are fixed. (TS 102 223 V4.0.0 Section 6.8.2)
+            device_identities = DeviceIdentities()
+            device_identities.from_dict({'source_dev_id': 'terminal', 'dest_dev_id': 'uicc'})
+
+            # Testing hint: The value of tail does not influence the behavior
+            # of an SJA2 that sent ans SMS, so this is implemented only
+            # following TS 102 223, and not fully tested.
+            tail = command_details.to_tlv() + device_identities.to_tlv() + result.to_tlv()
+            # Testing hint: In contrast to the above, this part is positively
+            # essential to get the SJA2 to provide the later parts of a
+            # multipart SMS in response to an OTA RFM command.
+            terminal_response = '80140000' + b2h(len(tail).to_bytes(1, 'big') + tail)
+
+            terminal_response_rv = self.send_apdu(terminal_response)
+            last_sw = terminal_response_rv[1]
+
+        if not sw_match(rv[1], sw):
+            raise SwMatchError(rv[1], sw.lower(), self.sw_interpreter)
+        return rv
+
+def argparse_add_reader_args(arg_parser: argparse.ArgumentParser):
+    """Add all reader related arguments to the given argparse.Argumentparser instance."""
+    from pySim.transport.serial import SerialSimLink
+    from pySim.transport.pcsc import PcscSimLink
+    from pySim.transport.modem_atcmd import ModemATCommandLink
+    from pySim.transport.calypso import CalypsoSimLink
+
+    SerialSimLink.argparse_add_reader_args(arg_parser)
+    PcscSimLink.argparse_add_reader_args(arg_parser)
+    ModemATCommandLink.argparse_add_reader_args(arg_parser)
+    CalypsoSimLink.argparse_add_reader_args(arg_parser)
+
+    return arg_parser
+
+
+def init_reader(opts, **kwargs) -> LinkBase:
+    """
+    Init card reader driver
+    """
+    if opts.pcsc_dev is not None or opts.pcsc_regex is not None:
+        from pySim.transport.pcsc import PcscSimLink
+        sl = PcscSimLink(opts, **kwargs)
+    elif opts.osmocon_sock is not None:
+        from pySim.transport.calypso import CalypsoSimLink
+        sl = CalypsoSimLink(opts, **kwargs)
+    elif opts.modem_dev is not None:
+        from pySim.transport.modem_atcmd import ModemATCommandLink
+        sl = ModemATCommandLink(opts, **kwargs)
+    else:  # Serial reader is default
+        print("No reader/driver specified; falling back to default (Serial reader)")
+        from pySim.transport.serial import SerialSimLink
+        sl = SerialSimLink(opts, **kwargs)
+
+    if os.environ.get('PYSIM_INTEGRATION_TEST') == "1":
+        print("Using %s reader interface" % (sl.name))
+    else:
+        print("Using reader %s" % sl)
+
+    return sl

pySim/transport/calypso.py

@@ -0,0 +1,175 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2018 Vadim Yanitskiy <axilirator@gmail.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import select
+import struct
+import socket
+import os
+import argparse
+from typing import Optional
+
+from pySim.transport import LinkBase
+from pySim.exceptions import ReaderError, ProtocolError
+from pySim.utils import h2b, b2h, Hexstr, ResTuple
+
+
+class L1CTLMessage:
+
+    # Every (encoded) L1CTL message has the following structure:
+    #  - msg_length (2 bytes, net order)
+    #  - l1ctl_hdr (packed structure)
+    #    - msg_type
+    #    - flags
+    #    - padding (2 spare bytes)
+    #  - ... payload ...
+
+    def __init__(self, msg_type, flags=0x00):
+        # Init L1CTL message header
+        self.data = struct.pack("BBxx", msg_type, flags)
+
+    def gen_msg(self):
+        return struct.pack("!H", len(self.data)) + self.data
+
+
+class L1CTLMessageReset(L1CTLMessage):
+
+    # L1CTL message types
+    L1CTL_RESET_REQ = 0x0d
+    L1CTL_RESET_IND = 0x07
+    L1CTL_RESET_CONF = 0x0e
+
+    # Reset types
+    L1CTL_RES_T_BOOT = 0x00
+    L1CTL_RES_T_FULL = 0x01
+    L1CTL_RES_T_SCHED = 0x02
+
+    def __init__(self, ttype=L1CTL_RES_T_FULL):
+        super().__init__(self.L1CTL_RESET_REQ)
+        self.data += struct.pack("Bxxx", ttype)
+
+
+class L1CTLMessageSIM(L1CTLMessage):
+
+    # SIM related message types
+    L1CTL_SIM_REQ = 0x16
+    L1CTL_SIM_CONF = 0x17
+
+    def __init__(self, pdu):
+        super().__init__(self.L1CTL_SIM_REQ)
+        self.data += pdu
+
+
+class CalypsoSimLink(LinkBase):
+    """Transport Link for Calypso based phones."""
+    name = 'Calypso-based (OsmocomBB) reader'
+
+    def __init__(self, opts: argparse.Namespace = argparse.Namespace(osmocon_sock="/tmp/osmocom_l2"), **kwargs):
+        sock_path = opts.osmocon_sock
+        super().__init__(**kwargs)
+        # Make sure that a given socket path exists
+        if not os.path.exists(sock_path):
+            raise ReaderError(
+                "There is no such ('%s') UNIX socket" % sock_path)
+
+        print("Connecting to osmocon at '%s'..." % sock_path)
+
+        # Establish a client connection
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        self.sock.connect(sock_path)
+
+        # Remember socket path
+        self._sock_path = sock_path
+
+    def __del__(self):
+        self.sock.close()
+
+    def wait_for_rsp(self, exp_len: int = 128):
+        # Wait for incoming data (timeout is 3 seconds)
+        s, _, _ = select.select([self.sock], [], [], 3.0)
+        if not s:
+            raise ReaderError("Timeout waiting for card response")
+
+        # Receive expected amount of bytes from osmocon
+        rsp = self.sock.recv(exp_len)
+        return rsp
+
+    def reset_card(self):
+        # Request FULL reset
+        req_msg = L1CTLMessageReset()
+        self.sock.send(req_msg.gen_msg())
+
+        # Wait for confirmation
+        rsp = self.wait_for_rsp()
+        rsp_msg = struct.unpack_from("!HB", rsp)
+        if rsp_msg[1] != L1CTLMessageReset.L1CTL_RESET_CONF:
+            raise ReaderError("Failed to reset Calypso PHY")
+
+    def connect(self):
+        self.reset_card()
+
+    def disconnect(self):
+        pass  # Nothing to do really ...
+
+    def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
+        pass  # Nothing to do really ...
+
+    def _send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
+
+        # Request FULL reset
+        req_msg = L1CTLMessageSIM(h2b(pdu))
+        self.sock.send(req_msg.gen_msg())
+
+        # Read message length first
+        rsp = self.wait_for_rsp(struct.calcsize("!H"))
+        msg_len = struct.unpack_from("!H", rsp)[0]
+        if msg_len < struct.calcsize("BBxx"):
+            raise ReaderError("Missing L1CTL header for L1CTL_SIM_CONF")
+
+        # Read the whole message then
+        rsp = self.sock.recv(msg_len)
+
+        # Verify L1CTL header
+        hdr = struct.unpack_from("BBxx", rsp)
+        if hdr[0] != L1CTLMessageSIM.L1CTL_SIM_CONF:
+            raise ReaderError("Unexpected L1CTL message received")
+
+        # Verify the payload length
+        offset = struct.calcsize("BBxx")
+        if len(rsp) <= offset:
+            raise ProtocolError("Empty response from SIM?!?")
+
+        # Omit L1CTL header
+        rsp = rsp[offset:]
+
+        # Unpack data and SW
+        data = rsp[:-2]
+        sw = rsp[-2:]
+
+        return b2h(data), b2h(sw)
+
+    def __str__(self) -> str:
+        return "osmocon:%s" % (self._sock_path)
+
+    @staticmethod
+    def argparse_add_reader_args(arg_parser: argparse.ArgumentParser):
+        osmobb_group = arg_parser.add_argument_group('OsmocomBB Reader', """Use an OsmocomBB compatible phone
+to access the SIM inserted to the phone SIM slot.  This will require you to run the OsmocomBB firmware inside
+the phone (can be ram-loaded).  It also requires that you run the ``osmocon`` program, which provides a unix
+domain socket to which this reader driver can attach.""")
+        osmobb_group.add_argument('--osmocon', dest='osmocon_sock', metavar='PATH', default=None,
+                                  help='Socket path for Calypso (e.g. Motorola C1XX) based reader (via OsmocomBB)')

pySim/transport/modem_atcmd.py

@@ -0,0 +1,187 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2020 Vadim Yanitskiy <axilirator@gmail.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import logging as log
+import time
+import re
+import argparse
+from typing import Optional
+import serial
+
+from pySim.utils import Hexstr, ResTuple
+from pySim.transport import LinkBase
+from pySim.exceptions import ReaderError, ProtocolError
+
+# HACK: if somebody needs to debug this thing
+# log.root.setLevel(log.DEBUG)
+
+
+class ModemATCommandLink(LinkBase):
+    """Transport Link for 3GPP TS 27.007 compliant modems."""
+    name = "modem for Generic SIM Access (3GPP TS 27.007)"
+
+    def __init__(self, opts: argparse.Namespace = argparse.Namespace(modem_dev='/dev/ttyUSB0',
+                                                                     modem_baud=115200), **kwargs):
+        device = opts.modem_dev
+        baudrate = opts.modem_baud
+        super().__init__(**kwargs)
+        self._sl = serial.Serial(device, baudrate, timeout=5)
+        self._echo = False		# this will be auto-detected by _check_echo()
+        self._device = device
+        self._atr = None
+
+        # Check the AT interface
+        self._check_echo()
+
+        # Trigger initial reset
+        self.reset_card()
+
+    def __del__(self):
+        if hasattr(self, '_sl'):
+            self._sl.close()
+
+    def send_at_cmd(self, cmd, timeout=0.2, patience=0.002):
+        # Convert from string to bytes, if needed
+        bcmd = cmd if isinstance(cmd, bytes) else cmd.encode()
+        bcmd += b'\r'
+
+        # Clean input buffer from previous/unexpected data
+        self._sl.reset_input_buffer()
+
+        # Send command to the modem
+        log.debug('Sending AT command: %s', cmd)
+        try:
+            wlen = self._sl.write(bcmd)
+            assert wlen == len(bcmd)
+        except Exception as exc:
+            raise ReaderError('Failed to send AT command: %s' % cmd) from exc
+
+        rsp = b''
+        its = 1
+        t_start = time.time()
+        while True:
+            rsp = rsp + self._sl.read(self._sl.in_waiting)
+            lines = rsp.split(b'\r\n')
+            if len(lines) >= 2:
+                res = lines[-2]
+                if res == b'OK':
+                    log.debug('Command finished with result: %s', res)
+                    break
+                if res == b'ERROR' or res.startswith(b'+CME ERROR:'):
+                    log.error('Command failed with result: %s', res)
+                    break
+
+            if time.time() - t_start >= timeout:
+                log.info('Command finished with timeout >= %ss', timeout)
+                break
+            time.sleep(patience)
+            its += 1
+        log.debug('Command took %0.6fs (%d cycles a %fs)', time.time() - t_start, its, patience)
+
+        if self._echo:
+            # Skip echo chars
+            rsp = rsp[wlen:]
+        rsp = rsp.strip()
+        rsp = rsp.split(b'\r\n\r\n')
+
+        log.debug('Got response from modem: %s', rsp)
+        return rsp
+
+    def _check_echo(self):
+        """Verify the correct response to 'AT' command
+        and detect if inputs are echoed by the device
+
+        Although echo of inputs can be enabled/disabled via
+        ATE1/ATE0, respectively, we rather detect the current
+        configuration of the modem without any change.
+        """
+        # Next command shall not strip the echo from the response
+        self._echo = False
+        result = self.send_at_cmd('AT')
+
+        # Verify the response
+        if len(result) > 0:
+            if result[-1] == b'OK':
+                self._echo = False
+                return
+            if result[-1] == b'AT\r\r\nOK':
+                self._echo = True
+                return
+        raise ReaderError('Interface \'%s\' does not respond to \'AT\' command' % self._device)
+
+    def reset_card(self):
+        # Reset the modem, just to be sure
+        if self.send_at_cmd('ATZ') != [b'OK']:
+            raise ReaderError('Failed to reset the modem')
+
+        # Make sure that generic SIM access is supported
+        if self.send_at_cmd('AT+CSIM=?') != [b'OK']:
+            raise ReaderError('The modem does not seem to support SIM access')
+
+        log.info('Modem at \'%s\' is ready!', self._device)
+
+    def connect(self):
+        pass  # Nothing to do really ...
+
+    def disconnect(self):
+        pass  # Nothing to do really ...
+
+    def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
+        pass  # Nothing to do really ...
+
+    def _send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
+        # Make sure pdu has upper case hex digits [A-F]
+        pdu = pdu.upper()
+
+        # Prepare the command as described in 8.17
+        cmd = 'AT+CSIM=%d,\"%s\"' % (len(pdu), pdu)
+        log.debug('Sending command: %s',  cmd)
+
+        # Send AT+CSIM command to the modem
+        rsp = self.send_at_cmd(cmd)
+        if rsp[-1].startswith(b'+CME ERROR:'):
+            raise ProtocolError('AT+CSIM failed with: %s' % str(rsp))
+        if len(rsp) != 2 or rsp[-1] != b'OK':
+            raise ReaderError('APDU transfer failed: %s' % str(rsp))
+        rsp = rsp[0]  # Get rid of b'OK'
+
+        # Make sure that the response has format: b'+CSIM: %d,\"%s\"'
+        try:
+            result = re.match(b'\+CSIM: (\d+),\"([0-9A-F]+)\"', rsp)
+            (_rsp_pdu_len, rsp_pdu) = result.groups()
+        except Exception as exc:
+            raise ReaderError('Failed to parse response from modem: %s' % rsp) from exc
+
+        # TODO: make sure we have at least SW
+        data = rsp_pdu[:-4].decode().lower()
+        sw = rsp_pdu[-4:].decode().lower()
+        log.debug('Command response: %s, %s',  data, sw)
+        return data, sw
+
+    def __str__(self) -> str:
+        return "modem:%s" % self._device
+
+    @staticmethod
+    def argparse_add_reader_args(arg_parser: argparse.ArgumentParser):
+        modem_group = arg_parser.add_argument_group('AT Command Modem Reader', """Talk to a SIM Card inside a
+mobile phone or cellular modem which is attached to this computer and offers an AT command interface including
+the AT+CSIM interface for Generic SIM access as specified in 3GPP TS 27.007.""")
+        modem_group.add_argument('--modem-device', dest='modem_dev', metavar='DEV', default=None,
+                                 help='Serial port of modem for Generic SIM Access (3GPP TS 27.007)')
+        modem_group.add_argument('--modem-baud', type=int, metavar='BAUD', default=115200,
+                                 help='Baud rate used for modem port')

pySim/transport/pcsc.py

@@ -1,12 +1,7 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-""" pySim: PCSC reader transport link
-"""
-
-#
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
-# Copyright (C) 2010  Harald Welte <laforge@gnumonks.org>
+# Copyright (C) 2010-2023  Harald Welte <laforge@gnumonks.org>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -22,59 +17,110 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
+import argparse
+import re
+from typing import Optional
+
+from smartcard.CardConnection import CardConnection
 from smartcard.CardRequest import CardRequest
-from smartcard.Exceptions import NoCardException, CardRequestTimeoutException
+from smartcard.Exceptions import NoCardException, CardRequestTimeoutException, CardConnectionException
 from smartcard.System import readers
 
-from pySim.exceptions import NoCardError
+from pySim.exceptions import NoCardError, ProtocolError, ReaderError
 from pySim.transport import LinkBase
-from pySim.utils import h2i, i2h
+from pySim.utils import h2i, i2h, Hexstr, ResTuple
 
 
 class PcscSimLink(LinkBase):
+    """ pySim: PCSC reader transport link."""
+    name = 'PC/SC'
 
-	def __init__(self, reader_number=0):
-		r = readers();
-		self._reader = r[reader_number]
-		self._con = self._reader.createConnection()
+    def __init__(self, opts: argparse.Namespace = argparse.Namespace(pcsc_dev=0), **kwargs):
+        super().__init__(**kwargs)
+        self._reader = None
+        r = readers()
+        if opts.pcsc_dev is not None:
+            # actual reader index number (integer)
+            reader_number = opts.pcsc_dev
+            if reader_number >= len(r):
+                raise ReaderError('No reader found for number %d' % reader_number)
+            self._reader = r[reader_number]
+        else:
+            # reader regex string
+            cre = re.compile(opts.pcsc_regex)
+            for reader in r:
+                if cre.search(reader.name):
+                    self._reader = reader
+                    break
+            if not self._reader:
+                raise ReaderError('No matching reader found for regex %s' % opts.pcsc_regex)
 
-	def __del__(self):
-		self._con.disconnect()
-		return
+        self._con = self._reader.createConnection()
 
-	def wait_for_card(self, timeout=None, newcardonly=False):
-		cr = CardRequest(readers=[self._reader], timeout=timeout, newcardonly=newcardonly)
-		try:
-			cr.waitforcard()
-		except CardRequestTimeoutException:
-			raise NoCardError()
-		self.connect()
+    def __del__(self):
+        try:
+            # FIXME: this causes multiple warnings in Python 3.5.3
+            self._con.disconnect()
+        except:
+            pass
 
-	def connect(self):
-		try:
-			self._con.connect()
-		except NoCardException:
-			raise NoCardError()
+    def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
+        cr = CardRequest(readers=[self._reader],
+                         timeout=timeout, newcardonly=newcardonly)
+        try:
+            cr.waitforcard()
+        except CardRequestTimeoutException as exc:
+            raise NoCardError() from exc
+        self.connect()
 
-	def disconnect(self):
-		self._con.disconnect()
+    def connect(self):
+        try:
+            # To avoid leakage of resources, make sure the reader
+            # is disconnected
+            self.disconnect()
 
-	def reset_card(self):
-		self._con.disconnect()
-		try:
-			self._con.connect()
-		except NoCardException:
-			raise NoCardError()
-		return 1
+            # Explicitly select T=0 communication protocol
+            self._con.connect(CardConnection.T0_protocol)
+        except CardConnectionException as exc:
+            raise ProtocolError() from exc
+        except NoCardException as exc:
+            raise NoCardError() from exc
 
-	def send_apdu_raw(self, pdu):
-		"""see LinkBase.send_apdu_raw"""
+    def get_atr(self) -> Hexstr:
+        return self._con.getATR()
 
-		apdu = h2i(pdu)
+    def disconnect(self):
+        self._con.disconnect()
 
-		data, sw1, sw2 = self._con.transmit(apdu)
+    def reset_card(self):
+        self.disconnect()
+        self.connect()
+        return 1
 
-		sw = [sw1, sw2]
+    def _send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
 
-		# Return value
-		return i2h(data), i2h(sw)
+        apdu = h2i(pdu)
+
+        data, sw1, sw2 = self._con.transmit(apdu)
+
+        sw = [sw1, sw2]
+
+        # Return value
+        return i2h(data), i2h(sw)
+
+    def __str__(self) -> str:
+        return "PCSC[%s]" % (self._reader)
+
+    @staticmethod
+    def argparse_add_reader_args(arg_parser: argparse.ArgumentParser):
+        pcsc_group = arg_parser.add_argument_group('PC/SC Reader',
+        """Use a PC/SC card reader to talk to the SIM card.  PC/SC is a standard API for how applications
+access smart card readers, and is available on a variety of operating systems, such as Microsoft
+Windows, MacOS X and Linux.  Most vendors of smart card readers provide drivers that offer a PC/SC
+interface, if not even a generic USB CCID driver is used.  You can use a tool like ``pcsc_scan -r``
+to obtain a list of readers available on your system. """)
+        dev_group = pcsc_group.add_mutually_exclusive_group()
+        dev_group.add_argument('-p', '--pcsc-device', type=int, dest='pcsc_dev', metavar='PCSC', default=None,
+                               help='Number of PC/SC reader to use for SIM access')
+        dev_group.add_argument('--pcsc-regex', type=str, dest='pcsc_regex', metavar='REGEX', default=None,
+                               help='Regex matching PC/SC reader to use for SIM access')

pySim/transport/serial.py

@@ -1,10 +1,5 @@
-#!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-""" pySim: Transport Link for serial (RS232) based readers included with simcard
-"""
-
-#
 # Copyright (C) 2009-2010  Sylvain Munaut <tnt@246tNt.com>
 #
 # This program is free software: you can redistribute it and/or modify
@@ -21,204 +16,237 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-from __future__ import absolute_import
-
-import serial
 import time
+import os
+import argparse
+from typing import Optional
+import serial
 
 from pySim.exceptions import NoCardError, ProtocolError
 from pySim.transport import LinkBase
-from pySim.utils import h2b, b2h
+from pySim.utils import h2b, b2h, Hexstr, ResTuple
 
 
 class SerialSimLink(LinkBase):
+    """ pySim: Transport Link for serial (RS232) based readers included with simcard"""
+    name = 'Serial'
 
-	def __init__(self, device='/dev/ttyUSB0', baudrate=9600, rst='-rts', debug=False):
-		self._sl = serial.Serial(
-				port = device,
-				parity = serial.PARITY_EVEN,
-				bytesize = serial.EIGHTBITS,
-				stopbits = serial.STOPBITS_TWO,
-				timeout = 1,
-				xonxoff = 0,
-				rtscts = 0,
-				baudrate = baudrate,
-			)
-		self._rst_pin = rst
-		self._debug = debug
+    def __init__(self, opts = argparse.Namespace(device='/dev/ttyUSB0', baudrate=9600), rst: str = '-rts',
+                 debug: bool = False, **kwargs):
+        super().__init__(**kwargs)
+        if not os.path.exists(opts.device):
+            raise ValueError("device file %s does not exist -- abort" % opts.device)
+        self._sl = serial.Serial(
+            port=opts.device,
+            parity=serial.PARITY_EVEN,
+            bytesize=serial.EIGHTBITS,
+            stopbits=serial.STOPBITS_TWO,
+            timeout=1,
+            xonxoff=0,
+            rtscts=0,
+            baudrate=opts.baudrate,
+        )
+        self._rst_pin = rst
+        self._debug = debug
+        self._atr = None
 
-	def __del__(self):
-		self._sl.close()
+    def __del__(self):
+        if hasattr(self, "_sl"):
+            self._sl.close()
 
-	def wait_for_card(self, timeout=None, newcardonly=False):
-		# Direct try
-		existing = False
+    def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
+        # Direct try
+        existing = False
 
-		try:
-			self.reset_card()
-			if not newcardonly:
-				return
-			else:
-				existing = True
-		except NoCardError:
-			pass
+        try:
+            self.reset_card()
+            if not newcardonly:
+                return
+            existing = True
+        except NoCardError:
+            pass
 
-		# Poll ...
-		mt = time.time() + timeout if timeout is not None else None
-		pe = 0
+        # Poll ...
+        mt = time.time() + timeout if timeout is not None else None
+        pe = 0
 
-		while (mt is None) or (time.time() < mt):
-			try:
-				time.sleep(0.5)
-				self.reset_card()
-				if not existing:
-					return
-			except NoCardError:
-				existing = False
-			except ProtocolError:
-				if existing:
-					existing = False
-				else:
-					# Tolerate a couple of protocol error ... can happen if
-					# we try when the card is 'half' inserted
-					pe += 1
-					if (pe > 2):
-						raise
+        while (mt is None) or (time.time() < mt):
+            try:
+                time.sleep(0.5)
+                self.reset_card()
+                if not existing:
+                    return
+            except NoCardError:
+                existing = False
+            except ProtocolError:
+                if existing:
+                    existing = False
+                else:
+                    # Tolerate a couple of protocol error ... can happen if
+                    # we try when the card is 'half' inserted
+                    pe += 1
+                    if pe > 2:
+                        raise
 
-		# Timed out ...
-		raise NoCardError()
+        # Timed out ...
+        raise NoCardError()
 
-	def connect(self):
-		self.reset_card()
+    def connect(self):
+        self.reset_card()
 
-	def disconnect(self):
-		pass # Nothing to do really ...
+    def get_atr(self) -> Hexstr:
+        return self._atr
 
-	def reset_card(self):
-		rv = self._reset_card()
-		if rv == 0:
-			raise NoCardError()
-		elif rv < 0:
-			raise ProtocolError()
+    def disconnect(self):
+        pass  # Nothing to do really ...
 
-	def _reset_card(self):
-		rst_meth_map = {
-			'rts': self._sl.setRTS,
-			'dtr': self._sl.setDTR,
-		}
-		rst_val_map = { '+':0, '-':1 }
+    def reset_card(self):
+        rv = self._reset_card()
+        if rv == 0:
+            raise NoCardError()
+        if rv < 0:
+            raise ProtocolError()
+        return rv
 
-		try:
-			rst_meth = rst_meth_map[self._rst_pin[1:]]
-			rst_val  = rst_val_map[self._rst_pin[0]]
-		except:
-			raise ValueError('Invalid reset pin %s' % self._rst_pin);
+    def _reset_card(self):
+        self._atr = None
+        rst_meth_map = {
+            'rts': self._sl.setRTS,
+            'dtr': self._sl.setDTR,
+        }
+        rst_val_map = {'+': 0, '-': 1}
 
-		rst_meth(rst_val)
-		time.sleep(0.1)  # 100 ms
-		self._sl.flushInput()
-		rst_meth(rst_val ^ 1)
+        try:
+            rst_meth = rst_meth_map[self._rst_pin[1:]]
+            rst_val = rst_val_map[self._rst_pin[0]]
+        except Exception as exc:
+            raise ValueError('Invalid reset pin %s' % self._rst_pin) from exc
 
-		b = self._rx_byte()
-		if not b:
-			return 0
-		if ord(b) != 0x3b:
-			return -1;
-		self._dbg_print("TS: 0x%x Direct convention" % ord(b))
+        rst_meth(rst_val)
+        time.sleep(0.1)  # 100 ms
+        self._sl.flushInput()
+        rst_meth(rst_val ^ 1)
 
-		while ord(b) == 0x3b:
-			b = self._rx_byte()
+        b = self._rx_byte()
+        if not b:
+            return 0
+        if ord(b) != 0x3b:
+            return -1
+        self._dbg_print("TS: 0x%x Direct convention" % ord(b))
 
-		if not b:
-			return -1
-		t0 = ord(b)
-		self._dbg_print("T0: 0x%x" % t0)
+        while ord(b) == 0x3b:
+            b = self._rx_byte()
 
-		for i in range(4):
-			if t0 & (0x10 << i):
-				self._dbg_print("T%si = %x" % (chr(ord('A')+i), ord(self._rx_byte())))
+        if not b:
+            return -1
+        t0 = ord(b)
+        self._dbg_print("T0: 0x%x" % t0)
+        self._atr = [0x3b, ord(b)]
 
-		for i in range(0, t0 & 0xf):
-			self._dbg_print("Historical = %x" % ord(self._rx_byte()))
+        for i in range(4):
+            if t0 & (0x10 << i):
+                b = self._rx_byte()
+                self._atr.append(ord(b))
+                self._dbg_print("T%si = %x" % (chr(ord('A')+i), ord(b)))
 
-		while True:
-			x = self._rx_byte()
-			if not x:
-				break
-			self._dbg_print("Extra: %x" % ord(x))
+        for i in range(0, t0 & 0xf):
+            b = self._rx_byte()
+            self._atr.append(ord(b))
+            self._dbg_print("Historical = %x" % ord(b))
 
-		return 1
+        while True:
+            x = self._rx_byte()
+            if not x:
+                break
+            self._atr.append(ord(x))
+            self._dbg_print("Extra: %x" % ord(x))
 
-	def _dbg_print(self, s):
-		if self._debug:
-			print s
+        return 1
 
-	def _tx_byte(self, b):
-		self._sl.write(b)
-		r = self._sl.read()
-		if r != b:	# TX and RX are tied, so we must clear the echo
-			raise ProtocolError("Bad echo value. Expected %02x, got %s)" % (ord(b), '%02x'%ord(r) if r else '(nil)'))
+    def _dbg_print(self, s):
+        if self._debug:
+            print(s)
 
-	def _tx_string(self, s):
-		"""This is only safe if it's guaranteed the card won't send any data
-		during the time of tx of the string !!!"""
-		self._sl.write(s)
-		r = self._sl.read(len(s))
-		if r != s:	# TX and RX are tied, so we must clear the echo
-			raise ProtocolError("Bad echo value (Expected: %s, got %s)" % (b2h(s), b2h(r)))
+    def _tx_byte(self, b):
+        self._sl.write(b)
+        r = self._sl.read()
+        if r != b:  # TX and RX are tied, so we must clear the echo
+            raise ProtocolError("Bad echo value. Expected %02x, got %s)" % (
+                ord(b), '%02x' % ord(r) if r else '(nil)'))
 
-	def _rx_byte(self):
-		return self._sl.read()
+    def _tx_string(self, s):
+        """This is only safe if it's guaranteed the card won't send any data
+        during the time of tx of the string !!!"""
+        self._sl.write(s)
+        r = self._sl.read(len(s))
+        if r != s:  # TX and RX are tied, so we must clear the echo
+            raise ProtocolError(
+                "Bad echo value (Expected: %s, got %s)" % (b2h(s), b2h(r)))
 
-	def send_apdu_raw(self, pdu):
-		"""see LinkBase.send_apdu_raw"""
+    def _rx_byte(self):
+        return self._sl.read()
 
-		pdu = h2b(pdu)
-		data_len = ord(pdu[4])	# P3
+    def _send_apdu_raw(self, pdu: Hexstr) -> ResTuple:
 
-		# Send first CLASS,INS,P1,P2,P3
-		self._tx_string(pdu[0:5])
+        pdu = h2b(pdu)
+        data_len = pdu[4]  # P3
 
-		# Wait ack which can be
-		#  - INS: Command acked -> go ahead
-		#  - 0x60: NULL, just wait some more
-		#  - SW1: The card can apparently proceed ...
-		while True:
-			b = self._rx_byte()
-			if b == pdu[1]:
-				break
-			elif b != '\x60':
-				# Ok, it 'could' be SW1
-				sw1 = b
-				sw2 = self._rx_byte()
-				nil = self._rx_byte()
-				if (sw2 and not nil):
-					return '', b2h(sw1+sw2)
+        # Send first CLASS,INS,P1,P2,P3
+        self._tx_string(pdu[0:5])
 
-				raise ProtocolError()
+        # Wait ack which can be
+        #  - INS: Command acked -> go ahead
+        #  - 0x60: NULL, just wait some more
+        #  - SW1: The card can apparently proceed ...
+        while True:
+            b = self._rx_byte()
+            if ord(b) == pdu[1]:
+                break
+            if b != '\x60':
+                # Ok, it 'could' be SW1
+                sw1 = b
+                sw2 = self._rx_byte()
+                nil = self._rx_byte()
+                if (sw2 and not nil):
+                    return '', b2h(sw1+sw2)
 
-		# Send data (if any)
-		if len(pdu) > 5:
-			self._tx_string(pdu[5:])
+                raise ProtocolError()
 
-		# Receive data (including SW !)
-		#  length = [P3 - tx_data (=len(pdu)-len(hdr)) + 2 (SW1/2) ]
-		to_recv = data_len - len(pdu) + 5 + 2
+        # Send data (if any)
+        if len(pdu) > 5:
+            self._tx_string(pdu[5:])
 
-		data = ''
-		while (len(data) < to_recv):
-			b = self._rx_byte()
-			if (to_recv == 2) and (b == '\x60'): # Ignore NIL if we have no RX data (hack ?)
-				continue
-			if not b:
-				break;
-			data += b
+        # Receive data (including SW !)
+        #  length = [P3 - tx_data (=len(pdu)-len(hdr)) + 2 (SW1//2) ]
+        to_recv = data_len - len(pdu) + 5 + 2
 
-		# Split datafield from SW
-		if len(data) < 2:
-			return None, None
-		sw = data[-2:]
-		data = data[0:-2]
+        data = bytes(0)
+        while len(data) < to_recv:
+            b = self._rx_byte()
+            if (to_recv == 2) and (b == '\x60'):  # Ignore NIL if we have no RX data (hack ?)
+                continue
+            if not b:
+                break
+            data += b
 
-		# Return value
-		return b2h(data), b2h(sw)
+        # Split datafield from SW
+        if len(data) < 2:
+            return None, None
+        sw = data[-2:]
+        data = data[0:-2]
+
+        # Return value
+        return b2h(data), b2h(sw)
+
+    def __str__(self) -> str:
+        return "serial:%s" % (self._sl.name)
+
+    @staticmethod
+    def argparse_add_reader_args(arg_parser: argparse.ArgumentParser):
+        serial_group = arg_parser.add_argument_group('Serial Reader', """Use a simple/ultra-low-cost serial reader
+attached to a (physical or USB/virtual) RS232 port.  This doesn't work with all RS232-attached smart card
+readers, only with the very primitive readers following the ancient `Phoenix` or `Smart Mouse` design.""")
+        serial_group.add_argument('-d', '--device', metavar='DEV', default='/dev/ttyUSB0',
+                                  help='Serial Device for SIM access')
+        serial_group.add_argument('-b', '--baud', dest='baudrate', type=int, metavar='BAUD', default=9600,
+                                  help='Baud rate used for SIM access')

pySim/ts_102_222.py

@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+
+# Interactive shell for working with SIM / UICC / USIM / ISIM cards
+#
+# (C) 2022 by Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from typing import List
+import argparse
+
+import cmd2
+from cmd2 import CommandSet, with_default_category
+
+from pySim.utils import b2h, auto_uint8, auto_uint16, is_hexstr
+
+from pySim.ts_102_221 import *
+
+@with_default_category('TS 102 222 Administrative Commands')
+class Ts102222Commands(CommandSet):
+    """Administrative commands for telecommunication applications."""
+
+    delfile_parser = argparse.ArgumentParser()
+    delfile_parser.add_argument('--force-delete', action='store_true',
+            help='I really want to permanently delete the file. I know pySim cannot re-create it yet!')
+    delfile_parser.add_argument('NAME', type=str, help='File name or FID to delete')
+
+    @cmd2.with_argparser(delfile_parser)
+    def do_delete_file(self, opts):
+        """Delete the specified file. DANGEROUS!  See TS 102 222 Section 6.4.
+        This will permanently delete the specified file from the card.
+        pySim has no support to re-create files yet, and even if it did, your card may not allow it!"""
+        if not opts.force_delete:
+            self._cmd.perror("Refusing to permanently delete the file, please read the help text.")
+            return
+        f = self._cmd.lchan.get_file_for_selectable(opts.NAME)
+        (_data, _sw) = self._cmd.lchan.scc.delete_file(f.fid)
+
+    def complete_delete_file(self, text, line, begidx, endidx) -> List[str]:
+        """Command Line tab completion for DELETE FILE"""
+        index_dict = {1: self._cmd.lchan.selected_file.get_selectable_names()}
+        return self._cmd.index_based_complete(text, line, begidx, endidx, index_dict=index_dict)
+
+    termdf_parser = argparse.ArgumentParser()
+    termdf_parser.add_argument('--force', action='store_true',
+            help='I really want to terminate the file. I know I can not recover from it!')
+    termdf_parser.add_argument('NAME', type=str, help='File name or FID')
+
+    @cmd2.with_argparser(termdf_parser)
+    def do_terminate_df(self, opts):
+        """Terminate the specified DF. DANGEROUS!  See TS 102 222 6.7.
+        This is a permanent, one-way operation on the card. There is no undo, you can not recover
+        a terminated DF.  The only permitted command for a terminated DF is the DLETE FILE command."""
+        if not opts.force:
+            self._cmd.perror("Refusing to terminate the file, please read the help text.")
+            return
+        f = self._cmd.lchan.get_file_for_selectable(opts.NAME)
+        (_data, _sw) = self._cmd.lchan.scc.terminate_df(f.fid)
+
+    def complete_terminate_df(self, text, line, begidx, endidx) -> List[str]:
+        """Command Line tab completion for TERMINATE DF"""
+        index_dict = {1: self._cmd.lchan.selected_file.get_selectable_names()}
+        return self._cmd.index_based_complete(text, line, begidx, endidx, index_dict=index_dict)
+
+    @cmd2.with_argparser(termdf_parser)
+    def do_terminate_ef(self, opts):
+        """Terminate the specified EF. DANGEROUS!  See TS 102 222 6.8.
+        This is a permanent, one-way operation on the card. There is no undo, you can not recover
+        a terminated EF.  The only permitted command for a terminated EF is the DLETE FILE command."""
+        if not opts.force:
+            self._cmd.perror("Refusing to terminate the file, please read the help text.")
+            return
+        f = self._cmd.lchan.get_file_for_selectable(opts.NAME)
+        (_data, _sw) = self._cmd.lchan.scc.terminate_ef(f.fid)
+
+    def complete_terminate_ef(self, text, line, begidx, endidx) -> List[str]:
+        """Command Line tab completion for TERMINATE EF"""
+        index_dict = {1: self._cmd.lchan.selected_file.get_selectable_names()}
+        return self._cmd.index_based_complete(text, line, begidx, endidx, index_dict=index_dict)
+
+    tcard_parser = argparse.ArgumentParser()
+    tcard_parser.add_argument('--force-terminate-card', action='store_true',
+            help='I really want to permanently terminate the card. It will not be usable afterwards!')
+
+    @cmd2.with_argparser(tcard_parser)
+    def do_terminate_card_usage(self, opts):
+        """Terminate the Card. SUPER DANGEROUS!  See TS 102 222 Section 6.9.
+        This will permanently brick the card and can NOT be recovered from!"""
+        if not opts.force_terminate_card:
+            self._cmd.perror("Refusing to permanently terminate the card, please read the help text.")
+            return
+        (_data, _sw) = self._cmd.lchan.scc.terminate_card_usage()
+
+    create_parser = argparse.ArgumentParser()
+    create_parser.add_argument('FILE_ID', type=is_hexstr, help='File Identifier as 4-character hex string')
+    create_parser._action_groups.pop()
+    create_required = create_parser.add_argument_group('required arguments')
+    create_optional = create_parser.add_argument_group('optional arguments')
+    create_required.add_argument('--ef-arr-file-id', required=True, type=str, help='Referenced Security: File Identifier of EF.ARR')
+    create_required.add_argument('--ef-arr-record-nr', required=True, type=auto_uint8, help='Referenced Security: Record Number within EF.ARR')
+    create_required.add_argument('--file-size', required=True, type=auto_uint16, help='Size of file in octets')
+    create_required.add_argument('--structure', required=True, type=str, choices=['transparent', 'linear_fixed', 'ber_tlv'],
+                                 help='Structure of the to-be-created EF')
+    create_optional.add_argument('--short-file-id', type=str, help='Short File Identifier as 2-digit hex string')
+    create_optional.add_argument('--shareable', action='store_true', help='Should the file be shareable?')
+    create_optional.add_argument('--record-length', type=auto_uint16, help='Length of each record in octets')
+
+    @cmd2.with_argparser(create_parser)
+    def do_create_ef(self, opts):
+        """Create a new EF below the currently selected DF.  Requires related privileges."""
+        file_descriptor = {
+            'file_descriptor_byte': {
+                'shareable': opts.shareable,
+                'file_type': 'working_ef',
+                'structure': opts.structure,
+            }
+        }
+        if opts.structure == 'linear_fixed':
+            if not opts.record_length:
+                self._cmd.perror("you must specify the --record-length for linear fixed EF")
+                return
+            file_descriptor['record_len'] = opts.record_length
+            file_descriptor['num_of_rec'] = opts.file_size // opts.record_length
+            if file_descriptor['num_of_rec'] * file_descriptor['record_len'] != opts.file_size:
+                raise ValueError("File size not evenly divisible by record length")
+        elif opts.structure == 'ber_tlv':
+            self._cmd.perror("BER-TLV creation not yet fully supported, sorry")
+            return
+        ies = [FileDescriptor(decoded=file_descriptor), FileIdentifier(decoded=opts.FILE_ID),
+               LifeCycleStatusInteger(decoded='operational_activated'),
+               SecurityAttribReferenced(decoded={'ef_arr_file_id': opts.ef_arr_file_id,
+                                                 'ef_arr_record_nr': opts.ef_arr_record_nr }),
+               FileSize(decoded=opts.file_size),
+               ShortFileIdentifier(decoded=opts.short_file_id),
+            ]
+        fcp = FcpTemplate(children=ies)
+        (_data, _sw) = self._cmd.lchan.scc.create_file(b2h(fcp.to_tlv()))
+        # the newly-created file is automatically selected but our runtime state knows nothing of it
+        self._cmd.lchan.select_file(self._cmd.lchan.selected_file)
+
+    createdf_parser = argparse.ArgumentParser()
+    createdf_parser.add_argument('FILE_ID', type=is_hexstr, help='File Identifier as 4-character hex string')
+    createdf_parser._action_groups.pop()
+    createdf_required = createdf_parser.add_argument_group('required arguments')
+    createdf_optional = createdf_parser.add_argument_group('optional arguments')
+    createdf_sja_optional = createdf_parser.add_argument_group('sysmoISIM-SJA optional arguments')
+    createdf_required.add_argument('--ef-arr-file-id', required=True, type=str, help='Referenced Security: File Identifier of EF.ARR')
+    createdf_required.add_argument('--ef-arr-record-nr', required=True, type=auto_uint8, help='Referenced Security: Record Number within EF.ARR')
+    createdf_optional.add_argument('--shareable', action='store_true', help='Should the file be shareable?')
+    createdf_optional.add_argument('--aid', type=is_hexstr, help='Application ID (creates an ADF, instead of a DF)')
+    # mandatory by spec, but ignored by several OS, so don't force the user
+    createdf_optional.add_argument('--total-file-size', type=auto_uint16, help='Physical memory allocated for DF/ADi in octets')
+    createdf_sja_optional.add_argument('--permit-rfm-create', action='store_true')
+    createdf_sja_optional.add_argument('--permit-rfm-delete-terminate', action='store_true')
+    createdf_sja_optional.add_argument('--permit-other-applet-create', action='store_true')
+    createdf_sja_optional.add_argument('--permit-other-applet-delete-terminate', action='store_true')
+
+    @cmd2.with_argparser(createdf_parser)
+    def do_create_df(self, opts):
+        """Create a new DF below the currently selected DF.  Requires related privileges."""
+        file_descriptor = {
+            'file_descriptor_byte': {
+                'shareable': opts.shareable,
+                'file_type': 'df',
+                'structure': 'no_info_given',
+            }
+        }
+        ies = []
+        ies.append(FileDescriptor(decoded=file_descriptor))
+        ies.append(FileIdentifier(decoded=opts.FILE_ID))
+        if opts.aid:
+            ies.append(DfName(decoded=opts.aid))
+        ies.append(LifeCycleStatusInteger(decoded='operational_activated'))
+        ies.append(SecurityAttribReferenced(decoded={'ef_arr_file_id': opts.ef_arr_file_id,
+                                                     'ef_arr_record_nr': opts.ef_arr_record_nr }))
+        if opts.total_file_size:
+            ies.append(TotalFileSize(decoded=opts.total_file_size))
+        # TODO: Spec states PIN Status Template DO is mandatory
+        if opts.permit_rfm_create or opts.permit_rfm_delete_terminate or opts.permit_other_applet_create or opts.permit_other_applet_delete_terminate:
+            toolkit_ac = {
+               'rfm_create': opts.permit_rfm_create,
+               'rfm_delete_terminate': opts.permit_rfm_delete_terminate,
+               'other_applet_create': opts.permit_other_applet_create,
+               'other_applet_delete_terminate': opts.permit_other_applet_delete_terminate,
+               }
+            ies.append(ProprietaryInformation(children=[ToolkitAccessConditions(decoded=toolkit_ac)]))
+        fcp = FcpTemplate(children=ies)
+        (_data, _sw) = self._cmd.lchan.scc.create_file(b2h(fcp.to_tlv()))
+        # the newly-created file is automatically selected but our runtime state knows nothing of it
+        self._cmd.lchan.select_file(self._cmd.lchan.selected_file)
+
+    resize_ef_parser = argparse.ArgumentParser()
+    resize_ef_parser.add_argument('NAME', type=str, help='Name or FID of file to be resized')
+    resize_ef_parser._action_groups.pop()
+    resize_ef_required = resize_ef_parser.add_argument_group('required arguments')
+    resize_ef_required.add_argument('--file-size', required=True, type=auto_uint16, help='Size of file in octets')
+
+    @cmd2.with_argparser(resize_ef_parser)
+    def do_resize_ef(self, opts):
+        """Resize an existing EF below the currently selected DF.  Requires related privileges."""
+        f = self._cmd.lchan.get_file_for_selectable(opts.NAME)
+        ies = [FileIdentifier(decoded=f.fid),
+               FileSize(decoded=opts.file_size)]
+        fcp = FcpTemplate(children=ies)
+        (_data, _sw) = self._cmd.lchan.scc.resize_file(b2h(fcp.to_tlv()))
+        # the resized file is automatically selected but our runtime state knows nothing of it
+        self._cmd.lchan.select_file(self._cmd.lchan.selected_file)
+
+    def complete_resize_ef(self, text, line, begidx, endidx) -> List[str]:
+        """Command Line tab completion for RESIZE EF"""
+        index_dict = {1: self._cmd.lchan.selected_file.get_selectable_names()}
+        return self._cmd.index_based_complete(text, line, begidx, endidx, index_dict=index_dict)

pySim/ts_102_310.py

@@ -0,0 +1,114 @@
+# coding=utf-8
+"""Utilities / Functions related to ETSI TS 102 310, the EAP UICC spec.
+
+(C) 2024 by Harald Welte <laforge@osmocom.org>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from pySim.construct import *
+from construct import *
+from construct import Optional as COptional
+
+#from pySim.utils import *
+from pySim.filesystem import CardDF, TransparentEF
+from pySim.tlv import BER_TLV_IE, TLV_IE_Collection
+
+# TS102 310 Section 7.1
+class EF_EAPKEYS(TransparentEF):
+    class Msk(BER_TLV_IE, tag=0x80):
+        _construct = HexAdapter(GreedyBytes)
+    class Emsk(BER_TLV_IE, tag=0x81):
+        _construct = HexAdapter(GreedyBytes)
+    class MskCollection(TLV_IE_Collection, nested=[EF_EAPKEYS.Msk, EF_EAPKEYS.Emsk]):
+        pass
+
+    def __init__(self, fid='4f01', name='EF.EAPKEYS', desc='EAP derived keys'):
+        super().__init__(fid, sfid=0x01, name=name, desc=desc, size=(1,None))
+        self._tlv = EF_EAPKEYS.MskCollection
+
+# TS 102 310 Section 7.2
+class EF_EAPSTATUS(TransparentEF):
+    def __init__(self, fid='4f02', name='EF.EAPSTATUS', desc='EAP Authentication Status'):
+        super().__init__(fid, sfid=0x02, name=name, desc=desc, size=(1,1))
+        self._construct = Enum(Int8ub, no_auth_started=0, authenticating=1,
+                               authenticated=2, held_auth_failure=3)
+
+# TS 102 310 Section 7.3
+class EF_PUId(TransparentEF):
+    def __init__(self, fid='4f03', name='EF.PUId', desc='Permanent User Identity'):
+        super().__init__(fid, sfid=0x03, name=name, desc=desc, size=(10,None))
+        self._construct = GreedyBytes
+
+# TS 102 310 Section 7.4
+class EF_Ps(TransparentEF):
+    def __init__(self, fid='4f04', name='EF.Ps', desc='Pseudonym'):
+        super().__init__(fid, sfid=0x04, name=name, desc=desc, size=(1,None))
+        self._construct = GreedyBytes
+
+# TS 102 310 Section 7.5
+class EF_CurID(TransparentEF):
+    def __init__(self, fid='4f20', name='EF.CurID', desc='Current Identity'):
+        super().__init__(fid, sfid=0x10, name=name, desc=desc, size=(1,None))
+        self._construct = Struct('type'/Enum(Int8ub, permanent=0, pseudonym=1, re_authentication=2, should_not_be_revealed=255),
+                                 '_len'/Int8ub,
+                                 'value'/Utf8Adapter(this._len))
+
+
+# TS 102 310 Section 7.6
+class EF_ReID(TransparentEF):
+    class Identity(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+    class Counter(BER_TLV_IE, tag=0x81):
+        _construct = GreedyInteger
+    class Collection(TLV_IE_Collection, nested=[EF_ReID.Identity, EF_ReID.Counter]):
+        pass
+
+    def __init__(self, fid='4f21', name='EF.ReID', desc='Re-Authentication Identity'):
+        super().__init__(fid, sfid=0x11, name=name, desc=desc, size=(1,None))
+        self._tlv = EF_ReID.Collection
+
+# TS 102 310 Section 7.7
+class EF_Realm(TransparentEF):
+    def __init__(self, fid='4f22', name='EF.Realm', desc='Relm value of the identity'):
+        super().__init__(fid, sfid=0x12, name=name, desc=desc, size=(1,None))
+        self._construct = Struct('_len'/Int8ub,
+                                 'realm'/Utf8Adapter(Bytes(this._len)))
+
+class DF_EAP(CardDF):
+    # DF.EAP has no default FID; it always must be discovered via the EF.DIR entry
+    # and the 0x73 "discretionary template"
+    def __init__(self, fid, name='DF.EAP', desc='EAP client', **kwargs):
+        super().__init__(fid=fid, name=name, desc=desc, **kwargs)
+        files = [
+            EF_EAPKEYS(),
+            EF_EAPSTATUS(),
+            EF_PUId(),
+            EF_CurID(),
+            EF_ReID(),
+        ]
+        self.add_files(files)
+
+
+# TS 102 310 Section 5.2
+class EapSupportedTypesList(BER_TLV_IE, tag=0x80):
+    _construct = GreedyRange(Int8ub)
+class EapDedicatedFilesList(BER_TLV_IE, tag=0x81):
+    _construct = GreedyRange(Int16ub)
+class EapLabel(BER_TLV_IE, tag=0x82):
+    _construct = GreedyBytes
+class EapAppSvcSpecData(BER_TLV_IE, tag=0xa0, nested=[EapSupportedTypesList, EapDedicatedFilesList, EapLabel]):
+    pass
+class DiscretionaryTemplate(BER_TLV_IE, tag=0x73, nested=[EapAppSvcSpecData]):
+    pass

pySim/ts_31_102_telecom.py

@@ -0,0 +1,311 @@
+# -*- coding: utf-8 -*-
+
+# without this, pylint will fail when inner classes are used
+# within the 'nested' kwarg of our TlvMeta metaclass on python 3.7 :(
+# pylint: disable=undefined-variable
+
+"""
+DF_PHONEBOOK, DF_MULTIMEDIA, DF_MCS as specified in 3GPP TS 31.102 V16.6.0
+Needs to be a separate python module to avoid cyclic imports
+"""
+
+#
+# Copyright (C) 2022 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from construct import Optional as COptional
+from construct import Struct, Int16ub, Int32ub
+
+from pySim.tlv import *
+from pySim.filesystem import *
+from pySim.construct import *
+
+# TS 31.102 Section 4.2.8
+class EF_UServiceTable(TransparentEF):
+    def __init__(self, fid, sfid, name, desc, size, table, **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, size=size, **kwargs)
+        self.table = table
+
+    @staticmethod
+    def _bit_byte_offset_for_service(service: int) -> Tuple[int, int]:
+        i = service - 1
+        byte_offset = i//8
+        bit_offset = i % 8
+        return (byte_offset, bit_offset)
+
+    def _decode_bin(self, in_bin):
+        ret = {}
+        for i in range(0, len(in_bin)):
+            byte = in_bin[i]
+            for bitno in range(0, 8):
+                service_nr = i * 8 + bitno + 1
+                ret[service_nr] = {
+                    'activated': True if byte & (1 << bitno) else False
+                }
+                if service_nr in self.table:
+                    ret[service_nr]['description'] = self.table[service_nr]
+        return ret
+
+    def _encode_bin(self, in_json):
+        # compute the required binary size
+        bin_len = 0
+        for srv in in_json.keys():
+            service_nr = int(srv)
+            (byte_offset, bit_offset) = EF_UServiceTable._bit_byte_offset_for_service(
+                service_nr)
+            if byte_offset >= bin_len:
+                bin_len = byte_offset+1
+        # encode the actual data
+        out = bytearray(b'\x00' * bin_len)
+        for srv in in_json.keys():
+            service_nr = int(srv)
+            (byte_offset, bit_offset) = EF_UServiceTable._bit_byte_offset_for_service(
+                service_nr)
+            if in_json[srv]['activated'] is True:
+                bit = 1
+            else:
+                bit = 0
+            out[byte_offset] |= (bit) << bit_offset
+        return out
+
+    def get_active_services(self, cmd):
+        # obtain list of currently active services
+        (service_data, _sw) = cmd.lchan.read_binary_dec()
+        active_services = []
+        for s in service_data.keys():
+            if service_data[s]['activated']:
+                active_services.append(s)
+        return active_services
+
+    def ust_service_check(self, cmd):
+        """Check consistency between services of this file and files present/activated"""
+        num_problems = 0
+        # obtain list of currently active services
+        active_services = self.get_active_services(cmd)
+        # iterate over all the service-constraints we know of
+        files_by_service = self.parent.files_by_service
+        try:
+            for s in sorted(files_by_service.keys()):
+                active_str = 'active' if s in active_services else 'inactive'
+                cmd.poutput("Checking service No %u (%s)" % (s, active_str))
+                for f in files_by_service[s]:
+                    should_exist = f.should_exist_for_services(active_services)
+                    try:
+                        cmd.lchan.select_file(f)
+                        sw = None
+                        exists = True
+                    except SwMatchError as e:
+                        sw = str(e)
+                        exists = False
+                    if exists != should_exist:
+                        num_problems += 1
+                        if exists:
+                            cmd.perror("  ERROR: File %s is selectable but should not!" % f)
+                        else:
+                            cmd.perror("  ERROR: File %s is not selectable (%s) but should!" %  (f, sw))
+        finally:
+            # re-select the EF.UST
+            cmd.lchan.select_file(self)
+        return num_problems
+
+    def ust_update(self, cmd, activate=[], deactivate=[]):
+        service_data, _sw = cmd.lchan.read_binary()
+        service_data = h2b(service_data)
+
+        for service in activate:
+            nbyte, nbit = EF_UServiceTable._bit_byte_offset_for_service(service)
+            if nbyte > len(service_data):
+                missing = nbyte - service_data
+                service_data.extend(missing * "00")
+            service_data[nbyte] |= (1 << nbit)
+
+        for service in deactivate:
+            nbyte, nbit = EF_UServiceTable._bit_byte_offset_for_service(service)
+            if nbyte > len(service_data):
+                missing = nbyte - service_data
+                service_data.extend(missing * "00")
+            service_data[nbyte] &= ~(1 << nbit)
+
+        service_data = b2h(service_data)
+        cmd.lchan.update_binary(service_data)
+
+# TS 31.102 Section 4.4.2.1
+class EF_PBR(LinFixedEF):
+    # TODO: a80ac0034f3a02c5034f0904aa0acb034f3d07c2034f4a06
+    def __init__(self, fid='4F30', name='EF.PBR', desc='Phone Book Reference', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+        #self._tlv = FIXME
+
+# TS 31.102 Section 4.4.2.12.2
+class EF_PSC(TransparentEF):
+    _construct = Struct('synce_counter'/Int32ub)
+    def __init__(self, fid='4F22', name='EF.PSC', desc='Phone Book Synchronization Counter', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+        #self._tlv = FIXME
+
+# TS 31.102 Section 4.4.2.12.3
+class EF_CC(TransparentEF):
+    _construct = Struct('change_counter'/Int16ub)
+    def __init__(self, fid='4F23', name='EF.CC', desc='Change Counter', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+
+# TS 31.102 Section 4.4.2.12.4
+class EF_PUID(TransparentEF):
+    _construct = Struct('previous_uid'/Int16ub)
+    def __init__(self, fid='4F24', name='EF.PUID', desc='Previous Unique Identifer', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+
+# TS 31.102 Section 4.4.2
+class DF_PHONEBOOK(CardDF):
+    def __init__(self, fid='5F3A', name='DF.PHONEBOOK', desc='Phonebook', **kwargs):
+        super().__init__(fid=fid, name=name, desc=desc, **kwargs)
+        files = [
+            EF_PBR(),
+            EF_PSC(),
+            EF_CC(),
+            EF_PUID(),
+            # FIXME: Those 4Fxx entries with unspecified FID...
+            ]
+        self.add_files(files)
+
+
+
+# TS 31.102 Section 4.6.3.1
+class EF_MML(BerTlvEF):
+    def __init__(self, fid='4F47', name='EF.MML', desc='Multimedia Messages List', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+
+# TS 31.102 Section 4.6.3.2
+class EF_MMDF(BerTlvEF):
+    def __init__(self, fid='4F48', name='EF.MMDF', desc='Multimedia Messages Data File', **kwargs):
+        super().__init__(fid, name=name, desc=desc, **kwargs)
+
+class DF_MULTIMEDIA(CardDF):
+    def __init__(self, fid='5F3B', name='DF.MULTIMEDIA', desc='Multimedia', **kwargs):
+        super().__init__(fid=fid, name=name, desc=desc, **kwargs)
+        files = [
+            EF_MML(),
+            EF_MMDF(),
+            ]
+        self.add_files(files)
+
+
+# TS 31.102 Section 4.6.4.1
+EF_MST_map = {
+    1: 'MCPTT UE configuration data',
+    2: 'MCPTT User profile data',
+    3: 'MCS Group configuration data',
+    4: 'MCPTT Service configuration data',
+    5: 'MCS UE initial configuration data',
+    6: 'MCData UE configuration data',
+    7: 'MCData user profile data',
+    8: 'MCData service configuration data',
+    9: 'MCVideo UE configuration data',
+    10: 'MCVideo user profile data',
+    11: 'MCVideo service configuration data',
+    }
+
+# TS 31.102 Section 4.6.4.2
+class EF_MCS_CONFIG(BerTlvEF):
+    class McpttUeConfigurationData(BER_TLV_IE, tag=0x80):
+        pass
+    class McpttUserProfileData(BER_TLV_IE, tag=0x81):
+        pass
+    class McsGroupConfigurationData(BER_TLV_IE, tag=0x82):
+        pass
+    class McpttServiceConfigurationData(BER_TLV_IE, tag=0x83):
+        pass
+    class McsUeInitialConfigurationData(BER_TLV_IE, tag=0x84):
+        pass
+    class McdataUeConfigurationData(BER_TLV_IE, tag=0x85):
+        pass
+    class McdataUserProfileData(BER_TLV_IE, tag=0x86):
+        pass
+    class McdataServiceConfigurationData(BER_TLV_IE, tag=0x87):
+        pass
+    class McvideoUeConfigurationData(BER_TLV_IE, tag=0x88):
+        pass
+    class McvideoUserProfileData(BER_TLV_IE, tag=0x89):
+        pass
+    class McvideoServiceConfigurationData(BER_TLV_IE, tag=0x8a):
+        pass
+    class McsConfigDataCollection(TLV_IE_Collection, nested=[McpttUeConfigurationData,
+                                  McpttUserProfileData, McsGroupConfigurationData,
+                                  McpttServiceConfigurationData, McsUeInitialConfigurationData,
+                                  McdataUeConfigurationData, McdataUserProfileData,
+                                  McdataServiceConfigurationData, McvideoUeConfigurationData,
+                                  McvideoUserProfileData, McvideoServiceConfigurationData]):
+        pass
+    def __init__(self, fid='4F02', sfid=0x02, name='EF.MCS_CONFIG', desc='MCS configuration data', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_MCS_CONFIG.McsConfigDataCollection
+
+# TS 31.102 Section 4.6.4.1
+class EF_MST(EF_UServiceTable):
+    def __init__(self, fid='4F01', sfid=0x01, name='EF.MST', desc='MCS Service Table', size=(2,2),
+                 table=EF_MST_map, **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, size=size, table=table)
+
+class DF_MCS(CardDF):
+    def __init__(self, fid='5F3D', name='DF.MCS', desc='Mission Critical Services', **kwargs):
+        super().__init__(fid=fid, name=name, desc=desc, **kwargs)
+        files = [
+            EF_MST(),
+            EF_MCS_CONFIG(),
+            ]
+        self.add_files(files)
+
+
+# TS 31.102 Section 4.6.5.2
+EF_VST_map = {
+    1: 'MCPTT UE configuration data',
+    2: 'MCPTT User profile data',
+    3: 'MCS Group configuration data',
+    4: 'MCPTT Service configuration data',
+    5: 'MCS UE initial configuration data',
+    6: 'MCData UE configuration data',
+    7: 'MCData user profile data',
+    8: 'MCData service configuration data',
+    9: 'MCVideo UE configuration data',
+    10: 'MCVideo user profile data',
+    11: 'MCVideo service configuration data',
+    }
+
+# TS 31.102 Section 4.6.5.2
+class EF_VST(EF_UServiceTable):
+    def __init__(self, fid='4F01', sfid=0x01, name='EF.VST', desc='V2X Service Table', size=(2,2),
+                 table=EF_VST_map, **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, size=size, table=table)
+
+# TS 31.102 Section 4.6.5.3
+class EF_V2X_CONFIG(BerTlvEF):
+    class V2xConfigurationData(BER_TLV_IE, tag=0x80):
+        pass
+    class V2xConfigDataCollection(TLV_IE_Collection, nested=[V2xConfigurationData]):
+        pass
+    def __init__(self, fid='4F02', sfid=0x02, name='EF.V2X_CONFIG', desc='V2X configuration data', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_V2X_CONFIG.V2xConfigDataCollection
+
+# TS 31.102 Section 4.6.5
+class DF_V2X(CardDF):
+    def __init__(self, fid='5F3E', name='DF.V2X', desc='Vehicle to X', **kwargs):
+        super().__init__(fid=fid, name=name, desc=desc, **kwargs)
+        files = [
+            EF_VST(),
+            EF_V2X_CONFIG(),
+            ]
+        self.add_files(files)

pySim/ts_31_103.py

@@ -0,0 +1,329 @@
+# -*- coding: utf-8 -*-
+
+"""
+Various constants from 3GPP TS 31.103 V16.1.0
+"""
+
+#
+# Copyright (C) 2020 Supreeth Herle <herlesupreeth@gmail.com>
+# Copyright (C) 2021 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from construct import Struct, Switch, this, Bytes, GreedyString
+from pySim.filesystem import *
+from pySim.utils import *
+from pySim.tlv import *
+from pySim.ts_51_011 import EF_AD, EF_SMS, EF_SMSS, EF_SMSR, EF_SMSP
+from pySim.ts_31_102 import ADF_USIM, EF_FromPreferred
+from pySim.ts_31_102_telecom import EF_UServiceTable
+import pySim.ts_102_221
+from pySim.ts_102_221 import EF_ARR
+from pySim.construct import *
+
+# Mapping between ISIM Service Number and its description
+EF_IST_map = {
+    1: 'P-CSCF address',
+    2: 'Generic Bootstrapping Architecture (GBA)',
+    3: 'HTTP Digest',
+    4: 'GBA-based Local Key Establishment Mechanism',
+    5: 'Support of P-CSCF discovery for IMS Local Break Out',
+    6: 'Short Message Storage (SMS)',
+    7: 'Short Message Status Reports (SMSR)',
+    8: 'Support for SM-over-IP including data download via SMS-PP as defined in TS 31.111 [31]',
+    9: 'Communication Control for IMS by ISIM',
+    10: 'Support of UICC access to IMS',
+    11: 'URI support by UICC',
+    12: 'Media Type support',
+    13: 'IMS call disconnection cause',
+    14: 'URI support for MO SHORT MESSAGE CONTROL',
+    15: 'MCPTT',
+    16: 'URI support for SMS-PP DOWNLOAD as defined in 3GPP TS 31.111 [31]',
+    17: 'From Preferred',
+    18: 'IMS configuration data',
+    19: 'XCAP Configuration Data',
+    20: 'WebRTC URI',
+    21: 'MuD and MiD configuration data',
+}
+
+# TS 31.103 Section 4.2.2
+class EF_IMPI(TransparentEF):
+    _test_de_encode = [
+        ( '803137333830303630303030303031303140696d732e6d6e633030302e6d63633733382e336770706e6574776f726b2e6f7267',
+          { "nai": "738006000000101@ims.mnc000.mcc738.3gppnetwork.org" } ),
+    ]
+
+    class nai(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+
+    def __init__(self, fid='6f02', sfid=0x02, name='EF.IMPI', desc='IMS private user identity', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_IMPI.nai
+
+# TS 31.103 Section 4.2.3
+class EF_DOMAIN(TransparentEF):
+    _test_de_encode = [
+        ( '8021696d732e6d6e633030302e6d63633733382e336770706e6574776f726b2e6f7267',
+          { "domain": "ims.mnc000.mcc738.3gppnetwork.org" } ),
+    ]
+    class domain(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+
+    def __init__(self, fid='6f03', sfid=0x05, name='EF.DOMAIN', desc='Home Network Domain Name', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_DOMAIN.domain
+
+# TS 31.103 Section 4.2.4
+class EF_IMPU(LinFixedEF):
+    _test_de_encode = [
+        ( '80357369703a37333830303630303030303031303140696d732e6d6e633030302e6d63633733382e336770706e6574776f726b2e6f7267',
+          { "impu": "sip:738006000000101@ims.mnc000.mcc738.3gppnetwork.org" } ),
+    ]
+    class impu(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+
+    def __init__(self, fid='6f04', sfid=0x04, name='EF.IMPU', desc='IMS public user identity', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_IMPU.impu
+
+# TS 31.103 Section 4.2.7
+class EF_IST(EF_UServiceTable):
+    def __init__(self, **kwargs):
+        super().__init__('6f07', 0x07, 'EF.IST', 'ISIM Service Table', (1, None), EF_IST_map)
+        # add those commands to the general commands of a TransparentEF
+        self.shell_commands += [self.AddlShellCommands()]
+
+    @with_default_category('File-Specific Commands')
+    class AddlShellCommands(CommandSet):
+        def __init__(self):
+            super().__init__()
+
+        def do_ist_service_activate(self, arg):
+            """Activate a service within EF.IST"""
+            selected_file = self._cmd.lchan.selected_file
+            selected_file.ust_update(self._cmd, [int(arg)], [])
+
+        def do_ist_service_deactivate(self, arg):
+            """Deactivate a service within EF.IST"""
+            selected_file = self._cmd.lchan.selected_file
+            selected_file.ust_update(self._cmd, [], [int(arg)])
+
+        def do_ist_service_check(self, arg):
+            """Check consistency between services of this file and files present/activated.
+
+            Many services determine if one or multiple files shall be present/activated or if they shall be
+            absent/deactivated.  This performs a consistency check to ensure that no services are activated
+            for files that are not - and vice-versa, no files are activated for services that are not.  Error
+            messages are printed for every inconsistency found."""
+            selected_file = self._cmd.lchan.selected_file
+            num_problems = selected_file.ust_service_check(self._cmd)
+            self._cmd.poutput("===> %u service / file inconsistencies detected" % num_problems)
+
+
+# TS 31.103 Section 4.2.8
+class EF_PCSCF(LinFixedEF):
+    _test_de_encode = [
+        ( '802c0070637363662e696d732e6d6e633030302e6d63633733382e7075622e336770706e6574776f726b2e6f7267',
+          {'pcscf_address': { "address": "pcscf.ims.mnc000.mcc738.pub.3gppnetwork.org", "type_of_address": "FQDN" } } ),
+        ( '800501c0a80c22',
+          {'pcscf_address': { "address": "192.168.12.34", "type_of_address": "IPv4" } } ),
+        ( '801102fe800000000000000042d7fffe530335',
+          {'pcscf_address': { "address": "fe80::42:d7ff:fe53:335", "type_of_address": "IPv6" } } ),
+    ]
+    class PcscfAddress(BER_TLV_IE, tag=0x80):
+        _construct = Struct('type_of_address'/Enum(Byte, FQDN=0, IPv4=1, IPv6=2),
+                            'address'/Switch(this.type_of_address,
+                                             {'FQDN': Utf8Adapter(GreedyBytes),
+                                              'IPv4': Ipv4Adapter(GreedyBytes),
+                                              'IPv6': Ipv6Adapter(GreedyBytes)}))
+
+    def __init__(self, fid='6f09', sfid=None, name='EF.P-CSCF', desc='P-CSCF Address', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_PCSCF.PcscfAddress
+
+# TS 31.103 Section 4.2.9
+class EF_GBABP(TransparentEF):
+    def __init__(self, fid='6fd5', sfid=None, name='EF.GBABP', desc='GBA Bootstrapping', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._construct = Struct('rand'/LV,
+                                 'b_tid'/LV,
+                                 'key_lifetime'/LV)
+
+# TS 31.103 Section 4.2.10
+class EF_GBANL(LinFixedEF):
+    class NAF_ID(BER_TLV_IE, tag=0x80):
+        _construct = Struct('fqdn'/Utf8Adapter(Bytes(this._.total_len-5)),
+                            'ua_spi'/HexAdapter(Bytes(5)))
+    class B_TID(BER_TLV_IE, tag=0x81):
+        _construct = Utf8Adapter(GreedyBytes)
+    # pylint: disable=undefined-variable
+    class GbaNlCollection(TLV_IE_Collection, nested=[NAF_ID, B_TID]):
+        pass
+    def __init__(self, fid='6fd7', sfid=None, name='EF.GBANL', desc='GBA NAF List', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_GBANL.GbaNlCollection
+
+# TS 31.103 Section 4.2.11
+class EF_NAFKCA(LinFixedEF):
+    _test_de_encode = [
+        ( '80296273662e696d732e6d6e633030302e6d63633733382e7075622e336770706e6574776f726b2e6f7267',
+          { 'naf_key_centre_address': 'bsf.ims.mnc000.mcc738.pub.3gppnetwork.org' } ),
+        ( '8030656e65746e61667830312e696d732e6d6e633030302e6d63633733382e7075622e336770706e6574776f726b2e6f7267',
+          { 'naf_key_centre_address': 'enetnafx01.ims.mnc000.mcc738.pub.3gppnetwork.org' }),
+    ]
+    class NafKeyCentreAddress(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+    def __init__(self, fid='6fdd', sfid=None, name='EF.NAFKCA', desc='NAF Key Centre Address', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_NAFKCA.NafKeyCentreAddress
+
+# TS 31.103 Section 4.2.16
+class EF_UICCIARI(LinFixedEF):
+    class iari(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+
+    def __init__(self, fid='6fe7', sfid=None, name='EF.UICCIARI', desc='UICC IARI', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_UICCIARI.iari
+
+# TS 31.103 Section 4.2.18
+class EF_IMSConfigData(BerTlvEF):
+    class ImsConfigDataEncoding(BER_TLV_IE, tag=0x80):
+        _construct = HexAdapter(Bytes(1))
+    class ImsConfigData(BER_TLV_IE, tag=0x81):
+        _construct = GreedyString
+    # pylint: disable=undefined-variable
+    class ImsConfigDataCollection(TLV_IE_Collection, nested=[ImsConfigDataEncoding, ImsConfigData]):
+        pass
+    def __init__(self, fid='6ff8', sfid=None, name='EF.IMSConfigData', desc='IMS Configuration Data', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_IMSConfigData.ImsConfigDataCollection
+
+# TS 31.103 Section 4.2.19
+class EF_XCAPConfigData(BerTlvEF):
+    class Access(BER_TLV_IE, tag=0x81):
+        pass
+    class ApplicationName(BER_TLV_IE, tag=0x82):
+        pass
+    class ProviderID(BER_TLV_IE, tag=0x83):
+        pass
+    class URI(BER_TLV_IE, tag=0x84):
+        pass
+    class XcapAuthenticationUserName(BER_TLV_IE, tag=0x85):
+        pass
+    class XcapAuthenticationPassword(BER_TLV_IE, tag=0x86):
+        pass
+    class XcapAuthenticationType(BER_TLV_IE, tag=0x87):
+        pass
+    class AddressType(BER_TLV_IE, tag=0x88):
+        pass
+    class Address(BER_TLV_IE, tag=0x89):
+        pass
+    class PDPAuthenticationType(BER_TLV_IE, tag=0x8a):
+        pass
+    class PDPAuthenticationName(BER_TLV_IE, tag=0x8b):
+        pass
+    class PDPAuthenticationSecret(BER_TLV_IE, tag=0x8c):
+        pass
+
+    class AccessForXCAP(BER_TLV_IE, tag=0x81):
+        pass
+    class NumberOfXcapConnParPolicy(BER_TLV_IE, tag=0x82):
+        _construct = Int8ub
+    # pylint: disable=undefined-variable
+    class XcapConnParamsPolicyPart(BER_TLV_IE, tag=0xa1, nested=[Access, ApplicationName, ProviderID, URI,
+                                 XcapAuthenticationUserName, XcapAuthenticationPassword,
+                                 XcapAuthenticationType, AddressType, Address, PDPAuthenticationType,
+                                 PDPAuthenticationName, PDPAuthenticationSecret]):
+        pass
+    class XcapConnParamsPolicy(BER_TLV_IE, tag=0xa0, nested=[AccessForXCAP, NumberOfXcapConnParPolicy, XcapConnParamsPolicyPart]):
+        pass
+    class XcapConnParamsPolicyDO(BER_TLV_IE, tag=0x80, nested=[XcapConnParamsPolicy]):
+        pass
+    def __init__(self, fid='6ffc', sfid=None, name='EF.XCAPConfigData', desc='XCAP Configuration Data', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_XCAPConfigData.XcapConnParamsPolicy
+
+# TS 31.103 Section 4.2.20
+class EF_WebRTCURI(TransparentEF):
+    class uri(BER_TLV_IE, tag=0x80):
+        _construct = Utf8Adapter(GreedyBytes)
+
+    def __init__(self, fid='6ffa', sfid=None, name='EF.WebRTCURI', desc='WebRTC URI', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_WebRTCURI.uri
+
+# TS 31.103 Section 4.2.21
+class EF_MuDMiDConfigData(BerTlvEF):
+    class MudMidConfigDataEncoding(BER_TLV_IE, tag=0x80):
+        _construct = HexAdapter(Bytes(1))
+    class MudMidConfigData(BER_TLV_IE, tag=0x81):
+        _construct = GreedyString
+    # pylint: disable=undefined-variable
+    class MudMidConfigDataCollection(TLV_IE_Collection, nested=[MudMidConfigDataEncoding, MudMidConfigData]):
+        pass
+    def __init__(self, fid='6ffe', sfid=None, name='EF.MuDMiDConfigData',
+                 desc='MuD and MiD Configuration Data', **kwargs):
+        super().__init__(fid=fid, sfid=sfid, name=name, desc=desc, **kwargs)
+        self._tlv = EF_MuDMiDConfigData.MudMidConfigDataCollection
+
+
+class ADF_ISIM(CardADF):
+    def __init__(self, aid='a0000000871004', has_fs=True, name='ADF.ISIM', fid=None, sfid=None,
+                 desc='ISIM Application'):
+        super().__init__(aid=aid, has_fs=has_fs, fid=fid, sfid=sfid, name=name, desc=desc)
+
+        files = [
+            EF_IMPI(),
+            EF_DOMAIN(),
+            EF_IMPU(),
+            EF_AD(),
+            EF_ARR('6f06', 0x06),
+            EF_IST(),
+            EF_PCSCF(service=5),
+            EF_GBABP(service=2),
+            EF_GBANL(service=2),
+            EF_NAFKCA(service=2),
+            EF_SMS(service=(6,8)),
+            EF_SMSS(service=(6,8)),
+            EF_SMSR(service=(7,8)),
+            EF_SMSP(service=8),
+            EF_UICCIARI(service=10),
+            EF_FromPreferred(service=17),
+            EF_IMSConfigData(service=18),
+            EF_XCAPConfigData(service=19),
+            EF_WebRTCURI(service=20),
+            EF_MuDMiDConfigData(service=21),
+        ]
+        self.add_files(files)
+        # add those commands to the general commands of a TransparentEF
+        self.shell_commands += [ADF_USIM.AddlShellCommands()]
+
+    def decode_select_response(self, data_hex):
+        return pySim.ts_102_221.CardProfileUICC.decode_select_response(data_hex)
+
+
+# TS 31.103 Section 7.1
+sw_isim = {
+    'Security management': {
+        '9862': 'Authentication error, incorrect MAC',
+        '9864': 'Authentication error, security context not supported',
+    }
+}
+
+
+class CardApplicationISIM(CardApplication):
+    def __init__(self):
+        super().__init__('ISIM', adf=ADF_ISIM(), sw=sw_isim)

pySim/ts_31_104.py

@@ -0,0 +1,60 @@
+# -*- coding: utf-8 -*-
+
+"""
+Support for 3GPP TS 31.104 V17.0.0
+"""
+
+# Copyright (C) 2023 Harald Welte <laforge@osmocom.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+from pySim.filesystem import *
+from pySim.utils import *
+from pySim.tlv import *
+from pySim.ts_31_102 import ADF_USIM
+from pySim.ts_51_011 import EF_IMSI, EF_AD
+import pySim.ts_102_221
+from pySim.ts_102_221 import EF_ARR
+
+
+class ADF_HPSIM(CardADF):
+    def __init__(self, aid='a000000087100A', has_fs=True, name='ADF.HPSIM', fid=None, sfid=None,
+                 desc='HPSIM Application'):
+        super().__init__(aid=aid, has_fs=has_fs, fid=fid, sfid=sfid, name=name, desc=desc)
+
+        files = [
+            EF_ARR(fid='6f06', sfid=0x06),
+            EF_IMSI(fid='6f07', sfid=0x07),
+            EF_AD(fid='6fad', sfid=0x03),
+        ]
+        self.add_files(files)
+        # add those commands to the general commands of a TransparentEF
+        self.shell_commands += [ADF_USIM.AddlShellCommands()]
+
+    def decode_select_response(self, data_hex):
+        return pySim.ts_102_221.CardProfileUICC.decode_select_response(data_hex)
+
+
+# TS 31.104 Section 7.1
+sw_hpsim = {
+    'Security management': {
+        '9862': 'Authentication error, incorrect MAC',
+    }
+}
+
+
+class CardApplicationHPSIM(CardApplication):
+    def __init__(self):
+        super().__init__('HPSIM', adf=ADF_HPSIM(), sw=sw_hpsim)