public/pysim
1
0
Fork 1
mirror of https://gitea.osmocom.org/sim-card/pysim.git synced 2026-05-08 02:15:07 +03:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: public:fixeria/btsap
Branches Tags
public:master public:pmaier/rcp public:neels/suci public:neels/saip2 public:neels/wip public:neels/smsp_test public:pmaier/fixgr public:osmith/saip2 public:pmaier/putkey public:pmaier/otatest public:pmaier/otatestprep public:daniel/ota public:neels/saip public:pmaier/pgsql public:lynxis/esim-cccprofile public:osmith/wip public:ewild/ossl_verify public:ewild/ppk_tests public:laforge/ota public:laforge/wsrc public:laforge/wip public:laforge/scp public:laforge/smdp_asn1 public:fixeria/cmd2 public:chrysn/ota public:chrysn/for-29033 public:fixeria/run_gsm_alg public:fixeria/btsap public:sysmocom/factory public:users/daniel/csv-format public:fairwaves/sim public:zecke/tmp2 public:ccc public:zecke/hacks public:27c3
public:1.0
...
compare: public:pmaier/rcp
Branches Tags
public:pmaier/rcp public:master public:neels/suci public:neels/saip2 public:neels/wip public:neels/smsp_test public:pmaier/fixgr public:osmith/saip2 public:pmaier/putkey public:pmaier/otatest public:pmaier/otatestprep public:daniel/ota public:neels/saip public:pmaier/pgsql public:lynxis/esim-cccprofile public:osmith/wip public:ewild/ossl_verify public:ewild/ppk_tests public:laforge/ota public:laforge/wsrc public:laforge/wip public:laforge/scp public:laforge/smdp_asn1 public:fixeria/cmd2 public:chrysn/ota public:chrysn/for-29033 public:fixeria/run_gsm_alg public:fixeria/btsap public:sysmocom/factory public:users/daniel/csv-format public:fairwaves/sim public:zecke/tmp2 public:ccc public:zecke/hacks public:27c3
public:1.0

1111 Commits
fixeria/bt ... pmaier/rcp

Author SHA1 Message Date
Philipp Maier
d9eef6fce7 WIP: Remote Card Procedure Framework 
Problem: When UICC/eUICC cards are deployed into the field it is often
difficult to perform modifications to those cards. One important factor
that makes after-deployment modifications often difficult is that the
key material needed to perform the task must not be handed to the card
holder due to security requirements.

The presented Remote Card Procedure Framework solves this problem. It
provides a so called Remote Card Procedure Client (RCPC), which is a
lightwight software client which can be run by the card holder on the
remote machine.

With the RCPC, the card holder can access a so called Remote Card
Procedure Server (RCPC), to which so called Remote Card Procedure
Modules (RCPM) can subscribe and publish their functionality. With
the RCPC, the card holder can browse the functionality offered by
those connected modules and eventually the card holder may execute
a certain procedure by passing a command to the RCPS.

When a procedure is carried out, the RCPS automatically retrieves the
required key material from a database or CSV file and passes those
keys on to the selected RCPM. The RCPM can then use the key material
to establish a secure channel to carry out the procedure. The procedure
is then protected by a secure channel and the key material is never
disclosed towards the card holder on the remote end.

The framework is desinged in such a way that existing pySim APIs and
functions can be used from the RCPM API user code. Also only minimal
boilerplate code is required. The implementation also ships with a
comprehensive example.

Related: SYS#6959
 2026-05-07 17:08:35 +02:00
Philipp Maier
35ca46d8fe pySim/global_platform: make functionality available outside of cmd2 
The nested class AddlShellCommands holds methods that encapsulate
the actual functionality from the related do_ method (e.g.
do_store_data calls self.store_data). This is already a good level
of separation but it does not allow us to call those methods from
programs that are not based on cmd2. Let's turn those methods into
functions so that non cmd2 applications have easy access to the
functionality of pySim.global_platform.

Let's also add a pySimLogger, so that we do not have to call
self._cmd.poutput

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
2184168c54 pySim/scp: migrate to pySimLogger 
The module scp.py predates the existence of the pySimLogger and still
uses an individually created logger. Let's migrate to pySimLogger to
avoid unexpected effects and to be uniform with the other modules.

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
7596045245 pySim/filesystem: use pySimLogger instead of print 
let's replace the stray print statements with proper logger calls.

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
9e43ada58f pySim/app: use pySimLogger instead of print 
let's replace the stray print statements with proper logger calls.

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
1b43839da3 pySimLogger: fix default log format string 
In format string we prepend when we log in verbose mode. We use %(module)s
as format string quaifier. This qualifier is replaced with the name of the
module from where the logger was called. This is mostly equal to the logger
name (__name__) we pass when we create the logger.

However, this is not the behavior we actually want. We want to log the
logger name that we passed when the logger was created. For this, we must
use %(name)s as qualifier.

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
191113baee card_key_provider: add a static method to parse --column-keys args 
The contents of the --column-keys arguments are currently parsed
in init_card_key_provider. Let's add a static method in
CardKeyFieldCryptor to simplify re-usage of the CardKeyFieldCryptor

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
3a713ca2a6 card_key_provider: pass CardKeyFieldCryptor to constructor 
We currently create the CardKeyFieldCryptor object inside the constructor
of the concrete CardKeyProvider classes. There is currently no problem
with that, but when we create the CardKeyFieldCryptor object first and
then pass it as parameter to the constructor, we gain more flexibility
in case we want to support other CardKeyFieldCryptor variants in the
future.

Related: SYS#6959
 2026-05-07 15:43:32 +02:00
Philipp Maier
4b7fef3277 card_key_provider: move boiler-plate code into helper functions 
in pySim-shell.py we add the commandline options for the card key
provider and do the setup accordingly. Let's put this boilerplate
code into helper functions instead, so that we can re-use it in
other pySim programs as well. Let's use pySim.transport as a
pattern.

Related: SYS#6959
 2026-05-07 12:20:45 +02:00
Philipp Maier
0fa3eecde9 cosmetic: remove excess linebreaks 
We should not use double-line breaks to separate things as it is very
difficult to maintain consistence with that scheme.
 2026-05-07 12:20:01 +02:00
Philipp Maier
593bfa0911 ts_51_011/EF.SMSP: fix handling of 'alpha_id' field 
The field 'alpha_id' is technically not an optional field, even though
the specification describes it as optional. Once the card manufacturer
decides that the field should be present, it must be always present and
vice versa.

(see code comment for a more detailed description)

Related: SYS#7765
Change-Id: I0ec99b2648b22c56f9145345e4cd8776f9217701
 2026-04-29 19:39:14 +00:00
Philipp Maier
8fa7727a14 pySim-prog/cards: fix programming of EF.SMSP 
The legacy code found in legacy/cards.py does not use the modern
construct based encoder (pySim-read uses it). The card classes either
use their own implementation of update_smsp or use the generic method
provided by the SimCard class. The latter one is true for FairwavesSIM
and WavemobileSim.

Unfortunately the implementation found in the SimCard is wrong. It
adds padding at the end of the file instead of the beginning. This
completely messes up the contents of EF.SMSP for the cards using this
method. To fix this, let's use the leftpad feature provided by
the update_record. This will ensure a correct alignment of the file
contents.

Related: SYS#7765
Change-Id: Ie112418f1f1461762d61365d3863181ca6be7245
 2026-04-29 19:39:14 +00:00
Philipp Maier
f1609424de pySim/transport: fix GET RESPONSE behaviour 
The current behavior we implement in the method __send_apdu_T0 is
incomplete. Some details discussed in ETSI TS 102 221,
section 7.3.1.1.4, clause 4 seem to be not fully implemented. We
may also end up sending a GET RESPONSE in other APDU cases than
case 4 (the only case that uses the GET RESPONSE command).

Related: OS#6970
Change-Id: I26f0566af0cdd61dcc97f5f502479dc76adc37cc
 2026-04-29 19:34:27 +00:00
Neels Hofmeyr
1167b65e2a comment in uicc.py on Security Domain Keys: add SCP81 
Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
cd4b01f67e personalization: fix SdKey.apply_val() implementation 
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
393de033d3 personalization: add get_typical_input_len() to ConfigurableParameter 
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.

Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
5f1c7d603c personalization: make AlgorithmID a new EnumParam 
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.

Implement the enum value part abstractly in new EnumParam.

Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn

Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
d7072e9263 personalization: indicate default ParamSource per ConfigurableParameter 
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.

This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.

For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.

BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:

  my_source = MyParam.default_source.from_str( MyParam.default_value )

Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
ac593bb14d personalization: implement reading back values from a PES 
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.

Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.

Implementation detail:

Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.

For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.

Rationales for reading back values:

This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).

Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
  I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.

Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Neels Hofmeyr
a95622a022 personalization: add param_source.py, add batch.py 
Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test
 2026-04-25 05:15:14 +07:00
Vadim Yanitskiy
03b58985a5 tests: pySim-smpp2sim_test.sh: fix copy-pasted comment 
Change-Id: I8167c6a3251bb6755810c96075010e920ceee8ac
Jenkins: skip-card-test
 2026-04-23 23:54:10 +07:00
Vadim Yanitskiy
cc71dbf899 contrib/jenkins.sh: add setup_venv() 
Reduece code duplication by factoring out virtualenv setup and
activation into a shell function.

Change-Id: Ibb193d12d5502c78104ef53badc6037f08e92df1
 2026-04-23 23:54:00 +07:00
Vadim Yanitskiy
aafc8d51c3 contrib/jenkins.sh: separate JOB_TYPE for card tests 
A separate job gives us a possibility to skip tests requiring physical
cards for specific commits that do not touch the core logic.  See the
related commits in osmo-ci.git.

Change-Id: If76d812ee43b7eb3b57fdc660c60bf31fbff5b16
Related: osmo-ci.git Ia48d1b468f65d7c2e6b4128eeac36d0f3d03c45e
Related: osmo-ci.git I986d88545f64e13cd571ba9ff56bc924822e39a0
 2026-04-23 23:52:55 +07:00
Philipp Maier
c50f4b4a02 requirements: ensure safe version of PyYAML >= 5.4 (CVE-2020-1747) 
PyYAML versions 5.1–5.3.1 are vulnerable to CVE-2020-1747, which allows
arbitrary code execution through yaml.FullLoader. While PyYAML 5.4+
patches this, the dependency specification (pyyaml >= 5.1) doesn't
guarantee a safe version. Let's increase the requirement to version
5.4 to ensure a safe version of is used.

This patch is based on suggestions from:
"YanTong C <chyeyantong03@gmail.com>"

Change-Id: I901c76c59e9c1bab030eab81038e04a475b32510
 2026-04-16 11:01:19 +00:00
YanTong C
816b31eb07 pySim-prog: fix Insecure PRNG for SIM Authentication Keys (CWE-338) 
Root Cause:
pySim-prog.py uses Python's random module (Mersenne Twister MT19937) to
generate Ki and OPC — the root authentication keys for SIM cards. MT19937
is a deterministic PRNG that is not cryptographically secure. Its internal
state (624 × 32-bit words, 19,937 bits) can be fully recovered after
observing 624 consecutive outputs.

Impact:
1. SIM Card Cloning: An attacker who determines the PRNG state can predict
all Ki/OPC values generated before and after. With these keys, SIM cards
can be cloned.
2. Network Authentication Bypass: Ki/OPC are used in the Milenage algorithm
for 3G/4G/5G authentication. Predictable keys mean an attacker can
authenticate as any subscriber whose SIM was provisioned with the weak RNG.
3. Batch Compromise: In bulk provisioning scenarios (pySim-prog's primary
use case), hundreds or thousands of SIMs may be programmed sequentially.
Compromising one batch means recovering the PRNG state to predict all keys.

Fix:
Replace random.randrange() with os.urandom()

Change-Id: Id3e00d3ec5386f17c1525cacfc7d3f5bba43381f
 2026-04-15 13:50:11 +02:00
Vadim Yanitskiy
f2567de387 pySim/ts_51_011.py: add multi-record test vector for EF_PL 
Change-Id: I9f7a444b18056b1683cbd52a25af950125531746
 2026-04-07 22:47:22 +07:00
Vadim Yanitskiy
6b5fa38f14 tests: fix TransRecEF _test_de_encode to operate at file level 
Previously, _test_de_encode vectors for TransRecEF subclasses were tested
via decode_record_hex()/encode_record_hex(), i.e. one record at a time,
with the decoded value being a scalar.

Switch test_de_encode_record() in TransRecEF_Test to use decode_hex() /
encode_hex() instead, so that vectors represent whole-file content
(decoded value is a list of records) -- consistent with how LinFixedEF
handles _test_de_encode.  Update all existing vectors accordingly.

Change-Id: I4a9610f9ee39833cd0c90f64f89f5fbdd6f0846d
 2026-04-07 22:47:22 +07:00
Vadim Yanitskiy
45220e00d5 filesystem: JsonEditor: offer interactive retry on error 
When json.loads() fails (e.g. the user made a syntax mistake), prompt
the user with "Re-open file for editing? [y]es/[n]o:" and loop back to
the editor if they answer 'y' or 'yes'.  If the user declines, return
the original unmodified value so no write is attempted; the temp file
is still cleaned up by __exit__() in that case.

Change-Id: I9161b7becea0d8dfd3f5f740fbb253da2f061a1d
Related: OS#6899
 2026-04-07 22:47:22 +07:00
Vadim Yanitskiy
5828c92c66 filesystem: JsonEditor: use NamedTemporaryFile 
A plain NamedTemporaryFile is sufficient here: we only need a single
file, not a directory to hold it.  Using NamedTemporaryFile is simpler
(no subdirectory to manage) and gives us a .json suffix for free,
which editors use for syntax highlighting.

Change-Id: If3b0bd0fcc90732407dbd03b9cc883f7abeb948e
 2026-04-07 22:47:22 +07:00
Vadim Yanitskiy
5e2fd148f8 filesystem: edit_{binary,record}_decoded: add encode/decode examples 
When invoking `edit_binary_decoded` or `edit_record_decoded`, the
temp file opened in the editor now contains the EF's encode/decode
test vectors as //-comment lines below the JSON content, similar to
how 'git commit' appends comments to the commit message template.
The comment block is stripped before JSON parsing on save,
so it has no effect on the written data.

The feature is implemented via a new module-level JsonEditor context
manager class that encapsulates the full edit cycle:

* write JSON + examples to a TemporaryDirectory
* invoke the editor
* read back, strip //-comments, parse and return the result

Change-Id: I5a046a9c7ba7e08a98cf643d5a26bc669539b38f
Related: OS#6900
 2026-04-07 15:32:13 +00:00
Vadim Yanitskiy
fc932a2ee9 docs: auto-generate Card Filesystem Reference 
Add a Sphinx extension (docs/pysim_fs_sphinx.py) that hooks into the
builder-inited event and generates docs/filesystem.rst before Sphinx
reads any source files.

The generated page contains a hierarchical listing of all implemented
EFs and DFs, organised by application/specification (UICC/TS 102 221,
ADF.USIM/TS 31.102, ADF.ISIM/TS 31.103, SIM/TS 51.011).  For each file,
the class docstring and any _test_de_encode / _test_decode vectors
are included as an encoding/decoding example table.

docs/filesystem.rst is fully generated at build time and is therefore
added to .gitignore.

Add tests/unittests/test_fs_coverage.py that walks all pySim.* modules
and verifies that every CardProfile, CardApplication, and standalone
CardDF subclass with EF/DF children is either listed in the SECTIONS
(and will appear in the docs) or explicitly EXCLUDED.

Change-Id: I06ddeefc6c11e04d7c24e116f3f39c8a6635856f
Related: OS#6316
 2026-04-07 15:32:13 +00:00
Philipp Maier
d5aa963caa pysim/pcsc: do not use getProtocol for protocol selection 
The documentation of the getProtocol provided by pyscard says:

"Return bit mask for the protocol of connection, or None if no
protocol set. The return value is a bit mask of
CardConnection.T0_protocol, CardConnection.T1_protocol,
CardConnection.RAW_protocol, CardConnection.T15_protocol"

This suggests that the purpose of getProtocol is not to determine
which protocols are supported. Its purpose is to determine which
protocol is currently selected (either through auto selection or
through the explicit selection made by the API user). This means
we are using getProtocol wrong.

So far this was no problem, since the auto-selected protocol
should be a supported protocol anyway. However, the automatic
protocol selection may not always return a correct result (see
bug report from THD-siegfried [1]).

Let's not trust the automatic protocol selection. Instead let's
parse the ATR and make the decision based on the TD1/TD2 bytes).

[1] https://osmocom.org/issues/6952

Related: OS#6952
Change-Id: Ib119948aa68c430e42ac84daec8b9bd542db7963
 2026-04-02 12:53:30 +02:00
Vadim Yanitskiy
19245d0d8b docs/conf.py: add autodoc_mock_imports for klein and twisted 
The eSIM SM-DP+ server modules (`pySim.esim.es2p`, `pySim.esim.es9p`,
`pySim.esim.http_json_api`) unconditionally import optional server-side
dependencies at module level:

  pySim.esim.es2p          -- from klein import Klein
  pySim.esim.http_json_api -- from twisted.web.server import Request

Both imports fail during a docs build if the packages are absent or
broken, causing three "autodoc: failed to import" warnings and three
missing chapters in the generated manual.

Even when klein and twisted are installed, twisted 23.10.0 (the
version pulled in transitively by smpp.twisted3's `Twisted~=23.10.0`
constraint) is incompatible with Python 3.13+ because twisted.web.http
unconditionally executes `import cgi`, a module that was removed from
the standard library in Python 3.13.

Fix: add `autodoc_mock_imports = ['klein', 'twisted']` to conf.py.
Sphinx inserts mock entries into sys.modules before each autodoc import
attempt, so the modules can be imported and documented without requiring
the real packages to be importable at build time.

Change-Id: I71650466f02a6a6d150650deed167c05d2cb6e64
 2026-03-31 19:17:13 +07:00
Vadim Yanitskiy
a786590906 docs/conf.py: silence autosectionlabel duplicate-label warnings 
sphinxarg.ext generates generic sub-headings ("Named arguments",
"Positional arguments", "Sub-commands", "General options", ...) for
every argparse command and tool.  These repeat across many files and
trigger large numbers of autosectionlabel duplicate-label warnings.

Two-pronged fix:

* `autosectionlabel_maxdepth = 3` eliminates the depth-4+ warnings
  (sub-headings inside each individual command block).
* `suppress_warnings` per file silences the residual depth-3 collisions
  ("serial reader", "decode_hex", "sub-commands", ...) that still
  appear across tool documentation files.

Cross-references into these generic argparse-generated sections are not
a supported use-case, so suppressing the warnings is appropriate.

Change-Id: I9cdf2a4f6cbd435b16b90ab668205600ffd7c3b0
 2026-03-31 19:17:13 +07:00
Philipp Maier
ca8fada7b6 pySim-prog/pySim-read: add pySimLogger and verbose cmdline argument 
pySim-prog and pySim-read do not integrate the pySimLogger yet. As we
may add more debug output that should not be visible on normal use, we
should ensure that the pySimLogger is correctly set up.

Change-Id: Ia2fa535fd9ce4ffa301c3f5d6f98c1f7a4716c74
 2026-03-31 11:29:46 +00:00
Philipp Maier
c995bb1ec2 pySim-shell/cosmetic: remove semicolon 
Change-Id: I629bacd432491211b939fcd2bed554b44ef441bc
 2026-03-31 11:29:46 +00:00
Philipp Maier
ee06ab987f PySimLogger: add parameter to set initial log-level/verbosity 
When we initialize a new PySimLogger, we always call the setup method
first and then use the set_verbose and set_level method to configure
the initial log level and the initial log verbosity. However, we
initialize the PySimLogger in all our programs the same way and we
end up with the same boilerplate code every time. Let's add a keyword
parameter to the setup method where we can pass our opts.verbose (bool)
parameter so that the setup method can do the work for the main program.

In case the caller wants a different default configuration he still can
call set_verbose and set_level methods as needed.

Change-Id: I4b8ef1e203186878910c9614a1d900d5759236a8
 2026-03-31 11:29:46 +00:00
Philipp Maier
a1d3b8f5e8 pySim-read: remove import random 
In pySim-read we do not have to compute any random numbers, so
we may remove random from the imports

Change-Id: Iae4ee6aafb339cc682345299b92b4ecd0bbca14e
 2026-03-31 11:46:28 +02:00
Vadim Yanitskiy
f7b86e1920 docs/put_key-tutorial: fix three typos 
"verifiation"        -> "verification"
  "this is identifies" -> "this identifies" (extra word)
  "and and"            -> "and" (doubled word)

Change-Id: I1ae6b01638cc2c3dd8355ba801f85cc179ca8bd3
 2026-03-31 08:26:07 +00:00
Vadim Yanitskiy
2cfb0972df docs/legacy: fix typo "EF,IMSI" -> "EF.IMSI" 
Change-Id: I1f246ec008a57b2373ed3f5531ab4166101f4dd0
 2026-03-31 08:26:07 +00:00
Vadim Yanitskiy
4215a3bfd3 docs/saip-tool: fix typo "insertaion" -> "insertion" 
Change-Id: Ie9c9235ec964a15fab19d6ca5a83b2b1ddf07e7b
 2026-03-31 08:26:07 +00:00
Vadim Yanitskiy
b42d417bbe docs/shell: fix copy-paste typos in editor command descriptions 
Two adjacent commands (`edit_record_decoded`, `edit_binary_decoded`)
had identical copy-pasted error messages with three typos each:

  "modificatiosn" -> "modifications"
  "us the"        -> "use the"
  "comamdn"       -> "command"

Change-Id: Ie23baba4634e2cc40f81439fb11b102778aed1f6
 2026-03-31 08:26:07 +00:00
Vadim Yanitskiy
74ac191ae6 docs/card-key-provider: fix heading levels and typo 
The "ADM PIN" and "SCP02 / SCP03" sub-sections of "Field naming" used
the '~' heading character, which Sphinx resolved to level 4 - skipping
level 3 and throwing build ERRORs.  As a result, both sub-sections
had no heading at all.  Change both to '^' (level 3) to match the
other sub-sections in this file.

While at it, fix a typo: "consisting if" -> "consisting of".

Change-Id: Ia56efc7fadcc0fd62e87e63850b929d2f80851ba
 2026-03-31 08:26:07 +00:00
Philipp Maier
add4b991b7 transport: change APDU format paradigm 
Unfortunately we have mixed up the concept of TPDUs and APDUs in
earlier versions of pySim-shell. This lead to problems with
detecteding the APDU case properly (see also ISO/IEC 7816-3) and
also prevented us from adding support for T=1.

This problem has been fixed long time ago and all APDUs sent from
the pySim-shell code should be well formed and valid according to
ISO/IEC 7816-3.

To ensure that we continue to format APDUs correctly as APDUs (and
not TPDUs) we have added a mechanism to the LinkBase class that
would either raise an exception or print a warning if someone
mistakenly tries to send an APDU that is really a TPDU. Whether a
warning is printed or an exception is raised is controlled via the
apdu_strict member in the LinkBase class, which is false (print
warning only) by default.

The reason why we have implemneted the mechanism this way was
because we wanted to ensure that existing APDU scripts (pySim-shell
apdu command) keep working, even though when those scripts uses
APDUs which are formally invalid.

Sending a TPDU instead of an APDU via a T=0 link will still work
in almost all cases. This is also the reason why this problem
slipped through unnoticed for long time. However, there may still
be subtile problems araising from this practice. The root of the
problem is that it is impossible to distinguish between APDU case
3 and 4 when a TPDU instead of an APDU is sent. However in order
to handle a case 4 APDU correctly we must be able to distinguish
the APDU case correctly to handle the case correctly.
ETSI TS 102 221, section 7.3.1.1.4, clause 4 is very clear about
the fact that not (only) the status word (e.g. 61xx) but the
APDU case is what matters.

To complete the logic in LinkBaseTpdu and to maintain compatibility
(older APDU scripts), we must still be able to switch between the
'apdu_strict' mode and the non-strict mode. However, since
pySim-shell, pySim-prog and pySim-read internally use proper APDUs,
we may enable the 'apdu_strict' mode by default.

At the same time we will limit the effect of pySim-shell's
apdu_strict setable to the apdu command only. By doing so, the
bahviour of the apdu command is not altered. Users will still
have to enable the 'strict' mode explicitly. At the same time
all the internal functionality of pySim-shell will always use
the 'strict' mode.

Related: OS#6970
Change-Id: I9a531a825def318b28bf58291d811cf119003fab
 2026-03-30 10:26:13 +00:00
Philipp Maier
8c81e2cdf9 docs/put_key: add tutorial that explains how to manage global platform keys 
With the increased interest in using GlobalPlatform features of
UICC and eUICCs (OTA-SMS, applets, etc.), also comes an increased
interest in how the related GlobalPlatform keys can be managed
(key rotation, adding/removing keysets from/to a Security Domain).

Unfortunately, many aspects of this topic are not immediately
obvious for the average user. Let's add a tutorial that contains
some practical examples to shine some light on the topic.

Related: SYS#7881
Change-Id: I163dfedca3df572cb8442e9a4a280e6c5b00327e
 2026-03-25 18:07:32 +00:00
Vadim Yanitskiy
d9d62ee729 global_platform: refactor gen_install_parameters() 
gen_install_parameters() had contradictory logic: the outer guard
required all three arguments to be non-None/non-empty (making them
mutually inclusive), while the inner checks then treated each one
as optional.

Make each parameter independently optional (defaulting to None) and
remove the all-or-nothing check.  Simplify the function body to a
straightforward single-pass construction of system_specific_params.

Change-Id: I8756fb38016cdf0527fe2e21edb44381d1dc557f
 2026-03-25 18:05:30 +00:00
Vadim Yanitskiy
c7e68e1281 global_platform: install_cap_parser: argument groups cannot be nested 
pySim-shell currently does not work on systems with Python 3.14+:

  File ".../pysim/pySim/global_platform/__init__.py", line 868, in AddlShellCommands
    install_cap_parser_inst_prm_g_grp = install_cap_parser_inst_prm_g.add_argument_group()
  File "/usr/lib/python3.14/argparse.py", line 1794, in add_argument_group
    raise ValueError('argument groups cannot be nested')
  ValueError('argument groups cannot be nested')

The problem is that install_cap_parser creates a nested group inside
of mutually exclusive group.  argparse never supported group nesting
properly, so it has been deprecated since Python 3.11, and eventually
got removed in Python 3.14.

Remove group nesting, adjust the usage string, and implement the
mutual exclusiveness enforcement manually in do_install_cap().

Change-Id: Idddf72d5a745345e134b23f2f01e0257d0667579
 2026-03-25 18:05:30 +00:00
Philipp Maier
969f9c0e4b pySim/EF.SMSP: fix encoding of TP-Destination Address 
The TP-Destination Address in EF.SMSP uses the same encoding as the
TS-Service Centre Address field. However, even though the encoding
of both fields looks almost identical, it actually isn't.

The TS-Service Centre Address field encodes the length field as
octets required for the call_number + one octet for ton_npi.
(see also: 3GPP TS 24.011, section 8.2.5.2)

The TP-Destination Address uses the number of digits of the
call_number directly in the length field.
(see also: 3GPP TS 23.040, section 9.1.2.5)

Related: SYS#7765
Change-Id: I55c123c9e244e5a6e71a0348f5d476ef03e618e8
 2026-03-25 15:34:56 +01:00
Philipp Maier
2ef9abf23e pySim/EF.SMSP: add an additional de_encode test for EF_SMSP 
Let's add another testvector where we test what happens when we populate
none of the fields except for the tp_sc_addr.

Related: SYS#7765
Change-Id: I12b600ab17d1acfdddaffe6006095acf1a4228c9
 2026-03-24 15:55:24 +00:00
Philipp Maier
473f31066c pySim/pcsc/cosmetic: reformat comment 
Change-Id: Ic04bdfbc6727cc670679c377c1afd1de53504b8f
 2026-03-23 19:03:19 +00:00
Philipp Maier
b59363b49e pySim/EF.SMSP: remove superflous line break 
Change-Id: Ie02e02546e708e2c339810812188bd8e8af2a720
 2026-03-23 16:44:38 +01:00
Vadim Yanitskiy
115b517c6a esim/saip: raise an exception properly 
Change-Id: Ia3749c02120fdc16e556214d0461cbeca032447b
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
99aef1fecf cdma_ruim: fix inaccurate comment for EF_AD 
Change-Id: I71ea27fd30e44685ff35f49843072ca392995973
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
caddd1c7a0 ts_31_102: EF_5G_PROSE_UIR: fix copy-pasted inner class name 
Change-Id: I460e5ad70f35026d0d794271a4aef17323c14dfb
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
11a7a7e3b1 ts_31_102: fix description for EF_5GS3GPPLOCI 
Change-Id: I9cf3adfce65090fedb3f0fd33c9b3d15a2c5fb8c
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
5138208ee6 ts_51_011: EF.EXT[6-7]: fix typo in desc 
Change-Id: I93df1c9fd8a4d588ed7ed19ec2dc1d304412fc3d
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
5b2fabde62 utils: DataObjectCollection.encode(): fix TypeError 
`members_by_name` is a plain dictionary.  Calling it with `()` raises:

  TypeError: 'dict' object is not callable

Change-Id: I7e0c09aa7303f1506fe3a025fdc3779919dd0e6c
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
24127e985a utils: dec_plmn(): remove redundant call 
Change-Id: Ic95c3992ed57eb8fee952ec2dc7f092dd7689579
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
09ae327f8b ota: OtaAlgo{Crypt,Auth}: fix algo_auth vs algo_crypt 
* OtaAlgoCrypt.from_keyset() searches by `otak.algo_crypt`
  but the error message prints `otak.algo_auth`.  Should be
  `otak.algo_crypt` instead.

* OtaAlgoAuth.__init__() checks `algo_auth` but the error message
  prints `algo_crypt`.  Should be `otak.algo_auth` instead.

Change-Id: Ia636fffaeadc68e3f6d5b65d477e753834c95895
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
d32bce19f6 sms: fix flags_construct in SMS_DELIVER 
* field `tp_rp` appears at bit positions 7 and 5
** bit 7 should be `tp_rp` (Reply Path)
** bit 5 should be `tp_sri` (Status Report Indication)
* field `tp_lp` is completely missing
** should be at bit position 3

Change-Id: I0274849f0fa07281b5e050af429ffda7d249f9e8
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
83bfdc0d3b ara_m: fix undefined variable used in a format-string 
Change-Id: I310a5d461bae2b5e4d8e07097000b079c23aa0f6
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
14ec52a06c ara_m: fix exceptions not being raised properly 
Exceptions are meant to be thrown/raised, not returned.

Change-Id: Id799c264447e22887edcd2dc7eb991cf0af1bbfc
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
209d13e233 global_platform: fix docstring for Scp03SessionKeys._get_icv() 
Change-Id: I8983bc27f581295544360ba8b4ae1d28b3ea850f
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
3b50e64c8b global_platform: fix s/GET/STORE/ DATA in docs 
Both `do_store_data` and `store_data` have identical docstrings that
incorrectly describe the command as GET DATA.  Should be "STORE DATA".
Take a chance to fix missing space between `v2.3` and `Section`.

Change-Id: I33fc80ab8ca50fadc38217b0005eec6169c8e34e
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
b76cc80ea1 global_platform: fix store_data() returning last chunk only 
The loop builds up `response` across multiple STORE DATA blocks,
but the function returns only `data` - the response from the
*last* block.  It should return the accumulated response instead.

Change-Id: I3e15c8004d1e366e8c3896e559656622f48bb1a2
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
3b87ba3cba global_platform: fix typo in ApplicationTemplate 
The keyword argument should be `nested=`.  As written `ApplicationAID`
is silently ignored - `ApplicationTemplate` will not descend into its
nested TLVs.

Change-Id: If45dbb0c9b09fe53560d109957ce339267a9f2b0
 2026-03-20 14:32:59 -07:00
Vadim Yanitskiy
ea1d5af383 global_platform: fix typo in SupportedTlsCipherSuitesForScp81 
The attribute name is misspelled.  The BER-TLV infrastructure looks
for `_construct`; this typo means `SupportedTlsCipherSuitesForScp81`
will never decode its content.

Change-Id: I0f637951b0eeb7eca2a8b543baa737f216a935ed
 2026-03-20 14:32:59 -07:00
Philipp Maier
0634f77308 esim/http_json_api: allow URL rewriting 
The URL used when HTTP requests are performed is defined statically
with the url_prefix passed to the constructor of JsonHttpApiClient
together with the path property in JsonHttpApiFunction.

For applications that require dynamic URLs there is no way to rewrite
the URL. Let's add a mechanism that allows API users to apply custom
URL reqriting rules by adding a rewrite_url method to
JsonHttpApiFunction. API users may then overload this method with a
custom implementation as needed.

Related: SYS#7918
Change-Id: Id2713a867079cc140517fe312189e5e2162608a5
 2026-03-17 11:17:12 +01:00
Vadim Yanitskiy
a5a5865c7c cdma_ruim: fix copy-pasted desc for EF.AD 
Change-Id: I2338f35c21978dd6b8916c0abd57b94f5e087655
 2026-03-10 17:43:10 +00:00
Harald Welte
3752aeb94e pySim.esim.saip.File: Support pinStatusTemplateDO + lcsi 
The tuples defining a DF or ADF in an eSIM template must contain a
pinStatusTemplateDO.  When parsing tuples into a File() instance, we
must save it, and re-create it at the time we re-encode that file.

Same applies to the lcsi (life cycle state indicator), which may
optionally exist for any file.

Change-Id: I073aa4374f2cd664d07fa0224bf0d4c809cdf4aa
Closes: OS#6955
 2026-03-10 16:34:48 +00:00
Philipp Maier
914abe3309 docs/smpp-ota-tool: Add documentation/tutorial 
We already have documentation that explains how to run pySim-smpp2sim.
With smpp-ota-tool we now have a counterpart for pySim-smpp2sim, so
let's add documentation for this tool as well.

Related: SYS#7881
Change-Id: If0d18a263f5a6dc035b90f5c5c6a942d46bbba49
 2026-03-10 09:23:03 +00:00
Philipp Maier
84754b6ebb contrib/smpp-ota-tool: define commandline arguments in global scope 
The commandline arguments are currently defined under __main__ in a
private scope. From there they are not reachable to the sphinx
argparse module. We have to define the arguments globally at the
top. (like in the other applications)

Related: SYS#7881
Change-Id: I2d9782e3f5b1cac78c22d206fdcac4118c7d5e7c
 2026-03-10 09:23:03 +00:00
Philipp Maier
c47005d408 contrib/smpp-ota-tool: use '-' instead of '_' in command line args 
Some commandline arguments have an underscore in their name. Let's
replace those with dashes.

Change-Id: Icbe9d753d59263997e9ca34d46ed0daca36ca16c
Related: SYS#6868
 2026-03-10 09:23:03 +00:00
Philipp Maier
2dfaac6e4f contrib/smpp-ota-tool: fix description string (copy+paste error) 
Change-Id: I559844bfa1ac372370ef9d148f2f8a6bf4ab4ef5
Related: SYS#6868
 2026-03-10 09:23:03 +00:00
Philipp Maier
a615ba5138 tests/pySim-smpp2sim_test: add testcases for AES128 and AES256 
Extend the existing test script so that it can handle multiple
testcases. Also add support for switching eUICC profiles.
Finally, add a testcases to test OTA-SMS (RFM) with AES128 and
AES256 encryption.

Change-Id: I1f10504f3a29a8c74a17991632d932819fecfa5a
Related: OS#6868
 2026-03-10 09:23:03 +00:00
Philipp Maier
8ee10ab1a5 tests/pySim-smpp2sim_test/card_sanitizer: update card backup with new test keyset 
In our test setup we run the card_sanitizer.py script regualary to ensure that
we have consistent start conditions when running our tests. In case a testcase
crashes for some reason and leaves messed up files on a test card. The
card_sanitizer.py script will ensure that any problem like that is cleaned up
over night.

For the testcases we are about to add in the patch following this one, we need
to provision a new test keyset to one of our test cards. This has been already
done manually. However since the card_sanitizer still has the old keys in its
backup we will have to update that as well.

Change-Id: I5aa8a413b19b3e43a79d03e904daab50b4b1e767
Related: OS#6868
 2026-03-10 09:23:03 +00:00
Philipp Maier
f10af30aed global_platform/scp: fix dek_encrypt/dek_decrypt for SCP02 
The methods dek_encrypt/dek_decrypt use the wrong algorithm and the
wrong key material. The algorithm should be 3DES rather then single
DES and the key must be the DEK session key instead of the static
DEK key from which the DEK session key is derived.

Related: SYS#7902
Change-Id: I3d0cc7378680b346fa39152c8b7074446d2c869d
 2026-03-06 15:51:19 +01:00
Neels Hofmeyr
d8f3c78135 SmspTpScAddr: set example_input 
Change-Id: Ie2c367788215d746807be24051478f0032a19448
 2026-03-04 00:24:02 +01:00
Neels Hofmeyr
6b9b46a5a4 MilenageRotationConstants: set example_input to 3GPP default 
Change-Id: I36a9434b2f96d26d710f489d5afce1f0ef05bba1
 2026-03-04 00:23:41 +01:00
Philipp Maier
b6b4501e37 contrib/smpp-ota-tool: fix boolean commandline parameters 
Boolean parameters should be false by default and use store_true when
set.

Change-Id: I0652b48d2ea5efbaaf5bc147aa8cef7ab8b0861d
Related: OS#6868
 2026-02-24 09:52:48 +01:00
Philipp Maier
54658fa3a9 contrib/smpp-ota-tool: add missing usage helpstrings 
Change-Id: Ic1521ba11b405f311a30fdb3585ad518375669ae
Related: OS#6868
 2026-02-24 09:52:48 +01:00
Philipp Maier
eb04bb1082 contrib/smpp-ota-tool: warn about mixed up KIC/KIC indexes 
Cards usually have multiple sets of KIC, KID (and KIK). The keys
are selected through an index. However, mixing keys from different
sets is concidered as a security violation and cards should reject
such configurations.

Let's print a warning to make users aware that something is off.

Change-Id: Ieb4e14145baba1c2cb4a237b612b04694940f402
Related: OS#6868
 2026-02-24 09:52:48 +01:00
Philipp Maier
453fde5a3a contrib/smpp-ota-tool: use correct kid index 
(normally KID index and KIC index should be the same since mixing keys
is a concidered as a security violation. However, in this tool we
want to allow users to specify different indexes for KIC and KIC so that
they can make tests to make sure their cards correctly reject mixed up
key indexes)

Change-Id: I8847ccc39e4779971187e7877b8902fca7f8bfc1
Related: OS#6868
 2026-02-17 15:24:25 +01:00
Philipp Maier
57237b650e contrib/smpp-ota-tool/cosmetic: use lazy formatting for logging 
Change-Id: I2540472a50b7a49b5a67d088cbdd4a2228eef8f4
Related: OS#6868
 2026-02-17 15:24:25 +01:00
Philipp Maier
1f94791240 contrib/smpp-ota-tool/cosmetic: fix sourcecode formatting 
Change-Id: Icbce41ffac097d2ef8714bc8963536ba54a77db2
Related: OS#6868
 2026-02-17 15:24:25 +01:00
Philipp Maier
1a28575327 pySim-shell_test/euicc: ensure test-profile is enabled 
When testing commands like get_profile_info, enable_profile,
disable_profile or the commands to manage notifications, we
should ensure that the correct profile is enabled before
executing the actual testcase.

Change-Id: Ie57b0305876bc5001ab3a9c3a3b5711408161b74
 2026-02-17 09:22:42 +00:00
Neels Hofmeyr
e7016b5b57 compile_asn1_subdir: filter compiled files by .asn suffix 
When I open the .asn file in vim, pySim should not attempt to read the
vim .swp file as asn.1.

	  File "/home/moi/osmo-dev/src/pysim/pySim/esim/saip/__init__.py", line 45, in <module>
	    asn1 = compile_asn1_subdir('saip')
	[...]
	  File "<frozen codecs>", line 325, in decode
	UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 21: invalid start byte

Related: OS#6937
Change-Id: I37df3fc081e51e2ed2198876c63f6e68ecc8fcd8
 2026-02-10 16:14:14 +00:00
Philipp Maier
e80f3160a9 pySim/euicc: fix encoding/decoding of Iccid 
The class Iccid uses a BcdAdapter to encoded/decode the ICCID. This
works fine for ICCIDs that have an even (20) number of digits. In case
the digit count is odd (19), the ICCID the last digit requires padding.

Let's switch to PaddedBcdAdapter for encoding/decoding, to ensure that
odd-length ICCIDs are padded automatically.

Change-Id: I527a44ba454656a0d682ceb590eec6d9d0ac883a
Related: OS#6868
 2026-02-10 13:26:45 +00:00
Neels Hofmeyr
917ad7f9f5 gitignore: fix vim swp file pattern 
Change-Id: I5a8351dc09f6ca7c8e9032ff8352e5cf1a4833a3
 2026-02-10 13:10:17 +00:00
Philipp Maier
8b2a49aa8e esim/http_json_api: add alternative API interface (follow up) 
This is a follow up patch to change:
I2a5d4b59b12e08d5eae7a1215814d3a69c8921f6

- do not ignore length of kwargs
- fix role parameter (roles other than 'legacy_client' can be used now)
- use startswith instead of match

Related: SYS#7866
Change-Id: Ifae13e82d671ff09bddf771f063a388d2ab283eb
 2026-02-10 13:42:44 +01:00
Harald Welte
7ee7173a2f pySim.esim.saip.personalization: Fix docstring errors + warnings 
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:27: ERROR: Unexpected indentation. [docutils]
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:29: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:34: ERROR: Unexpected indentation. [docutils]
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:35: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:52: ERROR: Unexpected indentation. [docutils]
pysim/pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.ConfigurableParameter:53: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]

Change-Id: I3918308856c3a1a5e6e90561c3e2a6b88040670d
 2026-02-09 12:50:47 +00:00
Harald Welte
0f99598b34 pySim.esim.saip.personalization: Fix docstring error 
pySim/esim/saip/personalization.py:docstring of pySim.esim.saip.personalization.MilenageXoringConstants:4: ERROR: Unexpected indentation. [docutils]

Change-Id: If6ae360b7f74c095fa9075ae9aa988440496e6de
 2026-02-09 12:50:47 +00:00
Harald Welte
d7901ef08d pysim.utils.decomposeATR: Fix docutils warning 
pySim/utils.py:docstring of pySim.utils.decomposeATR:9: WARNING: Block quote ends without a blank line; unexpected unindent. [docutils]

Change-Id: Ifda4ba15014ba97634fd5bd5c9b19d9110f4670e
 2026-02-09 12:50:47 +00:00
Harald Welte
edfac26824 pySim.esim.saip: Fix docstring warnings: 
this fixes the following two warnings:

pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNode.walk:1: WARNING: Inline strong start-string without end-string. [docutils]
pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNodeDF.walk:1: WARNING: Inline strong start-string without end-string. [docutils]

Change-Id: Id7debf9296923b735f76623808cee68967a1ece7
 2026-02-09 12:50:47 +00:00
Harald Welte
07a3978748 es2p.py: also allow 18 digit ICCID 
While at it, also use tuples (const) instead of lists (var).

Tweaked-by: nhofmeyr@sysmocom.de (docstring, tuples)
Change-Id: Iaa6e710132e3f4c6cecc5ff786922f6c0fcfb54e
 2026-02-09 12:46:03 +00:00
Vadim Yanitskiy
a297cdba73 ModemATCommandLink: fix SyntaxWarning: invalid escape sequence '\+' 
Change-Id: If8de5299a4dc5a8525ef6657213db95d30e3c83b
Fixes: OS#6948
 2026-02-09 12:44:41 +00:00
Philipp Maier
f9d7c82b4d esim/http_json_api: add alternative API interface 
unfortunately the API changes introduced in change

I277aa90fddb5171c4bf6c3436259aa371d30d092

broke the API interface of http_json_api.py. This was taken into
account and necessary to introduce add the server functionality next
to the already existing client functionality. The changes to the API
were minimal and all code locations that use http_json_api.py
were re-aligned.

Unfortunately it was not clear at this point in time that there are
out-of-tree projects that could be affected by API changes in
http_json_api.py

To mitigate the problem this patch introduces an alternative API
interface to the JsonHttpApiFunction base class. This alternative
API interface works like the old API interface when the class is
instantiated in the original way. To make use of the revised client
the API use has to pass an additional keyword argument that defines
the role.

Related: SYS#7866
Change-Id: I2a5d4b59b12e08d5eae7a1215814d3a69c8921f6
 2026-02-09 12:42:28 +00:00
Alexander Couzens
c6fa2b4007 saip-tool: rename parser_tree correctly 
parser_info is already defined and this seems to be a copy/paste
accident.

Change-Id: Icc30dbf02a266211fa4d3aee8e7cec14185e716c
 2026-02-09 12:34:35 +00:00
Philipp Maier
39d744010a pySim-shell_test/euicc: fix testcase method name 
We have two test_enable_disable_profile method, the second one should
be called test_set_nickname.

Change-Id: I5ff79218fdafc8c42c8b58cc00be3e56e09d808b
 2026-02-09 10:10:08 +01:00
Philipp Maier
15691233e1 tests/pySim-smpp2sim_test: add integration test 
At the moment pySim.ota codebase is not covered by any of the
integration tests (we have only normal unittests so far). To
increase the test coverage, let's add an integration test that
sends exchanges an RFM OTA-SMS with a real-world card.

However, there is no tool avaliable that can be used as an SMPP
client for pySim-smpp2sim yet. Let's use smpp_ota_apdu2.py on
laforge/ota to develop a tool that we can use to exchange SMS-TPDUs
that contain remote APDU scripts (RFM/RAM).

Finally let's use the tool we have created as a basis to create
an integration test that exchanges an SMS-TPDU with the RFM
application of a sysmoISIM-SJA5 card. The testcase shall pass
when we get the expected response from the card.

Related: OS#6868
Change-Id: If25e38be004cc1c7aeeb130431831377e78fe28d
 2026-02-04 14:05:07 +00:00
Philipp Maier
0a1c5a27d7 esim/http_json_api: add missing apidoc 
Change-Id: Ibf9cf06197c9e3203c7a3ea5d77004f0ca41cd3f
 2026-02-04 12:45:58 +01:00
Harald Welte
e0a9e73267 http_json_api: Only require Content-Type if response body is non-empty 
If there is an empty body returned, such as in the case of the response
to an es9p notification, then it is of course also legal to not set the
content-type header.

This patch fixes an exception when talking to certain SM-DP+ with
es9p_client.py:

DEBUG:pySim.esim.http_json_api:HTTP RSP-STS: [204] hdr: {'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
DEBUG:pySim.esim.http_json_api:HTTP RSP: b''
{'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
<Response [204]>
Traceback (most recent call last):
  File "gprojects/git/pysim/es9p/../contrib/es9p_client.py", line 315, in <module>
    c.do_notification()
    ~~~~~~~~~~~~~~~~~^^
  File "projects/git/pysim/es9p/../contrib/es9p_client.py", line 159, in do_notification
    res = self.peer.call_handleNotification(data)
  File "projects/git/pysim/contrib/pySim/esim/es9p.py", line 174, in call_handleNotification
    return self.handleNotification.call(data)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "projects/git/pysim/contrib/pySim/esim/http_json_api.py", line 335, in call
    if not response.headers.get('Content-Type').startswith(req_headers['Content-Type']):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'startswith'

Change-Id: I99e8f167b7bb869c5ff6d908ba673dac87fef71a
 2026-01-31 11:58:32 +01:00
Harald Welte
22c3797a89 es9p_client: MAke install notification code execute at all 
The caller specified 'install' but the do_notification() function
compared with 'download' :(

Change-Id: I2d441cfbc1457688eb163301d3d91a1f1fdc7a8c
 2026-01-31 10:48:28 +00:00
Harald Welte
4e35e2c357 es9p_client: Fix type conversion in installation result notification 
The asn.1 encoder expects bytes-like objects, we cannot simply pass
hex-strings to it without conversion

Change-Id: I83ad047e043dc6b3462b188ce6dd0b2cc0e52e87
 2026-01-31 10:48:28 +00:00
Philipp Maier
e62f160775 contrib/csv-to-pgsql: add missing copyright header 
Change-Id: Iad8b2c1abb6a80764d05c823fbd03a9eae0ec0ab
 2026-01-31 01:32:27 +00:00
Alexander Couzens
1f2db11d31 pySim/card_key_provider: fix typo in keys 
Change-Id: Ie76f351ae221da2a0aab65c311fafe8ae6d63663
 2026-01-31 01:32:12 +00:00
Vladimir Serbinenko
ae91245582 Print SMSC in pySim-read.py 
Change-Id: I17067b68086316d51fd71ba77049874605594e3f
 2026-01-31 01:30:32 +00:00
Harald Welte
429b12c8b5 pySim-trace: pySim.apdu_source.stdin_hex 
This introduces an "APDU source" for pySim-trace which enables the
decoding of APDUs that are copy+pasted from elsewhere, for example
APDU logs in text form created by proprietary tools, or to decode
personalization scripts or the like.

Change-Id: I5aacf13b7c27cea9efd42f01dacca61068c3aa33
 2026-01-31 01:22:48 +00:00
Neels Hofmeyr
ccc1a047ab personalization: set example input values 
For all ConfigurableParameter subclasses, provide an example_input.

This may be useful for downstream projects' user interaction, to suggest
a value or prefill an input field, as appropriate.

Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
db17529136 personalization: set some typical parameter names 
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".

(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)

Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
1c082da0ee personalization: refactor SmspTpScAddr 
Refactor SmspTpScAddr to the new ConfigurableParameter implementation
style.

Change-Id: I2600369e195e9f5aed7f4e6ff99ae273ed3ab3bf
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
1e98856105 personalization: refactor SdKey 
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.

But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().

For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.

Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
ae656c66a3 personalization: refactor AlgorithmID, K, Opc 
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.

K and Opc use a common abstract BinaryParam.

Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240

Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
d5b570b01d personalization: refactor Pin, Adm 
Refactor Pin1, Pin2, Adm1 and Adm2 to the new ConfigurableParameter
implementation style.

Change-Id: I54aef10b6d4309398d4b779a3740a7d706d68603
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
21641816ea personalization: refactor Puk 
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.

DecimalHexParam will also be used for Pin and Adm soon.

Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
 2026-01-30 19:34:13 +00:00
Neels Hofmeyr
742baeab56 personalization: refactor ConfigurableParameter, Iccid, Imsi 
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods

The new features are optional, and will be heavily used by batch
personalization patches coming soon.

Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.

So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.

Details:

1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)

2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.

Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
 2026-01-30 19:34:13 +00:00
Philipp Maier
a4895702d7 transport/init: use PySimLogger to print messages 
The module still uses print to output information. Let's replace
those print calls with the more modern PySimLogger method calls.

Change-Id: I2e2ec2b84f3b84dbd8a029ae9bb64b7a96ddbde3
 2026-01-28 12:19:54 +01:00
Philipp Maier
2b42877389 pySimLogger: user __name__ of the module when creating a new logger 
At the moment we use random identifiers as names when we create a
new logger for pySimLogger. Let's switch to consistently use the
module name here. For the top level modules let's use the program
name so that it will show up in the log instead of __init__.

Change-Id: I49a9beb98845f66247edd42ed548980c97a7151a
 2026-01-28 12:19:54 +01:00
Harald Welte
167d6aca36 pySim.esim.saip: Don't try to generate file contents for MF/DF/ADF 
only EFs have data content

Change-Id: I02a54a3b2f73a0e9118db87f8b514d1dbf53971f
 2026-01-26 21:19:17 +01:00
Harald Welte
d8c45dc07e pySim.esim.saip: Implement optimized file content encoding 
Make sure we make use of the fill pattern when encoding file contents:
Only encode the differences to the fill pattern of the file, in order
to reduce the profile download size.

Change-Id: I61e4a5e04beba5c9092979fc546292d5ef3d7aad
 2026-01-26 21:19:05 +01:00
Philipp Maier
0a36ba257c pySim/runtime: use log.warning instead of log.warn 
The python logger method warn is deprecated since pyton 3.3, let's us
the warning method as suggested.

Change-Id: I3a4c0ca43768198ac6011ebe79050f91c04862e5
 2026-01-26 15:21:37 +00:00
Philipp Maier
1f36c9c28a contrib: add utility to receive ES2+handleDownloadProgressInfo calls 
We already have a tool to work with the ES2+ API provided by an SMDP+
(es2p_client.py) With this tool we can only make API calls towards
an SMDP+. However, SGP.22 also defines a "reverse direction" ES2+
interface through wich the SMDP+ may make API calls towards the MNO.

At the moment the only possible MNO originated API call is
ES2+handleDownloadProgressInfo. Let's add a simple tool that runs a
HTTP server to receive and log the ES2+handleDownloadProgressInfo
requests.

Related: SYS#7825
Change-Id: I95af30cebae31f7dc682617b1866f4a2dc9b760c
 2026-01-22 19:38:01 +01:00
Philipp Maier
e00c0becca esim/http_json_api: extend JSON API with server functionality 
At the moment http_json_api only supports the client role. Let's also add
support for the server role.

This patch refactors the existing client code. This in particular means
that the following preperations have to be made:

- To use the existing JsonHttpApiFunction definitions in the client and
  server the scheme has to be symetric. It already is for the most part,
  but it treads the header field differently. So let's just treat the
  header field like any other mandatory field and add it input_params.
  (this does not affect the es9p.py code since in ES9+ the requests have
   no header messages, see also SGP.22, section 6.5.1.1)

- The JsonHttpApiFunction class currently also has the code to perform
  the client requests. Let's seperate that code in a JsonHttpApiClient
  class to which we pass an JsonHttpApiFunction object.

- The code that does the encoding and decoding in the client role has
  lots of conditions the treat the header differently. Let's do the
  decisions about the header in the JsonHttpApiClient. The encoder
  and decoder function should do the generic encoding and decoding
  only. (however, some generic header specific conditions will remain).

The code for the server role logically mirrors the code for the client
role. We add a JsonHttpApiServer class that can be used to create
API endpoints. The API user has to pass in a call_handler through which
the application logic is defined. Above that we add an Es2pApiServer
class in es2p. In this class we implement the logic that runs the
HTTP server and receives the requests. The Es2pApiServer supports all
ES2+ functions defined by GSMA SGP.22. The user may use the provided
Es2pApiServerHandler base class to define the application logic for each
ES2+ function.

Related: SYS#7825
Change-Id: I277aa90fddb5171c4bf6c3436259aa371d30d092
 2026-01-22 19:38:01 +01:00
Philipp Maier
148d0a6f90 esim/http_json_api: add missing check 
The line actual_sec = func_ex_status.get('statusCodeData', None) suggests
that 'statusCodeData' may be None under normal circumstances. So let's guard
sec.update(actual_sec) so that we won't run into an exception in case
'statusCodeData' is not in func_ex_status.

Related: SYS#7825
Change-Id: I8a1a3cd5e029dba4a3aec1a64702e19b0d694ae2
 2026-01-22 18:51:16 +01:00
Harald Welte
51da6263b7 Fix esim.saip.ProfileElementSequence.remove_naas_of_type 
This method did not work at all at the moment, likely due to API churn
over time.  This change makes the following exception go away:

Traceback (most recent call last):
  File "projects/git/pysim/contrib/saip-tool.py", line 473, in <module>
    do_remove_naa(pes, opts)
    ~~~~~~~~~~~~~^^^^^^^^^^^
  File "projects/git/pysim/contrib/saip-tool.py", line 203, in do_remove_naa
    pes.remove_naas_of_type(naa)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "projects/git/pysim/contrib/pySim/esim/saip/__init__.py", line 1748, in remove_naas_of_type
    if template in hdr.decoded['eUICC-Mandatory-GFSTEList']:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "projects/git/pysim/contrib/pySim/esim/saip/oid.py", line 48, in __eq__
    return (self.intlist == other.intlist)
                            ^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'intlist'

A subsequent patch should introduce unit tests to avoid such breakage in
the future.

Change-Id: I88d862d751198c3d1648ab7f11d6e6a8fdbc41c9
 2026-01-20 09:50:03 +01:00
Harald Welte
4f1d7d7ac6 saip.validation: Verify unused mandatory services in header 
This adds a new check method to the pySim.esim.saip.validation.CheckBasicStructure
class, which ensures that no unused authentication algorithm related mandatory
services are indicated in the ProfileHeader.

So if a profile e.g. states in the header it requires
usim-test-algorithm, but then the actual akaParameter instances do not
actually use that algorithm, it would raise an exception.

Change-Id: Id0e1988ae1936a321d04bc7c3c3a33262c767d30
Related: SYS#7826
 2026-01-20 09:50:03 +01:00
Alexander Couzens
8557ec86be saip: ProfileElementSD: call _post_decode() when instantiating with decoded argument 
Otherwise self.keys is not generated from the given data and encoding will fail.

Change-Id: I3020f581a908fecc01d5d255ab5991ce1652e3ec
 2026-01-17 21:52:38 +00:00
Alexander Couzens
2e7944cc98 saip: calculate the number of records for LF and CY 
Some templates (e.g. for 5GS) define files which aren't completely defined.
5GS OPL5G: doesn't have a file size defined in the template,
but a record size.

Change-Id: I5ec1757d6852eb24d3662ec1c3fc88365e90a616
 2026-01-14 00:21:33 +00:00
Alexander Couzens
1347d5ffa2 saip: rework file sizes for "half-defined" template files 
Define the file size early if possible.
Some templates (e.g. for 5GS) define files which aren't completely defined.
Fixes the parsing for 5GS SUCI_Calc_Info which doesn't have a file size defined.

The saip-tool will other crash when reading a 5G enabled profile:
```
Traceback (most recent call last):
  File "./contrib/saip-tool.py", line 458, in <module>
    pes = ProfileElementSequence.from_der(f.read())
  File "pySim/esim/saip/__init__.py", line 1679, in from_der
    inst.parse_der(der)
    ~~~~~~~~~~~~~~^^^^^
  File "pySim/esim/saip/__init__.py", line 1552, in parse_der
    self.pe_list.append(ProfileElement.from_der(first_tlv, pe_sequence=self))
                        ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pySim/esim/saip/__init__.py", line 557, in from_der
    inst._post_decode()
    ~~~~~~~~~~~~~~~~~^^
  File "pySim/esim/saip/__init__.py", line 668, in _post_decode
    self.pe2files()
    ~~~~~~~~~~~~~^^
  File "pySim/esim/saip/__init__.py", line 655, in pe2files
    file = File(k, v, template.files_by_pename.get(k, None))
  File "pySim/esim/saip/__init__.py", line 133, in __init__
    self.from_tuples(l)
    ~~~~~~~~~~~~~~~~^^^
  File "pySim/esim/saip/__init__.py", line 358, in from_tuples
    self._body = self.file_content_from_tuples(l)
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "pySim/esim/saip/__init__.py", line 393, in file_content_from_tuples
    stream.write(self.template.expand_default_value_pattern(self.file_size))
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "pySim/esim/saip/templates.py", line 123, in expand_default_value_pattern
    raise ValueError("%s does not have a default length" % self)
ValueError: FileTemplate(EF.SUCI_Calc_Info) does not have a default length
```

Change-Id: I7c4a0914aef1049a416e6b091f23daab39a1dd9c
 2026-01-14 00:21:33 +00:00
Philipp Maier
fddab8639f card_key_provider: add PostgreSQL support 
The Card Key Provider currently only has support for CSV files
as input. Unfortunately using CSV files does not scale very well
when the card inventory is very large and continously updated.
In this case a centralized storage in the form of a database
is the more suitable approach.

This patch adds PostgreSQL support next to the existing CSV
file support. It also adds an importer tool to import existing
CSV files into the database.

Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3
Related: SYS#7725
 2026-01-12 10:57:27 +01:00
Philipp Maier
eb7c5d85d0 runtime/cosmetic: add line break 
The other source files have a line break between the character encoding
qualifier line and the python comment. Let's add a line break here
as well to maintain consistency.

Change-Id: Ied6b77eede748f1ddf6fde17c9b434fa4dd1114a
 2026-01-06 15:03:53 +01:00
Philipp Maier
eda6182edd transport/init/cosmetic: move copryight header to the top 
The copyright header seems to be misplaced, let's move it to the top.

Change-Id: I8358cca3bc9adb5a186a8b38a3bd90d7aec60d5c
 2026-01-06 15:00:49 +01:00
Alexander Couzens
725ffffda1 RFC: saip: templates: fix naming of EF.SUPI_NAI 
Fixes parsing of a 2.3 UICC profile.
This might be the wrong end as the spec says this is
NSI, but somehow it's working

Change-Id: I3cde1093156db274458d76e2c1c2e304d55a8466
 2026-01-07 12:28:25 +00:00
Alexander Couzens
777d005350 saip: templates: IsimOptional: add missing pe_name=ef-pcscf 
The file EF.P-CSCF is named ef-pcscf in the asn1 (tested with version 2.3)

Change-Id: I0cfba8f4e97fd6e2d8e21edf0439692b58a78ded
 2026-01-07 12:28:25 +00:00
Neels Hofmeyr
6e9625213a fix typo in doc TuakNumberOfKeccak 
Change-Id: Ie6f2260d5632dea7409cffd3afa7c8d0b1986a7c
 2026-01-07 00:00:21 +01:00
Philipp Maier
4c8a9478c2 cosmetic: fix company name in copyright header. 
The correct abbreviated version of the company name is
"sysmocom - s.f.m.c. GmbH", i.e. lowercase and with dash.

Change-Id: Id768d2f4b78162ff83320a800e4e66f1bd324d6d
 2026-01-06 21:41:23 +00:00
Philipp Maier
dfe4d9c8ac contrib: add a tool to parse the SIMA response from an eUICC 
When an eUICC performs a profile installation it returns a (concatenated)
series of ASN.1 encoded strings as "simaResponse". In case the profile
installation fails for some reason the simaResponse contains diagnostic
information to diagnose why the profile installation failed.

Unfortunately there are currently no practical tools available to decode
and display the information in the simaResponse. Let's add a tool for that.

Related SYS#7617

Change-Id: Ida4c3c5446653b283a3869c0c387f328ae51e55e
 2026-01-06 21:41:23 +00:00
Philipp Maier
8e048820d4 pySim-shell: renovate version command 
In case pySim-shell is used directly from the git repository (not
installed via a package manager), the version command fails with an
exception because pkg_resources.get_distribution('pySim') fails.

Let's renovate the version command and migrate from pkg_resources to
importlib.resources. There are many users and developers out there who
retrieve pySim-shell directly from the git repository and not via pip3.
To accommodate for that, let's check if pySim-shell.py is located in a
git repository and if so, let's display the HEAD commit hash instead.

Since the version of the currently installed pyosmocom version also
plays a critical role, let's display the pyosmocom version as well.

Related: OS#6830
Change-Id: I2b9038f88cfcaa07894a2f09c7f5ad8a5474083d
 2026-01-06 21:11:16 +00:00
Philipp Maier
c2ace3d8cf unittests/test_utils: add unittests for enc_imsi and dec_imsi 
So far we seem to have no unittests for enc_imsi and dec_imsi.

Change-Id: Iae55485c5ec7763aa4aaa25fd1910b854adaab60
 2026-01-06 21:10:40 +00:00
Harald Welte
097d565310 esim.saip: Better docstring about FsNode class 
Change-Id: Id9d196e8d9b1d1b892ec50100b170d72d2c3910b
 2026-01-06 21:10:29 +00:00
Harald Welte
a8ae89a041 pySim.esim.saip.ProfileElementSequence: Update type annotations 
The type annotations didn't reflect reality in two cases.

Change-Id: Ib99c00a38bf009c63180b4a593d6cc796ff282d3
 2026-01-06 21:10:19 +00:00
Philipp Maier
d764659a30 pySim-shell: do not show user home path in help text 
At the moment, the help text for the --csv option shows the path to
the users home. This is due to the default value, which is dynamically
generated. Let's use a static string with "~/" and resolve the full
path later when we need it.

Related: SYS#7725
Change-Id: Ied8b1e553de8f5370369c4485a2360906c874ed2
 2026-01-05 14:54:26 +00:00
Philipp Maier
3ca25219bc pySim-shell/cosmetic: remove unnecessary brackets 
Change-Id: I6929aa4fa189414217c05a4ef5180d4ed44eb3a4
 2026-01-05 14:42:49 +00:00
Harald Welte
1da34c1a4f Fix more odd-length digit sequences via PaddedBcdAdapter 
There are more files where trailing digits are indicated using 'f' and
should be stripped during decode, including EF.MSISDN and EF.VGCS

This is not just a presentation issue, but actually rendered wrong data
before, see the modified test output where our "read_record_uicc.ok"
file contained "bcd_len: 7" but then only 6 BCD digits due to this bug.

Change-Id: I4571482da924a3d645caa297108279d182448d21
 2025-12-23 20:57:03 +01:00
Harald Welte
381519556c ts_31_102.EF_ECC: Use PaddedBcdAdapter to skip trailing 'f' 
The emergency numbers from the example are 911 / 913, and not 911f / 311f

Change-Id: Ibfe1e23431aa803b936dd8529e0542e93d9df0b9
 2025-12-23 20:57:03 +01:00
Harald Welte
0fe432fec9 pySim.esim.saip.personalization: Support for EF.SMSP personalization 
It's a not-too-uncommon requirement to modify the SMSC address stored in
EF.SMSP.  This adds a ConfigurableParameter for this purpose.

Change-Id: I6b0776c2e753e0a6d158a8cf65cb030977782ec2
 2025-12-23 20:57:03 +01:00
Harald Welte
c6fd1d314a esim.saip.FsProfileElement: Add file2pe() for single file conversion 
We've had files2pe() for re-encoding all of the files, but let's add
a specific one for re-encoding only one of the files (such as commonly
needed during personalization)

Change-Id: I7b7f61aae6b7df6946dadf2f78fddf92995603ec
 2025-12-23 20:57:01 +01:00
Harald Welte
88aff4c577 pySim.ts_51_011.EF_SMSP: Properly handle odd-length ScAddr / TpAddr 
As the input phone number ("address") might be of an odd length of
digits, let's use PaddedBcdAdapter to fix two problems:

1) strip any potential trailing f in decoding
2) fix truncation of last digit during encoding

Change-Id: I1e9865e172bc29b8a31c281106d903934e81c686
Depends: pyosmocom Ib5afb5ab5c2bc9b519dc92818fc6974f7eecba16 (0.0.12
 2025-12-23 16:21:22 +00:00
Harald Welte
5fe76bb680 pySim/ts_51_011: Properly re-compute ScAddr length 
EF.SMSP contains up to two addresses: Both are stored in a fixed-length
field of 12 octets.  However, the actually used size depends on the
number of digits in the respective number.  Let's compute that length
field properly

Change-Id: Idef54a3545d1a5367a1efa2f0a6f7f0c1f860105
 2025-12-23 16:21:22 +00:00
Harald Welte
c058c6a34d ts_51_011: Improve testing of EF_SMSP 
* add another set of test data (from a real-world SIM card)
* switch from test_decode to test_de_encode as our encoder now works due
  to previous commits.

Change-Id: I8d16e195641bb59b2c26072008f88434692c0cab
 2025-12-23 16:21:22 +00:00
Philipp Maier
3d42106ad9 pysim/log: also accept ANSI strings to specify the log message colors 
the PySimLogger class currently only accepts cmd2 color enum values.
This is what we need for pySim-shell.py. However, in case we want to
use the PySimLogger in stand-alone programs that do not use cmd2, this
is a bit bulky. Let's add some flexibility to PySimLogger, so that we
can specify the colors as raw ANSI strings as well.

Change-Id: I93543e19649064043ae8323f82ecd8c423d1d921
Related: SYS#7725
 2025-12-19 16:12:31 +01:00
Harald Welte
9a23eab163 unittests/test_files: Pass to-be-encoded length to encoder functions 
Some of the encoders can only generate valid output if they are told
the expected output size.  This is due to variable-length fields that
depend on the size of the total record (or file).  Let's always pass
the expected length to the encoder methods.

Change-Id: I88f957e49b0c88a121a266d3582e79822fa0e214
 2025-12-18 20:38:59 +01:00
Harald Welte
82b57403c7 unittest/test_files.TransparentEF_Test: Actually test encoder 
In the test_encode_file() method, we should actually test the encoder,
and not the decoder.  I suppose this was a copy+paste mistake at some
point?  In the LinearFixedEF_Test.test_encoder_record we were already
testing the encoder. Just TransparentEF_Test got it wrong...

Change-Id: Id23305a78ab9acd2e006f2b26b72408795844d23
 2025-12-18 20:38:59 +01:00
Harald Welte
a62fb2b987 ts_51_011/EF.SMSP: Fix parsing of parameter_indicators 
There's a 3-bit RFU field that (unlike everything else in USIM/UICC)
considers '1' to be the default.  Let's make sure we get that right
during encode.

Change-Id: Ibe24a07f5f73d875d2077fa55471dbfc4e90da23
 2025-12-18 20:38:59 +01:00
Harald Welte
111f9da4f5 pyshark_gsmtap: Adjust display filter for some wireshark versions 
On my debian unstable system with wireshark 4.6.2-3, the pyshark_gsmtap
APDU source misses to report any ATRs, as those are not part of what's
reported with the 'gsm_sim' display filter.  This is due to
wireshark.git commit bcd82e2370d18e20983b378d494964d89c191cef first part
of the 4.6.0 release, which splits the ATR dissection into a separate
sub-dissector.

We cannot use the seemingly logical 'gsmtap.type == 4' instead, as old
wireshark simply bypasses any output for the gsmtap header if the SIM
sub-dissector is used.

Hence, 'gsm_sim || iso7816.atr' is something compatible with older and
newer wireshark versions.

Change-Id: I53c1c8ed58a82c37cd4be4af3890af21da839e86
 2025-12-18 20:35:49 +01:00
Harald Welte
ddbf91fc4a pySim.esim.saip.personalization: Support Milenage customization 
Milenage offers the capability for operators to modify the r1-r5
rotation constants as well as the c1-c5 xor-ing constants; let's
add ConfigurableParameters for that.

Change-Id: I397df6c0c708a8061e4adc0fde03a3f746bcb5b6
Related: SYS#7787
 2025-12-18 14:42:52 +01:00
Harald Welte
45bffb53f9 pySim.ts_51_011.EF_SMSP: Also permit UCS2 for the alpha_id 
TS 51.011 Section 10.5.6 refers to clause 10.5.1 (EF.ADN),
and the latter permits UCS2 in addition to 7-bit GSM alphabet.

Change-Id: If10b3d6d8b34ece02dc0350ca9ea9c3f8fbf3c9e
 2025-12-16 16:31:14 +01:00
Harald Welte
cc15b2b4c3 ts_51_011.EF_SMSP: Use integer division during encode 
Otherwise we might compute float values and fail encoding like this:

> construct.core.FormatFieldError: Error in path (building) -> tp_vp_minutes
> struct '>B' error during building, given value 169.0

Change-Id: I989669434c7ddee9595ee81a0822f9966907a844
 2025-12-16 16:31:12 +01:00
Harald Welte
11dfad88e6 pySim.esim.saip: Fix compatibility with pytohn < 3.11 
In python up to 3.10, the byteorder argument of the int.to_bytes()
method was mandatory, even if the length of the target byte sequence
is 1 and there factually is no byteorder.

https://docs.python.org/3.10/library/stdtypes.html#int.to_bytes
vs
https://docs.python.org/3.11/library/stdtypes.html#int.to_bytes

See also: https://discourse.osmocom.org/t/assistance-required-with-saip-pysim-tool-error-while-adding-applets-to-exiting-upp-der/2413/2

Change-Id: I8ba29d42c8d7bf0a36772cc3370eff1f6afa879f
 2025-12-14 13:58:34 +01:00
Harald Welte
572a81f2af pySim.runtime: Fix file selection by upper case hex FID 
When trying to remove a file (e.g. DF.5G_ProSe, 5FF0),
there seems to be a case sensitive check when checking for the dict:
pySim/runtime.py: get_file_for_filename():

478          def get_file_for_filename(self, name: str):
479              """Get the related CardFile object for a specified filename."""
480              sels = self.selected_file.get_selectables()
481              return sels[name]

The dict sels contains 5ff0, but not 5FF0.
The type of argument name is str. So a case sensitive check will be used.

Change-Id: Idd0db1f4bbd3ee9eec20f5fd0f4371c2882950cd
Closes: OS#6898
 2025-12-10 13:34:27 +00:00
Neels Hofmeyr
ff4f2491b8 fix downstream error: ImportError: cannot import name 'style' from 'cmd2' 
cmd2 version 3.0 was released, with significant API changes. Limit the
dependency to below 3.0, as already reflected in requirements.txt.

Seeing but not changing the discrepancy in minimum version:
requirements.txt has >2.6.2 while setup.py has >= 1.5.0.

Related: SYS#7775 SYS#7777
Change-Id: I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6
 2025-12-10 04:06:43 +01:00
Harald Welte
05fd870d1b contrib/saip-tool: Use repr() on security domain keys 
Let's not reinvent the wheel of printing such data structures and use
the repr method provided by the respective class instead.  This also
adds the missing key_usage_qualifier information to the print-out,
as well as the mac_len of the key components.

Change-Id: Iaead4a02f07130fd00bcecc43e1c843f1c221e63
 2025-12-09 16:23:49 +00:00
Harald Welte
c07ecbae52 pySim.esim.saip: Hex representation of SecurityDomainKey 
Let's print the key_usage_qualifier in hexadecimal notation (more compact)

Change-Id: Ic9a92d53d73378eafca1760dd8351215bce1157a
 2025-12-09 16:23:49 +00:00
Alexander Couzens
e20f9e6cdf ts_102_221: EF.ARR: fix read_arr_record 
`read_arr_record 1` failed with an AttributeError exception
because RECORD_NR must be all caps.

Change-Id: If44f9f2271293d3063f6c527e5a68dcfaeb5942e
 2025-12-04 15:32:16 +01:00
Philipp Maier
3f3f4e20e2 docs/conf.py: update copyright year 
The copyright year of the docs is still at 2023, let's update it
to the current year.

Change-Id: Icf64670847d090a250f732d94d18e780e483239b
 2025-11-25 17:14:54 +01:00
Philipp Maier
c2fb84251b card_key_provider: add missing type annotation 
Related: SYS#7725
Change-Id: I45751d2b31976c04c03006d8baa195eef2576b4f
 2025-11-21 17:49:09 +01:00
Philipp Maier
61541e7502 card_key_provider: refactor code and optimize out get_field method 
The method get_field in the base class can be optimized out. This
also allows us to remove code dup in the card_key_provider_get_field
function.

Let's also fix the return code behavior. A get method in a
CardKeyProvider implementation should always return None in case
nothing is found. Also it should not crash in that case. This will
allow the card_key_provider_get function to move on to the next
CardKeyProvider. In case no CardKeyProvider yields any results, an
exception is appropriate since it is pointless to continue execution
with "None" as key material.

To make the debugging of problems easier, let's also print some debug
messages that inform the user what key/value pair and which
CardKeyProvider was queried. This will make it easier to investigate
in case an expected result was not found.

Related: SYS#7725
Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9
 2025-11-21 17:49:09 +01:00
Philipp Maier
579214c4d0 card_key_provider: remove method _verify_get_data from base class 
The method _verify_get_data was intended to be used to verify the
user input before it further processed but ended up to be a simple
check that only checks the name of the key column very basically.

Unfortunately it is difficult to generalize the check code as the
concrete implementation of those checks is highly format dependent.
With the advent of eUICCs, we now have two data formats with
different lookup keys, so a static list with valid lookup keys is
also no longer up to the task.

After all it makes not much sense to keep this method, so let's
remove it.

(From the technical perspective, the key column is not limitied to
any specif field. In theory it would even be possible to use the KI
as lookup key as well, even though it would not make sense in
practice)

Related: SYS#7725
Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9
 2025-11-21 17:49:09 +01:00
Philipp Maier
4a7651eb65 pySim-shell: re-organize Card Key Provider related options 
As we plan to support other formats as data source for the Card Key
Provider soon, the more commandline options may be added and it makes
sense to group the Card Key Provider options in a dedicated group.

Let's also rename the option "--csv-column-key" to just "--column-key".
The column encryption is a generic concept and not CSV format specific.
(let's silently keep the "--csv-column-key" argument so maintain backward
compatibility)

Related: SYS#7725
Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c
 2025-11-21 17:49:09 +01:00
Philipp Maier
01a6724153 pySim-shell: add command to manually query the Card Key Provider 
The Card Key Provider is a built in mechanism of pySim-shell which
allows the user to read key material from a CSV file in order to
avoid having to lookup and enter the key material himself. The
lookup normally done by the pySim-shell commands automatically.

However, in some cases it may also be useful to be able to query the
CSV file manually in order to get certain fields displayed. Such a
command is in particular helpful to check and diagnose the CSV data
source.

Related: SYS#7725
Change-Id: I76e0f883572a029bdca65a5a6b3eef306db1c221
 2025-11-21 17:49:09 +01:00
Philipp Maier
a6ca5b7cd1 card_key_provider: remove unnecessary class property definitions 
The two properties csv_file and csv_filename are defined by the
constructor anyway, let's remove the declaration in the class body
because it is not needed.

Change-Id: Ibbe8e17b03a4ba0041c0e9990a5e9614388d9c03
 2025-11-21 17:49:09 +01:00
Philipp Maier
bcca2bffc2 card_key_provider: rename parameter filename to csv_filename 
let's rename the parameter filename to csv_filename to make it
more clear to what kind of file this parameter refers.

Change-Id: Id5b7c61b5e72fb205e30d2787855b2c276840a7b
 2025-11-21 17:49:09 +01:00
Philipp Maier
e80f96cc3b card_key_provider: use case-insensitive field names 
It is common in CSV files that the columns have uppercase names, so we
have adopted this scheme when we started using the card_key_provider.

This also means that the API of the card_key_provider_get() and
card_key_provider_get_field() function now implicitly requires
uppercase field names like 'ICCID', 'ADM1', etc.

Unfortunately this may be unreliable, so let's convert the field
names to uppercase as soon as we receive them. This makes the API
case-insensitive and gives us the assurance that all field names
we ever work with are in uppercase.

Related: SYS#7725
Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79
 2025-11-21 17:49:09 +01:00
Philipp Maier
4550574e03 card_key_provider: separate and refactor CSV column encryption 
The CardKeyProviderCsv class implements a column decryption scheme
where columns are protected using a transport key. The CSV files
are enrcypted using contrib/csv-encrypt-columns.py.

The current implementation has two main problems:

- The decryption code in CardKeyProviderCsv is not specific to CSV files.
  It could be re-used in other formats, for example to decrypt columns
  (fields) red from a database. So let's split the decryption code in a
  separate class.

- The encryption code in csv-encrypt-columns.py accesses methods and
  properties in CardKeyProviderCsv. Also having the coresponding
  encryption code somewhere out of tree may be confusing. Let's improve
  the design and put encryption and decryption functions in a single
  class. Let's also make sure the encryption/decryption is covered by
  unittests.

Related: SYS#7725
Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab
 2025-11-21 17:49:09 +01:00
Philipp Maier
08565e8a98 pySim-shell: use log level INFO by default 
The default log level of the PySimLogger is DEBUG by default. This is
to ensure that all messages are printed in an unconfigured setup.

However in pySim-Shell we care about configuring the logger, so let's
set the debug log level to INFO in startup. This will allow us to
turn debug messages on and off using the verbose switch.

Change-Id: I89315f830ce1cc2d573887de4f4cf4e19d17543b
Related: SYS#7725
 2025-11-21 17:49:09 +01:00
Harald Welte
fb20b7bc58 contrib: Add a small command line script to generate StoreMetadataRequest 
It's occasionally useful to be able to manually generate a
SGP.22 StoreMetadataRequest (tag BF25), so let's add a small utility
program doing exactly that.

Change-Id: I56ebd040f09dcd167b0b22148c2f1af56240b3b5
 2025-11-21 11:41:29 +00:00
Harald Welte
52df66cd56 pySim.esim.es8p: Support non-operational ProfileMetadata 
If no profileClass is given, ProfileMetadata defaults to operational.
Let's add the capability to also generate metadata for test or provisioning profiles.

Change-Id: Id55537ed03e2690c1fc9545bb3c49cfc76d8e331
 2025-11-21 11:41:29 +00:00
Philipp Maier
784cebded4 card_key_provider: add unit-test 
There is no unit-test for the CardKeyProviderCsv class yet. Let's add
one to ensure that the CardKeyProviderCsv class keeps working as expected.

Related: SYS#7725
Change-Id: I52519847a4c4a13a7bca49985133872b01c4aaab
 2025-11-18 15:47:40 +01:00
Philipp Maier
4f75aa1c8f card_key_provider: fix sourcecode formatting 
Change-Id: I5675f9f087086646937ca077d3545d2729ccd812
 2025-11-18 14:08:42 +01:00
Philipp Maier
94811ab585 pySim-shell: allow user friendly selection of the pin type 
The CHV commands (verify_chv, enable_chv, disable_chv, unblock_chv)
provide a --pin-nr parameter.

The --pin-nr is a decimal parameter that specifies the pin type to be
used. The exact pin type numbers are specified in ETSI TS 102.221,
Table 9.3.

Unfortunately the --pin-nr parameter is not very intuitive to use, it
it requires the user to manually lookup the numeric value. The specs
list that value as hexadecimal, so the user also has to convert it
to decimal. To make this less complicated, let's also accept
hexadecimal numbers with the --pin-nr parameter.

However, this alone does not improve the user expierience much. Let's
also add a --pin-type parameter (similar to the --adm-type parameter
of the verify_adm command) to specifiy the pin type in a human
readable form.

Change-Id: I0b58c402d95cbc4fe690e6edb214829d463e9f2c
 2025-11-01 14:03:23 +00:00
Philipp Maier
f3e6e85f99 osmo-smdpp: update documentation 
osmo-smdpp has built-in SSL/TLS support for quite some time now. The manual does not
yet mention this feature yet.

Change-Id: I2db5ae32914386a34eab1ed7d2aff8cae82bfa9b
 2025-10-31 16:31:45 +01:00
Philipp Maier
7f2cb157c8 osmo-smdpp: update commandline help and default port 
osmo-smdpp has built-in TLS support for some time now. Let's update
update the commandline help to be more concise.

Since the built-in SSL/TLS support is enabled by default, let's also
update the default port from 8000 to 443.

Change-Id: Ib5a069a8612beb1a9716a7514b498ec70d141178
 2025-10-31 16:31:40 +01:00
Philipp Maier
f94f366cf9 runtime: check record/file size before write 
When writing data to a transparent or linear fixed (record oriented)
and the data to write exceeds the record/file size, then the UICC will
respond with an error "6700: Checking errors - Wrong length"

In particular when the data is supplied as a JSON object and not as a
hex string, it may not be immediately obvious to the average user what
the problem actually is.

Let's check the record/file size before writing the data and raise an
exception in case the data excieeds the record/file size. Let's also
print an informative string message in case the data length is less
than the record/file size to make the user aware of unwritten bytes
at the end of a record/file.

Related: OS#6864
Change-Id: I7fa717d803ae79398d2c5daf92a7336be660c5ad
 2025-10-28 13:39:35 +01:00
Philipp Maier
4429e1cc70 pySim-shell: add a logger class to centralize logging 
In many sub modules we still use print() to occassionally print status
messages or warnings. This technically does not hurt, but it is an unclean
solution which we should replace with something more mature.

Let's use python's built in logging framework to create a static logger
class that fits our needs. To maintain compatibility let's also make sure
that the logger class will behave like a normal print() statement when no
configuration parameters are supplied by the API user.

To illustrate how the approach can be used in sub-modules, this patch
replaces the print statements in runtime.py. The other print statements
will the be fixed in follow-up patches.

Related: OS#6864
Change-Id: I187f117e7e1ccdb2a85dfdfb18e84bd7561704eb
 2025-10-25 19:46:34 +00:00
Philipp Maier
1ab2f8dd9d commands: do not use b2h with a string 
The function h2b expects a bytearray and must not be used on a string.
This is also true for nullstrings ('').

Related: OS#6869
Change-Id: I0e28e6ec476901bf19aa0f8640e41c74aa6e3aa2
 2025-10-21 17:17:21 +02:00
Oliver Smith
e5f39fbd34 Pass pylint 3.3.4 from debian trixie 
************* Module osmo-smdpp
osmo-smdpp.py:657:72: E0606: Possibly using variable 'iccid_str' before assignment (possibly-used-before-assignment)

=> False-positive: code paths that don't set iccid_str raise an error, so
   this shouldn't be a problem.

************* Module pySim-smpp2sim
pySim-smpp2sim.py:427:4: E1101: Module 'twisted.internet.reactor' has no 'run' member (no-member)

=> False-positive: pylint doesn't recognize dynamically set attributes.

************* Module es9p_client
contrib/es9p_client.py:126:11: E0606: Possibly using variable 'opts' before assignment (possibly-used-before-assignment)

=> Real bug, should be "self.opts".

Related: https://stackoverflow.com/a/18712867
Change-Id: Id042ba0944b58d98d27e1222ac373c7206158a91
 2025-10-02 09:06:44 +02:00
Harald Welte
947154639c pySim.esim.saip.FsNodeADF: Fix __str__ method 
It's quite common for a FsNodeADF to not have a df_name, so we need
to guard against that during stringification to avoid an exception.

Change-Id: I919d7c46575e0ebcdf3b979347a5cdd1a9feb294
 2025-09-24 17:59:17 +00:00
Kian-Meng Ang
4ee99c18cd Fix typos 
Found via `codespell -S tests -L ist,adn,ciph,ue,ot,readd,te,oce,tye`

Change-Id: I00a72e4f479dcef88f7d1058ce53edd0129d336a
 2025-09-24 17:59:17 +00:00
Eric Wild
5d2e2ee259 bsp: fix maxpayloadsize 
Change-Id: I08f544877b79681ad1f758a1ca31c292eae9f868
 2025-09-24 15:04:36 +00:00
Harald Welte
92841f2cd5 docs/suci-keytool.rst: spelling fix 
Change-Id: Idb45086d9d5963072fbc97835d551e2f78ad847f
 2025-09-04 18:57:27 +02:00
Bjoern Riemer
caa955b3ac Identify cards by Historical bytes of ATR 
- try to identify the CardModel by just comparing the Historical Bytes if matching by Whole ATR failed
- add decompose ATR code from pyscard-contrib

Related: OS#6837
Change-Id: Id7555e42290d232a0e0efc47e7d97575007d846f
 2025-08-28 21:44:24 +00:00
Bjoern Riemer
4dddcf932a Make sure to select MF before probing for files/Addons 
Change-Id: I685367c282f57a8856668a3172a9391a5bbcf2e2
 2025-08-28 21:44:24 +00:00
Oliver Smith
10fe0e3aae docs: fix authors line exceeding the page 
Fix that the authors get cut off as they exceed the page in the PDF
version. This can currently be seen here:
https://downloads.osmocom.org/docs/pysim/master/osmopysim-usermanual.pdf

Change-Id: Iacbba6c2f74bf2b9f96057e71bde017a11f703a8
 2025-08-27 14:31:13 +02:00
Oliver Smith
076fec267a requirements: set cmd2>=2.6.2,<3.0 
Remove the previous workaround that set cmd2==2.4.3 in jenkins.sh. The
bug this worked around has been fixed in 2.6.2.

3.0 will break unless we use some new additional decorator.

Related: OS#6776
Change-Id: I4ba65ed486247c5670313b75f43a242d264df14b
 2025-08-27 14:31:13 +02:00
Eric Wild
b4a12ecc14 smdp: update tls certs 
Old expired today, new from https://www.gsma.com/solutions-and-impact/technologies/esim/wp-content/uploads/2021/07/SGP.26_v1.5-17-July-2025_files_v3.zip

Change-Id: I585efe3360a0aac2a49a79d5fef2789dea2b169d
 2025-08-21 14:54:32 +00:00
Eric Wild
6cffb31b42 memory backed ephermal session store for easy concurrent runs 
Change-Id: I05bfd6ff471ccf1c8c2b5f2b748b9d4125ddd4f7
 2025-08-15 13:04:02 +02:00
Eric Wild
6aed97d6c8 smdpp: fix asn1tool OBJECT IDENTIFIER decoding 
While at it make the linter happy.
The feature to ignore blocks is making slow progress:
https://github.com/astral-sh/ruff/issues/3711#

Change-Id: Ic678e6c4a4c1a01de87a8dce26f4a5e452e8562a
 2025-08-15 13:04:02 +02:00
Eric Wild
cb7d5aa3a7 smdpp: add proper brp cert support 
Change-Id: I6906732f7d193a9c2234075f4a82df5e0ed46100
 2025-08-15 13:04:02 +02:00
Eric Wild
70fedb5a46 smdpp: verify cert chain 
Change-Id: I1e4e4b1b032dc6a8b7d15bd80d533a50fe0cff15
 2025-08-15 13:04:02 +02:00
Eric Wild
7798ea9c5c x509 cert: fix weird cert check 
Change-Id: I18beab0e1b24579724704c4141a2c457b2d4cf99
 2025-08-15 13:04:02 +02:00
Eric Wild
0b1d3c85fd smdpp: less verbose by default 
Those data blobs are huge.

Change-Id: I04a72b8f52417862d4dcba1f0743700dd942ef49
 2025-08-15 13:04:02 +02:00
Eric Wild
3c1a59640c smdp: clean up accidental commited trash 
and update gitinore to ensure this does not happen again

Change-Id: Ieeb2da3bdb006b08e2f82bfc3b5252f4abf4420b
 2025-08-15 13:04:01 +02:00
Eric Wild
ccefc98160 smdpp: add proper tls support, cert generation FOR TESTING 
If TLS is enabled (default) it will automagically generate missing pem files + dh params.

A faithful reproduction of the certs found in SGP.26_v1.5_Certificates_18_07_2024.zip available at
https://www.gsma.com/solutions-and-impact/technologies/esim/gsma_resources/sgp-26-test-certificate-definition-v1-5/
can be generated by running contrib/generate_certs.py. This allows adjusting the expiry dates, CA flag,
and other parameters FOR TESTING. Certs can be used by the smdpp by running
$ python -u osmo-smdpp.py -c generated

Change-Id: I84b2666422b8ff565620f3827ef4d4d7635a21be
 2025-06-25 10:22:42 +02:00
Eric Wild
79805d1dd7 smdpp: reorder imports 
Change-Id: Ib72089fb75d71f0d33c9ea17e5491dd52558f532
 2025-06-25 10:22:42 +02:00
Eric Wild
5969901be5 smdpp: Verify EID is within permitted range of EUM certificate 
Change-Id: Ice704548cb62f14943927b5295007db13c807031
 2025-06-21 13:18:38 +00:00
Eric Wild
5316f2b1cc smdpp: verify request headers 
Change-Id: Ic1221bcb87a9975a013ab356266d3cb76d9241f1
 2025-06-21 12:19:42 +00:00
Eric Wild
9572cbdb61 smdpp: update certs 
TLS will expire again:
$ find . -iname "CERT*NIST*der" | xargs -ti  openssl x509 -in {} -inform DER -noout -checkend $((24*3600*90))
...
openssl x509 -in ./smdpp-data/generated/DPtls/CERT_S_SM_DP_TLS_NIST.der -inform DER -noout -checkend 7776000
Certificate will expire
...

Grabbed from SGP.26_v1.5_Certificates_18_07_2024.zip available at
https://www.gsma.com/solutions-and-impact/technologies/esim/gsma_resources/sgp-26-test-certificate-definition-v1-5/

Change-Id: I25442d6f55a385019bba1e47ad3d795120f850ad
 2025-06-21 12:19:33 +00:00
Eric Wild
7fe7bff3d8 smdpp: optional deps 
Works locally, too:
$ pip install -e ".[smdpp]"

Change-Id: If69b2bd5f8bc604443108c942c17eba9c22f4053
 2025-06-16 13:37:43 +02:00
Harald Welte
c7c48718ba Get rid of [now] superfluous HexAdapter 
With the introduction of using osmocom.construct.{Bytes,GreedyBytes}
in Change-Id I1c8df6350c68aa408ec96ff6cd1e405ceb1a4fbb we don't have a
need for wrapping each instance of Bytes or GreedyBytes into a
HexAdapter anymore.  The osmocom.construct.{Bytes,GreedyBytes} will
automatically perform the related hex-string-to-bytes conversion if
needed - and during printing we have osmocom.utils.JsonEncoder that
makes sure to convert any bytes type to a hex-string.

Change-Id: I9c77e420c314f5e74458628dc4e767eab6d97123
 2025-05-07 19:35:54 +02:00
Harald Welte
e37cdbcd3e docs: Better python doc-strings for better pySim.esim manual 
Change-Id: I7be6264c665a2a25105681bb5e72d0f6715bbef8
 2025-05-07 10:50:47 +02:00
Harald Welte
89070a7c67 docs: Build the pySim.esim library documentation 
... we added doc-strings but missed to actually render them in the
manual so far.

Change-Id: Iff2baca86376e68898a8af0252906f802ffa79eb
 2025-05-06 21:43:46 +02:00
Vadim Yanitskiy
004b06eab1 jenkins.sh: workaround for 'usage: build.py' in docs 
Recent versions of cmd2 have changed how the 'prog' attribute is
automatically set for ArgumentParser instances.  As a result, we
are now seeing an unexpected 'build.py' artifact appearing in
the generated documentation.

Let's use an older release of cmd2, which retains the old expected
behavior.  Use it specifically for building documentation.

Change-Id: Ifbad35adc5e9d3141acfd024d7dee2a25f1cb62e
Related: https://github.com/python-cmd2/cmd2/issues/1414
Related: OS#6776
 2025-05-01 02:58:56 +07:00
Harald Welte
949c2a2d57 Use osmocom.construct.{Bytes,GreedyBytes} for hexstring input support 
The upstream construct.{Bytes,GreedyBytes} only support bytes/bytearray
input data for the encoder, while the [newly-created]
osmocom.construct.{Bytes,GreedyBytes} support alternatively hex-string input.

This is important in the context of encoding construct-based types from
JSON, where our osmocom.utils.JsonEncoder will automatically convert any
bytes to hex-string, while re-encoding those hex-strings will fail prior
to this patch.

Change-Id: I1c8df6350c68aa408ec96ff6cd1e405ceb1a4fbb
Closes: OS#6774
 2025-04-28 09:32:52 +02:00
Harald Welte
19f3759306 osmo-smdpp: Renew SGP.26 TLS certificate for SM-DP+ 
The SGP.26 v3.0 certificate had expired on July 11, 2024. Let's replace
it with a cert of 10 year validity period to facilitate uninterrupted testing
with osmo-smdpp.

@@ -1,12 +1,12 @@
 Certificate:
     Data:
         Version: 3 (0x2)
-        Serial Number: 9 (0x9)
+        Serial Number: 10 (0xa)
         Signature Algorithm: ecdsa-with-SHA256
         Issuer: CN=Test CI, OU=TESTCERT, O=RSPTEST, C=IT
         Validity
-            Not Before: Jun  9 19:04:42 2023 GMT
-            Not After : Jul 11 19:04:42 2024 GMT
+            Not Before: Apr 23 15:23:05 2025 GMT
+            Not After : Apr 21 15:23:05 2035 GMT
         Subject: O=ACME, CN=testsmdpplus1.example.com
         Subject Public Key Info:
             Public Key Algorithm: id-ecPublicKey

Change-Id: I6f67186b9b1b9cc81bfb0699a9d3984d08be8821
 2025-04-24 13:47:06 +00:00
Harald Welte
d838a95c2a edit_{binary,record}_decoded: Support hex-decode of bytes 
We've created + used osmocom.utils.JsonEncoder as an encoder class
for json.{dump,dumps} for quite some time.  However, we missed to
use this decoder class from the edit_{binary,record}_decoded commands
in the pySim-shell VTY.

Change-Id: I158e028f9920d8085cd20ea022be2437c64ad700
Related: OS#6774
 2025-04-24 13:47:06 +00:00
Vadim Yanitskiy
fbe6d02ce3 docs/saip-tool: fix ERROR: Unexpected indentation 
According to [1], the literal block must be indented (and, like all
paragraphs, separated from the surrounding ones by blank lines).

[1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#literal-blocks

While at it, fix tabs-vs-spaces: use 2 spaces like in other places.

Change-Id: If548bf66339433c1f3f9e2a557821e808c6afa26
 2025-04-24 03:05:30 +07:00
Vadim Yanitskiy
aace546900 filesystem: fix WARNING: Block quote ends without a blank line 
Use the 'r' (raw) qualifier to avoid rendering '\n' as the actual
line break in the auto-generated documentation.

Change-Id: Ie7f59685a78534eb2c43ec4bc39685d3fd264778
 2025-04-24 02:50:27 +07:00
Vadim Yanitskiy
08e6336fc9 doc/card-key-provider: fix WARNING: Title underline too short 
Change-Id: I29fda8350de75c4e7c0020fa4dce4cd0e5defda7
 2025-04-24 02:37:19 +07:00
Philipp Maier
d5da431fd4 saip-tool: add commandline option to edit mandatory services list 
Change-Id: I120b98d4b0942c26674bc1365c5711101ec95235
 2025-04-16 10:35:15 +02:00
Philipp Maier
59faa02f9a ara_m: add command to lock write access to the ARA-M rules. 
Recent versions of the ARA-M applet from Bertrand Martel can lock
the write access to ARA-M rules. Let's add a command for that and
some documentation.

Related: SYS#7245
Change-Id: I71581a0c9f146f9a0921093d9b53b053b4a8946c
 2025-04-14 11:14:36 +00:00
Philipp Maier
1dea0f39dc saip-tool: add features to add, remove and inspect application PEs 
The PE-Application object is used to provision JAVA-card applications
into an eUICC during profile installation. Let's extend the SAIP-tool
so that we are able to add, remove and inspect applications.

Change-Id: I41db96f2f0ccc29c1725a92215ce6b17d87b76ce
 2025-04-14 11:01:24 +00:00
Harald Welte
a2bfd397ba pySim-smpp2sim.py: Simulate SMSC+CN+RAN+UE for OTA testing 
The pySim-smpp2sim.py program exposes two interfaces:
* SMPP server-side port, so external programs can rx/tx SMS
* APDU interface towards the SIM card

It therefore emulates the SMSC, Core Network, RAND and UE parts
that would normally be encountered in an OTA setup.

Change-Id: Ie5bae9d823bca6f6c658bd455303f63bace2258c
 2025-04-08 18:14:18 +00:00
Philipp Maier
40e795a825 saip-tool: add ProfileElement class for application PE 
The application profile element has no ProfileElement class yet, so
let's create a ProfileElementApplication class and move the existing
extract-apps code into a method of ProfileElementApplication.

Change-Id: Iaa43036d388fbf1714c53cab1fc21092c4667a21
 2025-03-31 12:27:24 +02:00
Philipp Maier
dc2b9574c9 saip-tool: allow removing of profile elements by type 
At the moment it is only possible to remove profile elements by their identification
number. However, there may be cases where we want to remove all profile elements of
a certain type at once (e.g. when removing all applications).

Change-Id: I92f9f9d5b4382242963f1b3ded814a0d013c4808
 2025-03-28 14:35:40 +01:00
Philipp Maier
2b3b2c2a3b saip-tool: add option to extact profile elements to file 
In some cases it may be helpful to extract a single profile element
from the sequence to a dedicated file.

Change-Id: I77a80bfaf8970660a84fa61f7e08f404ffc4c2da
 2025-03-28 14:34:55 +01:00
Philipp Maier
02a7a2139f saip-tool: add function to write PE sequence 
To prevent code duplication and to make the implementation simpler,
let's add a function that takes care of writing the PE sequnece
to an output file.

Change-Id: I38733422270f5b9c18187b7f247b84bf21f9121b
 2025-03-28 13:25:30 +00:00
Harald Welte
701e011e14 [cosmetic] pySim.transport: Fix spelling/typos in comment 
Change-Id: Ia20cc2439bf00c1b6479f36c05514945ac4faf71
 2025-03-28 09:13:11 +01:00
Harald Welte
f57f6a95a5 pySim/commands: Fix envelope command APDU case after T=1 support 
When we merged I8b56d7804a2b4c392f43f8540e0b6e70001a8970 for T=1
support, the ENVELOPE C-APDU was not adjusted to reflect the correct
case.  ENVELOPE expects a response and hence needs a Le byte present.

This avoids below related message when performing e.g. OTA via SMS

  Warning: received unexpected response data, incorrect APDU-case (3, should be 4, missing Le field?)!

Change-Id: Ice12675e02aa5438cf9f069f8fcc296c64aabc5a
Related: OS#6367
 2025-03-28 09:13:11 +01:00
Philipp Maier
8da8b20f58 es8p: fix typo 
Change-Id: I241efe0c7ceab190b7729a6d88101501ca37652e
 2025-03-10 19:16:20 +00:00
Philipp Maier
74be2e202f filesystem: do not decode short TransRecEF records 
A TransRecEF is based on a TransparentEF. This means that a TransRecEF
is basically normal TransparentEF that holds a record oriented data
structure. This also requires that the total length of the TransRecEF
is a multiple of the record length of the data structure that is stored
in it. When this is not the case, the last record will be cut short and
the decoding will fail. We should guard against this case.

Related: OS#6598
Change-Id: Ib1dc4d7ce306f1f0b080bb4b6abc36e72431d3fa
 2025-03-10 18:59:08 +00:00
Neels Hofmeyr
cabb8edd53 pylint: ota.py: fix E0606 possibly-used-before-assignment 
************* Module pySim.ota
pySim/ota.py:430:24: E0606: Possibly using variable 'cpl' before assignment (possibly-used-before-assignment)

Change-Id: Ibbae851e458bbe7426a788b0784d553753c1056f
 2025-03-07 21:27:01 +01:00
Neels Hofmeyr
19e1330ce8 pylint: personalization.py: fix E1135: permitted_len unsupported-membership-test 
pre-empt this from coming up in patch
I60ea8fd11fb438ec90ddb08b17b658cbb789c051:

E1135: Value 'self.permitted_len' doesn't support membership test (unsupported-membership-test) pickermitted_len

Change-Id: I0343f8dbbffefb4237a1cb4dd40b576f16111073
 2025-03-07 21:26:54 +01:00
Neels Hofmeyr
e91488d21f .gitignore: smdpp-data/sm-dp-sessions from running osmo-smdpp.py 
Change-Id: I02a4ad4bc8e612e64111b16bc11c8c3d4dd41c45
 2025-03-01 23:17:57 +01:00
Neels Hofmeyr
9e8143723d .gitignore tags (from ctags) 
Change-Id: I1ae374e687b885399e0abfa39fcd750d944ae7ce
 2025-03-01 23:17:57 +01:00
Neels Hofmeyr
15df7cbf88 add PEM cert as used in docs/osmo-smdpp.rst 
Add PEM version of smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_NIST.der

A CERT_S_SM_DP_TLS_NIST.pem file is referenced in docs/osmo-smdpp.rst --
nginx apparently cannot use DER certs, so it is convenient for beginners
if the example from the docs just works without having to know that:

The added file was produced using

    openssl x509 -inform DER -in CERT_S_SM_DP_TLS_NIST.der -outform PEM -out CERT_S_SM_DP_TLS_NIST.pem

Change-Id: I41ba6ebacb71df0eb8a248c0c3c9ccd709718d74
 2025-03-01 23:17:56 +01:00
Neels Hofmeyr
1d962ec8c8 osmo-smdpp.py: enable --host and --port cmdline args (and document) 
Change-Id: Ic98dac1e1e713d74c3f8052c5bbeb44445aa8ab4
 2025-03-01 23:17:56 +01:00
Neels Hofmeyr
80a5dd1cf6 docs/osmo-smdpp.rst: fix typo apostrophe 
Change-Id: I32b18a61301fc2784675fa8acbeadb996ebcd821
 2025-03-01 23:17:56 +01:00
Philipp Maier
c4a6b8b3e7 pySim-shell: obey quit command in startup commands+scripts 
Startup scripts are executed using the cmd2 provided onecmd_plus_hooks
method. This method can run arbitrary commands, which also includes
the command "run_scrit" that we use to execute startup scripts.

When a script executes a quit command, or when someone issues a quit
command using the --execute-command or the command argument, then
this commands is executed. However a quit command won't actually quit
the process. All it does is to change the return code of
app.onecmd_plus_hooks (see [1]). So we must evaluate the return code
and take care of the quitting ourselves.

[1] https://cmd2.readthedocs.io/en/0.9.15/api/cmd.html#cmd2.cmd2.Cmd.onecmd_plus_hooks

Related: OS#6731
Change-Id: Ic6e9c54cdb6955d65011af3eb5a025eee5da4143
 2025-02-25 14:55:49 +01:00
Harald Welte
de91b0dc97 euicc: Add euicc_memory_reset shell command 
This implements the ES10c eUICC Memory Reset procedure

Change-Id: Ib462f5b7de3e500e51c0f3d6e2b9b0c2d3ba7e20
 2025-02-14 12:32:41 +01:00
Neels Janosch Hofmeyr
30e40ae520 setup.py: install esim.asn1 resources, install esim.saip 
These changes are necessary to successfully run
./tests/unittests/test_esim_saip.py with a pySim installed via
'pip install'.

For example:

   virtualenv venv
   source venv/bin/activate
   git clone ssh://gerrit.osmocom.org:29418/pysim
   pip install pysim/
   cd pysim
   ./tests/unittests/test_esim_saip.py

Before this patch, that would result first in package pySim.esim.saip
being unknown (not installed at all), and when that is added to
setup.py, in this error:

	Traceback (most recent call last):
	  File "/home/moi/osmo-dev/src/pysim/tests/unittests/./test_esim_saip.py", line 23, in <module>
	    from pySim.esim.saip import *
	  File "/home/moi/s/esim/sysmo_esim_mgr/venv/lib/python3.13/site-packages/pySim/esim/saip/__init__.py", line 41, in <module>
	    asn1 = compile_asn1_subdir('saip')
	  File "/home/moi/s/esim/sysmo_esim_mgr/venv/lib/python3.13/site-packages/pySim/esim/__init__.py", line 56, in compile_asn1_subdir
	    for i in resources.files('pySim.esim').joinpath('asn1').joinpath(subdir_name).iterdir():
		     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
	  File "/usr/lib/python3.13/pathlib/_local.py", line 577, in iterdir
	    with os.scandir(root_dir) as scandir_it:
		 ~~~~~~~~~~^^^^^^^^^^
	FileNotFoundError: [Errno 2] No such file or directory: '/home/moi/s/esim/sysmo_esim_mgr/venv/lib/python3.13/site-packages/pySim/esim/asn1/saip'

After this patch, the test completes successfully.

	......
	----------------------------------------------------------------------
	Ran 6 tests in 0.067s

	OK

Related: sysmocom's eSIM manager product that is currently in
development needs to fully use pySim.esim.saip, ideally from a regular
'pip install', and not from using the pySim source tree directly.

Related: SYS#6768
Change-Id: I0d7d6962a308eccca589a42c22546d508ff686f5
 2025-02-08 02:02:18 +01:00
Neels Janosch Hofmeyr
8a61498ba6 .gitignore: dist subdir, may be created by pip 
Change-Id: Ib23a687845842bd25d83f87aa00ae0c278abc842
 2025-02-08 01:59:44 +01:00
Philipp Maier
edcd62435d pySim/transport: add abstract get_atr method to LinkBase 
The implementations that inheret from the LinkBase class are expected to
implement a get_atr method. This method is mandatory, since it is one of
the most basic functionalities of pySim to display an ATR. Also the ATR
is sometimes needed to distinguish between different card models.

The modem_atcmd and calypso implementation completely lack the get_atr
method. Apparantly it is not possible to get an ATR in those
environments, so lets add a dummy method there.

Related: OS#6322
Change-Id: I4fc020ca45658af78e495a5c1b985213f83cbb50
 2025-01-29 13:35:44 +01:00
Harald Welte
08ba187fd4 ATR: align get_atr() return value type 
type annotations claimed the return type was Hexstr, but in reality
it was a list of integers.  Let's fix that.

Change-Id: I01b247dad40ec986cf199302f8e92d16848bd499
Closes: OS#6322
 2025-01-29 13:00:53 +01:00
Philipp Maier
d871e4696f ATR: use lowercase hex strings without spaces as ATR constants 
The ATR constants are the only hex string constants where the hex
bytes digits are separated with spaces. Also the hex digits are in
lowercase. Let's use a lowercase string without spaces here like
we do in many other code locations.

Related: OS#6322
Change-Id: I95118115b02523ed262a2fbe4369ace3996cd8f5
 2025-01-29 11:26:54 +01:00
Philipp Maier
15140aae44 global_platform: add new command "install_cap" 
Installing JAVA-card applets from a CAP file is a multi step process, which is
difficult when done manually. Fortunately it is easy to automate the process,
so let's add a dedicated command for that.

Change-Id: I6cbd37f0fad5579b20e83c27349bd5acc129e6d0
Related: OS#6679
 2025-01-22 16:46:32 +01:00
Harald Welte
a0071b32ff global_platform: LOAD and INSTALL [for load] support 
In this patch we add the commands "install_for_load" and "load".

Depends: pyosmocom.git I86df064fa41db85923eeb0d83cc399504fdd4488
Change-Id: I924aaeecbb3a72bdb65eefbff6135e4e9570579e
Related: OS#6679
 2025-01-22 15:42:09 +01:00
Philipp Maier
f688d28107 global_platform: fix usage of the Key Version Number (kvn) 
The kvn parameter is used to select a keyset when establishin a secure channel.
At the moment this is a mandatory parameter and it must be within a certain
range.

However GPC_SPE_034 explicitly defines a reserved kvn value 0, that always
refers to the first available key. That effectively makes it an optional
parameter and the commandline interface should have the --key-ver parameter
as an optional parameter.

The ranges also have to be extended to allow 0 as kvn value. We also have to
put a range to support the sysmoUSIM-SJS1, which uses kvn value 1, which is
a non standard value.

Related: OS#6679
Change-Id: I42be2438c7f199b238f2ec7a9434cec5393210a7
 2025-01-15 15:02:46 +01:00
Harald Welte
14d6e68ff7 cards: Avoid exception seen with (some) GSM-R SIM cards 
Some old cards are classic SIM and not based on UICCs.  Such cards
do not offer the capability of selecting applications.  Let's avoid
running into an exception by providing dummy methods that simply fail
for each AID selection.

Change-Id: Ib3457496380c0c5096052ad7799970ee620dee33
Closes: OS#6691
 2025-01-12 14:31:50 +01:00
Philipp Maier
712946eddb javacard: add parser for JAVA-card CAP file format 
To install JAVA-card applets we need to be able to extract the executeable
loadfile and the AIDs of the applet and the loadfile. This patch adds the
parser and related unittests.

Related: OS#6679
Change-Id: I581483ccb9d8a254fcecc995fec3c811c5cf38eb
 2025-01-06 11:25:14 +01:00
Philipp Maier
6d2e3853b4 global_platform: add spec reference to help of --install-parameters 
Related: OS#6679
Change-Id: I7e8174d469e09ad130d2866663a65bdeb4afc35a
 2024-12-20 15:54:17 +01:00
Philipp Maier
2a833b480a global_platform: fix command "delete" 
The delete command formats a TPDU, not APDU, which leads to warning messages

Related: OS#6679
Change-Id: Id04c89acbd4f449cb974d3cb05024f11dba4684e
 2024-12-19 18:26:58 +01:00
Philipp Maier
6287db4855 global_platform: remove unused code 
This commented out part is not needed anymore.

Related: OS#6679
Change-Id: If1de0218f841159789ac86f6a13740c1cbd0a57a
 2024-12-19 18:08:54 +01:00
Philipp Maier
9df5e2f171 javacard, cosmetic: fix sourcecode fromatting and improve docstring 
The line with TAGS is longer than 120 columns and there is some
comment that should be moved to the python docstring.

Related: OS#6679
Change-Id: I1d02098320cfbe17a0eb1bfdcf3bc85034cc8e20
 2024-12-19 18:05:06 +01:00
Philipp Maier
25319c5184 ara_m fix export of AID-REF-DO (empty) 
GPD_SPE_013 Table 6-3 defines two types of AID-REF-DO objects (both
are fully independed TLV IEs with the same name). The version with
tag '4F' identifies an SE application. It may contain an AID prefix
or even be of length 0 in case the rule should apply to all SE
applications. Then there is the version with tag 'C0', which must
always have length 0 and serves a flag to apply the rule to the
implicitly selected SE application. Technically both are completely
different things, so we must also treat them separately in the
pySim-shell code.

Related: OS#6681
Change-Id: I771d5e860b12215280e3d0a8c314ce843fe0d6a2
 2024-12-11 11:11:44 +01:00
Philipp Maier
8711bd89b0 ara_m: fix spec reference. 
there are multiple references to a specification "SEID". As it seems this is
a reference to the GlobalPlatform "Secure Element Access Control" spec, which
has the document reference "GPD_SPE_013". Let's use "GPD_SPE_013" to referene
the spec.

Related: SYS#6681
Change-Id: I77895f1b84126563380ce89aa07a3b448d8784a3
 2024-12-06 17:33:40 +01:00
Harald Welte
16920aeacd README.md update / re-wording 
Let's give a better description of what the project is all about, and
differentiate reading/exploring any SIM from writing/updating a special
programmable one where you know the ADM credentials.

Change-Id: Ied2a9626594e9735d92d4eabe6c6b90f92aa2909
 2024-12-05 16:33:34 +00:00
Philipp Maier
67c0fff15b pySim-shell: change Prompt character to "#" after "verify_adm" 
Let's change the prompt from ">" to "#" when the user gains admin
privilegs using verify_adm.

Related: OS#6640
Change-Id: I957b9df7b5069b6fce5bf958c94e8ffda833c77f
 2024-11-27 14:41:38 +01:00
Philipp Maier
9f9e931378 pySim-shell: reset card in method equip 
When the equip method is running, all kinds of states in pySim-shell are reset.
To be sure that the card state is also reset (normally this is the case because
usually init_card is called before equip), we should send an explicit reset to
the card as well.

Related: OS#6640
Change-Id: I622a2df2c9184841f72abd18483bfbfd00b2f464
 2024-11-27 14:41:38 +01:00
Philipp Maier
45d1b43393 ts_31_102: fix testcase for EF_ePDGSelection 
the testcase EF_ePDGSelection has a wrong testvector in the plmn field.
This test vector is accepted because there is a complementary error in
pyosmocom. However, the root problem got fixed (see depends), which means
that the test vector of EF_ePDGSelection now needs to be updated.

Depends: pyosmocom.git: I3811b227d629bd4e051a480c9622967e31f8a376
Change-Id: I96fd4c13c8e58ef33ddf9e3124617b1b59b9b2c1
Related: OS#6598
 2024-11-27 10:07:51 +01:00
JPM
ceed99ad3c Fixing 3-digit MNC PLMN Encoding/Decoding tests expected values for EF_OPL and EF_ePDGSelection. 
Related: pyosmocom.git I3811b227d629bd4e051a480c9622967e31f8a376
Change-Id: Ib2b586cb570dbe74a617c45c0fca276b08bb075e
 2024-11-27 07:22:33 +00:00
Harald Welte
2debf5dc4b docs/shell: Fix documentation for eUICC ISD-R specific commands 
Back in January 2024 in change 7ba09f9392
we migrate dthe commands from 'class ADF_ISDR' to CardApplicationISDR
without updating the sphinx-argparse references in the documentation.

Let's fix that, making the syntax reference for those commands re-appear
in the documentation.

Change-Id: I1d7e2d1a5dfbdcc11b1fdb3e89845787f7cddbfc
 2024-11-26 21:24:56 +01:00
Harald Welte
708a45bcee es2p_client: Print the activation code after confirmOrder success 
Change-Id: I92608ff0cdc35b184edff0c656221644ba36f257
 2024-11-25 20:29:59 +01:00
Harald Welte
1be2e9b713 contrib/suci-keytool.py: Convenience tool for SUCI key generation 
This adds a small utility program that can be used for generating
keys used for SUCI in 5G SA networks, as well as for dumping them
in a format that's compatible with what is needed on the USIM.

Change-Id: I9e92bbba7f700e160ea9c58da5f23fa4c31d40c6
 2024-11-25 20:29:59 +01:00
Harald Welte
73c76e02ce contrib/esim-qrcode.py: Small command line tool to encode eSIM QR codes 
Change-Id: I7983de79937124cc258efd459c51f812f5fa79cb
 2024-11-25 20:29:59 +01:00
Harald Welte
d1ddb1e352 docs: Add documentation about contrib/sim-rest-{server,client} 
Those programs have been around since 2021 but we never had any
documentation here. Let's fix that.

Change-Id: I7c471cac9500db063a0c8f5c5eb7b6861b3234ed
 2024-11-25 20:29:56 +01:00
Harald Welte
0bb8b44ea8 esim.saip.ProfileElementUSIM: Fix IMSI decode if [only] template based 
In case the fileDescriptor of EF.IMSI is purely template based and only
the file content is given in the actual profile, we must pass a template
reference to the File() constructor before we can read the IMSI.

This fixes the following exception for some profiles:
	ValueError: File(ef-imsi): No fileDescriptor found in tuple, and none set by template before

Change-Id: I14157a7b62ccd9b5b42de9b8060f2ebc5f91ebb3
 2024-11-23 15:43:12 +01:00
Harald Welte
9d7caef810 esim.saip.FsProfileElement: Add create_file() method 
So far we mainly created File() instances when parsing existing
profiles.  However, sometimes we want to programmatically create Files
and we should offer a convenience helper to do so, rather than asking
API users to worry about low-level details.

Change-Id: I0817819af40f3d0dc0c3d2b91039c5748dd31ee2
 2024-11-22 21:02:35 +01:00
Harald Welte
9ac4ff3229 esim.saip.File: Suppress encoding attributes that are like template 
The point of the SAIP template mechanism is to reduce the size of the
encoded profile.  Therefore, our encoder in the to_fileDescriptor()
method should suppress generating attributes if their value is identical
to that of the template (if any).

Change-Id: I337ee6c7e882ec711bece17b7a0def9da36b0ad7
 2024-11-22 21:00:47 +01:00
Harald Welte
0f1ffd20ef esim.saip.File: Proper ARR conversion of template (into) to file (bytes) 
The encoding of the access rule reference is different in FileTemplate
vs File, let's make sure we properly convert it when instantiating a
File from a FileTemplate.

Change-Id: Ibb8afb85cc0006bc5c59230ebf28b2c0c1a8a8ed
 2024-11-22 20:59:19 +01:00
Harald Welte
0516e4c47a esim.saip.File: Re-compute file_size when changing body 
If the API user modifies the size of the body, we need to check if we
need to re-compute the file_size attribute which is later encoded into
the fileDescriptor.  The size obviously must be large enough to fit the
body.  Let's do this implicitly by introducing a setter for File.body

Change-Id: I1a908504b845b7c90f31294faf2a6e988bdd8049
 2024-11-22 20:56:58 +01:00
Harald Welte
3442333760 esim.saip: New methods for inserting ProfileElement into sequence 
ProfileElements.insert_after_pe() is a convenience method to insert
a new PE after an existing one in the sequence.  This is a frequent
task as there are strict ordering requirements in the SAIP format.

Change-Id: I4424926127b4867931c2157e9340bacd2682ff0c
 2024-11-22 20:49:24 +01:00
Harald Welte
5354fc22d0 [cosmetic] esim: Fix various typos in comments/messages/docs 
Change-Id: I806c7a37951e72027ab9346169a3f8fe241f2c46
 2024-11-22 17:04:30 +01:00
Harald Welte
93237f4407 [cosmetic] esim.saip: Fix various typos in comments/docs/messages 
Change-Id: I4fc603634a0f2b53e432a77f05e811a38ba065c2
 2024-11-22 16:59:26 +01:00
Harald Welte
779092b0cd esim.saip: Fix computation of file content 
When generating the file content (body), we need to proceed in the
following order:

1a) If FCP contains fillPattern/repeatPattern, compute file content from those

1b) If FCP doesn't contain fillPattern/repeatPattern but template
    exists, compute file content from template

2)  Apply any fillFileConten / fillFileOffset from the SAIP File on top
    of the above

Change-Id: I822bb5fbec11a3be35910a496af7168458fd949c
Closes: OS#6642
 2024-11-22 16:03:58 +01:00
Harald Welte
6046102cbb esim.saip: Compute number of records from efFileSize and record_len 
If we know the efFileSize and record_len, but Fcp doesn't contain
the number of records, we can simply compute it.

Change-Id: I0cc8e7241e37ee23df00c2622422904e7ccdca77
 2024-11-22 16:01:58 +01:00
Harald Welte
118624d256 pySim.esim.saip: Treat "Readable and Updateable when deactivated" flag 
There's a second flag hidden in the TS 102 222 "Special File
Information"; let's parse + re-encode it properly.

Change-Id: I7644d265f746c662b64f7156b3be08a01e3a97aa
Related: OS#6643
 2024-11-22 16:01:58 +01:00
Harald Welte
599845394e esim.saip: Fix parsing/generating fillPattern + repeatPattern 
So far we only thought of default filling coming from a template.
However, filling can happen from the Fcp, and we need to properly parse
and [re-]encode that information.

Change-Id: Iff339cbe841112a01c9c617f43b0e69df2521b51
Related: OS#6643
 2024-11-22 16:01:25 +01:00
Philipp Maier
de8cc322f1 docs: add topic about remote UICC/eUICC access 
With osmo-remsim and Android APDU proxy we have two powerful solutions to
allow remote acces to UICC/eUICC cards. Let's add a section where we give
a brief overview about those solutions, so that pySim-shell users get
awre of them.

Related: OS#6367
Change-Id: I73de4de2e5d4a01d6d91989ee684cbdb680de8ef
 2024-11-19 10:56:26 +01:00
Philipp Maier
385d4407da pySim-shell_test: add new testcase for card initialization 
The card initialization normally takes place automatically. Nearly all
testcases implicitly cover this code-path. However, it is also possible
to skip the card initialization and do it at some later point. This is
commonly the case for unprovisioned card that require some custom APDUs
in a basic initialization step. When this step is done one would use
the "equip" command to level up to the full featured mode. This patch
adds a testcase for this scenario

Related: OS#6367
Change-Id: I01a03fa07d8c62164453bd707c5943288ff1a972
 2024-11-19 10:56:26 +01:00
Philipp Maier
852eff54df pySim/transport add support for T=1 protocol and fix APDU/TPDU layer conflicts 
ETSI TS 102 221, section 7.3 specifies that UICCs (and eUICCs) may support two
different transport protocols: T=0 or T=1 or both. The spec also says that the
terminal must support both protocols.

This patch adds the necessary functionality to support the T=1 protocol
alongside the T=0 protocol. However, this also means that we have to sharpen
the lines between APDUs and TPDUs.

As this patch also touches the low level interface to readers it was also
manually tested with a classic serial reader. Calypso and AT command readers
were not tested.

Change-Id: I8b56d7804a2b4c392f43f8540e0b6e70001a8970
Related: OS#6367
 2024-11-19 10:56:26 +01:00
Philipp Maier
f951c56449 global_platform/scp: refactor _wrap_cmd_apdu 
The _wrap_cmd_apdu methods for SCP02 and SCP03 are a bit hard to read. Let's
refactor them so that it is easier to understand what happens. In particular
that one can not have encryption (cenc) without signing (cmac)

Related: OS#6367
Change-Id: I4c5650337779a4bd1f98673650c6c3cb526d518b
 2024-11-19 09:55:48 +00:00
Philipp Maier
90881a2fff docs/osmo-smdpp: restructure subsection "osmo-smdpp" 
Sphinx is complaining about a duplicate label "osmo-smdpp". Apparantly
because we use this label twices as section headline. The subsection
"osmo-smdpp" in "Running osmo-smdpp" talks about the commandline and the
supplementary files that osmo-smdpp needs to run. Let's split the two
topics into two different sections.

Change-Id: I8bc4979160a00d36a03b9cd10679562a08c2c55c
 2024-11-18 10:49:25 +01:00
Philipp Maier
4aaccf8751 docs/legacy: remove unused '::' paragraph. 
Change-Id: If51564665d3793d9108053ffeb97d81ae93ced51
 2024-11-18 10:49:20 +01:00
Philipp Maier
3ef2c40951 docs/osmo-smdpp: fix typo 
Change-Id: I9978c5e02c1affe95a3b72d63e88965d7af5303e
 2024-11-18 10:49:02 +01:00
Philipp Maier
b845aab473 docs/osmo-smdpp: fix markup 
Change-Id: I4a0ed6fb2eedf1892835c43d304a53c995f028c8
 2024-11-18 10:48:50 +01:00
Philipp Maier
30c59fce42 pySim-shell_test/utils: treat cmd2 error "not a recognized command... as exception 
When a pySim-shell command is not recognized, cmd2 prints "xyz is not a
recognized command, alias, or macro." This string is a normal print out
and not an exception, but during tests, it may point out a severe problem
and therefore it should be tread like an exception.

Related: OS#6367
Change-Id: I17be6af1547b31170622e17b9cfb9c492597670d
 2024-11-04 15:01:16 +01:00
Philipp Maier
ec30022b1a pySim-shell: add new commandline option "--skip-card-init" 
by default pySim-shell does all kinds of probing and file selection
on startup. This is to determine the card type and to find a suitable
card profile. However, in case the card is non yet provisioned this
probing may cause a error messages and even might upset the cards
internal state. So let's have a commandline option thrugh which we
can instruct pySim-shell to skip any initialization and to give us
a prompt immediately, so that we can enter custom APDUs

Related: OS#6367
Change-Id: I1d8a57de201fe7ad7cbcbc6f72969ea8521e821d
 2024-11-04 14:54:33 +01:00
Philipp Maier
daa1c74047 pySim-shell: fix reset command for no-profile mode 
There are situations where no card profile can be determined. In this case no
RuntimeState will be present. This is in particular the case when pySim-shell
is used on a card that is not provisioned/initialized yet. In those cases we
have to go the direct route and reset the card directly.

Related: OS#6367
Change-Id: I27bf9fdb131d8bdeba07f4dfd2b76b38f9bfdd17
 2024-11-04 11:28:05 +01:00
Philipp Maier
5887fb70fb pySim-shell: allow checking of APDU responses 
The "apdu" command allows us to send custom APDUs to a card. This command is
often used in low level initialization scripts or tests. To stop the script
execution in case of an error, the command allows us to specify a status word
that must match the status word of the response. But we have no such mechanism
for the response itself. Let's add another parameter where we can pass a regex
that the response must match.

Related: OS#6367
Change-Id: I97bbcdf37bdcf00ad50a875b96940c211de7073d
 2024-11-04 11:28:05 +01:00
Philipp Maier
882e24677f pySim-shell_test/utils: print logfile on all types of errors 
When pySim-shell has problems starting up, it exits with an error
code. This is detected by the testsuite, but it also causes an
early exit, so that the log file content are not printed.

Change-Id: Ic0f34eda32a7c557810abcb05a84e343741fdb8a
 2024-11-04 11:28:05 +01:00
Philipp Maier
f4c156ae57 global_platform/scp: mapdu may be undeclared 
when we sign and encrypt the APDU in _wrap_cmd_apdu (SCP03) we return an "mapdu"
at the end. However, in the (unlikely?) case where self.do_cencand
self.do_cmac are false, mapdu will be undeclared. In _wrap_cmd_apdu for SCP02
we just re-use the apdu variable and return it at the end, so when no
encryption and no signing is applied, the APDU falls just through without any
modifications. We should have the same mechanism for the SCP03 wrapping as
well.

Related: OS#6367

Change-Id: Ic7089a69dffd7313572c5b3e5953200be5925766
 2024-11-04 11:28:05 +01:00
Philipp Maier
59593e0f28 pySim-shell-test: improve global platform tests 
The tests that check the establishment of a secure channel currently only test
security level 3. Also the get_data command after it only tests data reception
from the card.

Let's extend the test coverage and test the SCP establishment for security
level 1 as well. Let's also add a get_status command to make sure sending data
to the card also works (without exceptions).

Related: OS#6367
Change-Id: Idff40b414a249e532df1bdce2a8deb9b0cb9718f
 2024-11-04 11:28:05 +01:00
Philipp Maier
35b9b3c542 commands: fix apidoc (wrong order of parameters) 
Change-Id: I4d17c71c7f992ecd795dd214d34f2e094c0a5b53
 2024-11-01 10:29:27 +01:00
Philipp Maier
464d1ac2be commands: fix double space character in apidoc 
Change-Id: Id0dbe4578fd212bc240aac80e1e416cb57e92cc7
 2024-11-01 10:29:27 +01:00
Philipp Maier
909b8c1611 global_platform/scp: fix typo 
Change-Id: Ib26d983c6a80419326de812af2781c5e710dbcfc
 2024-11-01 10:29:27 +01:00
Philipp Maier
5d54f3b8d8 commands: fix typo 
Change-Id: I4103b7474063a26f09666361aef72abcd35bc12d
 2024-11-01 10:29:15 +01:00
Philipp Maier
98f4ea1447 pySim-shell_test/utils: display pySim-shell logfile content 
When we configure the tests to display file content, we only display files that
we compare, let's also display log file contents from pySim-shell. This will
be useful in situations where we only have log output from the tests, but no
access to the file system of the test host.

Related: OS#6601
Change-Id: Ibf6f78d7e71c213c7ca1caaf21c4c890e892261e
 2024-10-25 23:41:29 +00:00
Philipp Maier
32d6a9ab5f pySim-shell_test/utils: enumerate pySim-shell logs 
When pySim-shell is called by a testcase, a logfile is createted. The logfile
filename contains the testcase name. However, a testcase may run pySim-shell
multiple times. In this case we overwrite the log from previous run. Let's use a
counter to generate unique file names for each run, so that we won't lose logs
from previous runs.

Related: OS#6601
Change-Id: Ib2195d9b2231f74d0a6c4fb28f4889b6c45efb1e
 2024-10-25 23:41:29 +00:00
Philipp Maier
d8d52bdf77 pySim-shell_test/utils: delete log files in general 
When we get rid of temporary files, we delete those using a wildcard,
but for the logs from pySim-shell we explicitly memorize the name
of the pySim-shell logfile and delete it later by this explicit
name. This is not necessary, let's just delete all log files present
using a wildcard.

Related: OS#6601
Change-Id: I09dc7e59d1a3dcb68f54e3a8dccb86a1bc6c9ee6
 2024-10-25 23:41:29 +00:00
Harald Welte
12328c090d pySim.ts_31_102: Add support for EF.EARFCNList 
This adds a construct + pyosmocore.tlv based declarative encoder/decoder
for the EF.EARFCNList file used in the context of NB-IoT in later
release USIMs.

Change-Id: I16797ca58c3ad6ebaf588d04fec011a0cbcfcef3
 2024-10-25 18:09:19 +02:00
Philipp Maier
ba22e238f3 global_platform: ensure ArgumentParser gets a list for choices 
When we use the argument parser with choices, we sometimes use a
list that we derive from a dictionary. However if we do that we
still must ensure that we really put a list and not a dict_values
or dict_keys class as parameter because this won't work any longer
with cmd2 version 2.5.0

Related: OS#6601
Change-Id: I165fefd8feb0d96cedc15d036fb32da381f711b3
 2024-10-25 15:30:12 +02:00
Harald Welte
f9631fb361 pySim.esim.saip.templates: Fix DF_TELECOM FileID (7F10, not 7F11) 
Change-Id: I4bc37f9d99c046cd6c6accaaf460a39eb79b660f
 2024-10-20 10:14:34 +02:00
Harald Welte
f4dd9b5ceb docs/shell: Add missing :ref: when referencing other command 
Change-Id: I18f110e6313932d82b19ecaa7e07ef00c2339513
 2024-10-20 10:14:19 +02:00
Philipp Maier
82b0f1b39a pySim-shell_test: re-enable test_list_and_rm_notif 
The problems with test_list_and_rm_notif (see also change id
I7d0b6a998499d84f0eb4e24592ad43210ac54806) are now resolved, so
we can re-enable the testcase now.

Closes: SYS#7094
Change-Id: I95eb3b9c02a69653797851197e882ea9316805fc
 2024-10-11 16:13:07 +02:00
Harald Welte
3a905d637c pySim.euicc: Fix ASN.1 encoding of integer values 
Change-Id: I26ee41705f5e95c5fa3a9997cbaebdacca3e89a7
Closes: SYS#7094
 2024-10-11 16:13:07 +02:00
Oliver Smith
a8cfeb0111 docs/Makefile: make SPHINXBUILD work in venv 
sphinx-build doesn't use the PYTHONPATH from the venv, unless it runs
as python3 -m sphinx.cmd.build. We need it to use the imports from
PYTHONPATH, so we can update the pyosmocom version in requirements.txt
in a patch, and this new version will be used in the jenkins job that
runs during gerrit review. Otherwise the previously installed version
(from the docker image) will be used.

Related: https://github.com/sphinx-doc/sphinx/issues/8910
Change-Id: I487e1af6a3493df5b806cc2d3d2b70bc5233b89f
 2024-10-11 16:10:51 +02:00
Philipp Maier
7c62fc5ec4 jenkins: build docs in virtualenv as well 
The build system uses a virtual environment, in which it installs
pysim and its dependencies. This is done for the integration tests,
but not when building the sphinx documentation. However, the
documentation build process also invokes pysim code to generate
documentation from the docstrings. This means we need pysim with
all its dependencies for the doc building as well.

Change-Id: I6381eeef7fa19873ca0cc330a0ab43b7ef5096e4
Related: SYS#7094
 2024-10-11 15:59:58 +02:00
Philipp Maier
7429bc0ca0 tests: sanitize all cards before running tests 
Even though our tests are written in a way that they shouldn't interfere
with each other, it may happen that one testrun writes content to a file
that upsets a different testrun. The resulting problems are often
difficult to diagnose.

To minimize the problem, let's add code that can reset the cards to a
defined state. This can be done using pySim-shell's export
feature. We can generate a backup from a known good state and then play
back the backup to reset files that have been changed. Files that didn't
change will not be written thanks to the conserve_write feature of
pySim-shell.

Related: OS#4384
Change-Id: I42eaf61280968518164f2280245136fd30a603ce
 2024-10-05 08:44:20 +00:00
Philipp Maier
93c89856c8 utils: move enc_msisdn and dec_msisdn to legacy/utils.py 
We now have a construct based encoder/decoder for the record content
of EF.MSISDN. This means that we do not need the functions enc_msisdn
and dec_msisdn in the non-legacy code anymore. We can now move both
to legacy/utils.py.

Related: OS#5714
Change-Id: I19ec8ba14551ec282fc0cc12ae2f6d528bdfc527
 2024-09-27 18:17:19 +02:00
Philipp Maier
1f45799188 ts_102_221: se _test_de_encode instead of _test_decode in EF.DIR unittest 
The unittest for EF.DIR only runs with _test_decode, but it also runs with
_test_de_encode without any problems

Related: OS#5714
Change-Id: If459073c6ff927c1cc1790d506e3979243b1fb4c
 2024-09-27 18:17:19 +02:00
Philipp Maier
10ea4a0714 ts_51_011: use _test_de_encode instead of _test_decode in EF.CFIS unittest 
The unittest for EF.CFIS only runs with _test_decode, but it also runs with
_test_de_encode without any problems

Related: OS#5714
Change-Id: Ib876fd799f871fe64ced2a7b64847ffd09e16ed9
 2024-09-27 18:17:19 +02:00
Philipp Maier
dc2ca5d6be ts_51_011: fix unittest for EF.ADN 
The unittest for EF.ADN can run with _test_de_encode. However, the original
test vectors seem to be from a card with a slightly larger record size, so
they need a bit of re-alignment

Related: OS#5714
Change-Id: I241792e66ee6167be6ddc076453344b6307d6265
 2024-09-27 18:17:19 +02:00
Philipp Maier
39552464d8 ts_51_011: replace encoding of EF.MSISDN with construct model 
The encoding of EF.MSISDN is currently done with enc_msisdn and
dec_msisdn from utils.py. Let's replace this with a construct
based model, similar to the one we already use with EF.ADN

Related: OS#5714
Change-Id: I647f5c63f7f87902a86c0c5d8e92fdc7f4350a5a
 2024-09-27 18:17:19 +02:00
Philipp Maier
4045146f62 cosmetic: use **kwargs instead of **_kwargs 
Some methods sometimes have a **_kwargs parameter, let's be consistent
and use **kwargs only.

Related: OS#5714
Change-Id: I98857cc774185e55a604eb4fbfbf62ed4bd6ded7
 2024-09-27 18:17:19 +02:00
Philipp Maier
efddffe015 filesystem: pass total_len to construct of when encoding file contents 
In our construct models we frequently use a context parameter "total_len",
we also pass this parameter to construct when we decode files, but we
do not pass it when we generate files. This is a problem, because when
total_len is used in the construct model, this parameter must be known
also when decoding the file.

Let's make sure that the total_len is properly determined and and passed
to construct (via pyosmocom).

Related: OS#5714
Change-Id: I1b7a51594fbc5d9fe01132c39354a2fa88d53f9b
 2024-09-27 18:17:19 +02:00
Harald Welte
78c22a7d63 pySim-shell: New '-e' command line argument 
Using '-e' it is possible to specify *multiple* pySim-shell commands
which shall be executed at startup.  This extends the current ability
to execute just a single command.

Example:
 ./pySim-shell.py -p0 -e 'select ADF.USIM/EF.IMSI' -e 'read_binary_decoded'

Change-Id: I74004f46105553f077c039ca0f86f75afccc7342
 2024-09-23 16:13:16 +00:00
Philipp Maier
d96d04718e pySim-shell_test: disable test_list_and_rm_notif 
The testcase euicc.test_list_and_rm_notif fails due to a problem
with the eUICC. The eUICC reports the following error when a
delete notification attempt is made:

"delete_notification_status": "undefinedError"

Let's temporarily disable this testcase until the problem is resolved.

Change-Id: I7d0b6a998499d84f0eb4e24592ad43210ac54806
 2024-09-23 13:52:51 +02:00
Oliver Smith
7b95fac022 contrib/jenkins: add SKIP_CLEAN_WORKSPACE 
In order to run this script from pyosmocom's contrib/jenkins.sh script,
we want to skip the clean workspace step. Add an environment variable to
do that.

Related: OS#6570
Change-Id: Ic8dc9b85da17719195f7374d37eccb4dedba6ce8
 2024-09-23 08:38:55 +02:00
Oliver Smith
c09d4cc6b8 gitignore: add files generated with jenkins.sh 
Change-Id: Iaffe04a3dfebd46efc479dc3665fd67f2c95f375
 2024-09-23 08:38:55 +02:00
Philipp Maier
f87a00c04f Add testsuite for pySim-shell with real cards 
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.

Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
 2024-09-20 17:53:27 +02:00
Philipp Maier
d7032955c5 pySim-prog_test: add test vectors for sysmoISIM-SJA5 
The sysmoISIM-SJA5 has no testvectors yet

Change-Id: Ia6684ab3ee6c85cfe7bc0ab80d34a26e3499907a
 2024-09-20 17:18:50 +02:00
Philipp Maier
26ee39bebf pySim-shell: recognize ADP pins longer than 8 digits as hexadecimal 
When a hexadecimal formatted ADM pin is retrieved via the
card_key_provider, it still requires the --pin-is-hex parameter so
that sanitize_pin_adm knows the correct format.

This unfortunately ruins the card_key_provider feature for all cards
that use hexadecimal pins, because the --pin-is-hex would also be
required in scripts, which makes a script either useable for cards
with hexadecimal ADM or for for cards with ASCII ADM.

To minimize the problem, let's recognize all ADM pins longer than 8
digits as hexadecimal in case --pin-is-hex is not set.

Related: OS#4348
Change-Id: Iad9398365d448946c499ce89e3cfb2c3af5d525e
 2024-09-20 15:46:50 +02:00
Philipp Maier
01a96cd8e4 pySim-prog_test: individual ICCIDs for all cards 
Our test cards need to stay recognizable, so it is important that
each card has a unique ICCID. This means we must write an individual
ICCID, when we test writing the ICCID.

Related: OS#4384
Change-Id: I858a35e526e7b4868e901222d587258412779f41
 2024-09-20 12:15:34 +00:00
Philipp Maier
dca641aaa2 pySim-prog_test: do not set an ICCID parameter for sysmoISIM-SJA2 
The sysmoISIM-SJA2 does not support changing of the ICCID.
pySim-prog will also reject this, so let's remove the ICCID
from the parameter list.

Related: OS#4384
Change-Id: I89571f2bf7c4cec4d621c322a58687b7781b0ed2
 2024-09-20 12:15:34 +00:00
Philipp Maier
154e29c89a requirements: require at least construct version 2.10.70 
Older construct versions seem to have problems, in particular with
evaluating COptional() correctly. With 2.10.70 no such problems
were observed.

Related: OS#5714
Change-Id: If59dc708a7194649d1f42c4cf33f6328edcb80d2
 2024-09-19 18:09:33 +00:00
Philipp Maier
d5ddd04f33 pySim-shell: improve command "desc" 
The "desc" command displays a string with a file description, let's also
display some size information as part of the description as well.

Related: OS#5714
Change-Id: I98e139ba2bf35df5524245cdd96f5c52cf09b986
 2024-09-18 10:41:34 +02:00
Philipp Maier
6942a40909 filesystem, cosmetic: remove excess whitespace 
Change-Id: I902670590ae75a5d197616ae37d8268a60125121
 2024-09-18 10:41:34 +02:00
Philipp Maier
9a6425b6f2 runtime: add new API functions to get the record len and file size 
We have an API function to get the number of records, let's now also
add API functions to get the record length and the overall size of
the currently selected file.

Related: OS#5714
Change-Id: Ica7811c04161d8098b40c7219ed6b939df716cfd
 2024-09-17 17:59:46 +00:00
Philipp Maier
94ecf9a929 pySim-prog: rework documentation 
The documentation for the classic pySim-prog application is a bit
sparse. Let's rework it so that it includes the most important
information that is required to operate pySim-prog. Let's also add
a section about how the batch mode and CSV files are used.

Related: SYS#4120
Change-Id: I1d1a65154cea7fa77428b412fcf8c7b4cba629b1
 2024-09-17 15:47:30 +00:00
Philipp Maier
3eb74829df pySim-prog: fix commandline parameter check for CSV mode 
The CSV mode needs one of the four additional parameters: --imsi
--iccid, --read-iccid or --read-imsi. Also this check is unrelated
to the batch mode. The CSV file parameter reading works independently
from the batch mode.

Related: SYS#4120
Change-Id: I1292afb85122ed2b7944d02ede69c928a453866f
 2024-09-17 15:47:30 +00:00
Philipp Maier
3dc0496913 pySim-prog: treat --imsi and --iccid equally 
When using a CSV file, we can either read the IMSI or ICCID from the
CSV file before programming. However, should also be possible to
supply both manually to identify the CSV file entry using the --imsi
or --iccid option. This currently only works for the IMSI, but not
for the ICCID.

Related: SYS#4120
Change-Id: Id3083c7794a7bd59501997f22afdc23bad3069e6
 2024-09-17 15:47:30 +00:00
Philipp Maier
39e4a4b7c5 pySim-prog: add FIXME note to tell that writing hlr.db files is broken 
The writing to osmo-hlr SQLITE files is broken since the SQLITE format
has evolved over time. Let's add a FIXME note to tell that this needs
fixing.

Related: SYS#4120
Change-Id: I2b23f8bb9f3c2adeb48b010834057f5b4fb1e626
 2024-09-17 15:47:30 +00:00
Harald Welte
87e1ba6c18 update pyosmocom dependency to 0.0.3 
0.0.3 fixes an important problem related to enabling callers of build_construct()
to pass in a total_len value in order to specify the target output size.

Change-Id: I01687bb54e65bf5cc318745df588c3d6ea14eb83
 2024-09-17 17:25:43 +02:00
Harald Welte
ad3d73e734 docs: Bring osmo-smdpp documentation up to date with code 
Change-Id: Ibaab1fadd5d35ecdb356bed1820074b1b0a1752e
Closes: OS#6418
 2024-09-17 15:22:45 +00:00
Harald Welte
8e42a12048 docs: remove traces of modules migrated to pyosmocom 
Change-Id: I2ebb17f9781c90a81e9e554bddd7a851ef51c82a
 2024-09-17 15:22:45 +00:00
Harald Welte
84857accf3 pySim-shell: Detect different eUICC types and print during start-up 
Change-Id: I54ea4ce663693f3951040dcc8a16bf532bf99c02
 2024-09-17 15:22:45 +00:00
Harald Welte
72186cce84 pySim.profile: Further refactor card <-> profile matching 
The new architecture avoids sim/ruim/uicc specific methods in
pySim.profile and instead moves the profile-specific code into the
profile; it also solves everything within the class hierarchy, no need
for global methods.

Change-Id: I3b6c44d2f5cce2513c3ec8a3ce939a242f3e4901
 2024-09-17 15:22:45 +00:00
Harald Welte
5f2dfc28ff pySim/profile: Change match_with_profile from static to class method 
This was suggested by vyanitskiy during gerrit patch review in
https://gerrit.osmocom.org/c/pysim/+/38049 in order to make the
upcoming eUICC CardProfiles simpler.

Change-Id: Ia7c049b31cb1c5c5bb682406d9dd7a73bcd43185
 2024-09-17 15:22:45 +00:00
Philipp Maier
bd762c77ae pySim-prog: fix sourcecode formatting 
Change-Id: Ie4d4ec6d1752013fa8aa39912aa600c2b4afac74
 2024-09-16 12:20:43 +02:00
Philipp Maier
492379e61a pySim-prog: fix sourcecode formatting 
Change-Id: I0e567a1a681891f33f170c5df50c691b20b6a978
 2024-09-16 12:20:19 +02:00
Philipp Maier
7633a11239 pySim-shell: print cardinfo hexstrings in lowercase 
To ensure consistency, let's print the cardinfo hexstrings in lowercase
only.

Related: OS#6531
Change-Id: Ia6a8bd0e700c7fd933fb6c1b1050ed9494462d60
 2024-09-11 14:37:24 +02:00
Harald Welte
07b67439f8 pySim.euicc: Add 'get_data sgp02_eid' in ADF.ECASD of M2M eUICC 
The M2M eUICC are completely different from the consumer/IoT eUICC.

Obtaining the EID works via GET DATA in the ECASD.  Let's add support
for that.

Change-Id: I6cca6f75d268229244c90b3f1f88e26c89a2b4e0
 2024-09-10 20:40:16 +02:00
Harald Welte
c3fe111c0e pySim.commands: use _checksw during get_data() method 
All other methods use send_apdu_checksw, just get_data()
was missing the _checksw part.

Change-Id: Ic784bf0c30b22e5e83843aa6694e2706b4b2ac48
 2024-09-10 20:40:16 +02:00
Harald Welte
2fe9b6a3e9 pySim.transport: Also trace card reset events in ApduTracer 
Change-Id: Ia46b65124520eb2b8015dfa3f0a135b497668b92
 2024-09-10 20:37:56 +02:00
Harald Welte
241d65db12 pySim.transport: Add support for generic stdout apdu tracer 
Any program using argparse_add_reader_args() will get a new
long-opt '--apdu-trace' which enables a raw APDU trace to the console.

Change-Id: I4bc3d2e023ba360f07f024d7b661a93322f87530
 2024-09-07 14:43:58 +02:00
Harald Welte
bf0689a48e pySim.app: Properly reset card state after reading EID 
The code had two problems:

* the RESET was only performed in the successful case, but not if
  some exceptio was raised

* the RESET was a low-level reset bypassing the RuntimeState,
  so the lchan.selected_file was stale afterwards

Fixes: Change-Id Idc2ea1d9263f39b3dff403e1535a5e6c4e88b26f

Change-Id: Ib23d3d5b58b456a25157a622c1010c81cd8b2213
 2024-09-07 14:43:58 +02:00
Harald Welte
726097e51f transport: define TERMINAL RESPONSE content within ProactiveHandler 
So far the core proactive handling code would always generate a positive
response, with no way for the ProactiveHandler call-back to influence
that or to include additional IEs/TLVs.

Let's change that.

Change-Id: Ic772b3383533f845689ac97ad03fcf67cf59c208
 2024-09-05 11:30:53 +00:00
Harald Welte
ee9ac2f7ff suci-tutorial: fix typo s/symo/sysmo/ 
Change-Id: I0d3bdcf590e8dfef6deabc9967fd2f04152e1020
 2024-09-04 09:54:15 +02:00
Philipp Maier
398fdd7e8c pySim-shell: use upper case letters for positional arguments 
When we define positional arguments for the argument parser, we usually
use upper case letters only. However, there are some code locations that
use lower case letters. Let's translate those to capital letters to
have a consistent appeariance.

Related: OS#6531
Change-Id: Iec1ff8262bc6e9cf87c3cbf7b32fa5f753b7e574
 2024-09-04 09:37:28 +02:00
Philipp Maier
639806cc5a pySim-shell: do not display 'AIDs:' when there are none 
The command cardinfo also displays the AIDs of the card applications.
However, on classic GSM sim cards there are no applications. In this
case cardinfo will still display the string 'AIDs:', but it will of
course not list any AIDs under this string.

Related: OS#6531
Change-Id: Ifb111ce43fdebe85d30857dfc61ab570380b68d1
 2024-09-04 09:36:44 +02:00
Philipp Maier
471162dc76 suci-tutorial: add section about SUCI calculation by the USIM 
The tutorial describes how SUCI calculation in the UE is configured,
let's now add a section about SUCI calculation by the USIM.

Related: OS#6531
Change-Id: I45d47f9278b30d99ebde6891de0ba8cc74b1a0a0
 2024-09-04 09:36:43 +02:00
Philipp Maier
f81331808f pySim-shell: rework startup procedure and introduce non interactive mode 
When pySim-shell is used in a scripted environment, we may easily get trapped in
the pySim-shell prompt. This may happen in particular in case the script file
is not executed due to problem with the reader initialization. In such a case
pySim-shell will not exit automatically and the shellscript that was calling
pySim-shell will stall indefinetly.

To make the use of pySim-shell more reliable in scripted environments, let's
add a --noprompt option that ensures the interactive mode is never entered.
Let's also exit with an appropriate return code in case of initialization
errors, so that the calling script can know that something went wrong.

Related: OS#6531
Change-Id: I07ecb27b37e2573629981a0d032cc95cd156be7e
 2024-09-04 07:15:14 +00:00
Philipp Maier
bd7c21257c commands: avoid double lchan patching, get rid of cla_byte getter+setter methods 
The SimCardCommands has a cla_byte @property method, which automatically
returns the lchan patched CLA byte. We use cla_byte property to build
the UICC command APDUs inside SimCardCommands and then we hand the APDU
over to the send_apdu* methods. The cla_byte @property method as well
as the send_apdu* methods perform the lchan patching. This means the CLA
byte gets patched twice, which is technically not an issue, but can be
confusing when trying to understand the code.

To fix this, let's remove the @property methods and turn cla_byte into
a normal property again. This is also more accurate since the cla_byte
property originally was introduced to switch between UICC and classic
SIM APDU commands, which have almost identcal APDUs.

Related: OS#6531
Change-Id: I420f8a5f7ff8d9e5ef94d6519fb3716d6c7caf64
 2024-09-03 21:17:28 +00:00
Harald Welte
6aabb92c38 esim.saip.templates: Fix expand_default_value_pattern for length==0 
The original code treated length==0 like length==None (unspecified),
which is wrong.

Change-Id: I39fa1e2b1b9d6d1c671ea37bdbec1d6f97e8a5e7
 2024-09-03 21:57:47 +02:00
Harald Welte
b22bab0b20 pySim.esim.saip.ProfileElementGFM: Initialize 'fileManagementCMD' 
When constructing a ProfileElmentGFM from scratch, initialize the
decoded['fileManagementCMD'], as it is a mandatory member during
ASN.1 encode.

Change-Id: Iaae99348d36b7f0c739daf039d6ea2305b7ca9db
 2024-09-03 21:57:47 +02:00
Harald Welte
981220641d pySim.esim.saip.File: Turn file_size into a computed property 
This way, we can use file_size for both record-oriented and transparent EF

Change-Id: Ib787cabe969202073a8c10042e200f3d2c29db73
 2024-09-03 21:57:47 +02:00
Harald Welte
73dd3d0637 pySim.esim.saip: Add missing initialization of File.df_name 
Change-Id: Iaf596a8914850ccae584c3b78dc7711db736ac80
 2024-09-03 21:57:47 +02:00
Harald Welte
65cbe48953 pySim.esim.saip: Another naming irregularity. 
The choice member is called df-5gprose but the header is called
'df-5g-prose-header' (note the '-' between '5g' and 'prose'). WTF.

Change-Id: I86004ac2e18a187c26c5e470344908512d21fb9e
 2024-09-03 21:57:47 +02:00
Harald Welte
52735f3685 pySim.esim.saip: Fix weird DF names 
Sometimes the struct member is called like df-telecom, but in other
cases it's called df-df-saip  with a double 'df' in front.  That makes
no sense, but we have to deal with it from our constructors...

Change-Id: If5e670441f03a47fa34e97a326909b24927c12f7
 2024-09-03 21:57:47 +02:00
Harald Welte
9036d6d3fb remove pySim.gsmtap as it has moved to osmopython.gsmtap 
Change-Id: I631bb85bc6e76b089004d9f2e2082d70cbccf200
 2024-09-03 21:57:47 +02:00
Harald Welte
a3962b2076 Migrate over to using pyosmocom 
We're creating a 'pyosmocom' pypi module which contains a number of core
Osmocom libraries / interfaces that are not specific to SIM card stuff
contained here.

The main modules moved in this initial step are pySim.tlv, pySim.utils
and pySim.construct. utils is split, not all of the contents is
unrelated to SIM Cards.  The other two are moved completely.

Change-Id: I4b63e45bcb0c9ba2424dacf85e0222aee735f411
 2024-09-03 21:57:47 +02:00
Harald Welte
a437d11135 contrib/jenkins.sh: Install dependencies before calling pylint 
This is the only way we can make sure pylint has all required
information about imports from packages we depend upon.

Change-Id: I29582aa3d7f9ace9ce832d5b907420aaf14881fb
 2024-09-03 21:56:19 +02:00
Philipp Maier
aa182e9815 pySim-prog_test: supress stderr when probing for cards 
When probing for cards, the probing might fail in case the terminal
is empty. This results into lengthy error log output that is not
of interest.

Related: OS#6532
Change-Id: I1d44f9458a05992d79b0152d3affcfeb783cccff
 2024-09-03 12:38:30 +02:00
Philipp Maier
4d1f4fde4f pySim-prog_test: tolerate missing .data files 
When the test detects a card, but does not find the .data faile for
it, then it fails. This can be a problem in case we want to intentionally
exclude a specific card model.

Related: OS#6532
Change-Id: Iba196ada0076385de7bffcb157a85fda0a6c1852
 2024-09-03 12:38:30 +02:00
Philipp Maier
33256ddfed pySim-prog_test: tolerate empty reader slots 
The test currently expects all reader slots to be populated. This means
the cards may plugged into a random order, but there may not be any empty
slots in the system at all. This requirement is easy to meet when each
card has its own single-slot USB PCSC-reader, but as soon as multislot
readers are used there may be some empty slots.

Related: OS#6532
Change-Id: I7ba1d1350b6998d65e90408184accdb212556a7c
 2024-09-03 12:38:20 +02:00
Philipp Maier
f0034e4fe8 suci-tutorial: fix spec reference 
Related: OS#6531
Change-Id: If98c0b1093c7d19ea0278758c635b8405b465a2e
 2024-09-02 17:23:08 +02:00
Philipp Maier
df08441472 suci-tutorial: put download links for specs to the front 
The section Technical References has direct download links for the relevant specs.
Then later in th Key Provisioning section another download link follows and another
one is redundant. Let's put all download links into the Technical References section
and then only use the spec numbers in the following. This way we have all download
links in one location.

Related: OS#6531
Change-Id: Ibcbc6bb5d836d32c381922a35afa3b73b5f90621
 2024-09-02 17:22:08 +02:00
Philipp Maier
4d99c2b204 tests: move pySim-prog test and its data into a sub directory 
We currently have the shell script that performs the test in the
tests directory and the related data in pysim-testdata directory.
This is confusing, let's have evrything in a dedicated sub directory

Change-Id: Ic995a7f600d164fc0be3c2eb8255dbe043429bea
Related: OS#6531
 2024-09-02 17:07:21 +02:00
Philipp Maier
eb4ca1189c tests: move pySim-trace test and its data into a sub directory 
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.

Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
 2024-09-02 17:07:21 +02:00
Harald Welte
8ac2647004 contrib: script to generate "update" commands from diff of fsdumps 
Change-Id: I08897cd353093575f98c68580afbc68b6f2f878f
 2024-08-31 13:12:05 +00:00
Philipp Maier
e0241037e7 tests: move unittests into a sub directory 
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.

Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
 2024-08-30 05:25:20 +00:00
Philipp Maier
8680698f97 suci-tutorial: fix incorrect hnet_pubkey value 
The first hnet_pubkey value with the identifier 27 seems to be incorrect.
It differs from the value suggested in 3GPP TS 31.121, section 4.9.4 and
also does not work with the on card SUCI calculation.

The tutorial also contains a reference to 3GPP TS 33.501, Annex C.4. This
spec specifies an ECIES Profile A and an ECIES Profile B. The tutorial
recommends to use a key from profile B, but it actually uses a key from
profile A.

Related: OS#6531
Change-Id: I6fddf8a6efc28ad0d40b1715973429904e00d2b2
 2024-08-30 05:24:18 +00:00
Philipp Maier
a90bf12ea1 ts_31_102: Add mssing help string for get_identity parameter --nswo-context 
Related: OS#6531
Change-Id: I3ebd3a2ceb7f2580f4cd939b3f002f38f236d7f2
 2024-08-30 05:15:50 +00:00
Philipp Maier
c595221bc3 scp: fix key length in dek_encrypt and dek_decrypt 
When creating the DES cipher object with DES.new, we use the property
card_keys.dek. This property may hold a 16 byte key, but DES uses
an 8 byte key (56 bit + 8 bit integrity). Pycryptodome does not
automatically ignore excess key bytes. Instead it throws an
exception. This means we need to make sure to supply only the first
8 bytes of card_keys.dek

See also: https://pycryptodome.readthedocs.io/en/latest/src/cipher/des.html

Related: OS#6531
Change-Id: I92e0dc6a6196b532bd8b53fca7b9e78070d6903f
 2024-08-30 05:05:38 +00:00
Philipp Maier
d8637f3a70 commands: get rid of cla4lchan 
The send_apdu* methods now support lchan patching, so there is no longer
a need for computing the class byte manually (which is prone get forgotten)
before calling a send_apdu*. It is now enough to supply an APDU that has
a class byte with the default channel selected. This also means we do not
need cla4lchan anymore, so let's restruture the code and get rid of it
completely.

Related: OS#6531
Change-Id: Ia795f3c16a8875484fce3b44e61497d5aa52b447
 2024-08-28 12:53:14 +02:00
Philipp Maier
caabee4ccb ara_m: use class byte of current lchan 
The ara_m commands use APDUs with a fix class byte (0x80). This means
that all ARA-M related features only work in the basic logical channel.
To fix this, let's compute the class byte for the current logical channel
dynamically inside the send_apdu methods of SimCardCommands. This will
fix the problem globally.

Related: OS#6531
Change-Id: Ie3e48678f178a488bfaea6cc2b9a3e18145a8d10
 2024-08-28 12:53:14 +02:00
Philipp Maier
cc4c021bb1 global_platform: use scp_key_identity ICCID for ADF.ISD 
Related: OS#6531
Change-Id: I73a6f7088321a2b703074aa5228910709050cab2
 2024-08-28 12:53:14 +02:00
Philipp Maier
1034a9749f global_platform: fix help description for establish_scp03 
The argument parser object for establish_scp03 (est_scp03_parser) is
copied from est_scp02_parser. This object still has the .description
property set, which is the description for establish_scp02. To get
the description string that is defined in do_establish_scp03, we must
remove the old description string first.

Related: OS#6531
Change-Id: Ibb26bddf88b2e644a7f0c6b2a06bde228aa8afc7
 2024-08-28 12:52:24 +02:00
Harald Welte
f807983a98 pySim.esim.saip: Add missing entry for 'rfm' to class4petype 
Change-Id: I5fec2b026fc6a1197fc1e18d880ea6d10fd4a611
 2024-08-27 14:23:40 +00:00
Philipp Maier
8c1a1c5cc5 pySim-shell: prevent opening/closing logical channel 0 
The basic logical channel 0 is always present. It cannot be created or
closed. Let's restrict the value range of chan_nr, so that only valid
lchan numbers can be passed.

Related: OS#6531
Change-Id: I4eebd9f15fadd18e1caeb033fda36c59446fcab8
 2024-08-26 16:58:10 +02:00
Philipp Maier
d5943934a5 pySim-shell, cosmetic: define positional arguments last 
When we define command arguments using the ArgumentParser, we sometimes
define the positional arguments first. However, since positional arguments
usually follow after the optional (--xyz) arguments, we should define the
positional arguments last.

Related: OS#6531
Change-Id: I2412eb6e7dc32ae95a575f31d4489ce210d85ea0
 2024-08-26 16:58:10 +02:00
Philipp Maier
edf266726d filesystem: add command to delete all contents from a BER-TLV EF 
When working with BER-TLF files, we can only delete one tag at a time.
There is no way to delete all tags at once. This may make working with
BER-TLV files difficult, in particular when scripting is used and the
script needs to start with an empty file. Also export has problems,
since it does not reset the file before setting the new values there
may be unexpected results in case there still tags in the file that
are not set during import. To fill the gap, let's add a commandd that
deletes all tags in a BER-TLV EF at once.

Related: OS#6531
Change-Id: I5d6bcfe865df7cb8fa6dd0052cab3b364d929f94
 2024-08-26 16:58:10 +02:00
Philipp Maier
d20be98ed1 pySim-shell: fix sourcecode formatting 
Change-Id: I7133e93366eaacca5ace301172a08ae84e211c0e
 2024-08-26 16:58:10 +02:00
Philipp Maier
585e16a923 filesystem: fix double space in docstring 
Change-Id: I69ef171ac2dd2e2717404b1f3b10f986af419f6e
 2024-08-23 13:17:27 +02:00
Philipp Maier
1f92031079 pySim-shell: fix CardKeyProvider for chv management commands 
The CardKeyProvider support for the commands enable_chv, disable_chv,
verify_chv, change_chv and unblock_chv is broken. The reason for this
is the annotation "type=is_decimal" in the argument parser. This annotation
prevents the usage of string placeholders ("PIN1", "PUK1", etc).

Let's fix this by finding a better solution. We can also replace any
missing PIN/PUK code by checking if it is supplied or not. If not,
we query the CardKeyProvider. This also makes the usage of the *_chv
commands more uniform with the verify_adm command.

Related: OS#6531
Change-Id: I565b56ac608e801c67ca53d337bdec9efa3f3817
 2024-08-23 06:51:37 +00:00
Philipp Maier
89dbdbdccc runtime: fix get_file_by_name 
The method get_file_by_name compares the selectable directly with the
given file name. This is not correct. The comparison should be with the
path element from the pathlist.

Related: OS#6092
Change-Id: Id2d0704678935d9b9e2f1aeb6eaccbff6fa9d429
 2024-08-23 06:51:37 +00:00
Harald Welte
a5e2a8dbfd contrib/saip-tool: Add 'tree' command to display filesystem tree of profile 
Change-Id: I5cda7ef814648543c63938ac6a4fb9dba79379ff
 2024-08-23 06:51:07 +00:00
Harald Welte
a86b1abc03 osmo-smdpp: Proper error handling in case ctxParams1 is missing member 
************* Module osmo-smdpp
osmo-smdpp.py:373:15: E0601: Using variable 'iccid_str' before assignment (used-before-assignment)

Change-Id: I52bef18cbcc9f5d14519ff1473532c8502d45908
 2024-08-23 06:51:07 +00:00
Harald Welte
6d4c566fd7 Fix pySim.esim.es2p.Param.timestamp._encode 
************* Module pySim.esim.es2p
pySim/esim/es2p.py:107:19: E1101: Class 'datetime' has no 'toisoformat' member (no-member)

Change-Id: Ib762792d595048bf6d7d6f5acbe2715f137ae5bb
 2024-08-23 06:51:07 +00:00
Harald Welte
c6f8457ff1 pySim.esim.saip: maintain a parsed fileystem hierarchy 
With this change, the ProfileElementSequence object will maintain a
representation of the filesystem hierarchy of the eSIM profile.  Every
file that is added by a ProfileElement will add a FsNode into that tree,
and each FsNode will point to the File object for the respective file.

This allows us to find files by their path, as well as add files by
path.

Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
 2024-08-23 06:51:07 +00:00
Vadim Yanitskiy
5e2b93eb55 jenkins: use osmo-clean-workspace.sh before and after build 
Related: osmo-ci.git I2409b2928b4d7ebbd6c005097d4ad7337307dd93
Change-Id: I5ebebfa27e4b0c7b2fb3aa60618a82c1bfdaa19a
Fixes: OS#6546
 2024-08-21 19:04:04 +00:00
Harald Welte
cd22b9aee3 pySim.esim.saip.File: move away from stream for file content 
Let's linearize the file content in a bytes member variable self.body.

Change-Id: I6cb23a3a644854abd3dfd3b50b586ce80da21353
 2024-08-18 19:38:44 +02:00
Harald Welte
39613da6a7 pySim.esim.saip: Fix key used in FsProfileElement.files2pe 
The self.files member is a dict.  Hence we should use those dict
keys when [re]building the decoded dict. The previous code ignored
it and re-constructed the key from File.pe_name - but that's not
always identical.

Change-Id: I0e6c97721fb1cfc6b5c21595d85bd374d485b573
 2024-08-18 19:38:44 +02:00
Harald Welte
ab3e04fdb1 pySim.esim.saip: Fix typo in ProfileElementAKA.set_mapping() method 
Change-Id: Icd1594c6c2a8536a4ab8d1fc698307f05f539bdb
 2024-08-18 19:38:44 +02:00
Harald Welte
3a95fa12f6 pySim.esim.saip: Add some more docstring comments 
Change-Id: I70cf2b4dff1952f581efa3b21211c542f43ce565
 2024-08-18 19:38:44 +02:00
Harald Welte
b349149a88 pySim.esim.saip: Back-reference from ProfileElement to ProfileElementSequence 
Store a back-reference to the PE-Sequence in the PE object; this is
neccessary for some upcoming patches, e.g. to determine the position in
the sequence, access the global filesystem hierarchy, etc.

Change-Id: I24b692e47e4dd0afb5a17b04d5e0251dded3d611
 2024-08-18 19:38:44 +02:00
Harald Welte
3b30994ff0 pySim.esim.saip: pass up **kwargs from ProfileElement sub-class constructors 
Change-Id: Ib2b7f6d7428d03e9a8c23af39a61f450096c12bc
 2024-08-18 19:38:44 +02:00
Harald Welte
6a1e5eb4ee pySim.esim.saip: Move AKA specific post_dec + pre_enc to AKA subclass 
Having AKA specific code in the generic ProfileElement base class dated
back to when we didn't have a ProfileElementAKA subclass.

Change-Id: Icd332183758b8ef20a77507b728f5e455698def0
 2024-08-18 19:38:44 +02:00
Harald Welte
31c3c9a1e3 pySim.esim.saip: Refactor file size encoding into a method 
Change-Id: I46b8cb81ef8cc1794c11b61e0adfb575f937b349
 2024-08-18 19:38:44 +02:00
Harald Welte
6d495fb24d pySim.esim.saip: Improve File.from_template feature support 
When populating a File from a FileTemplate, let's make sure we
* correctly treat the maximum file size for BER-TLV files
* respect the default value pattern / repeat pattern
* respect the high_update flag.

Change-Id: I3ba092e0893f53a18264dff5fa37b12ccd9bd47e
 2024-08-18 19:38:44 +02:00
Harald Welte
01ddec2fdc contrib/saip-tool: Add command-line arguments to configure log level 
Change-Id: I4257d7b76193cdaad8c8571ff49f29067e8ab8c8
 2024-08-18 19:38:43 +02:00
Harald Welte
b2970d4bbe pySim.esim.saip.oid: Allow OID instance in prefix_match() 
So far the prefix_match() required a string argument; let's also
permit another OID object to be passed; we internally convert that
to string.

Change-Id: I0feb7782d1813cc46ec78f170eb0fce804aebe3a
 2024-08-16 18:06:12 +02:00
Harald Welte
1f477495ec saip-tool: Set default log level to INFO (instead of DEBUG) 
most users don't want to debug the program.

Change-Id: I54ae558cf8d87bf64cc75431cc4edcc694fa9084
 2024-08-16 17:46:41 +02:00
Harald Welte
97dfcaa9c7 pySim.filesystem: Permit Path object construction from FID integer list 
we so far supported construction of the Path object from a string or
a list of strings.  Let's also add the option of constructing it from a
path consisting of a list of integer FID values.

Change-Id: Ia7e9375b3258d1fbfdc892cefba3e3bbe841c550
 2024-08-16 17:46:41 +02:00
Harald Welte
022d562ae1 pySim.ts_102_221: Make sure FileDescriptor for BER-TLV contains file_type 
before this change, structure == 'ber_tlv' was missing the
file_type == working_ef attribute.  So for linear_fixed, transparent
and cyclic, the file_type attribute was present, but for ber_tlv it was
missing. This is illogical from a user point of vie and makes downstream code
potentially more complex, as it cannot match on working_ef for all EF
types.

Change-Id: If0076cc6dd35a818c08309885f6ef1c1704052c6
 2024-08-15 19:48:25 +02:00
Harald Welte
89dff98fb6 pySim.esim.saip.templates: Introduce dependency/hierarchy information 
The SAIP specification is very weird in a way that it treats the DF and
EF descriptions as some kind of flat structure without describing the
hierarchy.  So when creating a DF, sometimes it should be created below
the current DF, and sometimes it should be adjacent next to the current
DF.

Let's introduce
* a 'ppath' property of FileTemplate to indicate if a file is anything
  but a direct sibling of the 'base DF' of the PE
* an 'extends' property of ProfileTemplate to indicate that a given
  template does not have its own 'base DF', but that its contents merely
  extends that of another ProfileTemplate
* a 'parent' property of ProfileTemplate to indicate a parent
  ProfileTemplate below whose 'base DF' our files should be placed.

Change-Id: Ieab4835cd21008b289713784c0eb7170af2ccfb9
 2024-08-15 19:48:25 +02:00
Philipp Maier
526fdae6e5 pySim-shell: improve fsdump 
In the previous patch we have improved the export command. Since
the implementation of the fsdump command is very similar to the
implementation of the export command we can now apply the same
improvements to the fsdump command as well.

Change-Id: I4d2ef7b383025a5bbf122f18ecd51b7d73aaba14
Related: OS#6092
 2024-08-12 12:30:27 +02:00
Philipp Maier
c421645ba6 pySim-shell: improve export and enable exportation of DF and ADF files 
Since we now have the ability to provide export methods for all file
types in the file system (this also includes DF and ADF files), we need
to support this at shell command level as well. Let's also renovate the
walk method and the action method that does the actual exporting.

Related: OS#6092
Change-Id: I3ee661dbae5c11fec23911775f352ac13bc2c6e5
 2024-08-08 15:42:27 +02:00
Philipp Maier
12cc6821c4 runtime: add method to lookup a file by name without selecting it 
In some cases it might come in handy to be able to lookup a random file
in the file system tree before actually selecting it. This would be
very useful in situations where we need to check the presence of the
file or if we need to check certain file attributes before performing
some task.

Related: OS#6092
Change-Id: I6b6121e749cea843163659e1a26bb3893c032e29
 2024-08-08 15:37:35 +02:00
Philipp Maier
8597b64ee6 runtime: integrate escape route for applications without ADF support 
the select_parent method in RuntimeLchan currently implements a way
to escape from an application that has no filesystem support. However,
this escape route can be integrated directly into the select_file
method. This will give us the benefit that it will work transparently
in all code locations.

(This also means we can get rid of the select_parent method again)

Related: OS#6120
Change-Id: Ie6f37d13af880d24a9c7a8a95cef436b603587c7
 2024-08-08 15:37:35 +02:00
Philipp Maier
2d235f8143 filesystem: fix typo 
Change-Id: I17f184bbcf494c5fe944602224cf72d6a22cbc9d
 2024-08-08 15:37:35 +02:00
Philipp Maier
b92f4f52cc ara_m: add export support for the ARA-M application 
This patch adds an export method to the CardApplicationARAM class.
This method reads the ARA-M configuration and transforms it into
executeable command lines, which can be executed as a script later
to restore an ARA-M configuration.

Related: OS#6092
Change-Id: I811cb9d25cb8ee194b4ead5fb2cabf1fdc0c1c43
 2024-08-08 10:47:59 +02:00
Philipp Maier
03901cc9ce filesystem: add export method for ADF files 
This patch adds an export method to CardADF, which calls the application
specific export method in CardApplication class

Related: OS#6092
Change-Id: I8129656096ecaf41b36e5f2afbbfbebcd0587886
 2024-08-08 10:46:49 +02:00
Philipp Maier
b4530e71b7 filesystem: add placeholder export method in CardFile base class 
We add export methods in subclasses of CardFile but the base class
itself lacks an export method. To make the code more readable and
to avoid unnecessary exceptions, les's add a default export method
that just returns a comment.

Related: OS#6092
Change-Id: Ife2a9bad14750db84a87fab907297028c33f1f7d
 2024-08-08 10:46:39 +02:00
Harald Welte
7d9c6583ef pySim.cards: Make file_exists() check for activated/deactivated 
The Card.file_exists() method is only called by legacy pySim-{read,prog}
when it wants to determine if it can read/write a file.  Therefore
it actually doesn't only want to know if the file exists, but also
if it's not deactivated.

Change-Id: I73bd1ab3780e475c96a10cd5dbdd45b829c67335
Closes: OS#6530
 2024-08-07 14:10:21 +00:00
Philipp Maier
4515f1cf87 ara_m: fix --apdu-filter setting 
The code for the --apdu-filter commandline option is not yet finished.
Let's finish it and make it work.

Related: OS#6092
Change-Id: Ib5fb388972fde0d50c3db0082ebf40bcca404681
 2024-08-06 09:20:44 +02:00
Harald Welte
10e9e97724 pySim.esim.saip.templates: Add expand_default_value() method 
This method can be used to expand the default value pattern of the
file system template for the file to the specified (record, file) length.

Change-Id: Id3eb16910c0bdfa572294e14ca1cd44ca95ca69f
 2024-08-05 15:56:15 +00:00
Harald Welte
8f5fd37b4a pySim.esim.saip.templates: Fix '...' notation in default value 
The default value must contain '...' to indicate a variable-length
default value section, not '..'

Change-Id: I8d78278065c145b86460acf8eb723babe777c4f6
 2024-08-05 15:56:15 +00:00
Harald Welte
ca1b00f99e pySim.esim.saip.templates: Explicitly specifiy repeatable default value 
Change-Id: I9ae2c36f5bffac392c1219bb6ea21c1c05dff4b9
 2024-08-05 15:56:15 +00:00
Harald Welte
465d1a07e0 pySim.esim.saip.templates: Add SaipSpecVersion 
The SAIP specification version implicitly determines which filesystem
templates (or versions thereof) are supported.  So if a given eUICC
states it implements SAIP version 2.3.0, then we have to translate
this into which template versions that means.  The new SaipSpecVersion
and its derived classes do exactly that.

Change-Id: I3a894c72c22e42bd2067e067be80a67197ad1bf2
 2024-08-05 15:56:15 +00:00
Philipp Maier
44d51a7b16 pySim-shell: fix typo 
Change-Id: I1bd995ae9eb59a44a48da62a7b0262faa84a4f2b
 2024-08-05 15:20:15 +02:00
Harald Welte
5b513a543f pySim.esim.saip.oid: Fix OID defininitions for v3.3.1 IoT templates 
Change-Id: Iac620362ae9336199f3b3b168a4bfeda3e2b7c35
 2024-08-04 12:46:12 +02:00
Harald Welte
46bc37fa65 pySim.filesystem: Add __len__ method to Path object 
This returns the length of the path.

Change-Id: I5e3ba726ed180405c4218ebeee240a3a40527f99
 2024-08-04 12:46:12 +02:00
Harald Welte
19328e3bbd pySim.esim.saip.templates: Update to SAIP v3.3.1 (July 2023) 
This new TCA SAIP version introduces a number of aditional templates

Change-Id: Ie8aeae3f5b36a3141a70f670c220932389d241a6
 2024-08-04 12:46:12 +02:00
Harald Welte
d2254377b6 pySim.esim.saip.templates: Add a notion of the path of a file 
The SAIP data format is inherently flat and doesn't intrinsically
have an idea of the tree-like structure of a filesystem.  However,
if we want to (for example) convert a physical USIM into an eSIM
profile, we need to find the template for a given file, where the file
is identified by its path.

Let's expose a path property of the FileTemplate object, and populate
that when creating the FileTemplate as part of a ProfileTemplate.

Change-Id: Ie145ba159081daf8fbfa544f6d4248f05b7eea96
 2024-08-04 12:46:12 +02:00
Harald Welte
041a1b33fc pySim.esim.saip.template: Permit file-size for BER-TLV files 
We previously only permitted this for transparent files (TR), but
file size can of course also be specified for BER-TLV files.

Change-Id: Ie007cf2ccde0a17d0fb853a96b833f064ae52c59
 2024-08-04 12:46:12 +02:00
Harald Welte
d3a6bbc215 pySim.esim.saip: Add subcasses for EAP, DF.SNPN and DF.5G_ProSe 
Change-Id: I8f29e72d387c66c99ceccffc9de23a68fd15dc46
 2024-08-04 12:46:12 +02:00
Harald Welte
08d7c10211 pySim-shell: Support other ADMx values beyond ADM1 from 'verify_adm' 
Change-Id: Icce6903c1e449889f8bc5003ccfe6af767a26d44
 2024-08-04 12:46:02 +02:00
Harald Welte
fdae0ff90d pySim-shell: Support hexadecimal ADM pin in 'verify_adm' 
Change-Id: I4191ed79ebe7869d8411d280a32ac2d4bbc210e3
Closes: OS#6480
 2024-08-01 11:38:18 +02:00
Harald Welte
7c06bcdd57 Support EF.ICCID and EF.PL on classic TS 51.011 SIM 
So far we only had the EF.ICCID and EF.PL within our UICC card profile.
However, a classic GSM SIM card is not an UICC, so the CardProfileSIM
also needs those files.

To avoid circular dependencies, move the definitions from ts_102_221.py
to ts_51_011.py

Change-Id: I6eaa5b579f02c7d75f443ee2b2cc8ae0ba13f2fe
Closes: OS#6485
 2024-07-31 23:17:13 +02:00
Harald Welte
d81c2086c8 pySim.tlv: Fix from_dict of nested TLVs 
The existing logic is wrong.  How we call from_dict() doesn't differ if
a member IE itself contains a nested collection: We always must pass a
single-entry dict with the snak-case name of the class to from_dict().

Change-Id: Ic1f9db45db75b887227c2e20785198814cbab0f5
Fixes: OS#6453
 2024-07-31 23:17:13 +02:00
Harald Welte
d3fb38965b ara_m: Fix pySim.tlv.IE.from_dict() calls 
Historically, to_dict and from_dict were not symmetric; this has been
fixed in I07e4feb3800b420d8be7aae8911f828f1da9dab8 in December 2023.

This however broke the ara_m legacy use of the from_dict() methods.
We've just introduced a from_val_dict() method in
I81654ea54aed9e598943f41a26a57dcc3a7f10c2, let's make use of it.

Change-Id: I3aaec40eb665d6254be7b103444c04ff48aac36d
 2024-07-29 13:06:27 +02:00
Harald Welte
4fd3fa445c pySim.esim.saip: Add subclasses for gsm-access, phonebook, 5gs, saip 
Those are all optional ProfileElements related to the USIM NAA.

Change-Id: I621cc3d2440babdc11b4b038f16acf418bbc88ad
 2024-07-29 13:06:27 +02:00
Harald Welte
4f9ee0fa75 pySim.esim.saip: Refactor from_der() method to have class_for_petype() 
Change-Id: I2e70dddb0b3adb41781e4db76de60bff2ae4fdb7
 2024-07-29 13:06:27 +02:00
Harald Welte
6b1c6a986c pySim.esim.saip.templates: Build tree from template files 
Change-Id: I13e80e9dbddbb145411378a0d9e01461aef75db4
 2024-07-29 13:06:27 +02:00
Harald Welte
3d6a712e8c Fix missing AIDs in pySim.saip templates 
Change-Id: Ie02e2d27ece0fbd9719468c8d31febd1937468f8
 2024-07-29 13:06:27 +02:00
Harald Welte
8b1060a30e Reference pySim.filesystem derived classes from SAIP templates 
Change-Id: Ia1c810262f1cfa48dae192c7de620c7f0fb69c25
 2024-07-29 13:06:27 +02:00
Harald Welte
e354ef7d05 pySim.esim.saip: Initial support for parsing GenericFileManagement 
Change-Id: I4a92f5849158a59f6acca05121d38adc0a495906
 2024-07-29 13:06:16 +02:00
Harald Welte
e3e964589f pySim.ts_102_221: Add ProprietaryInformation sub-IEs of TS 102 222 
We put those in ts_102_221 because that's where ProprietaryInformation
is defined, and we don't want to risk circular dependencies.

Change-Id: I526acfeacee9e4f7118f280b3549fd04fdb74336
 2024-07-29 13:03:21 +02:00
Harald Welte
cf65d92039 pySim.ts_102_221: Fix FileDescriptor encoding for BER-TLV case 
This fixes a long-standing bug in the FileDescriptor IE class which so
far only supported decoding, but not encoding of BER-TLV file
descriptors.

Change-Id: I598b0e1709ee004bcf01a53beb91f68470e1f3da
 2024-07-29 13:03:21 +02:00
Harald Welte
f3b3ba15b8 pySim.filesystem: Add Path for abstraction/utility around file system paths 
Change-Id: I202baa378988431a318850e3593ff1929d94d268
 2024-07-29 13:01:51 +02:00
Harald Welte
bff8902ce1 pySim.commands: make use of status word interpreter for CHV 
Related: OS#6398
Change-Id: I71efe9d6804c4845bb81f1b3b443215dad0ac301
 2024-07-29 13:01:51 +02:00
Harald Welte
de5de0e9db pySim-shell: add "fsdump" command 
This command exports the entire filesystem state as one JSON document,
which can be useful for storing it in a noSQL database, or for doing a
structured diff between different such dumps.

It's similar to "export", but then reasonably different to rectify a
separate command.

Change-Id: Ib179f57bc04d394efe11003ba191dca6098192d3
 2024-07-29 10:48:22 +00:00
Harald Welte
d29f244aad pySim.tlv: Separate {to,from}_val_dict() from {to,from}_dict() 
There are some situations where we want to work with a type-name-wrapped
dict that includes the type information, and others where we don't want
that.  The main reason is that nested IEs can only be reconstructed if
we can determine the type/class of the nested IE from the dict data.

Let's explicitly offer {to,from}_val_dict() methods that work with
the value-part only

Related: OS#6453
Change-Id: I81654ea54aed9e598943f41a26a57dcc3a7f10c2
 2024-07-27 10:30:26 +02:00
Harald Welte
eda408fba3 pySim.commands: Don't convert SwMatchError to ValueError 
In the read and write command implementations, we used to catch
lower-layer exceptions (usually SwMatchError) and "translate" that into
a value error, only to add more information to the exception.  This
meant that higher-layer code could no longer detect this was actually
a SwMatchError exception type.

Let's instead use the add_note() method to amend the existing exception,
rather than raising a new one of different type.

Change-Id: Ic94d0fe60a8a5e15aade56ec418192ecf31ac5e7
 2024-07-27 10:30:26 +02:00
Harald Welte
2a963a7ac0 pySim.runtime: Be more verbose if incompatible method is called 
Change-Id: I57190d50a63e0c22a8c5921e1348fae31b23e3d4
 2024-07-27 10:30:26 +02:00
Harald Welte
75a109419c pySim.tlv: Add convenience methods to IE class 
The new methods allow programmatic resolution of nested IEs from
a parent, assuming there's only one child of a given type (which is
often but not always the case).

Change-Id: Ic95b74437647ae8d4bf3cdc481832afb622e3cf0
 2024-07-27 10:30:26 +02:00
Harald Welte
d25ea35e7e pySim.esim.saip: Decode each 'File' element in ProfileElement 
When loading a ProfileElement from its DER-ecoded format, populate
a dict with a pySim.esim.saip.File object for each file.

Change-Id: Ie2791c10289eb28daed2904467b0c5e5b11c94c2
 2024-07-27 10:30:26 +02:00
Harald Welte
6d2e385acf pySim.esim.saip: Add OID comparison functions 
Change-Id: Iab642e0cc6597f667ea126827ea888652f0f2689
 2024-07-27 10:30:26 +02:00
Philipp Maier
e931966a06 ara_m: fix misspelled object name 
Related: OS#6092
Change-Id: I2c2289658f099aa1d25a4ab3292dea9a7c16c123
 2024-07-27 08:22:57 +00:00
Philipp Maier
2c0e3358a7 ara_m: fix sourcecode formatting 
Related: OS#6092
Change-Id: I374eefdd1a2763552c98c1928753197e9f753e2b
 2024-07-27 08:22:57 +00:00
Philipp Maier
43fc875168 pySim-shell: fix comment formatting 
Related: OS#6092
Change-Id: Icea88c061436d26a3240fc666fcc3fe1bd36d2ba
 2024-07-27 08:22:57 +00:00
Philipp Maier
dff7bb0687 pySim-shell: clean up method calls in do_switch_channel 
The function do_switch_channel method calls methods in RuntimeLchan
that should be private. There is also a code duplication in
RuntimeLchan that should be cleaned up.

Related: OS#6092
Change-Id: Ie5e5f45787abaaf032e1b49f51d447653cf2c996
 2024-07-27 08:22:04 +00:00
Philipp Maier
4fefac78b8 pySim-shell: fix reset command 
The reset command resets the card using the card object. This unfortunately
leaves the RuntimeState uninformed about the event. However, the RuntimeState
class also has a reset method that resets the card and the RuntimeState. Let's
use this reset method. Also fix this method so that it ensures that the SCP is
also no longer present.

Related: OS#6092
Change-Id: I1ad29c9e7ce7d80bebc92fa173ed7a44ee4c2998
 2024-07-27 08:22:04 +00:00
Philipp Maier
7858f591fe pySim-shell: turn "ADF-escape-code" into an lchan method. 
When we traverse the file system using the command "export" we will
also select all ADFs but not all ADFs may have UICC file system support.
This makes it impossible to exit those ADFs again. To exit anyway we
select an application with filesystem support first and then the parent
EF we wanted to select originally. This method may not only be useful
when traversing the filesystem, so let's put it into the RuntimeLchan
class and change it a little so that it would also work if the ADF in
question is an a sub DF.

Related: OS#6092
Change-Id: I72de51bc7519fafbcc71d829719a8af35d774342
 2024-07-27 08:22:04 +00:00
Philipp Maier
d29bdbc2c8 pySim-shell: move export code into filesystem class model 
The code that generates the filesystem export lines for the various
different file structures can be moved into the filesystem class model.

This simplifies the code since we do not need any extra logic to
distinguish between the different file structures.

Related: OS#6092
Change-Id: Icc2ee60cfc4379411744ca1033d79a1ee9cff5a6
 2024-07-27 08:22:04 +00:00
Harald Welte
34dce409b9 pySim.global_platform.ota: Support KVN 0x70 for SCP02 
This is a non-standard extension of sysmocom products.

Change-Id: I00d52f7629aae190ee487ea3453f42b5f94cf42f
 2024-07-26 08:38:17 +02:00
Harald Welte
c60944a7de saip-tool: Fix TAR display for implicit TAR 
Until Change-Id Ifba1048e3000829d54769b0420f5134e2f9b04e1 the TAR
output was working for implicit tar.  With said commit we fixed it
for explicit tar but broke implicit tar.

With this commit it works for both implicit and explicit TAR.

Change-Id: I76133b0e02996a138257f3fba5ceb0d2fc6fad80
 2024-07-26 08:38:17 +02:00
Harald Welte
0c022944ff pySim.apdu.global_platform: Decode the INSTALL command parameters 
Change-Id: I1c323c1cb1be504c6ad5b7efb0fa85d87eaa8cf7
 2024-07-26 08:38:17 +02:00
Harald Welte
4f2a6ebf1f pySim.ota: Add construct definition for SIM File + TK Param definition 
Change-Id: Ie5aa2babaf66af49eb5223e5e9d4451089baf055
 2024-07-26 08:38:17 +02:00
Philipp Maier
f26042f92d pySim-shell: fix comment formatting 
Related: OS#6092
Change-Id: I101868a6f0220b62977c5e633df2607467cfba91
 2024-07-26 06:24:07 +00:00
Philipp Maier
9aeadea4c3 ts_31_103_shared: fix file structure of EF.WebRTCURI 
EF_WebRTCURI should inherit from LinFixedEF intead of TransparentEF.
(See also 3gpp TS 31.103, section 4.2.20)

Related: OS#6092
Change-Id: I903c483a8553fbe599fa7b5a2aefb28bc85b5078
 2024-07-26 06:24:07 +00:00
Philipp Maier
c78ea1ffa6 runtime: rename get_file_for_selectable to get_file_for_filename 
Let's rename get_file_for_selectable to get_file_for_filename so that it
is immediately clear what the method does.

Related: OS#6092
Change-Id: Ifed860814229857ad8b969e50849debbf5d8918f
 2024-07-26 06:24:07 +00:00
Philipp Maier
2cca36e8fd runtime: add missing docstring 
Change-Id: Iee2702c5326f1ec2a32c40b675ba1647387c40c8
Related: OS#6092
 2024-07-26 06:24:07 +00:00
Harald Welte
87b4f99a90 pySim.apdu: Get rid of HexAdapter 
In the past, we always wrapped a HexAdapter around bytes-like data in
order to make sure it's printed as hex-digits.  However, now that we are
doing JSON output it's much easier to let the pySim.utils.JsonEncoder
take care of this in a generic way.

We should do a similar migration all over pySim (pySim-shell,
filesystem, etc.) - but for now only do it in the low-hanging fruit of
pySim-trace aka pySim.apdu

Change-Id: I0cde40b2db08b4db9c10c1ece9ca6fdd42aa9154
 2024-07-26 06:20:46 +00:00
Harald Welte
c800f2a716 pySim-trace: display decoded result as JSON, not as python dict 
This means users can copy+paste or otherwise post-process the data in a
standard format.

Change-Id: I3135f2f52b8d61684a71b836915b43da5c48422b
 2024-07-24 10:37:05 +02:00
Harald Welte
699b49ef1b pySim.apdu.ts_102_222: APDU decoding for administrative commands 
Change-Id: I77c97221da19e1a67d96f7cfb69785baefc675c0
 2024-07-24 10:37:05 +02:00
Harald Welte
d93d774dcc pySim.apdu: Fix APDU CLA matching 
The cla values as hex strings must be compared in case insensitive manner

Change-Id: I890bc385d6209e6cfe9b0c38bd9deee7ae50e5f5
 2024-07-19 18:24:29 +02:00
Harald Welte
289d2343fa pySim.apdu: Refactor cmd_to_dict() method 
Let's factor out the "automatic processing using _tlv / _construct" as a
separate method.  This way we enable a derived class to first call that
automatic processing method, and then amend its output in a second step.

Change-Id: I1f066c0f1502020c88d99026c25bf2e283c3b4f5
 2024-07-19 18:23:36 +02:00
Harald Welte
03eae595a3 pySim.ts_31_102: Fix name of EF.VBSCA 
It's VGCS but VBS.  There's no VBCS.

Change-Id: I3c4a7ec9cd6a56fe7b85832afc68685f8dccbfd1
 2024-07-18 12:15:43 +02:00
Harald Welte
f174ad6885 ts_31_102: Make use of ts_31_103_shared and add Rel 18 files 
Change-Id: I68ca15084f9654468bd37526c02a66322085b25b
 2024-07-18 12:15:43 +02:00
Harald Welte
6f5a0498bf [cosmetic] ts_31_102: Note in comment which release introdcued recent files 
Change-Id: I0c1250b532992ae954b1d8ab20993cb9fa947695
 2024-07-17 18:37:15 +02:00
Harald Welte
fb56f35546 move parts of pySim.ts_31_103 to pySim.ts_31_103_shared 
This is requird to make some definitions available to USIM / ts_31_102
without introducing circular dependencies.

Change-Id: I32e29f400d2da047e821bf732316b21805b5a1e2
 2024-07-17 18:37:15 +02:00
Harald Welte
282aeadcc4 pySim.ts_31_103: update to spec v18.1.0 Release 18 
This adds two new EFs and one new IST service.

Change-Id: Iced1700046b459399a3e8305e1387ec65eeb3536
 2024-07-17 18:37:15 +02:00
Harald Welte
92bae20b49 osmo-smdpp + es9p_client: HTTP status 204 is used for handleNotification 
As SGP.22 states, the handleNotification endpoint uses HTTP status 204,
not 200 (due to its empty body).

Change-Id: I890bdbd3e1c4578d2d5f0367958fdce26e338cac
 2024-07-17 18:37:02 +02:00
Harald Welte
e18586ddf0 pySim.globalplatform: Add 'http' submodule for GP Amd B RAM over HTTPS 
This implements the first parts of the "GlobalPlatform Remote
Application Management over HTTP Card Specification v2.3 - Amendment B,
Versoin 1.2".  Specifically, this patch covers the TLV definitions for
the OTA message used for HTTPS session triggering.

This also adds some more unit test coverage to pySim.cat, based on
real-world data that was captured nested inside the HTTPS Administration
session triggering parameters.

Change-Id: Ia7d7bd6df41bdf1249011bad9a9a38b7669edc54
 2024-07-17 18:05:57 +02:00
Harald Welte
03194c0877 pySim.esim.es8p: Add support for encoding icon in ProfileMetadata 
Change-Id: I8c6a0c628f07c2a9608174457d20b8955114731a
 2024-07-17 18:05:57 +02:00
Harald Welte
84077f239f osmo-smdpp: Request enable/disable/delete notifications in metadata 
this way, the eUICC will send us notifications whenever our profiles are
enabled/disabled/deleted.

Change-Id: I2861290864522b691b30b079c7c2e1466904df2d
 2024-07-17 18:05:57 +02:00
Harald Welte
5370178ca2 osmo-smdpp: Implement 'other' notification signature validation 
"other" notifications (enable, disable, delete) contain ECDSA
signatures that also need verification.

Change-Id: If610058b7af6f9fc7822576c93f9970e2ce9aba9
 2024-07-17 18:05:57 +02:00
Harald Welte
3ad3da8995 contrib/es9p_client: Add support for reporting notifications to SM-DP+ 
The ES9+ interface is not only used for downloading eSIM profiles, but
it is also used to report back the installation result as well as
profile management operations like enable/disable/delete.

Change-Id: Iefba7fa0471b34eae30700ed43531a515af0eb93
 2024-07-17 18:05:57 +02:00
Harald Welte
9d0c2947f1 es9p_client: Move code into a class; do common steps in constructor 
This is in preparation of supporting more than just 'download'

Change-Id: I5a165efcb97d9264369a9c6571cd92022cbcdfb0
 2024-07-17 15:22:09 +02:00
Harald Welte
0519e2b7e1 osmo-smdpp: Make sure to return empty HTTP response in handleNotification 
SGP.22 is quite clear in that handleNotification shall return an empty
HTTP response body.  Let's make sure we comply to that and don't report
a JSON response.

Change-Id: I1cad539accbc3e7222bfd4780955b3b1ff694c5b
 2024-07-17 15:22:09 +02:00
Harald Welte
96e2a521e9 pySim.esim.http_json_api: 'header' is not always present in response 
For example, the ES9+ handleNotification function is defined with an
empty response body, so we cannot unconditionally assume that every HTTP
response will contain a JSON "header" value.

Change-Id: Ia3c5703b746c1eba91f85f8545f849a3f2d56e0b
 2024-07-16 16:58:55 +00:00
Harald Welte
23dd13542e saip-tool: Fix output of TAR values in "print" subcommand 
Change-Id: Ifba1048e3000829d54769b0420f5134e2f9b04e1
 2024-07-16 15:06:57 +00:00
Harald Welte
5fdfa1463e pySim.cat: More spec references + explanations in comments 
Change-Id: I4a89156075ae225594740451b33c3dec8983cf04
 2024-07-15 12:40:10 +02:00
Harald Welte
c805f00bff transport: Implement treatment of 62xx and 63xx warning/error responses 
TS 102 221 specifies that (in case of a class 4 command) and as SW
62xx or 63xx, we should send a GET RESPONSE just like in the 61xx
case in order to get the respective response.

As we don't really know if it's a case1/2/3/4 command in the
pySim.transport, let's always send the GET RESPONSE in case SW 62xx or
63xx are received.  It shouldn't hurt - in the worst case there's no
response available...

Change-Id: Ibb1398194a16fc1f1f9bc46af6c66fb6575240cd
 2024-07-13 23:09:02 +02:00
Harald Welte
12902730bf pySim.commands: Check return value of TERMINAL PROFILE command 
Change-Id: Iaede74caf22970869c2c85b42d1e6f70d52c65cb
 2024-07-13 23:07:22 +02:00
Harald Welte
0c40a2245b pySim.ota: Raise exception if encoded length would exceed 140 bytes 
SMS cannot exceed 140 bytes, and TS 31.115 explicitly states that larger
messages must use multi-part SMS, which we don't yet implement here.

Change-Id: I8a1543838be2add1c3cfdf7155676cf2b9827e6e
 2024-07-13 23:07:22 +02:00
Harald Welte
dacacd206d pySim.ota: Handle cases where 'secured_data' is empty 
while it's true that in situations where response_status == 'por_ok'
we are guaranteed to have a 'secured_data' key in the dict, its value
could well be b'', which in turn causes us to run into an exception,
calling a decoder on an empty byte value; let's avoid that.

Change-Id: I7c919f9987585d3b42347c54bd3082a54b8c2a0a
 2024-07-13 23:07:22 +02:00
Harald Welte
b865d383aa pySim.transport: Fix proactive_handler from_dict() calls 
Change-Id: I2aa19ef6a19085d77c1b4f2d434a01ee241bd9a8
 2024-07-13 23:04:20 +02:00
Harald Welte
1c2ec93164 pySim.tlv: Add COMPACT_TLV_IE TLV variant 
the COMPACT-TLV variant is a TLV variant that ISO7816 uses for encoding
tag and length into a single octet. This is used (for example) in ATR
historical bytes.

Let's add support for this to our pySim TLV encoder/decoder.

Change-Id: I9e98d150b97317ae0c6be2366bdaaeaeddf8031c
 2024-07-10 18:10:39 +02:00
Harald Welte
76b3488829 saip-tool: Also dump RFM information in "info" command 
example output:

Number of RFM instances: 2
RFM instanceAID: d276000005aa060200000000b00000 (-> TAR: b00000)
        MSL: 0x16
RFM instanceAID: d276000005aa060200000000b00001 (-> TAR: b00001)
        MSL: 0x16
        ADF AID: a0000000871002ff33ffff8901010100

Change-Id: I534267c7420fc5bd96eaded6078e986161729073
 2024-07-10 06:51:23 +00:00
Harald Welte
37320da4ab saip-tool: Dump information about security domains from "info" command 
output looks like this:

Number of security domains: 1
Security domain Instance AID: a000000151000000
        KVN=0x01, KID=0x01, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]
        KVN=0x01, KID=0x02, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]
        KVN=0x01, KID=0x03, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]

Change-Id: Ia25f5ca6d7e888f7032301dd2561d066a3870010
 2024-07-10 06:51:23 +00:00
Harald Welte
b5679386d7 pySim.esim.saip: Add methods to rebuild "mandatory" lists in ProfileHeader 
The ProfileHeader PE contain lists of template-oids and services that
are mandatory in this profile.  Let's add methods that can be used to
(re-) compute those lists based on the actual PE contents of the
sequence.

The idea is that during programmatic construction of a profile, those methods
would be called after appending all PEs, just before encoding the
profile as DER.

Change-Id: Ib43db8695c6eb63965756364fda7546d82df0beb
 2024-07-10 06:51:23 +00:00
Harald Welte
03aebf5b43 pySim.esim.saip: ProfileElement{Header,End} classes 
Change-Id: I88e18c1ee4907eeac3ae5d04d7bc30d6765f91fa
 2024-07-10 06:51:23 +00:00
Harald Welte
5f9b8a8fc1 pySim.esim.saip: Move initialization of PE header to base class 
Let's avoid the copy+paste in the subclass constructors and initialize the profile
element header in the base class constructor.

Change-Id: I6e69ae1f0d33d963247fc506db33b3840c10c19a
 2024-07-10 06:51:23 +00:00
Harald Welte
3b7e2ae2c1 pySim.saip: Add ProfileElementRFM class 
Change-Id: I547e02c12345932deafa4b914fcaeaa183b69798
 2024-07-10 06:51:23 +00:00
Harald Welte
2668eb6148 pySim.esim.saip: Add ProfileElementOpt{USIM,ISIM} classes 
Change-Id: Iebff2e767baa19f272eeddc62d7d5b3a8f665db5
 2024-07-10 06:51:23 +00:00
Harald Welte
3c530c3c1a pySim.saip.oid: Properly differentiate optional from non-optional templates 
There are e.g. templates for usim and for opt-usim, and they should not
be confused with each other.  Let's reflect that in the naming.

Change-Id: Ic6d04ce3172dc969c6b8c018b8d305eb6fd3f550
 2024-07-10 06:51:23 +00:00
Harald Welte
992e60902a tests: Add ProfileElementSD and ProfileElementSSD to test_constructor_encode 
Change-Id: Idc6f37b487dfa8a69ac7a50a537cfc317113d501
 2024-07-10 06:51:23 +00:00
Harald Welte
292191d67a pySim.esim.saip: Add ProfileElementAKA constructor + methods 
This helps us to construct an akaParameter PE from scratch.

Change-Id: I4cc42c98bf82aec085ab7f48aea4ff7efa0eae9e
 2024-07-10 06:51:23 +00:00
Harald Welte
c0ea149555 pySim.esim: Allow calling compile_asn1_subdir() with non-DER coddec 
this isn't needed for the on-wire format, but can be useful for debug
output in GSER or JER.

Change-Id: I1de4b9506a92d60f582c328a180760332584f9e4
 2024-07-10 06:51:23 +00:00
Harald Welte
200bf6eb8b pySim.esim.saip: Meaningful defaults in PE Constructor + test 
Let's make sure the constructor of ProfileElement subclasses set
meaningful defaults to the self.decoded member, so that the to_der()
method can actually encode it.   This is required when constructing
a profile from scratch, as opposed to loading an existing one from DER.

Also, add a test to verify that the encoder passes without exception;
doesn't test the generated binary data.

Change-Id: I401bca16e58461333733877ec79102a5ae7fe410
 2024-07-10 06:51:23 +00:00
Harald Welte
698886247f pySim.tlv: Fix ComprTlvMeta() not passing kwargs to parent __new__ 
This fixes commit cdf661b24c
"pySim.tlv.COMPR_TLV_IE: Patch comprehension bit if derived class misses it"
where we introduce a comprehension-TLV specific derived metaclass, which forgets
to pass the kwargs through to the parent metaclass.

Change-Id: If65a8169bcf91bb2f943d0316f1140e07f0b8b8e
 2024-07-10 08:39:40 +02:00
Harald Welte
b6532b56d2 saip-tool: Add 'extract-apps' to dump all applications from eSIM profile 
This new action can be used to dump all java applications as either raw
IJC file or converted to CAP format (the usual format generated by
JavaCard toolchains).

Change-Id: I51cffa5ba3ddbea491341d678ec9249d7cf470a5
 2024-06-11 08:45:27 +02:00
Harald Welte
3d70f659f3 saip-tool: Add new 'info' action to print general information 
It will print something like this:

SAIP Profile Version: 2.1
Profile Type: 'GSMA Generic eUICC Test Profile'
ICCID: 8949449999999990023f
Mandatory Services: usim, isim, csim, javacard, usim-test-algorithm

NAAs: mf[1], usim[1], csim[1], isim[1]
NAA mf
NAA usim (a0000000871002ff49ff0589)
        IMSI: 001010123456063
NAA csim
NAA isim (a0000000871004ff49ff0589)

Number of applications: 0

Change-Id: I107d457c3313a766229b569453c18a8d69134bec
 2024-06-10 13:39:40 +02:00
Harald Welte
ecb65bc2f2 esim.saip: Remove debug print() 
Change-Id: I8dfe29302225d951e656d1321bbd249bfe242602
 2024-06-10 13:39:40 +02:00
Harald Welte
f36e9fd39f es9p_client: Use a plausible TAC (copy from lpac) 
Some SM-DP+ (notably Idemia) fail if the TAC is not valid.

Change-Id: I48890c4a56147410d0cd5c4e47647b8eb5ad9998
 2024-06-10 13:39:40 +02:00
Harald Welte
36276e7b2a contrib/jenkins.sh: Execute pylint also on all contrib python scripts 
This way we get linting coverage for sim-rest-{server,client}, eidtool,
unber and others.

Change-Id: I2d6271d493d0f6765e6a184f8ae32f8325317be2
 2024-06-10 11:39:28 +00:00
Harald Welte
5341bf902f unber.py: work-around pylint reporting (possibly-used-before-assignment) 
contrib/unber.py:39:22: E0606: Possibly using variable 'content' before assignment (possibly-used-before-assignment)

Change-Id: I725cd5e05e3121c853669eb4bbfe5ba51b79eb75
 2024-06-10 11:39:28 +00:00
Harald Welte
5964bdd5a4 osmo-smdpp: use NIST-P256 by default 
The eSIM specs allow for both brainpool and nist; in reality the
deployments use the NIST P256 curve.

osmo-smdpp currently only supports a single certificate; let's use the
NIST one by default.

Change-Id: Idc7809f320505279c8a75e9b667be0a2af802f6b
 2024-06-10 11:39:28 +00:00
Harald Welte
1aa77c5d74 tests/ota_test.py: Allow stand-alone execution 
Let's add a __main__ section to allow stand-alone execution via
	python3 ./tests/test_ota.py

Change-Id: Ic3940ac23c7ddc1013e21f41eae6076a11dfd4f4
 2024-06-10 11:39:28 +00:00
Harald Welte
32401a54e6 pySim.ota.OtaDialectSms: Implement command decoding 
So far we only implemented command encoding and response decoding.
Let's also add command decoding, which is useful for example when
decoding protocol traces.

Change-Id: Id666cea8a91a854209f3c19c1f09b512bb493c85
 2024-06-10 11:39:28 +00:00
Harald Welte
8bd551af32 pySim.ota.OtaDialectSms: Move SMS header construct up to class level 
this way we can use it in other [future] methods.

Change-Id: If296f823c18864fddcfb9cb1b82a087bac8875d4
 2024-06-10 07:45:00 +00:00
Harald Welte
1a9cabbbf0 pySim/ota: Don't modify input argument in OtaDialectSms.encode_cmd 
Change-Id: I4c4c44002762696b931ed3580ffe54daf62ffa61
 2024-06-10 08:59:39 +02:00
Harald Welte
4a191089dc pySim.cat: Add more alredy-defined IEs to ProactiveCmd classes 
... also add some spec references

Change-Id: If071abdc61c7c881bdea5292d12c74a1024f6784
 2024-06-10 08:59:39 +02:00
Harald Welte
3b4a673de4 add contrib/saip-tool.py 
This is a tool to work with eSIM profiles in SAIP format.  It allows
to dump the contents, run constraint checkers as well as splitting
of the PE-Sequence into the individual PEs.

Change-Id: I396bcd594e0628dfc26bd90233317a77e2f91b20
 2024-06-10 08:59:39 +02:00
Harald Welte
a5634c248b jenkins.sh: Include es9p_client in pylint 
Change-Id: I06f6773b8b5d3dfa588617d5af81c2fddb474a3d
 2024-06-09 22:49:33 +02:00
Harald Welte
cdf661b24c pySim.tlv.COMPR_TLV_IE: Patch comprehension bit if derived class misses it 
Our current implementation assumes that all COMPR_TLV_IE are created
with a raw tag value that has the comprehension bit set.  Check for this
during the class __new__ method and print a warning if we have to fix it up

Change-Id: I299cd65f32dffda9040d18c17a374e8dc9ebe7da
 2024-06-09 22:49:33 +02:00
Harald Welte
05349a0c65 pySim.cat: Make sure to always set comprehension bit in COMPR_TLV_IE 
our implementation currently assumes that all derived classes are
created with a tag value that has the comprehension bit set.

Change-Id: I6e5f2a69c960c03015c3f233f8fbc2a7a802f07e
 2024-06-09 22:17:51 +02:00
Harald Welte
144bae3f37 pySim.tlv: Correctly parse COMPREHENSION-TLV without comprehension bit 
The uppermost bit of COMPREHENSION-TLV tags indicates whether the
recipient is required to "comprehend" that IE or not. So every IE
actually has two tag values: one with and one without that bit set.

As all our existing TLV definitions of COMPR_TLV_IE have that bit set,
let's assume this is the default, but use the same definition also for
the situation where that bit is not set.

Change-Id: I58d04ec13be0c12d9fb8cb3d5a0480d0defb6c95
 2024-06-09 12:18:16 +02:00
Harald Welte
4680503acc esim.saip: Add ProfileElementSequence.remove_naas_of_type 
This method allows the caller to remove all NAAs of a certain type,
for example to remove all CSIM instances from a given profile.

Change-Id: I64438bf0be58bad7a561c3744b7e9b1338a7857c
 2024-06-09 12:18:16 +02:00
Harald Welte
0cb0e02c5c esim.saip: Introduce ProfileElement.identification property 
Change-Id: I6525bb78619e574296488843e021d505e0632d99
 2024-06-09 12:18:16 +02:00
Harald Welte
50d9e2a6d8 esim.es9p: Suppress sending requestHeader on ES9+ 
SGP.22 states that ES9+ should not include a requestHeader

Change-Id: Ic9aa874a82241d7b26e2bcb0423961173e103020
 2024-06-09 12:18:16 +02:00
Harald Welte
888c6e5647 add contrib/es9p_client: Perform ES9+ client functions like LPA+eUICC 
This tool can be used to test the SM-DP+. It implements the full dance
of all HTTPs API operations to get to the downloadProfile, and will
decrypt the BPP to the UPP, which is then subsequently stored as file on
disk.

Needless to say, this will only work if you have an eUICC certificate +
private key that is compatible with the CI of your SM-DP+.

Change-Id: Idf8881e82f9835f5221c58b78ced9937cf5fb520
 2024-06-09 12:18:16 +02:00
Harald Welte
f07161d396 http_json_api / es9p: Add User-Agent header 
ES9+ (And ES11) require the use of User-Agent, while ES2+ not.

Change-Id: Iffe64d82087940a82fbfa73bf5d2b7e864ae5d67
 2024-06-09 12:18:16 +02:00
Harald Welte
0d1dea01df add pySim.esim.es9p with definitions of the ES9+ HTTP Interface 
Let's use the infrastructure of pySim.esim.http_json_api to define
the ES9+ API Functions.  This can in turn be used by clients or even
osmo-smdpp can be ported over to using this infratructure rather than
open-coding a lot of the encoding/decoding of API request/response
parameters.

Change-Id: I194ef1d186391f36245c099cc70a4813185ecf9c
 2024-06-09 12:18:16 +02:00
Harald Welte
f1495c1e4e esim.es2p: Split generic part of HTTP/REST API from ES2+ 
This way we can reuse it for other eSIM RSP HTTP interfaces like
ES9+, ES11, ...

Change-Id: I468041da40a88875e8df15b04d3ad508e06f16f7
 2024-06-09 12:18:16 +02:00
Harald Welte
7b3d4b805c pySim/cat: Fix "Decode the "Type of Comand" from numeric value to a string" 
This fixes a bug introduced in Change-Id: I833ec02bf281fe49de2be326018e91f521de52c0

Change-Id: I8b466c123173a5be335df3e1d77ef1c5f717a7d9
 2024-06-09 12:17:51 +02:00
Harald Welte
2c39d81b4b pySim/cat: Decode the "Type of Comand" from numeric value to a string 
This makes pySim-trace of proactive UICC much more readable.

Change-Id: I833ec02bf281fe49de2be326018e91f521de52c0
 2024-06-08 20:15:08 +02:00
Harald Welte
2eea70f6bc pySim.apdu.ts_102_221: Decode FETCH and TERMINAL RESPONSE body 
This gives a meaningful decode during pySim-trace.

Change-Id: Ifa410e1fefc25e87ffa8e3a2230af80180a36a18
 2024-06-08 18:38:22 +02:00
Harald Welte
f22637f151 pySim.apdu.ts_102_221: Decode the ENVELOPE command body using pySim.cat TLV 
This will decode the ENVELOPE body in pySim-trace further.

Before:

00 ENVELOPE                                             -        9000 {'p1': 0, 'p2': 0, 'cmd': 'd14682028381060291978b3c40048111227ff6407070611535002d02700000281516011212000001eae1bd578fa25791898128811b2206cc71639ca292ec2526da8aef4273d2fe2e', 'rsp': '027100001f0a00000100000001200000ab12800101230d08a0000001510000000f829000'}

After:

00 ENVELOPE                                             -        9000 {'p1': 0, 'p2': 0, 'cmd': [{'smspp_download': [{'device_identities': {'source_dev_id': 'network', 'dest_dev_id': 'uicc'}}, {'address': {'ton_npi': 145, 'call_number': '79'}}, {'sms_tpdu': {'tpdu': '40048111227ff6407070611535002d02700000281516011212000001eae1bd578fa25791898128811b2206cc71639ca292ec2526da8aef4273d2fe2e'}}]}], 'rsp': '027100001f0a00000100000001200000ab12800101230d08a0000001510000000f829000'}

Change-Id: I5ecdbe0b5fa8856cb723569896b73cd49778ed5f
 2024-06-08 18:38:22 +02:00
Harald Welte
5529a41a63 pySim.cat: More TLV Definitions for Event Download 
Change-Id: I713f12577cab1678cdf97b7ae0e6f3815a42242c
 2024-06-08 18:38:22 +02:00
Harald Welte
33a6daee6d pySim.apdu: Allow TLV based decoders for APDU command and response body 
So far we only supported construct.

Change-Id: Ibb80d328c9a1f464aa5338ca0ca1d6bfb00734e1
 2024-06-08 18:38:22 +02:00
Harald Welte
16749075f9 pySim-trace: Add support for the TCA Loader log file format 
The "TCA Loader" is a freeware utility program published by the
Trusted Connectivity Alliance for testing SCP80, SCP81, SCP02 and SCP03
in UICCs.  It can generate text log files of the APDUs it exchanges;
let's add this file format to pySim-trace

Change-Id: Ie76d36bb18c6bd8968d2a5b74ec1b8c5ccaaa409
 2024-06-08 18:38:22 +02:00
Harald Welte
add30ecbff global_platform/euicc: Implement obtaining SCP keys from CardKeyProvider 
Now that CardKeyProvider is capable of storing key materials
transport-key-encrypted, we can use this functionality to look up the
SCP02 / SCP03 key material for a given security domain.

This patch implements this for the ISD-R and ECASD using a look-up by
EID inside the CSV.

Change-Id: I2a21f031ab8af88019af1b8390612678b9b35880
 2024-06-04 23:18:37 +02:00
Harald Welte
1aaf978d9f CardKeyProvider: Implement support for column-based transport key encryption 
It's generally a bad idea to keep [card specific] key material lying
around unencrypted in CSV files.  The industry standard solution in the
GSMA is a so-called "transport key", which encrypts the key material.

Let's introduce support for this in the CardKeyProvider (and
specifically, the CardKeyProviderCSV) and allow the user to specify
transport key material as command line options to pySim-shell.

Different transport keys can be used for different key materials, so
allow specification of keys on a CSV-column base.

The higher-level goal is to allow the CSV file not only to store
the ADM keys (like now), but also global platform key material for
establishing SCP towards various security domains in a given card.

Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff
 2024-06-04 23:18:37 +02:00
Harald Welte
a3d41a147f document the CardKeyProvider 
Change-Id: Ie6fc24695dd956a4f9fd6f243d3b0ef66acf877b
 2024-06-04 23:18:37 +02:00
Harald Welte
0251367ddb pySim.esim.saip: Meaningful constructors for [I]SD + SSD 
So far the main use case was to read a ProfileElement-SD from
a DER file.  But when we want to construct one from scratch,
we need to have the constructor put some meaningful [default]
values into the class members.

Change-Id: I69e104f1d78165c12291317326dbab05977a1574
 2024-06-04 23:18:37 +02:00
Harald Welte
bc949649da esim.saip: Implement ProfileElement.header_name for more PE types 
We now cover all PE types as of PE_Definitions-3.3.1.asn

Change-Id: I37951a0441fe53fce7a329066aebd973389cb743
 2024-06-04 23:00:46 +02:00
Harald Welte
4d5d2f5849 pySim.esim.saip.validation: Ensure unique PE identification value 
Change-Id: I37b9eb4cfb74de79b0493986d976c8a5f8ccd8ea
 2024-06-04 20:51:57 +00:00
Harald Welte
77256d0c48 esim.saip: Implement SecurityDomainSD.{add,has,remove}_key() methods 
This way it's possible to programmatically inspect and modify the
high-level decoded key material inside a securityDomain profile element.

Change-Id: I18b1444303de80eaddd840a7e0061ea0098a8ba1
 2024-06-04 20:51:57 +00:00
Harald Welte
80976b65e5 esim.saip: Introduce ProfileElement derived classes 
It's rather useful to have derived classes implementing specific
functions related to that SAIP profile type.  Let's introruce that
concept and a first example for securityDomain, where methods allow
checking/adding/removing support for SCPs.

Change-Id: I0929cc704b2aabddbc2ddee79ab8b674b1ed4691
 2024-06-04 20:51:57 +00:00
Harald Welte
fe28a1d87d esim.bsp: Fix a bug in demac_only_one() 
When de-MAC-ing at the recipient side, we must increment the cipher(!)
block number even if no ciphering is done at all.

We did this correctly for MAC (sender) case, but not on the de-MAC
(receiver) case.

Change-Id: I97993f9e8357b36401d435aaa15558d1c7e411eb
 2024-06-03 16:07:57 +00:00
Harald Welte
ee7be44528 utils: Introduce BER-TLV parsers that return raw tag or even raw TLV 
In the eSIM RSP univers there are some rather ugly layering violatoins
where ASN.1 cannot be parsed but we have to mess with raw TLVs and the
details of DER encoding.  Let's add two funtions that make it more
convenient to work with this: They return the raw tag as integer, or
even the entire encoded TLV rather than the value part only.

Change-Id: I1e68a4003b833e86e9282c77325afa86ce144b98
 2024-06-03 16:07:57 +00:00
Harald Welte
2755b54ded [cosmetic] fix typos in comments 
Change-Id: I549ef7002e6ebef3f13af620cad8d03c7f4d891a
 2024-06-02 18:23:31 +00:00
Harald Welte
ddbfc043ac add globalplatform.uicc 
GlobalPlatform has a [non-public] "UICC Configuration" spec, which
defines some specific aspects of implementing GlobalPlatform in the
context of an UICC.  Let's add some python definitions about it.

Change-Id: If4cb110a9bc5f873b0e097c006bef59264ee48fa
 2024-05-30 20:06:59 +02:00
Harald Welte
64a5901c4c osmo-smdpp: Make error message more descriptive 
Before this patch we had three different error causes that would cause a
"Verification failed" error message.  Let's state explicitly which part
of verification did actually fail.

Change-Id: I5030758fe365bb802ae367b494aace5a66bc7a91
 2024-05-30 20:06:59 +02:00
Harald Welte
56912caac7 osmo-smdpp: Don't re-encode euiccSigned1/euiccSigned2 
We used to re-encode those parts of a decoded ASN.1 struct that is
cryptographically signed in the GSMA SGP.22 specification.  However, if
the received data follows a later spec and contains new/unknown records,
then our poor-man's attempt at re-encoding will render a different
binary, which in turn means the signature check will fail.

Let's instead do a manual step-by-step raw decode of the DER TLV
structure to extract the actual binary information of parts of ASN.1
objects.

Change-Id: I4e31fd4b23ec3be15b9d07c2c30a3e31e22bdda1
Closes: OS#6473
 2024-05-30 20:06:59 +02:00
Harald Welte
3dabbafdba docs/shell: Mention GlobalPlatform and eUICC commands in overview 
Change-Id: I5b6ad752fea09ed9632f150dfbbabf2156a5a9c0
 2024-05-30 20:06:59 +02:00
Harald Welte
e4450afb4e pySim.app: Attempt to retrieve the EID of a SGP.22 / SGP.32 eUICC 
... and populate the RuntimeState.identity['EID'] wit it, so other
[future] parts of the system can use it.

Let's also print the EID (if available) from the 'cardinfo' shell
command.

Change-Id: Idc2ea1d9263f39b3dff403e1535a5e6c4e88b26f
 2024-05-26 11:01:29 +02:00
Harald Welte
7f6102365c pySim-shell: Migrate PySimApp.iccid to RuntimeState.identity['ICCID'] 
In the previous patch, we've introduced a new 'identities' dict as part
of the runtime state.  Let's migrate our ICCID storage into it for
consistency.

Change-Id: Ibdcf9a7c4e7e445201640bce33b768bcc4460db1
 2024-05-26 11:01:29 +02:00
Harald Welte
f47433863e runtime: Introduce an 'identity' dict for things like ATR, ICCID, EID 
This patch introduces the dict, as well as its first use for ATR storage

Change-Id: Ief5ceaf5afe82800e33da233573293527befd2f4
 2024-05-26 11:01:29 +02:00
Harald Welte
3ba10b61e1 pysim/euicc: Remove duplicated code 
The get_eid command is actually sending the command apdu twice, as
it contains both an older implementation (result unused) and the newer
one.

Change-Id: Ie82bb09f4fc30bc879029b83147dad5614792b48
 2024-05-26 11:01:29 +02:00
Harald Welte
a823ce89f6 pySim/commands: STATUS: Use indeterminate length Le/P3 == '00' 
Let's have the card tell us what the length is by indicating '00'
instead of stating 'FF'.  This is better aligned with general practice
and won't break assumptions in other parts of the code like SCP
transport.

Change-Id: Ied63c6e1970e3dfc675da5e5f94579fbb06fea51
 2024-05-26 11:01:29 +02:00
Harald Welte
8844603941 pySim/global_platform: Fix install_for_personalization command 
A mix-up betewen underscore and dash resulted in:

Change-Id: I49d12b7c7ae2a343940e87d5069c0ae44a9bc50c
AttributeError: 'Namespace' object has no attribute 'application_aid'
 2024-05-26 11:01:29 +02:00
Oliver Smith
6add18ea08 contrib/sim-rest-client: don't crash without args 
When running without an argument, let argparse print a nice usage error:

  $ ./sim-rest-client.py
  usage: sim-rest-client.py [-h] [-H HOST] [-p PORT] [-v] [-n SLOT_NR] {auth,info} ...
  sim-rest-client.py: error: the following arguments are required: {auth,info}

Instead of:

  $ ./sim-rest-client.py
  Traceback (most recent call last):
    File "/usr/share/pysim/contrib/./sim-rest-client.py", line 185, in <module>
      main(sys.argv)
    File "/usr/share/pysim/contrib/./sim-rest-client.py", line 181, in main
      args.func(args)
      ^^^^^^^^^
  AttributeError: 'Namespace' object has no attribute 'func'

Change-Id: I92998d9b94dcfb9dcfc3da161fe5d8f45f242b78
 2024-05-24 20:23:35 +00:00
Oliver Smith
56264669a7 pcsc: don't assume opts.pcsc_shared is present 
Fixes running contrib/sim-rest-server.py:
  builtins.AttributeError: 'Namespace' object has no attribute 'pcsc_shared'

Change-Id: I864f65849c5d43cf7c73e60f1935afdf4273f696
 2024-05-24 20:23:01 +00:00
Harald Welte
172c9f7ca6 pySim/cat: Fix contruct for Address class/IE 
Something like "this._.total_len-1" only works during decode. Let's
use GreedyBytes instead, working for encode and decode.

Change-Id: Idf8326298cab7ebc68b09c7e829bfc2061222f51
 2024-05-23 16:54:53 +02:00
Harald Welte
daeba3c1fb sysmocom_sjs2: Make sure 'Const' is imported 
File "/crypt/space/home/laforge/projects/git/pysim/pySim/sysmocom_sja2.py", line 180, in __init__
    self._construct = Struct(Const(b'\x82'), 'time_unit'/self.TimeUnit, 'value'/Int8ub,
                             ^^^^^
NameError: name 'Const' is not defined

Change-Id: If34a48e349680ef84e68a4a1a19dde536ecda0e6
 2024-05-22 18:03:59 +02:00
Harald Welte
91ec099680 euicc: clarify which eUICCs are supported 
We currently do not support M2M eUICC

Change-Id: I3deb9f181075411484158471012ed449c83028fa
 2024-05-22 18:03:59 +02:00
Harald Welte
568d8cf5db pySim-trace.py: Resolve possible variable use before assignment 
pySim-trace.py:198:27: E0606: Possibly using variable 's' before assignment (possibly-used-before-assignment)

Change-Id: I28c137a20143b2cd6ea9a0d5461ab61fcd6fe935
 2024-05-22 18:03:59 +02:00
Harald Welte
a3f22ea259 pySim-prog.py: Resolve possible variable use before assignment 
pySim-prog.py:741:7: E0606: Possibly using variable 'cp' before assignment (possibly-used-before-assignment)

Change-Id: I6ab307db378d2ca76dfeae53dc3befa7c103974d
 2024-05-22 18:03:59 +02:00
Harald Welte
81bc26cc31 osmo-smdpp.py: Resolve possible variable use before assignment 
osmo-smdpp.py:374:72: E0601: Using variable 'iccid_str' before assignment (used-before-assignment)

Let's raise an exception in the erroneous case.

Change-Id: I01b308226e12f91699b1b5c6bb06f853be47e185
 2024-05-22 18:03:59 +02:00
Harald Welte
c3d04ab193 euicc.py: Resolve possible variable use before assignment 
pySim/euicc.py:436:31: E0606: Possibly using variable 'p_id' before assignment (possibly-used-before-assignment)
pySim/euicc.py:455:31: E0606: Possibly using variable 'p_id' before assignment (possibly-used-before-assignment)
pySim/euicc.py:473:31: E0606: Possibly using variable 'p_id' before assignment (possibly-used-before-assignment)

Let's raise an exception in the erroneous case.

Change-Id: Ifdf4651e503bae6ea3e91c89c2121b416a12fb1a
 2024-05-22 18:03:59 +02:00
Harald Welte
bb2cba83c5 commands.py: Resolve possible variable use before assignment 
pySim/commands.py:608:39: E0606: Possibly using variable 'p2' before assignment (possibly-used-before-assignment)

Let's raise an exception in the erroneous case.

Change-Id: I23adf2e89aa8a13246cc20ef022c84f0113eb2cd
 2024-05-22 18:03:59 +02:00
Harald Welte
45b7d0126b commands.py: Resolve possible variable use before assignment 
pySim/commands.py:223:18: E0606: Possibly using variable 'skip' before assignment (possibly-used-before-assignment)

Let's raise an exception in the erroneous case.

Change-Id: Id1a892c3446e472699e77f076c2414277e92c98d
 2024-05-22 18:03:59 +02:00
Harald Welte
73a5c74114 pySim-trace: Support decoding of eUICC traces 
Let's register the ISD-R and ECASD applications so we avoid the warnings
printed when processing an eUICC protocol trace:

WARNING  pySim.apdu.ts_102_221: SELECT UNKNOWN AID a0000005591010ffffffff8900000100

Change-Id: I362a1a7f12d979ff0b7971d5300db9ed56bb1ee5
 2024-05-10 20:30:58 +02:00
Harald Welte
a644fecc01 pySim.global_platform: Fix key encryption with DEK 
When a SCP is active, the DEK is used to encrypt any key material
that's installed using PUT KEY.  The code prior to this patch fails
to handle this case as it calls the encrypt_key() method on the wrong
object.

Change-Id: I6e10fb9c7881ba74ad2986c36bba95b336470838
 2024-05-10 18:28:32 +00:00
Harald Welte
900b04559b euicc: Fix shell command for SGP.31 get_certs 
Change-Id: I2e59070992bb522d14a5e4956f0d8e738a785dd8
 2024-05-10 18:19:29 +00:00
Harald Welte
57df6f6e68 filesystem: Enforce lower-case hex AID 
our utils.b2h() returns values in lower-case hex string notation,
so let's make sure the CardADF and CardApplication AID values are also
stored in lower case notation, othewise the matching baesd on AIDs
returned from the card will not work, specifically as we use uppercase
AIDs in pySim.euicc for CardApplicationECASD and CardApplicationISDR.

Rather than change those two instances, let's solve it in a generic way.

We already do the same for the CardFile.fid member.

Change-Id: Ie42392412d9eb817fbc563d9165faab198ffa7a9
 2024-05-10 19:58:53 +02:00
Harald Welte
1d1ba8e4cc esim.esp2: Allow HTTP methods other than POST 
While all official/standardized ES2+ API functions use POST, there
are some vendor-specific extensions using different HTTP methods.  Be
flexible enough to allow derived classes to easily specify other methods.

Change-Id: I4b1a0dc7e6662485397c7708933bf16e5ed56e10
 2024-04-03 00:49:33 +02:00
Harald Welte
b2b29cfed1 esim.es2p: Permit ApiParamInteger to be an actual integer 
Usually, the specifications say that the integer type is actually
transmitted as a JSON string type.  However, it seems some
implementations do return a native JSON integer type.  Let's be
tolerant in that regard.

Change-Id: I5b47f8bba01225d53eff2ca086e53a2133abed7f
 2024-04-03 00:49:31 +02:00
Harald Welte
7aeeb4f475 Add funding link to github mirror 
see https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository

Change-Id: Ib23e6f406ab9546f59ec9e8c6b3eaf27c3dce410
 2024-03-23 09:22:55 +00:00
Harald Welte
f3432eef4c README.md: Add link to issue tracker 
Change-Id: I33f4e05486d609b2c903c8341dccf1ee01e90577
 2024-03-23 10:06:48 +01:00
Harald Welte
60eef0264a README.md: Link to discourse forum 
Change-Id: Ia5ecbd4f2c2a5dfa1ba69ae2b5712da7abc93c4e
 2024-03-23 10:06:48 +01:00
Harald Welte
0c5dfd9d23 README.md: Point to simtrace mailing list 
SIM card related topics are best kept there and not on openbsc.

Change-Id: I0dedd2ed0ab07c6020f9d30857654c5600c53814
 2024-03-23 10:06:48 +01:00
Vadim Yanitskiy
a412c436b4 contrib/jenkins.sh: add 'distcheck' job to check package integrity 
The idea of this new job is to catch package integrity problems,
like the missing entries in setup.py/packages[] or missing deps.

Change-Id: Ic72d58494e8fd0cab8d66ce60f7b70593b770872
Related: osmo-ci.git I9d4d9e9de2b16a4b745791f3c9c93507f43bfa6d
 2024-03-21 18:39:00 +00:00
Cody Harris
479aeb0b00 add missing modules to setup.py 
Change-Id: I330d5e35e5f1b508c6209b6894009b5fdd35d660
 2024-03-20 15:01:46 -07:00
Harald Welte
24a7f168bd pcsc: open reader/card in EXCLUSIVE mode by default 
There was a support request hinting that other applications
concurrently accessed the SIM and were messing up the card state while
pySim-shell was running.

Let's avoid such situations by opening the card/reader in EXCLUSIVE mode
by default.  If somebody really has a special use case, they can now add
the --pcsc-shared flag to restore the legacy behavior (SHARED mode).

Change-Id: I90d887714b559a4604708d3c6dd23b5e05f40576
 2024-03-15 20:33:09 +00:00
Harald Welte
3aa0b41f39 pySim-prog: convert from optparse to argparse 
We already use argparse everywhere else, and we have moved reader-driver
argument parsing into the library expecting argparse.

Change-Id: I7407496643247c754d002656688e9fdcbcf644a8
 2024-03-15 20:33:09 +00:00
Philipp Maier
7b524fa079 osmo-smdpp: fix generation of transactionId 
The hex string of the generated transactionId contains lowercase hex
digits. However SGP.22 explicitly spcifies to use uppercase hex digits
when using JSON fromatted messages. See section 6.5.2.6 for example.

Related: SYS#6720
Change-Id: I8439aa9d70f6fe798fa88b623bac13debdc19ca1
 2024-03-15 09:27:36 +01:00
Harald Welte
ee4db7010b sysmocom_sja2: Add test vectors for EF_USIM_AUTH_KEY 
Change-Id: I8be62ba52fbbf6d470f771906a5d3734cca5bac8
 2024-03-14 12:05:24 +01:00
Harald Welte
2c219cd706 docs/shell: Give users some hints on what to do if encoding/decoding fails 
Change-Id: I557991da748126f3585b88b27706b29e0264635b
Related: OS#6385
 2024-03-11 12:55:29 +01:00
Vadim Yanitskiy
decb468092 tests: assertEquals() is deprecated, use assertEqual() 
This fixes deprecation warnings printed by Python 3.11.7.

Change-Id: I1de93b0fee9e8439f7da8a3b9fd2a6974973fb4f
 2024-03-02 01:36:19 +07:00
Harald Welte
b18c7d9be0 saip.personalization: Fix encoding of ICCID in ProfileHeader 
To make things exciting, they decided that the ICCID in the profile
header is encoded different from the ICCID contained in EF.ICCID...

Change-Id: I5eacdcdc6bd0ada431eb047bfae930d79d6e3af8
 2024-02-21 09:23:58 +01:00
Harald Welte
6d63712b51 saip.personalization: automatically compute class 'name' attribute 
We can use the metaclass to set a proper non-camel-case name attribute.

Change-Id: If02df436c8f5ce01d21e9ee077ad3736e669d103
 2024-02-21 09:23:58 +01:00
Harald Welte
2de552e712 saip.personalization: differentiate input_value from value 
When personalizing e.g. the ICCID, the input_value is the raw
incrementing counter.  From that, we calculate the Luhn check digit,
and that "output" value is what we'll put in to the EF.ICCID specific
encoder.

However, we also store that output value in the instance in order
to generate the output CSV file containig the card-specific
personalization data.

Change-Id: Idfcd26c8ca9d73a9c2955f7c97e711dd59a27c4e
 2024-02-21 09:23:55 +01:00
Harald Welte
19fa98e7d0 saip.personalization: Add support for SCP80/81/02/03 keys 
Those keys are normally per-card unique, and hence the personalization
must be able to modify them in the profile.

Change-Id: Ibe4806366f1cce8edb09d52613b1dd56250fa5ae
 2024-02-21 09:22:40 +01:00
Harald Welte
318faef583 saip.personalization: include encode/decode of value; add validation method 
Change-Id: Ia9fa39c25817448afb191061acd4be894300eeef
 2024-02-21 09:22:40 +01:00
Harald Welte
aa76546d16 osmo-smdpp: Add TS.48 profiles modified for unique ICCIDs 
The original TS.48 profiles have shared/overlapping ICCIDs meaning you
can always install one of them on a given eUICC.  Let's add a set of
modified TS.48 profiles so  you can install any number of them in
parallel on a single eUICC, switching between them via your LPA.

Change-Id: Id5019b290db1ee90ae1c72b312f08bf3184908ea
 2024-02-21 09:22:40 +01:00
Harald Welte
8449b14d08 osmo-smdpp: Get rid of hard-coded ICCID 
Read the ICCID from the header of the UPP when building the
ProfileMetdata.  This allows the download of profiles with arbitrary ICCID.

Change-Id: I1b9e17f757f9935436828e6dc1ab75ff17d1d1a4
 2024-02-20 23:55:37 +01:00
Harald Welte
922b8a279c saip: improve docstrings 
Change-Id: I0ca82a434e0bde3dc1b304dfc179d568588631c6
 2024-02-18 22:30:08 +01:00
Harald Welte
7d88b076ad pylint: esim/saip/validation.py 
pySim/esim/saip/validation.py:95:42: C0117: Consider changing "not not ('usim' in m_svcs or 'isim' in m_svcs)" to "'usim' in m_svcs or 'isim' in m_svcs" (unnecessary-negation)
pySim/esim/saip/validation.py:129:0: C0305: Trailing newlines (trailing-newlines)

Change-Id: Idcc9871d6a7068e8aedbd8cd81f4156918af5e50
 2024-02-18 22:30:08 +01:00
Harald Welte
5ff0bafcda pylint: esim/saip/__init__.py 
pySim/esim/saip/__init__.py:28:0: R0402: Use 'from pySim.esim.saip import templates' instead (consider-using-from-import)
pySim/esim/saip/__init__.py:166:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/esim/saip/__init__.py:206:4: W0612: Unused variable 'tagdict' (unused-variable)
pySim/esim/saip/__init__.py:273:23: C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len)

Change-Id: I12ef46c847d197fb0c01e624818aeac14eb99e31
 2024-02-18 22:30:08 +01:00
Harald Welte
d16a20ccc3 saip: profile processing; merging with templates 
Introduce code that makes use of the information from
pySim.esim.saip.templates to build a complete representation of a file
by merging the template with the ProfileElement decribing the file.

This happens within the class pySim.esim.saip.File, whose instances are
created from ProfileElement + Template.

Change-Id: Ib1674920e488ade9597cb039e4e2047dcbc7864e
 2024-02-18 22:30:08 +01:00
Harald Welte
54b4f0ccbd asn1/saip: Fix typo in original ASN.1: Compontents -> Components 
Change-Id: I6bec5625579873a9ec267d896584608c9d5e3a2f
 2024-02-18 22:30:08 +01:00
Harald Welte
efdf423a7f utils: Add function to verify Luhn check digits and to sanitize ICCIDs 
Change-Id: I7812420cf97984dd834fca6a38c5e5ae113243cb
 2024-02-18 22:30:08 +01:00
Harald Welte
979c837286 Dynamically determine maximum CMD data length depending on SCP 
If we're using a Secure Channel Protocol, this will add overhead
in terms of the C-MAC appended to the C-APDU.  This means in turn that
the useable length of the data field shrinks by a certain number of
bytes.

Let's make sure the SCP instances expose an 'overhead' property
of how much overhead they add - and that other commands use this to
determine the maximum command data field length.

Change-Id: I0a081a23efe20c77557600e62b52ba90a401058d
 2024-02-15 20:35:29 +01:00
Harald Welte
1432af5150 Add terminal_capability command to send TERMINAL CAPABILITY 
TS 102 221 specifies a TERMINAL CAPABILITY command using which the
terminal (Software + hardware talking to the card) can expose their
capabilities.  This is also used in the eUICC universe to let the eUICC
know which features are supported.

Change-Id: Iaeb8b4c34524edbb93217bf401e466399626e9b0
 2024-02-12 18:59:54 +01:00
Harald Welte
eac459fe24 ts_31_102: Add support for "USIM supporting non-IMSI SUPI Type" 
This type of USIM was introduced in Release 16.4. It is basically
a copy of ADF.USIM without the EF.IMSI file and a dedicated AID.

Change-Id: Ifcde27873a398273a89889bb38537f79859383e9
 2024-02-12 18:04:19 +01:00
Harald Welte
95873a964e Introduce code for ES2+ API client functionality 
Change-Id: Id652bb4c2df8893a824b8bb44beeafdfbb91de3f
 2024-02-09 21:41:51 +01:00
Harald Welte
e1c0b626d8 global_platform: Add --suppress-key-check option to put_key command 
In some cases we may not want to auto-generate the Key Check Values.

Change-Id: I244b717b3e3aae6eb3ad512f9e23ff0b65958bb7
 2024-02-06 20:36:32 +01:00
Harald Welte
d6ecf272f5 pySim-shell: Fix regression in 'apdu' command on cards without profile 
Cards where no profile was detected don't have a logical channel, and
hence must use the raw APDU at all times.

Change-Id: I08e5d190bdb4e62ee808bfd77584cb3e0b85a8ae
Fixes: Change-Id Id0c364f772c31e11e8dfa21624d8685d253220d0
 2024-02-05 17:54:51 +01:00
Harald Welte
9d1487af6d global_platform: Fix INSTALL [for personalization] 
The APDU hex string needs to use %02x instead of %02u...

Change-Id: Ic3b30ba623ee04f5190c77afd226b52165b3183f
 2024-02-05 17:54:30 +01:00
Harald Welte
908634396f pylint: global_platform/__init__.py 
pySim/global_platform/__init__.py:468:4: W0221: Number of parameters was 2 in 'CardFile.decode_select_response' and is now 1 in overriding 'ADF_SD.decode_select_response' method (arguments-differ)
pySim/global_platform/__init__.py:473:8: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/global_platform/__init__.py:491:19: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:528:22: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:559:12: C0200: Consider using enumerate instead of iterating with range and len (consider-using-enumerate)
pySim/global_platform/__init__.py:587:18: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:617:20: W0612: Unused variable 'dec' (unused-variable)
pySim/global_platform/__init__.py:645:12: W0612: Unused variable 'data' (unused-variable)
pySim/global_platform/__init__.py:645:18: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:746:15: C0121: Comparison 'opts.key_id == None' should be 'opts.key_id is None' (singleton-comparison)
pySim/global_platform/__init__.py:746:39: C0121: Comparison 'opts.key_ver == None' should be 'opts.key_ver is None' (singleton-comparison)
pySim/global_platform/__init__.py:750:15: C0121: Comparison 'opts.key_id != None' should be 'opts.key_id is not None' (singleton-comparison)
pySim/global_platform/__init__.py:752:15: C0121: Comparison 'opts.key_ver != None' should be 'opts.key_ver is not None' (singleton-comparison)
pySim/global_platform/__init__.py:787:16: W0612: Unused variable 'rsp_hex' (unused-variable)
pySim/global_platform/__init__.py:787:25: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:836:30: W0612: Unused variable 'sw' (unused-variable)
pySim/global_platform/__init__.py:839:12: W0612: Unused variable 'ext_auth_resp' (unused-variable)
pySim/global_platform/__init__.py:846:33: W0613: Unused argument 'opts' (unused-argument)
pySim/global_platform/__init__.py:878:15: R1716: Simplify chained comparison between the operands (chained-comparison)
pySim/global_platform/__init__.py:886:29: W0613: Unused argument 'kvn' (unused-argument)
pySim/global_platform/__init__.py:893:0: C0413: Import "from Cryptodome.Cipher import DES, DES3, AES" should be placed at the top of the module (wrong-import-position)
pySim/global_platform/__init__.py:23:0: C0411: standard import "from typing import Optional, List, Dict, Tuple" should be placed before "from construct import Optional as COptional" (wrong-import-order)
pySim/global_platform/__init__.py:24:0: C0411: standard import "from copy import deepcopy" should be placed before "from construct import Optional as COptional" (wrong-import-order)
pySim/global_platform/__init__.py:893:0: C0411: third party import "from Cryptodome.Cipher import DES, DES3, AES" should be placed before "from pySim.global_platform.scp import SCP02, SCP03" (wrong-import-order)
pySim/global_platform/__init__.py:893:0: C0412: Imports from package Cryptodome are not grouped (ungrouped-imports)

Change-Id: Iea6afb5e72e035637e761bb25535f48fd4bc99f4
 2024-02-05 17:54:30 +01:00
Harald Welte
55be7d48ee pylint: construct.py 
pySim/construct.py:47:0: W0311: Bad indentation. Found 16 spaces, expected 12 (bad-indentation)
pySim/construct.py:59:0: W0311: Bad indentation. Found 16 spaces, expected 12 (bad-indentation)
pySim/construct.py:82:0: W0311: Bad indentation. Found 16 spaces, expected 12 (bad-indentation)
pySim/construct.py:1:0: C0114: Missing module docstring (missing-module-docstring)
pySim/construct.py:14:0: W0105: String statement has no effect (pointless-string-statement)
pySim/construct.py:178:29: W0613: Unused argument 'instr' (unused-argument)
pySim/construct.py:199:15: C0121: Comparison 'codepoint_prefix == None' should be 'codepoint_prefix is None' (singleton-comparison)
pySim/construct.py:269:15: C0121: Comparison 'v == False' should be 'v is False' if checking for the singleton value False, or 'not v' if testing for falsiness (singleton-comparison)
pySim/construct.py:271:17: C0121: Comparison 'v == True' should be 'v is True' if checking for the singleton value True, or 'v' if testing for truthiness (singleton-comparison)
pySim/construct.py:385:15: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/construct.py:392:15: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/construct.py:408:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/construct.py:421:7: R1701: Consider merging these isinstance calls to isinstance(c, (Container, dict)) (consider-merging-isinstance)
pySim/construct.py:444:11: R1729: Use a generator instead 'all(v == 255 for v in raw_bin_data)' (use-a-generator)
pySim/construct.py:434:81: W0613: Unused argument 'exclude_prefix' (unused-argument)
pySim/construct.py:544:12: W0707: Consider explicitly re-raising using 'raise IntegerError(str(e), path=path) from e' (raise-missing-from)
pySim/construct.py:561:8: R1731: Consider using 'nbytes = max(nbytes, minlen)' instead of unnecessary if block (consider-using-max-builtin)
pySim/construct.py:573:12: W0707: Consider explicitly re-raising using 'raise IntegerError(str(e), path=path) from e' (raise-missing-from)
pySim/construct.py:3:0: C0411: standard import "import typing" should be placed before "from construct.lib.containers import Container, ListContainer" (wrong-import-order)
pySim/construct.py:10:0: C0411: third party import "import gsm0338" should be placed before "from pySim.utils import b2h, h2b, swap_nibbles" (wrong-import-order)
pySim/construct.py:11:0: C0411: standard import "import codecs" should be placed before "from construct.lib.containers import Container, ListContainer" (wrong-import-order)
pySim/construct.py:12:0: C0411: standard import "import ipaddress" should be placed before "from construct.lib.containers import Container, ListContainer" (wrong-import-order)
pySim/construct.py:7:0: W0611: Unused BitwisableString imported from construct.core (unused-import)

Change-Id: Ic8a06d65a7bcff9ef399fe4e7e5d82f271c946bb
 2024-02-05 17:54:30 +01:00
Harald Welte
6db681924c pylint: tlv.py 
pySim/tlv.py:29:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/tlv.py:43:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/tlv.py:66:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/tlv.py:89:11: C0121: Comparison 'self.decoded == None' should be 'self.decoded is None' (singleton-comparison)
pySim/tlv.py:170:8: R1703: The if statement can be replaced with 'return bool(test)' (simplifiable-if-statement)
pySim/tlv.py:202:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:257:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:308:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/tlv.py:383:15: C0121: Comparison 'tag == None' should be 'tag is None' (singleton-comparison)
pySim/tlv.py:382:17: W0612: Unused variable 'r' (unused-variable)
pySim/tlv.py:389:16: W0612: Unused variable 'dec' (unused-variable)
pySim/tlv.py:461:22: R1718: Consider using a set comprehension (consider-using-set-comprehension)
pySim/tlv.py:473:0: C0206: Consider iterating with .items() (consider-using-dict-items)
pySim/tlv.py:473:58: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
pySim/tlv.py:20:0: W0611: Unused Optional imported from typing (unused-import)
pySim/tlv.py:20:0: W0611: Unused Dict imported from typing (unused-import)
pySim/tlv.py:20:0: W0611: Unused Any imported from typing (unused-import)
pySim/tlv.py:21:0: W0611: Unused bidict imported from bidict (unused-import)
pySim/tlv.py:28:0: W0611: Unused LV imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused HexAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused BcdAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused BitsRFU imported from pySim.construct (unused-import)
pySim/tlv.py:28:0: W0611: Unused GsmStringAdapter imported from pySim.construct (unused-import)
pySim/tlv.py:29:0: W0614: Unused import(s) NoCardError, ProtocolError, ReaderError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: Ic22d00d3ae73ad81167276d9482b7b86a04476ba
 2024-02-05 17:54:30 +01:00
Harald Welte
f2b20bf6ca pylint: utils.py 
pySim/utils.py:903:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/utils.py:153:16: R1719: The if expression can be replaced with 'bool(test)' (simplifiable-if-expression)
pySim/utils.py:158:16: R1719: The if expression can be replaced with 'bool(test)' (simplifiable-if-expression)
pySim/utils.py:166:16: R1719: The if expression can be replaced with 'bool(test)' (simplifiable-if-expression)
pySim/utils.py:222:19: R1719: The if expression can be replaced with 'not test' (simplifiable-if-expression)
pySim/utils.py:237:18: R1719: The if expression can be replaced with 'bool(test)' (simplifiable-if-expression)
pySim/utils.py:246:19: R1719: The if expression can be replaced with 'not test' (simplifiable-if-expression)
pySim/utils.py:279:11: W0612: Unused variable 'remainder' (unused-variable)
pySim/utils.py:541:7: R1714: Consider merging these comparisons with 'in' by using 'eutran_bits in (16384, 28672)'. Use a set instead if elements are hashable. (consider-using-in)
pySim/utils.py:550:7: R1714: Consider merging these comparisons with 'in' by using 'gsm_bits in (128, 140)'. Use a set instead if elements are hashable. (consider-using-in)
pySim/utils.py:614:7: C0121: Comparison 'imsi == None' should be 'imsi is None' (singleton-comparison)
pySim/utils.py:627:7: C0121: Comparison 'imsi == None' should be 'imsi is None' (singleton-comparison)
pySim/utils.py:733:7: R1714: Consider merging these comparisons with 'in' by using 'msisdn in ('', '+')'. Use a set instead if elements are hashable. (consider-using-in)
pySim/utils.py:774:8: W0612: Unused variable 'try_encode' (unused-variable)
pySim/utils.py:803:16: W0707: Consider explicitly re-raising using 'except ValueError as exc' and 'raise ValueError('PIN-ADM needs to be hex encoded using this option') from exc' (raise-missing-from)
pySim/utils.py:801:16: W0612: Unused variable 'try_encode' (unused-variable)
pySim/utils.py:821:7: C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len)
pySim/utils.py:836:4: W0612: Unused variable 'e' (unused-variable)
pySim/utils.py:892:7: C0121: Comparison 'str_list == None' should be 'str_list is None' (singleton-comparison)
pySim/utils.py:991:11: R1701: Consider merging these isinstance calls to isinstance(o, (BytesIO, bytearray, bytes)) (consider-merging-isinstance)

Change-Id: I190ae75964ef6e0ed43fae994693a8bccd21c7f7
 2024-02-05 17:54:30 +01:00
Harald Welte
472165f20f pylint: ts_102_222.py 
pySim/ts_102_222.py:195:0: W0311: Bad indentation. Found 15 spaces, expected 12 (bad-indentation)
pySim/ts_102_222.py:201:0: W0311: Bad indentation. Found 15 spaces, expected 12 (bad-indentation)
pySim/ts_102_222.py:26:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/ts_102_222.py:35:4: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/ts_102_222.py:52:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:52:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:73:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:73:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:89:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:89:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:107:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:107:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:152:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:152:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:203:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:203:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:220:9: W0612: Unused variable 'data' (unused-variable)
pySim/ts_102_222.py:220:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_222.py:24:0: C0411: standard import "import argparse" should be placed before "import cmd2" (wrong-import-order)
pySim/ts_102_222.py:23:0: W0611: Unused with_argparser imported from cmd2 (unused-import)
pySim/ts_102_222.py:27:0: W0611: Unused h2b imported from pySim.utils (unused-import)
pySim/ts_102_222.py:27:0: W0611: Unused swap_nibbles imported from pySim.utils (unused-import)
pySim/ts_102_222.py:27:0: W0611: Unused JsonEncoder imported from pySim.utils (unused-import)
pySim/ts_102_222.py:26:0: W0614: Unused import(s) NoCardError, ProtocolError, ReaderError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: If251c6cb10e637a13adaaf3ae848501908b9c345
 2024-02-05 17:54:30 +01:00
Harald Welte
f2322774c7 pylint: filesystem.py 
pySim/filesystem.py:823:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/filesystem.py:849:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/filesystem.py:43:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/filesystem.py:74:45: C0121: Comparison 'fid == None' should be 'fid is None' (singleton-comparison)
pySim/filesystem.py:94:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/filesystem.py:100:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/filesystem.py:149:8: W0105: String statement has no effect (pointless-string-statement)
pySim/filesystem.py:170:11: C0121: Comparison 'self.parent == None' should be 'self.parent is None' (singleton-comparison)
pySim/filesystem.py:283:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:309:8: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/filesystem.py:314:15: C0117: Consider changing "not 'fid' in kwargs" to "'fid' not in kwargs" (unnecessary-negation)
pySim/filesystem.py:317:24: R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal)
pySim/filesystem.py:418:11: C0121: Comparison 'name == None' should be 'name is None' (singleton-comparison)
pySim/filesystem.py:427:11: C0121: Comparison 'sfid == None' should be 'sfid is None' (singleton-comparison)
pySim/filesystem.py:452:28: R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal)
pySim/filesystem.py:508:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/filesystem.py:531:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/filesystem.py:576:8: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/filesystem.py:599:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:609:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:620:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:633:28: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:642:41: W0613: Unused argument 'opts' (unused-argument)
pySim/filesystem.py:644:24: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:696:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:723:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:749:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:777:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:797:8: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/filesystem.py:822:23: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:838:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:844:34: W0613: Unused argument 'opts' (unused-argument)
pySim/filesystem.py:848:23: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:866:23: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:878:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:893:28: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:910:24: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:967:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:995:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1023:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1051:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1114:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1141:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1167:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1194:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/filesystem.py:1226:8: W0246: Useless parent or super() delegation in method '__init__' (useless-parent-delegation)
pySim/filesystem.py:1236:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:1239:35: W0613: Unused argument 'opts' (unused-argument)
pySim/filesystem.py:1252:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:1263:19: W0612: Unused variable 'sw' (unused-variable)
pySim/filesystem.py:1315:24: R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal)
pySim/filesystem.py:35:0: C0411: standard import "import argparse" should be placed before "import cmd2" (wrong-import-order)
pySim/filesystem.py:37:0: C0411: standard import "from typing import cast, Optional, Iterable, List, Dict, Tuple, Union" should be placed before "import cmd2" (wrong-import-order)
pySim/filesystem.py:27:0: W0611: Unused import code (unused-import)
pySim/filesystem.py:34:0: W0611: Unused with_argparser imported from cmd2 (unused-import)
pySim/filesystem.py:41:0: W0611: Unused i2h imported from pySim.utils (unused-import)
pySim/filesystem.py:41:0: W0611: Unused Hexstr imported from pySim.utils (unused-import)
pySim/filesystem.py:44:0: W0611: Unused js_path_find imported from pySim.jsonpath (unused-import)
pySim/filesystem.py:43:0: W0614: Unused import(s) NoCardError, ProtocolError, ReaderError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: I94e1f5791e9fc34a60d0254978a35fd6ab2ff8d7
 2024-02-05 17:51:59 +01:00
Harald Welte
8829f8e690 pylint: commands.py 
pySim/commands.py:443:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/commands.py:446:0: C0325: Unnecessary parens after 'elif' keyword (superfluous-parens)
pySim/commands.py:669:0: C0325: Unnecessary parens after 'elif' keyword (superfluous-parens)
pySim/commands.py:27:0: W0622: Redefining built-in 'BlockingIOError' (redefined-builtin)
pySim/commands.py:27:0: W0401: Wildcard import construct (wildcard-import)
pySim/commands.py:30:0: W0404: Reimport 'Hexstr' (imported line 29) (reimported)
pySim/commands.py:42:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:48:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:98:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:114:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:131:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:223:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:234:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/commands.py:252:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/commands.py:271:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/commands.py:274:18: W0612: Unused variable 'sw' (unused-variable)
pySim/commands.py:326:16: W0707: Consider explicitly re-raising using 'raise ValueError('%s, failed to read (offset %d)' % (str_sanitize(str(e)), offset)) from e' (raise-missing-from)
pySim/commands.py:386:16: W0707: Consider explicitly re-raising using 'raise ValueError('%s, failed to write chunk (chunk_offset %d, chunk_len %d)' % (str_sanitize(str(e)), chunk_offset, chunk_len)) from e' (raise-missing-from)
pySim/commands.py:443:12: R1720: Unnecessary "elif" after "raise", remove the leading "el" from "elif" (no-else-raise)
pySim/commands.py:521:14: R1714: Consider merging these comparisons with 'in' by using 'sw in ('62f1', '62f2')'. Use a set instead if elements are hashable. (consider-using-in)
pySim/commands.py:532:11: R1701: Consider merging these isinstance calls to isinstance(data, (bytearray, bytes)) (consider-merging-isinstance)
pySim/commands.py:666:8: R1720: Unnecessary "elif" after "raise", remove the leading "el" from "elif" (no-else-raise)
pySim/commands.py:762:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/commands.py:776:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)

Change-Id: Idfcd6f799d5de9ecacd2c3d1e0d1f7d932f2b8db
 2024-02-05 12:41:38 +01:00
Harald Welte
09f9663005 pylint: pySim/euicc.py 
pySim/euicc.py:27:0: W0622: Redefining built-in 'BlockingIOError' (redefined-builtin)
pySim/euicc.py:27:0: W0401: Wildcard import construct (wildcard-import)
pySim/euicc.py:37:7: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/euicc.py:47:9: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/euicc.py:337:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/euicc.py:325:63: W0613: Unused argument 'exp_sw' (unused-argument)
pySim/euicc.py:335:15: W0612: Unused variable 'sw' (unused-variable)
pySim/euicc.py:361:13: W0612: Unused variable 'data' (unused-variable)
pySim/euicc.py:361:19: W0612: Unused variable 'sw' (unused-variable)
pySim/euicc.py:363:52: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:380:41: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:386:37: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:392:37: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:398:39: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:415:39: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:478:29: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:480:13: W0612: Unused variable 'data' (unused-variable)
pySim/euicc.py:480:19: W0612: Unused variable 'sw' (unused-variable)
pySim/euicc.py:500:31: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:506:48: W0613: Unused argument 'opts' (unused-argument)
pySim/euicc.py:26:0: C0411: third party import "from construct import Optional as COptional" should be placed before "from pySim.tlv import *" (wrong-import-order)
pySim/euicc.py:27:0: C0411: third party import "from construct import *" should be placed before "from pySim.tlv import *" (wrong-import-order)
pySim/euicc.py:28:0: C0411: standard import "import argparse" should be placed before "from construct import Optional as COptional" (wrong-import-order)
pySim/euicc.py:29:0: C0411: third party import "from cmd2 import cmd2, CommandSet, with_default_category" should be placed before "from pySim.tlv import *" (wrong-import-order)
pySim/euicc.py:30:0: C0412: Imports from package pySim are not grouped (ungrouped-imports)
pySim/euicc.py:31:0: W0611: Unused CardADF imported from pySim.filesystem (unused-import)
pySim/euicc.py:31:0: W0611: Unused CardApplication imported from pySim.filesystem (unused-import)

Change-Id: I6c33e2361a042a16f27e66cb883c392333b8383d
 2024-02-05 12:37:54 +01:00
Harald Welte
356a6c0f99 pylint: runtime.py 
pySim/runtime.py:272:0: C0325: Unnecessary parens after 'raise' keyword (superfluous-parens)
pySim/runtime.py:276:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/runtime.py:280:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/runtime.py:349:0: C0325: Unnecessary parens after 'raise' keyword (superfluous-parens)
pySim/runtime.py:549:0: C0305: Trailing newlines (trailing-newlines)
pySim/runtime.py:29:4: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/runtime.py:119:16: W0612: Unused variable 'data' (unused-variable)
pySim/runtime.py:153:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/runtime.py:161:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/runtime.py:212:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/runtime.py:273:12: W0707: Consider explicitly re-raising using 'raise RuntimeError('%s: %s - %s' % (swm.sw_actual, k[0], k[1])) from swm' (raise-missing-from)
pySim/runtime.py:267:19: W0612: Unused variable 'sw' (unused-variable)
pySim/runtime.py:343:27: W0612: Unused variable 'sw' (unused-variable)
pySim/runtime.py:397:15: W0612: Unused variable 'sw' (unused-variable)
pySim/runtime.py:527:14: W0612: Unused variable 'sw' (unused-variable)
pySim/runtime.py:528:8: W0612: Unused variable 'tag' (unused-variable)
pySim/runtime.py:528:13: W0612: Unused variable 'length' (unused-variable)
pySim/runtime.py:528:28: W0612: Unused variable 'remainder' (unused-variable)

Change-Id: I2e164dbaa2070116bed3bac63b0fa5b8aa5b1331
 2024-02-05 11:27:36 +01:00
Harald Welte
f01c4b2c98 pylint: ara_m.py 
pySim/ara_m.py:29:0: W0622: Redefining built-in 'BlockingIOError' (redefined-builtin)
pySim/ara_m.py:29:0: W0401: Wildcard import construct (wildcard-import)
pySim/ara_m.py:68:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/ara_m.py:89:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/ara_m.py:282:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/ara_m.py:280:15: W0612: Unused variable 'sw' (unused-variable)
pySim/ara_m.py:312:34: W0613: Unused argument 'opts' (unused-argument)
pySim/ara_m.py:318:37: W0613: Unused argument 'opts' (unused-argument)
pySim/ara_m.py:356:15: C0121: Comparison 'opts.aid != None' should be 'opts.aid is not None' (singleton-comparison)
pySim/ara_m.py:385:37: W0613: Unused argument 'opts' (unused-argument)
pySim/ara_m.py:309:8: W0238: Unused private member `AddlShellCommands.__init(self)` (unused-private-member)
pySim/ara_m.py:309:8: W0238: Unused private member `ADF_ARAM.AddlShellCommands.__init(self)` (unused-private-member)

Change-Id: I5a739187a8966cdb0ae5c6cbc7bc5d4115433aeb
 2024-02-05 11:27:36 +01:00
Harald Welte
a5fafe8b48 pylint: ts_102_221.py 
pySim/ts_102_221.py:20:0: W0622: Redefining built-in 'BlockingIOError' (redefined-builtin)
pySim/ts_102_221.py:30:0: R0402: Use 'from pySim import iso7816_4' instead (consider-using-from-import)
pySim/ts_102_221.py:20:0: W0401: Wildcard import construct (wildcard-import)
pySim/ts_102_221.py:235:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/ts_102_221.py:272:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/ts_102_221.py:281:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/ts_102_221.py:484:12: W0715: Exception arguments suggest string formatting might be intended (raising-format-tuple)
pySim/ts_102_221.py:486:12: W0715: Exception arguments suggest string formatting might be intended (raising-format-tuple)
pySim/ts_102_221.py:488:12: W0715: Exception arguments suggest string formatting might be intended (raising-format-tuple)
pySim/ts_102_221.py:523:11: C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len)
pySim/ts_102_221.py:647:0: W0613: Unused argument 'kwargs' (unused-argument)
pySim/ts_102_221.py:747:19: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_102_221.py:26:0: C0411: third party import "from bidict import bidict" should be placed before "from pySim.construct import *" (wrong-import-order)
pySim/ts_102_221.py:27:0: C0412: Imports from package pySim are not grouped (ungrouped-imports)
pySim/ts_102_221.py:29:0: W0611: Unused match_sim imported from pySim.profile (unused-import)
pySim/ts_102_221.py:34:0: W0611: Unused DF_GSM imported from pySim.ts_51_011 (unused-import)
pySim/ts_102_221.py:34:0: W0611: Unused DF_TELECOM imported from pySim.ts_51_011 (unused-import)

Change-Id: I99d408bdf2551527f097a04240e857728b738621
 2024-02-05 09:56:02 +01:00
Harald Welte
a5630dc45c pylint: apdu/ts_102_221.py 
pySim/apdu/ts_102_221.py:60:16: R1724: Unnecessary "else" after "continue", remove the "else" and de-indent the code inside it (no-else-continue)
pySim/apdu/ts_102_221.py:107:16: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/apdu/ts_102_221.py:294:8: R1703: The if statement can be replaced with 'return bool(test)' (simplifiable-if-statement)
pySim/apdu/ts_102_221.py:294:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/ts_102_221.py:299:31: W0613: Unused argument 'lchan' (unused-argument)
...
pySim/apdu/ts_102_221.py:389:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu/ts_102_221.py:421:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu/ts_102_221.py:425:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu/ts_102_221.py:438:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/ts_102_221.py:26:0: C0411: standard import "from typing import Optional, Dict, Tuple" should be placed before "from construct import GreedyRange, Struct" (wrong-import-order)

Change-Id: Id5caac8da4c965dbaf88d624cdc9dcc8fc168b8c
 2024-02-05 09:55:56 +01:00
Harald Welte
4b56c6cd3e pylint: ts_31_102.py 
Change-Id: I5b72ad476d338aa4048bb15a74796ef69191f028
 2024-02-05 09:55:50 +01:00
Harald Welte
0d9c8f73a8 pylint: sysmocom_sja2.py 
pySim/sysmocom_sja2.py:20:0: W0401: Wildcard import pytlv.TLV (wildcard-import)
pySim/sysmocom_sja2.py:27:0: W0401: Wildcard import construct (wildcard-import)
pySim/sysmocom_sja2.py:21:0: C0411: standard import "from struct import pack, unpack" should be placed before "from pytlv.TLV import *" (wrong-import-order)
pySim/sysmocom_sja2.py:27:0: C0411: third party import "from construct import *" should be placed before "from pySim.utils import *" (wrong-import-order)
pySim/sysmocom_sja2.py:21:0: W0611: Unused pack imported from struct (unused-import)
pySim/sysmocom_sja2.py:25:0: W0611: Unused CardProfileUICC imported from pySim.ts_102_221 (unused-import)

Change-Id: I0e5b5c6f3179f9710464af4cba91d682412b8a09
 2024-02-05 09:55:43 +01:00
Harald Welte
4f3976d77f pylint: cdma_ruim.py 
pySim/cdma_ruim.py:30:0: W0401: Wildcard import construct (wildcard-import)
pySim/cdma_ruim.py:188:4: W0237: Parameter 'data_hex' has been renamed to 'resp_hex' in overriding 'CardProfileRUIM.decode_select_response' method (arguments-renamed)
pySim/cdma_ruim.py:30:0: C0411: third party import "from construct import *" should be placed before "from pySim.utils import *" (wrong-import-order)

Change-Id: I4c384f37a6a317c6eddef8742572fcfa76a5fc20
 2024-02-05 09:53:59 +01:00
Harald Welte
4c0b80415e pylint: global_platform/scp.py 
pySim/global_platform/scp.py:27:0: W0404: Reimport 'Optional' (imported line 20) (reimported)
pySim/global_platform/scp.py:157:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/global_platform/scp.py:165:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/global_platform/scp.py:182:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/global_platform/scp.py:189:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/global_platform/scp.py:266:4: W0221: Variadics removed in overriding 'SCP02._wrap_cmd_apdu' method (arguments-differ)
pySim/global_platform/scp.py:298:4: W0237: Parameter 'rsp_apdu' has been renamed to 'apdu' in overriding 'SCP02.unwrap_rsp_apdu' method (arguments-renamed)
pySim/global_platform/scp.py:314:7: C0121: Comparison 'l == None' should be 'l is None' (singleton-comparison)
pySim/global_platform/scp.py:436:11: C0121: Comparison 'host_challenge == None' should be 'host_challenge is None' (singleton-comparison)
pySim/global_platform/scp.py:506:4: W0237: Parameter 'rsp_apdu' has been renamed to 'apdu' in overriding 'SCP03.unwrap_rsp_apdu' method (arguments-renamed)
pySim/global_platform/scp.py:27:0: C0411: standard import "from typing import Optional" should be placed before "from Cryptodome.Cipher import DES3, DES" (wrong-import-order)

Change-Id: Idd2b779a6628c88d9a48c94b8581525209824426
 2024-02-05 09:53:54 +01:00
Harald Welte
530bf73cbc pylint: esim/saip/oid.py 
pySim/esim/saip/oid.py:30:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/esim/saip/oid.py:46:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)

Change-Id: I65c9cd1bb2b6a1747a7fbb25052adc75605bc870
 2024-02-05 09:53:50 +01:00
Harald Welte
5f6b64dc25 pylint: esim/saip/templates.py 
pySim/esim/saip/templates.py:106:0: R1707: Disallow trailing comma tuple (trailing-comma-tuple)
pySim/esim/saip/templates.py:56:37: C0121: Comparison 'self.fid != None' should be 'self.fid is not None' (singleton-comparison)
pySim/esim/saip/templates.py:57:28: C0121: Comparison 'self.arr != None' should be 'self.arr is not None' (singleton-comparison)
pySim/esim/saip/templates.py:58:37: C0121: Comparison 'self.sfi != None' should be 'self.sfi is not None' (singleton-comparison)
pySim/esim/saip/templates.py:96:11: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/esim/saip/templates.py:591:0: W1404: Implicit string concatenation found in list (implicit-str-concat)

Change-Id: I181578ba630c8bdb558297e990411b59593652a0
 2024-02-05 09:53:45 +01:00
Harald Welte
1258e464a1 pylint: esim/saip/personalization.py 
pySim/esim/saip/personalization.py:104:0: W0311: Bad indentation. Found 17 spaces, expected 16 (bad-indentation)
pySim/esim/saip/personalization.py:105:0: W0311: Bad indentation. Found 17 spaces, expected 16 (bad-indentation)
pySim/esim/saip/personalization.py:151:0: C0305: Trailing newlines (trailing-newlines)
pySim/esim/saip/personalization.py:36:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/esim/saip/personalization.py:56:4: W0237: Parameter 'pe_seq' has been renamed to 'pes' in overriding 'Iccid.apply' method (arguments-renamed)
pySim/esim/saip/personalization.py:19:0: W0611: Unused Optional imported from typing (unused-import)

Change-Id: I70b3e266bbafabbfcec3d48027d50b45c2c17809
 2024-02-05 09:53:40 +01:00
Harald Welte
2b84644c08 pylint: esim/rsp.py 
pySim/esim/rsp.py:101:4: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/rsp.py:27:0: C0411: standard import "from collections.abc import MutableMapping" should be placed before "from cryptography.hazmat.primitives.asymmetric import ec" (wrong-import-order)
pySim/esim/rsp.py:22:0: W0611: Unused import copyreg (unused-import)
pySim/esim/rsp.py:27:0: W0611: Unused MutableMapping imported from collections.abc (unused-import)

Change-Id: Id87dbf82cd41ce6e5276e5bdd7af1877d77e3fab
 2024-02-05 09:53:35 +01:00
Harald Welte
f235b799e9 pylint: esim/x509_cert.py 
pySim/esim/x509_cert.py:70:4: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/x509_cert.py:91:20: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/esim/x509_cert.py:105:28: W3101: Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (missing-timeout)
pySim/esim/x509_cert.py:163:0: C0413: Import "from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature" should be placed at the top of the module (wrong-import-position)
pySim/esim/x509_cert.py:20:0: C0411: standard import "from typing import Optional, List" should be placed before "import requests" (wrong-import-order)
pySim/esim/x509_cert.py:163:0: C0411: third party import "from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature" should be placed before "from pySim.utils import b2h" (wrong-import-order)
pySim/esim/x509_cert.py:163:0: C0412: Imports from package cryptography are not grouped (ungrouped-imports)
pySim/esim/x509_cert.py:22:0: W0611: Unused padding imported from cryptography.hazmat.primitives.asymmetric (unused-import)
pySim/esim/x509_cert.py:24:0: W0611: Unused InvalidSignature imported from cryptography.exceptions (unused-import)

Change-Id: Ic435c9a7cfcc18cacec3a3d872925bd737fb5cd9
 2024-02-05 09:53:30 +01:00
Harald Welte
e6e74229c9 pylint: pySim/esim/bsp.py 
pySim/esim/bsp.py:1:0: C0114: Missing module docstring (missing-module-docstring)
pySim/esim/bsp.py:28:0: C0413: Import "import abc" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:29:0: C0413: Import "from typing import List" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:30:0: C0413: Import "import logging" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:33:0: C0413: Import "from cryptography.hazmat.primitives import hashes" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:34:0: C0413: Import "from cryptography.hazmat.primitives.kdf.x963kdf import X963KDF" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:36:0: C0413: Import "from Cryptodome.Cipher import AES" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:37:0: C0413: Import "from Cryptodome.Hash import CMAC" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:39:0: C0413: Import "from pySim.utils import bertlv_encode_len, bertlv_parse_one, b2h" should be placed at the top of the module (wrong-import-position)
pySim/esim/bsp.py:48:55: W0613: Unused argument 'padding' (unused-argument)
pySim/esim/bsp.py:55:45: W0613: Unused argument 'multiple' (unused-argument)
pySim/esim/bsp.py:84:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/bsp.py:89:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/bsp.py:94:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/bsp.py:169:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/esim/bsp.py:292:8: W0612: Unused variable 'tdict' (unused-variable)
pySim/esim/bsp.py:292:15: W0612: Unused variable 'l' (unused-variable)
pySim/esim/bsp.py:292:23: W0612: Unused variable 'remain' (unused-variable)

Change-Id: I64bd634606c375e767676a4b5ba7c2cc042350c2
 2024-02-05 09:53:26 +01:00
Harald Welte
9ef65099d2 pylint: apdu/__init__.py 
pySim/apdu/__init__.py:41:0: W0105: String statement has no effect (pointless-string-statement)
pySim/apdu/__init__.py:55:4: C0204: Metaclass class method __new__ should have 'mcs' as first argument (bad-mcs-classmethod-argument)
pySim/apdu/__init__.py:187:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu/__init__.py:200:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:208:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:216:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:224:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:239:11: C0117: Consider changing "not 'p1' in self.cmd_dict" to "'p1' not in self.cmd_dict" (unnecessary-negation)
pySim/apdu/__init__.py:295:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:313:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:416:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:429:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:455:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/apdu/__init__.py:31:0: C0411: standard import "import typing" should be placed before "from termcolor import colored" (wrong-import-order)
pySim/apdu/__init__.py:32:0: C0411: standard import "from typing import List, Dict, Optional" should be placed before "from termcolor import colored" (wrong-import-order)

Change-Id: I5657912df474f3ed0e277458a8eb33e28aeb2927
 2024-02-05 09:53:21 +01:00
Harald Welte
0930bcbbb7 pylint: apdu/ts_31_102.py 
pySim/apdu/ts_31_102.py:12:0: W0401: Wildcard import construct (wildcard-import)
pySim/apdu/ts_31_102.py:38:0: W0404: Reimport 'ApduCommand' (imported line 18) (reimported)
pySim/apdu/ts_31_102.py:38:0: W0404: Reimport 'ApduCommandSet' (imported line 18) (reimported)

Change-Id: I191147af95142adcd7d768d7dae6480b0c7513fc
 2024-02-05 09:53:16 +01:00
Harald Welte
295d4a4907 pylint: apdu_source/pyshark_rspro 
pySim/apdu_source/pyshark_rspro.py:19:0: W0611: Unused import sys (unused-import)
pySim/apdu_source/pyshark_rspro.py:21:0: W0611: Unused pprint imported from pprint as pp (unused-import)
pySim/apdu_source/pyshark_rspro.py:25:0: W0611: Unused b2h imported from pySim.utils (unused-import)

Change-Id: Ibe8482d8adbb82a74f36b0d64bc5dae27da02b73
 2024-02-05 09:53:12 +01:00
Harald Welte
9bc016e777 pylint: apdu_source/pyshark_gsmtap 
pySim/apdu_source/pyshark_gsmtap.py:90:0: C0305: Trailing newlines (trailing-newlines)
pySim/apdu_source/pyshark_gsmtap.py:68:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu_source/pyshark_gsmtap.py:30:0: C0411: first party import "from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/pyshark_gsmtap.py:31:0: C0411: first party import "from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/pyshark_gsmtap.py:32:0: C0411: first party import "from pySim.apdu.global_platform import ApduCommands as GpApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/pyshark_gsmtap.py:19:0: W0611: Unused import sys (unused-import)
pySim/apdu_source/pyshark_gsmtap.py:21:0: W0611: Unused pprint imported from pprint as pp (unused-import)
pySim/apdu_source/pyshark_gsmtap.py:25:0: W0611: Unused b2h imported from pySim.utils (unused-import)
pySim/apdu_source/pyshark_gsmtap.py:26:0: W0611: Unused Tpdu imported from pySim.apdu (unused-import)

Change-Id: I0f2bfed2f671e02fc48bcc2a03c785edc691584f
 2024-02-05 09:53:06 +01:00
Harald Welte
528d922510 pylint: apdu_source/gsmtap.py 
pySim/apdu_source/gsmtap.py:48:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/apdu_source/gsmtap.py:44:20: W0612: Unused variable 'addr' (unused-variable)
pySim/apdu_source/gsmtap.py:22:0: C0411: first party import "from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/gsmtap.py:23:0: C0411: first party import "from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/gsmtap.py:24:0: C0411: first party import "from pySim.apdu.global_platform import ApduCommands as GpApduCommands" should be placed before "from . import ApduSource, PacketType, CardReset" (wrong-import-order)
pySim/apdu_source/gsmtap.py:19:0: W0611: Unused GsmtapMessage imported from pySim.gsmtap (unused-import)

Change-Id: I672e8838ebe11015863fd4fd6047181a3f184658
 2024-02-05 09:52:58 +01:00
Harald Welte
c5ff0a6ab5 pylint: apdu_source/__init__.py 
pySim/apdu_source/__init__.py:1:0: C0114: Missing module docstring (missing-module-docstring)
pySim/apdu_source/__init__.py:17:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/apdu_source/__init__.py:34:16: W0133: Exception statement has no effect (pointless-exception-statement)

Change-Id: Iec552afd6004b849132132642910a4c7f91e3473
 2024-02-05 09:52:52 +01:00
Harald Welte
49d69335b2 pylint: transport/__init__.py 
pySim/transport/__init__.py:139:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)

Change-Id: Ibeeb7d6fde40bb37774bbd09ad185203ac7bcb48
 2024-02-05 09:52:46 +01:00
Harald Welte
fdaefd9a8a pylint: transport/serial.py 
pySim/transport/serial.py:54:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/transport/serial.py:89:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/transport/serial.py:225:0: C0325: Unnecessary parens after 'while' keyword (superfluous-parens)
pySim/transport/serial.py:63:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/transport/serial.py:106:8: R1720: Unnecessary "elif" after "raise", remove the leading "el" from "elif" (no-else-raise)
pySim/transport/serial.py:124:12: W0707: Consider explicitly re-raising using 'except Exception as exc' and 'raise ValueError('Invalid reset pin %s' % self._rst_pin) from exc' (raise-missing-from)
pySim/transport/serial.py:204:12: R1723: Unnecessary "elif" after "break", remove the leading "el" from "elif" (no-else-break)
pySim/transport/serial.py:20:0: C0411: standard import "import time" should be placed before "import serial" (wrong-import-order)
pySim/transport/serial.py:21:0: C0411: standard import "import os" should be placed before "import serial" (wrong-import-order)
pySim/transport/serial.py:22:0: C0411: standard import "import argparse" should be placed before "import serial" (wrong-import-order)
pySim/transport/serial.py:23:0: C0411: standard import "from typing import Optional" should be placed before "import serial" (wrong-import-order)

Change-Id: I82ef12492615a18a13cbdecf0371b3a5d02bbd5c
 2024-02-05 09:52:40 +01:00
Harald Welte
7781c70c09 pylint: transport/pcsc.py 
pySim/transport/pcsc.py:26:0: W0404: Reimport 'CardConnectionException' (imported line 26) (reimported)
pySim/transport/pcsc.py:60:4: R1711: Useless return at end of function or method (useless-return)
pySim/transport/pcsc.py:74:12: W0707: Consider explicitly re-raising using 'except CardRequestTimeoutException as exc' and 'raise NoCardError() from exc' (raise-missing-from)
pySim/transport/pcsc.py:86:12: W0707: Consider explicitly re-raising using 'except CardConnectionException as exc' and 'raise ProtocolError() from exc' (raise-missing-from)
pySim/transport/pcsc.py:88:12: W0707: Consider explicitly re-raising using 'except NoCardException as exc' and 'raise NoCardError() from exc' (raise-missing-from)
pySim/transport/pcsc.py:22:0: W0611: Unused Union imported from typing (unused-import)

Change-Id: I0ef440d8825300d6efb8959a67da095ab5623f9c
 2024-02-05 09:52:29 +01:00
Harald Welte
181becb676 pylint: transport/modem_atcmd.py 
pySim/transport/modem_atcmd.py:70:0: C0325: Unnecessary parens after 'assert' keyword (superfluous-parens)
pySim/transport/modem_atcmd.py:28:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/transport/modem_atcmd.py:60:22: C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck)
pySim/transport/modem_atcmd.py:72:12: W0707: Consider explicitly re-raising using 'except Exception as exc' and 'raise ReaderError('Failed to send AT command: %s' % cmd) from exc' (raise-missing-from)
pySim/transport/modem_atcmd.py:120:12: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/transport/modem_atcmd.py:138:8: W1201: Use lazy % formatting in logging functions (logging-not-lazy)
pySim/transport/modem_atcmd.py:170:12: W0707: Consider explicitly re-raising using 'except Exception as exc' and 'raise ReaderError('Failed to parse response from modem: %s' % rsp) from exc' (raise-missing-from)
pySim/transport/modem_atcmd.py:168:13: W0612: Unused variable 'rsp_pdu_len' (unused-variable)
pySim/transport/modem_atcmd.py:21:0: C0411: standard import "import time" should be placed before "import serial" (wrong-import-order)
pySim/transport/modem_atcmd.py:22:0: C0411: standard import "import re" should be placed before "import serial" (wrong-import-order)
pySim/transport/modem_atcmd.py:23:0: C0411: standard import "import argparse" should be placed before "import serial" (wrong-import-order)
pySim/transport/modem_atcmd.py:24:0: C0411: standard import "from typing import Optional" should be placed before "import serial" (wrong-import-order)
pySim/transport/modem_atcmd.py:28:0: W0614: Unused import(s) NoCardError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: I2c8994eabd973b65132af1030429b1021d0c20df
 2024-02-05 09:52:24 +01:00
Harald Welte
c4d80870e8 pylint: transport/calypso.py 
pySim/transport/calypso.py:27:0: W0401: Wildcard import pySim.exceptions (wildcard-import)
pySim/transport/calypso.py:61:23: W0622: Redefining built-in 'type' (redefined-builtin)
pySim/transport/calypso.py:62:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
pySim/transport/calypso.py:73:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
pySim/transport/calypso.py:27:0: W0614: Unused import(s) NoCardError and SwMatchError from wildcard import of pySim.exceptions (unused-wildcard-import)

Change-Id: I6b10d5f3370c00b07288300b537c6f0e17c84a87
 2024-02-05 09:52:17 +01:00
Harald Welte
f5a8e70f44 pylint: gsm_r.py 
pySim/gsm_r.py:97:0: W0311: Bad indentation. Found 11 spaces, expected 12 (bad-indentation)
pySim/gsm_r.py:32:0: C0411: standard import "from struct import pack, unpack" should be placed before "from pySim.utils import *" (wrong-import-order)
pySim/gsm_r.py:33:0: C0411: third party import "from construct import Struct, Bytes, Int8ub, Int16ub, Int24ub, Int32ub, FlagsEnum" should be placed before "from pySim.utils import *" (wrong-import-order)
pySim/gsm_r.py:34:0: C0411: third party import "from construct import Optional as COptional" should be placed before "from pySim.utils import *" (wrong-import-order)
pySim/gsm_r.py:35:0: C0412: Imports from package pySim are not grouped (ungrouped-imports)
pySim/gsm_r.py:30:0: W0611: Unused import enum (unused-import)
pySim/gsm_r.py:32:0: W0611: Unused pack imported from struct (unused-import)
pySim/gsm_r.py:32:0: W0611: Unused unpack imported from struct (unused-import)
pySim/gsm_r.py:39:0: W0611: Unused import pySim.ts_51_011 (unused-import)

Change-Id: I2a3bba5994d0d4d90fcd3f51bee962fec3a8b0dc
 2024-02-05 09:52:12 +01:00
Harald Welte
fd9188d306 pylint: cat.py 
pySim/cat.py:586:4: W0237: Parameter 'do' has been renamed to 'x' in overriding 'PlmnWactList._from_bytes' method (arguments-renamed)
pySim/cat.py:981:8: W0120: Else clause on loop without a break statement, remove the else and de-indent all the code inside it (useless-else-on-loop)
pySim/cat.py:1000:4: W0221: Number of parameters was 3 in 'TLV_IE_Collection.from_bytes' and is now 2 in overriding 'ProactiveCommand.from_bytes' method (arguments-differ)
pySim/cat.py:1010:12: W0612: Unused variable 'dec' (unused-variable)
pySim/cat.py:1010:17: W0612: Unused variable 'remainder' (unused-variable)
pySim/cat.py:1022:4: W0221: Number of parameters was 2 in 'TLV_IE_Collection.to_bytes' and is now 1 in overriding 'ProactiveCommand.to_bytes' method (arguments-differ)
pySim/cat.py:22:0: C0411: standard import "from typing import List" should be placed before "from bidict import bidict" (wrong-import-order)
pySim/cat.py:26:0: C0411: third party import "from construct import Int8ub, Int16ub, Byte, Bytes, Bit, Flag, BitsInteger" should be placed before "from pySim.utils import b2h, h2b, dec_xplmn_w_act" (wrong-import-order)
pySim/cat.py:27:0: C0411: third party import "from construct import Struct, Enum, Tell, BitStruct, this, Padding, RepeatUntil" should be placed before "from pySim.utils import b2h, h2b, dec_xplmn_w_act" (wrong-import-order)
pySim/cat.py:28:0: C0411: third party import "from construct import GreedyBytes, Switch, GreedyRange, FlagsEnum" should be placed before "from pySim.utils import b2h, h2b, dec_xplmn_w_act" (wrong-import-order)
pySim/cat.py:23:0: W0611: Unused h2b imported from pySim.utils (unused-import)
pySim/cat.py:26:0: W0611: Unused Bit imported from construct (unused-import)
pySim/cat.py:26:0: W0611: Unused Flag imported from construct (unused-import)
pySim/cat.py:27:0: W0611: Unused Tell imported from construct (unused-import)
pySim/cat.py:27:0: W0611: Unused Padding imported from construct (unused-import)
pySim/cat.py:27:0: W0611: Unused RepeatUntil imported from construct (unused-import)

Change-Id: I0c6327a7a8045736e8678b7286a7ed685c96fb71
 2024-02-05 09:52:07 +01:00
Harald Welte
6088b554ef pylint: app.py 
pySim/app.py:113:0: C0305: Trailing newlines (trailing-newlines)
pySim/app.py:23:0: W0611: Unused NoCardError imported from pySim.exceptions (unused-import)

Change-Id: I3cac6892f4d3da66f116cecd49f751da227528a4
 2024-02-05 09:51:59 +01:00
Harald Welte
eb18ed08b0 pylint: ts_31_102_telecom.py 
pySim/ts_31_102_telecom.py:45:0: C0325: Unnecessary parens after '=' keyword (superfluous-parens)
pySim/ts_31_102_telecom.py:33:0: W0401: Wildcard import construct (wildcard-import)
pySim/ts_31_102_telecom.py:76:15: C0121: Comparison 'in_json[srv]['activated'] == True' should be 'in_json[srv]['activated'] is True' if checking for the singleton value True, or 'in_json[srv]['activated']' if testing for truthiness (singleton-comparison)
pySim/ts_31_102_telecom.py:85:23: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_31_102_telecom.py:124:22: W0612: Unused variable 'sw' (unused-variable)
pySim/ts_31_102_telecom.py:32:0: C0411: third party import "from construct import Optional as COptional" should be placed before "from pySim.tlv import *" (wrong-import-order)
pySim/ts_31_102_telecom.py:33:0: C0411: third party import "from construct import *" should be placed before "from pySim.tlv import *" (wrong-import-order)

Change-Id: I4ee0d0e1b5b418b8527b4674141cbaef896a64a2
 2024-02-05 09:51:00 +01:00
Harald Welte
33cd964c1a pylint: profile.py 
pySim/profile.py:169:0: C0325: Unnecessary parens after 'assert' keyword (superfluous-parens)
pySim/profile.py:189:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/profile.py:197:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/profile.py:27:0: C0411: standard import "import abc" should be placed before "from pySim.commands import SimCardCommands" (wrong-import-order)
pySim/profile.py:28:0: C0411: standard import "import operator" should be placed before "from pySim.commands import SimCardCommands" (wrong-import-order)
pySim/profile.py:29:0: C0411: standard import "from typing import List" should be placed before "from pySim.commands import SimCardCommands" (wrong-import-order)

Change-Id: Ifd55e8ab5ab04da06c8d11e50bc15740580b2900
 2024-02-05 09:50:54 +01:00
Harald Welte
e8439d9639 pylint: sms.py 
pySim/sms.py:23:0: W0404: Reimport 'Flag' (imported line 23) (reimported)
pySim/sms.py:54:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:60:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:120:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:132:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:137:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:141:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:188:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:192:8: W0612: Unused variable 'flags' (unused-variable)
pySim/sms.py:209:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:228:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:236:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:279:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:306:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:311:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:319:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:339:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:347:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:373:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-nam

Change-Id: I8082a01443ef568eebda696239572f0af7b56f1b
 2024-02-05 09:50:47 +01:00
Harald Welte
8e7d28cad7 pylint: ota.py 
pySim/ota.py:21:0: W0401: Wildcard import construct (wildcard-import)
pySim/ota.py:129:8: R1705: Unnecessary "elif" after "return", remove the leading "el" from "elif" (no-else-return)
pySim/ota.py:150:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/ota.py:192:8: W0612: Unused variable 'padded_data' (unused-variable)
pySim/ota.py:202:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/ota.py:207:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/ota.py:210:4: C0103: Method name "fromKeyset" doesn't conform to snake_case naming style (invalid-name)
pySim/ota.py:239:8: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/ota.py:242:4: C0103: Method name "fromKeyset" doesn't conform to snake_case naming style (invalid-name)
pySim/ota.py:328:4: W0221: Number of parameters was 4 in 'OtaDialect.encode_cmd' and is now 5 in overriding 'OtaDialectSms.encode_cmd' method (arguments-differ)
pySim/ota.py:392:4: W0221: Number of parameters was 3 in 'OtaDialect.decode_resp' and is now 4 in overriding 'OtaDialectSms.decode_resp' method (arguments-differ)

Change-Id: Icb8d690e541dbaf1406085a8446a0c67641fefff
 2024-02-05 09:44:53 +01:00
Harald Welte
cb4c0cf1e8 pylint: exceptions.py 
pySim/exceptions.py:27:4: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/exceptions.py:32:4: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/exceptions.py:37:4: W0107: Unnecessary pass statement (unnecessary-pass)

Change-Id: Ibd0725ceb5fdcdc0995c39a4449ada2fd6088552
 2024-02-05 09:44:53 +01:00
Harald Welte
c5c9728127 pylint: cards.py 
pySim/cards.py:30:0: W0401: Wildcard import pySim.utils (wildcard-import)
pySim/cards.py:41:8: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/cards.py:55:4: R1711: Useless return at end of function or method (useless-return)
pySim/cards.py:78:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
pySim/cards.py:91:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
pySim/cards.py:159:12: R1705: Unnecessary "else" after "return", remove the "else" and de-indent the code inside it (no-else-return)
pySim/cards.py:28:0: C0411: standard import "import abc" should be placed before "from pySim.ts_102_221 import EF_DIR" (wrong-import-order)
pySim/cards.py:25:0: W0611: Unused Dict imported from typing (unused-import)
pySim/cards.py:28:0: W0611: Unused import abc (unused-import)

Change-Id: I708da28caffb417ed2f8413f9611526b18b29cd4
 2024-02-05 09:44:53 +01:00
Harald Welte
f57912ea15 pylint: gsmtap.py 
pySim/gsmtap.py:28:0: W0401: Wildcard import construct (wildcard-import)
pySim/gsmtap.py:26:0: W0611: Unused List imported from typing (unused-import)
pySim/gsmtap.py:26:0: W0611: Unused Dict imported from typing (unused-import)
pySim/gsmtap.py:26:0: W0611: Unused Optional imported from typing (unused-import)

Change-Id: I53739874edef0a9ae25a8599e7cc7eee9dedb703
 2024-02-05 09:44:53 +01:00
Harald Welte
0f2ac70397 pylint: card_key_provider.py, card_handler.py, iso7816_4.py, jsonpath.py 
pySim/card_key_provider.py:57:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)
pySim/card_key_provider.py:61:0: C0325: Unnecessary parens after 'if' keyword (superfluous-parens)

pySim/card_handler.py:100:0: C0325: Unnecessary parens after '=' keyword (superfluous-parens)
pySim/card_handler.py:100:24: C0121: Comparison 'self.cmds.get('verbose') == True' should be 'self.cmds.get('verbose') is True' if checking for the singleton value True, or 'bool(self.cmds.get('verbose'))' if testing for truthiness (singleton-comparison)
pySim/card_handler.py:29:0: C0411: standard import "import subprocess" should be placed before "from pySim.transport import LinkBase" (wrong-import-order)
pySim/card_handler.py:30:0: C0411: standard import "import sys" should be placed before "from pySim.transport import LinkBase" (wrong-import-order)
pySim/card_handler.py:31:0: C0411: third party import "import yaml" should be placed before "from pySim.transport import LinkBase" (wrong-import-order)

pySim/iso7816_4.py:20:0: W0401: Wildcard import construct (wildcard-import)

pySim/jsonpath.py:1:0: C0114: Missing module docstring (missing-module-docstring)
pySim/jsonpath.py:6:0: W0105: String statement has no effect (pointless-string-statement)
pySim/jsonpath.py:2:0: W0611: Unused import json (unused-import)
pySim/jsonpath.py:3:0: W0611: Unused import pprint (unused-import)

Change-Id: I780595d69000f727ad0fbaff4b89918b91b3122e
 2024-02-05 09:44:18 +01:00
Harald Welte
62bd7d3df2 global_platform: Add DEK (key) encryption support 
Change-Id: I940cc2e16a1d3e3cdef4ebcf3f15fc2c8de21284
 2024-02-05 01:45:02 +01:00
Harald Welte
2bb2ff4aeb global_platform: INSTALL [for install] support 
Change-Id: I4c1da90f1aa8ad9609602272f374078d1e1faa11
 2024-02-05 01:41:49 +01:00
Harald Welte
7156a40187 construct: Add StripTrailerAdapter 
In smart cards, we every so often encounter data types that contain a
bit-mask whose length depends on whether or not there are any of the
least-significant bits are set.  So far we worked around this with
some kind of Struct('byte1', 'byte2'/COptional, 'byte3'/COptional)
approach.

Let's do thisin a generic way using the new StripTrailerAdapter.

Change-Id: I659aa7247c57c680895b0bf8412f9e477fc3587d
 2024-02-05 01:39:39 +01:00
Harald Welte
cd8e16fdfe global_platform: KCV support for PUT KEY 
GlobalPlatform requires the use of the KCV for DES + AES keys. Let's
implement that.

(11.8.2.3.3: "For all key types described in section B.6, the Key Check
Value shall be present.")

Change-Id: Ief168a66dee58b56f4126db12829b3a98906c8db
 2024-02-04 21:27:00 +01:00
Harald Welte
e55fcf66bf Be more conservative in our imports 
Try to avoid '*' from anything into various modules, polluting the
namespace.

Change-Id: Iba749d18e1863ded88ba2d2183e2e8d718b2d612
 2024-02-04 21:27:00 +01:00
Harald Welte
bc8e2e1664 contrib/jenkins.sh: include tests/*.py in pylint 
Change-Id: I9c8113acf5341b198d91040710b6b10cb2b6ef38
 2024-02-04 21:27:00 +01:00
Harald Welte
57f73f8de7 make our tests pass pylint 
Change-Id: If3a9f178c3f915123178efe00269fce74f6e585d
 2024-02-04 21:27:00 +01:00
Harald Welte
af8826a02b Implement Global Platform SCP03 
This adds an implementation of the GlobalPlatform SCP03 protocol. It has
been tested in S8 mode for C-MAC, C-ENC, R-MAC and R-ENC with AES using
128, 192 and 256 bit key lengh.  Test vectors generated while talking to
a sysmoEUICC1-C2T are included as unit tests.

Change-Id: Ibc35af5474923aed2e3bcb29c8d713b4127a160d
 2024-02-04 17:56:59 +01:00
Harald Welte
13a1723c2e rename global_platform.scp02 to global_platform.scp 
This is in preparation of extending it to cover SCP03 in a follow-up
patch.

Change-Id: Idc0afac6e95f89ddaf277a89f9c95607e70a471c
 2024-02-04 17:56:59 +01:00
Harald Welte
afd89ca36d Contstrain argparse integers to permitted range 
In many casese we used type=int permitting any integer value, positive
or negative without a constratint in size.  However, in reality often
we're constrained to unsigned 8 or 16 bit ranges.  Let's use the
auto_uint{8,16} functions to enforce this within argparse before
we even try to encode something that won't work.

Change-Id: I35c81230bc18e2174ec1930aa81463f03bcd69c8
 2024-02-04 17:56:59 +01:00
Harald Welte
a30ee17246 global_platform: Fix --key-id argument 
The key-id is actually a 7-bit integer and on the wire the 8th bit
has a special meaning which can be derived automatically.

Let's unburden the user from explicitly encoding that 8th bit and
instead set it automatically.

Change-Id: I8da37aa8fd064e6d35ed29a70f5d7a0e9060be3a
 2024-02-04 17:56:59 +01:00
Harald Welte
bdf8419966 global_platform: add delete_key and delete_card_content 
This GlobalPlatform command is used to delete applications/load-files
or keys.

Change-Id: Ib5d18e983d0e918633d7c090c54fb9a3384a22e5
 2024-02-04 17:56:59 +01:00
Harald Welte
a7eaefc8d9 global_platform: add set_status command 
Using this command, one can change the life cycle status of on-card
applications, specifically one can LOCK (disable) them and re-enable
them as needed.

Change-Id: Ie14297a119d01cad1284f315a2508aa92cb4633b
 2024-02-04 17:56:59 +01:00
Harald Welte
4d5fd25f31 global_platform: Add install_for_personalization command 
This allows us to perform STORE DATA on applications like ARA-M/ARA-D
after establishing SCP02 to the related security domain.

Change-Id: I2ce766b97bba42c64c4d4492b505be66c24f471e
 2024-02-04 17:56:59 +01:00
Harald Welte
321973ad20 pySim-shell: Make 'apdu' command use logical (and secure) channel 
The 'apdu' command so far bypassed the logical channel and also
the recently-introduced support for secure channels.  Let's change
that, at least by default.  If somebody wants a raw APDU without
secure / logical channel processing, they may use the --raw option.

Change-Id: Id0c364f772c31e11e8dfa21624d8685d253220d0
 2024-02-04 17:43:11 +01:00
Harald Welte
41a7379a4f Introduce GlobalPlatform SCP02 implementation 
This implementation of GlobalPlatform SCP02 currently only supports
C-MAC and C-ENC, but no R-MAC or R-ENC yet.

The patch also introduces the notion of having a SCP instance associated
with a SimCardCommands instance.  It also adds the establish_scp0w and
release_scp shell commands to all GlobalPlatform Security Domains.

Change-Id: I56020382b9dfe8ba0f7c1c9f71eb1a9746bc5a27
 2024-02-04 17:42:30 +01:00
Harald Welte
762a72b308 global_platform 'put_key': constrain ranges of KVN + KID in argparse 
The earlier we catch errors in user input, the better.

Change-Id: Icee656f1373a993b6883ffaab441fe178c0fe8cb
 2024-02-03 13:32:41 +01:00
Harald Welte
a2f1654051 move global_platform.py to global_platform/__init__.py 
This will allow us to have multiple different modules for different
aspects of global_platform.

Change-Id: Ieca0b20c26a2e41eb11455941164474b76eb3c7a
 2024-02-01 12:06:07 +01:00
Harald Welte
eecef54eee commands.py: Wrap the transport send_apdu* methods 
Let's not have higher level code directly call the transports send_apdu*
methods.  We do this as a precursor to introducing secure channel
support, where the secure channel driver would add MAC and/or encrypt
APDUs before they are sent to the transport.

Change-Id: I1b870140959aa8241cda2246e74576390123cb2d
 2024-02-01 12:06:07 +01:00
Harald Welte
5918345c78 global_platform: implement GET STATUS command 
The GlobalPlatform GET STATUS command is used to display information
about ISD / Applications / ExecutabLoad Files / Modules on the card.

Change-Id: Ic92f96c1c6a569aebc93a906c62a43b86fe3b811
 2024-01-31 22:24:42 +01:00
Harald Welte
93bdf00967 pySim.esim: Add class for parsing/encoding eSIM activation codes 
Change-Id: I2256722c04b56e8d9c16a65e3cd94f6a46f4ed85
 2024-01-30 21:33:41 +01:00
Harald Welte
d7715043a3 osmo-smdpp: Add more GSMA TS.48 test profiles 
We now have all v1, v2, v3, v4 and v5 profiles from
https://github.com/GSMATerminals/Generic-eUICC-Test-Profile-for-Device-Testing-Public

Change-Id: Ia204c055b9d04552ae664130ff8ae4fc4163b88c
 2024-01-30 21:33:41 +01:00
Harald Welte
8a39d00cc3 osmo-smdpp: Support multiple different profiles 
Let's simply use the matchingId for filesystem lookup of the UPP file.

This way we can have any number of profiles by simply creating the
respeective files.

Change-Id: I0bc3a14b9fdfcc6322917dd0c69d8295de486950
 2024-01-30 21:33:41 +01:00
Harald Welte
3f3fd1a841 add SAIP template handling + v3.1 definitions 
This adds classes for describing profile templates as well
as derived classes defining the profile templates of the
"Profile Interoperability Technical Specification", specifically
it's "ANNEX A (Normative): File Structure Templates Definition"

We need a machine-readable definition of those templates, so
we can fully interpret an unprotected profile package (UPP),
as the UPP usually only contains the increment/difference to
a given teplate.

Change-Id: I79bc0a480450ca2de4b687ba6f11d0a4ea4f14c8
 2024-01-29 09:23:36 +01:00
Harald Welte
263e3094ba requirements.txt: Switch to osmocom fork of asn1tools 
This is sadly required as the Interoperable Profile format must process
elements of an ASN.1 sequence in order, which doesn't work if the parser
puts the elements in a python dict.

The osmocom fork of asn1tools hence uses OrderedDict to work around
this problem.

Change-Id: Id28fcf060f491bb3d76aa6d8026aa76058edb675
 2024-01-29 09:21:57 +01:00
Harald Welte
e815e79db9 esim.saip: More type annotations 
Change-Id: Ib549817ee137bab610aea9c89a5ab86c2a7592ea
 2024-01-29 09:21:53 +01:00
Harald Welte
9f55da998f esim.saip: Move OID to separate sub-module 
This helps us to prevent circular imports in follow-up code.

Change-Id: I94f85f2257d4702376f4ba5eb995a544a2e53fd3
 2024-01-29 08:06:12 +01:00
Harald Welte
488427993d saip.personalization: Fix ICCID fillFileContent replacement 
Change-Id: Ic267fdde3b648b376ea6814783df1e90ea9bb9ad
 2024-01-28 22:03:11 +01:00
Harald Welte
0bce94996f saip.personalization: Also drop any fillFileOffset 
When replacing a file's contents, we must not just remove any
fillFileContent tuples, but also the fillFileOffset.

Change-Id: I3e4d97ae9de8a78f7bc0165ece5954481568b800
 2024-01-28 22:03:11 +01:00
Harald Welte
3d6df6ce13 [cosmetic] ara_m: Give a spec reference for the PERM-AR-DO 
PERM-AR-DO actually originates in a different spec than all other parts
of the ara_m.py, so let's explicitly mention that.

Change-Id: I6e0014c323f605860d0f70cd0c04d7e461e8a9de
 2024-01-27 20:50:12 +00:00
Harald Welte
7f2263b4a0 runtime: Reset selected_file_fcp[_hex] if SELECT returns no data 
In case SELECT doesn't return any response data, we must reset
the lchan.selected_file_fcp* members to None to prevent pySim-shell
preventing stale data from the previously selected file.

Change-Id: Ia04b8634e328e604e8df7e8d59b7fd532242d2ca
 2024-01-27 21:47:13 +01:00
Harald Welte
9b1a9d9b2e ara_m: Use GlobalPlatform SELECT decoding 
As the ARA-M applet is a GlobalPlatform applet, its SELECT response
decoding should be used, not the ETSI EUICC TS 102 221 fall-back.

Change-Id: I1a30b88a385f6de663aa837483dd32c0d104856f
 2024-01-27 21:47:13 +01:00
Harald Welte
5e0439f881 ara_m: Permit encoding of empty AID (--aid '') in ARA-M rules 
Encoding an empty AID-REF-DO (4F) is neccessary to achieve the meaning
described in "Secure Element Access Control - Public Release v1.0"
Table 6-1: "Empty: Indicates that the rules to be stored or retrieved
are associated with all SE applications not covered by a specific rule".

Change-Id: Iac6c3d78bc9ce36bac47589e5f7a0cc78e2efc38
 2024-01-27 21:47:13 +01:00
Harald Welte
9fd4bbe42e osmo-smdpp: Constrain selection of CI certificate 
We can only choose a CI certificate which is supported both by the eUICC
as well as which has signed our own SM-DP+ certificates.

Change-Id: I0b9130f06d501ca7d484063d56d606cfdd2544f4
 2024-01-25 19:16:57 +01:00
Harald Welte
18d0a7de96 global_platform: Add shell command for PUT KEY 
This command is used for installation of GlobalPlatform keys.  We only
implement the command without secure messaging at this point, as it is
used during card personalization.  Authentication will later be handled
by generic implementations of SCP02 and/or SCP03.

Change-Id: Icffe9e7743266d7262fbf440dd361b21eed7c5cf
 2024-01-25 19:16:57 +01:00
Harald Welte
280a9a3408 docs: Add missing global_platform store_data command docs 
In If30c5d31b4e7dd60d3a5cfb1d1cbdcf61741a50e we introduced a store_data
comamnd, but forgot to add it to the pySim-shell manual.

Change-Id: I6039818c2c0c5373b4a4ef1e33e152de7fbbd01a
 2024-01-25 19:16:57 +01:00
Harald Welte
e6124b0aba add contrib/eidtool.py: Tool for checking + computing EID checksum 
Change-Id: If6c560a2de51718948fb99b816e080f2aff4d0ed
 2024-01-25 19:16:57 +01:00
Harald Welte
6dadb6c215 docs: Update osmo-smdpp with pointer to sysmoEUICC1-C2T and SGP.26 
Change-Id: Id031ca48549a3c2ac21c93a169262570843d8e2d
 2024-01-25 19:16:57 +01:00
Harald Welte
af87cd544f osmo-smdpp: Implement eUICC + EUM certificate signature chain validation 
Change-Id: I961827c50ed5e34c6507bfdf853952ece5b0d121
 2024-01-22 19:08:09 +01:00
Harald Welte
45b7dc9466 Move X.509 related code from osmo-smdpp to pySim.esim.x509_cert 
Change-Id: I230ba0537b702b0bf6e9da9a430908ed2a21ca61
 2024-01-22 17:57:55 +01:00
Harald Welte
c83a963877 New pySim.esim.x509_cert module for X.509 certificate handling 
Change-Id: Ia8cc2dac02fcd96624dc6d9348f103373eeeb614
 2024-01-22 17:57:55 +01:00
Harald Welte
667d589f20 pySim.utils: Support datetime.datetime in JsonEncoder 
Change-Id: I6223475cec8eb45c6fc4278109ad9dd1cb557800
 2024-01-18 16:58:48 +01:00
Harald Welte
ebb6f7f938 osmo-smdpp: Actually dump Rx/Tx JSON in JSON format and not as python dict 
Change-Id: Ieea3fd2d0f0239acfa6a5c4cfdbfd558d1a3e0ea
 2024-01-18 16:58:48 +01:00
Harald Welte
0311c92e96 Fix encoding of decoded/json data in update_{record_binary}_decoded 
The patch introducing the is_hexstr type into the argparser was
accidentially also introduce in two locations where we actually don't
expect a hex-string.

This is a partial revert of I6426ea864bec82be60554dd125961a48d7751904

Change-Id: I3c3d2a2753aa7a2566a3b1add7ba70c86499d293
Closes: #6331
 2024-01-18 16:58:48 +01:00
Harald Welte
66b337079a pySim-shell: Permit 'reset' command also in unqeuipped stage 
If we are not 'equipped' as we could not detect any known applications
on the card, we used to only permit the 'apdu' command. However, we
should also permit the 'reset' command, as it also is something that's
possible with ever card, even of unknown types.

Change-Id: I23199da727973d7095ac18031f49e1e8423aa287
 2024-01-16 18:48:52 +00:00
Harald Welte
4f3d11b378 euicc: Implement EID checksum verification + computation 
Change-Id: I2cb342783137ee7e4b1be3b14e9c3747316f1995
 2024-01-16 19:04:19 +01:00
Harald Welte
cd18ed0a82 ts_102_221: Better explain 'selected file invalidated' 
Some specs call it 'invalidated', others call it 'deactivated'.  If the
user is unfamiliar with this, the error message about "invalidated"
might not be obvious enough; let's also mention 'deactivated' in the
message and explicitly mention that it needs to be activated before use.

Change-Id: I91488b0e7dc25a8970022b09e575485a4165eefa
 2024-01-16 19:04:10 +01:00
Harald Welte
ecfb09037e global_platform: More definitions to support key loading 
With the definitions from this commit, we can build key loading
TLVs, which is used to load ECC keys into eUICCs.

Change-Id: I853c94d37939ef3dd795f893232b0276a5a4af81
 2024-01-14 18:21:57 +01:00
Harald Welte
1f7a9bd5b4 TLV: Add DGI encoding of "GP Scripting Language Annex B" 
The DGI encoding is specified in Annex B of the
"GlobalPlatform Systems Scripting Language Specification v1.1.0"

which is an "archived" specification that is no longer published
by GlobalPlatform, despite it being referenced from the GlobalPlatform
Card Specification v2.3, which is the basis of the GSMA eSIM
specifications.

For some reason it was the belief of the specification authors that
yet another format of TLV encoding is needed, in addition to the BER-TLV
and COMPREHENSION-TLV used by the very same specifications.

The encoding of the tag is not really specified anywhere, but I've only
seen 16-bit examples.  The encoding of the length is specified and
implemented here accordingly.

Change-Id: Ie29ab7eb39f3165f3d695fcc1f02051338095697
 2024-01-14 17:42:01 +01:00
Harald Welte
d5be46ae7e global_platform: Implement generic store_data command 
Change-Id: If30c5d31b4e7dd60d3a5cfb1d1cbdcf61741a50e
 2024-01-14 17:32:53 +01:00
Harald Welte
7ba09f9392 euicc: Migrate ECASD + ISD-R over to global_platform.CardApplicationSD 
Actually, the GSMA eUICC is a kind of derivative of a GlobalPlatform
card, and the ECASD and ISD-R are security domains.  As such, we
should make them derived classes of global_platform.CardApplicationSD
which means they inherit some of the shared shell_commands etc.

Change-Id: I660e874d9bcbb8c28a64e4ef82dc53bee97aacfc
 2024-01-12 10:02:54 +01:00
Harald Welte
91842b471d Constrain user input to hex-string in argparse 
We do have an is_hexstr function which we should use anywhere
where we expect the user to input a string of hex digits.  This way
we validate the input before running in some random exception.

Change-Id: I6426ea864bec82be60554dd125961a48d7751904
 2024-01-12 10:02:54 +01:00
Harald Welte
d1cc8d0c1d euicc: Fix decoding of SubjectKeyIdentifier. 
There's actually no additional TLV structure inside the Tag 0x04.

Change-Id: Ic922355308747a888083c5b26765d272b6b20bd0
 2024-01-09 23:35:10 +01:00
Harald Welte
f2bcb44ccc pySim.saip.*: Support for parsing / operating on eSIM profiles 
This commit introduces the capability to parse and encode
SimAlliance/TCA "Interoperable Profiles" and apply personalization
operations on them.

Change-Id: I71c252a214a634e1bd6f73472107efe2688ee6d2
 2024-01-09 21:37:12 +00:00
Harald Welte
5bbb144a31 Initial proof-of-concept SM-DP+ for GSMA consumer eSIM RSP 
This commit introduces

* the osmo-smdpp.py program implementing the main procedures and the
  HTTP/REST based ES9+
* python modules for ES8+ and non-volatile RSP Session State storage
* the ASN.1 source files required to parse/encode RSP
* 3GPP test certificates from SGP.26
* an unsigned profile package (UPP) of a SAIP v2.3 TS48 test profile

As I couldn't get the 'Klein' tls support to work, the SM-DP+ code
currently does not support HTTPS/TLS but plan HTTP, so you either have
to modify your LPA to use HTTP instead of HTTPS, or put a TLS proxy in
front.

I have successfully installed an eSIM profile on a test eUICC that
contains certificate/key data within the test CI defined in GSMA SGP.26

Change-Id: I6232847432dc6920cd2bd08c84d7099c29ca1c11
 2024-01-09 21:37:12 +00:00
Harald Welte
e76fae9c4c pySim-shell: Update manual with examples for using with eUICC ISD-R 
Change-Id: I4a0acdad5c7478ee76f92c7610c0e2a5331dea46
 2024-01-08 20:56:32 +00:00
Harald Welte
c499dc79a8 euicc: Fix eUICC list_notifications command 
Prior to this patch, the command would always raise exceptions.

Change-Id: I75a7840c3f4b68bfc164a43908b100dd6e41e575
 2024-01-08 12:10:22 +00:00
Harald Welte
0002789a88 euicc: Fix delete_profile command 
Contrary to {enable,disable}_profile, the delete_profile does not use
the ProfileIdentifier TLV, but directly the Iccid / IsdpAid.

Change-Id: I43e298524048703264e16cbdd0b76d82ba976985
 2024-01-08 12:10:17 +00:00
Harald Welte
cfa62cb95b Allow logger to do lazy evaluation of format strings 
Change-Id: I39d26cdd5b85a61a06fd8c7a9d0a046e398819bd
 2024-01-08 12:10:11 +00:00
Harald Welte
d657708df2 add contrib/unber.py utility 
This tool is a replacement for asn1c 'unber' program with a much more
useful/readable output:
* contains hexadecimal raw tag values
* contains hexdump of value, rather than HTML entities in pseudo-XML

Change-Id: I22c1a461ccba04c2c8caaab7ca29ea6ae76e2ea3
 2024-01-08 11:18:49 +00:00
Harald Welte
242197b53d Add pySim.esim.bsp module implementing BSP (BPP Protection Protocol) 
This is the protocol used for the ES8+ interface between SM-DP+ and the
eUICC in the GSMA eSIM system.

Change-Id: Ic461936f2e68e1e6f7faab33d06acf3063e261e7
 2024-01-07 10:22:04 +01:00
Harald Welte
5b623a1247 ts_102_310: Add file definitions resembling ETSI TS 102 310 (EAP) 
The definitions are not used yet, as one would have to add that
dynamically based on which EF.DIR entries contain the 0x73 discretionary
template.  As I don't have any cards implementing this so far, I'll skip
that part.

Change-Id: I532ff2c94021ab1b4520fe2b6988c8960319d208
 2024-01-04 21:50:38 +01:00
Harald Welte
62e570b620 ts_31_103: Add TLV + construct for EF_NAFKCA 
Change-Id: I124064994eb695790e9a3aff40be8139b3a2f2cf
 2024-01-04 21:27:39 +01:00
Harald Welte
4fe7de8568 ts_31_103: Add construct for EF.GBABP and EF.GBANL 
Change-Id: Ife06f54c2443f3e048bd36f706f309843703403a
 2024-01-04 21:27:39 +01:00
Harald Welte
b0c9ccba66 construct: avoid StreamError exceptions due to files containing all-ff 
In smart cards, files/records containing all-ff means they are simply
not used/initialized.  Let's avoid raising exceptions when interpreting
0xff as length value and reading less bytes as value.

Change-Id: I09c3cb82063fc094eb047749996a6eceff757ea2
 2024-01-04 21:20:19 +01:00
Harald Welte
e13403b206 ts_31_102: Start to use construct for EF.SUCI_Calc_Info 
We cannot fully switch to construct for all of it easily due to
the priority value and the ordering/sorting by priority implemented
in the hand-coded version.  But we can at least migrate the
encode/decode of the hnet_pubkey_list via construct.

Change-Id: I4ad5ea57bab37c2dc218e7752d538aa4cdc36ee3
 2024-01-04 20:13:06 +01:00
Harald Welte
9a48aea263 fileystem/tlv: remove unused imports 
Change-Id: I519c7792c7fbe18be63ddc77d211f0d034afcd1f
 2024-01-02 21:13:30 +01:00
Harald Welte
19d2b93d7e move SUCI sub-classes to EF_SUCI_CalcInfo 
Change-Id: Iea6b176327881ff9414f4fe624e94811f9782927
 2023-12-29 18:51:25 +01:00
Harald Welte
9d607978fa global_platform: Add support for more GET DATA TLVs 
Example:

pySIM-shell (00:MF/ADF.ISD)> get_data extended_card_resources_info
{
    "extended_card_resources_info": [
        {
            "number_of_installed_app": 8
        },
        {
            "free_non_volatile_memory": 354504
        },
        {
            "free_volatile_memory": 10760
        }
    ]
}

Change-Id: I129e43c377b62dae1b9a88a0a2dc9663ac2a97da
 2023-12-29 18:51:25 +01:00
Harald Welte
1c0a249131 commands: Ignore exceptions during READ while UPDATE 
If we are reading a file to check if we can skip the write to conserve
writes, don't treat exceptions as fatal.  The file may well have the
access mode in a way that permits us to UPDATE but not to READ.  Simply
fall-back to unconditional UPDATE in this case.

Change-Id: I7bffdaa7596e63c8f0ab04a3cb3ebe12f137d3a8
 2023-12-29 18:51:25 +01:00
Harald Welte
db1684df04 sysmocom_sja2: Implement EF_CHV files using construct 
this has the advantage of getting the encoder for free (so far we only
had the decoder).  While at it, also add some tests data for the unit
tests.

Change-Id: Ifb8caf5cd96706d7fb6b452d6552b115c0828797
 2023-12-29 18:51:25 +01:00
Harald Welte
ce01f48b00 test_files: Test decoder also with ff-padded input 
It's customary in the SIM card universe to right-pad data with ff bytes.
So far we only test decoders without such padding, which is unrealistic.
Let's also tests the decoders with extra 'ff' padding present.

For some files this doesn't make sense, so we add a _test_no_pad class
attribute that can be spcified to prevent this new "test with ff-padding"
from being executed for the test data of the class.

Change-Id: I7f5cbb4a6f91040fe9adef9da0a1f30f9f156dae
 2023-12-29 18:51:25 +01:00
Harald Welte
bcd261583c tests_files.py: Reduce code duplication 
Change-Id: Ib84a0ae35262a19fce3e688afe8e1678a4c59eba
 2023-12-29 18:51:25 +01:00
Harald Welte
69bdcf5022 Fix TLV_IE_Collection.from_tlv in certain situations 
The existing code used to produce an empty output in situations where a
TLV_IE_Collection would be parsed from a single TLV only with some
additional trailing padding:

>>> from pySim.utils import h2b
>>> from pySim.ts_31_102 import EF_CSGT
>>> t = EF_CSGT.Csgt_TLV_Collection()
>>> t.from_tlv(h2b('8906810300666f6fff'))
[TextCsgType(foo)]
>>> t.to_dict()
[]

This was caused by an early return (actually returning the decoded
result) but *without updating self.children*.

Change-Id: I1c84ccf698c6ff7e7f14242f9aaf7d15ac2239f4
 2023-12-29 18:51:25 +01:00
Harald Welte
a77f7e1eb9 ts_31_102: Implement decoders/encoders for EFs below DF.HNB 
These files are mostly related to CSG (Closed Subscriber Group)
in the context of HomeNodeB (HNB), aka femtocells.

Change-Id: Ie57963381e928e2c1da408ad46549a780056242a
 2023-12-29 18:51:25 +01:00
Harald Welte
6e6caa8b4a support UCS-2 characters in EF.MMSUP, EF.ADN, EF.SPN, EF.PNN, EF.ECC 
Now that we have support for the UCS-2 encoding as per TS 102 221 Annex A,
we can start to make use of it from various file constructs.

As some specs say "Either 7-bit GSM or UCS-2" we also introduce
a related automatic GsmOrUcs2Adapter and GsmOrUcs2String class.

Change-Id: I4eb8aea0a13260a143e2c60fca73c3c4312fd3b2
 2023-12-29 18:51:25 +01:00
Harald Welte
f6fceb8684 Implement convoluted encoding of UCS-2 as per TS 102 221 Annex A 
TS 102 221 Annex A defines three variants of encoding UCS-2 characters
into byte streams in files on UICC cards: One rather simplistic one, and
two variants for optimizing memory utilization on the card.

Let's impelement a construct "Ucs2Adapter" class for this.

Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
 2023-12-29 18:51:21 +01:00
Harald Welte
842fbdb15d add PlmnAdapter for decoding PLMN bcd-strings like 262f01 to 262-01 
The human representation of a PLMN is usually MCC-MNC like 262-01
or 262-001.  Let's add a PlmnAdapter for use within construct, so we
can properly decode that.

Change-Id: I96f276e6dcdb54a5a3d2bcde5ee6dbaf981ed789
 2023-12-28 08:08:54 +01:00
Harald Welte
dffe7af578 Fix enumeration of GlobbalPlatformISDR during card_init() 
We used __subclasses__(), but this only returns the immediate
subclasses and not all further/nested subclasses.  Instead, we must
use the pySim.utils.all_subclasses() function to really get all of them.

The hack to use the method signature of the constructor to determine if
it's an intermediate class didn't work, as even GlobbalPlatformISDR
has a optional argument for non-default AIDs.  So let's introduce an
explicit class attribute for that purpose.

Change-Id: I7fb1637f8f7a149b536c4d77dac92736c526aa6c
 2023-12-27 22:17:38 +01:00
Harald Welte
722c11a7e9 global_platform: Add support for key types of v2.3.1 (including AES) 
Change-Id: Iae30f18435c2b0a349bfd9240b9c7cca06674534
 2023-12-27 15:16:03 +00:00
Harald Welte
45626271cf global_platform: Add TLV test data for Key Information Data 
Change-Id: Ib7b73cb28abea98986a66264a0779263873d7fb2
 2023-12-27 15:15:58 +00:00
Harald Welte
2538dd7621 global_platform: Correctly decode Key Information Data 
The list contains tuples of (key_type, key_length). Let's fix that.

Change-Id: Icf367827d62ed67afa27ee3d0ba9d5cd5bc65c99
 2023-12-27 15:15:54 +00:00
Harald Welte
ee6a951774 Add TLV decoder test data 
This adds some first test data for the new unitdata driven test cases
for the TLV encoder/decoder.

It also fixes a bug in the ts_102_221.FileDescriptor decoder for BER-TLV
structured files which was found and fixed while introducing the test
data.

Related: OS#6317
Change-Id: Ief156b7e466a772c78fb632b2fa00cba2eb1eba5
 2023-12-27 15:15:24 +00:00
Harald Welte
2a36c1b921 data-driven TLV unit data test support 
While we do have the _test_de_encode data driven tests for file
definitions, we don't yet have something similar for derived classes of
BER_TLV_IE. This means that TLVs used outside of the filesystem context
(for example, decoding the SELECT/STATUS response, but also eUICC and
other stuff) do not yet have test coverage.

This commit just adds the related test code, but no test data yet.

Related: OS#6317
Change-Id: Ied85f292bb57fde11dc188be84e3384dc3ff1601
 2023-12-27 15:15:17 +00:00
Harald Welte
a9b21bdb1f tlv: Fix from_dict() symmetry 
the to_dict() method generates a {class_name: value} dictionary,
for both the nested and non-nested case.  However, before this patch,
the from_dict() method expects a plain list of child IE dicts
in the nested case.  This is illogical.

Let's make sure from_dict always expectes a {class_name: value} dict
for both nested and non-nested situations.

Change-Id: I07e4feb3800b420d8be7aae8911f828f1da9dab8
 2023-12-27 15:14:48 +00:00
Harald Welte
a5eb924f9e filesystem: use pySim.utils.build_construct() 
We recently introduced a pySim.utils.build_construct() wrapper around
the raw call of the construct.build() method.  So far, this wrapper
was only used from pySim.tlv, but let's also use it from
pySim.filesystem.

Basically, whenever we use parse_construct(), we should use
build_construct() as the inverse operation.

Change-Id: Ibfd61cd87edc72882aa66d6ff17861a3e918affb
 2023-12-23 17:49:01 +01:00
Harald Welte
a4b9bdf238 pySim-trace_test.sh: Force termcolor to suppress color generation 
on some systems, the output would otherwise contain colored status
words, which in turn mean the test otuput no longer matches the expected
output.

Change-Id: Icb700f6e85a285748e00367a398975aa5e75dec5
 2023-12-23 10:38:21 +01:00
Harald Welte
caef0df663 construct/tlv: Pass optional 'context' into construct decoder/encoder 
The context is some opaque dictionary that can be used by the
constructs; let's allow the caller of parse_construct,  from_bytes,
from_tlv to specify it.

Also, when decoding a TLV_IE_Collection, pass the decode results of
existing siblings via the construct.

Change-Id: I021016aaa09cddf9d36521c1a54b468ec49ff54d
 2023-12-23 09:15:47 +00:00
Harald Welte
188869568a docs/shell: extend the introduction part; link to video presentation 
Change-Id: I77c30921f2b8c002c9dda244656c348c96b41f06
 2023-12-23 09:14:59 +00:00
Harald Welte
324175f8bd additional encode/decode test data for various files 
Change-Id: Ib563a2204922d2013b5f7c5abde0773051e17938
 2023-12-23 08:20:42 +01:00
Harald Welte
5376251993 31.102 + 51.011: Fix encode/decode of EF.CFIS 
The EF.CFIS definition is not identical to EF.ADN, so we cannot recycle
the EF.ADN class to decode EF.CFIS.

Change-Id: Idcab35cbe28332e3c8612bcb90226335b48ea973
 2023-12-23 08:20:42 +01:00
Harald Welte
542dbf6771 fix encode/decode of xPLMNwAcT 
There are some pretty intricate rules about how GSM and E-UTRAN are
encoded, let's make sure we fully  support both as per 3GPP TS 31.102
Release 17.  As part of this, switch to a sorted list of access technologies,
in order to have a defined order.  This makes comparing in unit tests
much easier.  However, it also means that we need to sort the set
when printing the list of AcT in pySim-read to generate deterministic
output.

Change-Id: I398ac2a2527bd11e9c652e49fa46d6ca8d334b88
 2023-12-23 08:20:42 +01:00
Harald Welte
e45168ef29 test/test_files: set maxDiff attribute 
Without this the diff between expected and actual output is truncated
and one instead reads the following output:

	Diff is 844 characters long. Set self.maxDiff to None to see it.

We actually want to see the full diff to see what's not matching.

Change-Id: I6e89705061454191b6db1255de7fe549ad720800
 2023-12-22 09:13:10 +01:00
Harald Welte
2822dca9ec tests: use case-insensitive compare of hex strings 
Change-Id: I080f6e173fec40c27dd3ebbf252eaddf5a0e15ba
 2023-12-22 09:13:10 +01:00
Harald Welte
0ecbf63a02 transport: Extend the documentation for each transport driver 
This driver description we add to the code is automatically added to the
respective user manual sections.

Change-Id: I8807bfb11f43b167f1321d556e09ec5234fff629
 2023-12-21 12:33:12 +01:00
Harald Welte
baec4e9c81 transport: Move printing of reader number/name to generic code 
Let's avoid copy+pasting print statements everywhere.  The instances
do already have a __str__ method for the purpose of printing their name in a
generic way.

Change-Id: I663a9ea69bf7e7aaa6502896b6a71ef692f8d844
 2023-12-21 12:33:12 +01:00
Harald Welte
ad002797e2 transport/pcsc: Allow opening PC/SC readers by a regex of their name 
Opening PC/SC readers by index/number is very error-prone as the order
is never deterministic in any system with multiple (hot-plugged, USB)
readers.  Instead, let's offer the alternative of specifying a regular
expression to match the reader name (similar to remsim-bankd).

Change-Id: I983f19c6741904c1adf27749c9801b44a03a5d78
 2023-12-21 12:33:12 +01:00
Harald Welte
0f177c1d29 transport: Pass argparse.Namespace directly into transport classes 
It's odd that the individual transport driver specifies their argparse
options but then the core transport part evaluates them individually.
This means we cannot add new options within a transport.

Let's pass the Namespace instance into the constructor of the
specific transport to improve this.

Change-Id: Ib977007dd605ec9a9c09a3d143d2c2308991a12c
 2023-12-21 11:31:57 +00:00
Harald Welte
c108595041 move {enc,dec}_addr_tlv functions from pySim.util to pySim.legacy.util 
In the previous commit we've stopped using those functions from modern
pySim-shell code.  Hence, the only remaining user is the legacy tools,
so we can move the code to the legacy module.

Change-Id: I6f18ccb36fc33bc204c01f9ece135676510e67ec
 2023-12-17 10:46:31 +00:00
Harald Welte
301d6ed14a isim: Replace legacy imperative address TLV encoder/decoder with construct 
We've recently introduced IPv{4,6}Adapter construct classes and can
switch to this instead of using the old imperative encoder/decoder
functions {enc,dec}_addr_tlv().

Aside from code cleanup, this also means we now support the IPv6 address
type in EF.PCSCF.

Change-Id: I4d01ccfe473a8a80fbee33fdcbd8a19b39da85ac
 2023-12-17 10:46:31 +00:00
Harald Welte
b3c46135bb bertlv_parse_len: Fix input data is smaller than num length octets 
This can happen if there's a file with invalid encoding on the card,
such as a tag followed by all-ff.  Let's gracefully ignore it and
return zero bytes as response.

Change-Id: Ic44557368a6034dbf4bb021ab23a57927c22def0
 2023-12-17 10:46:31 +00:00
Harald Welte
6e9ae8a584 usim: Properly decode/encode IPv4 + IPv6 addresses 
use normal textual representation for IPv4 and IPv6 addresses

Change-Id: I2c6c377f4502af37639e555826c85d5dcf602f9b
 2023-12-17 10:46:31 +00:00
Harald Welte
478b5fe8e3 usim: ePDGId + ePDGSelection: Fix encoder/decoder + add test cases 
Change-Id: Idca19b6fdabae6cc708e92c7714fa0903ea5a1ee
 2023-12-17 10:46:31 +00:00
Harald Welte
cdfe1c24af usim: Add EF.ePDGSelection + EF.ePDGSelectionEm support 
Change-Id: I760a394ae1eac5f1175dc9b86c11b4a60671582e
 2023-12-17 10:46:31 +00:00
Harald Welte
5277b5cf2c USIM: add support for EG.ePDGIdEm (Emergency ePDG) 
Change-Id: I71cb7a4b9323f57b96db2d9f12f1567eda63f742
 2023-12-17 10:46:31 +00:00
Philipp Maier
a5707c7dfb filesystem: fix typo 
Change-Id: I721875d302ab69340d56b33102297b56c070465f
 2023-12-13 12:47:36 +01:00
Philipp Maier
82cc7cc11a runtime: refactor file selection methods select and select_file 
The implementation of the methods select and select_file of class
RuntimeLchan is a bit complex. We access the card directly in several
places which makes it difficult to track the state changes. We should
clean this up so that we call self.rs.card.select_adf_by_aid/
self.scc.select_file from a single place only.

This means that the method select uses the method select_file. This
results in a much cleaner implementation. We also should take care
that the important states that we track (selected_file, selected_adf,
etc.) are updated by a single private method. Since the update always
must happen after a select _select_post is a good place to do this.

Related: OS#5418
Change-Id: I9ae213f3b078983f3e6d4c11db38fdbe504c84f2
 2023-12-13 12:47:36 +01:00
Philipp Maier
14bf003dad filesystem: use sort path when selecting an application 
The method build_select_path_to uses the internal file system tree model
to find the path to a given file. This works the same for applications
(ADF) as it works for normal files (EF/DF). However, an application can
be selected anytime from any location in the filesystem tree. There is
no need to select a specific path leading to that application first.
This means that if there is an ADF somewhere in the resulting
inter_path, we may clip everything before that ADF.

Related: OS#5418
Change-Id: I838a99bb47afc73b4274baecb04fff31abf7b2e2
 2023-12-13 12:45:46 +01:00
Philipp Maier
174fd32f17 runtime: explain how file probing works 
We use a trick to probe a file (that does not exist in the local file
model yet). Let's explain further how that works, in particular why we
do not have to upate any state if probing fails.

Change-Id: I2a8af73654251d105af8de1c17da53dfa10dc669
Related: OS#5418
 2023-12-13 09:02:30 +00:00
Harald Welte
b582c3c7ea euicc: Fix TLV IE definitions for SetNickname{Req,Resp} 
The metaclass uese the 'nested' attribute, while the existing code
accidentially used the 'children' attribute.  The latter is used
by instances for actual child classes, while the Class/nested
attribute is for the list of classes whose instancse could be potential
children.

Change-Id: I968bd84d074dcdcec37d99be5d3d4edac9c35a0c
 2023-12-07 23:29:11 +01:00
Harald Welte
c20d442695 euicc: Fix encoding of Lc value in STORE DATA 
The length value "of course" is a hex value, don't use %02u but %02x

This fixes any eUICC command with a Lc > 10 bytes.

Change-Id: I1e1efbfb9916fc43699602cc889cf4b3d42736f2
 2023-12-07 22:46:40 +01:00
Harald Welte
2b6deddcdc euicc: the ICCID TLV object uses bcd-swapped-nibble encoding 
Change-Id: I050f9e0fb128f3e1d472e2330b136a753794a5a1
 2023-12-07 14:21:43 +01:00
Philipp Maier
5482737f31 pySim-shell: don't get trapped in applications without file system 
When we traverse the file system, we may also end up selecting
applications (ADF), which do not support an USIM/ISIM like file system.
This will leave us without the ability to select the MF (or any other
file) again. The only way out is to select the ISIM or USIM application
again to get the access to the file system again.

Change-Id: Ia2fdd65f430c07acb1afdaf265d24c6928b654e0
Related: OS#5418
 2023-12-07 13:21:07 +00:00
Harald Welte
008cdf4664 euicc: Fix encoding of {enable,disable,delete}_profile 
The encoding was missing a "CHOICE" container and missed the
fact that the refreshFlag presence is mandatory for enable+disable.

Change-Id: I12e2b16b2c1b4b01dfad0d1fb485399827f25ddc
 2023-12-07 13:19:52 +00:00
Harald Welte
0f7d48ed69 tlv: Fix encoding of zero-valued TLVs 
If a TLV was elementary (no nested IEs), and it had only a single
integer content whose value is 0, we erroneously encoded that as
zero-length TLV (len=0, no value part):

>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b''
>>> rf.to_tlv()
b'\x81\x00'

After this change it is correct:

>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b'\x00'
>>> rf.to_tlv()
b'\x81\x01\x00'

Change-Id: I5f4c0555cff7df9ccfc4a56da12766d1bf89122f
 2023-12-07 13:19:52 +00:00
Philipp Maier
c038cccdd8 runtime: cosmetic: prnounce file reference data 
One of the most important properties of the RuntimeLchan are the
selected_file/adf properties. Let's reformat the code so that those
properties are more pronounced.

Change-Id: I4aa028f66879b7d6c2a1cd102cda8d8ca5ff48b1
Related: OS#5418
 2023-12-07 12:29:17 +01:00
Philipp Maier
e30456b07a runtime: explain why we may access the card object directly 
When we are in the constructor of RuntimeState, we may/must access the
card object directly. Let's explain why, since it may not be immediately
obvious.

Change-Id: I01f74d5f021d46679d1c9fa83fb8753382b0f88f
Related: OS#5418
 2023-12-07 12:28:57 +01:00
Philipp Maier
b8b61bf8af runtime: do not use the _scc object of the card object to select MF 
The constructor of the RuntimeState object selects the MF befor it does
some other steps. However it does this through the _scc object of the
card object. This method is before we had lchan abstraction, so we
should now use the lchan object like in all other places.

Related: OS#5418
Change-Id: I9a751c0228c77077e3fabb50a9a68e4489e7151c
 2023-12-07 12:28:39 +01:00
Harald Welte
880db37356 flatten_dict_lists(): Don't flatten lists with duplicate keys 
If we have a list of dicts, and we flatten that into a dict: Only
do that if there are no dicts with duplocate key values in the list,
as otherwise we will loose information during the transformation.

Change-Id: I7f6d03bf323a153f3172853a3ef171cbec8aece7
Closes: OS#6288
 2023-12-06 09:02:38 +01:00
Harald Welte
9c38711773 ara_m: Fix encoding of DeviceInterfaceVersionDO 
Ever since commit 30de9fd8ab in July
we are (properly) using snake_case names in the from_dict (to become
bijective with to_dict).   This code was not updated by accident,
creating an exception when using the `aram_get_config`

Change-Id: If216b56b38ab17d13896074aa726278b9ba16923
Related: OS#6119
 2023-12-06 01:07:35 +00:00
Philipp Maier
a1850aeccc filesystem: add flag to tell whether an ADF supports an FS or not 
An ADF may or may not support a file system. For example ADF.ARA-M does
not have any filesystem support, which means the SELECT we may use from
this ADF is limited and an can only select a different application. To
know about this in advance let's add a flag that we set when we
instantiate an ADF.

Change-Id: Ifd0f7c34164685ea18d8a746394e55416fa0aa66
Related: OS#5418
 2023-12-05 17:37:36 +00:00
Harald Welte
4e02436dba perform multiple GET RESPONSE cycles if more data is available 
So far we implemented only one round of "Send the APDU, get SW=61xx,
call GET RESPONSE".  This permitted us to receive only data up to 256
bytes.

Let's extend that to doing multiple rounds, concatenating the result.
This allows us to obtain arbitrary-length data from the card.

See Annex C.1 of ETSI TS 102 221 for examples showing multiple 61xx
iterations.

Change-Id: Ib17da655aa0b0eb203c29dc92690c81bd1300778
Closes: OS#6287
 2023-12-04 21:38:50 +01:00
Philipp Maier
1c207a2499 pySim-shell: Do not use self.lchan.scc when sending raw APDUs. 
When sending raw APDUs, we access the scc (SimCardCommands) object via
the scc member in the lchan object. Unfortunately self.lchan will not be
populated when the rs (RuntimeState) object is missing. This is in
particular the case when no profile could be detected for the card,
which is a common situation when we boostrap an unprovisioned card.

So let's access the scc object through the card object. This is also
more logical since when we send raw APDUs we work below the level of
logical channels.

Change-Id: I6bbaebe7d7a2013f0ce558ca2da7d58f5e6d991a
Related: OS#6278
 2023-11-29 15:24:10 +01:00
Philipp Maier
eb3b0dd379 pySim-shell: refuse to execute a startup script on initialization errors 
When there is an error on initialization (e.g. card not present), we
should not continue to execute a startup script that was passed with the
pySim-shell commandline. Instead we should print a message that the
startup script was ignored due to errors.

Related: OS#6271
Change-Id: I61329988e0e9021b5b0ef8e0819fb8e23cabf38b
 2023-11-24 12:41:18 +01:00
Philipp Maier
f1e1e729c4 app: do not catch exceptions in init_card 
The function init_card catches all exceptions and then returns None
objects for card or rs in case of an error. This does not fit in the
style we pursue in pySim. This is in particular true for library
functions. We want those functions to raise exceptions when something is
wrong, so that we can catch the exception at top level. Let's fix this
for init_card now.

Related: OS#6271
Change-Id: I581125d8273ef024f6dbf3a5db6116be15c5c95d
 2023-11-24 12:41:18 +01:00
iw0
40ef226030 ts_31_102: correct name of EF_ePDGId 
In 31.102 v17.10, file 6ff3 is called "EF_ePDGId". Adjust the spelling to match.

Change-Id: I2c27a7f325f75274e2110eb312b623cf9e7dab47
 2023-11-14 13:18:36 +00:00
Philipp Maier
578cf12e73 runtime: fix tracking of selected_adf 
The class property selected_adf is not updated in all locations where an
ADF is selected, this means that we may loose track of the currently
selected ADF in some locations

Change-Id: I4cc0c58ff887422b4f3954d35c8380ddc00baa1d
Related: OS#5418
 2023-11-09 14:43:08 +00:00
Harald Welte
8fab463e67 pySim-shell: Move init_card() function to new pySim.app module 
The point of this is to move generic code out of pySim-shell.py,
paving the way for more/other executables using the full power of
our class model without having to reinvent the wheel.

Change-Id: Icf557ed3064ef613ed693ce28bd3514a97a938bd
 2023-11-09 12:36:47 +00:00
Harald Welte
2d44f03af2 transport: Log it explicitly if user doesn't specify a reader 
Change-Id: I37e9d62fabf237ece7e49d8f2253c606999d3d02
 2023-11-04 15:48:55 +00:00
Harald Welte
45477a767b Use construct 'Flag' instead of 'Bit' for type descriptions 
It's better for the human reader (and more obvious that it's a boolean
value) if we decode single Bits as True/False instead of 1/0.

Change-Id: Ib025f9c4551af7cf57090a0678ab0f66a6684fa4
 2023-11-04 15:48:44 +00:00
Harald Welte
7be68b2980 sysmocom_sja2: Add some de/encode test vectors 
This increases test coverage and also shows where we so far only
have decoders but no encoders yet

Change-Id: I7932bab7c81a2314c1b9477f50b82a46f24d074e
 2023-11-03 00:43:17 +01:00
Harald Welte
1c849f8bc2 pySim-shell: Reject any non-decimal PIN values 
Don't even send any non-decimal PIN values to the card, but reject
them when parsing the command arguments.

Change-Id: Icec1698851471af7f76f20201dcdcfcd48ddf365
 2023-11-03 00:43:17 +01:00
Harald Welte
977c5925a1 pySim-shell: permit string with spaces for 'echo' command 
before this patch:

pySIM-shell (00:MF)> echo foo bar baz
usage: echo [-h] string
echo: error: unrecognized arguments: bar baz

after this patch:

pySIM-shell (00:MF)> echo foo bar baz
foo bar baz

Change-Id: I1369bc3aa975865e3a8a574c132e469813a9f6b9
 2023-11-03 00:43:17 +01:00
Harald Welte
4e59d89a5d pySim-shell: Validate that argument to 'apdu' command is proper hexstr 
Let's not even send anything to the card if it's not an even number
of hexadecimal digits

Change-Id: I58465244101cc1a976e5a17af2aceea1cf9f9b54
 2023-11-03 00:43:17 +01:00
Harald Welte
f9ea63ea51 pySim-shell: Improved argument validation for verify_adm argument 
Let's make sure we don't even bother to ask the card to verify
anything as ADM1 pin which is not either a sequence of decimal digits
or an even number of hex digits (even number of bytes).

Change-Id: I4a193a3cf63462fad73d145ab1481070ddf767ca
 2023-11-03 00:43:17 +01:00
Harald Welte
469db9393f pySim-shell: Use argparser for verify_adm to support --help 
Let's add a proper argparser instance for the 'verify_adm' command,
avoiding situations where the user types 'verif_adm --help' and then
--help is interpreted as the PIN value, removing one more attempt from
the failed ADM1 counter.

Let's use that opportunity to improve the documentation of the command.

Change-Id: I3321fae66a11efd00c53b66c7890fce84796e658
 2023-11-02 21:46:38 +00:00
Harald Welte
0ba3fd996a pySim-shell: Add copyright statement and link to online manual to banner 
This way the users are reminded where they can go to read the manual.

Change-Id: Ie86822e73bccb3c585cecc818d4462d4ca6e43c2
 2023-11-02 21:46:13 +00:00
Harald Welte
3d16fdd8da docs: shell: Various documentation updates/extensions 
* examples for export, verify_adm, reset, apdu
* explain CSV option for verify_adm
* fix 'tree' example (--help shouldn't be there)

Change-Id: I6ed8d8c5cf268ad3534e988eff9501f388b8d80f
 2023-11-02 21:46:08 +00:00
Harald Welte
aa07ebcdac docs: shell: update output in examples 
pySim-shell output has changed over time, so some examples were
showing outdated content.  Let's update those.

Change-Id: I4058719c32b61689522e90eba37253e8accb8ba5
 2023-11-02 21:46:01 +00:00
Harald Welte
6663218ab8 docs: Fix docstring syntax to avoid warnings 
pySim/tlv.py:docstring of pySim.tlv.IE.from_bytes:1: ERROR: Unknown target name: "part".
pySim/tlv.py:docstring of pySim.tlv.IE.to_bytes:1: ERROR: Unknown target name: "part".

Change-Id: I170176910c4519005b9276dbe5854aaaecb58efb
 2023-11-02 21:45:54 +00:00
Harald Welte
0c25e922be docs: shell: Re-order the command sections/classes 
the generic pysim command should precede those from specs like ISO7816

Change-Id: I11e66757f10cc28fda547244ae09d51dacd70824
 2023-11-02 21:45:48 +00:00
Harald Welte
350cfd822b docs: shell: link to cmd2 documentation 
Change-Id: I532cb33781f95fe847db7fae7a5264b5d9c416de
 2023-11-02 21:44:46 +00:00
Harald Welte
0f2faa59fb docs: shell: By now we have encoders/decoders for most files 
Change-Id: Ia771f9969ae7eb0094d1768af3f7f54cc9d0d581
 2023-11-01 17:26:35 +01:00
Harald Welte
47bb33f937 docs: shell: Clarify various different card support 
Change-Id: Ibf8e3538aa3c954df72c11ec0a2f885031b54b0e
 2023-11-01 17:26:35 +01:00
Philipp Maier
a24755e066 filesystem: fix method build_select_path_to 
The method build_select_path_to chops off the first element of the
current path. This is done to prevent re-selection of the first file in
the current path.

Unfortunately chopping off the first element in the current path does
not work properly in a situation when the current path points to the MF.
This would chop off the first and last element in the list and the for
loop below would run 0 times.

To fix this, let's keep the first element and chop it off from the
resulting path.

Related: OS#5418
Change-Id: Ia521a7ac4c25fd3a2bc8edffdc45ec89ba4b16eb
 2023-10-31 17:25:55 +01:00
Philipp Maier
1da8636c0f runtime: cosmetic: fix formatting of comment 
Change-Id: I4e949a08c1bfab413b82e958a64404390e58148f
 2023-10-31 17:25:51 +01:00
Philipp Maier
4af63dc760 transport: print reader device/number on init 
When we initialize the reader, we currently tell only which type of
interface we are using, but we do not print the reader number or the
device path.

Let's extend the messages so that the path is printed. To prevent
problems with integration-tests, let's also add an environment variable
that we can use to detect when pySim runs inside a integration-test.

Related: OS#6210
Change-Id: Ibe296d51885b1ef5f9c9ecaf1d28da52014dcc4b
 2023-10-26 15:17:07 +00:00
Harald Welte
cbc0bdfaa9 euicc: add some first IoT eUICC commands (GSMA SGP.32) 
this is far from being complete, just some basic first commands
to get the certificates and eIM configuration.

Change-Id: Ie05108e635ed9c6de10f0ba431cb1b13893f6be8
 2023-10-26 15:16:30 +00:00
Harald Welte
884eb551af euicc: Add get_profiles_info command 
Example output:

pySIM-shell (02:MF/ADF.ISD-R)> get_profiles_info
{
    "profile_info_seq": {
        "profile_info": {
            "iccid": "98940462222222222222",
            "isdp_aid": "a0000005591010ffffffff8900001200",
            "profile_state": "enabled",
            "service_provider_name": "foobar",
            "profile_name": "foobar",
            "profile_class": "provisioning"
        }
    }
}

Change-Id: I52d136f99dc0eb29905e7ca0cd0865486d3cf65b
 2023-10-26 15:16:30 +00:00
Harald Welte
268a2025db Initial support for eUICC 
This just adds basic support for the ISD-R application and its
associated STORE DATA command which is used for the ES10x interfaces
between off-card entities and the on-card ISD-R.

Change-Id: Ieab37b083e25d3f36c20f6e9ed3e4bdfdd14a42a
Closes: OS#5637
 2023-10-26 15:16:30 +00:00
Philipp Maier
8c82378bfd transport: move argument parser setup into concrete classes 
The argument parser is set up globally for all LinkBase objects in
__init__.py. Since we tend to have only platform independed code in
__init__.py, we should move the argument parser setup into the
specific LinkBase classes.

Related: OS#6210
Change-Id: I22c32aa81ca0588e3314c3ff4546f6e5092c11df
 2023-10-24 19:28:34 +00:00
Philipp Maier
3077343739 transport: move init message into concrete classes 
In in the module __init__.py we print an init message (which type of
LinkBase class is providing the SimLink). However in __init__.py we tend
to have only platform independed code but the message string can already
be categorized as platform depened. Let's put the init message into the
constructor of the concrete classes of LinkBase.

Related: OS#6210
Change-Id: I0a6dd7deb79a5f3e42b29094a1cf2535075fa430
 2023-10-24 19:28:34 +00:00
Harald Welte
10669f2ddf utils: Fix bertlv_encode_tag() for multi-byte tags 
We used to support only single-byte tags in bertlv_encode_tag,
let's fix that.  The easy option is to simply call bertlv_parse_tag,
as that already supported multi-byte tags.

Change-Id: If0bd9137883c4c8b01c4dfcbb53cabeee5c1ce2b
 2023-10-24 15:10:01 +02:00
Harald Welte
237ddb5bb3 pySim-shell: Include current logical channel in prompt 
Now that pySim-shell can switch between logical channels, let's state
the currently used logical channel in the prompt.

Change-Id: I45781a6fba205eeb4ac7f58d5cb642b7131bdd88
Related: OS#6230
 2023-10-24 15:10:01 +02:00
Harald Welte
20650997e8 pySim-shell: Add 'switch_channel' command 
We've already had the 'open_channel' and 'close_channel' commands,
which were sent to (and acknowledged by) the card.  However,
those commands didn't affect the pySim-shell state, i.e. all
communication would still happen through the default channel '0'.

With this patch we introduce a 'switch_channel' command, using which
the user can determine which of the (previously opened) logical channels
shall be used by pySim-shell.

Change-Id: Ia76eb45c4925882ae6866e50b64d9610bd4d546d
Closes: OS#6230
 2023-10-24 15:10:01 +02:00
Harald Welte
6dd6f3e12c prevent SimCardCommands.select_adf_by_aid bypassing lchan 
Now that pySim-shell is aware of logical channels and issues almost
all of its APDUs on the currently selected channel, we must also make
sure that ADF selection by AID (implemented by the CardBase class)
issues the SELECT on the respective logical channel.

Before this patch, SELECT ADF by AID would always be issued on the
primary logical channel (0), irrespective of the currently active
RuntimeLchan.

Change-Id: Idf05c297e6a2e24ca539408b8912e348c0782bb4
Related: OS#6230
 2023-10-24 15:10:01 +02:00
Harald Welte
46255121e0 pySim-shell: Create + use per-RuntimeLchan SimCardCommands 
This new approach will "fork" separate SimCardCommands instances
for each RuntimeLchan.  Higher-layer code should now always use the
RuntimeLchan.scc rather than the RuntimeState.card._scc in order to
make sure commands use the correct logical channel.

Change-Id: I13e2e871f2afc2460d9fd1cd566de42267c7d389
Related: OS#6230
 2023-10-24 15:10:01 +02:00
Harald Welte
3dfab9dede commands.py: Add support for multiple logical channels. 
Historically we always only had one instance of SimCardCommands, but
with this patch we can now have multiple instances, one for each lchan.

The SimCardCommands class is aware of the logical channel it runs on
and will patch the CLA byte accordingly.

Change-Id: Ibe5650dedc0f7681acf82018a86f83377ba81d30
Related: OS#6230
 2023-10-24 15:10:01 +02:00
Harald Welte
91eeecfbf3 docs: Fix command reference for 'apdu' command 
This fixes the below error during build of the documentation:

pysim/docs/shell.rst:349: ERROR: "<class 'pySim-shell.PySimCommands'>" has no attribute "apdu_cmd_parser"

Change-Id: If89b66a45ea18b5a3fc56bf77b05e679463da5a8
 2023-10-23 22:30:31 +02:00
Harald Welte
49acc06327 RuntimeState: Add type annotation for 'card' argument 
Change-Id: I3c5138a918f7e45aabe3972883714d05ee704877
 2023-10-21 21:47:04 +02:00
Harald Welte
bdf595756e pySim-shell: Create/delete RuntimeLchan objects on open/close of channel 
We already have the open channel and close_channel commands in
pySim-shell. They are sent to the card and acknowledged, respectively.

We also already do have code that can track multiple different logical
channels (the rs.lchan array).  However, this is currently only used by
pySim-trace, and not by pySim-shell.  Let's change that.

Change-Id: Idacee2dc57e8afe85c79bc85b259064e7f5b83a2
Related: OS#6230
 2023-10-21 21:47:04 +02:00
Harald Welte
7997252267 cards.py: Fix type annotation 
The CardBaes 'scc' member refers to a SimCardCommands instance,
not to a LinkBase.

Change-Id: If4c0dfbd8c9a03d1a0bc4129bb3c5d5fa492d4cb
 2023-10-21 21:47:04 +02:00
Philipp Maier
7c0cd0a93b pySim-shell: do not fail when EF.ICCID does not exist 
An eUICC that has no active eSIM profile does not have an ICCID. (The
reason for this is that EF.ICCID is part of the eSIM profile).
Unfortunately pySim-shell insists on reading the ICCID from EF.ICCID on
startup in order to use it as a lookup key for verify_adm later.

To solve the problem, let's add a try/except block around the section
where EF.ICCID is read. In case of failure we set the ICCID to None,

Related: OS#5636
Change-Id: I8d18c5073946c5a6bb1f93be0ce692a599f46f8c
 2023-10-20 20:51:24 +00:00
Harald Welte
509ecf84fa Use keyword argument for file description argument 
While our base classes (TransparentEF / LinFixedEF) always have the
dsecription as 4th argument after "fid, sfid, name", most of the derived
file-specific classes do not share that same argument order.

As seen in the bug fixed by previous Change-Id I7f32c9fd01094620b68b0e54536ecc6cdbe67903
this can have serious consequences.  Let's avoid using unnamed
(positional) arguments for the description text altogether.

Change-Id: Icfb3fd1bae038c54fa14a91aa9f75219d839968c
 2023-10-18 23:32:57 +02:00
Harald Welte
28accc88c3 ts_31_102: Fix initialization of file size 
We were using positional arguments when instantiating instances
of classes like EF_5GS3GPPLOCI with non-default names/fids/...

However, we got the argument order wrong and were passing the
description string in the position of the file size, which causes
exceptions like the following from pySim-trace:

Traceback (most recent call last):
  File "/home/laforge/projects/git/pysim/./pySim-trace.py", line 198, in <module>
    tracer.main()
  File "/home/laforge/projects/git/pysim/./pySim-trace.py", line 125, in main
    inst.process(self.rs)
  File "/home/laforge/projects/git/pysim/pySim/apdu/__init__.py", line 259, in process
    self.processed = method(self.lchan)
  File "/home/laforge/projects/git/pysim/pySim/apdu/ts_102_221.py", line 152, in process_on_lchan
    if self.cmd_dict['offset'] != 0 or self.lr < self.file.size[0]:
TypeError: '<' not supported between instances of 'int' and 'str'

Let's use named initializers for any arguments after the usual "fid, sfid, name"
initial arguments.

Change-Id: I7f32c9fd01094620b68b0e54536ecc6cdbe67903
 2023-10-18 23:21:46 +02:00
Philipp Maier
af4e5bb18c transport: do not catch exceptions in init_reader 
We currently catch any exceptions that may occur when the card reader is
initialized. Then we print the exception string or the exception type
when no string is available. However, a failure during the reader
initialization is usually a severe problem, so a traceback would provde
a lot of helpful information to debug the issue. So lets not catch any
exceptions at this level so that we get the full backtrace.

Related: OS#6210
Change-Id: I4c4807576fe63cf71a7d33b243a3f8fea0b7ff23
 2023-10-16 14:36:02 +02:00
Philipp Maier
58e89eb15a transport: add return type annotation to method __str__ 
The abstract class LinkBase has no return type annotation on its
__str__ method.

Related: OS#6210
Change-Id: I26d3d2714708dbe957704b60d17ba2afa325b2c4
 2023-10-10 12:06:57 +02:00
Philipp Maier
6bfa8a8533 pySim-shell: print device info in case an exception occurs 
When an exception occurs while initializing or handling the card we
print a traceback, but we do not print any info that allows us to
identify the device that was involved when the exception occurred. Let's
include the device path or number in the error message before we print
the traceback.

In order to make it easier to print the device information, let's add a
__str__() method to all of our devices. This method shall return the
device number or path.

Related: OS#6210
Change-Id: I200463e692245da40ea6d5b609bfc0ca02d15bdb
 2023-10-10 11:51:08 +02:00
Philipp Maier
8e03f2f2ed pySim-shell: do not pass failed card object to PysimApp 
When the try block in which we also call init_card() fails, there may be
no card object, so we must not pass the card object to PysimApp in the
except block. This is also no problem, PysimApp will run without the
card object until the user executes do_equip for a second attempt.

Related: OS#6210
Change-Id: I28195f442ce007f05f7610c882bbc4a6520a8ce6
 2023-10-10 11:26:56 +02:00
Philipp Maier
91c971bf82 pySim-prog, pySim-shell do not use global variables 
When __main__ runs different variables get assigned. In particular opts,
scc, sl and ch. Those variables are available in any scope and
technically it is possible to access them. However, lets not do this
since it leads to confusion. Also, pylint will complain about those code
locations.

In pySim-shell.py
- Let's use the proper locations (sl and ch are stored in PysimApp.
- Scc can be assigned in init_card.
- In method walk, the use of the variable opts to call ection_df is wrong,
  lets use **kwargs (see also usage of action_ef).
- The constructor of Cmd2ApduTracer has a parameter cmd2_app, but usese
  the global variable app. Let's use cmd2_app instead.

In pySim-prog.py
- Do not use opts.num in find_row_in_csv_file, use num instead.
- Pass scc to process_card as parameter so that it won't access scc
  in the global scope.

Change-Id: I7f09e9a6a6bfc658de75e86f7383ce73726f6666
Related: OS#6210
 2023-10-09 12:37:47 +02:00
Philipp Maier
37e57e0c45 filesystem: add attribute "leftpad" to class LinFixedEF 
In some cases, the specs do not specify an absolute record length.
Instead there may be only a minimum record length specified. The card
vendor may then chose to use larger record length at will. This usually
is no problem since the data is usually written from the left and the
remaining bytes are padded at the end (right side) of the data. However
in some rare cases (EF.MSISDN, see also 3GPP TS 51.011, section 10.5.5)
the data must be written right-aligned towards the physical record
length. This means that the data is padded from the left in this case.

To fix this: Let's add a "leftpad" flag to LinFixedEF, which we set to
true in those corner cases. The code that updates the record in
commands.py must then check this flag and padd the data accordingly.

Change-Id: I241d9fd656f9064a3ebb4e8e01a52b6b030f9923
Related: OS#5714
 2023-09-07 14:19:26 +02:00
Philipp Maier
0ac4d3c7dc commands: make method verify_binary and verify_record private 
The methods verify_binary and verify_record are only used internally
in class SimCardCommands, they can be both private methods. Also lets
move them above the method that uses them.

Related: OS#5714
Change-Id: I57c9af3d6ff45caa4378c400643b4ae1fa42ecac
 2023-09-07 13:23:08 +02:00
Philipp Maier
4840d4dc8f pySim-shell: fix commandline option -a (verify_adm) 
The commandline option -a, which does an ADM verification on startup,
does no longer work since the verify_adm method is no longer available
in the card base classes (cards.py). Let's use the verify_chv method
from SimCardCommands instead.

Related: RT#68294
Change-Id: Ic1e54d0e9e722d64b3fbeb044134044d47946f7c
 2023-09-06 14:57:55 +02:00
Philipp Maier
3a37ad015c sim-reset-server: fix error printing sw_match_error 
In the last line of the if,elif,else branch, when we print the ApiError
object, we pass the variable sw to str() instead passing it to
ApiError() like we do it in the lines above. This is not correct and
causes strange exceptions.

Related: OS#67094
Change-Id: I5a1d19abeb00c2c9dc26517abc44a5c916f2d658
 2023-09-06 12:59:24 +02:00
Philipp Maier
7d13845285 sim-rest-server: fix REST method info 
The REST megthd info uses deprecated methods to read the ICCID and the
IMSI from the card. However, we can replace those methods by selecting
the files we are interested in manually and then reading them.

Related: RT#67094
Change-Id: Ib0178823abb18187404249cfed71cfb3123d1d74
 2023-08-25 09:52:48 +02:00
Philipp Maier
91b379a039 sim-rest-server: use UiccCardBase instead of UsimCard 
The class UsimCard is deprecated and only still used in very old
legacy applications. let's use the more modern UiccCardBase class
instead.

Related: RT#67094
Change-Id: I3676f033833665751c0d953176eafe175b20c14a
 2023-08-21 18:36:10 +00:00
Philipp Maier
71a3fb8b3a sim-rest-server: do not select ADF.USIM in connect_to_card 
When the function connect_to_card is done, it selects ADF.USIM. This
might be contraproductive in case someone needs to access files on MF
level in one of the REST methods. Instead fo ADF.USIM, let's use MF as a
common ground to start from.

At the moment the only existing REST (info, auth) immediately select
ADF.USIM after calling connect_to_card already, so there are no further
modifications necessary.

Related: RT#67094
Change-Id: I16e7f3c991c83f81989ecc4e4764bb6cc799c01d
 2023-08-21 18:36:10 +00:00
Philipp Maier
a42ee6f99d cards: get rid of method read_iccid 
The method read_iccid in class CardBase should be put back to
legacy/cards.py. The reason for this is that it falls in the same
category like read_imsi, read_ki, etc. We should not use those old
methods in future programs since we have a more modern infrastructure
(lchan) now.

Also pySim-shell.py is the only caller of this method now. It is not
used in any other place.

Related: RT#67094
Change-Id: Ied3ae6fd107992abcc1b5ea3edb0eb4bdcd2f892
 2023-08-21 18:36:10 +00:00
Florian Klink
09ff0e2b43 README.md: sort dependencies, document smpp.pdu 
This dependency is currently only mentioned in requirements.txt, it
makes sense to also document it here.

Change-Id: I89760dd4008829c91fafbd442483d076c92a7ed4
 2023-08-13 12:16:16 +02:00
Florian Klink
83222abf2e setup.py: fix package name 
The package providing the serial python module seems to be called
pyserial, which also matches what's written in requirements.txt.

Change-Id: I71ef6a19a487101e552219f10f2fa6215b966abd
 2023-08-13 12:10:16 +02:00
Philipp Maier
e6cba76a36 pySim-shell: check presence of runtime state before accessing it 
When the command equip (do_equip) is executed, it accesses
self.rs.profile to see if there are any commands that need to be
unregistered before moving on with the card initialization.

However, it may be the case that no runtime state exists at this point.
This is in particular the case when the card is completely empty and
hence no profile is picked and no runtime state exists.

Change-Id: I0a8be66a69b630f1f2898b62dc752a8eb5275301
 2023-08-11 11:28:31 +02:00
Philipp Maier
63e8a18883 pySim-prog_test: fix typo 
Related: OS#6094
Change-Id: I6432ee3ee948fea697067fb3857cb9b83b1f8422
 2023-08-01 16:10:14 +02:00
Philipp Maier
a380e4efbe pySim-trace_test: verify output of pySim-trace.py 
At the moment we only verify that no exceptions occurred but the output
is not yet verfied.

Related: OS#6094
Change-Id: I3aaa779b5bd8f30936c284a80dbdcb2b0e06985c
 2023-08-01 16:10:14 +02:00
Philipp Maier
7124ad1031 pySim-trace_test: fix shebang line 
Related: OS#6094
Change-Id: Ib2d3a4659f5db9772ddcd9a4ae73c04fec1070fc
 2023-08-01 16:01:47 +02:00
Philipp Maier
d62182ca43 runtime: make sure applications are always listed in the same order 
When we print the profile applications. which are not registered in
EF.DIR, we use python sets to subtract the applications which were part
of EF.DIR and hence already listed. Since we use sets the order may be
arbitrary. This is so far not a problem, since the output is meant to be
read by humans, but as soon as we try to use the output for unit-test
verifications we need a consistent order (sorted)

Related: OS#6094
Change-Id: Ie75613910aaba14c27420c52b6596ab080588273
 2023-08-01 15:47:27 +02:00
Philipp Maier
600e284a7b README.md: Add note about pySim-trace.py dependencies 
Related: OS#6094
Change-Id: I2e03f9c827bd6ee73891bba34bd2f2efe3ded7e6
 2023-07-29 08:56:07 +00:00
Philipp Maier
1cdcbe4f57 pysim-test: rename pysim-test.sh to pySim-prog_test.sh 
We now have pySim-shell and pySim-trace. Let's give pysim-test.sh a more
distinctive name so that it is clear to which program it refers.

Related: OS#6094
Change-Id: I438f63f9580ebd3c7cc78cc5dab13c9937ac6e3a
 2023-07-29 08:56:07 +00:00
Philipp Maier
ec9cdb73e7 tests: add test script for pySim-trace 
pySim-trace has no test coverage yet. Let's use a script to run a
GSAMTAP pcacp through it and check that no exceptions are raised.

Related: OS#6094
Change-Id: Icfabfa7c59968021eef0399991bd05b92467d8d2
 2023-07-29 08:56:07 +00:00
Alexander Couzens
c8facea845 Fix the remaining functions using the broken Card.update_ust() call 
Card.update_ust() got replaced by the file operation ust_update().
In addition to Change-Id I7a6a77b872a6f5d8c478ca75dcff8ea067b8203e

Fixes: f8d2e2ba08 ("split pySim/legacy/{cards,utils} from pySim/{cards,utils}")
Change-Id: Ie6405cae37493a2101e5089a8d11766fbfed4518
 2023-07-29 06:23:15 +00:00
Alexander Couzens
2dd59edd74 ARA-M: fix encoding of the PkgRefDO when using aram_store_ref_ar_do 
The command wasn't used the correct dict to allow the encoders to work.

Related: OS#6121
Change-Id: Ic2bc179b413a6b139e07e3e55b93ff921cb020a9
 2023-07-29 06:23:15 +00:00
Alexander Couzens
760e421be5 utils.py: remove superfluous import from itself 
b2h() is already available.

Change-Id: Ied513a08cc8b5091dd467106250f1e6b5067c3a8
 2023-07-29 06:21:54 +00:00
Alexander Couzens
6c5c3f8b2b Reimplement ust_service_activate and ust_service_deactivate for USIM/EF.UST 
Fixes: f8d2e2ba08 ("split pySim/legacy/{cards,utils} from pySim/{cards,utils}")
Change-Id: I7a6a77b872a6f5d8c478ca75dcff8ea067b8203e
 2023-07-28 15:34:40 +00:00
Philipp Maier
8dc2ca2d37 pySim-trace: catch StopIteration exception on trace file end 
When the trace file end is reaced, pyShark raises a StopIteration
exception. Let's catch this exception and exit gracefully.

Related: OS#6094
Change-Id: I6ab5689b909333531d08bf46e5dfea59b161a79e
 2023-07-28 10:36:52 +02:00
Philipp Maier
162ba3af3e pySim-trace: mark card reset in the trace 
The trace log currently does not contain any information about card
resets. This makes the trace difficult to follow. Let's use the
CardReset object to display the ATR in the trace.

Related: OS#6094
Change-Id: Ia550a8bd2f45d2ad622cb2ac2a2905397db76bce
 2023-07-28 10:14:19 +02:00
Philipp Maier
1f46f07e3c utils: tolerate uninitialized fields in dec_addr_tlv 
TLV fields holding an address may still be uninitialized and hence
filled with 0xff bytes. Lets interpret those fields in the same way as
we interpret empty fields.

Related: OS#6094
Change-Id: Idc0a92ea88756266381c8da2ad62de061a8ea7a1
 2023-07-28 10:14:19 +02:00
Philipp Maier
784b947b11 pySim-trace: remove stray debug print 
Related: OS#6094
Change-Id: I5f030a8552a84f721bd12ab4751933fc6eeae256
 2023-07-28 10:14:19 +02:00
Philipp Maier
407c95520f pySim-trace: add commandline option --show-raw-apdu 
The trace log currently only shows the parsed APDU. However, depending
on the problem to investigate it may be required to see the raw APDU
string as well. Let's add an option for this.

Related: OS#6094
Change-Id: I1a3bc54c459e45ed3154479759ceecdc26db9d37
 2023-07-28 10:07:35 +02:00
Philipp Maier
791f80a44f construct: add adapter Utf8Adapter to safely interpret utf8 text 
Uninitialized Files, File records or fields in a File record or File
usually contain a string of 0xff bytes. This becomes a problem when the
content is normally encoded/decoded as utf8 since by the construct
parser. The parser will throw an expection when it tries to decode the
0xff string as utf8. This is especially a serious problem in pySim-trace
where an execption stops the parser.

Let's fix this by interpreting a string of 0xff as an empty string.

Related: OS#6094
Change-Id: Id114096ccb8b7ff8fcc91e1ef3002526afa09cb7
 2023-07-26 17:13:54 +02:00
farhadh
fec721fcb1 Fixed mnc encoding 
According to 3GPP TS 24.008 section 10.5.5.36 PLMN identity of the CN operator

Change-Id: I400435abfa8b67da886fc39c801e1abba39725bf
 2023-07-21 11:09:49 +00:00
Philipp Maier
92b9356ed2 runtime: fix lchan deletion in method reset 
When we perform a reset while multiple channels are open (this is in
particular the case when parsing real world traces with pySim-trace). To
delete those channels during the reset we iterate over the dictionary
using the keys and delete the channels one by one. However, this must
not be done using the keys as index directly. Python will then throw an
exception: "RuntimeError: dictionary changed size during iteration".

Instead using the keys directly we should cast them into a list and then
using that list for the iteration.

Related: OS#6094
Change-Id: I430ef216cf847ffbde2809f492ee9ed9030343b6
 2023-07-21 11:52:10 +02:00
Philipp Maier
7d86fe1d8a apdu/ts_102_221: extract channel number from dict before calling del_lchan 
When the method del_lchan is called, closed_channel_nr still contains a dict
that contains the channel number under the key 'logical_channel_number'.
This will lead to an exception. We must extact the channel number from
the dict before we can use it with del_lchan. (See also
created_channel_nr)

Related: OS#6094
Change-Id: I399856bc227f17b66cdb4158a69a35d50ba222a7
 2023-07-20 15:50:16 +00:00
Philipp Maier
cfb665bb3f pySim-shell: fix verify_adm command 
The comman verify_adm does no longer work since the verify_adm method is
no longer available in the card base classes (cards.py). Let's use the
verify_chv method from SimCardCommands instead.

Change-Id: Ic87e1bff221b10d33d36da32b589e2737f6ca9cd
 2023-07-20 17:36:08 +02:00
Philipp Maier
3175d61eb2 cards: fix swapped PIN mapping number 
The constant for _adm_chv_num is swapped. It should be 0x0A, rather than
0xA0

Change-Id: I5680d2deee855ef316a98058e8c8ff8cf4edbbb2
 2023-07-20 17:36:04 +02:00
Harald Welte
38306dfc04 pySim-shell: Add a mode where a pySim-shell cmd can be passed by shell 
This adds a new operation mode for pySim-shell, where a single command
can be passed to pySim-shell, which then is executed before pySim-shell
terminates.

Example: ./pySim-shell.py -p0 export --json

Change-Id: I0ed379b23a4b1126006fd8f9e7ba2ba07fb01ada
Closes: OS#6088
 2023-07-12 22:05:14 +02:00
Harald Welte
531894d386 move Runtime{State,Lchan} from pySim.filesystem to new pySim.runtime 
Those two are really separate concepts, so let's keep them in separate
source code files.

Change-Id: I9ec54304dd8f4a4cba9487054a8eb8d265c2d340
 2023-07-12 22:05:14 +02:00
Harald Welte
b77063b9b7 pySim/filesystem.py: remove unused class FileData 
Change-Id: I62eb446e4995a532227a45c8cc521f5f80535d93
 2023-07-12 22:05:14 +02:00
Harald Welte
6ad9a247ef pySim-shell: Iterate over CardApplication sub-classes 
Rather than having to know and explicitly list every CardApplication,
let's iterate over the __subclasses__ of the CardApplication base class.

Change-Id: Ia6918e49d73d80acfaf09506e604d4929d37f1b6
 2023-07-12 22:05:14 +02:00
Harald Welte
2d5959bf47 ts_102_221: Remove CardProfileUICCSIM 
This profile has always been a hack/work-around for the situation that
a classic GSM SIM is not a UICC, and we didn't yet have the concept of
CardProfileAddons yet, so there was no way to probe and add something
to an UICC which was not an application with its own AID/ADF.

Since now we have CardProfileAddons (including one for GSM SIM),
and pySim-trace (the other user of CardProfileUICCSIM) has also switched
over to using CardProfileUICC + addons, we can remove this work-around.

Change-Id: I45cec68d72f2003123da4c3f86ed6a5a90988bd8
 2023-07-12 22:05:14 +02:00
Harald Welte
323a35043f Introduce concept of CardProfileAddon 
We have a strict "one CardProfile per card" rule.  For a modern UICC
without legacy SIM support, that works great, as all applications
have AID and ADF and can hence be enumerated/detected that way.

However, in reality there are mostly UICC that have legacy SIM, GSM-R
or even CDMA support, all of which are not proper UICC applications
for historical reasons.

So instead of having hard-coded hacks in various places, let's introduce
the new concept of a CardProfileAddon.  Every profile can have any
number of those.  When building up the RuntimeState, we iterate over the
CardProfile addons, and probe which of those are actually on the card.
For those discovered, we add their files to the filesystem hierarchy.

Change-Id: I5866590b6d48f85eb889c9b1b8ab27936d2378b9
 2023-07-12 22:05:14 +02:00
Harald Welte
f9e2df1296 cdma_ruim: Fix unit tests and actually enable them 
As pySim.cdma_ruim was not imported by test_files.py, the unit tests
were apparently never executed and hence didn't pass.  Let's fix both
of those problems.

Change-Id: Icdf4621eb68d05a4948ae9efeb81a007d48e1bb7
 2023-07-12 22:05:14 +02:00
Harald Welte
659d7c11ca cards: all UICC should use sel_ctrl="0400" and SIM "0000" 
Hence move this from the derived classes into the respective base
classes SimCardBase and UiccCardBase

Change-Id: Iad197c2b560c5ea05c54a122144361de5742aafd
 2023-07-12 22:05:14 +02:00
Harald Welte
775ab01a2b cards: cosmetic rename, argument name should be scc, not ssc 
ssc = SimCardCommands

Change-Id: I9d690a0a5b9b49ea342728a29b7d4ed10ac31e4e
 2023-07-12 22:05:14 +02:00
Harald Welte
172c28eba8 cards: All derived of SimCardBase use CLA=A0; all UiccCardBase use CLA=00 
Change-Id: Id61b549f68410631529349ee62b08a102f609405
 2023-07-12 22:05:14 +02:00
Harald Welte
b314b9be34 ts_31_102, ts_31_103: Move legacy-only code to pySim.legacy 
Change-Id: Ifebfbbc00ef0d01cafd6f058a32d243d3696e97e
 2023-07-12 22:05:14 +02:00
Harald Welte
57ad38e661 create pySim.legacy.ts_51_011.py and move legacy code there 
Those old flat dicts indicating FID to string-name mapping have long
been obsoleted by the pySim.filsystem based classes.

Change-Id: I20ceea3fdb02ee70d8c8889c078b2e5a0f17c83b
 2023-07-12 22:05:14 +02:00
Harald Welte
a3961298ef pySim/cards: Add type annotations 
Change-Id: Id5752a64b59097584301c860ebf74d858ed3d240
 2023-07-12 22:05:14 +02:00
Harald Welte
f8d2e2ba08 split pySim/legacy/{cards,utils} from pySim/{cards,utils} 
There are some functions / classes which are only needed by the legacy
tools pySim-{read,prog}, bypassing our modern per-file transcoder
classes.  Let's move this code to the pySim/legacy sub-directory,
rendering pySim.legacy.* module names.

The long-term goal is to get rid of those and have all code use the
modern pySim/filesystem classes for reading/decoding/encoding/writing
any kind of data on cards.

Change-Id: Ia8cf831929730c48f90679a83d69049475cc5077
 2023-07-12 22:03:59 +02:00
Harald Welte
263fb0871c pySim/cards: Split legacy classes away from core SIM + UICC 
This introduces an internal split between
* the code that is shared between pySim-shell and legacy tools, which is
  now in the new class hierarchy {Card,SimCard,UiccCard}Base
* the code that is only used by legacy tools,
  which is using the old class names inherited from the *Base above

All users still go through the legacy {Sim,Usim,Isim}Card classes, they
will be adjusted in subsequent patches.

Change-Id: Id36140675def5fc44eedce81fc7b09e0adc527e1
 2023-07-12 21:35:17 +02:00
Harald Welte
02a7f7441f filesystem: Support selecting MF from MF 
This was currently not handled in build_select_path_to(), resulting in
weird exceptions like 'Cannot determine path from MF(3f00) to MF(3f00)'

Change-Id: I41b9f047ee5dc6b91b487f370f011af994aaca04
 2023-07-11 17:50:48 +02:00
Harald Welte
284efda086 pySim-prog: Also accept 18-digit ICCIDs 
There are cards with 18-digit ICCIDs, so let's be a bit more tolerant.

Change-Id: I5395daeb2e96987335f6f9bf540c28d516001394
 2023-07-11 11:09:00 +02:00
Harald Welte
fdcf3c5702 GlobalPlatform ADF.SD: Add command line reference + error message 
The get_data shell command didn't have any interactive help / syntax,
and no meaningful error message in case an unknown data object name
was specified by the user.  Let's fix that.

Change-Id: I09faaf5d45118635cf832c8c513033aede1427e5
 2023-07-11 08:54:04 +02:00
Harald Welte
a1561fe9ae ts_102_222: Remove unneeded imports 
Change-Id: I0fc54a042f03ecf707fde81a859c7dd65a7009cc
 2023-07-11 08:42:12 +02:00
Harald Welte
f9f8d7a294 pySim/transport: Use newly-defined ResTuple type 
Let's use the newly-added ResTuple type annotation rather than
open-coding it everywhere.

Change-Id: I122589e8aec4bf66dc2e86d7602ebecb771dcb93
 2023-07-11 08:42:12 +02:00
Harald Welte
fdb187d7ff pySim/commands.py: Better type annotations 
Change-Id: I68081b5472188f80a964ca48d5ec1f03adc70c4a
 2023-07-11 08:42:12 +02:00
Harald Welte
ab6897c4cd pySim/transport: More type annotations 
Change-Id: I62e081271e3a579851a588a4ed7282017e56f852
 2023-07-11 08:42:12 +02:00
Harald Welte
f5e26ae954 pySim/utils: define 'Hexstr' using NewType 
This means Hexstr is no longer an alias for 'str', but a distinct
new type, a sub-class of 'str'.

Change-Id: Ifb787670ed0e149ae6fcd0e6c0626ddc68880068
 2023-07-11 08:42:12 +02:00
Harald Welte
2352f2dcdd pySim/tlv.py: Fix TLV_IE_Collection from_dict with nested collections 
This is all quite complicated.  In general, the TLV_IE.to_dict() method
obviously is expected to return a dict (with key equal to the snake-case
name of the class, value to the decode IE value).  This single-entry
dict can then be passed back to the from_dict() method to build the
binary representation.

However, with a TLV_IE_Collection, any TLV_IE can occur any number of
times, so we need an array to represent it (dict would need unique key,
which doesn't exist in multiple instances of same TLV IE).  Hence, the
TLV_IE_Collection.to_dict() method actually returns a list of dicts,
rather than a dict itself.  Each dict in the list represents one TLV_IE.

When encoding such a TLV_IE_Collection back from the list-of-dicts, we
so far didn't handle this special case and tried to de-serialize with
a class-name-keyed dict, which doesn't work.

This patch fixes a regression in the aram_store_ref_ar_do pySim-shell
command which got introduced in Change-Id I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8

While we're fixing it, add some additional comments to why things are
how they are.

Change-Id: Ibdd30cf1652c864f167b1b655b49a87941e15fd5
 2023-07-11 08:42:12 +02:00
Harald Welte
ba955b650e pySim/tlv.py: Don't create an exception from within raise 
An invalid variable used in a raise ValueError() would cause a further
exception, depriving the user of a meaningful error message.

Change-Id: I6eb31b91bd69c311f07ff259a424edc58b57529a
 2023-07-11 08:42:12 +02:00
Harald Welte
30de9fd8ab TLV_IE_Collection: use snake-style names during from_dict() 
The TLV_IE_Collection, just like the individual TLV classes, do
use their snake-style names when converting from binary to dict
using the to_dict() method.  It is inconsistent (and a bug) to
expect the CamelCase names during encoding (from_dict).  After all,
we want the output of to_dict() to be used as input to from_dict().

Change-Id: Iabd1ad98c3878659d123eef919c22ca824886f8a
 2023-07-11 08:42:12 +02:00
iw0
f818acd5eb pySim-shell: Unregister profile commands during equip 
This avoids error messages about re-registering 'AddlShellCommands' commandsets during 'equip()' in the bulk_script command.

Change-Id: I893bb5ae95f5c6e4c2be2d133754e427bc92a33d
 2023-07-09 08:12:28 +00:00
Harald Welte
f4a01472bf pySim-shell: Support USIM specific methods/commands on unknown UICC 
So far, if no known programmable card (like sysmoISIM) has been found,
we were using the SimCard base class.  However, once we detect an UICC,
we should have switched to the UsimCard class, as otherwise the various
methods called by USIM/ISIM specific commands don't exist and we get
weird 'SimCard' object has no attribute 'update_ust' execptions.

The entire auto-detection and the legacy SimCard / UsimCard classes
are showing the legacy of the code base and should probably be
re-architected.  However, let's fix the apparent bug for now.

Change-Id: I5a863198084250458693f060ca10b268a58550a1
Closes: OS#6055
 2023-07-04 21:17:19 +02:00
Harald Welte
fa9f348180 ts_31_103: enable encode tests for files containing single TLV IE 
Now that we have fixed OS#6073 in the previous commit, we can enable
the so-far disabled encoder tests for EF.{DOMAIN,IMPU,IMPI} and
remove associated FIXMEs.

Change-Id: I79bfc5b77122907d6cc2f75605f9331b5e650286
 2023-06-27 09:29:37 +02:00
Harald Welte
579ac3ec0e tlv: Fix IE.from_dict() method 
The existing IE.from_dict() method *supposedly* accepts a dict as
input value, but it actually expects the raw decoded value, unless it is
a nested IE.  This is inconsistent in various ways, and results in a bug
visible at a higher layer, such as files like EF.{DOMAIN,IMPI,IMPU},
which are transparent files containing a single BER-TLV IE.

Decoding such files worked, but re-encoding them did not, due to the
fact that we'd pass a dict to the from_dict method, which then gets
assigned to self.decoded and further passed along to any later actual
encoder function like to_bytes or to_tlv.  In that instance, the dict
might be handed to a self._construct which has no idea how to process
the dict, as it expects the raw decoded value.

Change-Id: I3dd5204510e5c32ef1c4a999258d87cb3f1df8c8
Closes: OS#6073
Related: OS#6072
 2023-06-27 09:29:37 +02:00
Harald Welte
0ec01504ab cosmetic: Implement cmd2.Settable backwards-compat via wrapper class 
Let's avoid too many open-coded if-clauses and simply wrap it in
a compatibility class.

Change-Id: Id234f3fa56fe7eff8e1153d71b9be8a2e88dd112
 2023-06-27 09:29:25 +02:00
Harald Welte
985ff31efa work-around what appears to be a pylint bug 
smpp.pdu.pdu_types.DataCodingScheme.GSM_MESSAGE_CLASS very much exists,
and I can prove that manually in the python shell.  So let's assume this
is a pylint bug and work around it

pySim/sms.py:72:21: E1101: Instance of 'DataCodingScheme' has no 'GSM_MESSAGE_CLASS' member (no-member)

Change-Id: Iab34bae06940fecf681af9f45b8657e9be8cbc7b
 2023-06-27 09:26:28 +02:00
Harald Welte
e126872a29 Fix run-editor bug with cmd2 >= 2.0.0 compatibility 
In cmd2, the upstream authors decided to rename a method in 2.0.0
without providing a backwards compatibility wrapper.  Let's add that
locally.

Change-Id: Iaa17b93db13ba330551799cce5f0388c78217224
Closes: OS#6071
 2023-06-25 08:22:56 +02:00
Harald Welte
721ba9b31f tests: Add new, data-driven OTA tests 
Rather than writing one test class with associated method for each
OTA algorithm / test, let's do this in a data-driven way, where new
test cases just have to provide test data, while the code iterates over
it.

Change-Id: I8789a21fa5a4793bdabd468adc9fee3b6e633c25
 2023-06-18 10:50:50 +02:00
Harald Welte
0b32725f80 Add support for encoding/decoding SMS in TPDU and SMPP format 
This is important when talking OTA with a SIM.

Change-Id: I0d95e62c1e7183a7851d1fe38df0f5133830cb1f
 2023-06-18 10:46:23 +02:00
Harald Welte
7e55569f3a docs: Add section on pySim-trace to user manual 
Change-Id: I5edb222818f00e36ed5b067e0f8d5786f39ae887
 2023-06-13 15:10:25 +00:00
Philipp Maier
e345e1126d pySim-shell: fix reset command 
The API of the lchan object has changed. It no longer features the reset
method used by the pySim-shell reset command. Let's fix this by using
the reset method of the card object.

Change-Id: I55511d1edb97e8fa014724598ec173dd47fe25c1
 2023-06-11 19:11:37 +00:00
Harald Welte
f422eb1886 Add ".py" suffix to sphinx-argparse generated docs 
This is important to produce the right command syntax when generating
command line reference in the user manual.  However, we shouldn't add
this kludge to the individual programs, but only to the documentation
using the :prog: syntax.

Change-Id: I2ec7ab00c63d5d386f187e54755c71ffc2dce429
 2023-06-09 11:50:18 +02:00
Harald Welte
f9a5ba5e0f 31.102: Fix EF.Routing_Indicator for odd number of digits 
The routing indicator is BCD-encoded but has an arbitrary length of
1, 2, 3 or 4 digits.

In order to support the odd lengths of 1 or 3, we must not pad on the
byte level, but on the nibble level. This requires a slight extension of
the Rpad() Adapter.

Change-Id: I6c26dccdd570de7b7a4cd48338068e230340ec7c
Fixes: OS#6054
 2023-06-09 09:19:53 +02:00
Harald Welte
1dce498a67 README: remove redundancy 'Manual' and 'Documentation 
Also, re-order sections oriented more towards the user (Docs first)

Change-Id: I4fc76222a1c22685131cb6926721ce24f0373046
 2023-06-08 21:46:24 +02:00
Harald Welte
555cf6f6db README: rephrase initial section; add HPSIM; programmable vs. standard 
Change-Id: Ied7bce9fc4ebc9a71093ac41d9c1b8e67fe04d7e
 2023-06-08 21:46:24 +02:00
Harald Welte
75e31c5d5b test_ota: Add one first OTA SMS AES128 unit test 
Change-Id: Id4a66bbfaec2d8610e8a7a2c72c0dfd08332edcd
 2023-06-08 17:29:46 +02:00
Harald Welte
19b4a971e9 SJA5: EF.USIM_AUTH_KEY: Display / enforce proper length TUAK K 
The K value in case of TUAK can be 16 or 32 bytes long.  We used to
permit/parse/display 32 bytes even if only 16 bytes was configured.

Let's enforce the correct length of "K".

Fixes: OS#6053
Change-Id: Ia0f9a2138f16dce72f3118001e95baa1c80f23ce
 2023-06-08 17:28:40 +02:00
Harald Welte
7ec822373e ts_31_102: Add shell command for GET IDENTITY 
GET IDENTITY is used in the "SUCI computation on USIM" feature.

Change-Id: I619d397900dbd6565f8f46acdabcee511903830c
 2023-06-07 15:54:17 +00:00
Philipp Maier
621f78c943 serial: return a return code in reset_card() 
The method reset_card does not return a return code, while the
coresponding pcsc implementation does return 1 on success.

Change-Id: I658dd6857580652696b4a77e7d6cfe5778f09eff
 2023-06-07 10:00:52 +00:00
Matan Perelman
60951b0c17 utils: Remove format_xplmn leading zeros in MNC 
Change-Id: I803edafbd892c2b32b884d0b39fed61967a3d68b
 2023-06-07 10:00:07 +00:00
Matan Perelman
777ee9e54d Add FPLMN read and program 
Change-Id: I9ce8c1af691c28ea9ed69e7b5f03f0c02d1f029b
 2023-06-07 10:00:07 +00:00
Harald Welte
1de62c41d7 pySim/apdu/ts_31_102.py: Add Rel17 5G NSWO context for GET IDENTITY 
Change-Id: I6ce5848ca4cf04430be7767e9cb2d18f4c5a5531
 2023-06-07 11:14:07 +02:00
Harald Welte
b0e0dce80a ts_102221: Add "resume_uicc" command 
We've had a "suspend_uicc" command since commit
ec95053249 in 2021, but didn't yet
have the corresponding "resume" pair.

Note that you cannot really execute this in a reasonable way from
within pySim, as it is required to power-cycle the card
between SUSPEND and RESUME, see TS 102 221 Section 11.1.22.3.2

Change-Id: I3322fde74f680e77954e1d3e18a32ef5662759f2
 2023-06-07 11:13:34 +02:00
Harald Welte
659781cbe1 Move "suspend_uicc" command from pySim-shell to ts_102_221.py 
The SUSPEND UICC command is a TS 102 221 (UICC) command, so move
it to the UICC Card Profile.

Also, make sure that any shell command sets specified in the
CardProfile are actually installed during equip().

Change-Id: I574348951f06b749aeff986589186110580328bc
 2023-06-07 11:10:33 +02:00
Philipp Maier
4e5aa304fc ts_31_102: fix typo 
Change-Id: Ic8f93a55b974984472356f48518da91c6a521409
 2023-06-06 19:24:29 +02:00
Harald Welte
c85ae4188f Fix result parsing of "suspend_uicc" 
prior to this patch, the suspend_uicc command would always cause a
python exception as a list of integers was returned by decode_duration rather than a single integer (that can be used with %u format string).

Change-Id: I981e9d46607193176b28cb574564e6da546501ba
 2023-06-06 17:36:39 +02:00
Harald Welte
892526ffd0 pySim-shell: Unregister TS 102 222 commands during 'equip' 
This avoids error messages about re-registering the same TS 102 222
commands during executing the 'equip' command.

Change-Id: I3567247fe84e928e3ef404c07eff8250ef04dfe9
 2023-06-06 17:36:39 +02:00
Harald Welte
e619105249 HPSIM application support 
Support HPSIM as specified in 3GPP TS 31.104

Change-Id: I2729fd2b88cd13c36d7128753ad8d3e3d08a9b52
 2023-06-06 17:36:39 +02:00
Harald Welte
d75fa3f7c9 Switch from pycryptodome to pycryptodomex 
So for some weird historical reasons, the same python module is
available as pycryptodome (Crypto.* namespace) and pycryptodomex
(Cryptodome.* namespace).  See the following information on the project
homepage: https://www.pycryptodome.org/src/installation

To make things extra-weird, Debian choose to package pycryptodomex as
python3-pycryptodome
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886291).

So in order to support both Debian-packaged and differently-installed
packages, let's switch to pycryotodomex on all platforms/installers.

Change-Id: I04daed01f51f9702595ef9f9e0d7fcdf1e4adb62
 2023-06-05 20:58:11 +02:00
Harald Welte
219a5f369c OTA: Fix padding of AES CMAC 
When using AES CMAC for authentication of OTA messages, we must not pad
the user data before calling the CMAC function. This is unlike the DES
MAC, where padding to the DES block size is mandatory.

This bug was discovered when trying to talk OTA with AES to a
sysmoISIM-SJA5.  This patch makes the OTA AES interoperate with the
card.  Also, with this patch the cryptographic results of pySim/ota.py
are identical to those of the java code
org.opentelecoms.gsm0348.impl.crypto.CipheringManager

Change-Id: I4b40b5857f95ccb21c35795abe7a1995e368bac3
 2023-06-03 12:45:35 +00:00
Harald Welte
03650582e0 SJA5: Proper encode/decode of TUAK data in EF.USIM_AUTH_KEY 
Unfortunately, TUAK requires a number of additional (and
differently-sized) parameters, so the format of EF.USIM_AUTH_KEY
differs significantly depending on TUAK or non-TUAK case.

Change-Id: I0dcfe05777510fb34973dc2259b137133d8e199d
 2023-06-03 12:45:35 +00:00
Harald Welte
557c13685e SJA5: Add TUAK + XOR-2G algorithm definitions for EF_[U]SIM_AUTH_KEY 
Change-Id: I62a7255d991fa1ed09a7c9bcf8be4b68acfa61a7
 2023-06-03 12:45:35 +00:00
Harald Welte
954ce95a16 SJA2: Implement DF.SYSTEM/EF.0348_KEY using construct 
This implicitly adds support for JSON->binary encoding, not just
decoding (previous code predating construct support).

Change-Id: I0994d9f66a504dd3c60b43ed5cf6645515dcbc6a
 2023-06-03 12:45:35 +00:00
Harald Welte
ba6d6ab64f ts_31_102: EF_SUPI_NAI: Decode/Encode GLI+GCI as UTF-8 strings 
According to TS 23.003 Section 28.15 and 28.16 both GLI and GCI
are NAI as defined in IETF RFC 7542, which in turn specifies they
are encoded in UTF-8.

Change-Id: I0a82bd0d0a2badd7bc4a1f8de2c3e3c144ee5b12
 2023-06-03 12:45:35 +00:00
Harald Welte
455611c9a3 ts_31_102: Add decoder/encoder for DF.5GS/EF.Routing_Indicator 
This file is rather important for 5G SA operation, so we should have
a proper encoder/decoder in place.

Change-Id: I1b37fdfc2807976880b2cafb61951f08eebeb344
 2023-06-03 12:45:35 +00:00
Tobias Engel
d70ac22618 modem_atcmd: raise ProtocolError instead of ReaderError on CME ERROR 
Also accept ProtocolError in addition to SwMatchError in filesystem.py
when probing for applications

Change-Id: I82b50408328f8eaaee5c9e311c4620d20f930642
 2023-06-02 15:35:43 +00:00
Vadim Yanitskiy
bca01523df setup.py: fix syntax errors (missing commas) 
Change-Id: Ia53a659ad9652d582e2bf4a039a3e18631435072
Fixes: 2b15e315 "setup: add missing pyyaml to setup.py and README.md"
Fixes: 93aac3ab "pySim-shell: fix compatibility problem with cmd2 >= 2.0.0 (Settable)"
 2023-05-28 17:13:21 +07:00
Matan Perelman
c296cb593e cards: Add support for Gialer SIM cards 
Change-Id: Icd2021aec630ac018f66ab565e03112047389e17
 2023-05-27 12:37:16 +02:00
Merlin Chlosta
69b69d4d84 docs: add SUPI/SUCI usage example 
Change-Id: I2908ea9df7e78c596554731085902e2ab7278328
 2023-05-27 12:37:12 +02:00
Harald Welte
0489ae67cf cards.py: support ATR-based detection of sysmoISIM-SJA5 
The cards are 99% software-compatible to the SJA2, so let's just
derive the SJA5 class from the SJA2

Change-Id: I706631baaf447c49904277886bc9a3f6ba3f5532
 2023-05-25 22:23:07 +02:00
Harald Welte
2bee70cbac ts_31_102: Add DF.SAIP support 
DF.SAIP (SIMalliance Interoperable Profile) is not part of 31.102,
but something from the eSIM/eUICC universe of TCA (formerly known as
SIMalliance).  However, as 3GPP does not specify how/where the card
stores the information required for SUCI calculation, the
TCA/SIMalliance standard is the only standard there is.  Some CardOS
start to use this standard even for non-eSIM/eUICC use cases.

Change-Id: Iffb65af335dfdbd7791fca9a0a6ad4b79814a57c
 2023-05-25 09:58:34 +02:00
Harald Welte
24e77a7758 ts_31_102: Fix FID + SFI of EF.MCHPPLMN 
Change-Id: I7e24c904e47cc6f90e90b8634cbed478bd14231f
 2023-05-25 07:55:44 +00:00
Harald Welte
5206429c0c ts_31_102: Fix FID of EF.OPL5G (it's 4F08 instead of 6F08) 
Change-Id: I68c7ad93dabd768d80ae629498aee29d7bab5542
 2023-05-25 07:55:44 +00:00
Harald Welte
04bd5140fd ts_31_102: Fix EF.NIA FID 
The FID in ADF.USIM is different from the FID in DF.GSM.  So while
we can re-use the ts_51_011 EF_NIA class definition, we must pass in
a different fid to the constructor.

Change-Id: Ib414d5b476666e276824266e33b341175a2ee05a
 2023-05-25 07:55:44 +00:00
Harald Welte
33eef850c0 ts_51_011: Fix EF.Phase FID (it's 6FAE, not 6FA3) 
Change-Id: I11df83b17b8d6eaab309908cbee646c888abab0d
 2023-05-25 07:55:44 +00:00
Harald Welte
10a1a0a22e ts_51_011: Fix FID of EF.BCCH 
It's 6F74, not 6F7F! (see TS 51.011 Section 10.3.14)

Change-Id: I9d90fa05a0f926f99a5d4832341cc8a9449df7ae
 2023-05-25 07:55:44 +00:00
Harald Welte
fc67de2219 ts_31_102: Extend from Rel16 to Rel17 
This adds definitions for a variety of files which were added in Release
17 of 3GPP TS 31.102.

Change-Id: I61badc1988b006a1065bdfdcc8a93b758e31f79b
 2023-05-25 07:55:44 +00:00
Harald Welte
c224b3b5f1 ts_51_011: Add sst_service_[de]{activate,allocate} shell commands 
Just like the existing commands for UST/IST: Allow the user to
activate/deactivate individual services.  As EF.SST also contains
information about "allocation" of a service, let's have commands for
allocation and activation.

Change-Id: If959d06248cb1a9d2c0a21cdd40d438726cbc5f0
 2023-05-25 07:55:44 +00:00
Vadim Yanitskiy
ade366d2a9 setup.py: add missing packages for pySim-trace.py 
pySim-trace.py is broken if pySim is installed using setup.py:

  fixeria@DELL:~$ pySim-trace.py
  Traceback (most recent call last):
    File "/usr/bin/pySim-trace.py", line 8, in <module>
      from pySim.apdu import *
  ModuleNotFoundError: No module named 'pySim.apdu'

Change-Id: I371143cb4009db46275ec7a020497b909dcc3b4e
 2023-05-24 09:03:01 +00:00
Vadim Yanitskiy
a793552b4f contrib/jenkins.sh: print pylint version before running it 
Change-Id: Icc96ff16af482581dc97a387bcff1374fbb620f3
Related: OS#6034
 2023-05-23 13:12:41 +02:00
Oliver Smith
e47ea5f2e5 Fix pylint errors 
In a previous patch the dependency on cmd2 was changed from cmd2==1.5 to
cmd2>=1.5. After this was merged, this lead to the docker images getting
rebuilt and now having a higher cmd2 version that gets used in the CI
checks. So while the patch was in review, pylint was actually running
with a lower cmd2 version and was taking different code paths.

Fix for:
pySim-shell.py:30:4: E0611: No name 'fg' in module 'cmd2' (no-name-in-module)
pySim-shell.py:30:4: E0611: No name 'bg' in module 'cmd2' (no-name-in-module)
pySim-shell.py:154:8: E1123: Unexpected keyword argument 'use_ipython' in method call (unexpected-keyword-arg)
pySim-shell.py:171:30: E1120: No value for argument 'settable_object' in constructor call (no-value-for-parameter)
pySim-shell.py:173:30: E1120: No value for argument 'settable_object' in constructor call (no-value-for-parameter)
pySim-shell.py:175:30: E1120: No value for argument 'settable_object' in constructor call (no-value-for-parameter)
pySim-shell.py:176:30: E1120: No value for argument 'settable_object' in constructor call (no-value-for-parameter)

Fixes: f8a3d2b3 ("requirements.txt: allow cmd2 versions greater than 1.5")
Fixes: OS#6034
Change-Id: I182d3a2b87e70ed551a70c88d3d531a36bf53f53
 2023-05-23 13:12:33 +02:00
Philipp Maier
3bcc22f73d README.md: add missing pycryptodome to dependency list 
Change-Id: Ib3cf13a1ad38749ac82d1b36fa32d9c5aba29e1a
 2023-05-17 17:30:49 +02:00
Philipp Maier
2b15e315e2 setup: add missing pyyaml to setup.py and README.md 
Change-Id: I1d35f38b17a315dd58e8dd91a27bfa6c2c85905d
 2023-05-17 17:30:49 +02:00
Philipp Maier
f8a3d2b3db requirements.txt: allow cmd2 versions greater than 1.5 
Since we now have fixed the compatibility issues with recent cmd2
versions, we may allow also versions greater than 1.5 in the
requirements.txt

Change-Id: I87702c5250a3660c84458939167bffdca9c06059
 2023-05-17 17:30:49 +02:00
Harald Welte
961b803ec4 pySim-shell: fix compatibility problem with cmd2 >= 2.3.0 (bg) 
cmd2.fg and cmd2.bg have been deprecated in cmd2 2.3.0 and removed
in cmd2 2.4.0. Let's work around this by a version check.

Related upstream commits:
(See also: https://github.com/python-cmd2/cmd2)

Commit f57b08672af97f9d973148b6c30d74fe4e712d14
Author: Kevin Van Brunt <kmvanbrunt@gmail.com>
Date:   Mon Oct 11 15:20:46 2021 -0400

and

Commit f217861feae45a0a1abb56436e68c5dd859d64c0
Author: Kevin Van Brunt <kmvanbrunt@gmail.com>
Date:   Wed Feb 16 13:34:13 2022 -0500

Change-Id: I9fd32c0fd8f6d40e00a318602af97c288605e8e5
 2023-05-17 17:30:49 +02:00
Harald Welte
c85d4067fd pySim-shell: fix compatibility problem with cmd2 >= 2.0.0 (include_ipy) 
In version 2.0.0, the use_ipython parameter in the Cmd constructor is
renamed to include_ipy. There are still plenty of older cmd2
installations around, so let's work around this using a version check.

See also: https://github.com/python-cmd2/cmd2

Commit: 2397280cad072a27a51f5ec1cc64908039d14bd1
Author: Kevin Van Brunt <kmvanbrunt@gmail.com>
Date: 2021-03-26 18:56:33

This commit is based on pySim gerrit changes:
Ifce40410587c85ae932774144b9548b154ee8ad0
I19d28276e73e7024f64ed693c3b5e37c1344c687

Change-Id: Ibc0e18b002a03ed17933be4d0b4f4e86ad99c26e
 2023-05-17 17:30:49 +02:00
Harald Welte
93aac3abe6 pySim-shell: fix compatibility problem with cmd2 >= 2.0.0 (Settable) 
In cmd2 relase 2.0.0 the constructor of Settable adds a settable_object
parameter, which apparantly was optional at first, but then became
mandatory. Older versions must not have the settable_object parameter
but versions from 2.0.0 on require it. Let's add a version check so that
we stay compatible to cmd2 versions below and above 2.0.0.

See also: https://github.com/python-cmd2/cmd2

Commit 486734e85988d0d0160147b0b44a37759c833e8a
Author: Eric Lin <anselor@gmail.com>
Date:   2020-08-19 20:01:50

and

Commit 8f981f37eddcccc919329245b85fd44d5975a6a7
Author: Eric Lin <anselor@gmail.com>
Date: 2021-03-16 17:25:34

This commit is based on pySim gerrit change:
Ifce40410587c85ae932774144b9548b154ee8ad0

Change-Id: I38efe4702277ee092a5542d7d659df08cb0adeff
 2023-05-17 17:30:49 +02:00
Vadim Yanitskiy
87dd020d5f Add very basic profile for R-UIM (CDMA) cards 
R-UIM (CDMA) cards are pretty much like the normal GSM SIM cards and
"speak" the same 2G APDU protocol, except that they have their own file
hierarchy under MF(3f00)/DF.CDMA(7f25).  They also have DF.TELECOM(7f10)
and even DF.GSM(7f20) with a limited subset of active EFs.  The content
of DF.CDMA is specified in 3GPP2 C.S0023-D.

This patch adds a very limited card profile for R-UIM, including auto-
detecion and a few EF definitions under DF.CDMA.  This may be useful
for people willing to explore or backup their R-UIMs.  To me this was
useful for playing with an R-UIM card from Skylink [1] - a Russian
MNO, which provided 450 MHz CDMA coverage until 2016.

[1] https://en.wikipedia.org/wiki/Sky_Link_(Russia)

Change-Id: Iacdebdbc514d1cd1910d173d81edd28578ec436a
 2023-05-10 00:14:13 +00:00
Vadim Yanitskiy
6b19d80229 ts_51_011: fix EF_ServiceTable: use self for static method 
Even though _bit_byte_offset_for_service() is a @staticmethod, it's
still available via self, just like any non-static method.

Change-Id: I3590dda341d534deb1b7f4743ea31ab16dbd6912
 2023-05-10 00:14:13 +00:00
Vadim Yanitskiy
e63cb2cc4d setup.py: add missing pySim-trace.py' to scripts[] 
Change-Id: I44dfcf48ae22182bd7aaa908559f3d1e1e31acce
 2023-05-05 15:12:17 +07:00
Vadim Yanitskiy
b34f23448c filesystem: define more convenient codec for EF.ACC 
This patch improves the output of the 'read_binary_decoded' command:

pySIM-shell (MF/DF.GSM/EF.ACC)> read_binary_decoded
{
    "ACC0": false,
    "ACC1": false,
    "ACC2": false,
    "ACC3": false,
    "ACC4": false,
    "ACC5": false,
    "ACC6": false,
    "ACC7": false,
    "ACC8": false,
    "ACC9": false,
    "ACC10": false,
    "ACC11": false,
    "ACC12": false,
    "ACC13": false,
    "ACC14": false,
    "ACC15": true
}

And allows to set/unset individual ACCs using 'update_binary_decoded':

pySIM-shell (MF/DF.GSM/EF.ACC)> update_binary_decoded --json-path 'ACC15' 0
"0000"
pySIM-shell (MF/DF.GSM/EF.ACC)> update_binary_decoded --json-path 'ACC8' 1
"0100"
pySIM-shell (MF/DF.GSM/EF.ACC)> update_binary_decoded --json-path 'ACC0' 1
"0101"

Change-Id: I805b3277410745815d3fdc44b9c0f8c5be8d7a10
Related: SYS#6425
 2023-04-18 04:36:34 +07:00
Vadim Yanitskiy
0d80fa9150 pySim-prog.py: fix SyntaxWarning: using is with a literal 
Change-Id: If9460bf827242a1dfc518213e3faa9137a21869a
 2023-04-14 00:11:19 +07:00
Philipp Maier
7b9e24482d pySim-shell: add cardinfo command 
It may sometimes be helpful to get a bit of general information about
the card. To sort out problems it sometimes helps to get an idea what
card type and ICCID pySim-shell has in memory.

Change-Id: If31ed17102dc0108e27a5eb0344aabaaf19b19f9
 2023-03-27 10:37:28 +02:00
Harald Welte
61ef1571f9 pySim-shell.py: add a command for RUN GSM ALGORITHM 
Change-Id: Id7876d83d018aca79253784411d3a9d54a249a0a
 2023-03-22 09:57:32 +00:00
Vadim Yanitskiy
9970f59f4f SimCardCommands.run_gsm(): use send_apdu_checksw() 
Change-Id: Ib713cf8154a3aba72bc5776a8d99ec47631ade28
 2023-03-22 09:57:32 +00:00
Vadim Yanitskiy
1dd5cb540d fix SimCardCommands.run_gsm(): always use CLA=0xa0 
Depending on the card type (SIM or USIM/ISUM), self.cla_byte may
be either 0xa0 or 0x00.  Sending RUN GSM ALGORITHM with CLA=0x00
fails with SW=6985 (Command not allowed), so let's make sure
that we always use CLA=0xa0 regardless of the card type.

Change-Id: Ia0abba136dbd4cdea8dbbc3c4d6abe12c2863680
 2023-03-22 09:57:32 +00:00
Oliver Smith
41fbf12dba gitignore: add manuals related files 
Change-Id: I93a63b33032f93f381b8ef451aecc97d3011ce8c
 2023-03-20 13:30:38 +01:00
Oliver Smith
308d7cdf78 docs/Makefile: don't forward shrink to sphinx 
Adjust the catch-all target at the end of the Makefile that is supposed
to route all unknown targets to sphinx, so it doesn't do this for the
shrink target. The shrink target has recently been added to
Makefile.common.inc in osmo-gsm-manuals, which gets included right above
the catch-all target. So it isn't an unknown target, but for some reason
the sphinx catch-all runs in addition to the shrink target (runs
shrink-pdfs.sh, see output below) and fails. As I did not add the
catch-all logic, preserve it but add an exception for the shrink rule.

Fix for:
  + make -C docs publish publish-html
  make: Entering directory '/build/docs'
  /opt/osmo-gsm-manuals/build/shrink-pdfs.sh _build/latex/osmopysim-usermanual.pdf
  * _build/latex/osmopysim-usermanual.pdf: 272K (shrunk from 336K)
  Running Sphinx v5.3.0

  Sphinx error:
  Builder name shrink not registered or available through entry point

Related: SYS#6380
Change-Id: If2802bb93909aba90debe5e03f3047cec73e2f54
 2023-03-20 12:28:06 +01:00
Harald Welte
0707b80ad3 ts_102_222: Implement support for RESIZE FILE for an EF 
This adds pySim-shell support for the RESIZE FILE command in order
to change the size of linear fixed or transparent EF.

Change-Id: I03fbb683e26231c75f345330ac5f914ac88bbe7a
 2023-03-09 09:49:40 +00:00
Oliver Smith
da1f562294 docs: change upload path for html docs 
Upload it to pysim/master/html instead of latest/pysim.

Related: OS#5902
Change-Id: I0b338bd7d1fb2620d63e651eeb8e40c7d8e722e2
 2023-03-07 12:44:14 +01:00
Harald Welte
a07d509de6 docs: Document the file-specific commands for ADF.USIM/EF.EST 
Change-Id: Iddba9f25ba957f03ca25628a7742fe40fd79c030
 2023-02-23 10:02:49 +01:00
Harald Welte
18b7539925 31.102: EF.EST enables/disables services; name commands accordingly 
EF.EST is the *enabled* services table.  Let's call the shell commands
enable and disable, rather than activate/deactivate.

Change-Id: Iacbdab42bc08e2be38ad7233d903fa7cda0d95b6
 2023-02-23 10:00:51 +01:00
Harald Welte
577312a04e docs: Add reference for various commands 
A number of more recently introduced commands were not yet listed in the
manual, let's fix that.

Change-Id: I39150f55eecb5d8ff48292dc5cc0f9e16dd4398c
 2023-02-23 09:52:44 +01:00
Philipp Maier
8490240ce6 cards: sysmo-isim-sja2: make sure an ADF is present in EF.DIR before selecting it 
sysmo-isim-sja2 may come in different configurations, so some may
intentionally lack ADF.USIM or ADF.ISIM. Since select_adf_by_aid() may
raise an exception when selecting a non existent file we should make
sure that the ADF we intend to select is indeed present. A reliable way
to do this is to check if the application is registered in EF.DIR.

Change-Id: Icf6f6b36f246398af408ec432d493fe3f22963dd
 2023-02-10 18:28:39 +01:00
Harald Welte
865eea68c3 filesystem: add unit tests for encoder/decoder methods 
Lets add test vectors for the per-record/per-file encode/decode of
our various classes for the Elementary Files.

We keep the test vectors as class variables of the respective EF-classes
to ensure implementation and test vectors are next to each other.

The test classes then iterate over all EF subclasses and execute the
decode/encode functions using the test vectors from the class variables.

Change-Id: I02d884547f4982e0b8ed7ef21b8cda75237942e2
Related: OS#4963
 2023-02-01 10:52:23 +01:00
Harald Welte
d2edd414a8 ts_51_011: Fix decoding/encoding of EF_LOCIGPRS 
The P-TMSI signature is a 3-byte value, not a 1-byte value.

Change-Id: I06e8d3efe0b3cf3970159c913acfd2f72280302d
 2023-01-31 17:26:09 +01:00
Harald Welte
caa94b5a81 Assume first record number if caller specifies none 
This fixes a regression introduced in Change-Id
I02d6942016dd0631b21d1fd301711c13cb27962b which added support for
different encoding/decoding of records by their record number.

Change-Id: I0c5fd21a96d2344bfd9551f31030eba0769636bf
 2023-01-31 17:26:09 +01:00
Harald Welte
9b9efb6a7a ts_31_102: Fix several bugs in EF_ECC encoder 
The encoder function apparently was never tested, it didn't match at all
the output of the decoder, not even in terms of the string keys of the
dict.

Change-Id: Id67bc39d52c4dfb39dc7756d8041cbd552ccbbc4
 2023-01-31 17:26:09 +01:00
Harald Welte
136bdb065b ts_51_011: EF_SMSP: Use integer division in ValidityPeriodAdapter 
ValidityPeriodAdapter() must return integer values when encoding a
value, as only integer values can be expressed in the binary format.

Change-Id: I0b431a591ac1761d875b5697a71b6d59241db87d
 2023-01-31 17:26:09 +01:00
Harald Welte
9181a69a55 gsm_r: EF_IC: Network String Table Index is 16bit, not 8bit 
As per EIRENE GSM-R SIM-Card FFFIS, EF_IC conatains records of 1+2+2+2
bytes, the network string table index is 16bit and not 8bit as we
implemented so far.

Change-Id: I9e3d4a48b3cb6fb0ecf887b04c308e903a99f547
 2023-01-31 16:00:20 +00:00
Harald Welte
5924ec4d97 ts_51_011: Improve decoding of SELECT response for classic SIM 
When decoding the SELECT response of a clasic GSM SIM without
UICC functionality, we
* did not decode the record length or number of records
* accidentially reported the EF file_size as available_memory (like DF)

Let's fix those two, and also add a comment on how the output dict
of decode_select_response() should look like.

As a result, code like 'read_records' now knows the number of records
and can iterate over them rather than raising exceptions.

Change-Id: Ia8e890bda74e3b4dacca0673d6e5ed8692dabd87
Closes: OS#5874
 2023-01-27 20:46:08 +01:00
Harald Welte
a1bb3f7147 ts_51_011: Support EF.LND 
This file is a optional file specified by TS 51.011, storing the last
numbers dialled.  As the EIRENE FFFIS for GSM-R SIM refers to this,
we must implement it to have full GSM-R support in pySim.

Change-Id: I3b7d6c7e7504b7cc8a1b62f13e8c0ae83a91d0f0
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
0dc6c201e5 ts_51_011, ts_31_102: point to proper EF_EXTn file 
We're using a shared class to implement the identical file encoding
for EF.{ADN,SDN,MBDN,BDN,FDN,CFIS}.  However, they all point to
different extension files.

Previosly for EF.SDN:
    "ext1_record_id": 255

Now for EF.SDN:
    "ext3_record_id": 255

Change-Id: I5301d41225266d35c05e41588811502e5595520d
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
f11f1308b1 ts_51_011: Implement Extended BCD Coding 
TS 51.011 specifies an "Extended BCD Coding" in Table 12 of Section
10.5.1. It allows to express the '*' and '#' symbols used in GSM
SS and/or USSD codes.

This improves decoding from
    "dialing_nr": "a753b1200f",
to
    "dialing_nr": "*753#1200f",

Change-Id: Ifcec13e9b296dba7bec34b7872192b7ce185c23c
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
9ba68df3cc ts_51_011: Support EF.SDN 
DF.TELECOM/EF.SDN (Service Dialling Numbers) is specified in section
10.5.9 of TS 51.011 and required by EIRENE for GSM-R.

Let's use the pre-existing EF.ADN decoder to decode this file.

Change-Id: If91332b10138096d465a9dccf90744de2c14b2be
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
5b9472db7a ts_51_011: Fix bit-order in EF.VGCSS and EF.VBSS 
Those files contain a bit-mask of active group IDs stored at the
respective positions in EV.VGCS and EF.VBS.  However, the bit-order
of each byte is reversed.

Change-Id: I77674c23823aae71c9504b1a85cd75266edadc6f
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
73a7fea357 gsm_r: Fix byte/nibble ordering of predefined_value1 
Change-Id: Ia0dd8994556548a17a0a3101225c23e804511717
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
6bf2d5f216 gsm_r: EF_Predefined: Decode first record different from others 
In their infinite wisdom, the authors of the EIRENE FFFIS for GSM-R SIM
cards invented yet a new way of encoding data in SIM card files: The
first record of a file may be encoded differently than further records
of files.

This patch implements the feature based on the newly-introduced way by
which we pass the record number to the encoder and decoder methods.

Change-Id: Ib526f6c3c2ac9a945b8242e2e54536628376efc0
Related: OS#5784
 2023-01-27 20:46:08 +01:00
Harald Welte
f6b37af721 Prepare for decoding/encoding records differently based on record number 
In their infinite wisdom, the authors of the EIRENE FFFIS for GSM-R SIM
cards invented yet a new way of encoding data in SIM card files: The
first record of a file may be encoded differently than further records
of files.

Let's add the required infrastructure to pySim so that the encode and
decode methods for record-oriented files get passed in the current
record number.

Change-Id: I02d6942016dd0631b21d1fd301711c13cb27962b
Related: OS#5784
 2023-01-24 20:03:02 +01:00
Harald Welte
8dbf714e96 gsm_r: Fix decoding of EF.FN 
This fixes the below exception when trying to decode records of EF.FN:

EXCEPTION of type 'TypeError' occurred with message: 'unsupported operand type(s) for &: 'str' and 'int''

Change-Id: I3723a0d59f862fa818bea1622fe43a7b56c92847
Related: OS#5784
 2023-01-24 14:37:18 +01:00
Harald Welte
e6d7b14f43 gsm_r: Fix typo (it's EF.FN, not EF.EN) 
Related: OS#5784
Change-Id: I2c97a02973d2a1eda2cea5412391144726bb0525
 2023-01-24 14:37:13 +01:00
Harald Welte
bc7437d3b6 pySim-trace: Also consider SW 91xx as successful 
Change-Id: I9e4170721be30342bdce7fb4beeefd1927263ca6
 2023-01-24 13:50:51 +01:00
Harald Welte
7489947046 pySim-trace: Fix missing MANAGE CHANNEL decode 
old output:

00 MANAGE CHANNEL 01       9110

new output:

00 MANAGE CHANNEL 01       9110 {'mode': 'open_channel', 'created_channel': 1}

Change-Id: Iac5b24c14d2b68d526ab347462b72548b8731b30
 2023-01-24 13:50:51 +01:00
Harald Welte
c95f6e2124 pySim-trace: Add support for reading GSMTAP from pcap files 
So far we supported
* GSMTAP live traces via a UDP socket
* RSPRO traces from pcap files (or live)

We were lacking support for reading GSMTAP stored in pcap, which
is what this patch implements.

Change-Id: I46d42774b39a2735500ff5804206ddcfa545568c
 2023-01-24 13:50:51 +01:00
Philipp Maier
284ac104af cards: also program EF.AD under ADF.USIM 
DF.GSM and ADF.USIM have an EF.AD with nearly the same contents. Usually
there is one file physically present and the other is just a link.
Apparantly this is not always the case for sysmo-ismi-sja2 cards, so
lets program EF.AD in both locations.

Change-Id: Ic9dd4acc8d9a72acbb7376ddf3e2128125d4a8f5
Related: OS#5830
 2023-01-19 10:32:13 +01:00
Philipp Maier
de0cf1648c cards: fix typo 
Change-Id: I81a6074776bdf67b7bea359fe7a24f906936f46d
 2023-01-03 13:30:02 +01:00
Philipp Maier
4237ccfb45 pySim-prog: add python docstring for read_params_csv 
Change-Id: I098ff56ef38208f2f321194625ff4279ece2023c
 2022-12-20 11:33:03 +01:00
Philipp Maier
5f0cb3c5f2 pySim-prog: rename write_parameters function. 
The function name "write_parameters" is very generic and since it is
called during the programming cycle it should be made clear that it is
not about writing parameters to the card.

Change-Id: Idaba672987230d7d0dd500409f9fe0b94ba39370
 2022-12-20 11:33:03 +01:00
Philipp Maier
cbb8c02d25 pySim-prog: make dry-run more realistic 
The process_card function has a dry-run mode where one can test
parameters without actually writing to the card. However, the dry-run
feature also does not perform read operations and connects to the card
reader at a different point in time. Lets be more accurate here and
perform all operations a normal programming cycle would perform but
without calling the card.program() method.

Change-Id: I3653bada1ad26bcf75109a935105273ee9d1525c
 2022-12-20 11:33:03 +01:00
Philipp Maier
0a8d9f05b8 cards: check length of mnc more restrictively 
Since we now ensure that mnc always has a valid length lets make the
check in cards.py more strict.

Related: OS#5830
Change-Id: Iee8f25416e0cc3be96dff025affb1dc11d919fcd
 2022-12-20 11:33:03 +01:00
Philipp Maier
32c0434540 pySim-prog: fix handling of mnclen parameter. 
The handling of the mnclen parameter does not work. Lets fix it so that
it can be used again with CSV and normal card programming. Lets ensure
that depending on the parameter and the defaults it is always ensured
that the mnc string has the correct length so that lower layers can
deduct the length of the mnc properly by the string length of the mnc.

Change-Id: I48a109296efcd7a3f37c183ff53c5fc9544e3cfc
Related: OS#5830
 2022-12-20 11:21:07 +01:00
Philipp Maier
2688ddf459 pySim-prog: clean up csv file reader function 
The function that goes through the CSV file and searches for either IMSI
or ICCID or picks a specific line by number is very hard to read and
understand. Lets clean it up and add useful error messages

Change-Id: I7ae995aa3297e77b983e59c75e1c3ef17e1d7cd4
Related: OS#5830
 2022-12-20 11:12:52 +01:00
Philipp Maier
4f888a0414 sysmocom_sja2: simplify and fix op/opc decoder/encoder 
The decoder/encoder of that decodes the EF.xSIM_AUTH_KEY files has an
overcomplicated handling for op/and opc. There is a condition that
checks if milenage is configured and another one that checks if the
string is recognized as OP or OPc. Both is not correct and seems not to
work (op and opc is always displayed as "null")

The encoder/decoder should focus on the physical file layout and
regardless of any other conriguration the OP/OPc field is physically
present and should be displayd and presented for editing.

Change-Id: I6fa3a07e5e473273498d3f13d4cfa33743b787e1
 2022-12-02 12:38:08 +01:00
Christian Amsüss
5d26311efc OTA: Adjust IV length for AES 
Change-Id: I854c844418244c100c328f9e76c0f37850d3db00
 2022-11-25 04:00:55 +01:00
Oliver Smith
8e45b75711 contrib/jenkins.sh: split test/pylint/docs 
Split the jenkins job up in three parts, so each of them can run in
parallel, and the test part that has to run on a specific node (and
blocks it while running), finishes faster.

Don't install depends of pylint/docs jobs as they will run in docker
and the depends get installed once in the container.

Related: OS#5497
Depends: docker-playground Id5c75725d2fab46b29773fa4f637fa2d73fa7291
Depends: osmo-ci Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
Change-Id: I5245c529db729e209d78a02ab9c917a90d0e0206
 2022-11-04 13:13:14 +01:00
Oliver Smith
0529c1906d docs: allow overriding OSMO_GSM_MANUALS_DIR 
Related: OS#5497
Change-Id: I433217b7aa1cdcddc52a89721e03e44b417bacb1
 2022-10-21 16:24:47 +02:00
Oliver Smith
507b5271ac contrib/jenkins.sh: set PYTHONUNBUFFERED=1 
Make sure all python output is printed immediatelly.

Change-Id: I5d334bbc34e4df39ac54472642299c567894f449
 2022-10-18 16:50:03 +02:00
Vadim Yanitskiy
4e64e72766 Revert "contrib/jenkins.sh: pylint v2.15 is unstable, pin v2.14.5" 
This reverts commit 12175d3588.

The upstream has fixed the regression:

https://github.com/PyCQA/pylint/issues/7375

which was actually not in pylint itself but in its dependency:

https://github.com/PyCQA/astroid/pull/1763
https://github.com/PyCQA/astroid/releases/tag/v2.12.6

Change-Id: I1bf36e0c6db14a10ff4eab57bae238401dbd7fd0
Closes: OS#5668
 2022-09-14 05:00:33 +00:00
Harald Welte
75a58d1a87 Add new pySim.ota library, implement SIM OTA crypto 
This introduces a hierarchy of classes implementing

* ETS TS 102 225 (general command structure)
* 3GPP TS 31.115 (dialects for SMS-PP)

In this initial patch only the SMS "dialect" is supported,
but it is foreseen that USSD/SMSCB/HTTPS dialects can be
added at a later point.

Change-Id: I193ff4712c8503279c017b4b1324f0c3d38b9f84
 2022-09-08 15:45:55 +02:00
Vadim Yanitskiy
7d05e49f11 README.md: update installation instructions for Debian 
Change-Id: Icefa33570a34960a4fff145f3c1b6585d867605c
 2022-09-05 23:15:11 +07:00
Vadim Yanitskiy
98ea2a0f7a README.md: update git URLs (git -> https; gitea) 
Change-Id: Ia86979f656557e442b0f432b0646aa7661c293e9
 2022-09-05 23:15:11 +07:00
Vadim Yanitskiy
0a8d27ad7a README.md: list recent dependencies from requirements.txt 
Change-Id: Ia486dbc7f630c1404e51728b5353cf5a0d643415
 2022-09-05 23:15:11 +07:00
Vadim Yanitskiy
9550a0a45b README.md: fix module name: s/serial/pyserial/ 
Change-Id: I5fd308fb161cd5bd5f702845691296877e523248
 2022-09-05 23:15:11 +07:00
Vadim Yanitskiy
b5eaf14991 README.md,requirements.txt: add missing construct version info 
Change-Id: I90da0df431f0d7dbfa4aa428366fbf0e35db388f
Related: OS#5666
 2022-09-05 23:15:10 +07:00
Vadim Yanitskiy
bdac3f61be Bump minimum required construct version to v2.9.51 
With this version I can get all unittests passing:

  python -m unittest discover tests/

We're passing argument 'path' to stream_read_entire(), which was
added in [1] and become available since v2.9.51.

Change-Id: I4223c83570d333ad8d79bc2aa2d8bcc580156cff
Related: [1] bfe71315b027e18e62f00ec4de75043992fd2316 construct.git
Related: OS#5666
 2022-09-05 23:15:10 +07:00
Vadim Yanitskiy
05d30eb666 construct: use Python's API for int<->bytes conversion 
Argument 'signed' was added in [1] and become available since v2.10.63.
Therefore using bytes2integer() and integer2bytes() from construct.core
bumps the minimum required version of construct to v2.10.63.  For
instance, debian:bullseye currently ships v2.10.58.

There is no strict requirement to use construct's API, so let's use
Python's API instead.  This allows using older construct versions
from the v2.9.xx family.

Change-Id: I613dbfebe993f9c19003635371941710fc1b1236
Related: [1] 660ddbe2d9a351731ad7976351adbf413809a715 construct.git
Related: OS#5666
 2022-09-05 23:15:10 +07:00
Vadim Yanitskiy
7800f9d356 contrib/jenkins.sh: install dependencies from requirements.txt 
Change-Id: I99af496e9a3758ea624ca484f4fbc51b262ffaf4
 2022-09-05 23:15:10 +07:00
Vadim Yanitskiy
7ce04a5a29 contrib/jenkins.sh: execute this script with -x and -e 
-x  Print commands and their arguments as they are executed
  -e  Exit immediately if a command exits with a non-zero status

Change-Id: I13af70ef770936bec00b050b6c4f988e53ee2833
 2022-09-05 23:15:06 +07:00
Vadim Yanitskiy
b3ea021b32 contrib/jenkins.sh: speed up pylint by running multiple processes 
Use multiple processes to speed up pylint.  Specifying -j0 will
auto-detect the number of processors available to use.

On AMD Ryzen 7 3700X this significantly reduces the exec time:

  $ time python -m pylint -j1 ... pySim *.py
  real    0m12.409s
  user    0m12.149s
  sys     0m0.136s

  $ time python -m pylint -j0 ... pySim *.py
  real    0m5.541s
  user    0m58.496s
  sys     0m1.213s

Change-Id: I76d1696c27ddcab358526f807c4a0a7f0d4c85d4
 2022-08-30 17:15:53 +07:00
Vadim Yanitskiy
12175d3588 contrib/jenkins.sh: pylint v2.15 is unstable, pin v2.14.5 
pylint v2.15 is crashing, let's fall-back to a known to work v2.14.5.

Change-Id: Ie29be6ec6631ff2b3d8cd6b2dd9ac0ed8f505e4f
Related: https://github.com/PyCQA/pylint/issues/7375
Related: OS#5668
 2022-08-30 17:12:03 +07:00
Christian Amsüss
59f3b1154f proactive: Send a Terminal Response automatically after a Fetch 
Change-Id: I43bc994e7517b5907fb40a98d84797c54056c47d
 2022-08-21 11:54:33 +00:00
Christian Amsüss
98552ef1bd proactive: Avoid clobbering the output of the command that triggered the FETCH 
Change-Id: I2b794a5c5bc808b9703b4bc679c119341a0ed41c
 2022-08-21 11:54:00 +00:00
Harald Welte
cab26c728c pySim-shell: Use pySim.cat definitions to print decoded proactive cmds 
Register a ProactiveHandler with pySim.transport and call the decoder
from pySim.cat to print a decoded version:

Example usage (exact data only works on my specific card due to the
encrpyted payload):

pySIM-shell (MF/ADF.USIM)> envelope_sms 400881214365877ff6227052000000000302700000201506393535b000118dd46f4ad6b015922f62292350d60af4af191adcbbc35cf4
FETCH: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
SendShortMessage(CommandDetails({'command_number': 1, 'type_of_command': 19, 'command_qualifier': 0}),DeviceIdentities({'source_dev_id': 'uicc', 'dest_dev_id': 'network'}),SMS_TPDU({'tpdu': '410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c'}))
SW: 9000, data: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c

Change-Id: Ia4cdf06a44f46184d0da318bdf67077bc8ac9a1a
 2022-08-06 18:56:42 +02:00
Harald Welte
fd476b4d62 pySim.transport: Add mechanism for handling for CAT/USAT proactive cmds 
This introduces an optional argument to the LinkBase class constructor,
where the application can pass an instance of a ProactiveHandler derived
class in order to handle the proactive commands that the LinkBase is
automatically fetching whenever the card indicates so.

Change-Id: I844504e2fc1b27ce4fc7ede20b2307e698baa0f6
 2022-08-06 18:56:42 +02:00
Harald Welte
5a4891a5b7 Add TLV definitions for *a lot more* CAT / USAT data objects 
This adds deciding for the bulk of the TLV objects used in the
ETSI CAT (Card Application Toolkit) and 3GPP USAT (USIM Application
Toolkit) systems.

This patch just adds the definitions, but doesn't use them anywhere yet.

Change-Id: I0c66912dbc10164e040e2fec358cef13c45a66ec
 2022-08-06 18:56:42 +02:00
Harald Welte
7d8029eb23 tlv: Use self._compute_tag() method rather than direct self.tag 
The TLV_IE.from_tlv() method is part of a base class that is inherited
by more specific classes.  The official way to obtain the tag is the
inherited-class-provided self._compute_tag() method, and *not* a direct
reference to the self.tag member.

This allows for some more obscure TLV parsers, such as the upcoming one
for Proactive Commands in the CAT/OTA context.

Change-Id: I0cd70e31567edc5a0584336efcb5e4282734f6dd
 2022-08-06 13:19:16 +02:00
Harald Welte
f56b6b2a1c ts_31_102: Add missing imports for envelope_sms command 
The envelope_sms command fails due to some missing imports prior to
this patch.

Change-Id: I98e692745e7e1cfbc64b88b248700b1e54915b96
 2022-07-30 16:37:01 +02:00
Harald Welte
51b3abb000 ts_31_102: Fix terminal_profile, envelope and envelope_sms commands 
In commit Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5 I accidentially
broke those commands by adding argparse definitions for better
documentation.  When adding the  @cmd2.with_argparser decorator,
the method argument changes from the raw string to an argparse.Namespace
object.

This patch fixes the below exception:

pySIM-shell (MF/ADF.USIM)> terminal_profile ffffffff
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2129, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2559, in onecmd
    stop = func(statement)
  File "/usr/local/lib/python3.10/dist-packages/cmd2/decorators.py", line 336, in cmd_wrapper
    return func(*args_list, **kwargs)
  File "/space/home/laforge/projects/git/pysim/pySim/ts_31_102.py", line 1274, in do_terminal_profile
    (data, sw) = self._cmd.card._scc.terminal_profile(arg)
  File "/space/home/laforge/projects/git/pysim/pySim/commands.py", line 583, in terminal_profile
    data_length = len(payload) // 2
TypeError: object of type 'Namespace' has no len()

Change-Id: Ia861eeb2970627d3ecfd0ca73f75ca571c6885b2
Fixes: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
 2022-07-30 16:37:01 +02:00
Harald Welte
7416d463a4 Fix printing of SwMatchError after introduction of logical channels 
the interpret_sw() method was moved from RuntimeState to RuntimeLchan
in Change-Id I7aa994b625467d4e46a2edd8123240b930305360 - but the code
in pySim/exceptions.py was not adjusted accordingly.

Change-Id: I0614436c99c6a6ebc22c4dc14fb361c5f5f16686
 2022-07-30 16:37:01 +02:00
Harald Welte
93c34aac89 apdu/ts_102_221: SELECT: allow select of SELF 
While in the pySim-shell, it's useful to filter the currently selected
file from the choice of available files for select, this doesn't apply
for the tracing case: It's perfectly valid for the UE to SELECT the
file that's already selected right now.  The operation basically
becomes equivalent to a STATUS.

Change-Id: I1a20fb3ba70426333ac34448c6cb782c51363965
 2022-07-25 14:25:11 +02:00
Harald Welte
dcc689d9c4 apdu/ts_102_221: SELECT: allow select of parent/ancestor DFs 
We need to pass the 'PARENT' flag to get_selectables() to be able
to track SELECT on any of the parent/ancestor DF FID.

Change-Id: Ia7ac627d5edccb97160c90688d720d887fad6ec7
 2022-07-25 14:25:11 +02:00
Harald Welte
f5ff1b896e filesystem: We can select not just immediate parent DF but all ancestors 
I didn't check the specs, but at least experience with real-world cards
(and modems) shows that it's not just permitted to select the immediate
parent DF, but all ancestors of the currently selected file.

So adjust the get_selectables() method to not just return the immediate
parent, but to recurse all the way up and report the FID of any ancestor
DF.

Change-Id: Ic9037aa9a13af6fb0c2c22b673aa4afa78575b49
 2022-07-25 14:25:11 +02:00
Harald Welte
8e9c844130 apdu/ts_102_221: Fix SELECT of 3f00 
In order to be able to explicitly select the MF via 3f00,
we need to pass the 'MF' to get_selectables(), so the record
is included in the list of selectable files from the current
working directory.

Change-Id: I27085896142fe547a6e93e01e63e59bbc65c8b8a
 2022-07-24 11:56:35 +02:00
Harald Welte
498361f3b5 apdu/ts_102_221: Implement SELECT case "df_ef_or_mf_by_file_id" 
This was [sadly] simply missing from the implementation so far.

Change-Id: I7bbd13ce29f5adc1ca3ca01bffabbe02dd17db20
 2022-07-24 11:56:35 +02:00
Harald Welte
d2c177b396 filesystem.py: Make CardDF.get_selectables() respect the flags 
All other get_selectables() understand a flag like 'FIDS' to request
only the hexadecimal FIDs and not the file names.  However, the
CardEF.get_selectables() ignored those flags and unconditionally
returned the names.

Change-Id: Icdc37cae3eecd36d167da76c30224b9d48c844fd
 2022-07-24 11:56:35 +02:00
Harald Welte
86d698d310 pySim-trace: Don't print argparse object at start-up 
Change-Id: I881471d026457d8ffcfdbd412c7aae0d0bff9344
 2022-07-24 10:23:50 +02:00
Harald Welte
72c5b2d796 pySim-trace: Fix --no-suppress-{select.status} command line arguments 
The Tracer implemented those options and the argparser handled it,
but we didn't ever connect the two.

Change-Id: I7d7d5fc475a8d09efdb63d3d6f1cc1de1996687b
 2022-07-24 10:23:50 +02:00
Harald Welte
c61fbf4daa pySim-trace: Support SELECT with empty response body 
If the modem/UE doesn't ask for the FCP to be returned, a SELECT
can exit with 9000 and no response body.  Don't crash in that case.

Change-Id: I66788717bec921bc54575e60f3f81adc80584dbc
 2022-07-24 09:46:11 +02:00
Harald Welte
04897d5f25 sim-rest-server: Report meaningful error message if PIN is blocked 
Instead of a cryptic backtrace, we now return a meaningful error like this:

{"error": {"message": "Security Status not satisfied - Card PIN enabled?", "status_word": "6982"}

Change-Id: I6dafd37dfd9fa3d52ca2c2e5ec37a6d274ba651b
Closes: OS#5606
 2022-07-23 14:07:00 +02:00
Harald Welte
3f3b45a27b sim-rest-server: Render error messages as JSON 
Let's make sure even error messages are returned in JSON format.

While at it, also reduce some code duplication between the 'auth'
and 'info' route handlers by using the klein handle_errors decorator
instead of manual exception catching.

Change-Id: I1e0364e28ba7ce7451993f57c8228f9a7ade6b0e
Closes: OS#5607
 2022-07-23 13:46:52 +02:00
Harald Welte
fc31548c11 pySim-shell: Add a "version" command to print the pySim package version 
It may be interesting to know which pySim-shell version a user is running.

Change-Id: Ib9a1fbff71aa8a2cfbaca9e23efcf7c68bf5af1a
Closes: OS#5459
 2022-07-23 12:49:14 +02:00
Harald Welte
21caf32e3d Introduce APDU/TPDU trace decoder 
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between
SIM/UICC/USIM/ISIM card and UE.

It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.

This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.

Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats.  So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.

Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
 2022-07-23 12:18:57 +02:00
Harald Welte
cfa3015bcf sysmocom_sja2: Prevent KeyError/None exception on encode 
Fix a bug in the pySim.sysmocom_sja2 module, where we defined unnamed
bits in BitStruct without a default value causing exceptions like this:

	EXCEPTION of type 'KeyError' occurred with message: 'None'

Change-Id: Ib2da5adda4fae374ab14bb8100f338691aef719a
Closes: OS#5575
 2022-07-23 12:17:21 +02:00
Harald Welte
1272129ea7 ts_31_102: Fix EF_EPSLOCI argument ordering 
We were invoking the constructor with the description as 4th positional
argument, but that was actually the 'size' argument in this case.

Let's swap the order to be aligned with other file constructors.

Change-Id: I9acee757f096fef0d8bacbec3b52f56267cd52f6
 2022-07-21 22:48:59 +02:00
Harald Welte
99e4cc02e5 filesystem: Use Tuple for record length 
The size should be a *tuple*.  In reality we so far passed a set.  The
problem with the set is that ordering is not guaranteed, and hence we
cannot assume the first and second item have meaning (minimum vs.
default record length).

Change-Id: I470f4e69c83cb2761861b3350bf8d49e31f4d957
 2022-07-21 22:48:59 +02:00
Harald Welte
13edf30d6c filesystem: Use Tuple for transparent file size 
As the documentation strings say: The size should be a *tuple*.  In
reality we so far passed a set.  The problem with the set is that
ordering is not guaranteed, and hence we cannot assume the first and
second item have meaning (minimum vs. default size).

While at it, use a type annotation to catch such bugs easily.

Change-Id: I553616f8c6c4aaa8f635b3d7d94e8e8f49ed5a56
 2022-07-21 22:48:59 +02:00
Harald Welte
b2e4b4a300 introduce fully_qualified_path_str() method 
Reduce all the copy+pasted '/'.join(path_list) constructs with
a method returning the formatted path string.

Change-Id: I5e9bfb425c3a3fade13ca4ccd2b891a0c21ed56d
 2022-07-20 19:35:58 +02:00
Harald Welte
3c98d5e91d Never use Bytes without any 'Adapter' 
Otherwise we have binary/bytes as values inside the dict, rather than a
hexadecimal string.  That's ugly when printing without json formatting.

Change-Id: Ia3e7c4791d11bd4e3719a43d58e11e05ec986d1f
 2022-07-20 19:35:58 +02:00
Harald Welte
857f110492 EF.AD: Avoid NotImplementedErrror regarding network names 
Even while we don't yet have a proper decoder, let's at least represent
the network name as hex-string

Change-Id: I4ed626699d1e4e484d4ffd04349676dadff626a0
 2022-07-20 19:35:58 +02:00
Harald Welte
ea600a8451 tlv: Make NotImplementedError more verbose 
This helps to understand immediately _what_ is not implemented for which
type.

Change-Id: I017eb4828e9deee80338024c41c93c0f78db3f3b
 2022-07-20 19:35:58 +02:00
Harald Welte
fc8a9cca7b README: Mention the manual can also be built from the source 
Change-Id: Ic73a9ebaecab1b14668aaffe4cd39b3749a19fc7
 2022-07-20 19:35:58 +02:00
Harald Welte
363edd9d34 ts_31_102: Add support for obsolete EF.RPLMNAcT 
This file existed in earlier specs like Release 3.8.0, but was removed
in later revisions.  Still, there are cards around implementing that
older spec, so let's add a decoder.

Change-Id: Ic7163b2a01f64ef1223cf15b8d0813d3edf5b61a
 2022-07-18 09:35:35 +02:00
Harald Welte
d90ceb86be ts_31_102: Add support for DF.GSM-ACCESS 
Change-Id: I244c3eea13587e6213062d9a58e821697614a86a
 2022-07-17 22:12:06 +02:00
Harald Welte
228ae8e1dc ts_31_102: Support for files of DF.V2X (Vehicle 2 X) 
Change-Id: I7246f165aebbc42a685f36a7a6f973498b23b614
 2022-07-17 22:01:50 +02:00
Harald Welte
650f612d74 ts_31_102: Support for DF_MCS (Mission Critical Services) 
Change-Id: I0485a14c7820f7b345eeba6109a93b6d4bc639bf
 2022-07-17 22:01:29 +02:00
Harald Welte
6f8a870c65 move EF_UServiceTable from ts_31_102 to ts_31_102_telecom 
We want to use this class in an upcoming patch for DF_MCS support,
and in order to avoid cyclic imports, EF_UServiceTable must be moved.

Change-Id: I9cd6ab795bfd92f845eb943679a3d6302f1003ce
 2022-07-17 21:55:37 +02:00
Harald Welte
a0452216a4 minimalistic support for DF.MULTIMEDIA 
No decode of the payload of the files yet, but let's at least
name them.

Change-Id: I2d9c56bdea08fe6629978b6a1f7c139f487d075a
 2022-07-17 21:55:15 +02:00
Harald Welte
a6c0f880da filesystem: Introduce the basic notion of 'logical channels' 
cards can have multiple logical channels; each logical channel
has its own state of what is the current selected file + application.

Let's split the RuntimeState class into the global RuntimeState and the
per-lchan-specific RuntimeLchan class.

This code doesn't actually introduce any code that uses lchans other
than the basic logical channel (0), but just modifies the data model
to accomodate those in the future.

Change-Id: I7aa994b625467d4e46a2edd8123240b930305360
 2022-07-17 21:55:15 +02:00
Harald Welte
de4c14c0dc Add very simplistic DF_PHONEBOOK support 
This at least gives us the names for the DF and those EFs inside.

Change-Id: I12f70ae78e219e765ecb44cacff421d64c7b3f19
 2022-07-17 21:55:11 +02:00
Harald Welte
afe093ce41 ts_31_103: Fix typos related to IMSConfigData + MudMidConfigData 
s/neted/nested/

Change-Id: I9049ed12b8e7e6d1fdb7d19ed0b98ce8b46f9b0e
 2022-07-17 21:52:57 +02:00
Harald Welte
eb882052f5 ts_31_102: Fix FID in DF.HNB 
The FID are all specified as 4f8x and not 4f0x

Change-Id: I0cfd623693a5017efe01bc6891640db22ba3f9f9
 2022-07-17 21:52:57 +02:00
Harald Welte
4b00365c6e fileystem: Use human-readable ADF name if available. 
When using __str__ for a CardDF we would get "DF(DF.TELECOM)"
but when using it on CardADF we would get ADF(a0000000871002)"
instead of "ADF(ADF.USIM)".  Let's fix that.

Change-Id: I5801a08bcc28cb222734af6d9ee835227f4fee69
 2022-07-17 21:52:57 +02:00
Harald Welte
1e52b0d3b7 pySim-shell: Remove unused imports 
Those might have been used some time ago, but they are not.

Change-Id: I00f096fc8049c0aebc1127f9a1725638d973af0e
 2022-07-16 11:53:21 +02:00
Harald Welte
46a7a3fcc2 filesystem: keep track of currently selected ADF 
As it is possible to select files relative to the currently selected
ADF, we should keep track of that.

Change-Id: I83c93fdcd23b1d3877644ef0bf72d330343fbbc7
 2022-07-16 11:50:09 +02:00
Harald Welte
d56f45d720 filesystem: raise exception only when applicable 
We should first see if any of the files in the tree actually
require a service mapping before raising
ValueError('TODO: implement recursive service -> file mapping')

Change-Id: I9c339f0cac020e7eec7f4f840748040e5f77923d
 2022-07-16 11:50:08 +02:00
Vadim Yanitskiy
c655518654 pySim/ts_102_222.py: remove ununsed imports from 'cmd2' 
Change-Id: If6c686c8248cd0ad4edb68b84886a6f5f558d0f7
 2022-07-14 19:12:21 +07:00
Vadim Yanitskiy
0d9f088853 pySim-shell.py: remove unused imports of 'bg' from 'cmd2' 
Change-Id: Ic2a73a98f322be391e54215bc5fc3358776da0ae
 2022-07-14 19:11:25 +07:00
Harald Welte
6f8cf9b315 sim-rest-server: Set Content-Type: application/json on response 
Change-Id: Ib80a650f3e8d3e3ee6295db6de0981dfc23d3feb
 2022-07-08 20:47:46 +02:00
Harald Welte
77d510b4be scripts/deactivate-5g.script: Also disable service 126 
Service 126 relates to DF.5GS/EF.UAC_AIC.  As we are deactivating that
file in the script, we should also disable the related EF.UST service.

Change-Id: Id35035aaf23b2163caed3197786288c87be03cfa
 2022-07-08 20:47:35 +02:00
Vadim Yanitskiy
04b5d9d7ab Py2 -> Py3: do not inherit classes from object 
https://stackoverflow.com/questions/4015417/why-do-python-classes-inherit-object/45062077

Change-Id: I15003ba591510d68f3235f71526ad5d8a456088e
 2022-07-07 03:05:30 +07:00
Philipp Maier
bda52830c9 cards: populate ADM1 key reference member 
In class SimCard, we specify the key reference for ADM1 as 0x04. in the
UsimCard class, which inherits from SimCard nothing is specified, even
though ETSI TS 102 221 specifies 0x0A as key reference. Lets set the
member in UsimCard accordingly to be closer to the spec.

Note: For the moment this is a cosmetic fix, it does not change the
behaviour since all card classes derived from UsimCard set the key
reference properly.

Change-Id: I96af395b1832f4462a6043cca3bb3812fddac612
 2022-06-21 09:56:49 +02:00
Philipp Maier
2403125a34 pySim-shell: set default ADM key reference 
ETSI TS 102 221, Table 9.3 specifies 0x0A as default key reference for
ADM1. Lets make sure pySim-shell uses this key-reference if the card is
a generic UICC.

Change-Id: I8a96244269dc6619f39a5369502b15b83740ee45
 2022-06-14 16:22:39 +02:00
Philipp Maier
541a9154da ts_102_221: The BTLV IEs FILE SIZE and TOTAL FILE SIZE have a min length 
The TLV IEs FILE SIZE and TOTAL FILE SIZE have a minimum length of 2
byte. Even when the length is in the single digit range two bytes must
be used. See also: ETSI TS 102 221, section 11.1.1.4.1 and 11.1.1.4.2

Change-Id: Ief113ce8fe3bcae2c9fb2ff4138df9ccf98d26ff
 2022-06-10 16:26:54 +02:00
Philipp Maier
40ea4a4a1c commands: add ".." notation to expand hexstrings 
When updating files and records there are sometimes huge portions that
are just 0xff. Mostly this is at the end of a file or record that is not
completely used. Lets add a notation to tell PySim-shell how to fill
those sections.

Change-Id: Iedd7887bf7d706878f4a3beca8dbea456404610b
 2022-06-03 10:26:58 +02:00
Philipp Maier
f16ac6acf8 pySim-shell: catch exceptions from walk() while exporting 
When we run the exporter we also get an error summary at the end.
However, if walk() throws an eception this stops the exporter
immediately and we won't get the summpary. Lets catch exceptions from
walk as well so that we are able to end gracefully.

Change-Id: I3edc250ef2a84550c5b821a72e207e4d685790a5
 2022-06-03 10:18:09 +02:00
Philipp Maier
7b138b0d2d pySim-shell: extend walk() so that we can also have an action of ADF or DF 
The walk() method that we use to traverse the whole file system tree is
currently only able to execute action callbacks on EFs. Lets add a
mechanism that allows us to have a second callback that is executed when
we hit a DF or ADF.

Change-Id: Iabcd78552a14a2d3f8f31273dda7731e1f640cdb
 2022-06-03 08:17:57 +00:00
Philipp Maier
e7d1b67d80 pySim-shell: match SW in apdu command 
The apdu command has no option to match the resulting SW. Lets add a new
option for this.

Change-Id: Ic5a52d7cf533c51d111850eb6d8147011a48ae6c
 2022-06-03 10:08:37 +02:00
Philipp Maier
7226c09569 pySim-shell: make APDU command available on the lowest level 
The apdu command is used to communicate with the card on the lowest
possible level. Lets make it available even before a card profile (rs)
is avalable. This is especially useful when the card has no files on it,
in this situation pySim-shell will not be able to assign a profile to
the card at all. We can then use the apdu command to equip the card with
the most basic files and start over.

Change-Id: I601b8f17bd6af41dcbf7bbb53c75903dd46beee7
 2022-06-03 08:07:42 +00:00
Philipp Maier
373b23c372 ts_102_221: fix SFI generation 
The generation of the SFI does not work. The result is always a zero
length TLV IE.

Change-Id: Iaa38d2be4719f12c1d7b30a8befe278f1ed78ac1
 2022-06-02 08:43:54 +00:00
Philipp Maier
6b8eedc501 filesystem: also return the encoded FCP from probe_file 
he method probe_file returns the decoded FCP after it managed to
successfully probe the file. Lets also return the encoded FCP string, as
it is needed by the caller.

Change-Id: Ia5659e106fb0d6fb8b77506a10eba309e764723e
 2022-06-01 18:10:04 +02:00
Philipp Maier
9a4091d93a pySim-shell: more generic export options 
The as_json parameter has been added as an additional parameter to the
export function. Lets use a dictionary here and put the parameter in it.
This makes it easier to add more options in the future

Change-Id: Ie860eec918e7cdb01651642f4bc2474c9fb1924f
 2022-05-30 11:53:22 +02:00
Philipp Maier
ea81f75e94 pySim-shell: explain why we insist on a DF or ADF 
Change-Id: I155cefb10864432d59a0a66410783b4c9772f8a4
 2022-05-19 10:14:44 +02:00
Christian Amsüss
e17e277a24 ts_102_222: Set number of records when creating linear files 
This information is mandatory for linear files as per TS 102 221 V15
section 11.1.1.4.3. This might not have been spotted earlier because
cards of type sysmoISIM-SJA2 accept creation without it as well.

Change-Id: I8aeb869c601ee5d1c8b02da6d72eb3c50e347982
 2022-05-06 11:04:51 +00:00
Vadim Yanitskiy
e6b86872ce transport/pcsc: throw ReaderError with a message 
Before this patch:

  $ ./pySim-shell.py -p 0
  Card reader initialization failed with an exception of type:
  <class 'pySim.exceptions.ReaderError'>

after:

  $ ./pySim-shell.py -p 0
  Card reader initialization failed with exception:
  No reader found for number 0

Change-Id: Id08c4990857f7083a8d1cefc90ff85fc20ab6fef
 2022-04-25 18:24:41 +03:00
Vadim Yanitskiy
b95445159b SimCard.reset(): fix SyntaxWarning: 'is' with a literal 
Change-Id: I5860179acd1cb330e91dbe5b57cd60cd520f2d9d
 2022-04-21 16:46:09 +03:00
Harald Welte
c30bed235e ts_102_221: Add encode/write support of EF.ARR records 
With this change, we can also encode/write EF.ARR records, not just
decode/read.

Change-Id: Id0da2b474d05aba12136b9cae402ad8326700182
 2022-04-05 14:45:18 +02:00
Harald Welte
0dcdfbfe94 utils: Add DataObjectSequence.encode_multi() 
This is the analogous to the decode_multi() method.

Change-Id: Ifdd1b1bd4d67f447638858c3e92742ca6f884bfa
 2022-04-05 14:42:48 +02:00
Harald Welte
785d484709 utils: Fix bugs in DataObject encoders 
The DataObject is some weird / rarely used different code than the
normal TLV encoder/decoder.  It has apparently so far only been used
for decoding, without testing the encoding side, resulting in related
bugs.

Let's fix those that I encountered today, and add a test case.

Change-Id: I31370066f43c22fc3ce9e2b9ee75986a652f6fc4
 2022-04-05 14:33:00 +02:00
æstrid smith
b7f35ac163 ts_31_103: Correct file-id of EF.DOMAIN in ADF.ISIM 
While the short ID of this file is 05, the actual file-id is 6f03.
Reference to TS 31.103 section 4.2.3.

Change-Id: Idd572ab064ea38e74dffd583c27ea505b23214a2
 2022-03-27 10:43:38 +00:00
Harald Welte
ab91d874e4 ts_31_102: Avoid pylint false positive 
This should avoid the following pylint error:

************* Module pySim.ts_31_102
pySim/ts_31_102.py:621:100: E0601: Using variable 'sw' before assignment (used-before-assignment)

Change-Id: I0bb9607cdab0e6e3cd17b4d27129a51a607bc0f2
 2022-03-27 12:33:55 +02:00
Harald Welte
aefd0649a2 pySim-shell: Add 'decode_hex' command for transparent + linear EF 
These commands can be used to decode a user-provided hex-string,
instead of decoding the data read from the file.  This is useful
for quickly manually decoding some values read from other locations,
such as e.g. copy+pasted from a eSIM profile in ASN.1 value notation.

Change-Id: I81f73bce2c26e3e5dfc7538d223bb2d2483c7fa0
 2022-03-01 16:48:22 +00:00
Harald Welte
34eb504b3b Initial support for GlobalPlatform 
One can now select the Issuer Security Domain (hard-coded to
a000000003000000) and issue get_data requests.  FCI and other TLV
objects are dcoded, e.g.

pySIM-shell (MF)> select ADF.ISD
{
    "application_id": "a000000003000000",
    "proprietary_data": {
        "maximum_length_of_data_field_in_command_message": 255
    }
}
pySIM-shell (MF/ADF.ISD)> get_data CardData
{
    "card_data": [
        {
            "card_recognition_data": [
                {
                    "object_identifier": "2a864886fc6b01"
                },
                {
                    "card_management_type_and_version": [
                        {
                            "object_identifier": "2a864886fc6b02020101"
                        }
                    ]
                },
                {
                    "card_identification_scheme": [
                        {
                            "object_identifier": "2a864886fc6b03"
                        }
                    ]
                },
                {
                    "secure_channel_protocol_of_isd": [
                        {
                            "object_identifier": "2a864886fc6b040215"
                        }
                    ]
                }
            ]
        }
    ]
}

Change-Id: If11267d45ab7aa371eea8c143abd9320c32b54d0
 2022-03-01 16:32:15 +00:00
Harald Welte
a037762b04 ts_31_102: Further decode TAI in EF.OPL5G 
The TAI is not just an opaque bytestring but it consists of 3 fields.

Change-Id: Ie5a5ce74713deb0e151218ae553d3f3d96cef17d
 2022-02-25 15:45:09 +01:00
Harald Welte
3a5afff022 ts_31_102: Further decode LAI in EF_LOCI 
Change-Id: I21d9356e541eb320848a373804781ae0bef7d012
 2022-02-25 15:45:02 +01:00
Harald Welte
1459e45005 ts_51_011: Better decode of EF_OPL LAI 
before:
{
    "lai": "62f2300000fffe",
    "pnn_record_id": 1
}

after:
{
    "lai": {
        "mcc_mnc": "262f03",
        "lac_min": "0000",
        "lac_max": "fffe"
    },
    "pnn_record_id": 1
}

Change-Id: I82581220e9c33a8e67cbefd5dfeb40bbc2c31179
 2022-02-25 15:44:26 +01:00
Harald Welte
22a1cdde25 ts_51_011: Properly decode EF.OPL 
The OPL has 7 bytes "LAI" as the LAI actually contains a LAC
range (so two more bytes for the end of the 16bit range).

Change-Id: I74bcf10b0a8977af0f2844044a812c5780af1706
 2022-02-25 15:31:16 +01:00
Harald Welte
dd45d8ee3b ts_31_102: Fix decoding of UServiceTable 
range(0,7) in python is 0..6, and not 0..7, so we need range(0.8)
to produce the desired range covering all bits of a byte.

This resulted in services 8,16,24,... not being displayed in
the decoded output of EF.UST / EF.IST.

Change-Id: I22bbc481de342685352bf5b13d54931d3f37f9b7
 2022-02-25 15:31:16 +01:00
Harald Welte
4ebeebffca ts_102_221: Fix decoding the 'num_of_rec' field 
It is a 8bit integer, not a 16bit integer.  See TS 102 221 11.1.1.4.3

Change-Id: I3e258547dad21a248650cfbc02e0576268d3b3fd
 2022-02-25 09:48:20 +01:00
Harald Welte
5e9bd93bbd ts_102_221: properly decode short file identifier 
The SFI TLV contanins not the raw SFI, but it contains the SFI
shifted to left by 3 bits (for some strange reason).  So let's
un-shift it.

Change-Id: Ibc69b99010d2a25cbb69b6a3d1585d0cb63f1345
 2022-02-25 09:37:40 +01:00
Harald Welte
fa578bd601 add scripts/deactivate-ims.script to deactivate IMS related services 
Change-Id: I0cd93c8fa0024dd9d93647c565190abe94d3097e
 2022-02-21 09:57:09 +01:00
Harald Welte
c89a1a99ca Add scripts/deacivate-5g.script 
This script can be used to deactivate all 5G related services and files.

Change-Id: I5dc3e9f0ae76a7ae57484e5a3369e11ff02c7eca
 2022-02-17 12:42:14 +01:00
Harald Welte
12af793d4b doc: Improve documentation in various places 
* don't duplicate information between .rst files and docstrings
* if there's more than a trivial single-line documentation, put it as
  docstring into the python source and use ".. argparse" to pul it into
  the manual
* add documentation for some commands for which it was missing
* show one level deeper in the navigation table, listing the commands

Change-Id: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
 2022-02-15 16:40:45 +01:00
Harald Welte
d01bd3632c docs: Document missing 'status' command in 7816 section 
Change-Id: I9af85a36bc4f24c3a22b9b2a6b8e2abd86edfe4e
 2022-02-15 15:56:48 +01:00
Harald Welte
799c354827 shell: Proper argparser (for help + manual) activate_file 
Change-Id: I5929ae3deff4d15b5db4a1d866576271c57a955f
 2022-02-15 15:56:28 +01:00
Harald Welte
2bb17f3df9 pySim-shell: export: Add FCP template to export 
The FCP template provides us a lot of context, like the permissions of
a given file.  Let's make it part of the 'export' output, both in raw
and in decoded form.

Change-Id: I05f17bbebd7a9b3535204b821900851a5f66e88f
Closes: OS#5457
 2022-02-15 15:41:55 +01:00
Harald Welte
9e241435cc docs/legcay.txt: Point to pySim-shell as replacement 
Change-Id: I9ca6b9d8c35e23be2ec8752107bb7d1e4f6f9bc1
 2022-02-15 15:38:19 +01:00
Harald Welte
3c9b784825 pySim-shell: support TS 102 222 administrative commands 
This adds support for creating/deleting and terminating files,
as well as support for permanent card termination.

Change-Id: I5b1ffb1334afa18d62beb642268066a30deb7ea6
 2022-02-15 15:35:36 +01:00
Harald Welte
747a978478 ts_102_221: Implement File Descriptor using construct 
This automatically adds encoding support, which is needed for upcoming
CREATE FILE support.

Change-Id: Ia40dba4aab6ceb9d81fd170f7efa8dad1f9b43d0
 2022-02-15 15:35:36 +01:00
Harald Welte
ee670bc1c6 pySim-shell: Allow selecting of deep paths like DF.GSM/EF.IMSI 
With this patch applied, users can directly enter commands like

select DF.GSM/EF.IMSI or
select ADF.USIM/DF.5GS/EF.5GAUTHKEYS

This feature doesn't have tabl completion, so it's mostly useful
for when you know what to select, or for use within scripts.

Change-Id: I681a132eb2df4b2aba4c2ccbdd21c6d5b88443e3
 2022-02-15 15:35:36 +01:00
Harald Welte
226b866f51 ts_31_103: TLV definitions for IMS, XCAP and MudMid configuration 
Change-Id: I9a90ee978db668a70259eb48085ff5384cf696d6
 2022-02-15 15:35:36 +01:00
Harald Welte
540adb0ee6 ts_51_011: EF_CMI: Decoder the alpha_id string 
Change-Id: I45efe29ab98972945b4257229a995815f5632536
 2022-02-15 15:35:36 +01:00
Harald Welte
1e73d228f4 ts_51_011: Convert EF_ADN and EF_ACC to Construct 
this has the benefit of providing encoding support for free.

Change-Id: I31c118082e92892486c3688de2197c0c6dd2750e
 2022-02-15 15:35:36 +01:00
Harald Welte
bc0e209a9f ts_51_011: Proper decode of EF.SMSP 
Full decode of the SSM Parameters File

Change-Id: Iac5bb87ed3350978dc8b207f052510fdba2e4883
 2022-02-15 15:35:35 +01:00
Harald Welte
3bb516b2b1 Improve IST/UST check documentation (for the user manual) 
Change-Id: I18093d795721f2e729eff858c8922edde9e84451
 2022-02-15 15:35:35 +01:00
Harald Welte
aceb2a548a ust_service_check: proper treatment of files in sub-directories 
We must not only consider files in the current directory (ADF.USIM)
but also in its sub-directories.  This requires us to be able to
determine the path we need to traverse between the currently selected
file (EF.UST) and the respective file in some other directory,
which is implemented via CardFile.build_select_path_to().

Change-Id: I61797fefa9dafa36a8a62c11aa2cfaeecb015740
 2022-02-15 15:35:35 +01:00
Harald Welte
419bb496e1 ts_31_102: service annotations for DF.{5GS,WLAN,HNB} 
We had service annotations only for ADF.USIM so far, but not for
the related sub-directories.

Change-Id: Iaa56a26ba53eaf18fce14845ae07a27c52a2c58a
Note: The code doesn't make use of them in any reasonable way yet!
 2022-02-15 15:35:35 +01:00
Harald Welte
fa8b8d1160 ts_31_102: Use perror() instead of poutput() for errors 
This adds colorization and ensures they go to stderr and not stdout

Change-Id: I34b8f974b4ff13002679c4700bdf604db7d7f3cd
 2022-02-15 15:35:35 +01:00
Harald Welte
82f75c200f ts_31_102: Add more EF.UST checks to 'ust_service_check' command 
* check for service dependencies listed in TS 31.102
* print number of errors encountered

Change-Id: Id47f8f2c8de299bbf91243d0c8900d22a7d35b10
 2022-02-15 15:35:35 +01:00
Harald Welte
d53918c3e1 filesystem: Fix CardMF.get_app_names() 
This function was not used and doesn't work without this patch.

Change-Id: Id3dad7d97fe29a25792d2f8f0e879666c1d9c136
 2022-02-15 15:35:35 +01:00
Harald Welte
6ca2fa7a5d Split EF.UST handling from EF.IST and EF.SST 
The existing code had the following serious problems:
* when trying to update EF.SST or EF.IST, it would write to EF.UST !
* shell commands were called ust_* even for the EST/IST files

Let's introduce the proper separation between what is shared and what
is file-specific.

Change-Id: Ie55669ca37a4762fac9f71b1db528ca67056e8dd
 2022-02-15 15:35:35 +01:00
Harald Welte
4c5e2310fa ts_31_102: Add "ust_service_check" command. 
This command performs a consistency check between the services activated
in EF.UST/EF.IST and the files that should (or should not) be
active/selectable for the given service.

Produces output like:

Checking service No 48 (inactive)
  ERROR: File EF(EF.MWIS) is selectable but should not!
Checking service No 49 (active)
  ERROR: File EF(EF.CFIS) is not selectable (SW=6a82) but should!

Change-Id: Iea7166959e2015eb8fa34d86036560c9e42ce4d3
 2022-02-15 15:35:35 +01:00
Harald Welte
d16d904c57 README.md: Remove old usage examples, refer to user manual instead 
We want people to use pySim-shell and should not mislead them by
having usage examples of old tools in README.md.  Also, all
documentation should be in the manuals, let's try to have bits
and pieces in various places.

Change-Id: I8c07a2e0778ab95fb42be6074acb80874e681d20
 2022-02-15 15:35:35 +01:00
Harald Welte
3729c47651 commands: Add method to select parent DF ("cd ..") 
This is useful when walking around the filesystem tree.

Change-Id: Ib256c1b7319f2b5f9a06200fb96854ecb2b7f6bb
 2022-02-14 00:51:27 +01:00
Harald Welte
a630a3cd28 cosmetic: Remove extraneous empty lines between spec-section-comment and class 
This is an artefact of the recent autopep8 re-formatting.

Change-Id: I8b0e7781719d69e18856ada2f482de2c5396bcc3
 2022-02-14 00:51:27 +01:00
Harald Welte
6169c72f82 USIM + ISIM: Specify the services associated with each file 
This allows us [in a future patch] to perform consistency checking,
whether files exist for services not activated in EF.{UST,IST} or
vice-versa: Services are activated by files are not present or
deactivated.

Change-Id: I94bd1c3f9e977767553000077dd003423ed6dbd1
 2022-02-14 00:51:27 +01:00
Harald Welte
9170fbf08d filesystem: Maintain a 'service' attribute for all files on a card 
This can be populated by card profiles with the SST/IST/UST service
that is associated with the file.

Change-Id: I3b3f74b691368fa09967ecb377a9f7a6d8af7869
 2022-02-14 00:51:22 +01:00
Harald Welte
afb8d3f925 pySim-shell: introduce 'apdu' command for sending raw APDU to card 
This can be useful when playing around with cards, for example
sending commands for which pySim-shell doesn't yet have proper support.

Change-Id: Ib504431d26ed2b6f71f77a143ff0a7fb4f5ea02e
 2022-02-14 00:48:16 +01:00
Harald Welte
08b11abc2f pySim-shell: export: allow export as JSON instead of hex 
The primary use case of the --json option is to systematically execute
all of our decoder classes in order to find bugs.  As we don't have
encoders for all files yet, the output generated by 'export --json'
will in many cases not be executable as script again, unlike the normal
'export' output.

Change-Id: Idd820f8e3af70ebcbf82037b56fd2ae9655afbc5
 2022-02-14 00:48:16 +01:00
Harald Welte
c8c3327b6e ts_102_221: Proper parsing of FCP using pySim.tlv instead of pytlv 
pytlv is a nightmare of shortcomings, let's abandon it in favor of
our own meanwhile-created pySim.tlv.  This has the added benefit
that unknown tags finally no longer raise exceptions.

Change-Id: Ic8e0e0ddf915949670d620630d4ceb02a9116471
Closes: OS#5414
 2022-02-14 00:48:11 +01:00
Harald Welte
e4a6eafc6f tlv: Don't raise exception if somebody passes empty data to TLV decoder 
Change-Id: Id46994029d9b3cd6b67f4f7ee619466602cc8142
 2022-02-14 00:44:55 +01:00
Harald Welte
c975251a48 filesystem: Don't pass empty string to parse_select_response() 
This happens e.g. when selecting the ARA-M applet on sysmoISIM-SJA2:

pySIM-shell (MF)> select ADF.ARA-M
-> 00a4040409 a00000015141434c00
<- 9000:
Traceback (most recent call last):
  File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2064, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2494, in onecmd
    stop = func(statement)
  File "/space/home/laforge/projects/git/pysim/./pySim-shell.py", line 750, in do_select
    fcp_dec = self._cmd.rs.select(path, self._cmd)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 1314, in select
    select_resp = f.decode_select_response(data)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 193, in decode_select_response
    return self.parent.decode_select_response(data_hex)
  File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 378, in decode_select_response
    return profile.decode_select_response(data_hex)
  File "/space/home/laforge/projects/git/pysim/pySim/ts_102_221.py", line 796, in decode_select_response
    t.from_tlv(h2b(resp_hex))
  File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 231, in from_tlv
    (rawtag, remainder) = self.__class__._parse_tag_raw(do)
  File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 258, in _parse_tag_raw
    return bertlv_parse_tag_raw(do)
  File "/space/home/laforge/projects/git/pysim/pySim/utils.py", line 208, in bertlv_parse_tag_raw
    if binary[0] == 0xff and len(binary) == 1 or binary[0] == 0xff and binary[1] == 0xff:
IndexError: bytearray index out of range
EXCEPTION of type 'IndexError' occurred with message: 'bytearray index out of range'

Change-Id: I910e6deba27d1483dff1e986c89f1a1b2165f49b
 2022-02-14 00:44:55 +01:00
Harald Welte
81f4b4058b Extend unit test coverage for construct, add [some] tests for TLV 
Change-Id: I3470e0b2e978221aa0c1e46a4b65f71f71abef2e
 2022-02-14 00:41:24 +01:00
Harald Welte
d0519e0c37 construct: Add Construct for variable-length int 'GreedyInteger' 
We have a number of integers with variable-length encoding, so
add a Construct for this.  Naming inspired by GreedyBytes.

Related to https://github.com/construct/construct/issues/962

Change-Id: Ic6049b74ea3705fda24855f34b4a1d5f2c9327f7
 2022-02-14 00:41:24 +01:00
451 changed files with 83560 additions and 7092 deletions
Expand all files Collapse all files

2
.checkpatch.conf Normal file
View File

@@ -0,0 +1,2 @@
--exclude ^pySim/esim/asn1/.*\.asn$
--exclude ^smdpp-data/.*$

1
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1 @@
open_collective: osmocom

17
.gitignore vendored
View File

@@ -1,2 +1,17 @@
*.pyc
.*.swp
.*.sw?
/docs/_*
/docs/generated
/docs/filesystem.rst
/.cache
/.local
/build
/pySim.egg-info
/smdpp-data/sm-dp-sessions*
dist
tags
smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_NIST.pem
smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_BRP.pem
smdpp-data/generated
smdpp-data/certs/dhparam2048.pem

181
README.md
View File

@@ -1,32 +1,84 @@
pySim - Read, Write and Browse Programmable SIM/USIM Cards
====================================================
pySim - Tools for reading, decoding, browsing SIM/USIM/ISIM/HPSIM/eUICC Cards
=============================================================================
This repository contains Python programs that can be used
to read, program (write) and browse certain fields/parameters on so-called programmable
SIM/USIM cards.
This repository contains a number of Python programs related to working with
subscriber identity modules of cellular networks, including but not limited
to SIM, UICC, USIM, ISIM, HPSIMs and eUICCs.
Such SIM/USIM cards are special cards, which - unlike those issued by
regular commercial operators - come with the kind of keys that allow you
to write the files/fields that normally only an operator can program.
* `pySim-shell.py` can be used to interactively explore, read and decode contents
of any of the supported card models / card applications. Furthermore, if
you have the credentials to your card (ADM PIN), you can also write to the card,
i.e. edit its contents.
* `pySim-read.py` and `pySim-prog.py` are _legacy_ tools for batch programming
some very common parameters to an entire batch of programmable cards
* `pySim-trace.py` is a tool to do an in-depth decode of SIM card protocol traces
such as those obtained by [Osmocom SIMtrace2](https://osmocom.org/projects/simtrace2/wiki)
or [osmo-qcdiag](https://osmocom.org/projects/osmo-qcdiag/wiki).
* `osmo-smdpp.py` is a proof-of-concept GSMA SGP.22 Consumer eSIM SM-DP+ for lab/research
* there are more related tools, particularly in the `contrib` directory.
Note that the access control configuration of normal production cards
issue by operators will restrict significantly which files a normal
user can read, and particularly write to.
The full functionality of pySim hence can only be used with on so-called
programmable SIM/USIM/ISIM/HPSIM cards, such as the various
[sysmocom programmable card products](https://shop.sysmocom.de/SIM/).
Such SIM/USIM/ISIM/HPSIM cards are special cards, which - unlike those
issued by regular commercial operators - come with the kind of keys that
allow you to write the files/fields that normally only an operator can
program.
This is useful particularly if you are running your own cellular
network, and want to issue your own SIM/USIM cards for that network.
network, and want to configure your own SIM/USIM/ISIM/HPSIM cards for
that network.
Homepage and Manual
-------------------
Homepage
--------
Please visit the [official homepage](https://osmocom.org/projects/pysim/wiki)
for usage instructions, manual and examples.
Documentation
-------------
The pySim user manual can be built from this very source code by means
of sphinx (with sphinxcontrib-napoleon and sphinx-argparse). See the
Makefile in the 'docs' directory.
A pre-rendered HTML user manual of the current pySim 'git master' is
available from <https://downloads.osmocom.org/docs/latest/pysim/> and
a downloadable PDF version is published at
<https://downloads.osmocom.org/docs/latest/osmopysim-usermanual.pdf>.
A slightly dated video presentation about pySim-shell can be found at
<https://media.ccc.de/v/osmodevcall-20210409-laforge-pysim-shell>.
pySim-shell vs. legacy tools
----------------------------
While you will find a lot of online resources still describing the use of
`pySim-prog.py` and `pySim-read.py`, those tools are considered legacy by
now and have by far been superseded by the much more capable
`pySim-shell.py`. We strongly encourage users to adopt pySim-shell, unless
they have very specific requirements like batch programming of large
quantities of cards, which is about the only remaining use case for the
legacy tools.
Please visit the [official homepage](https://osmocom.org/projects/pysim/wiki) for usage instructions, manual and examples.
Git Repository
--------------
You can clone from the official Osmocom git repository using
```
git clone git://git.osmocom.org/pysim.git
git clone https://gitea.osmocom.org/sim-card/pysim.git
```
There is a cgit interface at <https://git.osmocom.org/pysim>
There is a web interface at <https://gitea.osmocom.org/sim-card/pysim>.
Installation
@@ -34,23 +86,38 @@ Installation
Please install the following dependencies:
- pyscard
- serial
- pytlv
- cmd2 >= 1.3.0 but < 2.0.0
- jsonpath-ng
- construct
- bidict
- gsm0338
- cmd2 >= 1.5.0
- colorlog
- construct >= 2.9.51
- pyosmocom
- jsonpath-ng
- packaging
- pycryptodomex
- pyscard
- pyserial
- pytlv
- pyyaml >= 5.4
- smpp.pdu (from `github.com/hologram-io/smpp.pdu`)
- termcolor
Example for Debian:
```
apt-get install python3-pyscard python3-serial python3-pip python3-yaml
pip3 install -r requirements.txt
```sh
sudo apt-get install --no-install-recommends \
pcscd libpcsclite-dev \
python3 \
python3-setuptools \
python3-pycryptodome \
python3-pyscard \
python3-pip
pip3 install --user -r requirements.txt
```
After installing all dependencies, the pySim applications ``pySim-read.py``, ``pySim-prog.py`` and ``pySim-shell.py`` may be started directly from the cloned repository.
In addition to the dependencies above ``pySim-trace.py`` requires ``tshark`` and the python package ``pyshark`` to be installed. It is known that the ``tshark`` package
in Debian versions before 11 may not work with pyshark.
### Archlinux Package
Archlinux users may install the package ``python-pysim-git``
@@ -69,19 +136,34 @@ sudo pacman -Rs python-pysim-git
```
Forum
-----
We welcome any pySim related discussions in the
[SIM Card Technology](https://discourse.osmocom.org/c/sim-card-technology/)
section of the osmocom discourse (web based Forum).
Mailing List
------------
There is no separate mailing list for this project. However,
discussions related to pysim-prog are happening on the
<openbsc@lists.osmocom.org> mailing list, please see
<https://lists.osmocom.org/mailman/listinfo/openbsc> for subscription
discussions related to pySim are happening on the simtrace
<simtrace@lists.osmocom.org> mailing list, please see
<https://lists.osmocom.org/mailman/listinfo/simtrace> for subscription
options and the list archive.
Please observe the [Osmocom Mailing List
Rules](https://osmocom.org/projects/cellular-infrastructure/wiki/Mailing_List_Rules)
when posting.
Issue Tracker
-------------
We use the [issue tracker of the pysim project on osmocom.org](https://osmocom.org/projects/pysim/issues) for
tracking the state of bug reports and feature requests. Feel free to submit any issues you may find, or help
us out by resolving existing issues.
Contributing
------------
@@ -91,48 +173,3 @@ Our coding standards are described at
We are using a gerrit-based patch review process explained at
<https://osmocom.org/projects/cellular-infrastructure/wiki/Gerrit>
Usage Examples
--------------
* Program customizable SIMs. Two modes are possible:
- one where you specify every parameter manually:
```
./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -i <IMSI> -s <ICCID>
```
- one where they are generated from some minimal set:
```
./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -z <random_string_of_choice> -j <card_num>
```
With ``<random_string_of_choice>`` and ``<card_num>``, the soft will generate
'predictable' IMSI and ICCID, so make sure you choose them so as not to
conflict with anyone. (for e.g. your name as ``<random_string_of_choice>`` and
0 1 2 ... for ``<card num>``).
You also need to enter some parameters to select the device:
-t TYPE : type of card (``supersim``, ``magicsim``, ``fakemagicsim`` or try ``auto``)
-d DEV : Serial port device (default ``/dev/ttyUSB0``)
-b BAUD : Baudrate (default 9600)
* Interact with SIMs from a python interactive shell (e.g. ipython):
```
from pySim.transport.serial import SerialSimLink
from pySim.commands import SimCardCommands
sl = SerialSimLink(device='/dev/ttyUSB0', baudrate=9600)
sc = SimCardCommands(sl)
sl.wait_for_card()
# Print IMSI
print(sc.read_binary(['3f00', '7f20', '6f07']))
# Run A3/A8
print(sc.run_gsm('00112233445566778899aabbccddeeff'))
```

112
contrib/analyze_simaResponse.py Executable file
View File

@@ -0,0 +1,112 @@
#!/usr/bin/env python3
# A tool to analyze the eUICC simaResponse (series of EUICCResponse)
#
# (C) 2025 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from osmocom.utils import h2b, b2h
from osmocom.tlv import bertlv_parse_one, bertlv_encode_tag, bertlv_encode_len
from pySim.esim.saip import *
parser = argparse.ArgumentParser(description="""Utility program to analyze the contents of an eUICC simaResponse.""")
parser.add_argument('SIMA_RESPONSE', help='Hexstring containing the simaResponse as received from the eUICC')
def split_sima_response(sima_response):
"""split an eUICC simaResponse field into a list of EUICCResponse fields"""
remainder = sima_response
result = []
while len(remainder):
tdict, l, v, next_remainder = bertlv_parse_one(remainder)
rawtag = bertlv_encode_tag(tdict)
rawlen = bertlv_encode_len(l)
result = result + [remainder[0:len(rawtag) + len(rawlen) + l]]
remainder = next_remainder
return result
def analyze_status(status):
"""
Convert a status code (integer) into a human readable string
(see eUICC Profile Package: Interoperable Format Technical Specification, section 8.11)
"""
# SIMA status codes
string_values = {0 : 'ok',
1 : 'pe-not-supported',
2 : 'memory-failure',
3 : 'bad-values',
4 : 'not-enough-memory',
5 : 'invalid-request-format',
6 : 'invalid-parameter',
7 : 'runtime-not-supported',
8 : 'lib-not-supported',
9 : 'template-not-supported ',
10 : 'feature-not-supported',
11 : 'pin-code-missing',
31 : 'unsupported-profile-version'}
string_value = string_values.get(status, None)
if string_value is not None:
return "%d = %s (SIMA status code)" % (status, string_value)
# ISO 7816 status words
if status >= 24576 and status <= 28671:
return "%d = %04x (ISO7816 status word)" % (status, status)
elif status >= 36864 and status <= 40959:
return "%d = %04x (ISO7816 status word)" % (status, status)
# Proprietary status codes
elif status >= 40960 and status <= 65535:
return "%d = %04x (proprietary)" % (status, status)
# Unknown status codes
return "%d (unknown, proprietary?)" % status
def analyze_euicc_response(euicc_response):
"""Analyze and display the contents of an EUICCResponse"""
print(" EUICCResponse: %s" % b2h(euicc_response))
euicc_response_decoded = asn1.decode('EUICCResponse', euicc_response)
pe_status = euicc_response_decoded.get('peStatus')
print(" peStatus:")
for s in pe_status:
print(" status: %s" % analyze_status(s.get('status')))
print(" identification: %s" % str(s.get('identification', None)))
print(" additional-information: %s" % str(s.get('additional-information', None)))
print(" offset: %s" % str(s.get('offset', None)))
if euicc_response_decoded.get('profileInstallationAborted', False) is None:
# This type is defined as profileInstallationAborted NULL OPTIONAL, so when it is present it
# will have the value None, otherwise it is simply not present.
print(" profileInstallationAborted: True")
else:
print(" profileInstallationAborted: False")
status_message = euicc_response_decoded.get('statusMessage', None)
print(" statusMessage: %s" % str(status_message))
if __name__ == '__main__':
opts = parser.parse_args()
sima_response = h2b(opts.SIMA_RESPONSE);
print("simaResponse: %s" % b2h(sima_response))
euicc_response_list = split_sima_response(sima_response)
for euicc_response in euicc_response_list:
analyze_euicc_response(euicc_response)

66
contrib/csv-encrypt-columns.py Executable file
View File

@@ -0,0 +1,66 @@
#!/usr/bin/env python3
# Utility program to perform column-based encryption of a CSV file holding SIM/UICC
# related key materials.
#
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import csv
import argparse
from Cryptodome.Cipher import AES
from osmocom.utils import h2b, b2h, Hexstr
from pySim.card_key_provider import CardKeyFieldCryptor
class CsvColumnEncryptor(CardKeyFieldCryptor):
def __init__(self, filename: str, transport_keys: dict):
self.filename = filename
self.crypt = CardKeyFieldCryptor(transport_keys)
def encrypt(self) -> None:
with open(self.filename, 'r') as infile:
cr = csv.DictReader(infile)
cr.fieldnames = [field.upper() for field in cr.fieldnames]
with open(self.filename + '.encr', 'w') as outfile:
cw = csv.DictWriter(outfile, dialect=csv.unix_dialect, fieldnames=cr.fieldnames)
cw.writeheader()
for row in cr:
for fieldname in cr.fieldnames:
row[fieldname] = self.crypt.encrypt_field(fieldname, row[fieldname])
cw.writerow(row)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('CSVFILE', help="CSV file name")
parser.add_argument('--csv-column-key', action='append', required=True,
help='per-CSV-column AES transport key')
opts = parser.parse_args()
csv_column_keys = {}
for par in opts.csv_column_key:
name, key = par.split(':')
csv_column_keys[name] = key
if len(csv_column_keys) == 0:
print("You must specify at least one key!")
sys.exit(1)
cce = CsvColumnEncryptor(opts.CSVFILE, csv_column_keys)
cce.encrypt()

301
contrib/csv-to-pgsql.py Executable file
View File

@@ -0,0 +1,301 @@
#!/usr/bin/env python3
# (C) 2025 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import argparse
import logging
import csv
import sys
import os
import yaml
import psycopg2
from psycopg2.sql import Identifier, SQL
from pathlib import Path
from pySim.log import PySimLogger
from packaging import version
log = PySimLogger.get(Path(__file__).stem)
class CardKeyDatabase:
def __init__(self, config_filename: str, table_name: str, create_table: bool = False, admin: bool = False):
"""
Initialize database connection and set the table which shall be used as storage for the card key data.
In case the specified table does not exist yet it can be created using the create_table_type parameter.
New tables are always minimal tables which follow a pre-defined table scheme. The user may extend the table
with additional columns using the add_cols() later.
Args:
tablename : name of the database table to create.
create_table_type : type of the table to create ('UICC' or 'EUICC')
"""
def user_from_config_file(config, role: str) -> tuple[str, str]:
db_users = config.get('db_users')
user = db_users.get(role)
if user is None:
raise ValueError("user for role '%s' not set up in config file." % role)
return user.get('name'), user.get('pass')
self.table = table_name.lower()
self.cols = None
# Depending on the table type, the table name must contain either the substring "uicc_keys" or "euicc_keys".
# This convention will allow us to deduct the table type from the table name.
if "euicc_keys" not in table_name and "uicc_keys" not in table_name:
raise ValueError("Table name (%s) should contain the substring \"uicc_keys\" or \"euicc_keys\"" % table_name)
# Read config file
log.info("Using config file: %s", config_filename)
with open(config_filename, "r") as cfg:
config = yaml.load(cfg, Loader=yaml.FullLoader)
host = config.get('host')
log.info("Database host: %s", host)
db_name = config.get('db_name')
log.info("Database name: %s", db_name)
table_names = config.get('table_names')
username_admin, password_admin = user_from_config_file(config, 'admin')
username_importer, password_importer = user_from_config_file(config, 'importer')
username_reader, _ = user_from_config_file(config, 'reader')
# Switch between admin and importer user
if admin:
username, password = username_admin, password_admin
else:
username, password = username_importer, password_importer
# Create database connection
log.info("Database user: %s", username)
self.conn = psycopg2.connect(dbname=db_name, user=username, password=password, host=host)
self.cur = self.conn.cursor()
# In the context of this tool it is not relevant if the table name is present in the config file. However,
# pySim-shell.py will require the table name to be configured properly to access the database table.
if self.table not in table_names:
log.warning("Specified table name (%s) is not yet present in config file (required for access from pySim-shell.py)",
self.table)
# Create a new minimal database table of the specified table type.
if create_table:
if not admin:
raise ValueError("creation of new table refused, use option --admin and try again.")
if "euicc_keys" in self.table:
self.__create_table(username_reader, username_importer, ['EID'])
elif "uicc_keys" in self.table:
self.__create_table(username_reader, username_importer, ['ICCID', 'IMSI'])
# Ensure a table with the specified name exists
log.info("Database table: %s", self.table)
if self.get_cols() == []:
raise ValueError("Table name (%s) does not exist yet" % self.table)
log.info("Database table columns: %s", str(self.get_cols()))
def __create_table(self, user_reader:str, user_importer:str, cols:list[str]):
"""
Initialize a new table. New tables are always minimal tables with one primary key and additional index columns.
Non index-columns may be added later using method _update_cols().
"""
# Create table columns with primary key
query = SQL("CREATE TABLE {} ({} VARCHAR PRIMARY KEY").format(Identifier(self.table),
Identifier(cols[0].lower()))
for c in cols[1:]:
query += SQL(", {} VARCHAR").format(Identifier(c.lower()))
query += SQL(");")
self.cur.execute(query)
# Create indexes for all other columns
for c in cols[1:]:
self.cur.execute(query = SQL("CREATE INDEX {} ON {}({});").format(Identifier(c.lower()),
Identifier(self.table),
Identifier(c.lower())))
# Set permissions
self.cur.execute(SQL("GRANT INSERT ON {} TO {};").format(Identifier(self.table),
Identifier(user_importer)))
self.cur.execute(SQL("GRANT SELECT ON {} TO {};").format(Identifier(self.table),
Identifier(user_reader)))
log.info("New database table created: %s", self.table)
def get_cols(self) -> list[str]:
"""
Get a list of all columns available in the current table scheme.
Returns:
list with column names (in uppercase) of the database table
"""
# Return cached col list if present
if self.cols:
return self.cols
# Request a list of current cols from the database
self.cur.execute("SELECT column_name FROM information_schema.columns where table_name = %s;", (self.table,))
cols_result = self.cur.fetchall()
cols = []
for c in cols_result:
cols.append(c[0].upper())
self.cols = cols
return cols
def get_missing_cols(self, cols_expected:list[str]) -> list[str]:
"""
Check if the current table scheme lacks any of the given expected columns.
Returns:
list with the missing columns.
"""
cols_present = self.get_cols()
return list(set(cols_expected) - set(cols_present))
def add_cols(self, cols:list[str]):
"""
Update the current table scheme with additional columns. In case the updated columns are already exist, the
table schema is not changed.
Args:
table : name of the database table to alter
cols : list with updated colum names to add
"""
cols_missing = self.get_missing_cols(cols)
# Depending on the table type (see constructor), we either have a primary key 'ICCID' (for UICC data), or 'EID'
# (for eUICC data). Both table formats different types of data and have rather differen columns also. Let's
# prevent the excidentally mixing of both types.
if 'ICCID' in cols_missing:
raise ValueError("Table %s stores eUCCC key material, refusing to add UICC specific column 'ICCID'" % self.table)
if 'EID' in cols_missing:
raise ValueError("Table %s stores UCCC key material, refusing to add eUICC specific column 'EID'" % self.table)
# Add the missing columns to the table
self.cols = None
for c in cols_missing:
self.cur.execute(query = SQL("ALTER TABLE {} ADD {} VARCHAR;").format(Identifier(self.table),
Identifier(c.lower())))
def insert_row(self, row:dict[str, str]):
"""
Insert a new row into the database table.
Args:
row : dictionary with the colum names and their designated values
"""
# Check if the row is compatible with the current table scheme
cols_expected = list(row.keys())
cols_missing = self.get_missing_cols(cols_expected)
if cols_missing != []:
raise ValueError("table %s has incompatible format, the row %s contains unknown cols %s" %
(self.table, str(row), str(cols_missing)))
# Insert row into datbase table
row_keys = list(row.keys())
row_values = list(row.values())
query = SQL("INSERT INTO {} ").format(Identifier(self.table))
query += SQL("({} ").format(Identifier(row_keys[0].lower()))
for k in row_keys[1:]:
query += SQL(", {}").format(Identifier(k.lower()))
query += SQL(") VALUES (%s")
for v in row_values[1:]:
query += SQL(", %s")
query += SQL(");")
self.cur.execute(query, row_values)
def commit(self):
self.conn.commit()
log.info("Changes to table %s committed!", self.table)
def open_csv(opts: argparse.Namespace):
log.info("CSV file: %s", opts.csv)
csv_file = open(opts.csv, 'r')
cr = csv.DictReader(csv_file)
if not cr:
raise RuntimeError("could not open DictReader for CSV-File '%s'" % opts.csv)
cr.fieldnames = [field.upper() for field in cr.fieldnames]
log.info("CSV file columns: %s", str(cr.fieldnames))
return cr
def open_db(cr: csv.DictReader, opts: argparse.Namespace) -> CardKeyDatabase:
try:
db = CardKeyDatabase(os.path.expanduser(opts.pgsql), opts.table_name, opts.create_table, opts.admin)
# Check CSV format against table schema, add missing columns
cols_missing = db.get_missing_cols(cr.fieldnames)
if cols_missing != [] and (opts.update_columns or opts.create_table):
log.info("Adding missing columns: %s", str(cols_missing))
db.add_cols(cols_missing)
cols_missing = db.get_missing_cols(cr.fieldnames)
# Make sure the table schema has no missing columns
if cols_missing != []:
log.error("Database table lacks CSV file columns: %s -- import aborted!", cols_missing)
sys.exit(2)
except Exception as e:
log.error(str(e).strip())
log.error("Database initialization aborted due to error!")
sys.exit(2)
return db
def import_from_csv(db: CardKeyDatabase, cr: csv.DictReader):
count = 0
for row in cr:
try:
db.insert_row(row)
count+=1
if count % 100 == 0:
log.info("CSV file import in progress, %d rows imported...", count)
except Exception as e:
log.error(str(e).strip())
log.error("CSV file import aborted due to error, no datasets committed!")
sys.exit(2)
log.info("CSV file import done, %d rows imported", count)
if __name__ == '__main__':
option_parser = argparse.ArgumentParser(description='CSV importer for pySim-shell\'s PostgreSQL Card Key Provider',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
option_parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False)
option_parser.add_argument('--pgsql', metavar='FILE',
default="~/.osmocom/pysim/card_data_pgsql.cfg",
help='Read card data from PostgreSQL database (config file)')
option_parser.add_argument('--csv', metavar='FILE', help='input CSV file with card data', required=True)
option_parser.add_argument("--table-name", help="name of the card key table", type=str, required=True)
option_parser.add_argument("--update-columns", help="add missing table columns", action='store_true', default=False)
option_parser.add_argument("--create-table", action='store_true', help="create new card key table", default=False)
option_parser.add_argument("--admin", action='store_true', help="perform action as admin", default=False)
opts = option_parser.parse_args()
PySimLogger.setup(print, {logging.WARN: "\033[33m"}, opts.verbose)
# Open CSV file
cr = open_csv(opts)
# Open database, create initial table, update column scheme
db = open_db(cr, opts)
# Progress with import
if not opts.admin:
import_from_csv(db, cr)
# Commit changes to the database
db.commit()

73
contrib/eidtool.py Executable file
View File

@@ -0,0 +1,73 @@
#!/usr/bin/env python3
# Command line tool to compute or verify EID (eUICC ID) values
#
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import argparse
from pySim.euicc import compute_eid_checksum, verify_eid_checksum
option_parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,
description="""pySim EID Tool
This utility program can be used to compute or verify the checksum of an EID
(eUICC Identifier). See GSMA SGP.29 for the algorithm details.
Example (verification):
$ eidtool.py --verify 89882119900000000000000000001654
EID checksum verified successfully
Example (generation, passing first 30 digits):
$ eidtool.py --compute 898821199000000000000000000016
89882119900000000000000000001654
Example (generation, passing all 32 digits):
$ eidtool.py --compute 89882119900000000000000000001600
89882119900000000000000000001654
Example (generation, specifying base 30 digits and number to add):
$ eidtool.py --compute 898821199000000000000000000000 --add 16
89882119900000000000000000001654
""")
group = option_parser.add_mutually_exclusive_group(required=True)
group.add_argument('--verify', help='Verify given EID csum')
group.add_argument('--compute', help='Generate EID csum')
option_parser.add_argument('--add', type=int, help='Add value to EID base before computing')
if __name__ == '__main__':
opts = option_parser.parse_args()
if opts.verify:
res = verify_eid_checksum(opts.verify)
if res:
print("EID checksum verified successfully")
sys.exit(0)
else:
print("EID checksum invalid")
sys.exit(1)
elif opts.compute:
eid = opts.compute
if opts.add:
if len(eid) != 30:
print("EID base must be 30 digits when using --add")
sys.exit(2)
eid = str(int(eid) + int(opts.add))
res = compute_eid_checksum(eid)
print(res)

84
contrib/es2p_client.py Executable file
View File

@@ -0,0 +1,84 @@
#!/usr/bin/env python3
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import copy
import argparse
from pySim.esim import es2p, ActivationCode
EID_HELP='EID of the eUICC for which eSIM shall be made available'
ICCID_HELP='The ICCID of the eSIM that shall be made available'
MATCHID_HELP='MatchingID that shall be used by profile download'
parser = argparse.ArgumentParser(description="""
Utility to manually issue requests against the ES2+ API of an SM-DP+ according to GSMA SGP.22.""")
parser.add_argument('--url', required=True, help='Base URL of ES2+ API endpoint')
parser.add_argument('--id', required=True, help='Entity identifier passed to SM-DP+')
parser.add_argument('--client-cert', help='X.509 client certificate used to authenticate to server')
parser.add_argument('--server-ca-cert', help="""X.509 CA certificates acceptable for the server side. In
production use cases, this would be the GSMA Root CA (CI) certificate.""")
subparsers = parser.add_subparsers(dest='command',help="The command (API function) to call")
parser_dlo = subparsers.add_parser('download-order', help="ES2+ DownloadOrder function")
parser_dlo.add_argument('--eid', help=EID_HELP)
parser_dlo.add_argument('--iccid', help=ICCID_HELP)
parser_dlo.add_argument('--profileType', help='The profile type of which one eSIM shall be made available')
parser_cfo = subparsers.add_parser('confirm-order', help="ES2+ ConfirmOrder function")
parser_cfo.add_argument('--iccid', required=True, help=ICCID_HELP)
parser_cfo.add_argument('--eid', help=EID_HELP)
parser_cfo.add_argument('--matchingId', help=MATCHID_HELP)
parser_cfo.add_argument('--confirmationCode', help='Confirmation code that shall be used by profile download')
parser_cfo.add_argument('--smdsAddress', help='SM-DS Address')
parser_cfo.add_argument('--releaseFlag', action='store_true', help='Shall the profile be immediately released?')
parser_co = subparsers.add_parser('cancel-order', help="ES2+ CancelOrder function")
parser_co.add_argument('--iccid', required=True, help=ICCID_HELP)
parser_co.add_argument('--eid', help=EID_HELP)
parser_co.add_argument('--matchingId', help=MATCHID_HELP)
parser_co.add_argument('--finalProfileStatusIndicator', required=True, choices=['Available','Unavailable'])
parser_rp = subparsers.add_parser('release-profile', help='ES2+ ReleaseProfile function')
parser_rp.add_argument('--iccid', required=True, help=ICCID_HELP)
if __name__ == '__main__':
opts = parser.parse_args()
#print(opts)
peer = es2p.Es2pApiClient(opts.url, opts.id, server_cert_verify=opts.server_ca_cert, client_cert=opts.client_cert)
data = {}
for k, v in vars(opts).items():
if k in ['url', 'id', 'client_cert', 'server_ca_cert', 'command']:
# remove keys from dict that should not end up in JSON...
continue
if v is not None:
data[k] = v
print(data)
if opts.command == 'download-order':
res = peer.call_downloadOrder(data)
elif opts.command == 'confirm-order':
res = peer.call_confirmOrder(data)
matchingId = res.get('matchingId', None)
smdpAddress = res.get('smdpAddress', None)
if matchingId:
ac = ActivationCode(smdpAddress, matchingId, cc_required=bool(opts.confirmationCode))
print("Activation Code: '%s'" % ac.to_string())
elif opts.command == 'cancel-order':
res = peer.call_cancelOrder(data)
elif opts.command == 'release-profile':
res = peer.call_releaseProfile(data)

100
contrib/es2p_server.py Executable file
View File

@@ -0,0 +1,100 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import argparse
import logging
import json
import asn1tools
import asn1tools.codecs.ber
import asn1tools.codecs.der
import pySim.esim.rsp as rsp
import pySim.esim.saip as saip
from pySim.esim.es2p import param, Es2pApiServerMno, Es2pApiServerHandlerMno
from osmocom.utils import b2h
from datetime import datetime
from analyze_simaResponse import split_sima_response
from pathlib import Path
logger = logging.getLogger(Path(__file__).stem)
parser = argparse.ArgumentParser(description="""
Utility to receive and log requests against the ES2+ API of an SM-DP+ according to GSMA SGP.22.""")
parser.add_argument("--host", help="Host/IP to bind HTTP(S) to", default="localhost")
parser.add_argument("--port", help="TCP port to bind HTTP(S) to", default=443, type=int)
parser.add_argument('--server-cert', help='X.509 server certificate used to provide the ES2+ HTTPs service')
parser.add_argument('--client-ca-cert', help='X.509 CA certificates to authenticate the requesting client(s)')
parser.add_argument("-v", "--verbose", help="enable debug output", action='store_true', default=False)
def decode_sima_response(sima_response):
decoded = []
euicc_response_list = split_sima_response(sima_response)
for euicc_response in euicc_response_list:
decoded.append(saip.asn1.decode('EUICCResponse', euicc_response))
return decoded
def decode_result_data(result_data):
return rsp.asn1.decode('PendingNotification', result_data)
def decode(data, path="/"):
if data is None:
return 'none'
elif type(data) is datetime:
return data.isoformat()
elif type(data) is tuple:
return {str(data[0]) : decode(data[1], path + str(data[0]) + "/")}
elif type(data) is list:
new_data = []
for item in data:
new_data.append(decode(item, path))
return new_data
elif type(data) is bytes:
return b2h(data)
elif type(data) is dict:
new_data = {}
for key, item in data.items():
new_key = str(key)
if path == '/' and new_key == 'resultData':
new_item = decode_result_data(item)
elif (path == '/resultData/profileInstallationResult/profileInstallationResultData/finalResult/successResult/' \
or path == '/resultData/profileInstallationResult/profileInstallationResultData/finalResult/errorResult/') \
and new_key == 'simaResponse':
new_item = decode_sima_response(item)
else:
new_item = item
new_data[new_key] = decode(new_item, path + new_key + "/")
return new_data
else:
return data
class Es2pApiServerHandlerForLogging(Es2pApiServerHandlerMno):
def call_handleDownloadProgressInfo(self, data: dict) -> (dict, str):
logging.info("ES2+:handleDownloadProgressInfo: %s" % json.dumps(decode(data)))
return {}, None
if __name__ == "__main__":
args = parser.parse_args()
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING,
format='%(asctime)s %(levelname)s %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
Es2pApiServerMno(args.port, args.host, Es2pApiServerHandlerForLogging(), args.server_cert, args.client_ca_cert)

318
contrib/es9p_client.py Executable file
View File

@@ -0,0 +1,318 @@
#!/usr/bin/env python3
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import sys
import argparse
import logging
import hashlib
from typing import List
from urllib.parse import urlparse
from cryptography import x509
from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat
from cryptography.hazmat.primitives.asymmetric import ec
from osmocom.utils import h2b, b2h, swap_nibbles, is_hexstr
from osmocom.tlv import bertlv_parse_one_rawtag, bertlv_return_one_rawtlv
import pySim.esim.rsp as rsp
from pySim.esim import es9p, PMO
from pySim.esim.x509_cert import CertAndPrivkey
from pySim.esim.es8p import BoundProfilePackage
logging.basicConfig(level=logging.DEBUG)
parser = argparse.ArgumentParser(description="""
Utility to manually issue requests against the ES9+ API of an SM-DP+ according to GSMA SGP.22.""")
parser.add_argument('--url', required=True, help='Base URL of ES9+ API endpoint')
parser.add_argument('--server-ca-cert', help="""X.509 CA certificates acceptable for the server side. In
production use cases, this would be the GSMA Root CA (CI) certificate.""")
parser.add_argument('--certificate-path', default='.',
help="Path in which to look for certificate and key files.")
parser.add_argument('--euicc-certificate', default='CERT_EUICC_ECDSA_NIST.der',
help="File name of DER-encoded eUICC certificate file.")
parser.add_argument('--euicc-private-key', default='SK_EUICC_ECDSA_NIST.pem',
help="File name of PEM-format eUICC secret key file.")
parser.add_argument('--eum-certificate', default='CERT_EUM_ECDSA_NIST.der',
help="File name of DER-encoded EUM certificate file.")
parser.add_argument('--ci-certificate', default='CERT_CI_ECDSA_NIST.der',
help="File name of DER-encoded CI certificate file.")
subparsers = parser.add_subparsers(dest='command',help="The command (API function) to call", required=True)
# download
parser_dl = subparsers.add_parser('download', help="ES9+ download")
parser_dl.add_argument('--matchingId', required=True,
help='MatchingID that shall be used by profile download')
parser_dl.add_argument('--output-path', default='.',
help="Path to which the output files will be written.")
parser_dl.add_argument('--confirmation-code',
help="Confirmation Code for the eSIM download")
# notification
parser_ntf = subparsers.add_parser('notification', help='ES9+ (other) notification')
parser_ntf.add_argument('operation', choices=['enable','disable','delete'],
help='Profile Management Operation whoise occurrence shall be notififed')
parser_ntf.add_argument('--sequence-nr', type=int, required=True,
help='eUICC global notification sequence number')
parser_ntf.add_argument('--notification-address', help='notificationAddress, if different from URL')
parser_ntf.add_argument('--iccid', type=is_hexstr, help='ICCID to which the notification relates')
# notification-install
parser_ntfi = subparsers.add_parser('notification-install', help='ES9+ installation notification')
parser_ntfi.add_argument('--sequence-nr', type=int, required=True,
help='eUICC global notification sequence number')
parser_ntfi.add_argument('--transaction-id', required=True,
help='transactionId of previous ES9+ download')
parser_ntfi.add_argument('--notification-address', help='notificationAddress, if different from URL')
parser_ntfi.add_argument('--iccid', type=is_hexstr, help='ICCID to which the notification relates')
parser_ntfi.add_argument('--smdpp-oid', required=True, help='SM-DP+ OID (as in CERT.DPpb.ECDSA)')
parser_ntfi.add_argument('--isdp-aid', type=is_hexstr, required=True,
help='AID of the ISD-P of the installed profile')
parser_ntfi.add_argument('--sima-response', type=is_hexstr, required=True,
help='hex digits of BER-encoded SAIP EUICCResponse')
class Es9pClient:
def __init__(self, opts):
self.opts = opts
self.cert_and_key = CertAndPrivkey()
self.cert_and_key.cert_from_der_file(os.path.join(opts.certificate_path, opts.euicc_certificate))
self.cert_and_key.privkey_from_pem_file(os.path.join(opts.certificate_path, opts.euicc_private_key))
with open(os.path.join(opts.certificate_path, opts.eum_certificate), 'rb') as f:
self.eum_cert = x509.load_der_x509_certificate(f.read())
with open(os.path.join(opts.certificate_path, opts.ci_certificate), 'rb') as f:
self.ci_cert = x509.load_der_x509_certificate(f.read())
subject_exts = list(filter(lambda x: isinstance(x.value, x509.SubjectKeyIdentifier), self.ci_cert.extensions))
subject_pkid = subject_exts[0].value
self.ci_pkid = subject_pkid.key_identifier
print("EUICC: %s" % self.cert_and_key.cert.subject)
print("EUM: %s" % self.eum_cert.subject)
print("CI: %s" % self.ci_cert.subject)
self.eid = self.cert_and_key.cert.subject.get_attributes_for_oid(x509.oid.NameOID.SERIAL_NUMBER)[0].value
print("EID: %s" % self.eid)
print("CI PKID: %s" % b2h(self.ci_pkid))
print()
self.peer = es9p.Es9pApiClient(opts.url, server_cert_verify=opts.server_ca_cert)
def do_notification(self):
ntf_metadata = {
'seqNumber': self.opts.sequence_nr,
'profileManagementOperation': PMO(self.opts.operation).to_bitstring(),
'notificationAddress': self.opts.notification_address or urlparse(self.opts.url).netloc,
}
if self.opts.iccid:
ntf_metadata['iccid'] = h2b(swap_nibbles(self.opts.iccid))
if self.opts.operation == 'install':
pird = {
'transactionId': h2b(self.opts.transaction_id),
'notificationMetadata': ntf_metadata,
'smdpOid': self.opts.smdpp_oid,
'finalResult': ('successResult', {
'aid': h2b(self.opts.isdp_aid),
'simaResponse': h2b(self.opts.sima_response),
}),
}
pird_bin = rsp.asn1.encode('ProfileInstallationResultData', pird)
signature = self.cert_and_key.ecdsa_sign(pird_bin)
pn_dict = ('profileInstallationResult', {
'profileInstallationResultData': pird,
'euiccSignPIR': signature,
})
else:
ntf_bin = rsp.asn1.encode('NotificationMetadata', ntf_metadata)
signature = self.cert_and_key.ecdsa_sign(ntf_bin)
pn_dict = ('otherSignedNotification', {
'tbsOtherNotification': ntf_metadata,
'euiccNotificationSignature': signature,
'euiccCertificate': rsp.asn1.decode('Certificate', self.cert_and_key.get_cert_as_der()),
'eumCertificate': rsp.asn1.decode('Certificate', self.eum_cert.public_bytes(Encoding.DER)),
})
data = {
'pendingNotification': pn_dict,
}
#print(data)
res = self.peer.call_handleNotification(data)
def do_download(self):
print("Step 1: InitiateAuthentication...")
euiccInfo1 = {
'svn': b'\x02\x04\x00',
'euiccCiPKIdListForVerification': [
self.ci_pkid,
],
'euiccCiPKIdListForSigning': [
self.ci_pkid,
],
}
data = {
'euiccChallenge': os.urandom(16),
'euiccInfo1': euiccInfo1,
'smdpAddress': urlparse(self.opts.url).netloc,
}
init_auth_res = self.peer.call_initiateAuthentication(data)
print(init_auth_res)
print("Step 2: AuthenticateClient...")
#res['serverSigned1']
#res['serverSignature1']
print("TODO: verify serverSignature1 over serverSigned1")
#res['transactionId']
print("TODO: verify transactionId matches the signed one in serverSigned1")
#res['euiccCiPKIdToBeUsed']
# TODO: select eUICC certificate based on CI
#res['serverCertificate']
# TODO: verify server certificate against CI
euiccInfo2 = {
'profileVersion': b'\x02\x03\x01',
'svn': euiccInfo1['svn'],
'euiccFirmwareVer': b'\x23\x42\x00',
'extCardResource': b'\x81\x01\x00\x82\x04\x00\x04\x9ch\x83\x02"#',
'uiccCapability': (b'k6\xd3\xc3', 32),
'javacardVersion': b'\x11\x02\x00',
'globalplatformVersion': b'\x02\x03\x00',
'rspCapability': (b'\x9c', 6),
'euiccCiPKIdListForVerification': euiccInfo1['euiccCiPKIdListForVerification'],
'euiccCiPKIdListForSigning': euiccInfo1['euiccCiPKIdListForSigning'],
#'euiccCategory':
#'forbiddenProfilePolicyRules':
'ppVersion': b'\x01\x00\x00',
'sasAcreditationNumber': 'OSMOCOM-TEST-1', #TODO: make configurable
#'certificationDataObject':
}
euiccSigned1 = {
'transactionId': h2b(init_auth_res['transactionId']),
'serverAddress': init_auth_res['serverSigned1']['serverAddress'],
'serverChallenge': init_auth_res['serverSigned1']['serverChallenge'],
'euiccInfo2': euiccInfo2,
'ctxParams1':
('ctxParamsForCommonAuthentication', {
'matchingId': self.opts.matchingId,
'deviceInfo': {
'tac': b'\x35\x23\x01\x45', # same as lpac
'deviceCapabilities': {},
#imei:
}
}),
}
euiccSigned1_bin = rsp.asn1.encode('EuiccSigned1', euiccSigned1)
euiccSignature1 = self.cert_and_key.ecdsa_sign(euiccSigned1_bin)
auth_clnt_req = {
'transactionId': init_auth_res['transactionId'],
'authenticateServerResponse':
('authenticateResponseOk', {
'euiccSigned1': euiccSigned1,
'euiccSignature1': euiccSignature1,
'euiccCertificate': rsp.asn1.decode('Certificate', self.cert_and_key.get_cert_as_der()),
'eumCertificate': rsp.asn1.decode('Certificate', self.eum_cert.public_bytes(Encoding.DER))
})
}
auth_clnt_res = self.peer.call_authenticateClient(auth_clnt_req)
print(auth_clnt_res)
#auth_clnt_res['transactionId']
print("TODO: verify transactionId matches previous ones")
#auth_clnt_res['profileMetadata']
# TODO: what's in here?
#auth_clnt_res['smdpSigned2']['bppEuiccOtpk']
#auth_clnt_res['smdpSignature2']
print("TODO: verify serverSignature2 over smdpSigned2")
smdp_cert = x509.load_der_x509_certificate(auth_clnt_res['smdpCertificate'])
print("Step 3: GetBoundProfilePackage...")
# Generate a one-time ECKA key pair (ot{PK,SK}.DP.ECKA) using the curve indicated by the Key Parameter
# Reference value of CERT.DPpb.ECDSA
euicc_ot = ec.generate_private_key(smdp_cert.public_key().public_numbers().curve)
# extract the public key in (hopefully) the right format for the ES8+ interface
euicc_otpk = euicc_ot.public_key().public_bytes(Encoding.X962, PublicFormat.UncompressedPoint)
euiccSigned2 = {
'transactionId': h2b(auth_clnt_res['transactionId']),
'euiccOtpk': euicc_otpk,
#hashCC
}
# check for smdpSigned2 ccRequiredFlag, and send it in PrepareDownloadRequest hashCc
if auth_clnt_res['smdpSigned2']['ccRequiredFlag']:
if not self.opts.confirmation_code:
raise ValueError('Confirmation Code required but not provided')
cc_hash = hashlib.sha256(self.opts.confirmation_code.encode('ascii')).digest()
euiccSigned2['hashCc'] = hashlib.sha256(cc_hash + euiccSigned2['transactionId']).digest()
euiccSigned2_bin = rsp.asn1.encode('EUICCSigned2', euiccSigned2)
euiccSignature2 = self.cert_and_key.ecdsa_sign(euiccSigned2_bin + auth_clnt_res['smdpSignature2'])
gbp_req = {
'transactionId': auth_clnt_res['transactionId'],
'prepareDownloadResponse':
('downloadResponseOk', {
'euiccSigned2': euiccSigned2,
'euiccSignature2': euiccSignature2,
})
}
gbp_res = self.peer.call_getBoundProfilePackage(gbp_req)
print(gbp_res)
#gbp_res['transactionId']
# TODO: verify transactionId
print("TODO: verify transactionId matches previous ones")
bpp_bin = gbp_res['boundProfilePackage']
print("TODO: verify boundProfilePackage smdpSignature")
bpp = BoundProfilePackage()
upp_bin = bpp.decode(euicc_ot, self.eid, bpp_bin)
iccid = swap_nibbles(b2h(bpp.storeMetadataRequest['iccid']))
base_name = os.path.join(self.opts.output_path, '%s' % iccid)
print("SUCCESS: Storing files as %s.*.der" % base_name)
# write various output files
with open(base_name+'.upp.der', 'wb') as f:
f.write(bpp.upp)
with open(base_name+'.isdp.der', 'wb') as f:
f.write(bpp.encoded_configureISDPRequest)
with open(base_name+'.smr.der', 'wb') as f:
f.write(bpp.encoded_storeMetadataRequest)
if __name__ == '__main__':
opts = parser.parse_args()
c = Es9pClient(opts)
if opts.command == 'download':
c.do_download()
elif opts.command == 'notification':
c.do_notification()
elif opts.command == 'notification-install':
opts.operation = 'install'
c.do_notification()

48
contrib/esim-qrcode-gen.py Executable file
View File

@@ -0,0 +1,48 @@
#!/usr/bin/env python3
# Small command line utility program to encode eSIM QR-Codes
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import argparse
from pySim.esim import ActivationCode
option_parser = argparse.ArgumentParser(description="""
eSIM QR code generator. Will encode the given hostname + activation code
into the eSIM RSP String format as specified in SGP.22 Section 4.1. If
a PNG output file is specified, it will also generate a QR code.""")
option_parser.add_argument('hostname', help='FQDN of SM-DP+')
option_parser.add_argument('token', help='MatchingID / Token')
option_parser.add_argument('--oid', help='SM-DP+ OID in CERT.DPauth.ECDSA')
option_parser.add_argument('--confirmation-code-required', action='store_true',
help='Whether a Confirmation Code is required')
option_parser.add_argument('--png', help='Output PNG file name (no PNG is written if omitted)')
if __name__ == '__main__':
opts = option_parser.parse_args()
ac = ActivationCode(opts.hostname, opts.token, opts.oid, opts.confirmation_code_required)
print(ac.to_string())
if opts.png:
with open(opts.png, 'wb') as f:
img = ac.to_qrcode()
img.save(f)
print("# generated QR code stored to '%s'" % (opts.png))

40
contrib/esim_gen_metadata.py Executable file
View File

@@ -0,0 +1,40 @@
#!/usr/bin/env python3
# (C) 2025 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import argparse
from osmocom.utils import h2b, swap_nibbles
from pySim.esim.es8p import ProfileMetadata
parser = argparse.ArgumentParser(description="""Utility program to generate profile metadata in the
StoreMetadataRequest format based on input values from the command line.""")
parser.add_argument('--iccid', required=True, help="ICCID of eSIM profile");
parser.add_argument('--spn', required=True, help="Service Provider Name");
parser.add_argument('--profile-name', required=True, help="eSIM Profile Name");
parser.add_argument('--profile-class', choices=['test', 'operational', 'provisioning'],
default='operational', help="Profile Class");
parser.add_argument('--outfile', required=True, help="Output File Name");
if __name__ == '__main__':
opts = parser.parse_args()
iccid_bin = h2b(swap_nibbles(opts.iccid))
pmd = ProfileMetadata(iccid_bin, spn=opts.spn, profile_name=opts.profile_name,
profile_class=opts.profile_class)
with open(opts.outfile, 'wb') as f:
f.write(pmd.gen_store_metadata_request())
print("Written StoreMetadataRequest to '%s'" % opts.outfile)

169
contrib/fsdump-diff-apply.py Executable file
View File

@@ -0,0 +1,169 @@
#!/usr/bin/env python3
# The purpose of this script is to
# * load two SIM card 'fsdump' files
# * determine which file contents in "B" differs from that of "A"
# * create a pySim-shell script to update the contents of "A" to match that of "B"
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import json
import argparse
# Files that we should not update
FILES_TO_SKIP = [
"MF/EF.ICCID",
#"MF/DF.GSM/EF.IMSI",
#"MF/ADF.USIM/EF.IMSI",
]
# Files that need zero-padding at the end, not ff-padding
FILES_PAD_ZERO = [
"DF.GSM/EF.SST",
"MF/ADF.USIM/EF.UST",
"MF/ADF.USIM/EF.EST",
"MF/ADF.ISIM/EF.IST",
]
def pad_file(path, instr, byte_len):
if path in FILES_PAD_ZERO:
pad = '0'
else:
pad = 'f'
return pad_hexstr(instr, byte_len, pad)
def pad_hexstr(instr, byte_len:int, pad='f'):
"""Pad given hex-string to the number of bytes given in byte_len, using ff as padding."""
if len(instr) == byte_len*2:
return instr
elif len(instr) > byte_len*2:
raise ValueError('Cannot pad string of length %u to smaller length %u' % (len(instr)/2, byte_len))
else:
return instr + pad * (byte_len*2 - len(instr))
def is_all_ff(instr):
"""Determine if the entire input hex-string consists of f-digits."""
if all([x == 'f' for x in instr.lower()]):
return True
else:
return False
parser = argparse.ArgumentParser()
parser.add_argument('file_a')
parser.add_argument('file_b')
if __name__ == '__main__':
opts = parser.parse_args()
with open(opts.file_a, 'r') as file_a:
json_a = json.loads(file_a.read())
with open(opts.file_b, 'r') as file_b:
json_b = json.loads(file_b.read())
for path in json_b.keys():
print()
print("# %s" % path)
if not path in json_a:
raise ValueError("%s doesn't exist in file_a!" % path)
if path in FILES_TO_SKIP:
print("# skipped explicitly as it is in FILES_TO_SKIP")
continue
if not 'body' in json_b[path]:
print("# file doesn't exist in B so we cannot possibly need to modify A")
continue
if not 'body' in json_a[path]:
# file was not readable in original (permissions? deactivated?)
print("# ERROR: %s not readable in A; please fix that" % path)
continue
body_a = json_a[path]['body']
body_b = json_b[path]['body']
if body_a == body_b:
print("# file body is identical")
continue
file_size_a = json_a[path]['fcp']['file_size']
file_size_b = json_b[path]['fcp']['file_size']
cmds = []
structure = json_b[path]['fcp']['file_descriptor']['file_descriptor_byte']['structure']
if structure == 'transparent':
val_a = body_a
val_b = body_b
if file_size_a < file_size_b:
if not is_all_ff(val_b[2*file_size_a:]):
print("# ERROR: file_size_a (%u) < file_size_b (%u); please fix!" % (file_size_a, file_size_b))
continue
else:
print("# WARN: file_size_a (%u) < file_size_b (%u); please fix!" % (file_size_a, file_size_b))
# truncate val_b to fit in A
val_b = val_b[:file_size_a*2]
elif file_size_a != file_size_b:
print("# NOTE: file_size_a (%u) != file_size_b (%u)" % (file_size_a, file_size_b))
# Pad to file_size_a
val_b = pad_file(path, val_b, file_size_a)
if val_b != val_a:
cmds.append("update_binary %s" % val_b)
else:
print("# padded file body is identical")
elif structure in ['linear_fixed', 'cyclic']:
record_len_a = json_a[path]['fcp']['file_descriptor']['record_len']
record_len_b = json_b[path]['fcp']['file_descriptor']['record_len']
if record_len_a < record_len_b:
print("# ERROR: record_len_a (%u) < record_len_b (%u); please fix!" % (file_size_a, file_size_b))
continue
elif record_len_a != record_len_b:
print("# NOTE: record_len_a (%u) != record_len_b (%u)" % (record_len_a, record_len_b))
num_rec_a = file_size_a // record_len_a
num_rec_b = file_size_b // record_len_b
if num_rec_a < num_rec_b:
if not all([is_all_ff(x) for x in body_b[num_rec_a:]]):
print("# ERROR: num_rec_a (%u) < num_rec_b (%u); please fix!" % (num_rec_a, num_rec_b))
continue
else:
print("# WARN: num_rec_a (%u) < num_rec_b (%u); but they're empty" % (num_rec_a, num_rec_b))
elif num_rec_a != num_rec_b:
print("# NOTE: num_rec_a (%u) != num_rec_b (%u)" % (num_rec_a, num_rec_b))
i = 0
for r in body_b:
if i < len(body_a):
break
val_a = body_a[i]
# Pad to record_len_a
val_b = pad_file(path, body_b[i], record_len_a)
if val_a != val_b:
cmds.append("update_record %u %s" % (i+1, val_b))
i = i + 1
if len(cmds) == 0:
print("# padded file body is identical")
elif structure == 'ber_tlv':
print("# FIXME: Implement BER-TLV")
else:
raise ValueError('Unsupported structure %s' % structure)
if len(cmds):
print("select %s" % path)
for cmd in cmds:
print(cmd)

661
contrib/generate_smdpp_certs.py Executable file
View File

@@ -0,0 +1,661 @@
#!/usr/bin/env python3
"""
Faithfully reproduces the smdpp certs contained in SGP.26_v1.5_Certificates_18_07_2024.zip
available at https://www.gsma.com/solutions-and-impact/technologies/esim/gsma_resources/sgp-26-test-certificate-definition-v1-5/
Only usable for testing, it obviously uses a different CI key.
"""
import os
import binascii
from datetime import datetime
from cryptography import x509
from cryptography.x509.oid import NameOID, ExtensionOID
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.backends import default_backend
# Custom OIDs used in certificates
OID_CERTIFICATE_POLICIES_CI = "2.23.146.1.2.1.0" # CI cert policy
OID_CERTIFICATE_POLICIES_TLS = "2.23.146.1.2.1.3" # DPtls cert policy
OID_CERTIFICATE_POLICIES_AUTH = "2.23.146.1.2.1.4" # DPauth cert policy
OID_CERTIFICATE_POLICIES_PB = "2.23.146.1.2.1.5" # DPpb cert policy
# Subject Alternative Name OIDs
OID_CI_RID = "2.999.1" # CI Registered ID
OID_DP_RID = "2.999.10" # DP+ Registered ID
OID_DP2_RID = "2.999.12" # DP+2 Registered ID
OID_DP4_RID = "2.999.14" # DP+4 Registered ID
OID_DP8_RID = "2.999.18" # DP+8 Registered ID
class SimplifiedCertificateGenerator:
def __init__(self):
self.backend = default_backend()
# Store generated CI keys to sign other certs
self.ci_certs = {} # {"BRP": cert, "NIST": cert}
self.ci_keys = {} # {"BRP": key, "NIST": key}
def get_curve(self, curve_type):
"""Get the appropriate curve object."""
if curve_type == "BRP":
return ec.BrainpoolP256R1()
else:
return ec.SECP256R1()
def generate_key_pair(self, curve):
"""Generate a new EC key pair."""
private_key = ec.generate_private_key(curve, self.backend)
return private_key
def load_private_key_from_hex(self, hex_key, curve):
"""Load EC private key from hex string."""
key_bytes = binascii.unhexlify(hex_key.replace(":", "").replace(" ", "").replace("\n", ""))
key_int = int.from_bytes(key_bytes, 'big')
return ec.derive_private_key(key_int, curve, self.backend)
def generate_ci_cert(self, curve_type):
"""Generate CI certificate for either BRP or NIST curve."""
curve = self.get_curve(curve_type)
private_key = self.generate_key_pair(curve)
# Build subject and issuer (self-signed) - same for both
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COMMON_NAME, "Test CI"),
x509.NameAttribute(NameOID.ORGANIZATIONAL_UNIT_NAME, "TESTCERT"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "RSPTEST"),
x509.NameAttribute(NameOID.COUNTRY_NAME, "IT"),
])
# Build certificate - all parameters same for both
builder = x509.CertificateBuilder()
builder = builder.subject_name(subject)
builder = builder.issuer_name(issuer)
builder = builder.not_valid_before(datetime(2020, 4, 1, 8, 27, 51))
builder = builder.not_valid_after(datetime(2055, 4, 1, 8, 27, 51))
builder = builder.serial_number(0xb874f3abfa6c44d3)
builder = builder.public_key(private_key.public_key())
# Add extensions - all same for both
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.BasicConstraints(ca=True, path_length=None),
critical=True
)
builder = builder.add_extension(
x509.CertificatePolicies([
x509.PolicyInformation(
x509.ObjectIdentifier(OID_CERTIFICATE_POLICIES_CI),
policy_qualifiers=None
)
]),
critical=True
)
builder = builder.add_extension(
x509.KeyUsage(
digital_signature=False,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=True,
crl_sign=True,
encipher_only=False,
decipher_only=False
),
critical=True
)
builder = builder.add_extension(
x509.SubjectAlternativeName([
x509.RegisteredID(x509.ObjectIdentifier(OID_CI_RID))
]),
critical=False
)
builder = builder.add_extension(
x509.CRLDistributionPoints([
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-A.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
),
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-B.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
)
]),
critical=False
)
certificate = builder.sign(private_key, hashes.SHA256(), self.backend)
self.ci_keys[curve_type] = private_key
self.ci_certs[curve_type] = certificate
return certificate, private_key
def generate_dp_cert(self, curve_type, subject_cn, serial, key_hex,
cert_policy_oid, rid_oid, validity_start, validity_end):
"""Generate a DP certificate signed by CI - works for both BRP and NIST."""
curve = self.get_curve(curve_type)
private_key = self.load_private_key_from_hex(key_hex, curve)
ci_cert = self.ci_certs[curve_type]
ci_key = self.ci_keys[curve_type]
subject = x509.Name([
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "ACME"),
x509.NameAttribute(NameOID.COMMON_NAME, subject_cn),
])
builder = x509.CertificateBuilder()
builder = builder.subject_name(subject)
builder = builder.issuer_name(ci_cert.subject)
builder = builder.not_valid_before(validity_start)
builder = builder.not_valid_after(validity_end)
builder = builder.serial_number(serial)
builder = builder.public_key(private_key.public_key())
builder = builder.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_public_key(ci_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.SubjectAlternativeName([
x509.RegisteredID(x509.ObjectIdentifier(rid_oid))
]),
critical=False
)
builder = builder.add_extension(
x509.KeyUsage(
digital_signature=True,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=False,
crl_sign=False,
encipher_only=False,
decipher_only=False
),
critical=True
)
builder = builder.add_extension(
x509.CertificatePolicies([
x509.PolicyInformation(
x509.ObjectIdentifier(cert_policy_oid),
policy_qualifiers=None
)
]),
critical=True
)
builder = builder.add_extension(
x509.CRLDistributionPoints([
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-A.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
),
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-B.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
)
]),
critical=False
)
certificate = builder.sign(ci_key, hashes.SHA256(), self.backend)
return certificate, private_key
def generate_tls_cert(self, curve_type, subject_cn, dns_name, serial, key_hex,
rid_oid, validity_start, validity_end):
"""Generate a TLS certificate signed by CI."""
curve = self.get_curve(curve_type)
private_key = self.load_private_key_from_hex(key_hex, curve)
ci_cert = self.ci_certs[curve_type]
ci_key = self.ci_keys[curve_type]
subject = x509.Name([
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "ACME"),
x509.NameAttribute(NameOID.COMMON_NAME, subject_cn),
])
builder = x509.CertificateBuilder()
builder = builder.subject_name(subject)
builder = builder.issuer_name(ci_cert.subject)
builder = builder.not_valid_before(validity_start)
builder = builder.not_valid_after(validity_end)
builder = builder.serial_number(serial)
builder = builder.public_key(private_key.public_key())
builder = builder.add_extension(
x509.KeyUsage(
digital_signature=True,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=False,
crl_sign=False,
encipher_only=False,
decipher_only=False
),
critical=True
)
builder = builder.add_extension(
x509.ExtendedKeyUsage([
x509.oid.ExtendedKeyUsageOID.SERVER_AUTH,
x509.oid.ExtendedKeyUsageOID.CLIENT_AUTH
]),
critical=True
)
builder = builder.add_extension(
x509.CertificatePolicies([
x509.PolicyInformation(
x509.ObjectIdentifier(OID_CERTIFICATE_POLICIES_TLS),
policy_qualifiers=None
)
]),
critical=False
)
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_public_key(ci_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.SubjectAlternativeName([
x509.DNSName(dns_name),
x509.RegisteredID(x509.ObjectIdentifier(rid_oid))
]),
critical=False
)
builder = builder.add_extension(
x509.CRLDistributionPoints([
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-A.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
),
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-B.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
)
]),
critical=False
)
certificate = builder.sign(ci_key, hashes.SHA256(), self.backend)
return certificate, private_key
def generate_eum_cert(self, curve_type, key_hex):
"""Generate EUM certificate signed by CI."""
curve = self.get_curve(curve_type)
private_key = self.load_private_key_from_hex(key_hex, curve)
ci_cert = self.ci_certs[curve_type]
ci_key = self.ci_keys[curve_type]
subject = x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, "ES"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "RSP Test EUM"),
x509.NameAttribute(NameOID.COMMON_NAME, "EUM Test"),
])
builder = x509.CertificateBuilder()
builder = builder.subject_name(subject)
builder = builder.issuer_name(ci_cert.subject)
builder = builder.not_valid_before(datetime(2020, 4, 1, 9, 28, 37))
builder = builder.not_valid_after(datetime(2054, 3, 24, 9, 28, 37))
builder = builder.serial_number(0x12345678)
builder = builder.public_key(private_key.public_key())
builder = builder.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_public_key(ci_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.KeyUsage(
digital_signature=False,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=True,
crl_sign=False,
encipher_only=False,
decipher_only=False
),
critical=True
)
builder = builder.add_extension(
x509.CertificatePolicies([
x509.PolicyInformation(
x509.ObjectIdentifier("2.23.146.1.2.1.2"), # EUM policy
policy_qualifiers=None
)
]),
critical=True
)
builder = builder.add_extension(
x509.SubjectAlternativeName([
x509.RegisteredID(x509.ObjectIdentifier("2.999.5"))
]),
critical=False
)
builder = builder.add_extension(
x509.BasicConstraints(ca=True, path_length=0),
critical=True
)
builder = builder.add_extension(
x509.CRLDistributionPoints([
x509.DistributionPoint(
full_name=[x509.UniformResourceIdentifier("http://ci.test.example.com/CRL-B.crl")],
relative_name=None,
reasons=None,
crl_issuer=None
)
]),
critical=False
)
# Name Constraints
constrained_name = x509.Name([
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "RSP Test EUM"),
x509.NameAttribute(NameOID.SERIAL_NUMBER, "89049032"),
])
name_constraints = x509.NameConstraints(
permitted_subtrees=[
x509.DirectoryName(constrained_name)
],
excluded_subtrees=None
)
builder = builder.add_extension(
name_constraints,
critical=True
)
certificate = builder.sign(ci_key, hashes.SHA256(), self.backend)
return certificate, private_key
def generate_euicc_cert(self, curve_type, eum_cert, eum_key, key_hex):
"""Generate eUICC certificate signed by EUM."""
curve = self.get_curve(curve_type)
private_key = self.load_private_key_from_hex(key_hex, curve)
subject = x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, "ES"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "RSP Test EUM"),
x509.NameAttribute(NameOID.SERIAL_NUMBER, "89049032123451234512345678901235"),
x509.NameAttribute(NameOID.COMMON_NAME, "Test eUICC"),
])
builder = x509.CertificateBuilder()
builder = builder.subject_name(subject)
builder = builder.issuer_name(eum_cert.subject)
builder = builder.not_valid_before(datetime(2020, 4, 1, 9, 48, 58))
builder = builder.not_valid_after(datetime(7496, 1, 24, 9, 48, 58))
builder = builder.serial_number(0x0200000000000001)
builder = builder.public_key(private_key.public_key())
builder = builder.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_public_key(eum_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.SubjectKeyIdentifier.from_public_key(private_key.public_key()),
critical=False
)
builder = builder.add_extension(
x509.KeyUsage(
digital_signature=True,
content_commitment=False,
key_encipherment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=False,
crl_sign=False,
encipher_only=False,
decipher_only=False
),
critical=True
)
builder = builder.add_extension(
x509.CertificatePolicies([
x509.PolicyInformation(
x509.ObjectIdentifier("2.23.146.1.2.1.1"), # eUICC policy
policy_qualifiers=None
)
]),
critical=True
)
certificate = builder.sign(eum_key, hashes.SHA256(), self.backend)
return certificate, private_key
def save_cert_and_key(self, cert, key, cert_path_der, cert_path_pem, key_path_sk, key_path_pk):
"""Save certificate and key in various formats."""
# Create directories if needed
os.makedirs(os.path.dirname(cert_path_der), exist_ok=True)
with open(cert_path_der, "wb") as f:
f.write(cert.public_bytes(serialization.Encoding.DER))
if cert_path_pem:
with open(cert_path_pem, "wb") as f:
f.write(cert.public_bytes(serialization.Encoding.PEM))
if key and key_path_sk:
with open(key_path_sk, "wb") as f:
f.write(key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption()
))
if key and key_path_pk:
with open(key_path_pk, "wb") as f:
f.write(key.public_key().public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
))
def main():
gen = SimplifiedCertificateGenerator()
output_dir = "smdpp-data/generated"
os.makedirs(output_dir, exist_ok=True)
print("=== Generating CI Certificates ===")
for curve_type in ["BRP", "NIST"]:
ci_cert, ci_key = gen.generate_ci_cert(curve_type)
suffix = "_ECDSA_BRP" if curve_type == "BRP" else "_ECDSA_NIST"
gen.save_cert_and_key(
ci_cert, ci_key,
f"{output_dir}/CertificateIssuer/CERT_CI{suffix}.der",
f"{output_dir}/CertificateIssuer/CERT_CI{suffix}.pem",
None, None
)
print(f"Generated CI {curve_type} certificate")
print("\n=== Generating DPauth Certificates ===")
dpauth_configs = [
("BRP", "TEST SM-DP+", 256, "93:fb:33:d0:58:4f:34:9b:07:f8:b5:d2:af:93:d7:c3:e3:54:b3:49:a3:b9:13:50:2e:6a:bc:07:0e:4d:49:29", OID_DP_RID, "DPauth"),
("NIST", "TEST SM-DP+", 256, "0a:7c:c1:c2:44:e6:0c:52:cd:5b:78:07:ab:8c:36:0c:26:52:46:01:50:7d:ca:bc:5d:d5:98:b5:a6:16:d5:d5", OID_DP_RID, "DPauth"),
("BRP", "TEST SM-DP+2", 512, "0c:17:35:5c:01:1d:0f:e8:d7:da:dd:63:f1:97:85:cf:6c:51:cb:cd:46:6a:e8:8b:e8:f8:1b:c1:05:88:46:f6", OID_DP2_RID, "DP2auth"),
("NIST", "TEST SM-DP+2", 512, "9c:32:a0:95:d4:88:42:d9:ff:a4:04:f7:12:51:2a:a2:c5:42:5a:1a:26:38:6a:b6:a1:45:d5:81:1e:03:91:41", OID_DP2_RID, "DP2auth"),
]
for curve_type, cn, serial, key_hex, rid_oid, name_prefix in dpauth_configs:
cert, key = gen.generate_dp_cert(
curve_type, cn, serial, key_hex,
OID_CERTIFICATE_POLICIES_AUTH, rid_oid,
datetime(2020, 4, 1, 8, 31, 30),
datetime(2030, 3, 30, 8, 31, 30)
)
suffix = "_ECDSA_BRP" if curve_type == "BRP" else "_ECDSA_NIST"
gen.save_cert_and_key(
cert, key,
f"{output_dir}/DPauth/CERT_S_SM_{name_prefix}{suffix}.der",
None,
f"{output_dir}/DPauth/SK_S_SM_{name_prefix}{suffix}.pem",
f"{output_dir}/DPauth/PK_S_SM_{name_prefix}{suffix}.pem"
)
print(f"Generated {name_prefix} {curve_type} certificate")
print("\n=== Generating DPpb Certificates ===")
dppb_configs = [
("BRP", "TEST SM-DP+", 257, "75:ff:32:2f:41:66:16:da:e1:a4:84:ef:71:d4:87:4f:b0:df:32:95:fd:35:c2:cb:a4:89:fb:b2:bb:9c:7b:f6", OID_DP_RID, "DPpb"),
("NIST", "TEST SM-DP+", 257, "dc:d6:94:b7:78:95:7e:8e:9a:dd:bd:d9:44:33:e9:ef:8f:73:d1:1e:49:1c:48:d4:25:a3:8a:94:91:bd:3b:ed", OID_DP_RID, "DPpb"),
("BRP", "TEST SM-DP+2", 513, "9c:ae:2e:1a:56:07:a9:d5:78:38:2e:ee:93:2e:25:1f:52:30:4f:86:ee:b1:f1:70:8c:db:d3:c0:7b:e2:cd:3d", OID_DP2_RID, "DP2pb"),
("NIST", "TEST SM-DP+2", 513, "66:93:11:49:63:9d:ba:ac:1d:c3:d3:06:c5:8b:d2:df:d2:2f:73:bf:63:ac:86:31:98:32:90:b5:7f:90:93:45", OID_DP2_RID, "DP2pb"),
]
for curve_type, cn, serial, key_hex, rid_oid, name_prefix in dppb_configs:
cert, key = gen.generate_dp_cert(
curve_type, cn, serial, key_hex,
OID_CERTIFICATE_POLICIES_PB, rid_oid,
datetime(2020, 4, 1, 8, 34, 46),
datetime(2030, 3, 30, 8, 34, 46)
)
suffix = "_ECDSA_BRP" if curve_type == "BRP" else "_ECDSA_NIST"
gen.save_cert_and_key(
cert, key,
f"{output_dir}/DPpb/CERT_S_SM_{name_prefix}{suffix}.der",
None,
f"{output_dir}/DPpb/SK_S_SM_{name_prefix}{suffix}.pem",
f"{output_dir}/DPpb/PK_S_SM_{name_prefix}{suffix}.pem"
)
print(f"Generated {name_prefix} {curve_type} certificate")
print("\n=== Generating DPtls Certificates ===")
dptls_configs = [
("BRP", "testsmdpplus1.example.com", "testsmdpplus1.example.com", 9, "3f:67:15:28:02:b3:f4:c7:fa:e6:79:58:55:f6:82:54:1e:45:e3:5e:ff:f4:e8:a0:55:65:a0:f1:91:2a:78:2e", OID_DP_RID, "DP_TLS_BRP"),
("NIST", "testsmdpplus1.example.com", "testsmdpplus1.example.com", 9, "a0:3e:7c:e4:55:04:74:be:a4:b7:a8:73:99:ce:5a:8c:9f:66:1b:68:0f:94:01:39:ff:f8:4e:9d:ec:6a:4d:8c", OID_DP_RID, "DP_TLS_NIST"),
("NIST", "testsmdpplus2.example.com", "testsmdpplus2.example.com", 12, "4e:65:61:c6:40:88:f6:69:90:7a:db:e3:94:b1:1a:84:24:2e:03:3a:82:a8:84:02:31:63:6d:c9:1b:4e:e3:f5", OID_DP2_RID, "DP2_TLS"),
("NIST", "testsmdpplus4.example.com", "testsmdpplus4.example.com", 14, "f2:65:9d:2f:52:8f:4b:11:37:40:d5:8a:0d:2a:f3:eb:2b:48:e1:22:c2:b6:0a:6a:f6:fc:96:ad:86:be:6f:a4", OID_DP4_RID, "DP4_TLS"),
("NIST", "testsmdpplus8.example.com", "testsmdpplus8.example.com", 18, "ff:6e:4a:50:9b:ad:db:38:10:88:31:c2:3c:cc:2d:44:30:7a:f2:81:e9:25:96:7f:8c:df:1d:95:54:a0:28:8d", OID_DP8_RID, "DP8_TLS"),
]
for curve_type, cn, dns, serial, key_hex, rid_oid, name_prefix in dptls_configs:
cert, key = gen.generate_tls_cert(
curve_type, cn, dns, serial, key_hex, rid_oid,
datetime(2024, 7, 9, 15, 29, 36),
datetime(2025, 8, 11, 15, 29, 36)
)
gen.save_cert_and_key(
cert, key,
f"{output_dir}/DPtls/CERT_S_SM_{name_prefix}.der",
None,
f"{output_dir}/DPtls/SK_S_SM_{name_prefix.replace('_BRP', '_BRP').replace('_NIST', '_NIST')}.pem",
f"{output_dir}/DPtls/PK_S_SM_{name_prefix.replace('_BRP', '_BRP').replace('_NIST', '_NIST')}.pem"
)
print(f"Generated {name_prefix} certificate")
print("\n=== Generating EUM Certificates ===")
eum_configs = [
("BRP", "12:9b:0a:b1:3f:17:e1:4a:40:b6:fa:4e:d8:23:e0:cf:46:5b:7b:3d:73:24:05:e6:29:5d:3b:23:b0:45:c9:9a"),
("NIST", "25:e6:75:77:28:e1:e9:51:13:51:9c:dc:34:55:5c:29:ba:ed:23:77:3a:c5:af:dd:dc:da:d9:84:89:8a:52:f0"),
]
eum_certs = {}
eum_keys = {}
for curve_type, key_hex in eum_configs:
cert, key = gen.generate_eum_cert(curve_type, key_hex)
eum_certs[curve_type] = cert
eum_keys[curve_type] = key
suffix = "_ECDSA_BRP" if curve_type == "BRP" else "_ECDSA_NIST"
gen.save_cert_and_key(
cert, key,
f"{output_dir}/EUM/CERT_EUM{suffix}.der",
None,
f"{output_dir}/EUM/SK_EUM{suffix}.pem",
f"{output_dir}/EUM/PK_EUM{suffix}.pem"
)
print(f"Generated EUM {curve_type} certificate")
print("\n=== Generating eUICC Certificates ===")
euicc_configs = [
("BRP", "8d:c3:47:a7:6d:b7:bd:d6:22:2d:d7:5e:a1:a1:68:8a:ca:81:1e:4c:bc:6a:7f:6a:ef:a4:b2:64:19:62:0b:90"),
("NIST", "11:e1:54:67:dc:19:4f:33:71:83:e4:60:c9:f6:32:60:09:1e:12:e8:10:26:cd:65:61:e1:7c:6d:85:39:cc:9c"),
]
for curve_type, key_hex in euicc_configs:
cert, key = gen.generate_euicc_cert(curve_type, eum_certs[curve_type], eum_keys[curve_type], key_hex)
suffix = "_ECDSA_BRP" if curve_type == "BRP" else "_ECDSA_NIST"
gen.save_cert_and_key(
cert, key,
f"{output_dir}/eUICC/CERT_EUICC{suffix}.der",
None,
f"{output_dir}/eUICC/SK_EUICC{suffix}.pem",
f"{output_dir}/eUICC/PK_EUICC{suffix}.pem"
)
print(f"Generated eUICC {curve_type} certificate")
print("\n=== Certificate generation complete! ===")
print(f"All certificates saved to: {output_dir}/")
if __name__ == "__main__":
main()

99
contrib/jenkins.sh
View File

@@ -1,32 +1,77 @@
#!/bin/sh
#!/bin/sh -xe
# jenkins build helper script for pysim. This is how we build on jenkins.osmocom.org
#
# environment variables:
# * WITH_MANUALS: build manual PDFs if set to "1"
# * PUBLISH: upload manuals after building if set to "1" (ignored without WITH_MANUALS = "1")
# * JOB_TYPE: one of 'test', 'distcheck', 'pylint', 'docs'
# * SKIP_CLEAN_WORKSPACE: don't run osmo-clean-workspace.sh (for pyosmocom CI)
#
set -e
export PYTHONUNBUFFERED=1
if [ ! -d "./pysim-testdata/" ] ; then
setup_venv() {
virtualenv -p python3 venv --system-site-packages
. venv/bin/activate
}
if [ ! -d "./tests/" ] ; then
echo "###############################################"
echo "Please call from pySim-prog top directory"
echo "###############################################"
exit 1
fi
virtualenv -p python3 venv --system-site-packages
. venv/bin/activate
pip install pytlv
pip install 'pyyaml>=5.1'
pip install cmd2==1.5
pip install jsonpath-ng
pip install construct
pip install bidict
pip install gsm0338
if [ -z "$SKIP_CLEAN_WORKSPACE" ]; then
osmo-clean-workspace.sh
fi
case "$JOB_TYPE" in
"test")
setup_venv
pip install -r requirements.txt
pip install pyshark
# Execute automatically discovered unit tests first
python -m unittest discover -v -s tests/
python -m unittest discover -v -s tests/unittests
# Run pySim-trace test
tests/pySim-trace_test/pySim-trace_test.sh
;;
"card-test") # tests requiring physical cards
setup_venv
pip install -r requirements.txt
# Run pySim-prog integration tests
cd tests/pySim-prog_test/
./pySim-prog_test.sh
cd ../../
# Run pySim-shell integration tests
python3 -m unittest discover -v -s ./tests/pySim-shell_test/
# Run pySim-smpp2sim test
tests/pySim-smpp2sim_test/pySim-smpp2sim_test.sh
;;
"distcheck")
setup_venv
pip install .
pip install pyshark
for prog in venv/bin/pySim-*.py; do
$prog --help > /dev/null
done
;;
"pylint")
# Print pylint version
pip3 freeze | grep pylint
setup_venv
pip install .
# Run pylint to find potential errors
# Ignore E1102: not-callable
@@ -34,23 +79,29 @@ python -m unittest discover -v -s tests/
# Ignore E0401: import-error
# pySim/utils.py:276: E0401: Unable to import 'Crypto.Cipher' (import-error)
# pySim/utils.py:277: E0401: Unable to import 'Crypto.Util.strxor' (import-error)
pip install pylint
python -m pylint --errors-only \
python3 -m pylint -j0 --errors-only \
--disable E1102 \
--disable E0401 \
--enable W0301 \
pySim *.py
pySim tests/unittests/*.py *.py \
contrib/*.py
;;
"docs")
setup_venv
# attempt to build documentation
pip install sphinx
pip install sphinxcontrib-napoleon
pip3 install -e 'git+https://github.com/osmocom/sphinx-argparse@master#egg=sphinx-argparse'
(cd docs && make html latexpdf)
pip install -r requirements.txt
rm -rf docs/_build
make -C "docs" html latexpdf
if [ "$WITH_MANUALS" = "1" ] && [ "$PUBLISH" = "1" ]; then
make -C "docs" publish publish-html
fi
;;
*)
set +x
echo "ERROR: JOB_TYPE has unexpected value '$JOB_TYPE'."
exit 1
esac
# run the test with physical cards
cd pysim-testdata
../tests/pysim-test.sh
osmo-clean-workspace.sh

208
contrib/rcp/rcp_client.py Executable file
View File

@@ -0,0 +1,208 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import os
import websockets
import asyncio
import argparse
import logging
from copy import deepcopy
from pathlib import Path
from pySim.log import PySimLogger
from rcp_utils import CltConnHdlr, backtrace, pytype_to_type, load_ca_cert, load_json_schema, JsonValidator
from pySim.transport import init_reader, argparse_add_reader_args, LinkBase
SERVER_TIMEOUT = 10
log = PySimLogger.get(Path(__file__).stem)
option_parser = argparse.ArgumentParser(description='RCP Client',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
argparse_add_reader_args(option_parser)
option_parser.add_argument("--verbose", help="Enable verbose logging",
action='store_true', default=False)
option_parser.add_argument("--uri", help="URI of the RCP-Server")
option_parser.add_argument("--ca-cert", help="SSL/TLS CA-Certificate of the RCP-Server")
class RcpcCltConnHdlr(CltConnHdlr):
def __init__(self, sl, *args, **kwargs):
self.sl = sl
super().__init__(*args, **kwargs)
async def describe(self, suitable_for:dict) -> list:
log.info("Requesting module descriptions from RCP Server ...")
tx_json = {'rcpc_hello': {'suitable_for' : suitable_for}}
rx_json = await self._transact(tx_json)
module_descr = rx_json['rcpc_welcome']['module_descr']
if not module_descr:
raise ValueError("No RCP module available for this card")
return module_descr
async def run(self, cmd:str, cmd_argv) -> int:
log.info("Executing command with RCP Server ...")
tx_json = {'rcpc_command': {'cmd' : cmd, 'cmd_argv' : cmd_argv}}
while(True):
rx_json = await self._transact(tx_json)
tx_json = None
if 'rcpc_instr' in rx_json:
rcpc_instr = rx_json['rcpc_instr']
if 'c_apdu' in rcpc_instr:
c_apdu = rx_json['rcpc_instr']['c_apdu']
data, sw = sl.send_apdu(c_apdu)
tx_json = {'rcpc_result': {'r_apdu' : {'data': data.upper(), 'sw': sw.upper()}}}
elif 'reset' in rcpc_instr:
sl.reset_card()
atr = sl.get_atr()
tx_json = {'rcpc_result': {'atr' : atr.upper()}}
elif 'print' in rcpc_instr:
log.info(str(self) + " -- %s", rx_json['rcpc_instr']['print'])
tx_json = {'rcpc_result': {'empty' : None}}
elif 'rcpc_goodbye' in rx_json:
rc = rx_json['rcpc_goodbye']
log.info("Command execution done, rc: %d", rc)
return rc
def check_if_user_needs_basic_help(argv):
"""
The '--uri' argument is the minimum requirement to connect to the RCP Server to retrieve the information about the
dynamic commandline arguments. In case this argument is missing while '--help' or '-h' arguments are present. Then
we will fall back to display only a basic help that contains only the static commandline arguments (see above).
"""
if '--help' in argv or '-h' in argv:
if '--uri' not in argv:
option_parser.parse_args()
sys.exit(1)
def parse_known_arguemnts(argv):
"""
Parse the commandline arguments we know so far. Ignore unknown arguments and filter out '--help' and '-h'
arguments, in case those are present.
"""
argv_filtered = deepcopy(argv)
if '--help' in argv_filtered:
argv_filtered.remove('--help')
if '-h' in argv_filtered:
argv_filtered.remove('-h')
opts, unknown = option_parser.parse_known_args(argv_filtered)
return opts
async def run_rcp_session(opts, sl, ssl_context) -> int:
"""
Connect to the RCP Server, retrieve the module description, use the module description to complete the commandline
argument parser, execute the command that the user has selected.
"""
# Request ATR from card
card_atr = sl.get_atr().upper()
log.info("Detected Card with ATR: %s" % card_atr)
# Connect to RCP server
log.info("RCP Server URI: %s" % opts.uri)
async with websockets.connect(opts.uri, ssl=ssl_context) as websocket:
rcpc_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcpc_to_rcps_schema.json"))
rcps_to_rcpc_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcps_to_rcpc_schema.json"))
json_validator = JsonValidator(rcps_to_rcpc_schema, rcpc_to_rcps_schema)
client = RcpcCltConnHdlr(sl, websocket, SERVER_TIMEOUT, json_validator)
# Retrieve module description
module_descrs = await client.describe({"atr" : card_atr})
# Complete the commandlie parser and set up a dict that we can use as filter
# TODO: Maybe it makes sense to integrate this as a method into the RcpcCltConnHdlr class?
option_subparsers = option_parser.add_subparsers(dest='command', help="RCP command to use", required=True)
sys_argv_filter = {}
for module_descr in module_descrs:
cmd_descr = module_descr['cmd_descr']
for cmd in cmd_descr:
command_name = module_descr['name'] + "_" + cmd['name']
option_parser_cmd = option_subparsers.add_parser(command_name, help=cmd['help'])
sys_argv_filter[command_name] = []
for arg in cmd['args']:
arg['spec'] = pytype_to_type(arg['spec'])
option_parser_cmd.add_argument(arg['name'], **arg['spec'])
sys_argv_filter[command_name].append(arg['name'])
# Re-Parse commandline options with the completed commandline parser. In case commandline help is
# requested. The program is able to display the full helpscreen and exists.
opts = option_parser.parse_args()
# Filter the relevant command arguments from sys.argv
cmd_argv = []
next_is_value=False
for arg in sys.argv:
if arg in sys_argv_filter[opts.command]:
cmd_argv.append(arg)
next_is_value=True
elif next_is_value is True:
next_is_value=False
cmd_argv.append(arg)
# Run the command and close the connection
rc = await client.run(opts.command, cmd_argv)
await client.close()
return rc
if __name__ == '__main__':
# Setup logging
PySimLogger.setup(print, {logging.WARN: "\033[33m", logging.DEBUG: "\033[90m"}, '--verbose' in sys.argv)
# Since parts of the commandline arguments are retrieved dynamically, we have to resolve a chicken-egg-problem.
# We cannot call option_parser.parse_args() at the beginning, since we haven't received all information to
# complete the option_parser yet. However in order to retrieve the arguments correctly we need to get the
# URI and the parameters for the smartcard reader before we make the connection. The situation is even further
# complicated in case the user requests commandline help.
# To resolve the problem we first check if the user needs basic help (no '--uri' parameter present). If this is the
# case, the program will exit with a basic helpscreen.
check_if_user_needs_basic_help(sys.argv)
# In all other cases we parse the arguments we know so far. In case the user requests commandline help, we will
# ignore this request and continue. The full help is then displayed later when the option_parser is completed
# afer we have requested the commandline argument descriptions from the RCP Server. (see below)
opts = parse_known_arguemnts(sys.argv)
# Load SSL/TLS CA certificate from file
if opts.ca_cert:
ssl_context = load_ca_cert("RCP Server CA", opts.ca_cert)
else:
ssl_context = None
# Initialize card reader
try:
sl = init_reader(opts)
sl.connect()
except Exception as e:
backtrace("Card reader initialization")
sys.exit(1)
# Run the RCP session
try:
rc = asyncio.run(run_rcp_session(opts, sl, ssl_context))
sys.exit(rc)
except SystemExit as rc:
sys.exit(rc)
except:
backtrace("RCP session")
sys.exit(1)

405
contrib/rcp/rcp_module_utils.py Normal file
View File

@@ -0,0 +1,405 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import abc
import os
import argparse
import logging
import threading
import asyncio
import websockets
from argparse import Namespace
from copy import deepcopy
from pathlib import Path
from typing import Optional
from osmocom.utils import Hexstr, is_hexstr
from pySim.utils import ResTuple
from pySim.transport import LinkBase
from pySim.commands import SimCardCommands
from pySim.log import PySimLogger
from rcp_utils import SrvSyncConnHdlr, CltConnHdlr, backtrace, pytype_to_type, load_server_cert, load_ca_cert
from rcp_utils import dict_from_key_value_pairs, load_json_schema, JsonValidator
from websockets.sync.server import serve, ServerConnection
from pySim.app import init_card
from pySim.runtime import RuntimeState
from pySim.cards import CardBase
from pySim.card_key_provider import CardKeyFieldCryptor
# Response timeout towards the RCP Server (includes RCP Client latency)
RCP_SERVER_TIMEOUT = 30 # sec.
log = PySimLogger.get(Path(__file__).stem)
class RcpsSimLink(LinkBase):
"""
pySim: Transport Link for RCPM (Remote Card Procedure Module)
This is a 'headless' transport link implementation that can only be used from an RCPM module. It merely serves as
an adapter between the pySim transport API and the RCPM command server connection handler.
"""
name = 'RCPM'
def __init__(self, conn_hdlr: SrvSyncConnHdlr, **kwargs):
self.conn_hdlr = conn_hdlr
self._atr = None
super().__init__(**kwargs)
def __str__(self) -> str:
return "rcpm:" + str(self.conn_hdlr)
def _send_apdu(self, apdu: Hexstr) -> ResTuple:
tx_json = {'rcps_instr': {'c_apdu' : apdu.upper()}}
rx_json = self.conn_hdlr._transact(tx_json)
data = rx_json['rcps_result']['r_apdu']['data']
sw = rx_json['rcps_result']['r_apdu']['sw']
return data, sw
def wait_for_card(self, timeout: Optional[int] = None, newcardonly: bool = False):
# In this setting, we do not have/cannot to wait for a card since we are not the entity that handles the
# direct connection to the card. When the procedure begins, we assume that the remote end already has set up
# a connection to the card and made it ready to perform operations on it.
pass
def connect(self):
# In this setting, we do not have/cannot to connect because we are not the entity that handles the direct
# connection to the card. The connection is established by the remote end.
pass
def get_atr(self) -> Hexstr:
return self._atr
def disconnect(self):
# In this setting, we do not have/cannot disconnect because we are not the enitity that handles the direct
# connection to the card. The disconnect is eventually done by the remote end when the procedure has finished.
pass
def _reset_card(self):
tx_json = {'rcps_instr': {'reset' : None}}
rx_json = self.conn_hdlr._transact(tx_json)
self._atr = rx_json['rcps_result']['atr']
return 1
class RcpsCltConnHdlr(CltConnHdlr):
"""
The RCP Server client handler is used to connect to the RCP Server when RCP Module is started. The connection is
kept alive until the RCP Module is terminated. This connection is used to exchange management data with the RCP
Server.
"""
def __init__(self, cmd_srv_addr: str, cmd_srv_port: int, module, *args, **kwargs):
self.cmd_srv_addr = cmd_srv_addr
self.cmd_srv_port = cmd_srv_port
self.module = module
super().__init__(*args, **kwargs)
async def describe(self):
"""
Send a detailed description about this RCP Module to the RCP Server. This is also the initial message that
the RCP Server expects when an RCP Module connects.
"""
# The rules (dict) in suitable_for (array of dict) may contain hexstrings. Here we go through those rules
# and convert those hexstrings to uppercase, since this is the standard we have set for the JSON messages.
suitable_for = []
for rule in self.module.suitable_for:
rule_filtered = {}
for k in rule:
if is_hexstr(rule[k]):
rule_filtered[k] = rule[k].upper()
else:
rule_filtered[k] = rule[k]
suitable_for.append(rule_filtered)
# Publish RCP Module description on the RCP server
tx_json = {'rcpm_hello':
{'name' : self.module.name,
'cmd_descr' : self.module.cmd_descr,
'suitable_for' : suitable_for,
'addr' : self.cmd_srv_addr,
'port' : self.cmd_srv_port
}
}
rx_json = await self._transact(tx_json)
if 'rcpm_welcome' not in rx_json:
raise ValueError("description not accepted by RCP Server")
class RcpModule(abc.ABC):
"""
Base class to implement to derive a concrete RCPM module class
"""
# Module name used to identify the module in logs and user output. This module name should be short and concise.
name = "RCPM"
# Command description of this module. The command description consists of a short and concise command name, a
# helpstring and an argument specification in the form of a python dict. This specificaton, consisting of
# 'name', 'help', and 'args' is is directly passed to agparse on the client side.
#
# In addition to that, the API user may specify which keys the RCP Server shall retrieve before a command is
# executed. This is done via the 'get_keys' field. This field is optional and has the form of a dict with
# two optional fields 'uicc' and 'euicc'. The value part of both fields is a list of strings which name the
# columns that are passed to the CardKeyProvider for lookup. When the 'uicc' field is set, then the RCP Server
# will automatically request the ICCID from the card and do the lookup. When the 'euicc' field is set, the RCP
# Server will do the same with the EID. It is possible to mix both fields to request keys for the eUICC and the
# currently activated eSIM profile at the same time. However, this may be a very rare cornercase.
#
# Example:
# cmd_descr = [{'name' : 'reset',
# 'help': 'reset the card',
# 'args' : []},
# {'name' : 'read_binary',
# 'help': 'read binary data from a transparent file.',
# 'args' : [{ 'name' : '--fid',
# 'spec' : {'required' : True,
# 'help' : 'File identifier to of the file to read',
# 'action' : 'append',
# 'pytype' : 'str'},
# }
# ]},
# {'name' : 'unlock_aram',
# 'help': 'unlock a locked ARA-M applet on a sysmoISIM-SJA5',
# 'args' : [],
# 'get_keys' : {'uicc' : ['KIC', 'KID', 'KIK']}}
# ]
cmd_descr = []
# Card properties to determine if this module is suitable for a specific card type or card types. The RCP Server
# will match those properties against user requests to determine which module provides useful services to the
# user's card.
#
# Example: [{"atr" : "3b9f96803f87828031e073fe211f574543753130136502"}]
suitable_for = []
# In addition the above, the derived class must implement command methods for each command that is defined in the
# command description (see above). Each command method must begin with the prefix "cmd_" followed by the command
# name used in the command description. A command method must have the form as shown in the example shown below.
# Each method should return an integer value which will become the final return code of the RCP client program.
#
# Args:
# hdlr: RcpModuleHdlr object, this object is provided by the RcpmCmdSrvConnHdlr object, which calls
# the command method of the module. Through the RcpModuleHdlr object, the API user gets access
# to special service methods (e.g. print) and other required properties (e.g. the SimCardCommands
# objects, key material and others (see RcpModuleHdlr).
#
# Example:
# def cmd_reset(self, hdlr: RcpModuleHdlr) -> int: ...
# def cmd_read_binary(self, hdlr: RcpModuleHdlr) -> int: ...
# def cmd_unlock_aram(self, hdlr: RcpModuleHdlr) -> int: ...
class RcpmCmdSrvConnHdlr(SrvSyncConnHdlr):
"""
The RCP Module command server connection handler is used to handle dedicated connections from the RCP Server. Those
dedicated connections are technically transparent connections between the RCP Client and the RCP Module (this). The
RCP Server merely acts as a proxy at that point.
"""
def __init__(self, module: RcpModule, field_cryptor: CardKeyFieldCryptor, *args, **kwargs):
SrvSyncConnHdlr.__init__(self, *args, *kwargs)
self.module = module
self.crypt = field_cryptor
def _parse_cmd_argv(self, cmd_suffix: str, cmd_argv: list[str]) -> Namespace:
""" Parse (and validate) the received argument vector """
# Use the cmd_descr of the module to create a (temporary) argument parser for the received argument vector.
cmd_parser = argparse.ArgumentParser()
for cmd in self.module.cmd_descr:
if cmd['name'] == cmd_suffix:
args = deepcopy(cmd['args'])
for arg in args:
arg['spec'] = pytype_to_type(arg['spec'])
cmd_parser.add_argument(arg['name'], **arg['spec'])
# Parse the arguments and return the parsed Namespace object.
try:
return cmd_parser.parse_args(cmd_argv)
except SystemExit:
raise ValueError("unable to parse arguments: %s", str(cmd_argv), )
def print(self, message: str):
""" Print a message on the client side """
log.info(str(self) + " -- %s" % message)
tx_json = {'rcps_instr': {'print' : message}}
rx_json = self._transact(tx_json)
if rx_json != {'rcps_result': {'empty' : None}}:
raise ValueError("unexpected response from RCP Client: %s", rx_json)
def procedure(self):
""" Receive and process a command from the RCP Client (via RCP Server) """
# Receive the command request.
rx_json = self._recv()
cmd = rx_json['rcps_command']['cmd']
cmd_argv = rx_json['rcps_command']['cmd_argv']
keys = rx_json['rcps_command'].get('keys')
log.info(str(self) + " -- executing command: %s %s", cmd, " ".join(cmd_argv))
try:
# Make sure the command actually addresses this module.
cmd_prefix = self.module.name + "_"
if not cmd.startswith(cmd_prefix):
raise ValueError("invalid command: %s" % cmd)
# Make sure the module actually provides a command method for the requested command.
cmd_suffix = cmd[len(cmd_prefix):]
cmd_method = "cmd_" + cmd_suffix
if not hasattr(self.module, cmd_method):
raise ValueError("missing command method: %s" % cmd_method)
# Parse and validate command arguments.
cmd_args = self._parse_cmd_argv(cmd_suffix, cmd_argv)
# Setup a pySim RuntimeState, CardBase and a RuntimeLchan.
rs, card = init_card(RcpsSimLink(self))
# Hand over control to the command method provided by the specific module implementation.
rcp_module_hdlr = RcpModuleHdlr(self.print, rs, card, cmd_args, keys, self.crypt)
rs.reset()
try:
rc = getattr(self.module, cmd_method)(rcp_module_hdlr)
except Exception as e:
backtrace("command method")
rc = 1 # general error
except Exception as e:
backtrace("command parsing")
rc = 126 # cannot execute
# The prodedure is done, send "goodbye" message.
log.info(str(self) + " -- command execution done, rc: %d" % rc)
tx_json = {'rcps_goodbye': rc}
self._send(tx_json)
class RcpModuleHdlr():
"""
RCP Module handler class. This class is used by the RcpmCmdSrvConnHdlr to create the handler RcpModuleHdlr object
(hdlr), which is is passed to the command method. The RcpModuleHdlr gives the API user access to resources he can
use carry out the command.
"""
# The RuntimeState (rs), the CardBase (card) and the RuntimeLchan (lchan) are the three major objects through which
# an API user may interact with the UICC/eUICC on the other remote end. Those objects have the same objectives as
# in pySim-shell.py, with lchan representing the currently selected lchan (set to self.rs.lchan[0] by default, API
# users may change the reference to a different lchan)
rs = None
card = None
lchan = None
# The cmd_args property contains the parsed command arguments which were passed by the end-user to the RCP Client.
# The arguments are already parsed and validated against the cmd_dscr property of the RcpModule. The arguments are
# in the form of a Namespace object and can be accessed like any argparse output. However, since the arguments
# contain user input, some caution is required.
cmd_args = None
# In case the retrieve_uicc_keys property of the RcpModule is used retrieve UICC key material, this property will
# contain the key material in the form of a dictionary. The format is similar to the return value of
# card_key_provider_get() (see also pySim.card_key_provider).
keys_uicc = {}
# Same as self.keys_uicc, but contains eUICC related key material in case requested using retrieve_uicc_keys.
keys_euicc = {}
def __init__(self, print: callable, rs: RuntimeState, card: CardBase, cmd_args: Namespace,
keys: dict, field_cryptor: CardKeyFieldCryptor):
self.print = print
self.rs = rs
self.card = card
self.lchan = self.rs.lchan[0]
self.cmd_args = cmd_args
if keys:
if 'uicc' in keys:
self.keys_uicc = dict_from_key_value_pairs(keys['uicc'], keylabel='key', valuelabel='value')
for key in self.keys_uicc.keys():
self.keys_uicc[key] = field_cryptor.decrypt_field(key, self.keys_uicc.get(key))
if 'euicc' in keys:
self.keys_euicc = dict_from_key_value_pairs(keys['euicc'], keylabel='key', valuelabel='value')
for key in self.keys_euicc.keys():
self.keys_euicc[key] = field_cryptor.decrypt_field(key, self.keys_euicc.get(key))
def rcpm_setup_argparse(description: str):
"""Create argument parser and add the basic arguments all RCP Modules should have"""
option_parser = argparse.ArgumentParser(description='RCP Module: ' + description,
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
option_parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False)
option_parser.add_argument("--uri", help="URI of the RCP-Server", required=True)
option_parser.add_argument("--rcps-ca-cert", help="SSL/TLS CA-Certificate of the RCP-Server", required=True)
option_parser.add_argument("--rcpm-cmd-server-addr", help="Local Host/IP to bind RCP-Module-Command-Server to",
required=True)
option_parser.add_argument("--rcpm-cmd-server-port", help="Local TCP port to bind RCP-Module-Command-Server to",
required=True, type=int)
option_parser.add_argument("--rcpm-cmd-server-cert", help="SSL/TLS Certificate of the RCP-Module-Command-Server",
required=True)
CardKeyFieldCryptor.argparse_add_args(option_parser)
return option_parser
def rcpm_run_module(opts: Namespace, module: RcpModule, *args, **kwargs):
PySimLogger.setup(print, {logging.WARN: "\033[33m", logging.DEBUG: "\033[90m"}, opts.verbose)
log.info("RCP Module startup: %s", module.name)
log.debug("Main process ID: %d", os.getpid())
# Load SSL/TLS certificates.
rcpm_cmd_ssl_context = load_server_cert("RCPM Command Server", opts.rcpm_cmd_server_cert)
ssl_context = load_ca_cert("RCPM Server Client", opts.rcps_ca_cert)
# Load JSON schema for message validation between RCP Server and RCP Module (this process)
rcpm_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcpm_to_rcps_schema.json"))
rcps_to_rcpm_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcps_to_rcpm_schema.json"))
# Load JSON schema for message validation between RCP Server and RCP Module Command Server (this process)
rcpmcs_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcpmcs_to_rcps_schema.json"))
rcps_to_rcpmcs_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(), "rcps_to_rcpmcs_schema.json"))
# Start local RCP Client Command Server.
log.info("RCPC command server at: %s:%d" % (opts.rcpm_cmd_server_addr, opts.rcpm_cmd_server_port))
def rcpm_cmd_conn_hdlr(websocket: ServerConnection):
json_validator = JsonValidator(rcps_to_rcpmcs_schema, rcpmcs_to_rcps_schema)
transport_keys = CardKeyFieldCryptor.transport_keys_from_opts(opts)
field_cryptor = CardKeyFieldCryptor(transport_keys)
hdlr = RcpmCmdSrvConnHdlr(module(*args, *kwargs), field_cryptor, websocket, RCP_SERVER_TIMEOUT, json_validator)
hdlr.procedure()
hdlr.close()
server = serve(rcpm_cmd_conn_hdlr, opts.rcpm_cmd_server_addr, opts.rcpm_cmd_server_port, ssl=rcpm_cmd_ssl_context)
def rcpm_cmd_server():
log.debug("RCPC command server thread ID: %d", threading.get_native_id())
server.serve_forever()
rcpm_cmd_server_thread = threading.Thread(target = rcpm_cmd_server)
rcpm_cmd_server_thread.start()
# Connect to RCP Server and publish module description.
async def rcps_client():
async with websockets.connect(opts.uri, ping_timeout=10.0, ping_interval=1.0, ssl=ssl_context) as websocket:
json_validator = JsonValidator(rcps_to_rcpm_schema, rcpm_to_rcps_schema)
client = RcpsCltConnHdlr(opts.rcpm_cmd_server_addr, opts.rcpm_cmd_server_port, module, websocket,
RCP_SERVER_TIMEOUT, json_validator)
await client.describe()
await client.wait_close()
try:
asyncio.run(rcps_client())
except Exception as e:
backtrace("RCPS client")
# Shutdown
server.shutdown()
rcpm_cmd_server_thread.join()
log.info("RCP Module shutdown: %s", module.name)

437
contrib/rcp/rcp_server.py Executable file
View File

@@ -0,0 +1,437 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import sys
import argparse
import asyncio
import logging
from osmocom.utils import Hexstr
from pySim.utils import ResTuple
from copy import deepcopy
from pathlib import Path
from pySim.log import PySimLogger
from pySim.utils import dec_iccid
import websockets
from websockets.asyncio.server import serve, ServerConnection
from rcp_utils import SrvConnHdlr, CltConnHdlr, JsonValidator
from rcp_utils import load_json_schema, backtrace, pytype_to_type, load_server_cert, load_ca_cert
from rcp_utils import key_value_pairs_from_dict
from pySim.card_key_provider import argparse_add_card_key_provider_args, init_card_key_provider
from pySim.card_key_provider import card_key_provider_get_field, card_key_provider_get
# TODO: Logging is fine, however it may also be a good idea to log some higher level events to some sort of journal.
# We could use OpenObserve for that.
CLIENT_TIMEOUT = 10
log = PySimLogger.get(Path(__file__).stem)
runtime_state = None
option_parser = argparse.ArgumentParser(description='RCP Server',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
option_parser.add_argument("--verbose", help="Enable verbose logging",
action='store_true', default=False)
option_parser.add_argument("--rcpc-server-addr", help="Local Host/IP to bind RCP-Client-Server to",
required=True)
option_parser.add_argument("--rcpc-server-port", help="Local TCP port to bind RCP-Client-Server to",
required=True, type=int)
option_parser.add_argument("--rcpc-server-cert", help="SSL/TLS Certificate of the RCP-Client-Server",
required=True)
option_parser.add_argument("--rcpm-server-addr", help="Local Host/IP to bind RCP-Module-Server to",
required=True)
option_parser.add_argument("--rcpm-server-port", help="Local TCP port to bind RCP-Module-Server to",
required=True, type=int)
option_parser.add_argument("--rcpm-server-cert", help="SSL/TLS Certificate of the RCP-Module-Server",
required=True)
option_parser.add_argument("--rcpm-module-ca-cert", help="SSL/TLS CA-Certificate of the RCP-Module-Command-Server",
required=True)
argparse_add_card_key_provider_args(option_parser)
class ModuleRuntimeState:
def __init__(self, websocket:ServerConnection, name:str, cmd_descr:list, suitable_for:list, addr:str, port:int):
self.name = name
self.websocket = websocket
# Run the cmd_descr through argparse to catch malformed argument specifications early
for cmd in cmd_descr:
args = deepcopy(cmd['args'])
cmd_parser = argparse.ArgumentParser()
for arg in args:
try:
arg['spec'] = pytype_to_type(arg['spec'])
cmd_parser.add_argument(arg['name'], **arg['spec'])
except:
raise ValueError("invalid argument spec %s -- check RCP Module" % str(arg))
self.cmd_descr = cmd_descr
self.suitable_for = suitable_for
self.addr = addr
self.port = port
log.debug("new RCP module context created: '%s'", name)
def is_suitable(self, suitable_for:dict) -> bool:
"""Check if this module is 'suitable_for' a specific card"""
if suitable_for in self.suitable_for:
return True
return False
def describe(self) -> dict:
"""Describe this module towards the RCP Client"""
# The command description sent by the RCP Module also includes fields that are intended to be seen
# only by the RCP Server. Here we set up the command description as it is expected by the RCP Client.
cmd_descr = []
for descr in self.cmd_descr:
cmd_descr.append({'name' : descr['name'],
'help' : descr['help'],
'args' : descr['args']})
# Return module description
return {'name': self.name,
'cmd_descr': cmd_descr}
def get_cmd_descr(self, cmd: str) -> dict:
"""Get the description for a specific command of this module"""
for descr in self.cmd_descr:
if self.name + "_" + descr['name'] == cmd:
return descr
raise ValueError("command %s not found in command description %s" % (cmd_name, str(self.cmd_descr)))
def __str__(self) -> str:
return self.name
def __del__(self):
log.debug("RCP module context destroyed: '%s'", self.name)
class RuntimeState:
def __init__(self, rcpm_ca_ssl_context):
self.module_runtime_states = []
self.rcpm_ca_ssl_context = rcpm_ca_ssl_context
# Load JSON schema for message validation between RCP Client and RCP Server (this process)
self.rcpc_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcpc_to_rcps_schema.json"))
self.rcps_to_rcpc_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcps_to_rcpc_schema.json"))
# Load JSON schema for message validation between RCP Module and RCP Server (this process)
self.rcpm_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcpm_to_rcps_schema.json"))
self.rcps_to_rcpm_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcps_to_rcpm_schema.json"))
# Load JSON schema for message validation between RCP Module Command Server and RCP Server (this process)
self.rcpmcs_to_rcps_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcpmcs_to_rcps_schema.json"))
self.rcps_to_rcpmcs_schema = load_json_schema(os.path.join(Path(__file__).parent.resolve(),
"rcps_to_rcpmcs_schema.json"))
log.debug("new runtime context created.")
def __log_modules_available(self) -> str:
if self.module_runtime_states:
modules_str = ""
for module in self.module_runtime_states:
modules_str += "'" + str(module) + "', "
return "RCP modules available: %s" % modules_str[:-2]
else:
return "RCP modules available: none"
def module_add(self, module: ModuleRuntimeState):
self.module_runtime_states.append(module)
log.info("new RCP module, %s", self.__log_modules_available())
def module_remove(self, websocket:ServerConnection):
for module in self.module_runtime_states:
if module.websocket == websocket:
self.module_runtime_states.remove(module)
log.info("RCP module removed, %s", self.__log_modules_available())
return
log.warning("cannot remove RCP module, no RCP module associated with RCPC connection: %s:%d, %s" %
(*websocket.remote_address, self.__log_modules_available()))
def modules_find(self, suitable_for:dict) -> list[dict]:
modules = []
for module in self.module_runtime_states:
if module.is_suitable(suitable_for):
modules.append(module.describe())
if modules:
return modules
# It is absolutely tolerable if no suitable RCP module can be found. If this is the case, the client should
# display an empty help screen and exit normally.
log.warning("no suitable RCP module found, %s", self.__log_modules_available())
return []
def module_find(self, suitable_for:dict, cmd:str) -> ModuleRuntimeState:
modules = self.modules_find(suitable_for)
for m in modules:
module_name = m['name']
cmd_descr = m['cmd_descr']
for c in cmd_descr:
cmd_name = c['name']
if module_name + "_" + cmd_name == cmd:
break
for module_runtime_state in self.module_runtime_states:
if module_runtime_state.name == module_name:
return module_runtime_state
# Normally we should find the RCP module. When this method is called, we have already called modules_find
# before because we had to return the command descriptions to the client. If we cannot find the RCP module
# now, the module have been disconnected or the client somehow called a command that does not exist. In any
# case, ending up here means we cannot continue.
raise ValueError("RCP module not found for command: %s, " % (cmd, self.__log_modules_available()))
class RcpmCltConnHdlr(CltConnHdlr):
"""
The RCP Module client connection handler is the dedicated client that is used by the RCP Client connection handler
to handle the dedicated connection towards the RCP Module (see below)
"""
class RcpcSrvConnHdlr(SrvConnHdlr):
"""
The RCP Client connection handler takes care of the handling of client requests. Througout the lifetime of a
connection, the client will request a description of the available commands and then request the execution of a
procedure. To execute the procedure, the handler will make a dedicated connection to the RCP Module and then
transparently pass the messages from the RCP Client to the RCP Module and vice versa.
"""
module_client = None
async def describe(self):
"""
Collect the command/argument description of suitable modules and forward that definition to the RCP client. The
RCP client will then build an argument parser (commmandlien help, argument validation) from this information.
"""
rx_json = await self._recv()
self.suitable_for = rx_json['rcpc_hello']['suitable_for']
modules = runtime_state.modules_find(self.suitable_for)
tx_json = {'rcpc_welcome':
{'module_descr' : modules}
}
await self._send(tx_json)
async def _transact_apdu(self, apdu: Hexstr) -> ResTuple:
"""Private low level method to exchange an APDU"""
tx_json = {'rcpc_instr': {'c_apdu' : apdu.upper()}}
rx_json = await self._transact(tx_json)
if rx_json is None:
raise ValueError("RCP Client vanished unexpectetly")
data = rx_json['rcpc_result']['r_apdu']['data']
sw = rx_json['rcpc_result']['r_apdu']['sw']
return data, sw
async def _reset(self) -> Hexstr:
"""Private low level method to reset the UICC/eUICC"""
tx_json = {'rcpc_instr': {'reset' : None}}
rx_json = await self._transact(tx_json)
if rx_json is None:
raise ValueError("RCP Client vanished unexpectetly")
return rx_json['rcpc_result']['atr']
async def _read_iccid(self) -> Hexstr:
"""Private low level method to read the EID from an UICC (or eSIM)"""
data, sw = await self._transact_apdu("00A40000022FE200")
if sw != "9000":
raise ValueError("Unable to select EF.ICCID, sw: %s, " % sw)
data, sw = await self._transact_apdu("00B000000A")
if sw != "9000":
raise ValueError("Unable to read EF.ICCID, sw: %s, " % sw)
return dec_iccid(data)
async def _read_eid(self) -> Hexstr:
"""Private low level method to read the EID from an eUICC"""
data, sw = await self._transact_apdu("00A4040410A0000005591010FFFFFFFF890000010000")
if sw != "9000":
raise ValueError("Unable to select ISD-R, sw: %s, " % sw)
data, sw = await self._transact_apdu("80E2910006BF3E035C015A00")
if sw != "9000":
raise ValueError("Unable to retrieve EID, sw: %s, " % sw)
return data[10:]
async def print(self, message: str):
""" Print a message on the client side """
tx_json = {'rcpc_instr': {'print' : message}}
rx_json = await self._transact(tx_json)
if rx_json is None:
raise ValueError("RCP Client vanished unexpectetly")
if rx_json != {'rcpc_result': {'empty' : None}}:
raise ValueError("unexpected response from RCP Client: %s" % rx_json)
async def procedure(self):
"""
Receive a command from the client, pick a matching module, make a decdicated connection to that module and
forward instruction/response messages between RCP Client and RCP Module until the procedure is done.
"""
# Expect a command from the client
rx_json = await self._recv()
if rx_json is None:
log.debug(str(self) + " -- RCP client has closed the connection, no procedure executed")
return
command = rx_json['rcpc_command']
# Pick the matching RCP Module
module = runtime_state.module_find(self.suitable_for, command['cmd'])
# Retrieve keys (if the command requires them)
cmd_descr = module.get_cmd_descr(command['cmd'])
get_keys = cmd_descr.get('get_keys')
if get_keys:
keys = {}
get_keys_uicc = get_keys.get('uicc')
if get_keys_uicc:
iccid = await self._read_iccid()
keys_uicc = card_key_provider_get(get_keys_uicc, 'ICCID', iccid)
keys['uicc'] = key_value_pairs_from_dict(keys_uicc, keylabel='key', valuelabel='value')
get_keys_euicc = get_keys.get('euicc')
if get_keys_euicc:
eid = await self._read_eid()
keys_euicc = card_key_provider_get(get_keys_euicc, 'EID', eid)
keys['euicc'] = key_value_pairs_from_dict(keys_euicc, keylabel='key', valuelabel='value')
command['keys'] = keys
# Resetting card to ensure the card is in a defined state
await self._reset()
# Create a dedicated connection to the RCP Module and proxy the messages between RCP Client and RCP Module.
module_uri = "wss://%s:%d" % (module.addr, module.port)
log.info(str(self) + " -- executing procedure for command \"%s\" on module \"%s\" at: %s" %
(command['cmd'], module.name, module_uri))
async with websockets.connect(module_uri, ssl=runtime_state.rcpm_ca_ssl_context) as websocket:
# Create a connection to the RCP Module Command Server
json_validator = JsonValidator(runtime_state.rcpmcs_to_rcps_schema, runtime_state.rcps_to_rcpmcs_schema)
self.module_client = RcpmCltConnHdlr(websocket, CLIENT_TIMEOUT, json_validator)
# Prepare initial request to be send to the RCP Module Command Server
module_tx_json = {'rcps_command' : command}
while(True):
# Send request to the RCP Module Command Server
module_rx_json = await self.module_client._transact(module_tx_json)
# Forward the response to the RCP Client
if 'rcps_instr' in module_rx_json:
client_tx_json = {'rcpc_instr' : module_rx_json['rcps_instr']}
await self._send(client_tx_json)
elif 'rcps_goodbye' in module_rx_json:
log.info(str(self) + " -- command execution done, rc: %d" % module_rx_json['rcps_goodbye'])
client_tx_json = {'rcpc_goodbye' : module_rx_json['rcps_goodbye']}
await self._send(client_tx_json)
break
else:
raise ValueError("Unexpected response from RCP Module: %s" % str(module_rx_json))
# Receive the Result from the client, prepare request (module_tx_json) for the next turn
client_rx_json = await self._recv()
if client_rx_json is None:
raise ValueError("RCP client vanished unexpectetly")
if 'rcpc_result' in client_rx_json:
module_tx_json = {'rcps_result' : client_rx_json['rcpc_result']}
else:
raise ValueError("Unexpected result from RCP Client: %s" % str(client_rx_json))
async def close(self):
if self.module_client:
await self.module_client.close()
await super().close()
class RcpmSrvConnHdlr(SrvConnHdlr):
"""
The RCP Module connection handler is responsible to handle connect and disconnect events of RCP Modules. This
connection between the RCP Module and the RCP Server is used for management purposes only.
"""
async def describe(self):
"""
Receive the module description from an RCP Module. This description will be stored in an internal list until
the module is disconnected from the server.
"""
rx_json = await self._recv()
runtime_state.module_add(module = ModuleRuntimeState(self.websocket, **rx_json['rcpm_hello']))
tx_json = {'rcpm_welcome': None}
await self._send(tx_json)
def __del__(self):
"""
Remove RCPM from internal list when the connection is closed (and the handler is deleted)
"""
runtime_state.module_remove(self.websocket)
super().__del__()
async def rcpc_conn_hdlr(websocket: ServerConnection):
# TODO: Implement some sort of rate limit to protect against DoS. We may count the requests for each requesting
# IP address and reject the connection once a certain threshold is reached. (we plan to use the CardKeyProvider
# together with a database)
try:
json_validator = JsonValidator(runtime_state.rcpc_to_rcps_schema, runtime_state.rcps_to_rcpc_schema)
hdlr = RcpcSrvConnHdlr(websocket, CLIENT_TIMEOUT, json_validator)
except:
backtrace("RCPC connection handler (create)")
# If the handler creation fails, it makes no sense to continue
return
try:
await hdlr.describe()
await hdlr.procedure()
except:
backtrace("RCPC connection handler (client interaction)")
try:
await hdlr.close()
except:
backtrace("RCPC connection handler close")
async def rcpm_conn_hdlr(websocket: ServerConnection):
try:
json_validator = JsonValidator(runtime_state.rcpm_to_rcps_schema, runtime_state.rcps_to_rcpm_schema)
hdlr = RcpmSrvConnHdlr(websocket, CLIENT_TIMEOUT, json_validator)
await hdlr.describe()
await hdlr.close()
except:
backtrace("RCPM connection handler")
if __name__ == '__main__':
opts = option_parser.parse_args()
PySimLogger.setup(print, {logging.WARN: "\033[33m", logging.DEBUG: "\033[90m"}, opts.verbose)
# Load SSL/TLS certificates
rcpc_ssl_context = load_server_cert("RCP Client Server", opts.rcpc_server_cert)
rcpm_ssl_context = load_server_cert("RCP Module Server", opts.rcpm_server_cert)
rcpm_ca_ssl_context = load_ca_cert("RCP Module Command Server Client", opts.rcpm_module_ca_cert)
# Init card key provider for automatic card key retrieval
init_card_key_provider(opts)
# Start RCP server
runtime_state = RuntimeState(rcpm_ca_ssl_context)
async def rcp_server():
log.info("RCP Client Server at: %s:%d" % (opts.rcpc_server_addr, opts.rcpc_server_port))
log.info("RCP Module server at: %s:%d" % (opts.rcpm_server_addr, opts.rcpm_server_port))
async with serve(rcpc_conn_hdlr, opts.rcpc_server_addr, opts.rcpc_server_port, ssl=rcpc_ssl_context), \
serve(rcpm_conn_hdlr, opts.rcpm_server_addr, opts.rcpm_server_port, ssl=rcpm_ssl_context):
await asyncio.get_running_loop().create_future()
try:
asyncio.run(rcp_server())
except SystemExit:
pass
except:
backtrace("RCP Server")
sys.exit(1)

254
contrib/rcp/rcp_utils.py Normal file
View File

@@ -0,0 +1,254 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import sys
import ssl
import json
import abc
import asyncio
import websockets
import traceback
import threading
from copy import deepcopy
from websockets.asyncio.server import ServerConnection
from websockets.asyncio.client import ClientConnection
from pathlib import Path
from jsonschema import validate
from pySim.log import PySimLogger
from ssl import SSLContext
log = PySimLogger.get(Path(__file__).stem)
# TODO: Might be helpful for others as well, move this to pySim.utils?
def backtrace(what: str):
log.error("%s failed with an exception:", what)
log.error("---------------------8<---------------------")
traceback_lines = traceback.format_exc()
for line in traceback_lines.split("\n"):
if line:
log.error(line)
log.error("---------------------8<---------------------")
# TODO: Might be helpful for others as well, move this to pySim.utils?
def key_value_pairs_from_dict(keys: dict, keylabel: str='key', valuelabel: str='value') -> list:
key_list = []
for key in keys:
key_list.append({keylabel : key, valuelabel : keys[key]})
return key_list
# TODO: Might be helpful for others as well, move this to pySim.utils?
def dict_from_key_value_pairs(keys: list, keylabel: str='key', valuelabel: str='value') -> dict:
key_dict = {}
for key in keys:
key_dict[key[keylabel]] = key[valuelabel]
return key_dict
def pytype_to_type(dict_in: dict) -> dict:
"""
There is no way to properly express python types in JSON. This function can be used to replace
each ocurrence of "pytype", with "type", where the string type name is replaced with an actual
python type.
"""
dict_out = deepcopy(dict_in)
if dict_out.get('pytype'):
if dict_out['pytype'] == "str":
dict_out.pop('pytype')
dict_out['type'] = str
elif dict_out['pytype'] == "int":
dict_out.pop('pytype')
dict_out['type'] = int
else:
raise ValueError("invalid type in command argument specification: %s" % arg['spec']['type'])
return dict_out
def load_json_schema(filename: str) -> dict:
"""Load a JSON schema from file"""
log.info("loading JSON schema: %s", filename)
try:
with open(filename) as schema_file:
return json.load(schema_file)
except Exception as e:
backtrace("JSON schema load")
sys.exit(1)
def load_server_cert(what: str, filename: str) -> SSLContext:
"""Load an SSL/TLS server certificate"""
log.info("loading SSL/TLS server certificate (%s): %s", what, filename)
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain(filename)
return ssl_context
def load_ca_cert(what: str, filename: str) -> SSLContext:
"""Load an SSL/TLS CA certificate"""
log.info("loading SSL/TLS CA certificate (%s): %s", what, filename)
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ssl_context.load_verify_locations(filename)
return ssl_context
class JsonValidator():
"""
JSON validator class, can be passed to any ConnHdlr object to automatically validate the JSON messages which are
sent and and received.
"""
def __init__(self, rx_schema: dict, tx_schema: dict = None):
self.rx_schema = rx_schema
if tx_schema:
self.tx_schema = tx_schema
else:
self.tx_schema = None
def valid_rx_json(self, rx_json: dict):
validate(instance = rx_json, schema = self.rx_schema)
def valid_tx_json(self, tx_json: dict):
if self.tx_schema:
# We intentionally do not prevent the sending of an invalid JSON message. It is the responsibility of the
# receiving end to detect an invalid message and react accordingly. The purpose of this validation is to
# make developers/users aware of the problem.
try:
validate(instance = tx_json, schema = self.tx_schema)
except Exception as e:
backtrace("JSON schema validation (TX)")
class ConnHdlr(abc.ABC):
"""Base class that can be used to create a connection handler"""
def __init__(self, websocket: ServerConnection | ClientConnection, timeout: int,
json_validator: JsonValidator = None):
self.websocket = websocket
self.local_address = websocket.local_address
self.remote_address = websocket.remote_address
self.timeout = timeout
self.json_validator = json_validator
log.debug(str(self) + " -- new handler, timeout: %d sec.", self.timeout)
def _log_recv_peer(self, rx_json_str: str):
peer = "%s:%d<-%s:%d" % (self.local_address[0],
self.local_address[1],
self.remote_address[0],
self.remote_address[1])
log.debug(str(self) + " -- RX(%s): %s", peer, rx_json_str)
def _log_send_peer(self, tx_json_str: str):
peer = "%s:%d->%s:%d" % (self.local_address[0],
self.local_address[1],
self.remote_address[0],
self.remote_address[1])
log.debug(str(self) + " -- TX(%s): %s", peer, tx_json_str)
def __str__(self) -> str:
return "%s(%d)" % (type(self).__name__, id(self))
def __del__(self):
log.debug(str(self) + " -- closed handler")
class SrvConnHdlr(ConnHdlr):
"""Base class that can be used to create a connection handler for a server"""
async def _recv(self) -> dict:
"""Receive JSON message from client"""
async with asyncio.timeout(self.timeout):
try:
rx_json_str = await self.websocket.recv()
except websockets.exceptions.ConnectionClosedOK:
log.debug(str(self) + " -- no data received, connection is closed")
return None
self._log_recv_peer(rx_json_str)
rx_json = json.loads(rx_json_str)
if self.json_validator:
self.json_validator.valid_rx_json(rx_json)
return rx_json
async def _send(self, tx_json: dict):
"""Send JSON message to client"""
if self.json_validator:
self.json_validator.valid_tx_json(tx_json)
tx_json_str = json.dumps(tx_json)
self._log_send_peer(tx_json_str)
await self.websocket.send(tx_json_str)
async def _transact(self, tx_json: dict) -> dict:
"""Exchange JSON message with client"""
await self._send(tx_json)
return await self._recv()
async def close(self):
"""Wait for a connecion to close normally"""
await self.websocket.wait_closed()
log.debug(str(self) + " -- closed connection")
class SrvSyncConnHdlr(ConnHdlr):
"""Base class that can be used to create a synchronous connection handler for a server"""
def _recv(self) -> dict:
"""Receive JSON message from client"""
rx_json_str = self.websocket.recv(self.timeout)
self._log_recv_peer(rx_json_str)
rx_json = json.loads(rx_json_str)
if self.json_validator:
self.json_validator.valid_rx_json(rx_json)
return rx_json
def _send(self, tx_json: dict):
"""Send JSON message to client"""
if self.json_validator:
self.json_validator.valid_tx_json(tx_json)
tx_json_str = json.dumps(tx_json)
self._log_send_peer(tx_json_str)
self.websocket.send(tx_json_str)
def _transact(self, tx_json: dict) -> dict:
"""Exchange JSON message with client"""
self._send(tx_json)
return self._recv()
def close(self):
"""Close connection normally"""
self.websocket.close()
log.debug(str(self) + " -- closed connection")
class CltConnHdlr(ConnHdlr):
"""Base class that can be used to create a connection handler for a client"""
async def _transact(self, tx_json: dict) -> dict:
"""Exchange JSON message with server"""
if self.json_validator:
self.json_validator.valid_tx_json(tx_json)
tx_json_str = json.dumps(tx_json)
self._log_send_peer(tx_json_str)
async with asyncio.timeout(self.timeout):
await self.websocket.send(tx_json_str)
rx_json_str = await self.websocket.recv()
self._log_recv_peer(rx_json_str)
rx_json = json.loads(rx_json_str);
if self.json_validator:
self.json_validator.valid_rx_json(rx_json)
return rx_json
async def close(self):
"""Close connection normally"""
await self.websocket.close()
log.debug(str(self) + " -- closed connection")
async def wait_close(self):
"""Wait for a connecion to close normally"""
await self.websocket.wait_closed()
log.debug(str(self) + " -- closed connection")

85
contrib/rcp/rcpc_to_rcps_schema.json Normal file
View File

@@ -0,0 +1,85 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Client to RCP Server",
"type": "object",
"properties": {
"rcpc_hello": {
"type": "object",
"properties": {
"suitable_for": {
"type": "object",
"properties": {
"atr": {
"type": "string",
"pattern": "^[0-9,A-F]{0,66}$"
}
},
"oneOf": [
{ "required": [ "atr" ] }
],
"additionalProperties": false
}
},
"required": [ "suitable_for" ],
"additionalProperties": false
},
"rcpc_command": {
"type": "object",
"properties": {
"cmd": {
"type": "string",
"pattern": "^[0-9,A-Z,a-z,_]{0,40}$"
},
"cmd_argv": {
"type": "array",
"items": {
"type": "string",
"pattern": "^.{0,512}$"
},
"maxItems": 255
}
},
"required": [ "cmd", "cmd_argv" ],
"additionalProperties": false
},
"rcpc_result": {
"type": "object",
"properties": {
"r_apdu": {
"type": "object",
"properties": {
"data": {
"type": "string",
"pattern": "^[0-9,A-F]{0,512}$"
},
"sw": {
"type": "string",
"pattern": "^[0-9,A-F]{0,4}$"
}
},
"required": [ "data", "sw" ],
"additionalProperties": false
},
"atr": {
"type": "string",
"pattern": "^[0-9,A-F]{0,66}$"
},
"empty": {
"type": "null"
}
},
"oneOf": [
{ "required": [ "r_apdu" ] },
{ "required": [ "atr" ] },
{ "required": [ "empty" ] }
],
"additionalProperties": false
}
},
"oneOf": [
{ "required": [ "rcpc_hello" ] },
{ "required": [ "rcpc_command" ] },
{ "required": [ "rcpc_result" ] }
],
"additionalProperties": false
}

116
contrib/rcp/rcpm_to_rcps_schema.json Normal file
View File

@@ -0,0 +1,116 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Module to RCP Server",
"type": "object",
"properties": {
"rcpm_hello": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"cmd_descr": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"help": {
"type": "string"
},
"args": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"spec": {
"type": "object",
"properties": {
"required" : {
"type": "boolean"
},
"help": {
"type": "string"
},
"action": {
"type": "string"
},
"pytype": {
"type": "string"
},
"default" : {
"type": ["string", "integer"]
}
},
"required": [ "help" ],
"additionalProperties": false
}
},
"required": [ "name", "spec" ],
"additionalProperties": false
}
},
"get_keys": {
"type": "object",
"properties": {
"uicc" : {
"type": "array",
"items": {
"type": "string"
}
},
"euicc" : {
"type": "array",
"items": {
"type": "string"
}
}
},
"oneOf": [
{ "required": [ "uicc" ] },
{ "required": [ "euicc" ] }
],
"additionalProperties": false
}
},
"required": [ "name", "help", "args" ],
"additionalProperties": false
}
},
"suitable_for": {
"type": "array",
"items": {
"type": "object",
"properties": {
"atr": {
"type": "string",
"pattern": "^[0-9,A-F]{0,66}$"
}
},
"oneOf": [
{ "required": [ "atr" ] }
],
"additionalProperties": false
}
},
"addr": {
"type": "string"
},
"port": {
"type": "integer"
}
},
"required": [ "name", "cmd_descr", "suitable_for", "addr", "port" ],
"additionalProperties": false
}
},
"oneOf": [
{ "required": [ "rcpm_hello" ] }
],
"additionalProperties": false
}

36
contrib/rcp/rcpmcs_to_rcps_schema.json Normal file
View File

@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Module Command Server to RCP Server",
"type": "object",
"properties": {
"rcps_instr": {
"type": "object",
"properties": {
"print": {
"type": "string"
},
"reset": {
"type": "null"
},
"c_apdu": {
"type": "string",
"pattern": "^[0-9,A-F]{0,512}$"
}
},
"oneOf": [
{ "required": [ "print" ] },
{ "required": [ "reset" ] },
{ "required": [ "c_apdu" ] }
],
"additionalProperties": false
},
"rcps_goodbye": {
"type": "integer"
}
},
"oneOf": [
{ "required": [ "rcps_instr" ] },
{ "required": [ "rcps_goodbye" ] }
],
"additionalProperties": false
}

108
contrib/rcp/rcps_to_rcpc_schema.json Normal file
View File

@@ -0,0 +1,108 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Server to RCP Client",
"type": "object",
"properties": {
"rcpc_welcome": {
"type": "object",
"properties": {
"module_descr": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"cmd_descr": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"help": {
"type": "string"
},
"args": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"spec": {
"type": "object",
"properties": {
"required" : {
"type": "boolean"
},
"help": {
"type": "string"
},
"action": {
"type": "string"
},
"pytype": {
"type": "string"
},
"default" : {
"type": ["string", "integer"]
}
},
"required": [ "help" ],
"additionalProperties": false
}
},
"required": [ "name", "spec" ],
"additionalProperties": false
}
}
},
"required": [ "name", "help", "args" ],
"additionalProperties": false
}
}
},
"required": [ "name", "cmd_descr" ],
"additionalProperties": false
}
}
},
"required": [ "module_descr" ],
"additionalProperties": false
},
"rcpc_instr": {
"type": "object",
"properties": {
"print": {
"type": "string"
},
"reset": {
"type": "null"
},
"c_apdu": {
"type": "string",
"pattern": "^[0-9,A-F]{0,512}$"
}
},
"oneOf": [
{ "required": [ "print" ] },
{ "required": [ "reset" ] },
{ "required": [ "c_apdu" ] }
],
"additionalProperties": false
},
"rcpc_goodbye": {
"type": "integer"
}
},
"oneOf": [
{ "required": [ "rcpc_welcome" ] },
{ "required": [ "rcpc_instr" ] },
{ "required": [ "rcpc_goodbye" ] }
],
"additionalProperties": false
}

14
contrib/rcp/rcps_to_rcpm_schema.json Normal file
View File

@@ -0,0 +1,14 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Server to RCP Module",
"type": "object",
"properties": {
"rcpm_welcome": {
"type": "null"
}
},
"oneOf": [
{ "required": [ "rcpm_welcome" ] }
],
"additionalProperties": false
}

106
contrib/rcp/rcps_to_rcpmcs_schema.json Normal file
View File

@@ -0,0 +1,106 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RCP Server to RCP Module Command Server",
"type": "object",
"properties": {
"rcps_command": {
"type": "object",
"properties": {
"cmd": {
"type": "string",
"pattern": "^[0-9,A-Z,a-z,_]{0,40}$"
},
"cmd_argv": {
"type": "array",
"items": {
"type": "string",
"pattern": "^.{0,512}$"
},
"maxItems": 255
},
"keys" : {
"type": "object",
"properties": {
"uicc" : {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
},
"required": [ "key", "value" ],
"additionalProperties": false
}
},
"euicc" : {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
},
"required": [ "key", "value" ],
"additionalProperties": false
}
}
},
"oneOf": [
{ "required": [ "uicc" ] },
{ "required": [ "euicc" ] }
],
"additionalProperties": false
}
},
"required": [ "cmd", "cmd_argv" ],
"additionalProperties": false
},
"rcps_result": {
"type": "object",
"properties": {
"r_apdu": {
"type": "object",
"properties": {
"data": {
"type": "string",
"pattern": "^[0-9,A-F]{0,512}$"
},
"sw": {
"type": "string",
"pattern": "^[0-9,A-F]{0,4}$"
}
},
"required": [ "data", "sw" ],
"additionalProperties": false
},
"atr": {
"type": "string",
"pattern": "^[0-9,A-F]{0,66}$"
},
"empty": {
"type": "null"
}
},
"oneOf": [
{ "required": [ "r_apdu" ] },
{ "required": [ "atr" ] },
{ "required": [ "empty" ] }
],
"additionalProperties": false
}
},
"oneOf": [
{ "required": [ "rcps_command" ] },
{ "required": [ "rcps_result" ] }
],
"additionalProperties": false
}

2
contrib/rcp/usage_example/card_data.csv Normal file
View File

@@ -0,0 +1,2 @@
iccid,kic,kid,kik
8949440000001155306,F09C43EE1A0391665CC9F05AF4E0BD10,01981F4A20999F62AF99988007BAF6CA,8F8AEE5CDCC5D361368BC45673D99195
1 iccid kic kid kik
2 8949440000001155306 F09C43EE1A0391665CC9F05AF4E0BD10 01981F4A20999F62AF99988007BAF6CA 8F8AEE5CDCC5D361368BC45673D99195

2
contrib/rcp/usage_example/card_data.csv.encr Normal file
View File

@@ -0,0 +1,2 @@
"ICCID","KIC","KID","KIK"
"8949440000001155306","eae46224fa0a4ac1c12cba9d102f1188","3f14b978ddb38c08d832d4e4c2e0639d","9e19db4a5ed5cb8c4f5d96283eab273a"

20
contrib/rcp/usage_example/certs/example_ssl_rcp_ca_cert.crt Normal file
View File

@@ -0,0 +1,20 @@
-----BEGIN CERTIFICATE-----
MIIDSzCCAjOgAwIBAgIUEv1f0yjVtkr+RNYLItZ33eTJwHMwDQYJKoZIhvcNAQEL
BQAwFjEUMBIGA1UEAwwLRWFzeS1SU0EgQ0EwHhcNMjYwNDI5MTIwOTM0WhcNMzYw
NDI2MTIwOTM0WjAWMRQwEgYDVQQDDAtFYXN5LVJTQSBDQTCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBANXdkSyQlDzuo2cJmnBmFiZpc0V9tYBcNkpZd3Ac
R0WljazKKgXDWNmOcSO7891bi+1HZzz+nDfV0mJY776ScGkTqF43Hzpg9eZakMAx
yC24mT4h+uyRcPWZrBwaQhpiQrvZy4MRyuUB+BEgBSmhoDiuXP44kWiuEJHuzpOq
X6Q2dW8RIeQPDGGK6XPZIQLqx+krxkaqphd/vHgT1/yd7Ol5xxMc4x2UuPaVCj0D
OzslFsbb0Zu77ffCtHOVVnzSCzeEGGx1MPQm6hDVW+KUXXTwke1K55fmFZhu0gKO
HYSEjgPj6X8muDb+GvOAQX3fHmS6KvFS4fwWd2InZ3v2f3cCAwEAAaOBkDCBjTAM
BgNVHRMEBTADAQH/MB0GA1UdDgQWBBS6zY4Dd0pJFrvWLmyjn0vDTFqVqzBRBgNV
HSMESjBIgBS6zY4Dd0pJFrvWLmyjn0vDTFqVq6EapBgwFjEUMBIGA1UEAwwLRWFz
eS1SU0EgQ0GCFBL9X9Mo1bZK/kTWCyLWd93kycBzMAsGA1UdDwQEAwIBBjANBgkq
hkiG9w0BAQsFAAOCAQEAGJUXlbnVhh+xL+pyTyjwtd8nxhUcHzYZl+OT0bkGY9zT
S3NjHkKBbdnEftuYDYqp0uBuGFQ1WIOKiM3rp4IePKe84lSivZMVh9ObtNalcEQr
sqxBziNOMJM2mh5V2NdxiK2E1gCZ959wOQ8yzM6gGC+wW8w4zwULhv4JimQDjk+G
kAdiGL7+WAxrNWUulvm8khFt2nOlucJg4IAYVt2SI1AFMt/YSXoA4wMwM9QcHGj0
1A069IxX93WVhUpIL1Avwz+KJK0BPY6SM8LYUy6V50Hojp76BB7VD6SxQrSoceUo
6cRNDtCmofOlltfeUJLr1mI4S2tM50bQVsHD92EJBA==
-----END CERTIFICATE-----

115
contrib/rcp/usage_example/certs/example_ssl_rcpc_rcps_cert.pem Normal file
View File

@@ -0,0 +1,115 @@
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
42:38:a5:6f:70:53:40:e4:a4:1a:2c:0f:fc:81:13:42
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=Easy-RSA CA
Validity
Not Before: Apr 29 12:09:35 2026 GMT
Not After : Aug 1 12:09:35 2028 GMT
Subject: CN=example_ssl_rcpc_rcps_cert
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ae:0f:e1:ee:fc:f6:db:75:45:c0:f4:49:72:46:
3d:e3:db:0c:c4:34:d2:9e:49:d4:86:4f:19:0d:55:
70:50:81:e4:e6:64:56:a8:58:e8:e6:54:0a:16:bc:
f4:4b:84:cd:1d:b9:2e:ed:62:b6:cd:62:35:8b:81:
18:ab:ff:63:f5:c1:dc:16:3e:a8:dc:ac:11:dd:43:
12:f8:ef:f2:f1:af:84:fd:83:fe:a8:d3:46:7d:77:
e6:ae:95:61:a6:c9:99:6b:40:61:8d:6e:7e:66:1e:
97:77:b0:e8:b7:3d:3a:d5:d7:d3:ee:66:95:62:83:
14:cc:5e:32:ff:9e:bd:f1:06:e6:8d:6a:7c:0a:27:
22:19:b9:06:09:cf:ef:c7:dc:e8:8f:04:4b:83:0d:
cc:8d:b1:c2:cf:ab:40:25:6e:f2:bf:b7:c6:1d:8f:
d2:fc:3d:c8:a1:be:4a:09:b9:91:e3:76:4f:c7:9b:
fc:2f:de:d9:bb:eb:df:d3:d8:8c:72:79:bd:bf:10:
8b:01:e6:0f:7f:bb:f6:75:31:5a:40:ad:df:e1:07:
e6:12:12:b2:d3:99:d0:bd:24:5a:9a:ce:62:4f:da:
fe:0d:df:09:ae:da:04:83:54:e8:cb:68:c0:57:78:
c2:f4:68:42:d7:f4:81:4a:a3:b4:4e:0b:49:95:26:
1d:15
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Subject Key Identifier:
8E:99:9D:C0:70:98:57:16:08:8E:DF:6E:51:78:A6:86:18:FF:06:52
X509v3 Authority Key Identifier:
keyid:BA:CD:8E:03:77:4A:49:16:BB:D6:2E:6C:A3:9F:4B:C3:4C:5A:95:AB
DirName:/CN=Easy-RSA CA
serial:12:FD:5F:D3:28:D5:B6:4A:FE:44:D6:0B:22:D6:77:DD:E4:C9:C0:73
X509v3 Extended Key Usage:
TLS Web Server Authentication
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 Subject Alternative Name:
DNS:127.0.0.1, IP Address:127.0.0.1
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
3e:56:20:f9:3b:fa:13:6e:7e:a9:80:a6:15:18:01:82:f1:b8:
4d:1b:f1:ee:da:ed:50:f7:3b:13:01:a5:14:f9:4c:0e:34:57:
dc:e6:d1:7e:02:30:af:3b:fd:c9:ae:18:16:c9:3b:0a:4e:20:
da:cd:e8:cc:05:0c:b3:7d:6f:e5:15:ff:66:59:6b:fe:ff:1a:
ef:ca:b5:3a:1a:ad:dd:f6:19:43:d9:2b:61:18:29:95:b4:0c:
1e:b2:4a:ce:80:d3:1b:59:dc:62:ec:50:21:37:9c:2f:7a:4d:
c2:ac:de:1b:1d:a3:25:e0:e8:33:42:cf:77:31:2a:f2:44:36:
ef:59:89:da:6c:3e:9a:e8:d7:06:39:17:d5:78:82:6d:b6:63:
3f:9a:40:3b:e6:12:58:52:3d:63:4e:85:0b:02:cb:40:d2:8a:
59:8d:8f:ee:4a:c8:97:91:51:a9:2f:1b:15:81:9c:20:dd:94:
08:6f:ac:fa:c6:28:90:6c:17:5a:23:87:9a:5b:e5:c6:2e:f3:
09:66:de:76:1b:60:42:c1:5c:71:88:87:f6:7b:cb:e3:7e:14:
67:c9:a0:15:98:b6:7b:75:40:9a:08:fc:77:39:3a:23:cb:e3:
78:7d:57:f9:a7:66:36:b4:b5:07:de:61:3a:dd:07:58:b3:4f:
41:f6:f4:d9
-----BEGIN CERTIFICATE-----
MIIDhDCCAmygAwIBAgIQQjilb3BTQOSkGiwP/IETQjANBgkqhkiG9w0BAQsFADAW
MRQwEgYDVQQDDAtFYXN5LVJTQSBDQTAeFw0yNjA0MjkxMjA5MzVaFw0yODA4MDEx
MjA5MzVaMCUxIzAhBgNVBAMMGmV4YW1wbGVfc3NsX3JjcGNfcmNwc19jZXJ0MIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArg/h7vz223VFwPRJckY949sM
xDTSnknUhk8ZDVVwUIHk5mRWqFjo5lQKFrz0S4TNHbku7WK2zWI1i4EYq/9j9cHc
Fj6o3KwR3UMS+O/y8a+E/YP+qNNGfXfmrpVhpsmZa0BhjW5+Zh6Xd7Dotz061dfT
7maVYoMUzF4y/5698QbmjWp8CiciGbkGCc/vx9zojwRLgw3MjbHCz6tAJW7yv7fG
HY/S/D3Iob5KCbmR43ZPx5v8L97Zu+vf09iMcnm9vxCLAeYPf7v2dTFaQK3f4Qfm
EhKy05nQvSRams5iT9r+Dd8JrtoEg1Toy2jAV3jC9GhC1/SBSqO0TgtJlSYdFQID
AQABo4G+MIG7MAkGA1UdEwQCMAAwHQYDVR0OBBYEFI6ZncBwmFcWCI7fblF4poYY
/wZSMFEGA1UdIwRKMEiAFLrNjgN3SkkWu9YubKOfS8NMWpWroRqkGDAWMRQwEgYD
VQQDDAtFYXN5LVJTQSBDQYIUEv1f0yjVtkr+RNYLItZ33eTJwHMwEwYDVR0lBAww
CgYIKwYBBQUHAwEwCwYDVR0PBAQDAgWgMBoGA1UdEQQTMBGCCTEyNy4wLjAuMYcE
fwAAATANBgkqhkiG9w0BAQsFAAOCAQEAPlYg+Tv6E25+qYCmFRgBgvG4TRvx7trt
UPc7EwGlFPlMDjRX3ObRfgIwrzv9ya4YFsk7Ck4g2s3ozAUMs31v5RX/Zllr/v8a
78q1Ohqt3fYZQ9krYRgplbQMHrJKzoDTG1ncYuxQITecL3pNwqzeGx2jJeDoM0LP
dzEq8kQ271mJ2mw+mujXBjkX1XiCbbZjP5pAO+YSWFI9Y06FCwLLQNKKWY2P7krI
l5FRqS8bFYGcIN2UCG+s+sYokGwXWiOHmlvlxi7zCWbedhtgQsFccYiH9nvL434U
Z8mgFZi2e3VAmgj8dzk6I8vjeH1X+admNrS1B95hOt0HWLNPQfb02Q==
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCuD+Hu/PbbdUXA
9ElyRj3j2wzENNKeSdSGTxkNVXBQgeTmZFaoWOjmVAoWvPRLhM0duS7tYrbNYjWL
gRir/2P1wdwWPqjcrBHdQxL47/Lxr4T9g/6o00Z9d+aulWGmyZlrQGGNbn5mHpd3
sOi3PTrV19PuZpVigxTMXjL/nr3xBuaNanwKJyIZuQYJz+/H3OiPBEuDDcyNscLP
q0AlbvK/t8Ydj9L8PcihvkoJuZHjdk/Hm/wv3tm769/T2Ixyeb2/EIsB5g9/u/Z1
MVpArd/hB+YSErLTmdC9JFqazmJP2v4N3wmu2gSDVOjLaMBXeML0aELX9IFKo7RO
C0mVJh0VAgMBAAECggEAGBhmQhdeE+Cu1Ihsn2dWW3PAF2wpiNR3GVWbRfOBHf/x
QCx9K4ZNTU8ua1niZo7edyIiuyVaYWGaQHLRR8QNoiBhN2oapZujSHInzvKmeqr9
ubt7NgMzQ5ykwB+5OiW3uXda2cGFOV08QgspF+6ftakQMzUbslyrdSQIIscmi5Ya
uTDvE6+lBFinxy3RHFKVCZ3UrsDwfHR4eTUmgCHRB27joB7DFXL32amv0M8HjoGz
EZKGJgTwmRf9U4z4D4wCnOfVAPlsuthKUqMuTlBg0ZEstMZrzlP4suT2ieku0Usv
0XbJ38VozPYYFdR7nApVVvrJgHzI9cpoUbGto4BLOQKBgQDbXjFVLffOec8hv9dN
2VGZQmK61S9OrbvTnEOlxJd+kRid71X1pV5TuPKJJQtUJXf429bQOs40YbLeOmJt
BiRSR5yIBH7hDDC/c0ynqunstwDlgz+QX2Oh2B4alvVaWy0rZYF6NpBiI0+R5r5V
C2fHRS4LLPoflg83+CMubyLS6QKBgQDLIOXxlp1JQTzXhJkrkytLkafmEHAafovt
wbRD50/s+dl16BRX12sK0gXj2vwu0FleUD6Z7afDfspmvQdg3fyDxYw9Q+vw5LYQ
7BvoVU99o1m468yXwX/v36peCt4nOpwkJZKJfjgxjnMJByyeSUgL9uW4K+0D0LBV
a5Iv7QklTQKBgH30BkVPIHKIE/rfyIJlXemuaTu2/fOh4y9sEJdUWluMeeLssaFa
ct+FWJSQFYIaBVl4+E0VBqKi2e2o/ix1E1O+1ExwsF0M/8xdKk024BtPNA+TnWKK
so0Rpq9Dr9pScYvyOzZtr9b5SU2PfAcehlavDPHTwEV0hoZvTdvyab9JAoGADMBJ
7vp3cSvJN/Y470VTyHCiS4zonKEpA4nPWRviJowgnIgvDryVGZ7Jg94xSncFxSfg
ZiVHDLye1Ag1uFz3BwaVoRrsarjQvQs1TUZdsRNaBIO42iXpdBNkTHb+LxQ8zQAW
zM7BlErO6dgrctxCy416Ki+Ht1+YUiRojt2gX1kCgYBqytUy+XkPi5j3Ga29xcvP
WI3Uc8RI2GmoAmrw5QFiSG6lNXAzfo2ZNQbFnxgxeMOG9fV9yzBdIjXWNWr0E/KH
Fsb65R8iIrXQB9BZjuQqjz9nDm7eZZUBNGGbQ4DgSepnp194gXC5DoAElzuwOXbE
pY/kM1KwlpUR3J3LeF3i+Q==
-----END PRIVATE KEY-----

115
contrib/rcp/usage_example/certs/example_ssl_rcpm_rcps_cert.pem Normal file
View File

@@ -0,0 +1,115 @@
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
e7:09:ab:70:b5:dc:1f:11:d9:2a:23:04:39:87:34:f3
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=Easy-RSA CA
Validity
Not Before: Apr 29 12:09:35 2026 GMT
Not After : Aug 1 12:09:35 2028 GMT
Subject: CN=example_ssl_rcpm_rcps_cert
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:cc:79:9b:d3:f3:1f:41:9f:00:48:cd:47:0b:ae:
b9:1c:4e:3e:55:e2:4e:5f:a8:cc:13:d5:dd:bd:f0:
01:4c:19:ae:e3:a9:09:06:89:92:49:f7:bb:90:28:
fb:8c:22:69:b5:f5:a0:50:3d:97:0f:1e:1d:b1:a8:
57:9b:d7:e2:0d:99:67:7f:02:82:0c:9c:8e:dd:13:
03:28:93:b5:cb:7e:b5:78:06:10:bf:7b:55:c3:f7:
10:8b:20:4a:1c:f9:f1:b2:fa:f1:c7:44:9d:0a:ce:
ef:8d:f9:e8:ff:d1:c1:69:ec:8e:5f:11:cc:c9:98:
d5:1c:33:e2:5b:7a:4d:34:dc:76:c3:cd:db:4c:93:
d1:08:78:6f:3c:9a:ee:74:39:1e:cd:65:1e:c9:35:
cc:3b:2b:9e:d7:49:10:8e:58:85:b0:10:5b:90:1e:
f1:5e:d5:92:04:93:f9:33:c6:9d:77:63:d1:33:46:
5b:98:ff:9a:a8:f5:df:f7:84:21:e2:88:28:7a:a4:
c6:0d:9f:25:7e:0d:73:5b:d5:53:4a:90:79:94:37:
14:f3:c8:75:76:d4:1c:32:51:bf:58:16:74:d5:8d:
18:b6:53:f4:ab:cb:91:a8:8c:a3:ca:3c:5c:35:b6:
5f:62:57:37:5a:75:28:b7:4d:26:aa:ea:50:da:a4:
1c:55
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Subject Key Identifier:
47:92:B5:81:8B:5C:14:98:B3:83:B6:EB:06:9F:43:F3:3A:7E:ED:24
X509v3 Authority Key Identifier:
keyid:BA:CD:8E:03:77:4A:49:16:BB:D6:2E:6C:A3:9F:4B:C3:4C:5A:95:AB
DirName:/CN=Easy-RSA CA
serial:12:FD:5F:D3:28:D5:B6:4A:FE:44:D6:0B:22:D6:77:DD:E4:C9:C0:73
X509v3 Extended Key Usage:
TLS Web Server Authentication
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 Subject Alternative Name:
DNS:127.0.0.1, IP Address:127.0.0.1
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
6d:31:e6:29:d2:3b:a8:90:5c:4b:ac:61:15:95:5d:70:66:a5:
77:9d:88:47:49:73:75:be:70:69:d8:2f:62:82:5e:83:86:3b:
a8:48:3f:f1:5f:22:ae:81:23:64:c4:f2:2b:dd:4d:be:e5:6a:
26:a5:ea:c7:ba:1b:3e:6a:34:03:5a:f1:49:28:5f:56:4a:a6:
0e:1b:7a:07:48:76:95:b6:4b:f5:3f:b9:67:2e:e0:33:06:80:
d4:d6:01:a5:76:01:c0:a5:18:e5:38:8b:52:73:6e:6d:45:50:
b7:9a:ab:86:5d:e3:65:b4:b8:c7:ee:b2:dc:bf:e3:d5:bb:e4:
91:eb:f5:0c:38:22:5e:37:54:9e:ba:96:25:10:04:18:23:f7:
ae:73:4d:d0:aa:03:81:b4:89:36:97:15:da:1a:60:a0:98:5f:
03:f8:1b:22:83:57:41:4b:12:28:7d:8d:ea:88:74:24:28:5c:
53:41:89:5e:9a:da:fd:7b:bf:60:dc:de:9b:49:ce:5c:a3:b2:
01:7d:1d:cb:28:8c:ba:f4:7b:5d:2b:cb:15:5b:2a:97:1a:d1:
f9:e7:12:e3:43:b9:f4:2a:88:dd:6d:b6:a0:72:d3:bd:63:23:
e9:d7:f0:ac:b5:6d:0d:f2:d9:8b:2c:c4:35:5b:4d:83:dc:e8:
7d:0b:3d:a3
-----BEGIN CERTIFICATE-----
MIIDhTCCAm2gAwIBAgIRAOcJq3C13B8R2SojBDmHNPMwDQYJKoZIhvcNAQELBQAw
FjEUMBIGA1UEAwwLRWFzeS1SU0EgQ0EwHhcNMjYwNDI5MTIwOTM1WhcNMjgwODAx
MTIwOTM1WjAlMSMwIQYDVQQDDBpleGFtcGxlX3NzbF9yY3BtX3JjcHNfY2VydDCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMx5m9PzH0GfAEjNRwuuuRxO
PlXiTl+ozBPV3b3wAUwZruOpCQaJkkn3u5Ao+4wiabX1oFA9lw8eHbGoV5vX4g2Z
Z38Cggycjt0TAyiTtct+tXgGEL97VcP3EIsgShz58bL68cdEnQrO74356P/RwWns
jl8RzMmY1Rwz4lt6TTTcdsPN20yT0Qh4bzya7nQ5Hs1lHsk1zDsrntdJEI5YhbAQ
W5Ae8V7VkgST+TPGnXdj0TNGW5j/mqj13/eEIeKIKHqkxg2fJX4Nc1vVU0qQeZQ3
FPPIdXbUHDJRv1gWdNWNGLZT9KvLkaiMo8o8XDW2X2JXN1p1KLdNJqrqUNqkHFUC
AwEAAaOBvjCBuzAJBgNVHRMEAjAAMB0GA1UdDgQWBBRHkrWBi1wUmLODtusGn0Pz
On7tJDBRBgNVHSMESjBIgBS6zY4Dd0pJFrvWLmyjn0vDTFqVq6EapBgwFjEUMBIG
A1UEAwwLRWFzeS1SU0EgQ0GCFBL9X9Mo1bZK/kTWCyLWd93kycBzMBMGA1UdJQQM
MAoGCCsGAQUFBwMBMAsGA1UdDwQEAwIFoDAaBgNVHREEEzARggkxMjcuMC4wLjGH
BH8AAAEwDQYJKoZIhvcNAQELBQADggEBAG0x5inSO6iQXEusYRWVXXBmpXediEdJ
c3W+cGnYL2KCXoOGO6hIP/FfIq6BI2TE8ivdTb7laial6se6Gz5qNANa8UkoX1ZK
pg4begdIdpW2S/U/uWcu4DMGgNTWAaV2AcClGOU4i1Jzbm1FULeaq4Zd42W0uMfu
sty/49W75JHr9Qw4Il43VJ66liUQBBgj965zTdCqA4G0iTaXFdoaYKCYXwP4GyKD
V0FLEih9jeqIdCQoXFNBiV6a2v17v2Dc3ptJzlyjsgF9HcsojLr0e10ryxVbKpca
0fnnEuNDufQqiN1ttqBy071jI+nX8Ky1bQ3y2YssxDVbTYPc6H0LPaM=
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDMeZvT8x9BnwBI
zUcLrrkcTj5V4k5fqMwT1d298AFMGa7jqQkGiZJJ97uQKPuMImm19aBQPZcPHh2x
qFeb1+INmWd/AoIMnI7dEwMok7XLfrV4BhC/e1XD9xCLIEoc+fGy+vHHRJ0Kzu+N
+ej/0cFp7I5fEczJmNUcM+Jbek003HbDzdtMk9EIeG88mu50OR7NZR7JNcw7K57X
SRCOWIWwEFuQHvFe1ZIEk/kzxp13Y9EzRluY/5qo9d/3hCHiiCh6pMYNnyV+DXNb
1VNKkHmUNxTzyHV21BwyUb9YFnTVjRi2U/Sry5GojKPKPFw1tl9iVzdadSi3TSaq
6lDapBxVAgMBAAECggEADJFN6K9OWhYX1PcEWUgOLxqdCLd95Iccsfxot7ekcMUP
A4WnHRyACLqor9c2V3o2//IpU2fnB2IXu6ISmRd3WKl3hm4vnZmoIJeTpQm9Iv/g
+fqkyrbIgktcHDJUySal+n+jiYFNW2B1h1xXUT/scMz+FthNJg1Azfi0vorMFjCk
SBOSo7BQ2hiQ83FneVJU1TsxD4S4IBLx9fF6AW05norRmvm17Ip2pKk4QYzKiOI3
NIoSwbOgU0Vp1X3MMlilM5ZZdN3a9lI6lfBZE682WOxBmH67mKMQnR8aC+nwynQ+
45pUQIn8Fjx2hQHehbD7ZNw9Nob9AyGWqWGKFV74IQKBgQDox/j7dCHNm1mz1QRm
Q2YyN4OGXJI97t9Gd8UdNCv5bAkdx2cR2lmP3tRc6iAzrfNIpPCGaXo1vwJx477X
wO95W2b4hfm3j6v0cqRbsFzzVIHZUB77pXfAJfZeICpoqu0vxn5nb+yPgzgmToLX
pbIDdqWafzzrDLTmLCfwfKDI9QKBgQDg3tgpAXo8WCL4phNuW44XQ172lPTijpn+
wj1Z0rBrS806gL/+QvZZLS1WCym/QBV7TgGlxIJbmAfghcGyin3NliskSHAiccxG
/9eCSQes8czfsVj6qmBMwyff5r+wmk662qV0u07UHmuykYk3Dgs/zYdwq2SsTlL4
Y9eRjutp4QKBgQCONg0wYcR8/hmROeRULXzz1OJvZYKaf6K8RFOSAduTp6LyJG4d
hA4PTQzkLsy5hd4JVWr0UuAskaMGvSJMYTxsIaEI16C1ufpNfvRWZ6qBpfEmOEKV
boN4Sjj3TCNcioAZHeT/gGs/SeU10eUxpbLZVtTZTD6FQuAJdpR34UvBOQKBgFNM
mXxPLM2vxHyhYK9PwQoDDel/8lr+gjMqFvnwHyQP911FllmEyqbsIlAuYG+VOJ/t
nJSgf72YSsq0IbWWsdV3XFHbd5Z62zYtzdJYZTx+cesnUhPBC11EKcA6RSYRczqq
hgIA5MmU30ZNvSukyyv+Yb6t7uQZO4kByzgDXldhAoGBALgRkAHxgbKUXp5XyCDJ
e8dwVx0g9tfDM/DEZtU/Si5oUaunBaPV/Byov7OXOT02V8JLnA5ChUYgwUFI030x
QL/3eK12Qh5Gb9VabvYCicDRk4GzmqZU9Wcvm1zgbUr5jY8Lou44nFjol/Y1m70n
51WZbVkkWmBZO5m3NqN66SkJ
-----END PRIVATE KEY-----

115
contrib/rcp/usage_example/certs/example_ssl_rcps_rcpm_cert.pem Normal file
View File

@@ -0,0 +1,115 @@
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
28:96:2e:a1:40:e0:7e:f1:fb:63:1a:f4:53:6f:ce:fb
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=Easy-RSA CA
Validity
Not Before: Apr 29 12:09:35 2026 GMT
Not After : Aug 1 12:09:35 2028 GMT
Subject: CN=example_ssl_rcps_rcpm_cert
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ae:11:46:ef:d1:81:34:dd:23:5d:54:40:f3:9c:
85:35:95:a6:91:57:92:5c:bf:eb:40:34:69:eb:db:
c0:86:3c:7b:ff:9c:d7:ba:0e:41:57:84:15:cd:94:
f1:48:63:50:9c:34:97:ee:be:be:b0:27:d8:fd:cd:
8a:cf:85:ff:08:1f:07:d8:28:96:0e:e4:2d:d0:8b:
df:a8:fa:41:47:a0:a2:80:2e:2e:58:01:cc:6f:43:
5c:c2:fb:84:a7:ff:9e:97:bb:b3:a3:1f:63:64:73:
8d:73:dd:f4:7e:96:d7:6b:b3:cb:e2:35:59:55:e0:
e7:e3:c0:41:f8:b6:0f:c5:46:4c:cd:0e:91:80:ef:
e3:43:f0:72:26:12:10:be:83:a2:db:23:2d:b4:b1:
07:5a:b1:b3:10:9c:09:69:98:42:79:81:77:5e:22:
e4:71:47:70:27:15:2c:a7:13:c2:6d:44:59:b4:73:
c9:bb:27:7f:d6:e8:3d:85:bb:36:f6:cb:71:36:11:
b1:99:1a:1d:1a:15:dd:cd:65:7f:cd:cc:10:00:49:
ed:07:2d:7b:15:88:be:73:ba:1d:15:69:bc:d3:02:
55:ea:dc:2c:3f:0b:cd:18:57:59:7a:e3:09:b2:89:
cd:d6:e7:f6:95:c4:2e:8a:53:2b:a8:96:82:94:53:
00:77
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Subject Key Identifier:
60:BD:48:06:68:15:D4:DC:ED:EE:E4:C7:B1:9F:C4:93:6D:50:3A:77
X509v3 Authority Key Identifier:
keyid:BA:CD:8E:03:77:4A:49:16:BB:D6:2E:6C:A3:9F:4B:C3:4C:5A:95:AB
DirName:/CN=Easy-RSA CA
serial:12:FD:5F:D3:28:D5:B6:4A:FE:44:D6:0B:22:D6:77:DD:E4:C9:C0:73
X509v3 Extended Key Usage:
TLS Web Server Authentication
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 Subject Alternative Name:
DNS:127.0.0.1, IP Address:127.0.0.1
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
c5:35:61:58:23:e2:69:da:6c:d5:41:ab:a8:70:f4:dd:cc:a0:
a3:3d:84:89:93:b6:7f:69:7d:10:35:9d:c5:d1:0d:db:d2:d7:
36:af:d4:54:30:14:a7:5d:31:ca:5c:13:92:d5:60:50:f8:56:
4a:cb:16:b1:b3:b1:03:bf:96:53:77:1f:4a:0f:9c:29:2b:bf:
a4:e0:da:6f:ad:13:c7:2d:8e:18:c4:72:50:17:ed:1f:36:51:
7a:12:9f:fc:a6:d6:c8:55:e0:db:ea:16:d6:22:0d:a2:cb:eb:
b2:ba:07:92:2f:db:33:d6:a2:0c:ec:89:29:f1:96:40:e5:0b:
e6:1f:08:50:d6:29:87:a8:20:b2:e2:17:50:25:ff:53:36:ee:
7f:ce:e6:1d:ed:b3:16:61:18:42:a9:17:9e:a6:86:0d:a5:fc:
f9:42:c8:50:48:74:72:35:eb:8c:ff:4d:e8:98:88:a0:b4:b3:
d0:82:b3:2f:ea:19:d7:d5:ac:47:35:96:24:37:34:0c:7a:a2:
e0:4d:99:a7:55:61:85:1e:7e:6a:23:77:f5:07:13:e6:50:5c:
65:00:13:f6:b5:4b:5b:8c:11:c3:5d:af:ba:41:e9:84:1d:f1:
a4:70:16:28:c2:be:6e:d8:67:38:c5:a0:ba:8a:64:6f:27:ce:
63:a0:92:9b
-----BEGIN CERTIFICATE-----
MIIDhDCCAmygAwIBAgIQKJYuoUDgfvH7Yxr0U2/O+zANBgkqhkiG9w0BAQsFADAW
MRQwEgYDVQQDDAtFYXN5LVJTQSBDQTAeFw0yNjA0MjkxMjA5MzVaFw0yODA4MDEx
MjA5MzVaMCUxIzAhBgNVBAMMGmV4YW1wbGVfc3NsX3JjcHNfcmNwbV9jZXJ0MIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArhFG79GBNN0jXVRA85yFNZWm
kVeSXL/rQDRp69vAhjx7/5zXug5BV4QVzZTxSGNQnDSX7r6+sCfY/c2Kz4X/CB8H
2CiWDuQt0IvfqPpBR6CigC4uWAHMb0NcwvuEp/+el7uzox9jZHONc930fpbXa7PL
4jVZVeDn48BB+LYPxUZMzQ6RgO/jQ/ByJhIQvoOi2yMttLEHWrGzEJwJaZhCeYF3
XiLkcUdwJxUspxPCbURZtHPJuyd/1ug9hbs29stxNhGxmRodGhXdzWV/zcwQAEnt
By17FYi+c7odFWm80wJV6twsPwvNGFdZeuMJsonN1uf2lcQuilMrqJaClFMAdwID
AQABo4G+MIG7MAkGA1UdEwQCMAAwHQYDVR0OBBYEFGC9SAZoFdTc7e7kx7GfxJNt
UDp3MFEGA1UdIwRKMEiAFLrNjgN3SkkWu9YubKOfS8NMWpWroRqkGDAWMRQwEgYD
VQQDDAtFYXN5LVJTQSBDQYIUEv1f0yjVtkr+RNYLItZ33eTJwHMwEwYDVR0lBAww
CgYIKwYBBQUHAwEwCwYDVR0PBAQDAgWgMBoGA1UdEQQTMBGCCTEyNy4wLjAuMYcE
fwAAATANBgkqhkiG9w0BAQsFAAOCAQEAxTVhWCPiadps1UGrqHD03cygoz2EiZO2
f2l9EDWdxdEN29LXNq/UVDAUp10xylwTktVgUPhWSssWsbOxA7+WU3cfSg+cKSu/
pODab60Txy2OGMRyUBftHzZRehKf/KbWyFXg2+oW1iINosvrsroHki/bM9aiDOyJ
KfGWQOUL5h8IUNYph6ggsuIXUCX/Uzbuf87mHe2zFmEYQqkXnqaGDaX8+ULIUEh0
cjXrjP9N6JiIoLSz0IKzL+oZ19WsRzWWJDc0DHqi4E2Zp1VhhR5+aiN39QcT5lBc
ZQAT9rVLW4wRw12vukHphB3xpHAWKMK+bthnOMWguopkbyfOY6CSmw==
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCuEUbv0YE03SNd
VEDznIU1laaRV5Jcv+tANGnr28CGPHv/nNe6DkFXhBXNlPFIY1CcNJfuvr6wJ9j9
zYrPhf8IHwfYKJYO5C3Qi9+o+kFHoKKALi5YAcxvQ1zC+4Sn/56Xu7OjH2Nkc41z
3fR+ltdrs8viNVlV4OfjwEH4tg/FRkzNDpGA7+ND8HImEhC+g6LbIy20sQdasbMQ
nAlpmEJ5gXdeIuRxR3AnFSynE8JtRFm0c8m7J3/W6D2Fuzb2y3E2EbGZGh0aFd3N
ZX/NzBAASe0HLXsViL5zuh0VabzTAlXq3Cw/C80YV1l64wmyic3W5/aVxC6KUyuo
loKUUwB3AgMBAAECggEABGfLiSZJmYeUmDgZrLtkDlx16sJx9zGkR+u2V+cn6D3U
+uiCoo2EedfjSrYKT/AI35Xf19sGrc6ptJgPJfwY3aEWFxJv5HtB7ZVHWS98QiPT
+QqHgb1fPzGlQgoQ7Bo8GVBW1joPz0Bdbsv0ntTn1CyowauNUe8Z71mzpxJ0iQ7u
Z8LNoD7INEAZDBjHVov6pLGDHS9KFxGy20WG549mE37I4QxyxetJEgmuyhJkZOQ4
noEWiCMQjGsSg4YuSc1GS1jAVf3p2g3/TiheD/31r1jleY/T5s2qYC6MJ4vY+7yA
5sl7m8A47i0lHSKBdR3nWz+vXEsxB0nXK3Sulyt9AQKBgQDYRZ1nEe32CCZcWn/7
nG+9e6XNOe9E25skuvY/uEpikt92kdnyZOgHFfwM9Nv+w2IWGLZ8MmDDJ6/4hGvj
fJjcOUb3d/SJiivPvdRC9GYMrDUZe5AJ3p6fPqi4IeZOw7bMTcVB6aHAr9KoO//J
2t0WNvwzkOyl6KlhaQEIDK8bOQKBgQDOCvWvUEg8nHaKcmN0uQaKuPvCorhnyqUT
VFqLlSrYC79ffHCwp2y8nkFUnxpeHXENrtHtcrdnKBNkgGEn2l4xs63CjhKrBuIG
bDjrtp3vKlHRhQjX6HkrEEuUk52wYzuX7CfU8nTZnrtV74MocOEewJVW+84Rtwiw
vIUcgfgJLwKBgAbgj9TLOSntsGqXZiJ2Iwd/exI/mWAzK4fLejEkhxkDWp/Gm4ud
sdMn28/9qVE8nU3ek073uyP5ixr3+wZM2/+EwsDzy47kGeiNPMa0Rtp4T2f0CeyG
a7zcnTjduxkeGB3/CxrBdydNcAFxhvzAPO+L6BErtpq//0Ldt+6tmJPhAoGAFxEh
Cjx5qdd2ae9+dO3V7qfg/5xJ+sy0CGL0NBZCEqfWB/GdiBlmUgOBmuCpCgpPwtFk
jSm/oJva9/BrcBPBYd0Uweg37M+7dC6ffLwYGFNrj4JOSCWtkwWjAII6MCob3NlC
aFOwg0CDBo7m5xskCNZUocVU/6S3I1onqNZgF18CgYAczBf1NS4VPZvebrutTBEH
zyUX3XU9mR+dy0ncCGNVS8zYMtz7cweZInzNB2cTOfisIHzzdOnazm7D9uzPsREi
pKgaL+ErWYDlGiDTxMtGSRPTWGocYBYdU6y/0bobhZb0qyvyRhpGvPK0ReMUuvqu
FkNgoQ1lo0n6vawvxWW8Mw==
-----END PRIVATE KEY-----

49
contrib/rcp/usage_example/certs/make_certs.sh Executable file
View File

@@ -0,0 +1,49 @@
#!/bin/bash
EASYRSA=/usr/share/easy-rsa/easyrsa
CA_NAME="example_ssl_rcp_ca_cert"
export EASYRSA_PASSIN=pass:test
export EASYRSA_PASSOUT=pass:test
echo "Cleaning up..."
rm -rf ./ca
rm -rf ./*.pem
rm -rf ./*.key
rm -rf ./*.crt
echo "Creating CA cert..."
mkdir -p ./ca
cd ./ca
$EASYRSA init-pki
cp ../vars ./pki/
$EASYRSA --batch build-ca
cp ./pki/ca.crt ../$CA_NAME.crt
echo "Creating server certs..."
# Secures connection between RCP-Client and RCP-Server:
$EASYRSA --batch --subject-alt-name="DNS:127.0.0.1,IP:127.0.0.1" build-server-full example_ssl_rcpc_rcps_cert nopass
# Secures connection between RCP-Module and RCP-Server (module description):
$EASYRSA --batch --subject-alt-name="DNS:127.0.0.1,IP:127.0.0.1" build-server-full example_ssl_rcpm_rcps_cert nopass
# Secures connection between RCP-Server and RCP-Module (command execution):
$EASYRSA --batch --subject-alt-name="DNS:127.0.0.1,IP:127.0.0.1" build-server-full example_ssl_rcps_rcpm_cert nopass
echo "Collecting server certs..."
cp ./pki/issued/* ../
cp ./pki/private/* ../
cd ..
rm ./ca.key
echo "Merging server certs..."
for CRT in ./*.crt; do
CRT_NAME=`basename ${CRT%.*}`
if [ -f $CRT_NAME.key ]; then
cat $CRT_NAME.crt $CRT_NAME.key > $CRT_NAME.pem
rm $CRT_NAME.key
rm $CRT_NAME.crt
fi
done
echo "Finalizing..."
rm -rf ./ca

8
contrib/rcp/usage_example/encrypt_card_data.sh Executable file
View File

@@ -0,0 +1,8 @@
#!/bin/bash
. ./params.cfg
PYTHONPATH=$PYSIM_DIR ../../csv-encrypt-columns.py \
--csv-column-key kic:$CSV_COLUMN_KEY \
--csv-column-key kid:$CSV_COLUMN_KEY \
--csv-column-key kik:$CSV_COLUMN_KEY \
card_data.csv

40
contrib/rcp/usage_example/params.cfg Normal file
View File

@@ -0,0 +1,40 @@
# PYSIM_DIR passed to all components
PYSIM_DIR=../../../ # Points to the psyim top directory
# Verbosity switch passed to all components (comment-out to disable verbode mode)
#VERBOSE="--verbose"
# CSV column key to decrypt KIC, KID and KIK in csv_data.csv.encr
# (use encrypt_card_data.sh to regenerate csv_data.csv.encr from csv_data.csv)
CSV_COLUMN_KEY="00112233445566778899AABBCCDDEEFF"
# PCSC reader that the RCP Client shall use
PCSC_READER=0
# Since RCP Modules are custom implementations, they will most likely reside
# in a dedicated directory. This directory is passed together with PYSIM_DIR
# via PYTHONPATH to the module.
RCP_DIR=../
# CA of the certificates used in this example
CA_CERT="./certs/example_ssl_rcp_ca_cert.crt"
# Network interface where RCP Clients connect
RCPC_SERVER_PORT=8000
RCPC_SERVER_ADDR="127.0.0.1"
RCPC_SERVER_CERT="./certs/example_ssl_rcpc_rcps_cert.pem"
RCPC_SERVER_URI="wss://$RCPC_SERVER_ADDR:$RCPC_SERVER_PORT"
# Network interface where RCP Modules connect
RCPM_SERVER_PORT=8010
RCPM_SERVER_ADDR="127.0.0.1"
RCPM_SERVER_CERT="./certs/example_ssl_rcpm_rcps_cert.pem"
RCPM_SERVER_URI="wss://$RCPM_SERVER_ADDR:$RCPM_SERVER_PORT"
# Network interface where the (example) RCP Module binds its Command Server to.
# The command server is used by the RCP Server to run the command requested
# by the user. Each module needs a dedicated port. The address and port is
# automatically forwarded to the RCP Server.
RCPM_CMD_SERVER_PORT=8020
RCPM_CMD_SERVER_ADDR="127.0.0.1"
RCPM_CMD_SERVER_CERT="./certs/example_ssl_rcps_rcpm_cert.pem"

126
contrib/rcp/usage_example/rcp_module.py Executable file
View File

@@ -0,0 +1,126 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from pathlib import Path
from pySim.log import PySimLogger
from argparse import Namespace
from rcp_module_utils import rcpm_setup_argparse, rcpm_run_module, RcpModule, RcpmCmdSrvConnHdlr
from pySim.global_platform import GpCardKeyset, SCP02, establish_scp, release_scp, install, store_data
from Cryptodome.Random import get_random_bytes
from osmocom.utils import h2b, b2h
log = PySimLogger.get(Path(__file__).stem)
class ExmpleModule(RcpModule):
name = Path(__file__).stem
cmd_descr = [{'name' : 'reset',
'help': 'reset the card',
'args' : []},
{'name' : 'read_binary',
'help': 'read binary data from a transparent file.',
'args' : [{ 'name' : '--fid',
'spec' : {'required' : True,
'help' : 'File identifier to of the file to read',
'action' : 'append',
'pytype' : 'str'},
}
]},
{'name' : 'read_record',
'help': 'read binary data from a transparent file.',
'args' : [{ 'name' : '--fid',
'spec' : {'required' : True,
'help' : 'File identifier to of the file to read',
'action' : 'append',
'pytype' : 'str'},
},
{ 'name' : '--record',
'spec' : {'required' : True,
'help' : 'File record to read',
'default' : 1,
'pytype' : 'int'},
}
]},
{'name' : 'unlock_aram',
'help': 'unlock a locked ARA-M applet on a sysmoISIM-SJA5',
'args' : [],
'get_keys' : {'uicc' : ['KIC', 'KID', 'KIK']}}
]
suitable_for = [{'atr' : '3b9f96801f878031e073fe211b674a357530350265f8'}]
retrieve_uicc_keys = ['KIC', 'KID', 'KIK']
def cmd_reset(self, hdlr: RcpmCmdSrvConnHdlr) -> int:
hdlr.print("resetting UICC/eUICC ...")
hdlr.card._scc.reset_card()
hdlr.print("ATR is: %s" % hdlr.card._scc.get_atr())
return 0
def cmd_read_binary(self, hdlr: RcpmCmdSrvConnHdlr) -> int:
fid = hdlr.cmd_args.fid
hdlr.print("reading transparent file: %s ..." % fid)
(res, _) = hdlr.card._scc.read_binary(fid)
hdlr.print("file content is: %s" % res)
return 0
def cmd_read_record(self, hdlr: RcpmCmdSrvConnHdlr) -> int:
fid = hdlr.cmd_args.fid
record = hdlr.cmd_args.record
hdlr.print("reading linear-fixed file: %s ..." % fid)
(res, _) = hdlr.card._scc.read_record(fid, record)
hdlr.print("file content is: %s" % res)
return 0
def cmd_unlock_aram(self, hdlr: RcpmCmdSrvConnHdlr) -> int:
# Select ADF.ISD
hdlr.print("Selecting ADF.ISD ...")
hdlr.lchan.scc.send_apdu_checksw("00a4040408a00000000300000000")
# Estabish secure channel
hdlr.print("Establishing secure channel ...")
key_ver = 112
key_enc = hdlr.keys_uicc['KIC']
key_mac = hdlr.keys_uicc['KID']
key_dek = hdlr.keys_uicc['KIK']
security_level = 3
host_challenge_len = 8
host_challenge = get_random_bytes(host_challenge_len)
kset = GpCardKeyset(key_ver, h2b(key_enc), h2b(key_mac), h2b(key_dek))
scp = SCP02(card_keys=kset)
establish_scp(hdlr.lchan, scp, host_challenge, security_level)
# To prove that it works, we need to do something that actually requires to be authenticated
# via a secure channel. In this example we will send an unlock command to the ARA-M applet
# found on any sysmoISIM-SJA5 card. (see also: https://gitea.osmocom.org/sim-card/aram-applet)
hdlr.print("Unlocking ARA-M applet ...")
ara_m_aid = "a00000015141434c00"
install(hdlr.lchan, 0x20, 0x00, "0000%02x%s000000" % (len(ara_m_aid) // 2, ara_m_aid))
store_data(hdlr.lchan, h2b("A2"), structure = 'ber_tlv')
# Release the secure channel
hdlr.print("Done, releasing secure channel ...")
release_scp(hdlr.lchan)
return 0
if __name__ == '__main__':
option_parser = rcpm_setup_argparse("Example Module")
opts = option_parser.parse_args()
rcpm_run_module(opts, ExmpleModule)

16
contrib/rcp/usage_example/readme.txt Normal file
View File

@@ -0,0 +1,16 @@
How to try:
Go to the directory that contains the usage example:
cd pysim/contrib/rcp/usage_example
Edit card_data.csv to fill in the SCP02 keys for the ISD of your sysmoISIM-SJA5
Start the RCP Server:
./start_rcp_server.sh
Start the RCP Module:
./start_rcp_module.sh
Run the exmple scripts:
./run_rcp_client.sh
(it is also possible to call the run_rcp_client_*.sh scripts individually)

29
contrib/rcp/usage_example/run_rcp_client.sh Executable file
View File

@@ -0,0 +1,29 @@
#!/bin/bash
echo "basic help"
echo "===================================================================================="
./run_rcp_client_help.sh
echo "===================================================================================="
echo ""
echo ""
echo "help for which commands are available"
echo "===================================================================================="
./run_rcp_client_help_cmd.sh
echo "===================================================================================="
echo ""
echo ""
echo "help for specific commands"
echo "===================================================================================="
./run_rcp_client_help_cmd_specific.sh
echo "===================================================================================="
echo ""
echo ""
echo "run specific RCP commands"
echo "===================================================================================="
./run_rcp_client_cmd.sh
echo "===================================================================================="
echo ""
echo ""

28
contrib/rcp/usage_example/run_rcp_client_cmd.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI\
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_reset
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_read_binary --fid 3f00 --fid 2fe2
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_read_record --fid 3f00 --fid 2f00 --record 1
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_unlock_aram

6
contrib/rcp/usage_example/run_rcp_client_help.sh Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
-h

9
contrib/rcp/usage_example/run_rcp_client_help_cmd.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
-h

28
contrib/rcp/usage_example/run_rcp_client_help_cmd_specific.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_reset --help
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_read_binary --help
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_read_record --help
PYTHONPATH=$PYSIM_DIR ../rcp_client.py $VERBOSE \
--uri $RCPC_SERVER_URI \
--ca-cert $CA_CERT \
-p $PCSC_READER \
rcp_module_unlock_aram --help

14
contrib/rcp/usage_example/start_rcp_module.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR:$RCP_DIR ./rcp_module.py $VERBOSE \
--uri $RCPM_SERVER_URI \
--rcps-ca-cert $CA_CERT \
--rcpm-cmd-server-addr $RCPM_CMD_SERVER_ADDR \
--rcpm-cmd-server-port $RCPM_CMD_SERVER_PORT \
--rcpm-cmd-server-cert $RCPM_CMD_SERVER_CERT \
--column-key kic:$CSV_COLUMN_KEY \
--column-key kid:$CSV_COLUMN_KEY \
--column-key kik:$CSV_COLUMN_KEY

13
contrib/rcp/usage_example/start_rcp_server.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
. ./params.cfg
set -x
PYTHONPATH=$PYSIM_DIR ../rcp_server.py $VERBOSE \
--rcpc-server-addr $RCPC_SERVER_ADDR \
--rcpc-server-port $RCPC_SERVER_PORT \
--rcpc-server-cert $RCPC_SERVER_CERT \
--rcpm-server-addr $RCPM_SERVER_ADDR \
--rcpm-server-port $RCPM_SERVER_PORT \
--rcpm-server-cert $RCPM_SERVER_CERT \
--rcpm-module-ca-cert $CA_CERT \
--csv ./card_data.csv.encr

489
contrib/saip-tool.py Executable file
View File

@@ -0,0 +1,489 @@
#!/usr/bin/env python3
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import os
import sys
import argparse
import logging
from pathlib import Path as PlPath
from typing import List
from osmocom.utils import h2b, b2h, swap_nibbles
from osmocom.construct import GreedyBytes, StripHeaderAdapter
from pySim.esim.saip import *
from pySim.esim.saip.validation import CheckBasicStructure
from pySim.pprint import HexBytesPrettyPrinter
pp = HexBytesPrettyPrinter(indent=4,width=500)
parser = argparse.ArgumentParser(description="""
Utility program to work with eSIM SAIP (SimAlliance Interoperable Profile) files.""")
parser.add_argument('INPUT_UPP', help='Unprotected Profile Package Input file')
parser.add_argument("--loglevel", dest="loglevel", choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
default='INFO', help="Set the logging level")
parser.add_argument('--debug', action='store_true', help='Enable DEBUG logging')
subparsers = parser.add_subparsers(dest='command', help="The command to perform", required=True)
parser_split = subparsers.add_parser('split', help='Split PE-Sequence into individual PEs')
parser_split.add_argument('--output-prefix', default='.', help='Prefix path/filename for output files')
parser_dump = subparsers.add_parser('dump', help='Dump information on PE-Sequence')
parser_dump.add_argument('mode', choices=['all_pe', 'all_pe_by_type', 'all_pe_by_naa'])
parser_dump.add_argument('--dump-decoded', action='store_true', help='Dump decoded PEs')
parser_check = subparsers.add_parser('check', help='Run constraint checkers on PE-Sequence')
parser_rpe = subparsers.add_parser('extract-pe', help='Extract specified PE to (DER encoded) file')
parser_rpe.add_argument('--pe-file', required=True, help='PE file name')
parser_rpe.add_argument('--identification', type=int, help='Extract PE matching specified identification')
parser_rpe = subparsers.add_parser('remove-pe', help='Remove specified PEs from PE-Sequence')
parser_rpe.add_argument('--output-file', required=True, help='Output file name')
parser_rpe.add_argument('--identification', default=[], type=int, action='append', help='Remove PEs matching specified identification')
parser_rpe.add_argument('--type', default=[], action='append', help='Remove PEs matching specified type')
parser_rn = subparsers.add_parser('remove-naa', help='Remove specified NAAs from PE-Sequence')
parser_rn.add_argument('--output-file', required=True, help='Output file name')
parser_rn.add_argument('--naa-type', required=True, choices=NAAs.keys(), help='Network Access Application type to remove')
# TODO: add an --naa-index or the like, so only one given instance can be removed
parser_info = subparsers.add_parser('info', help='Display information about the profile')
parser_info.add_argument('--apps', action='store_true', help='List applications and their related instances')
parser_eapp = subparsers.add_parser('extract-apps', help='Extract applications as loadblock file')
parser_eapp.add_argument('--output-dir', default='.', help='Output directory (where to store files)')
parser_eapp.add_argument('--format', default='cap', choices=['ijc', 'cap'], help='Data format of output files')
parser_aapp = subparsers.add_parser('add-app', help='Add application to PE-Sequence')
parser_aapp.add_argument('--output-file', required=True, help='Output file name')
parser_aapp.add_argument('--applet-file', required=True, help='Applet file name')
parser_aapp.add_argument('--aid', required=True, help='Load package AID')
parser_aapp.add_argument('--sd-aid', default=None, help='Security Domain AID')
parser_aapp.add_argument('--non-volatile-code-limit', default=None, type=int, help='Non volatile code limit (C6)')
parser_aapp.add_argument('--volatile-data-limit', default=None, type=int, help='Volatile data limit (C7)')
parser_aapp.add_argument('--non-volatile-data-limit', default=None, type=int, help='Non volatile data limit (C8)')
parser_aapp.add_argument('--hash-value', default=None, help='Hash value')
parser_rapp = subparsers.add_parser('remove-app', help='Remove application from PE-Sequence')
parser_rapp.add_argument('--output-file', required=True, help='Output file name')
parser_rapp.add_argument('--aid', required=True, help='Load package AID')
parser_aappi = subparsers.add_parser('add-app-inst', help='Add application instance to Application PE')
parser_aappi.add_argument('--output-file', required=True, help='Output file name')
parser_aappi.add_argument('--aid', required=True, help='Load package AID')
parser_aappi.add_argument('--class-aid', required=True, help='Class AID')
parser_aappi.add_argument('--inst-aid', required=True, help='Instance AID (must match Load package AID)')
parser_aappi.add_argument('--app-privileges', default='000000', help='Application privileges')
parser_aappi.add_argument('--volatile-memory-quota', default=None, type=int, help='Volatile memory quota (C7)')
parser_aappi.add_argument('--non-volatile-memory-quota', default=None, type=int, help='Non volatile memory quota (C8)')
parser_aappi.add_argument('--app-spec-pars', default='00', help='Application specific parameters (C9)')
parser_aappi.add_argument('--uicc-toolkit-app-spec-pars', help='UICC toolkit application specific parameters field')
parser_aappi.add_argument('--uicc-access-app-spec-pars', help='UICC Access application specific parameters field')
parser_aappi.add_argument('--uicc-adm-access-app-spec-pars', help='UICC Administrative access application specific parameters field')
parser_aappi.add_argument('--process-data', default=[], action='append', help='Process personalization APDUs')
parser_rappi = subparsers.add_parser('remove-app-inst', help='Remove application instance from Application PE')
parser_rappi.add_argument('--output-file', required=True, help='Output file name')
parser_rappi.add_argument('--aid', required=True, help='Load package AID')
parser_rappi.add_argument('--inst-aid', required=True, help='Instance AID')
esrv_flag_choices = [t.name for t in asn1.types['ServicesList'].type.root_members]
parser_esrv = subparsers.add_parser('edit-mand-srv-list', help='Add/Remove service flag from/to mandatory services list')
parser_esrv.add_argument('--output-file', required=True, help='Output file name')
parser_esrv.add_argument('--add-flag', default=[], choices=esrv_flag_choices, action='append', help='Add flag to mandatory services list')
parser_esrv.add_argument('--remove-flag', default=[], choices=esrv_flag_choices, action='append', help='Remove flag from mandatory services list')
parser_tree = subparsers.add_parser('tree', help='Display the filesystem tree')
def write_pes(pes: ProfileElementSequence, output_file:str):
"""write the PE sequence to a file"""
print("Writing %u PEs to file '%s'..." % (len(pes.pe_list), output_file))
with open(output_file, 'wb') as f:
f.write(pes.to_der())
def do_split(pes: ProfileElementSequence, opts):
i = 0
for pe in pes.pe_list:
basename = PlPath(opts.INPUT_UPP).stem
if not pe.identification:
fname = '%s-%02u-%s.der' % (basename, i, pe.type)
else:
fname = '%s-%02u-%05u-%s.der' % (basename, i, pe.identification, pe.type)
print("writing single PE to file '%s'" % fname)
with open(os.path.join(opts.output_prefix, fname), 'wb') as outf:
outf.write(pe.to_der())
i += 1
def do_dump(pes: ProfileElementSequence, opts):
def print_all_pe(pes: ProfileElementSequence, dump_decoded:bool = False):
# iterate over each pe in the pes (using its __iter__ method)
for pe in pes:
print("="*70 + " " + pe.type)
if dump_decoded:
pp.pprint(pe.decoded)
def print_all_pe_by_type(pes: ProfileElementSequence, dump_decoded:bool = False):
# sort by PE type and show all PE within that type
for pe_type in pes.pe_by_type.keys():
print("="*70 + " " + pe_type)
for pe in pes.pe_by_type[pe_type]:
pp.pprint(pe)
if dump_decoded:
pp.pprint(pe.decoded)
def print_all_pe_by_naa(pes: ProfileElementSequence, dump_decoded:bool = False):
for naa in pes.pes_by_naa:
i = 0
for naa_instance in pes.pes_by_naa[naa]:
print("="*70 + " " + naa + str(i))
i += 1
for pe in naa_instance:
pp.pprint(pe.type)
if dump_decoded:
for d in pe.decoded:
print(" %s" % d)
if opts.mode == 'all_pe':
print_all_pe(pes, opts.dump_decoded)
elif opts.mode == 'all_pe_by_type':
print_all_pe_by_type(pes, opts.dump_decoded)
elif opts.mode == 'all_pe_by_naa':
print_all_pe_by_naa(pes, opts.dump_decoded)
def do_check(pes: ProfileElementSequence, opts):
print("Checking PE-Sequence structure...")
checker = CheckBasicStructure()
checker.check(pes)
print("All good!")
def do_extract_pe(pes: ProfileElementSequence, opts):
new_pe_list = []
for pe in pes.pe_list:
if pe.identification == opts.identification:
print("Extracting PE %s (id=%u) to file %s..." % (pe, pe.identification, opts.pe_file))
with open(opts.pe_file, 'wb') as f:
f.write(pe.to_der())
def do_remove_pe(pes: ProfileElementSequence, opts):
new_pe_list = []
for pe in pes.pe_list:
identification = pe.identification
if identification:
if identification in opts.identification:
print("Removing PE %s (id=%u) from Sequence..." % (pe, identification))
continue
if pe.type in opts.type:
print("Removing PE %s (type=%s) from Sequence..." % (pe, pe.type))
continue
new_pe_list.append(pe)
pes.pe_list = new_pe_list
pes._process_pelist()
write_pes(pes, opts.output_file)
def do_remove_naa(pes: ProfileElementSequence, opts):
if not opts.naa_type in NAAs:
raise ValueError('unsupported NAA type %s' % opts.naa_type)
naa = NAAs[opts.naa_type]
print("Removing NAAs of type '%s' from Sequence..." % opts.naa_type)
pes.remove_naas_of_type(naa)
write_pes(pes, opts.output_file)
def info_apps(pes:ProfileElementSequence):
def show_member(dictionary:Optional[dict], member:str, indent:str="\t", mandatory:bool = False, limit:bool = False):
if dictionary is None:
return
value = dictionary.get(member, None)
if value is None and mandatory == True:
print("%s%s: (missing!)" % (indent, member))
return
elif value is None:
return
if limit and len(value) > 40:
print("%s%s: '%s...%s' (%u bytes)" % (indent, member, b2h(value[:20]), b2h(value[-20:]), len(value)))
else:
print("%s%s: '%s' (%u bytes)" % (indent, member, b2h(value), len(value)))
apps = pes.pe_by_type.get('application', [])
if len(apps) == 0:
print("No Application PE present!")
return;
for app_pe in enumerate(apps):
print("Application #%u:" % app_pe[0])
print("\tloadBlock:")
load_block = app_pe[1].decoded['loadBlock']
show_member(load_block, 'loadPackageAID', "\t\t", True)
show_member(load_block, 'securityDomainAID', "\t\t")
show_member(load_block, 'nonVolatileCodeLimitC6', "\t\t")
show_member(load_block, 'volatileDataLimitC7', "\t\t")
show_member(load_block, 'nonVolatileDataLimitC8', "\t\t")
show_member(load_block, 'hashValue', "\t\t")
show_member(load_block, 'loadBlockObject', "\t\t", True, True)
for inst in enumerate(app_pe[1].decoded.get('instanceList', [])):
print("\tinstanceList[%u]:" % inst[0])
show_member(inst[1], 'applicationLoadPackageAID', "\t\t", True)
if inst[1].get('applicationLoadPackageAID', None) != load_block.get('loadPackageAID', None):
print("\t\t(applicationLoadPackageAID should be the same as loadPackageAID!)")
show_member(inst[1], 'classAID', "\t\t", True)
show_member(inst[1], 'instanceAID', "\t\t", True)
show_member(inst[1], 'extraditeSecurityDomainAID', "\t\t")
show_member(inst[1], 'applicationPrivileges', "\t\t", True)
show_member(inst[1], 'lifeCycleState', "\t\t", True)
show_member(inst[1], 'applicationSpecificParametersC9', "\t\t", True)
sys_specific_pars = inst[1].get('systemSpecificParameters', None)
if sys_specific_pars:
print("\t\tsystemSpecificParameters:")
show_member(sys_specific_pars, 'volatileMemoryQuotaC7', "\t\t\t")
show_member(sys_specific_pars, 'nonVolatileMemoryQuotaC8', "\t\t\t")
show_member(sys_specific_pars, 'globalServiceParameters', "\t\t\t")
show_member(sys_specific_pars, 'implicitSelectionParameter', "\t\t\t")
show_member(sys_specific_pars, 'volatileReservedMemory', "\t\t\t")
show_member(sys_specific_pars, 'nonVolatileReservedMemory', "\t\t\t")
show_member(sys_specific_pars, 'ts102226SIMFileAccessToolkitParameter', "\t\t\t")
additional_cl_pars = inst.get('ts102226AdditionalContactlessParameters', None)
if additional_cl_pars:
print("\t\t\tts102226AdditionalContactlessParameters:")
show_member(additional_cl_pars, 'protocolParameterData', "\t\t\t\t")
show_member(sys_specific_pars, 'userInteractionContactlessParameters', "\t\t\t")
show_member(sys_specific_pars, 'cumulativeGrantedVolatileMemory', "\t\t\t")
show_member(sys_specific_pars, 'cumulativeGrantedNonVolatileMemory', "\t\t\t")
app_pars = inst[1].get('applicationParameters', None)
if app_pars:
print("\t\tapplicationParameters:")
show_member(app_pars, 'uiccToolkitApplicationSpecificParametersField', "\t\t\t")
show_member(app_pars, 'uiccAccessApplicationSpecificParametersField', "\t\t\t")
show_member(app_pars, 'uiccAdministrativeAccessApplicationSpecificParametersField', "\t\t\t")
ctrl_ref_tp = inst[1].get('controlReferenceTemplate', None)
if ctrl_ref_tp:
print("\t\tcontrolReferenceTemplate:")
show_member(ctrl_ref_tp, 'applicationProviderIdentifier', "\t\t\t", True)
process_data = inst[1].get('processData', None)
if process_data:
print("\t\tprocessData:")
for proc in process_data:
print("\t\t\t" + b2h(proc))
def do_info(pes: ProfileElementSequence, opts):
def get_naa_count(pes: ProfileElementSequence) -> dict:
"""return a dict with naa-type (usim, isim) as key and the count of NAA instances as value."""
ret = {}
for naa_type in pes.pes_by_naa:
ret[naa_type] = len(pes.pes_by_naa[naa_type])
return ret
if opts.apps:
info_apps(pes)
return;
pe_hdr_dec = pes.pe_by_type['header'][0].decoded
print()
print("SAIP Profile Version: %u.%u" % (pe_hdr_dec['major-version'], pe_hdr_dec['minor-version']))
print("Profile Type: '%s'" % pe_hdr_dec['profileType'])
print("ICCID: %s" % b2h(pe_hdr_dec['iccid']))
print("Mandatory Services: %s" % ', '.join(pe_hdr_dec['eUICC-Mandatory-services'].keys()))
print()
naa_strs = ["%s[%u]" % (k, v) for k, v in get_naa_count(pes).items()]
print("NAAs: %s" % ', '.join(naa_strs))
for naa_type in pes.pes_by_naa:
for naa_inst in pes.pes_by_naa[naa_type]:
first_pe = naa_inst[0]
adf_name = ''
if hasattr(first_pe, 'adf_name'):
adf_name = '(' + first_pe.adf_name + ')'
print("NAA %s %s" % (first_pe.type, adf_name))
if hasattr(first_pe, 'imsi'):
print("\tIMSI: %s" % first_pe.imsi)
# applications
print()
apps = pes.pe_by_type.get('application', [])
print("Number of applications: %u" % len(apps))
for app_pe in apps:
print("App Load Package AID: %s" % b2h(app_pe.decoded['loadBlock']['loadPackageAID']))
print("\tMandated: %s" % ('mandated' in app_pe.decoded['app-Header']))
print("\tLoad Block Size: %s" % len(app_pe.decoded['loadBlock']['loadBlockObject']))
for inst in app_pe.decoded.get('instanceList', []):
print("\tInstance AID: %s" % b2h(inst['instanceAID']))
# security domains
print()
sds = pes.pe_by_type.get('securityDomain', [])
print("Number of security domains: %u" % len(sds))
for sd in sds:
print("Security domain Instance AID: %s" % b2h(sd.decoded['instance']['instanceAID']))
# FIXME: 'applicationSpecificParametersC9' parsing to figure out enabled SCP
for key in sd.keys:
print("\t%s" % repr(key))
# RFM
print()
rfms = pes.pe_by_type.get('rfm', [])
print("Number of RFM instances: %u" % len(rfms))
for rfm in rfms:
inst_aid = rfm.decoded['instanceAID']
print("RFM instanceAID: %s" % b2h(inst_aid))
print("\tMSL: 0x%02x" % rfm.decoded['minimumSecurityLevel'][0])
adf = rfm.decoded.get('adfRFMAccess', None)
if adf:
print("\tADF AID: %s" % b2h(adf['adfAID']))
tar_list = rfm.decoded.get('tarList', [inst_aid[-3:]])
for tar in tar_list:
print("\tTAR: %s" % b2h(tar))
def do_extract_apps(pes:ProfileElementSequence, opts):
apps = pes.pe_by_type.get('application', [])
for app_pe in apps:
package_aid = b2h(app_pe.decoded['loadBlock']['loadPackageAID'])
fname = os.path.join(opts.output_dir, '%s-%s.%s' % (pes.iccid, package_aid, opts.format))
print("Writing Load Package AID: %s to file %s" % (package_aid, fname))
app_pe.to_file(fname)
def do_add_app(pes:ProfileElementSequence, opts):
print("Applying applet file: '%s'..." % opts.applet_file)
app_pe = ProfileElementApplication.from_file(opts.applet_file,
opts.aid,
opts.sd_aid,
opts.non_volatile_code_limit,
opts.volatile_data_limit,
opts.non_volatile_data_limit,
opts.hash_value)
security_domain = pes.pe_by_type.get('securityDomain', [])
if len(security_domain) == 0:
print("profile package does not contain a securityDomain, please add a securityDomain PE first!")
elif len(security_domain) > 1:
print("adding an application PE to profiles with multiple securityDomain is not supported yet!")
else:
pes.insert_after_pe(security_domain[0], app_pe)
print("application PE inserted into PE Sequence after securityDomain PE AID: %s" %
b2h(security_domain[0].decoded['instance']['instanceAID']))
write_pes(pes, opts.output_file)
def do_remove_app(pes:ProfileElementSequence, opts):
apps = pes.pe_by_type.get('application', [])
for app_pe in apps:
package_aid = b2h(app_pe.decoded['loadBlock']['loadPackageAID'])
if opts.aid == package_aid:
identification = app_pe.identification
opts_remove_pe = argparse.Namespace()
opts_remove_pe.identification = [app_pe.identification]
opts_remove_pe.type = []
opts_remove_pe.output_file = opts.output_file
print("Found Load Package AID: %s, removing related PE (id=%u) from Sequence..." %
(package_aid, identification))
do_remove_pe(pes, opts_remove_pe)
return
print("Load Package AID: %s not found in PE Sequence" % opts.aid)
def do_add_app_inst(pes:ProfileElementSequence, opts):
apps = pes.pe_by_type.get('application', [])
for app_pe in apps:
package_aid = b2h(app_pe.decoded['loadBlock']['loadPackageAID'])
if opts.aid == package_aid:
print("Found Load Package AID: %s, adding new instance AID: %s to Application PE..." %
(opts.aid, opts.inst_aid))
app_pe.add_instance(opts.aid,
opts.class_aid,
opts.inst_aid,
opts.app_privileges,
opts.app_spec_pars,
opts.uicc_toolkit_app_spec_pars,
opts.uicc_access_app_spec_pars,
opts.uicc_adm_access_app_spec_pars,
opts.volatile_memory_quota,
opts.non_volatile_memory_quota,
opts.process_data)
write_pes(pes, opts.output_file)
return
print("Load Package AID: %s not found in PE Sequence" % opts.aid)
def do_remove_app_inst(pes:ProfileElementSequence, opts):
apps = pes.pe_by_type.get('application', [])
for app_pe in apps:
if opts.aid == b2h(app_pe.decoded['loadBlock']['loadPackageAID']):
print("Found Load Package AID: %s, removing instance AID: %s from Application PE..." %
(opts.aid, opts.inst_aid))
app_pe.remove_instance(opts.inst_aid)
write_pes(pes, opts.output_file)
return
print("Load Package AID: %s not found in PE Sequence" % opts.aid)
def do_edit_mand_srv_list(pes: ProfileElementSequence, opts):
header = pes.pe_by_type.get('header', [])[0]
for s in opts.add_flag:
print("Adding service '%s' to mandatory services list..." % s)
header.mandatory_service_add(s)
for s in opts.remove_flag:
if s in header.decoded['eUICC-Mandatory-services'].keys():
print("Removing service '%s' from mandatory services list..." % s)
header.mandatory_service_remove(s)
else:
print("Service '%s' not present in mandatory services list, cannot remove!" % s)
print("The following services are now set mandatory:")
for s in header.decoded['eUICC-Mandatory-services'].keys():
print("\t%s" % s)
write_pes(pes, opts.output_file)
def do_tree(pes:ProfileElementSequence, opts):
pes.mf.print_tree()
if __name__ == '__main__':
opts = parser.parse_args()
if opts.debug:
logging.basicConfig(level=logging.DEBUG)
else:
logging.basicConfig(level=logging.getLevelName(opts.loglevel))
with open(opts.INPUT_UPP, 'rb') as f:
pes = ProfileElementSequence.from_der(f.read())
print("Read %u PEs from file '%s'" % (len(pes.pe_list), opts.INPUT_UPP))
if opts.command == 'split':
do_split(pes, opts)
elif opts.command == 'dump':
do_dump(pes, opts)
elif opts.command == 'check':
do_check(pes, opts)
elif opts.command == 'extract-pe':
do_extract_pe(pes, opts)
elif opts.command == 'remove-pe':
do_remove_pe(pes, opts)
elif opts.command == 'remove-naa':
do_remove_naa(pes, opts)
elif opts.command == 'info':
do_info(pes, opts)
elif opts.command == 'extract-apps':
do_extract_apps(pes, opts)
elif opts.command == 'add-app':
do_add_app(pes, opts)
elif opts.command == 'remove-app':
do_remove_app(pes, opts)
elif opts.command == 'add-app-inst':
do_add_app_inst(pes, opts)
elif opts.command == 'remove-app-inst':
do_remove_app_inst(pes, opts)
elif opts.command == 'edit-mand-srv-list':
do_edit_mand_srv_list(pes, opts)
elif opts.command == 'tree':
do_tree(pes, opts)

31
contrib/saip-tool_example_add-app.sh Executable file
View File

@@ -0,0 +1,31 @@
#!/bin/bash
# This is an example script to illustrate how to add JAVA card applets to an existing eUICC profile package.
PYSIMPATH=../
INPATH=../smdpp-data/upp/TS48V1-A-UNIQUE.der
OUTPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-hello.der
APPPATH=./HelloSTK_09122024.cap
# Download example applet (see also https://gitea.osmocom.org/sim-card/hello-stk):
if ! [ -f $APPPATH ]; then
wget https://osmocom.org/attachments/download/8931/HelloSTK_09122024.cap
fi
# Step #1: Create the application PE and load the ijc contents from the .cap file:
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $INPATH add-app \
--output-file $OUTPATH --applet-file $APPPATH --aid 'D07002CA44'
# Step #2: Create the application instance inside the application PE created in step #1:
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $OUTPATH add-app-inst --output-file $OUTPATH \
--aid 'D07002CA44' \
--class-aid 'D07002CA44900101' \
--inst-aid 'D07002CA44900101' \
--app-privileges '00' \
--app-spec-pars '00' \
--uicc-toolkit-app-spec-pars '01001505000000000000000000000000'
# Display the contents of the resulting application PE:
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $OUTPATH info --apps
# For an explanation of --uicc-toolkit-app-spec-pars, see:
# ETSI TS 102 226, section 8.2.1.3.2.2.1

8
contrib/saip-tool_example_extract-apps.sh Executable file
View File

@@ -0,0 +1,8 @@
#!/bin/bash
# This is an example script to illustrate how to extract JAVA card applets from an existing eUICC profile package.
PYSIMPATH=../
INPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-hello.der
OUTPATH=./
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $INPATH extract-apps --output-dir ./ --format ijc

14
contrib/saip-tool_example_remove-app-inst.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
# This is an example script to illustrate how to remove a JAVA card applet instance from an application PE inside an
# existing eUICC profile package.
PYSIMPATH=../
INPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-hello.der
OUTPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-hello-no-inst.der
# Remove application PE entirely
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $INPATH remove-app-inst \
--output-file $OUTPATH --aid 'd07002ca44' --inst-aid 'd07002ca44900101'
# Display the contents of the resulting application PE:
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $OUTPATH info --apps

13
contrib/saip-tool_example_remove-app.sh Executable file
View File

@@ -0,0 +1,13 @@
#!/bin/bash
# This is an example script to illustrate how to remove a JAVA card applet from an existing eUICC profile package.
PYSIMPATH=../
INPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-hello.der
OUTPATH=../smdpp-data/upp/TS48V1-A-UNIQUE-no-hello.der
# Remove application PE entirely
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $INPATH remove-app \
--output-file $OUTPATH --aid 'D07002CA44'
# Display the contents of the resulting application PE:
PYTHONPATH=$PYSIMPATH python3 $PYSIMPATH/contrib/saip-tool.py $OUTPATH info --apps

1
contrib/sim-rest-client.py
View File

@@ -162,6 +162,7 @@ def main(argv):
parser.add_argument("-v", "--verbose", help="increase output verbosity", action='count', default=0)
parser.add_argument("-n", "--slot-nr", help="SIM slot number", type=int, default=0)
subp = parser.add_subparsers()
subp.required = True
auth_p = subp.add_parser('auth', help='UMTS AKA Authentication')
auth_p.add_argument("-c", "--count", help="Auth count", type=int, default=10)

119
contrib/sim-rest-server.py
View File

@@ -2,7 +2,7 @@
# RESTful HTTP service for performing authentication against USIM cards
#
# (C) 2021 by Harald Welte <laforge@osmocom.org>
# (C) 2021-2022 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -21,12 +21,15 @@ import json
import sys
import argparse
from klein import run, route
from klein import Klein
from pySim.transport import ApduTracer
from pySim.transport.pcsc import PcscSimLink
from pySim.commands import SimCardCommands
from pySim.cards import UsimCard
from pySim.cards import UiccCardBase
from pySim.utils import dec_iccid, dec_imsi
from pySim.ts_51_011 import EF_IMSI
from pySim.ts_102_221 import EF_ICCID
from pySim.exceptions import *
class ApduPrintTracer(ApduTracer):
@@ -35,24 +38,74 @@ class ApduPrintTracer(ApduTracer):
pass
def connect_to_card(slot_nr:int):
tp = PcscSimLink(slot_nr, apdu_tracer=ApduPrintTracer())
tp = PcscSimLink(argparse.Namespace(pcsc_dev=slot_nr), apdu_tracer=ApduPrintTracer())
tp.connect()
scc = SimCardCommands(tp)
card = UsimCard(scc)
card = UiccCardBase(scc)
# this should be part of UsimCard, but FairewavesSIM breaks with that :/
scc.cla_byte = "00"
scc.sel_ctrl = "0004"
card.read_aids()
card.select_adf_by_aid(adf='usim')
# ensure that MF is selected when we are done.
card._scc.select_file('3f00')
return tp, scc, card
class ApiError:
def __init__(self, msg:str, sw=None):
self.msg = msg
self.sw = sw
@route('/sim-auth-api/v1/slot/<int:slot>')
def auth(request, slot):
def __str__(self):
d = {'error': {'message':self.msg}}
if self.sw:
d['error']['status_word'] = self.sw
return json.dumps(d)
def set_headers(request):
request.setHeader('Content-Type', 'application/json')
class SimRestServer:
app = Klein()
@app.handle_errors(NoCardError)
def no_card_error(self, request, failure):
set_headers(request)
request.setResponseCode(410)
return str(ApiError("No SIM card inserted in slot"))
@app.handle_errors(ReaderError)
def reader_error(self, request, failure):
set_headers(request)
request.setResponseCode(404)
return str(ApiError("Reader Error: Specified SIM Slot doesn't exist"))
@app.handle_errors(ProtocolError)
def protocol_error(self, request, failure):
set_headers(request)
request.setResponseCode(500)
return str(ApiError("Protocol Error: %s" % failure.value))
@app.handle_errors(SwMatchError)
def sw_match_error(self, request, failure):
set_headers(request)
request.setResponseCode(500)
sw = failure.value.sw_actual
if sw == '9862':
return str(ApiError("Card Authentication Error - Incorrect MAC", sw))
elif sw == '6982':
return str(ApiError("Security Status not satisfied - Card PIN enabled?", sw))
else:
return str(ApiError("Card Communication Error %s" % failure.value, sw))
@app.route('/sim-auth-api/v1/slot/<int:slot>')
def auth(self, request, slot):
"""REST API endpoint for performing authentication against a USIM.
Expects a JSON body containing RAND and AUTN.
Returns a JSON body containing RES, CK, IK and Kc."""
@@ -62,61 +115,40 @@ def auth(request, slot):
rand = content['rand']
autn = content['autn']
except:
set_headers(request)
request.setResponseCode(400)
return "Malformed Request"
return str(ApiError("Malformed Request"))
try:
tp, scc, card = connect_to_card(slot)
except ReaderError:
request.setResponseCode(404)
return "Specified SIM Slot doesn't exist"
except ProtocolError:
request.setResponseCode(500)
return "Error"
except NoCardError:
request.setResponseCode(410)
return "No SIM card inserted in slot"
try:
card.select_adf_by_aid(adf='usim')
res, sw = scc.authenticate(rand, autn)
except SwMatchError as e:
request.setResponseCode(500)
return "Communication Error %s" % e
tp.disconnect()
set_headers(request)
return json.dumps(res, indent=4)
@route('/sim-info-api/v1/slot/<int:slot>')
def info(request, slot):
@app.route('/sim-info-api/v1/slot/<int:slot>')
def info(self, request, slot):
"""REST API endpoint for obtaining information about an USIM.
Expects empty body in request.
Returns a JSON body containing ICCID, IMSI."""
try:
tp, scc, card = connect_to_card(slot)
except ReaderError:
request.setResponseCode(404)
return "Specified SIM Slot doesn't exist"
except ProtocolError:
request.setResponseCode(500)
return "Error"
except NoCardError:
request.setResponseCode(410)
return "No SIM card inserted in slot"
try:
ef_iccid = EF_ICCID()
(iccid, sw) = card._scc.read_binary(ef_iccid.fid)
card.select_adf_by_aid(adf='usim')
iccid, sw = card.read_iccid()
imsi, sw = card.read_imsi()
res = {"imsi": imsi, "iccid": iccid }
except SwMatchError as e:
request.setResponseCode(500)
return "Communication Error %s" % e
ef_imsi = EF_IMSI()
(imsi, sw) = card._scc.read_binary(ef_imsi.fid)
res = {"imsi": dec_imsi(imsi), "iccid": dec_iccid(iccid) }
tp.disconnect()
set_headers(request)
return json.dumps(res, indent=4)
@@ -128,7 +160,8 @@ def main(argv):
args = parser.parse_args()
run(args.host, args.port)
srr = SimRestServer()
srr.app.run(args.host, args.port)
if __name__ == "__main__":
main(sys.argv)

240
contrib/smpp-ota-tool.py Executable file
View File

@@ -0,0 +1,240 @@
#!/usr/bin/env python3
# (C) 2026 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Harald Welte, Philipp Maier
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import argparse
import logging
import smpplib.gsm
import smpplib.client
import smpplib.consts
import time
from pySim.ota import OtaKeyset, OtaDialectSms, OtaAlgoCrypt, OtaAlgoAuth, CNTR_REQ, RC_CC_DS, POR_REQ
from pySim.utils import b2h, h2b, is_hexstr
from pathlib import Path
logger = logging.getLogger(Path(__file__).stem)
option_parser = argparse.ArgumentParser(description='Tool to send OTA SMS RFM/RAM messages via SMPP',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
option_parser.add_argument("--host", help="Host/IP of the SMPP server", default="localhost")
option_parser.add_argument("--port", help="TCP port of the SMPP server", default=2775, type=int)
option_parser.add_argument("--system-id", help="System ID to use to bind to the SMPP server", default="test")
option_parser.add_argument("--password", help="Password to use to bind to the SMPP server", default="test")
option_parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False)
algo_crypt_choices = []
algo_crypt_classes = OtaAlgoCrypt.__subclasses__()
for cls in algo_crypt_classes:
algo_crypt_choices.append(cls.enum_name)
option_parser.add_argument("--algo-crypt", choices=algo_crypt_choices, default='triple_des_cbc2',
help="OTA crypt algorithm")
algo_auth_choices = []
algo_auth_classes = OtaAlgoAuth.__subclasses__()
for cls in algo_auth_classes:
algo_auth_choices.append(cls.enum_name)
option_parser.add_argument("--algo-auth", choices=algo_auth_choices, default='triple_des_cbc2',
help="OTA auth algorithm")
option_parser.add_argument('--kic', required=True, type=is_hexstr, help='OTA key (KIC)')
option_parser.add_argument('--kic-idx', default=1, type=int, help='OTA key index (KIC)')
option_parser.add_argument('--kid', required=True, type=is_hexstr, help='OTA key (KID)')
option_parser.add_argument('--kid-idx', default=1, type=int, help='OTA key index (KID)')
option_parser.add_argument('--cntr', default=0, type=int, help='replay protection counter')
option_parser.add_argument('--tar', required=True, type=is_hexstr, help='Toolkit Application Reference')
option_parser.add_argument("--cntr-req", choices=CNTR_REQ.decmapping.values(), default='no_counter',
help="Counter requirement")
option_parser.add_argument('--no-ciphering', action='store_true', default=False, help='Disable ciphering')
option_parser.add_argument("--rc-cc-ds", choices=RC_CC_DS.decmapping.values(), default='cc',
help="message check (rc=redundency check, cc=crypt. checksum, ds=digital signature)")
option_parser.add_argument('--por-in-submit', action='store_true', default=False,
help='require PoR to be sent via SMS-SUBMIT')
option_parser.add_argument('--por-no-ciphering', action='store_true', default=False, help='Disable ciphering (PoR)')
option_parser.add_argument("--por-rc-cc-ds", choices=RC_CC_DS.decmapping.values(), default='cc',
help="PoR check (rc=redundency check, cc=crypt. checksum, ds=digital signature)")
option_parser.add_argument("--por-req", choices=POR_REQ.decmapping.values(), default='por_required',
help="Proof of Receipt requirements")
option_parser.add_argument('--src-addr', default='12', type=str, help='SMS source address (MSISDN)')
option_parser.add_argument('--dest-addr', default='23', type=str, help='SMS destination address (MSISDN)')
option_parser.add_argument('--timeout', default=10, type=int, help='Maximum response waiting time')
option_parser.add_argument('-a', '--apdu', action='append', required=True, type=is_hexstr, help='C-APDU to send')
class SmppHandler:
client = None
def __init__(self, host: str, port: int,
system_id: str, password: str,
ota_keyset: OtaKeyset, spi: dict, tar: bytes):
"""
Initialize connection to SMPP server and set static OTA SMS-TPDU ciphering parameters
Args:
host : Hostname or IPv4/IPv6 address of the SMPP server
port : TCP Port of the SMPP server
system_id: SMPP System-ID used by ESME (client) to bind
password: SMPP Password used by ESME (client) to bind
ota_keyset: OTA keyset to be used for SMS-TPDU ciphering
spi: Security Parameter Indicator (SPI) to be used for SMS-TPDU ciphering
tar: Toolkit Application Reference (TAR) of the targeted card application
"""
# Create and connect SMPP client
client = smpplib.client.Client(host, port, allow_unknown_opt_params=True)
client.set_message_sent_handler(self.message_sent_handler)
client.set_message_received_handler(self.message_received_handler)
client.connect()
client.bind_transceiver(system_id=system_id, password=password)
self.client = client
# Setup static OTA parameters
self.ota_dialect = OtaDialectSms()
self.ota_keyset = ota_keyset
self.tar = tar
self.spi = spi
def __del__(self):
if self.client:
self.client.unbind()
self.client.disconnect()
def message_received_handler(self, pdu):
if pdu.short_message:
logger.info("SMS-TPDU received: %s", b2h(pdu.short_message))
try:
dec = self.ota_dialect.decode_resp(self.ota_keyset, self.spi, pdu.short_message)
except ValueError:
# Retry to decoding with ciphering disabled (in case the card has problems to decode the SMS-TDPU
# we have sent, the response will contain an unencrypted error message)
spi = self.spi.copy()
spi['por_shall_be_ciphered'] = False
spi['por_rc_cc_ds'] = 'no_rc_cc_ds'
dec = self.ota_dialect.decode_resp(self.ota_keyset, spi, pdu.short_message)
logger.info("SMS-TPDU decoded: %s", dec)
self.response = dec
return None
def message_sent_handler(self, pdu):
logger.debug("SMS-TPDU sent: pdu_sequence=%s pdu_message_id=%s", pdu.sequence, pdu.message_id)
def transceive_sms_tpdu(self, tpdu: bytes, src_addr: str, dest_addr: str, timeout: int) -> tuple:
"""
Transceive SMS-TPDU. This method sends the SMS-TPDU to the SMPP server, and waits for a response. The method
returns when the response is received.
Args:
tpdu : short message content (plaintext)
src_addr : short message source address
dest_addr : short message destination address
timeout : timeout after which this method should give up waiting for a response
Returns:
tuple containing the response (plaintext)
"""
logger.info("SMS-TPDU sending: %s...", b2h(tpdu))
self.client.send_message(
# TODO: add parameters to switch source_addr_ton and dest_addr_ton between SMPP_TON_INTL and SMPP_NPI_ISDN
source_addr_ton=smpplib.consts.SMPP_TON_INTL,
source_addr=src_addr,
dest_addr_ton=smpplib.consts.SMPP_TON_INTL,
destination_addr=dest_addr,
short_message=tpdu,
# TODO: add parameters to set data_coding and esm_class
data_coding=smpplib.consts.SMPP_ENCODING_BINARY,
esm_class=smpplib.consts.SMPP_GSMFEAT_UDHI,
protocol_id=0x7f,
# TODO: add parameter to use registered delivery
# registered_delivery=True,
)
logger.info("SMS-TPDU sent, waiting for response...")
timestamp_sent=int(time.time())
self.response = None
while self.response is None:
self.client.poll()
if int(time.time()) - timestamp_sent > timeout:
raise ValueError("Timeout reached, no response SMS-TPDU received!")
return self.response
def transceive_apdu(self, apdu: bytes, src_addr: str, dest_addr: str, timeout: int) -> tuple[bytes, bytes]:
"""
Transceive APDU. This method wraps the given APDU into an SMS-TPDU, sends it to the SMPP server and waits for
the response. When the response is received, the last response data and the last status word is extracted from
the response and returned to the caller.
Args:
apdu : one or more concatenated APDUs
src_addr : short message source address
dest_addr : short message destination address
timeout : timeout after which this method should give up waiting for a response
Returns:
tuple containing the last response data and the last status word as byte strings
"""
logger.info("C-APDU sending: %s...", b2h(apdu))
# translate to Secured OTA RFM
secured = self.ota_dialect.encode_cmd(self.ota_keyset, self.tar, self.spi, apdu=apdu)
# add user data header
tpdu = b'\x02\x70\x00' + secured
# send via SMPP
response = self.transceive_sms_tpdu(tpdu, src_addr, dest_addr, timeout)
# Extract last_response_data and last_status_word from the response
sw = None
resp = None
for container in response:
if container:
container_dict = dict(container)
resp = container_dict.get('last_response_data')
sw = container_dict.get('last_status_word')
if resp is None:
raise ValueError("Response does not contain any last_response_data, no R-APDU received!")
if sw is None:
raise ValueError("Response does not contain any last_status_word, no R-APDU received!")
logger.info("R-APDU received: %s %s", resp, sw)
return h2b(resp), h2b(sw)
if __name__ == '__main__':
opts = option_parser.parse_args()
logging.basicConfig(level=logging.DEBUG if opts.verbose else logging.INFO,
format='%(asctime)s %(levelname)s %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
if opts.kic_idx != opts.kid_idx:
logger.warning("KIC index (%s) and KID index (%s) are different (security violation, card should reject message)",
opts.kic_idx, opts.kid_idx)
ota_keyset = OtaKeyset(algo_crypt=opts.algo_crypt,
kic_idx=opts.kic_idx,
kic=h2b(opts.kic),
algo_auth=opts.algo_auth,
kid_idx=opts.kid_idx,
kid=h2b(opts.kid),
cntr=opts.cntr)
spi = {'counter' : opts.cntr_req,
'ciphering' : not opts.no_ciphering,
'rc_cc_ds': opts.rc_cc_ds,
'por_in_submit': opts.por_in_submit,
'por_shall_be_ciphered': not opts.por_no_ciphering,
'por_rc_cc_ds': opts.por_rc_cc_ds,
'por': opts.por_req}
apdu = h2b("".join(opts.apdu))
smpp_handler = SmppHandler(opts.host, opts.port, opts.system_id, opts.password, ota_keyset, spi, h2b(opts.tar))
resp, sw = smpp_handler.transceive_apdu(apdu, opts.src_addr, opts.dest_addr, opts.timeout)
print("%s %s" % (b2h(resp), b2h(sw)))

52
contrib/suci-keytool.py Executable file
View File

@@ -0,0 +1,52 @@
#!/usr/bin/env python3
# small utility program to deal with 5G SUCI key material, at least for the ECIES Protection Scheme
# Profile A (curve25519) and B (secp256r1)
# (C) 2024 by Harald Welte <laforge@osmocom.org>
# SPDX-License-Identifier: GPL-2.0+
import argparse
from osmocom.utils import b2h
from Cryptodome.PublicKey import ECC
# if used with pycryptodome < v3.21.0 you will get the following error when using curve25519:
# "Cryptodome.PublicKey.ECC.UnsupportedEccFeature: Unsupported ECC purpose (OID: 1.3.101.110)"
def gen_key(opts):
# FIXME: avoid overwriting key files
mykey = ECC.generate(curve=opts.curve)
data = mykey.export_key(format='PEM')
with open(opts.key_file, "wt") as f:
f.write(data)
def dump_pkey(opts):
#with open("curve25519-1.key", "r") as f:
with open(opts.key_file, "r") as f:
data = f.read()
mykey = ECC.import_key(data)
der = mykey.public_key().export_key(format='raw', compress=opts.compressed)
print(b2h(der))
arg_parser = argparse.ArgumentParser(description="""Generate or export SUCI keys for 5G SA networks""")
arg_parser.add_argument('--key-file', help='The key file to use', required=True)
subparsers = arg_parser.add_subparsers(dest='command', help="The command to perform", required=True)
parser_genkey = subparsers.add_parser('generate-key', help='Generate a new key pair')
parser_genkey.add_argument('--curve', help='The ECC curve to use', choices=['secp256r1','curve25519'], required=True)
parser_dump_pkey = subparsers.add_parser('dump-pub-key', help='Dump the public key')
parser_dump_pkey.add_argument('--compressed', help='Use point compression', action='store_true')
if __name__ == '__main__':
opts = arg_parser.parse_args()
if opts.command == 'generate-key':
gen_key(opts)
elif opts.command == 'dump-pub-key':
dump_pkey(opts)

43
contrib/unber.py Executable file
View File

@@ -0,0 +1,43 @@
#!/usr/bin/env python3
# A more useful version of the 'unber' tool provided with asn1c:
# Give a hierarchical decode of BER/DER-encoded ASN.1 TLVs
import sys
import argparse
from osmocom.utils import b2h, h2b
from osmocom.tlv import bertlv_parse_one, bertlv_encode_tag
def process_one_level(content: bytes, indent: int):
remainder = content
while len(remainder):
tdict, l, v, remainder = bertlv_parse_one(remainder)
#print(tdict)
rawtag = bertlv_encode_tag(tdict)
if tdict['constructed']:
print("%s%s l=%d" % (indent*" ", b2h(rawtag), l))
process_one_level(v, indent + 1)
else:
print("%s%s l=%d %s" % (indent*" ", b2h(rawtag), l, b2h(v)))
option_parser = argparse.ArgumentParser(description='BER/DER data dumper')
group = option_parser.add_mutually_exclusive_group(required=True)
group.add_argument('--file', help='Input file')
group.add_argument('--hex', help='Input hexstring')
if __name__ == '__main__':
opts = option_parser.parse_args()
if opts.file:
with open(opts.file, 'rb') as f:
content = f.read()
elif opts.hex:
content = h2b(opts.hex)
else:
# avoid pylint "(possibly-used-before-assignment)" below
sys.exit(2)
process_one_level(content, 0)

25
docs/Makefile
View File

@@ -4,16 +4,22 @@
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SPHINXBUILD ?= python3 -m sphinx.cmd.build
SOURCEDIR = .
BUILDDIR = _build
# for osmo-gsm-manuals
OSMO_GSM_MANUALS_DIR=$(shell pkg-config osmo-gsm-manuals --variable=osmogsmmanualsdir 2>/dev/null)
OSMO_GSM_MANUALS_DIR ?= $(shell pkg-config osmo-gsm-manuals --variable=osmogsmmanualsdir 2>/dev/null)
OSMO_REPOSITORY = "pysim"
UPLOAD_FILES = $(BUILDDIR)/latex/osmopysim-usermanual.pdf
CLEAN_FILES = $(UPLOAD_FILES)
# Copy variables from Makefile.common.inc that are used in publish-html,
# as Makefile.common.inc must be included after publish-html
PUBLISH_REF ?= master
PUBLISH_TEMPDIR = _publish_tmpdir
SSH_COMMAND = ssh -o 'UserKnownHostsFile=$(OSMO_GSM_MANUALS_DIR)/build/known_hosts' -p 48
# Put it first so that "make" without argument is like "make help".
.PHONY: help
help:
@@ -23,7 +29,16 @@ $(BUILDDIR)/latex/pysim.pdf: latexpdf
@/bin/true
publish-html: html
rsync -avz -e "ssh -o 'UserKnownHostsFile=$(OSMO_GSM_MANUALS_DIR)/build/known_hosts' -p 48" $(BUILDDIR)/html/ docs@ftp.osmocom.org:web-files/latest/pysim/
rm -rf "$(PUBLISH_TEMPDIR)"
mkdir -p "$(PUBLISH_TEMPDIR)/pysim/$(PUBLISH_REF)"
cp -r "$(BUILDDIR)"/html "$(PUBLISH_TEMPDIR)/pysim/$(PUBLISH_REF)"
cd "$(PUBLISH_TEMPDIR)" && \
rsync \
-avzR \
-e "$(SSH_COMMAND)" \
"pysim" \
docs@ftp.osmocom.org:web-files/
rm -rf "$(PUBLISH_TEMPDIR)"
# put this before the catch-all below
include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.common.inc
@@ -32,4 +47,6 @@ include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.common.inc
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%:
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
@if [ "$@" != "shrink" ]; then \
$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O); \
fi

103
docs/cap-tutorial.rst Normal file
View File

@@ -0,0 +1,103 @@
Guide: Installing JAVA-card applets
===================================
Almost all modern-day UICC cards have some form of JAVA-card / Sim-Toolkit support, which allows the installation
of customer specific JAVA-card applets. The installation of JAVA-card applets is usually done via the standardized
GlobalPlatform (GPC_SPE_034) ISD (Issuer Security Domain) application interface during the card provisioning process.
(it is also possible to load JAVA-card applets in field via OTA-SMS, but that is beyond the scope of this guide). In
this guide we will go through the individual steps that are required to load JAVA-card applet onto an UICC card.
Preparation
~~~~~~~~~~~
In this example we will install the CAP file HelloSTK_09122024.cap [1] on an sysmoISIM-SJA2 card. Since the interface
is standardized, the exact card model does not matter.
The example applet makes use of the STK (Sim-Toolkit), so we must supply STK installation parameters. Those
parameters are supplied in the form of a hexstring and should be provided by the applet manufacturer. The available
parameters and their exact encoding is specified in ETSI TS 102 226, section 8.2.1.3.2.1. The installation of
HelloSTK_09122024.cap [1], will require the following STK installation parameters: "010001001505000000000000000000000000"
During the installation, we also have to set a memory quota for the volatile and for the non volatile card memory.
Those values also should be provided by the applet manufacturer. In this example, we will allow 255 bytes of volatile
memory and 255 bytes of non volatile memory to be consumed by the applet.
To install JAVA-card applets, one must be in the possession of the key material belonging to the card. The keys are
usually provided by the card manufacturer. The following example will use the following keyset:
+---------+----------------------------------+
| Keyname | Keyvalue |
+=========+==================================+
| DEK/KIK | 5524F4BECFE96FB63FC29D6BAAC6058B |
+---------+----------------------------------+
| ENC/KIC | 542C37A6043679F2F9F71116418B1CD5 |
+---------+----------------------------------+
| MAC/KID | 34F11BAC8E5390B57F4E601372339E3C |
+---------+----------------------------------+
[1] https://osmocom.org/projects/cellular-infrastructure/wiki/HelloSTK
Applet Installation
~~~~~~~~~~~~~~~~~~~
To prepare the installation, a secure channel to the ISD must be established first:
::
pySIM-shell (00:MF)> select ADF.ISD
{
"application_id": "a000000003000000",
"proprietary_data": {
"maximum_length_of_data_field_in_command_message": 255
}
}
pySIM-shell (00:MF/ADF.ISD)> establish_scp02 --key-dek 5524F4BECFE96FB63FC29D6BAAC6058B --key-enc 542C37A6043679F2F9F71116418B1CD5 --key-mac 34F11BAC8E5390B57F4E601372339E3C --security-level 1
Successfully established a SCP02[01] secure channel
.. warning:: In case you get an "EXCEPTION of type 'ValueError' occurred with message: card cryptogram doesn't match" error message, it is very likely that there is a problem with the key material. The card may lock the ISD access after a certain amount of failed tries. Carefully check the key material any try again.
When the secure channel is established, we are ready to install the applet. The installation normally is a multi step
procedure, where the loading of an executable load file is announced first, then loaded and then installed in a final
step. The pySim-shell command ``install_cap`` automatically takes care of those three steps.
::
pySIM-shell (SCP02[01]:00:MF/ADF.ISD)> install_cap /home/user/HelloSTK_09122024.cap --install-parameters-non-volatile-memory-quota 255 --install-parameters-volatile-memory-quota 255 --install-parameters-stk 010001001505000000000000000000000000
loading cap file: /home/user/HelloSTK_09122024.cap ...
parameters:
security-domain-aid: a000000003000000
load-file: 569 bytes
load-file-aid: d07002ca44
module-aid: d07002ca44900101
application-aid: d07002ca44900101
install-parameters: c900ef1cc80200ffc70200ffca12010001001505000000000000000000000000
step #1: install for load...
step #2: load...
Loaded a total of 573 bytes in 3 blocks. Don't forget install_for_install (and make selectable) now!
step #3: install_for_install (and make selectable)...
done.
The applet is now installed on the card. We can now quit pySim-shell and remove the card from the reader and test the
applet in a mobile phone. There should be a new STK application with one menu entry shown, that will greet the user
when pressed.
Applet Removal
~~~~~~~~~~~~~~
To remove the applet, we must establish a secure channel to the ISD (see above). Then we can delete the applet using the
``delete_card_content`` command.
::
pySIM-shell (SCP02[01]:00:MF/ADF.ISD)> delete_card_content D07002CA44 --delete-related-objects
The parameter "D07002CA44" is the load-file-AID of the applet. The load-file-AID is encoded in the .cap file and also
displayed during the installation process. It is also important to note that when the applet is installed, it cannot
be installed (under the same AID) again until it is removed.

342
docs/card-key-provider.rst Normal file
View File

@@ -0,0 +1,342 @@
Retrieving card-individual keys via CardKeyProvider
===================================================
When working with a batch of cards, or more than one card in general, it
is a lot of effort to manually retrieve the card-specific PIN (like
ADM1) or key material (like SCP02/SCP03 keys).
To increase productivity in that regard, pySim has a concept called the
`CardKeyProvider`. This is a generic mechanism by which different parts
of the pySim[-shell] code can programmatically request card-specific key material
from some data source (*provider*).
For example, when you want to verify the ADM1 PIN using the `verify_adm`
command without providing an ADM1 value yourself, pySim-shell will
request the ADM1 value for the ICCID of the card via the
CardKeyProvider.
There can in theory be multiple different CardKeyProviders. You can for
example develop your own CardKeyProvider that queries some kind of
database for the key material, or that uses a key derivation function to
derive card-specific key material from a global master key.
pySim already includes two CardKeyProvider implementations. One to retrieve
key material from a CSV file (`CardKeyProviderCsv`) and a second one that allows
to retrieve the key material from a PostgreSQL database (`CardKeyProviderPgsql`).
Both implementations equally implement a column encryption scheme that allows
to protect sensitive columns using a *transport key*
The CardKeyProviderCsv
----------------------
The `CardKeyProviderCsv` allows you to retrieve card-individual key
material from a CSV (comma separated value) file that is accessible to pySim.
The CSV file must have the expected column names, for example `ICCID`
and `ADM1` in case you would like to use that CSV to obtain the
card-specific ADM1 PIN when using the `verify_adm` command.
You can specify the CSV file to use via the `--csv` command-line option
of pySim-shell. If you do not specify a CSV file, pySim will attempt to
open a CSV file from the default location at
`~/.osmocom/pysim/card_data.csv`, and use that, if it exists.
The `CardKeyProviderCsv` is suitable to manage small amounts of key material
locally. However, if your card inventory is very large and the key material
must be made available on multiple sites, the `CardKeyProviderPgsql` is the
better option.
The CardKeyProviderPgsql
------------------------
With the `CardKeyProviderPgsql` you can use a PostgreSQL database as storage
medium. The implementation comes with a CSV importer tool that consumes the
same CSV files you would normally use with the `CardKeyProviderCsv`, so you
can just use your existing CSV files and import them into the database.
Requirements
^^^^^^^^^^^^
The `CardKeyProviderPgsql` uses the `Psycopg` PostgreSQL database adapter
(https://www.psycopg.org). `Psycopg` is not part of the default requirements
of pySim-shell and must be installed separately. `Psycopg` is available as
Python package under the name `psycopg2-binary`.
Setting up the database
^^^^^^^^^^^^^^^^^^^^^^^
From the perspective of the database, the `CardKeyProviderPgsql` has only
minimal requirements. You do not have to create any tables in advance. An empty
database and at least one user that may create, alter and insert into tables is
sufficient. However, for increased reliability and as a protection against
incorrect operation, the `CardKeyProviderPgsql` supports a hierarchical model
with three users (or roles):
* **admin**:
This should be the owner of the database. It is intended to be used for
administrative tasks like adding new tables or adding new columns to existing
tables. This user should not be used to insert new data into tables or to access
data from within pySim-shell using the `CardKeyProviderPgsql`
* **importer**:
This user is used when feeding new data into an existing table. It should only
be able to insert new rows into existing tables. It should not be used for
administrative tasks or to access data from within pySim-shell using the
`CardKeyProviderPgsql`
* **reader**:
To access data from within pySim shell using the `CardKeyProviderPgsql` the
reader user is the correct one to use. This user should have no write access
to the database or any of the tables.
Creating a config file
^^^^^^^^^^^^^^^^^^^^^^
The default location for the config file is `~/.osmocom/pysim/card_data_pgsql.cfg`
The file uses `yaml` syntax and should look like the example below:
::
host: "127.0.0.1"
db_name: "my_database"
table_names:
- "uicc_keys"
- "euicc_keys"
db_users:
admin:
name: "my_admin_user"
pass: "my_admin_password"
importer:
name: "my_importer_user"
pass: "my_importer_password"
reader:
name: "my_reader_user"
pass: "my_reader_password"
This file is used by pySim-shell and by the importer tool. Both expect the file
in the aforementioned location. In case you want to store the file in a
different location you may use the `--pgsql` commandline option to provide a
custom config file path.
The hostname and the database name for the PostgreSQL database is set with the
`host` and `db_name` fields. The field `db_users` sets the user names and
passwords for each of the aforementioned users (or roles). In case only a single
admin user is used, all three entries may be populated with the same user name
and password (not recommended)
The field `table_names` sets the tables that the `CardKeyProviderPgsql` shall
use to query to locate card key data. You can set up as many tables as you
want, `CardKeyProviderPgsql` will query them in order, one by one until a
matching entry is found.
NOTE: In case you do not want to disclose the admin and the importer credentials
to pySim-shell you may remove those lines. pySim-shell will only require the
`reader` entry under `db_users`.
Using the Importer
^^^^^^^^^^^^^^^^^^
Before data can be imported, you must first create a database table. Tables
are created with the provided importer tool, which can be found under
`contrib/csv-to-pgsql.py`. This tool is used to create the database table and
read the data from the provided CSV file into the database.
As mentioned before, all CSV file formats that work with `CardKeyProviderCsv`
may be used. To demonstrate how the import process works, let's assume you want
to import a CSV file format that looks like the following example. Let's also
assume that you didn't get the Global Platform keys from your card vendor for
this batch of UICC cards, so your CSV file lacks the columns for those fields.
::
"id","imsi","iccid","acc","pin1","puk1","pin2","puk2","ki","opc","adm1"
"card1","999700000000001","8900000000000000001","0001","1111","11111111","0101","01010101","11111111111111111111111111111111","11111111111111111111111111111111","11111111"
"card2","999700000000002","8900000000000000002","0002","2222","22222222","0202","02020202","22222222222222222222222222222222","22222222222222222222222222222222","22222222"
"card3","999700000000003","8900000000000000003","0003","3333","22222222","0303","03030303","33333333333333333333333333333333","33333333333333333333333333333333","33333333"
Since this is your first import, the database still lacks the table. To
instruct the importer to create a new table, you may use the `--create-table`
option. You also have to pick an appropriate name for the table. Any name may
be chosen as long as it contains the string `uicc_keys` or `euicc_keys`,
depending on the type of data (`UICC` or `eUICC`) you intend to store in the
table. The creation of the table is an administrative task and can only be done
with the `admin` user. The `admin` user is selected using the `--admin` switch.
::
$ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_01.csv --table-name uicc_keys --create-table --admin
INFO: CSV file: ./csv-to-pgsql_example_01.csv
INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1']
INFO: Using config file: /home/user/.osmocom/pysim/card_data_pgsql.cfg
INFO: Database host: 127.0.0.1
INFO: Database name: my_database
INFO: Database user: my_admin_user
INFO: New database table created: uicc_keys
INFO: Database table: uicc_keys
INFO: Database table columns: ['ICCID', 'IMSI']
INFO: Adding missing columns: ['PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC']
INFO: Changes to table uicc_keys committed!
The importer has created a new table with the name `uicc_keys`. The table is
now ready to be filled with data.
::
$ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_01.csv --table-name uicc_keys
INFO: CSV file: ./csv-to-pgsql_example_01.csv
INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1']
INFO: Using config file: /home/user/.osmocom/pysim/card_data_pgsql.cfg
INFO: Database host: 127.0.0.1
INFO: Database name: my_database
INFO: Database user: my_importer_user
INFO: Database table: uicc_keys
INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC']
INFO: CSV file import done, 3 rows imported
INFO: Changes to table uicc_keys committed!
A quick `SELECT * FROM uicc_keys;` at the PostgreSQL console should now display
the contents of the CSV file you have fed into the importer.
Let's now assume that with your next batch of UICC cards your vendor includes
the Global Platform keys so your CSV format changes. It may now look like this:
::
"id","imsi","iccid","acc","pin1","puk1","pin2","puk2","ki","opc","adm1","scp02_dek_1","scp02_enc_1","scp02_mac_1"
"card4","999700000000004","8900000000000000004","0004","4444","44444444","0404","04040404","44444444444444444444444444444444","44444444444444444444444444444444","44444444","44444444444444444444444444444444","44444444444444444444444444444444","44444444444444444444444444444444"
"card5","999700000000005","8900000000000000005","0005","4444","55555555","0505","05050505","55555555555555555555555555555555","55555555555555555555555555555555","55555555","55555555555555555555555555555555","55555555555555555555555555555555","55555555555555555555555555555555"
"card6","999700000000006","8900000000000000006","0006","4444","66666666","0606","06060606","66666666666666666666666666666666","66666666666666666666666666666666","66666666","66666666666666666666666666666666","66666666666666666666666666666666","66666666666666666666666666666666"
When importing data from an updated CSV format the database table also has
to be updated. This is done using the `--update-columns` switch. Like when
creating new tables, this operation also requires admin privileges, so the
`--admin` switch is required again.
::
$ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_02.csv --table-name uicc_keys --update-columns --admin
INFO: CSV file: ./csv-to-pgsql_example_02.csv
INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1', 'SCP02_DEK_1', 'SCP02_ENC_1', 'SCP02_MAC_1']
INFO: Using config file: /home/user/.osmocom/pysim/card_data_pgsql.cfg
INFO: Database host: 127.0.0.1
INFO: Database name: my_database
INFO: Database user: my_admin_user
INFO: Database table: uicc_keys
INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC']
INFO: Adding missing columns: ['SCP02_ENC_1', 'SCP02_MAC_1', 'SCP02_DEK_1']
INFO: Changes to table uicc_keys committed!
When the new table columns are added, the import may be continued like the
first one:
::
$ PYTHONPATH=../ ./csv-to-pgsql.py --csv ./csv-to-pgsql_example_02.csv --table-name uicc_keys
INFO: CSV file: ./csv-to-pgsql_example_02.csv
INFO: CSV file columns: ['ID', 'IMSI', 'ICCID', 'ACC', 'PIN1', 'PUK1', 'PIN2', 'PUK2', 'KI', 'OPC', 'ADM1', 'SCP02_DEK_1', 'SCP02_ENC_1', 'SCP02_MAC_1']
INFO: Using config file: /home/user/.osmocom/pysim/card_data_pgsql.cfg
INFO: Database host: 127.0.0.1
INFO: Database name: my_database
INFO: Database user: my_importer_user
INFO: Database table: uicc_keys
INFO: Database table columns: ['ICCID', 'IMSI', 'PIN2', 'PUK1', 'PUK2', 'ACC', 'ID', 'PIN1', 'ADM1', 'KI', 'OPC', 'SCP02_ENC_1', 'SCP02_MAC_1', 'SCP02_DEK_1']
INFO: CSV file import done, 3 rows imported
INFO: Changes to table uicc_keys committed!
On the PostgreSQL console a `SELECT * FROM uicc_keys;` should now show the
imported data with the added columns. All important data should now also be
available from within pySim-shell via the `CardKeyProviderPgsql`.
Column-Level CSV encryption
---------------------------
pySim supports column-level CSV encryption. This feature will make sure
that your key material is not stored in plaintext in the CSV file (or
database).
The encryption mechanism uses AES in CBC mode. You can use any key
length permitted by AES (128/192/256 bit).
Following GSMA FS.28, the encryption works on column level. This means
different columns can be decrypted using different key material. This
means that leakage of a column encryption key for one column or set of
columns (like a specific security domain) does not compromise various
other keys that might be stored in other columns.
You can specify column-level decryption keys using the
`--csv-column-key` command line argument. The syntax is
`FIELD:AES_KEY_HEX`, for example:
`pySim-shell.py --csv-column-key SCP03_ENC_ISDR:000102030405060708090a0b0c0d0e0f`
In order to avoid having to repeat the column key for each and every
column of a group of keys within a keyset, there are pre-defined column
group aliases, which will make sure that the specified key will be used
by all columns of the set:
* `UICC_SCP02` is a group alias for `UICC_SCP02_KIC1`, `UICC_SCP02_KID1`, `UICC_SCP02_KIK1`
* `UICC_SCP03` is a group alias for `UICC_SCP03_KIC1`, `UICC_SCP03_KID1`, `UICC_SCP03_KIK1`
* `SCP03_ECASD` is a group alias for `SCP03_ENC_ECASD`, `SCP03_MAC_ECASD`, `SCP03_DEK_ECASD`
* `SCP03_ISDA` is a group alias for `SCP03_ENC_ISDA`, `SCP03_MAC_ISDA`, `SCP03_DEK_ISDA`
* `SCP03_ISDR` is a group alias for `SCP03_ENC_ISDR`, `SCP03_MAC_ISDR`, `SCP03_DEK_ISDR`
NOTE: When using `CardKeyProviderPqsl`, the input CSV files must be encrypted
before import.
Field naming
------------
* For look-up of UICC/SIM/USIM/ISIM or eSIM profile specific key
material, pySim uses the `ICCID` field as lookup key.
* For look-up of eUICC specific key material (like SCP03 keys for the
ISD-R, ECASD), pySim uses the `EID` field as lookup key.
As soon as the CardKeyProvider finds a line (row) in your CSV file
(or database) where the ICCID or EID match, it looks for the column containing
the requested data.
ADM PIN
^^^^^^^
The `verify_adm` command will attempt to look up the `ADM1` column
indexed by the ICCID of the SIM/UICC.
SCP02 / SCP03
^^^^^^^^^^^^^
SCP02 and SCP03 each use key triplets consisting of ENC, MAC and DEK
keys. For more details, see the applicable GlobalPlatform
specifications.
If you do not want to manually enter the key material for each specific
card as arguments to the `establish_scp02` or `establish_scp03`
commands, you can make use of the `--key-provider-suffix` option. pySim
uses this suffix to compose the column names for the CardKeyProvider as
follows.
* `SCP02_ENC_` + suffix for the SCP02 ciphering key
* `SCP02_MAC_` + suffix for the SCP02 MAC key
* `SCP02_DEK_` + suffix for the SCP02 DEK key
* `SCP03_ENC_` + suffix for the SCP03 ciphering key
* `SCP03_MAC_` + suffix for the SCP03 MAC key
* `SCP03_DEK_` + suffix for the SCP03 DEK key
So for example, if you are using a command like `establish_scp03
--key-provider-suffix ISDR`, then the column names for the key material
look-up are `SCP03_ENC_ISDR`, `SCP03_MAC_ISDR` and `SCP03_DEK_ISDR`,
respectively.
The identifier used for look-up is determined by the definition of the
Security Domain. For example, the eUICC ISD-R and ECASD will use the EID
of the eUICC. On the other hand, the ISD-P of an eSIM or the ISD of an
UICC will use the ICCID.

38
docs/conf.py
View File

@@ -13,14 +13,23 @@
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
sys.path.insert(0, os.path.abspath('.')) # for local extensions (pysim_fs_sphinx, ...)
# -- Project information -----------------------------------------------------
project = 'osmopysim-usermanual'
copyright = '2009-2021 by Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle'
author = 'Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle'
copyright = '2009-2025 by Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta'
author = 'Sylvain Munaut, Harald Welte, Philipp Maier, Supreeth Herle, Merlin Chlosta'
# PDF: Avoid that the authors list exceeds the page by inserting '\and'
# manually as line break (https://github.com/sphinx-doc/sphinx/issues/6875)
latex_elements = {
"maketitle":
r"""\author{Sylvain Munaut, Harald Welte, Philipp Maier, \and Supreeth Herle, Merlin Chlosta}
\sphinxmaketitle
"""
}
# -- General configuration ---------------------------------------------------
@@ -31,7 +40,8 @@ extensions = [
"sphinx.ext.autodoc",
"sphinxarg.ext",
"sphinx.ext.autosectionlabel",
"sphinx.ext.napoleon"
"sphinx.ext.napoleon",
"pysim_fs_sphinx",
]
# Add any paths that contain templates here, relative to this directory.
@@ -56,3 +66,25 @@ html_theme = 'alabaster'
html_static_path = ['_static']
autoclass_content = 'both'
# Mock optional server-side deps of es2p and http_json_api/es9p,
# so that autodoc can import and document those modules.
autodoc_mock_imports = ['klein', 'twisted']
# Workaround for duplicate label warnings:
# https://github.com/sphinx-doc/sphinx-argparse/issues/14
#
# sphinxarg.ext generates generic sub-headings ("Named arguments",
# "Positional arguments", "Sub-commands", "General options", ...) for every
# argparse command/tool. These repeat across many files and trigger tons
# of autosectionlabel duplicate-label warnings - suppress them.
autosectionlabel_maxdepth = 3
suppress_warnings = [
'autosectionlabel.filesystem',
'autosectionlabel.saip-tool',
'autosectionlabel.shell',
'autosectionlabel.smpp2sim',
'autosectionlabel.smpp-ota-tool',
'autosectionlabel.suci-keytool',
'autosectionlabel.trace',
]

12
docs/index.rst
View File

@@ -31,15 +31,25 @@ pySim consists of several parts:
* a python :ref:`library<pySim library>` containing plenty of objects and methods that can be used for
writing custom programs interfacing with SIM cards.
* the [new] :ref:`interactive pySim-shell command line program<pySim-shell>`
* the [new] :ref:`pySim-trace APDU trace decoder<pySim-trace>`
* the [legacy] :ref:`pySim-prog and pySim-read tools<Legacy tools>`
.. toctree::
:maxdepth: 2
:maxdepth: 3
:caption: Contents:
shell
filesystem
trace
legacy
smpp2sim
library
library-esim
osmo-smdpp
sim-rest
suci-keytool
saip-tool
smpp-ota-tool
Indices and tables

195
docs/legacy.rst
View File

@@ -1,17 +1,20 @@
Legacy tools
Legacy tools
============
*legacy tools* are the classic ``pySim-prog`` and ``pySim-read`` programs that
existed long before ``pySim-shell``.
These days, it is highly recommended to use ``pySim-shell`` instead of these
legacy tools.
pySim-prog
----------
``pySim-prog`` was the first part of the pySim software suite. It started as
a tool to write ICCID, IMSI, MSISDN and Ki to very simplistic SIM cards, and
was later extended to a variety of other cards. As the number of features supported
became no longer bearable to express with command-line arguments, `pySim-shell` was
created.
``pySim-prog`` was the first part of the pySim software suite. It started as a
tool to write ICCID, IMSI, MSISDN and Ki to very simplistic SIM cards, and was
later extended to a variety of other cards. As the number of features supported
became no longer bearable to express with command-line arguments, `pySim-shell`
was created.
Basic use cases can still use `pySim-prog`.
@@ -19,31 +22,180 @@ Program customizable SIMs
~~~~~~~~~~~~~~~~~~~~~~~~~
Two modes are possible:
- one where you specify every parameter manually :
- one where the user specifies every parameter manually:
``./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -i <IMSI> -s <ICCID>``
This is the most common way to use ``pySim-prog``. The user will specify all relevant parameters directly via the
commandline. A typical commandline would look like this:
``pySim-prog.py -p <pcsc_reader> --ki <ki_value> --opc <opc_value> --mcc <mcc_value> --mnc <mnc_value>
--country <country_code> --imsi <imsi_value> --iccid <iccid_value> --pin-adm <adm_pin>``
Please note, that this already lengthy commandline still only contains the most common card parameters. For a full
list of all possible parameters, use the ``--help`` option of ``pySim-prog``. It is also important to mention
that not all parameters are supported by all card types. In particular, very simple programmable SIM cards will only
support a very basic set of parameters, such as MCC, MNC, IMSI and KI values.
- one where the parameters are generated from a minimal set:
It is also possible to leave the generation of certain parameters to ``pySim-prog``. This is in particular helpful
when a large number of cards should be initialized with randomly generated key material.
``pySim-prog.py -p <pcsc_reader> --mcc <mcc_value> --mnc <mnc_value> --secret <random_secret> --num <card_number> --pin-adm <adm_pin>``
The parameter ``--secret`` specifies a random seed that is used to generate the card individual parameters. (IMSI).
The secret should contain enough randomness to avoid conflicts. It is also recommended to store the secret safely,
in case cards have to be re-generated or the current card batch has to be extended later. For security reasons, the
key material, which is also card individual, will not be derived from the random seed. Instead a new random set of
Ki and OPc will be generated during each programming cycle. This means fresh keys are generated, even when the
``--num`` remains unchanged.
The parameter ``--num`` specifies a card individual number. This number will be managed into the random seed so that
it serves as an identifier for a particular set of randomly generated parameters.
In the example above the parameters ``--mcc``, and ``--mnc`` are specified as well, since they identify the GSM
network where the cards should operate in, it is absolutely required to keep them static. ``pySim-prog`` will use
those parameters to generate a valid IMSI that thas the specified MCC/MNC at the beginning and a random tail.
Specifying the card type:
``pySim-prog`` usually autodetects the card type. In case auto detection does not work, it is possible to specify
the parameter ``--type``. The following card types are supported:
* Fairwaves-SIM
* fakemagicsim
* gialersim
* grcardsim
* magicsim
* OpenCells-SIM
* supersim
* sysmoISIM-SJA2
* sysmoISIM-SJA5
* sysmosim-gr1
* sysmoSIM-GR2
* sysmoUSIM-SJS1
* Wavemobile-SIM
Specifying the card reader:
It is most common to use ``pySim-prog`` together with a PCSC reader. The PCSC reader number is specified via the
``--pcsc-device`` or ``-p`` option. However, other reader types (such as serial readers and modems) are supported. Use
the ``--help`` option of ``pySim-prog`` for more information.
- one where they are generated from some minimal set :
Card programming using CSV files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``./pySim-prog.py -n 26C3 -c 49 -x 262 -y 42 -z <random_string_of_choice> -j <card_num>``
To simplify the card programming process, ``pySim-prog`` also allows to read
the card parameters from a CSV file. When a CSV file is used as input, the
user does not have to craft an individual commandline for each card. Instead
all card related parameters are automatically drawn from the CSV file.
With <random_string_of_choice> and <card_num>, the soft will generate
'predictable' IMSI and ICCID, so make sure you choose them so as not to
conflict with anyone. (for eg. your name as <random_string_of_choice> and
0 1 2 ... for <card num>).
A CSV files may hold rows for multiple (hundreds or even thousands) of
cards. ``pySim-prog`` is able to identify the rows either by ICCID
(recommended as ICCIDs are normally not changed) or IMSI.
You also need to enter some parameters to select the device :
-t TYPE : type of card (supersim, magicsim, fakemagicsim or try 'auto')
-d DEV : Serial port device (default /dev/ttyUSB0)
-b BAUD : Baudrate (default 9600)
The CSV file format is a flexible format with mandatory and optional columns,
here the same rules as for the commandline parameters apply. The column names
match the command line options. The CSV file may also contain columns that are
unknown to pySim-prog, such as inventory numbers, nicknames or parameters that
are unrelated to the card programming process. ``pySim-prog`` will silently
ignore all unknown columns.
A CSV file may contain the following columns:
* name
* iccid (typically used as key)
* mcc
* mnc
* imsi (may be used as key, but not recommended)
* smsp
* ki
* opc
* acc
* pin_adm, adm1 or pin_adm_hex (must be present)
* msisdn
* epdgid
* epdgSelection
* pcscf
* ims_hdomain
* impi
* impu
* opmode
* fplmn
Due to historical reasons, and to maintain the compatibility between multiple different CSV file formats, the ADM pin
may be stored in three different columns. Only one of the three columns must be available.
* adm1: This column contains the ADM pin in numeric ASCII digit format. This format is the most common.
* pin_adm: Same as adm1, only the column name is different
* pin_adm_hex: If the ADM pin consists of raw HEX digits, rather then of numerical ASCII digits, then the ADM pin
can also be provided as HEX string using this column.
The following example shows a typical minimal example
::
"imsi","iccid","acc","ki","opc","adm1"
"999700000053010","8988211000000530108","0001","51ACE8BD6313C230F0BFE1A458928DF0","E5A00E8DE427E21B206526B5D1B902DF","65942330"
"999700000053011","8988211000000530116","0002","746AAFD7F13CFED3AE626B770E53E860","38F7CE8322D2A7417E0BBD1D7B1190EC","13445792"
"999700123053012","8988211000000530124","0004","D0DA4B7B150026ADC966DC637B26429C","144FD3AEAC208DFFF4E2140859BAE8EC","53540383"
"999700000053013","8988211000000530132","0008","52E59240ABAC6F53FF5778715C5CE70E","D9C988550DC70B95F40342298EB84C5E","26151368"
"999700000053014","8988211000000530140","0010","3B4B83CB9C5F3A0B41EBD17E7D96F324","D61DCC160E3B91F284979552CC5B4D9F","64088605"
"999700000053015","8988211000000530157","0020","D673DAB320D81039B025263610C2BBB3","4BCE1458936B338067989A06E5327139","94108841"
"999700000053016","8988211000000530165","0040","89DE5ACB76E06D14B0F5D5CD3594E2B1","411C4B8273FD7607E1885E59F0831906","55184287"
"999700000053017","8988211000000530173","0080","977852F7CEE83233F02E69E211626DE1","2EC35D48DBF2A99C07D4361F19EF338F","70284674"
The following commandline will instruct ``pySim-prog`` to use the provided CSV file as parameter source and the
ICCID (read from the card before programming) as a key to identify the card. To use the IMSI as a key, the parameter
``--read-imsi`` can be used instead of ``--read-iccid``. However, this option is only recommended to be used in very
specific corner cases.
``pySim-prog.py -p <pcsc_reader> --read-csv <path_to_csv_file> --source csv --read-iccid``
It is also possible to pick a row from the CSV file by manually providing an ICCID (option ``--iccid``) or an IMSI
(option ``--imsi``) that is then used as a key to find the matching row in the CSV file.
``pySim-prog.py -p <pcsc_reader> --read-csv <path_to_csv_file> --source csv --iccid <iccid_value>``
Writing CSV files
~~~~~~~~~~~~~~~~~
``pySim-prog`` is also able to generate CSV files that contain a subset of the parameters it has generated or received
from some other source (commandline, CSV-File). The generated file will be header-less and contain the following
columns:
* name
* iccid
* mcc
* mnc
* imsi
* smsp
* ki
* opc
A commandline that makes use of the CSV write feature would look like this:
``pySim-prog.py -p <pcsc_reader> --read-csv <path_to_input_csv_file> --read-iccid --source csv --write-csv <path_to_output_csv_file>``
Batch programming
~~~~~~~~~~~~~~~~~
In case larger card batches need to be programmed, it is possible to use the ``--batch`` parameter to run ``pySim-prog`` in batch mode.
The batch mode will prompt the user to insert a card. Once a card is detected in the reader, the programming is carried out. The user may then remove the card again and the process starts over. This allows for a quick and efficient card programming without permanent commandline interaction.
pySim-read
----------
``pySim-read`` allows you to read some data from a SIM card. It will only some files
of the card, and will only read files accessible to a normal user (without any special authentication)
``pySim-read`` allows to read some of the most important data items from a SIM
card. This means it will only read some files of the card, and will only read
files accessible to a normal user (without any special authentication)
These days, it is recommended to use the ``export`` command of ``pySim-shell``
instead. It performs a much more comprehensive export of all of the [standard]
files that can be found on the card. To get a human-readable decode instead of
the raw hex export, you can use ``export --json``.
Specifically, pySim-read will dump the following:
@@ -53,7 +205,7 @@ Specifically, pySim-read will dump the following:
* DF.GSM
* EF,IMSI
* EF.IMSI
* EF.GID1
* EF.GID2
* EF.SMSP
@@ -90,3 +242,4 @@ pySim-read usage
.. argparse::
:module: pySim-read
:func: option_parser
:prog: pySim-read.py

95
docs/library-esim.rst Normal file
View File

@@ -0,0 +1,95 @@
pySim eSIM libraries
====================
The pySim eSIM libraries implement a variety of functionality related to the GSMA eSIM universe,
including the various interfaces of SGP.21 + SGP.22, as well as Interoperable Profile decioding,
validation, personalization and encoding.
.. automodule:: pySim.esim
:members:
GSMA SGP.21/22 Remote SIM Provisioning (RSP) - High Level
---------------------------------------------------------
pySim.esim.rsp
~~~~~~~~~~~~~~
.. automodule:: pySim.esim.rsp
:members:
pySim.esim.es2p
~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.es2p
:members:
pySim.esim.es8p
~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.es8p
:members:
pySim.esim.es9p
~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.es9p
:members:
GSMA SGP.21/22 Remote SIM Provisioning (RSP) - Low Level
--------------------------------------------------------
pySim.esim.bsp
~~~~~~~~~~~~~~
.. automodule:: pySim.esim.bsp
:members:
pySim.esim.http_json_api
~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.http_json_api
:members:
pySim.esim.x509_cert
~~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.x509_cert
:members:
SIMalliance / TCA Interoperable Profile
---------------------------------------
pySim.esim.saip
~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.saip
:members:
pySim.esim.saip.oid
~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.saip.oid
:members:
pySim.esim.saip.personalization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.saip.personalization
:members:
pySim.esim.saip.templates
~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.saip.templates
:members:
pySim.esim.saip.validation
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: pySim.esim.saip.validation
:members:

12
docs/library.rst
View File

@@ -74,18 +74,6 @@ at 9600 bps. These readers are sometimes called `Phoenix`.
:members:
pySim construct utilities
-------------------------
.. automodule:: pySim.construct
:members:
pySim TLV utilities
-------------------
.. automodule:: pySim.tlv
:members:
pySim utility functions
-----------------------

149
docs/osmo-smdpp.rst Normal file
View File

@@ -0,0 +1,149 @@
osmo-smdpp
==========
`osmo-smdpp` is a proof-of-concept implementation of a minimal **SM-DP+** as specified for the *GSMA
Consumer eSIM Remote SIM provisioning*.
At least at this point, it is intended to be used for research and development, and not as a
production SM-DP+.
Unless you are a GSMA SAS-SM accredited SM-DP+ operator and have related DPtls, DPauth and DPpb
certificates signed by the GSMA CI, you **can not use osmo-smdpp with regular production eUICC**.
This is due to how the GSMA eSIM security architecture works. You can, however, use osmo-smdpp with
so-called *test-eUICC*, which contain certificates/keys signed by GSMA test certificates as laid out
in GSMA SGP.26.
At this point, osmo-smdpp does not support anything beyond the bare minimum required to download
eSIM profiles to an eUICC. Specifically, there is no ES2+ interface, and there is no built-in
support for profile personalization yet.
osmo-smdpp currently
* [by default] uses test certificates copied from GSMA SGP.26 into `./smdpp-data/certs`, assuming that your
osmo-smdpp would be running at the host name `testsmdpplus1.example.com`. You can of course replace those
certificates with your own, whether SGP.26 derived or part of a *private root CA* setup with matching eUICCs.
* doesn't understand profile state. Any profile can always be downloaded any number of times, irrespective
of the EID or whether it was downloaded before. This is actually very useful for R&D and testing, as it
doesn't require you to generate new profiles all the time. This logic of course is unsuitable for
production usage.
* doesn't perform any personalization, so the IMSI/ICCID etc. are always identical (the ones that are stored in
the respective UPP `.der` files)
* **is absolutely insecure**, as it
* does not perform all of the mandatory certificate verification (it checks the certificate chain, but not
the expiration dates nor any CRL)
* does not evaluate/consider any *Confirmation Code*
* stores the sessions in an unencrypted *python shelve* and is hence leaking one-time key materials
used for profile encryption and signing.
Running osmo-smdpp
------------------
osmo-smdpp comes with built-in TLS support which is enabled by default. However, it is always possible to
disable the built-in TLS support if needed.
In order to use osmo-smdpp without the built-in TLS support, it has to be put behind a TLS reverse proxy,
which terminates the ES9+ HTTPS traffic from the LPA, and then forwards it as plain HTTP to osmo-smdpp.
NOTE: The built in TLS support in osmo-smdpp makes use of the python *twisted* framework. Older versions
of this framework appear to have problems when using the example elliptic curve certificates (both NIST and
Brainpool) from GSMA.
nginx as TLS proxy
~~~~~~~~~~~~~~~~~~
If you chose to use `nginx` as TLS reverse proxy, you can use the following configuration snippet::
upstream smdpp {
server localhost:8000;
}
server {
listen 443 ssl;
server_name testsmdpplus1.example.com;
ssl_certificate /my/path/to/pysim/smdpp-data/certs/DPtls/CERT_S_SM_DP_TLS_NIST.pem;
ssl_certificate_key /my/path/to/pysim/smdpp-data/certs/DPtls/SK_S_SM_DP_TLS_NIST.pem;
location / {
proxy_read_timeout 600s;
proxy_hide_header X-Powered-By;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port $proxy_port;
proxy_set_header Host $host;
proxy_pass http://smdpp/;
}
}
You can of course achieve a similar functionality with apache, lighttpd or many other web server
software.
supplementary files
~~~~~~~~~~~~~~~~~~~
The `smdpp-data/certs` directory contains the DPtls, DPauth and DPpb as well as CI certificates
used; they are copied from GSMA SGP.26 v2. You can of course replace them with custom certificates
if you're operating eSIM with a *private root CA*.
The `smdpp-data/upp` directory contains the UPP (Unprotected Profile Package) used. The file names (without
.der suffix) are looked up by the matchingID parameter from the activation code presented by the LPA.
commandline options
~~~~~~~~~~~~~~~~~~~
Typically, you just run osmo-smdpp without any arguments, and it will bind its built-in HTTPS ES9+ interface to
`localhost` TCP port 443. In this case an external TLS reverse proxy is not needed.
osmo-smdpp currently doesn't have any configuration file.
There are command line options for binding:
Bind the HTTPS ES9+ to a port other than 443::
./osmo-smdpp.py -p 8443
Disable the built-in TLS support and bind the plain-HTTP ES9+ to a port 8000::
./osmo-smdpp.py -p 8000 --nossl
Bind the HTTP ES9+ to a different local interface::
./osmo-smdpp.py -H 127.0.0.2
DNS setup for your LPA
~~~~~~~~~~~~~~~~~~~~~~
The LPA must resolve `testsmdpplus1.example.com` to the IP address of your TLS proxy.
It must also accept the TLS certificates used by your TLS proxy. In case osmo-smdpp is used with built-in TLS support,
it will use the certificates provided in smdpp-data.
NOTE: The HTTPS ES9+ interface cannot be addressed by the LPA directly via its IP address. The reason for this is that
the included SGP.26 (DPtls) test certificates explicitly restrict the hostname to `testsmdpplus1.example.com` in the
`X509v3 Subject Alternative Name` extension. Using a bare IP address as hostname may cause the certificate to be
rejected by the LPA.
Supported eUICC
~~~~~~~~~~~~~~~
If you run osmo-smdpp with the included SGP.26 (DPauth, DPpb) certificates, you must use an eUICC with matching SGP.26
certificates, i.e. the EUM certificate must be signed by a SGP.26 test root CA and the eUICC certificate
in turn must be signed by that SGP.26 EUM certificate.
sysmocom (sponsoring development and maintenance of pySim and osmo-smdpp) is selling SGP.26 test eUICC
as `sysmoEUICC1-C2T`. They are publicly sold in the `sysmocom webshop <https://shop.sysmocom.de/eUICC-for-consumer-eSIM-RSP-with-SGP.26-Test-Certificates/sysmoEUICC1-C2T>`_.
In general you can use osmo-smdpp also with certificates signed by any other certificate authority. You
just always must ensure that the certificates of the SM-DP+ are signed by the same root CA as those of your
eUICCs.
Hypothetically, osmo-smdpp could also be operated with GSMA production certificates, but it would require
that somebody brings the code in-line with all the GSMA security requirements (HSM support, ...) and operate
it in a GSMA SAS-SM accredited environment and pays for the related audits.

836
docs/put_key-tutorial.rst Normal file
View File

@@ -0,0 +1,836 @@
Guide: Managing GP Keys
=======================
Most of today's smartcards follow the GlobalPlatform Card Specification and the included Security Domain model.
UICCs and eUCCCs are no exception here.
The Security Domain acts as an on-card representative of a card authority or administrator. It is used to perform tasks
like the installation of applications or the provisioning and rotation of secure channel keys. It also acts as a secure
key storage and offers all kinds of cryptographic services to applications that are installed under a specific
Security Domain (see also GlobalPlatform Card Specification, section 7).
In this tutorial, we will show how to work with the key material (keysets) stored inside a Security Domain and how to
rotate (replace) existing keys. We will also show how to provision new keys.
.. warning:: Making changes to keysets requires extreme caution as misconfigured keysets may lock you out permanently.
It's also strongly recommended to maintain at least one backup keyset that you can use as fallback in case
the primary keyset becomes unusable for some reason.
Selecting a Security Domain
~~~~~~~~~~~~~~~~~~~~~~~~~~~
A typical smartcard, such as an UICC will have one primary Security Domain, called the Issuer Security Domain (ISD).
When working with those cards, the ISD will show up in the UICC filesystem tree as `ADF.ISD` and can be selected like
any other file.
::
pySIM-shell (00:MF)> select ADF.ISD
{
"application_id": "a000000003000000",
"proprietary_data": {
"maximum_length_of_data_field_in_command_message": 255
}
}
When working with eUICCs, multiple Security Domains are involved. The model is fundamentally different from the classic
model with one primary Security Domain (ISD). In the case of eUICCs, an ISD-R (Issuer Security Domain - Root) and an
ISD-P (Issuer Security Domain - Profile) exist (see also: GSMA SGP.02, section 2.2.1).
The ISD-P is established by the ISD-R during the profile installation and serves as a secure container for an eSIM
profile. Within the ISD-P the eSIM profile establishes a dedicated Security Domain called `MNO-SD` (see also GSMA
SGP.02, section 2.2.4). This `MNO-SD` is comparable to the Issuer Security Domain (ISD) we find on UICCs. The AID of
`MNO-SD` is either the default AID for the Issuer Security Domain (see also GlobalPlatform, section H.1.3) or a
different value specified by the provider of the eSIM profile.
Since the AID of the `MNO-SD` is not a fixed value, it is not known by `pySim-shell`. This means there will be no
`ADF.ISD` file shown in the file system, but we can simply select the `ADF.ISD-R` first and then select the `MNO-SD`
using a raw APDU. In the following example we assume that the default AID (``a000000151000000``) is used The APDU
would look like this: ``00a4040408`` + ``a000000151000000`` + ``00``
::
pySIM-shell (00:MF)> select ADF.ISD-R
{
"application_id": "a0000005591010ffffffff8900000100",
"proprietary_data": {
"maximum_length_of_data_field_in_command_message": 255
},
"isdr_proprietary_application_template": {
"supported_version_number": "020300"
}
}
pySIM-shell (00:MF/ADF.ISD-R)> apdu 00a4040408a00000015100000000
SW: 9000, RESP: 6f108408a000000151000000a5049f6501ff
After that, the prompt will still show the `ADF.ISD-R`, but we are actually in `ADF.ISD` and the standard GlobalPlatform
operations like `establish_scpXX`, `get_data`, and `put_key` should work. By doing this, we simply have tricked
`pySim-shell` into making the GlobalPlatform related commands available for some other Security Domain we are not
interested in. With the raw APDU we then have swapped out the Security Domain under the hood. The same workaround can
be applied to any Security Domain, provided that the AID is known to the user.
Establishing a secure channel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before we can make changes to the keysets in the currently selected Security Domain we must first establish a secure
channel with that Security Domain. In the following examples we will use `SCP02` (see also GlobalPlatform Card
Specification, section E.1.1) and `SCP03` (see also GlobalPlatform Card Specification – Amendment D) to establish the
secure channel. `SCP02` is slightly older than `SCP03`. The main difference between the two is that `SCP02` uses 3DES
while `SCP03` is based on AES.
.. warning:: Secure channel protocols like `SCP02` and `SCP03` may manage an error counter to count failed login
attempts. This means attempting to establish a secure channel with a wrong keyset multiple times may lock
you out permanently. Double check the applied keyset before attempting to establish a secure channel.
.. warning:: The key values used in the following examples are random key values used for illustration purposes only.
Each UICC or eSIM profile is shipped with individual keys, which means that the keys used below will not
work with your UICC or eSIM profile. You must replace the key values with the values you have received
from your UICC vendor or eSIM profile provider.
Example: `SCP02`
----------------
In the following example, we assume that we want to establish a secure channel with the ISD of a `sysmoUSIM-SJA5` UICC.
Along with the card we have received the following keyset:
+---------+----------------------------------+
| Keyname | Keyvalue |
+=========+==================================+
| ENC/KIC | F09C43EE1A0391665CC9F05AF4E0BD10 |
+---------+----------------------------------+
| MAC/KID | 01981F4A20999F62AF99988007BAF6CA |
+---------+----------------------------------+
| DEK/KIK | 8F8AEE5CDCC5D361368BC45673D99195 |
+---------+----------------------------------+
This keyset is tied to the key version number KVN 122 and is configured as a DES keyset. We can use this keyset to
establish a secure channel using the SCP02 Secure Channel Protocol.
::
pySIM-shell (00:MF/ADF.ISD)> establish_scp02 --key-enc F09C43EE1A0391665CC9F05AF4E0BD10 --key-mac 01981F4A20999F62AF99988007BAF6CA --key-dek 8F8AEE5CDCC5D361368BC45673D99195 --key-ver 112 --security-level 3
Successfully established a SCP02[03] secure channel
Example: `SCP03`
----------------
The establishment of a secure channel via SCP03 works just the same. In the following example we will establish a
secure channel to the `MNO-SD` of an eSIM profile. The SCP03 keyset we use is tied to KVN 48 and looks like this:
+---------+------------------------------------------------------------------+
| Keyname | Keyvalue |
+=========+==================================================================+
| ENC/KIC | 63af517c29ad6ac6fcadfe6ac8a3c8a041d8141c7eb845ef1cba6112a325e430 |
+---------+------------------------------------------------------------------+
| MAC/KID | 54b9ad6713ae922f54014ed762132e7b59bdcd2a2a6beba98fb9afe6b4df27e1 |
+---------+------------------------------------------------------------------+
| DEK/KIK | cbb933ba2389da93c86c112739cd96389139f16c6f80f7d16bf3593e407ca893 |
+---------+------------------------------------------------------------------+
We assume that the `MNO-SD` is already selected (see above). We may now establish the SCP03 secure channel:
::
pySIM-shell (00:MF/ADF.ISD-R)> establish_scp03 --key-enc 63af517c29ad6ac6fcadfe6ac8a3c8a041d8141c7eb845ef1cba6112a325e430 --key-mac 54b9ad6713ae922f54014ed762132e7b59bdcd2a2a6beba98fb9afe6b4df27e1 --key-dek cbb933ba2389da93c86c112739cd96389139f16c6f80f7d16bf3593e407ca893 --key-ver 48 --security-level 3
Successfully established a SCP03[03] secure channel
Understanding Keysets
~~~~~~~~~~~~~~~~~~~~~
Before making any changes to keysets, it is recommended to check the status of the currently installed keysets. To do
so, we use the `get_data` command to retrieve the `key_information`. This command does not require the establishment of
a secure channel. We also cannot read back the key values themselves, but we get a summary of the installed keys
together with their KVN numbers, IDs, algorithm and key length values.
Example: `key_information` from a `sysmoISIM-SJA5`:
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> get_data key_information
{
"key_information": [
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 112,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 112,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 112,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 1,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 1,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 1,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 2,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 2,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 2,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 47,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 47,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 47,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
}
]
}
Example: `key_information` from a `sysmoEUICC1-C2T`:
::
pySIM-shell (SCP03[03]:00:MF/ADF.ISD-R)> get_data key_information
{
"key_information": [
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 50,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 50,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 50,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 64,
"key_types": [
{
"type": "aes",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 64,
"key_types": [
{
"type": "tls_psk",
"length": 16
}
]
}
}
]
}
The output from those two examples above may seem lengthy, but in order to move on and to provision own keys
successfully, it is important to understand each aspect of it.
Key Version Number (KVN)
------------------------
Each key is associated with a Key Version Number (KVN). Multiple keys that share the same KVN belong to the same
keyset. In the first example above we can see that four keysets with KVN numbers 112, 1, 2 and 47 are provisioned.
In the second example we see two keysets. One with KVN 50 and one with KVN 64.
The term "Key Version Number" is misleading as this number is not really a version number. It's actually a unique
identifier for a specific keyset that also defines with which Secure Channel Protocol a key can be used. This means
that the KVN is not just an arbitrary number. The following (incomplete) table gives a hint which KVN numbers may be
used with which Secure Channel Protocol.
+-----------+-------------------------------------------------------+
| KVN range | Secure Channel Protocol |
+===========+=======================================================+
| 1-15 | reserved for `SCP80` (OTA SMS) |
+-----------+-------------------------------------------------------+
| 17 | reserved for DAP specified in ETSI TS 102 226 |
+-----------+-------------------------------------------------------+
| 32-47 | reserved for `SCP02` |
+-----------+-------------------------------------------------------+
| 48-63 | reserved for `SCP03` |
+-----------+-------------------------------------------------------+
| 64-79 | reserved for `SCP81` (GSMA SGP.02, section 2.2.5.1) |
+-----------+-------------------------------------------------------+
| 112 | Token key (RSA public or DES, also used with `SCP02`) |
+-----------+-------------------------------------------------------+
| 113 | Receipt key (DES) |
+-----------+-------------------------------------------------------+
| 115 | DAP verification key (RS public or DES) |
+-----------+-------------------------------------------------------+
| 116 | reserved for CASD |
+-----------+-------------------------------------------------------+
| 117 | 16-byte DES key for Ciphered Load File Data Block |
+-----------+-------------------------------------------------------+
| 255 | reserved for ISD with SCP02 without SCP80 support |
+-----------+-------------------------------------------------------+
With that we can now understand that in the first example, the first and the last keyset is intended to be used with
`SCP02` and that the second and the third keyset is intended to be used with `SCP80` (OTA SMS). In the second example we
can see that the first keyset is intended to be used with `SCP03`, wheres the second should be usable with `SCP81`.
Key Identifier
--------------
Each keyset consists of a number of keys, where each key has a different Key Identifier. The Key Identifier is usually
an incrementing number that starts counting at 1. The Key Identifier is used to distinguish the keys within the keyset.
The exact number of keys and their attributes depends on the secure channel protocol for which the keyset is intended
for. Each secure channel protocol may have its specific requirements on how many keys of which which type, length or
Key Identifier have to be present.
However, almost all of the classic secure channel protocols (including `SCP02`, `SCP03` and `SCP81`) make use of the
following three-key scheme:
+----------------+---------+---------------------------------------+
| Key Identifier | Keyname | Purpose |
+================+=========+=======================================+
| 1 | ENC/KIC | encryption/decryption |
+----------------+---------+---------------------------------------+
| 2 | MAC/KID | cryptographic checksumming/signing |
+----------------+---------+---------------------------------------+
| 3 | DEK/KIK | encryption/decryption of key material |
+----------------+---------+---------------------------------------+
In this case, all three keys share the same length and are used with the same algorithm. The key length is often used
to implicitly select sub-types of an algorithm. (e.g. a 16 byte key of type `aes` is associated with `AES128`, where a 32
byte key would be associated with `AES256`).
The second example shows that different schemes are possible. The `SCP80` keyset from the second example uses a scheme
that works with two keys:
+----------------+---------+---------------------------------------+
| Key Identifier | Keyname | Purpose |
+================+=========+=======================================+
| 1 | TLS-PSK | pre-shared key used for TLS |
+----------------+---------+---------------------------------------+
| 2 | DEK/KIK | encryption/decryption of key material |
+----------------+---------+---------------------------------------+
It should also be noted that the order in which keysets and keys appear is an implementation detail of the UICC/eUICC
O/S. The order has no influence on how a keyset is interpreted. Only the Key Version Number (KVN) and the Key Identifier
matter.
Rotating a keyset
~~~~~~~~~~~~~~~~~
Rotating keys is one of the most basic tasks one might want to perform on an UICC/eUICC before using it productively. In
the following example we will illustrate how key rotation can be done. When rotating keys, only the key itself may
change. For example it is not possible to change the key length or the algorithm used (see also GlobalPlatform Card
Specification, section 11.8.2.3.3). Any key of the current Security Domain can be rotated, this also includes the key
that was used to establish the secure channel.
In the following example we assume that the Security Domain is selected and a secure channel is already established. We
intend to rotate the keyset with KVN 112. Since this keyset uses triple DES keys with a key length of 16, we must
replace it with a keyset with keys of the same nature.
The new keyset shall look like this:
+----------------+---------+----------------------------------+
| Key Identifier | Keyname | Keyvalue |
+================+=========+==================================+
| 1 | ENC/KIC | 542C37A6043679F2F9F71116418B1CD5 |
+----------------+---------+----------------------------------+
| 2 | MAC/KID | 34F11BAC8E5390B57F4E601372339E3C |
+----------------+---------+----------------------------------+
| 3 | DEK/KIK | 5524F4BECFE96FB63FC29D6BAAC6058B |
+----------------+---------+----------------------------------+
When passing the keys to the `put_key` commandline, we set the Key Identifier of the first key using the `--key-id`
parameter. This Key Identifier will be valid for the first key (KIC) we pass. For all consecutive keys, the Key
Identifier will be incremented automatically (see also GlobalPlatform Card Specification, section 11.8.2.2). To Ensure
that the new KIC, KID and KIK keys get the correct Key Identifiers, it is crucial to maintain order when passing the
keys in the `--key-data` arguments. It is also important that each `--key-data` argument is preceded by a `--key-type`
argument that sets the algorithm correctly (`des` in this case).
Finally we have to target the keyset we want to rotate by its KVN. The `--old-key-version-nr` argument is set to 112
as this identifies the keyset we want to rotate. The `--key-version-nr` is also set to 112 as we do not want
KVN to be changed in this example. Changing the KVN while rotating a keyset is possible. In case the KVN has to change
for some reason, the new KVN must be selected carefully to keep the key usable with the associated Secure Channel
Protocol.
The commandline that matches the keyset we had laid out above looks like this:
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> put_key --key-id 1 --key-type des --key-data 542C37A6043679F2F9F71116418B1CD5 --key-type des --key-data 34F11BAC8E5390B57F4E601372339E3C --key-type des --key-data 5524F4BECFE96FB63FC29D6BAAC6058B --old-key-version-nr 112 --key-version-nr 112
After executing this put_key commandline, the keyset identified by KVN 122 is equipped with new keys. We can use
`get_data key_information` to inspect the currently installed keysets. The output should appear unchanged as
we only swapped out the keys. All other parameters, identifiers etc. should remain constant.
.. warning:: It is technically possible to rotate a keyset in a `non atomic` way using one `put_key` commandline for
each key. However, in case the targeted keyset is the one used to establish the current secure channel,
this method should not be used since, depending on the UICC/eUICC model, half-written key material may
interrupt the current secure channel.
Removing a keyset
~~~~~~~~~~~~~~~~~
In some cases it is necessary to remove a keyset entirely. This can be done with the `delete_key` command. Here it is
important to understand that `delete_key` only removes one specific key from a specific keyset. This means that you
need to run a separate `delete_key` command for each key inside a keyset.
In the following example we assume that the Security Domain is selected and a secure channel is already established. We
intend to remove the keyset with KVN 112. This keyset consists of three keys.
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> delete_key --key-ver 112 --key-id 1
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> delete_key --key-ver 112 --key-id 2
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> delete_key --key-ver 112 --key-id 3
To verify that the keyset has been deleted properly, we can use the `get_data key_information` command to inspect the
current status of the installed keysets. We should see that the key with KVN 112 is no longer present.
Adding a keyset
~~~~~~~~~~~~~~~
In the following we will discuss how to add an entirely new keyset. The procedure is almost identical with the key
rotation procedure we have already discussed and it is assumed that all details about the key rotation are understood.
In this section we will go into more detail and illustrate how to provision new 3DES, `AES128` and `AES256` keysets.
It is important to keep in mind that storage space on smartcard is a precious resource. In many cases the amount of
keysets that a Security Domain can store is limited. In some situations you may be forced to sacrifice one of your
existing keysets in favor of a new keyset.
The main difference between key rotation and the adding of new keys is that we do not simply replace an existing key.
Instead an entirely new key is programmed into the Security Domain. Therefore the `put_key` commandline will have no
`--old-key-version-nr` parameter. From the commandline perspective, this is already the only visible difference from a
commandline that simply rotates a keyset. Since we are writing an entirely new keyset, we are free to chose the
algorithm and the key length within the parameter range permitted by the targeted secure channel protocol. Otherwise
the same rules apply.
For reference, it should be mentioned that it is also possible to add or rotate keyset using multiple `put_key`
commandlines. In this case one `put_key` commandline for each key is used. Each commandline will specify `--key-id` and
`--key-version-nr` and one `--key-type` and `--key-data` tuple. However, when rotating or adding a keyset step-by-step,
the whole process happens in a `non-atomic` way, which is less reliable. Therefore we will favor the `atomic method`
In the following examples we assume that the Security Domain is selected and a secure channel is already established.
Example: `3DES` key for `SCP02`
-------------------------------
Let's assume we want to provision a new 3DES keyset that we can use for SCP02. The keyset shall look like this:
+----------------+---------+----------------------------------+
| Key Identifier | Keyname | Keyvalue |
+================+=========+==================================+
| 1 | ENC/KIC | 542C37A6043679F2F9F71116418B1CD5 |
+----------------+---------+----------------------------------+
| 2 | MAC/KID | 34F11BAC8E5390B57F4E601372339E3C |
+----------------+---------+----------------------------------+
| 3 | DEK/KIK | 5524F4BECFE96FB63FC29D6BAAC6058B |
+----------------+---------+----------------------------------+
The keyset shall be a associated with the KVN 46. We have made sure before that KVN 46 is still unused and that this
KVN number is actually suitable for SCP02 keys. As we are using 3DES, it is obvious that we have to pass 3 keys with 16
byte length.
To program the key, we may use the following commandline. As we can see, this commandline is almost the exact same as
the one from the key rotation example where we were rotating a 3DES key. The only difference is that we didn't specify
an old KVN number and that we have chosen a different KVN.
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> put_key --key-id 1 --key-type des --key-data 542C37A6043679F2F9F71116418B1CD5 --key-type des --key-data 34F11BAC8E5390B57F4E601372339E3C --key-type des --key-data 5524F4BECFE96FB63FC29D6BAAC6058B --key-version-nr 46
In case of success, the keyset should appear in the `key_information` among the other keysets that are already present.
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> get_data key_information
{
"key_information": [
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 46,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 46,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 46,
"key_types": [
{
"type": "des",
"length": 16
}
]
}
},
...
]
}
Example: `AES128` key for `SCP80`
---------------------------------
In this example we intend to provision a new `AES128` keyset that we can use with SCP80 (OTA SMS). The keyset shall look
like this:
+----------------+---------+----------------------------------+
| Key Identifier | Keyname | Keyvalue |
+================+=========+==================================+
| 1 | ENC/KIC | 542C37A6043679F2F9F71116418B1CD5 |
+----------------+---------+----------------------------------+
| 2 | MAC/KID | 34F11BAC8E5390B57F4E601372339E3C |
+----------------+---------+----------------------------------+
| 3 | DEK/KIK | 5524F4BECFE96FB63FC29D6BAAC6058B |
+----------------+---------+----------------------------------+
In addition to that, we want to associate this key with KVN 3. We have inspected the currently installed keysets before
and made sure that KVN 3 is still unused. We are also aware that for SCP80 we may only use KVN values from 1 to 15.
For `AES128`, we specify the algorithm using the `--key-type aes` parameter. The selection between `AES128` and `AES256` is
done implicitly using the key length. Since we want to use `AES128` in this case, all three keys have a length of 16 byte.
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> put_key --key-id 1 --key-type aes --key-data 542C37A6043679F2F9F71116418B1CD5 --key-type aes --key-data 34F11BAC8E5390B57F4E601372339E3C --key-type aes --key-data 5524F4BECFE96FB63FC29D6BAAC6058B --key-version-nr 3
In case of success, the keyset should appear in the `key_information` among the other keysets that are already present.
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> get_data key_information
{
"key_information": [
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 3,
"key_types": [
{
"type": "aes",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 3,
"key_types": [
{
"type": "aes",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 3,
"key_types": [
{
"type": "aes",
"length": 16
}
]
}
},
...
]
}
Example: `AES256` key for `SCP03`
---------------------------------
Let's assume we want to provision a new `AES256` keyset that we can use for SCP03. The keyset shall look like this:
+----------------+---------+------------------------------------------------------------------+
| Key Identifier | Keyname | Keyvalue |
+================+=========+==================================================================+
| 1 | ENC/KIC | 542C37A6043679F2F9F71116418B1CD5542C37A6043679F2F9F71116418B1CD5 |
+----------------+---------+------------------------------------------------------------------+
| 2 | MAC/KID | 34F11BAC8E5390B57F4E601372339E3C34F11BAC8E5390B57F4E601372339E3C |
+----------------+---------+------------------------------------------------------------------+
| 3 | DEK/KIK | 5524F4BECFE96FB63FC29D6BAAC6058B5524F4BECFE96FB63FC29D6BAAC6058B |
+----------------+---------+------------------------------------------------------------------+
In addition to that, we assume that we want to associate this key with KVN 51. This KVN number falls in the range of
48 - 63 and is therefore suitable for a key that shall be usable with SCP03. We also made sure before that KVN 51 is
still unused.
With that we can go ahead and make up the following commandline:
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> put_key --key-id 1 --key-type aes --key-data 542C37A6043679F2F9F71116418B1CD5542C37A6043679F2F9F71116418B1CD5 --key-type aes --key-data 34F11BAC8E5390B57F4E601372339E3C34F11BAC8E5390B57F4E601372339E3C --key-type aes --key-data 5524F4BECFE96FB63FC29D6BAAC6058B5524F4BECFE96FB63FC29D6BAAC6058B --key-version-nr 51
In case of success, we should see the keyset in the `key_information`
::
pySIM-shell (SCP02[03]:00:MF/ADF.ISD)> get_data key_information
{
"key_information": [
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 51,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 51,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
{
"key_information_data": {
"key_identifier": 3,
"key_version_number": 51,
"key_types": [
{
"type": "aes",
"length": 32
}
]
}
},
...
]
}
Example: `AES128` key for `SCP81`
---------------------------------
In this example we will show how to provision a new `AES128` keyset for `SCP81`. We will provision this keyset under
KVN 64. The keyset we intend to apply shall look like this:
+----------------+---------+----------------------------------+
| Key Identifier | Keyname | Keyvalue |
+================+=========+==================================+
| 1 | TLS-PSK | 000102030405060708090a0b0c0d0e0f |
+----------------+---------+----------------------------------+
| 2 | DEK/KIK | 000102030405060708090a0b0c0d0e0f |
+----------------+---------+----------------------------------+
With that we can put together the following command line:
::
put_key --key-id 1 --key-type tls_psk --key-data 000102030405060708090a0b0c0d0e0f --key-type aes --key-data 000102030405060708090a0b0c0d0e0f --key-version-nr 64
In case of success, the keyset should appear in the `key_information` as follows:
::
pySIM-shell (SCP03[03]:00:MF/ADF.ISD-R)> get_data key_information
{
"key_information": [
...,
{
"key_information_data": {
"key_identifier": 2,
"key_version_number": 64,
"key_types": [
{
"type": "aes",
"length": 16
}
]
}
},
{
"key_information_data": {
"key_identifier": 1,
"key_version_number": 64,
"key_types": [
{
"type": "tls_psk",
"length": 16
}
]
}
}
]
}

267
docs/pysim_fs_sphinx.py Normal file
View File

@@ -0,0 +1,267 @@
"""
Sphinx extension: auto-generate docs/filesystem.rst from the pySim EF class hierarchy.
Hooked into Sphinx's ``builder-inited`` event so the file is always regenerated
from the live Python classes before Sphinx reads any source files.
The table of root objects to document is in SECTIONS near the top of this file.
EXCLUDED lists CardProfile/CardApplication subclasses intentionally omitted from
SECTIONS, with reasons. Both tables are read by tests/unittests/test_fs_coverage.py
to ensure every class with EF/DF content is accounted for.
"""
import importlib
import inspect
import json
import os
import sys
import textwrap
# Ensure pySim is importable when this module is loaded as a Sphinx extension
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from pySim.filesystem import (CardApplication, CardDF, CardMF, CardEF, # noqa: E402
TransparentEF, TransRecEF, LinFixedEF, CyclicEF, BerTlvEF)
from pySim.profile import CardProfile # noqa: E402
# Generic EF base classes whose docstrings describe the *type* of file
# (Transparent, LinFixed, ...) rather than a specific file's content.
# Suppress those boilerplate texts in the per-EF entries; they are only
# useful once, at the top of the document or in a dedicated glossary.
_EF_BASE_TYPES = frozenset([TransparentEF,
TransRecEF,
LinFixedEF,
CyclicEF,
BerTlvEF])
# ---------------------------------------------------------------------------
# Sections: (heading, module, class-name)
# The class must be either a CardProfile (uses .files_in_mf) or a CardDF
# subclass (uses .children).
# ---------------------------------------------------------------------------
SECTIONS = [
('MF / TS 102 221 (UICC)',
'pySim.ts_102_221', 'CardProfileUICC'),
('ADF.USIM / TS 31.102',
'pySim.ts_31_102', 'ADF_USIM'),
('ADF.ISIM / TS 31.103',
'pySim.ts_31_103', 'ADF_ISIM'),
('ADF.HPSIM / TS 31.104',
'pySim.ts_31_104', 'ADF_HPSIM'),
('DF.GSM + DF.TELECOM / TS 51.011 (SIM)',
'pySim.ts_51_011', 'CardProfileSIM'),
('CDMA / IS-820 (RUIM)',
'pySim.cdma_ruim', 'CardProfileRUIM'),
('DF.EIRENE / GSM-R',
'pySim.gsm_r', 'DF_EIRENE'),
('DF.SYSTEM / sysmocom SJA2+SJA5',
'pySim.sysmocom_sja2', 'DF_SYSTEM'),
]
# ---------------------------------------------------------------------------
# Excluded: {(module, class-name)}
# CardProfile and CardApplication subclasses that have EF/DF children but are
# intentionally absent from SECTIONS. Keeping this list explicit lets
# test_fs_coverage.py detect newly added classes that the developer forgot to
# add to either table.
# ---------------------------------------------------------------------------
EXCLUDED = {
# eUICC profiles inherit files_in_mf verbatim from CardProfileUICC; the
# eUICC-specific content lives in ISD-R / ISD-P applications, not in MF.
('pySim.euicc', 'CardProfileEuiccSGP02'),
('pySim.euicc', 'CardProfileEuiccSGP22'),
('pySim.euicc', 'CardProfileEuiccSGP32'),
# CardApplication* classes are thin wrappers that embed an ADF_* instance.
# The ADF contents are already documented via the corresponding ADF_* entry
# in SECTIONS above.
('pySim.ts_31_102', 'CardApplicationUSIM'),
('pySim.ts_31_102', 'CardApplicationUSIMnonIMSI'),
('pySim.ts_31_103', 'CardApplicationISIM'),
('pySim.ts_31_104', 'CardApplicationHPSIM'),
}
# RST underline characters ordered by nesting depth
_HEADING_CHARS = ['=', '=', '-', '~', '^', '"']
# Level 0 uses '=' with overline (page title).
# Level 1 uses '=' without overline (major sections).
# Levels 2+ use the remaining characters for DFs.
# ---------------------------------------------------------------------------
# RST formatting helpers
# ---------------------------------------------------------------------------
def _heading(title: str, level: int) -> str:
"""Return an RST heading string. Level 0 gets an overline."""
char = _HEADING_CHARS[level]
rule = char * len(title)
if level == 0:
return f'{rule}\n{title}\n{rule}\n\n'
return f'{title}\n{rule}\n\n'
def _json_default(obj):
"""Fallback serialiser: bytes -> hex, anything else -> repr."""
if isinstance(obj, (bytes, bytearray)):
return obj.hex()
return repr(obj)
def _examples_block(cls) -> str:
"""Return RST code-block examples (one per vector), or '' if none exist.
Each example is rendered as a ``json5`` code-block with the hex-encoded
binary as a ``// comment`` on the first line, followed by the decoded JSON.
``json5`` is used instead of ``json`` so that Pygments does not flag the
``//`` comment as a syntax error.
"""
vectors = []
for attr in ('_test_de_encode', '_test_decode'):
v = getattr(cls, attr, None)
if v:
vectors.extend(v)
if not vectors:
return ''
lines = ['**Examples**\n\n']
for t in vectors:
# 2-tuple: (encoded, decoded)
# 3-tuple: (encoded, record_nr, decoded) — LinFixedEF / CyclicEF
if len(t) >= 3:
encoded, record_nr, decoded = t[0], t[1], t[2]
comment = f'record {record_nr}: {encoded.lower()}'
else:
encoded, decoded = t[0], t[1]
comment = f'file: {encoded.lower()}'
json_str = json.dumps(decoded, default=_json_default, indent=2)
json_indented = textwrap.indent(json_str, ' ')
lines.append('.. code-block:: json5\n\n')
lines.append(f' // {comment}\n')
lines.append(json_indented + '\n')
lines.append('\n')
return ''.join(lines)
def _document_ef(ef: CardEF) -> str:
"""Return RST for a single EF. Uses ``rubric`` to stay out of the TOC."""
cls = type(ef)
parts = [ef.fully_qualified_path_str()]
if ef.fid:
parts.append(f'({ef.fid.upper()})')
if ef.desc:
parts.append(f'\u2014 {ef.desc}') # em-dash
title = ' '.join(parts)
lines = [f'.. rubric:: {title}\n\n']
# Only show a docstring if it is specific to this class. EFs that are
# direct instances of a base type (TransparentEF, LinFixedEF, ...) carry
# only the generic "what is a TransparentEF" boilerplate; named subclasses
# without their own __doc__ have cls.__dict__['__doc__'] == None. Either
# way, suppress the text here - it belongs at the document level, not
# repeated for every single EF entry.
doc = None if cls in _EF_BASE_TYPES else cls.__dict__.get('__doc__')
if doc:
lines.append(inspect.cleandoc(doc) + '\n\n')
examples = _examples_block(cls)
if examples:
lines.append(examples)
return ''.join(lines)
def _document_df(df: CardDF, level: int) -> str:
"""Return RST for a DF section and all its children, recursively."""
parts = [df.fully_qualified_path_str()]
if df.fid:
parts.append(f'({df.fid.upper()})')
if df.desc:
parts.append(f'\u2014 {df.desc}') # em-dash
title = ' '.join(parts)
lines = [_heading(title, level)]
cls = type(df)
doc = None if cls in (CardDF, CardMF) else cls.__dict__.get('__doc__')
if doc:
lines.append(inspect.cleandoc(doc) + '\n\n')
for child in df.children.values():
if isinstance(child, CardDF):
lines.append(_document_df(child, level + 1))
elif isinstance(child, CardEF):
lines.append(_document_ef(child))
return ''.join(lines)
# ---------------------------------------------------------------------------
# Top-level generator
# ---------------------------------------------------------------------------
def generate_filesystem_rst() -> str:
"""Walk all registered sections and return the full RST document as a string."""
out = [
'.. This file is auto-generated by docs/pysim_fs_sphinx.py — do not edit.\n\n',
_heading('Card Filesystem Reference', 0),
'This page documents all Elementary Files (EFs) and Dedicated Files (DFs) '
'implemented in pySim, organised by their location in the card filesystem.\n\n',
]
# Track already-documented classes so that DFs/EFs shared between profiles
# (e.g. DF.TELECOM / DF.GSM present in both CardProfileSIM and CardProfileRUIM)
# are only emitted once.
seen_types: set = set()
for section_title, module_path, class_name in SECTIONS:
module = importlib.import_module(module_path)
cls = getattr(module, class_name)
obj = cls()
if isinstance(obj, CardProfile):
files = obj.files_in_mf
elif isinstance(obj, CardApplication):
files = list(obj.adf.children.values())
elif isinstance(obj, CardDF):
files = list(obj.children.values())
else:
continue
# Filter out files whose class was already documented in an earlier section.
files = [f for f in files if type(f) not in seen_types]
if not files:
continue
out.append(_heading(section_title, 1))
for f in files:
seen_types.add(type(f))
if isinstance(f, CardDF):
out.append(_document_df(f, level=2))
elif isinstance(f, CardEF):
out.append(_document_ef(f))
return ''.join(out)
# ---------------------------------------------------------------------------
# Sphinx integration
# ---------------------------------------------------------------------------
def _on_builder_inited(app):
output_path = os.path.join(app.srcdir, 'filesystem.rst')
with open(output_path, 'w') as fh:
fh.write(generate_filesystem_rst())
def setup(app):
app.connect('builder-inited', _on_builder_inited)
return {'version': '0.1', 'parallel_read_safe': True}

46
docs/remote-access.rst Normal file
View File

@@ -0,0 +1,46 @@
Remote access to an UICC/eUICC
==============================
To access a card with pySim-shell, it is not strictly necessary to have physical
access to it. There are solutions that allow remote access to UICC/eUICC cards.
In this section we will give a brief overview.
osmo-remsim
-----------
osmo-remsim is a suite of software programs enabling physical/geographic
separation of a cellular phone (or modem) on the one hand side and the
UICC/eUICC card on the other side.
Using osmo-remsim, you can operate an entire fleet of modems/phones, as well as
banks of SIM cards and dynamically establish or remove the connections between
modems/phones and cards.
To access remote cards with pySim-shell via osmo-remseim (RSPRO), the
provided libifd_remsim_client would be used to provide a virtual PC/SC reader
on the local machine. pySim-shell can then access this reader like any other
PC/SC reader.
More information on osmo-remsim can be found under:
* https://osmocom.org/projects/osmo-remsim/wiki
* https://ftp.osmocom.org/docs/osmo-remsim/master/osmo-remsim-usermanual.pdf
Android APDU proxy
------------------
Android APDU proxy is an Android app that provides a bridge between a host
computer and the UICC/eUICC slot of an Android smartphone.
The APDU proxy connects to VPCD server that runs on the remote host (in this
case the local machine where pySim-shell is running). The VPCD server then
provides a virtual PC/SC reader, that pySim-shell can access like any other
PC/SC reader.
On the Android side the UICC/eUICC is accessed via OMAPI (Open Mobile API),
which is available in Android since API level Android 8 (API level 29).
More information Android APDU proxy can be found under:
* https://gitea.osmocom.org/sim-card/android-apdu-proxy

137
docs/saip-tool.rst Normal file
View File

@@ -0,0 +1,137 @@
saip-tool
=========
eSIM profiles are stored as a sequence of profile element (PE) objects in an ASN.1 DER encoded binary file. To inspect,
verify or make changes to those files, the `saip-tool.py` utility can be used.
NOTE: The file format, eSIM SAIP (SimAlliance Interoperable Profile) is specified in `TCA eUICC Profile Package:
Interoperable Format Technical Specification`
Profile Package Examples
~~~~~~~~~~~~~~~~~~~~~~~~
pySim ships with a set of TS48 profile package examples. Those examples can be found in `pysim/smdpp-data/upp`. The
files can be used as input for `saip-tool.py`. (see also GSMA TS.48 - Generic eUICC Test Profile for Device Testing)
See also: https://github.com/GSMATerminals/Generic-eUICC-Test-Profile-for-Device-Testing-Public
JAVA card applets
~~~~~~~~~~~~~~~~~
The `saip-tool.py` can also be used to manage JAVA-card applets (Application PE) inside a profile package. The user has
the option to add, remove and inspect applications and their instances. In the following we will discuss a few JAVA-card
related use-cases of `saip-tool.py`
NOTE: see also `contrib` folder for script examples (`saip-tool_example_*.sh`)
Inserting applications
----------------------
An application is usually inserted in two steps. In the first step, the application PE is created and populated with
the executable code from a provided `.cap` or `.ijc` file. The user also has to pick a suitable load block AID.
The application instance, which exists inside the application PE, is created in a second step. Here the user must
reference the load block AID and pick, among other application related parameters, a suitable class and instance AID.
Example: Adding a JAVA-card applet to an existing profile package
::
# Step #1: Create the application PE and load the ijc contents from the .cap file:
$ ./contrib/saip-tool.py upp.der add-app --output-file upp_with_app.der --applet-file app.cap --aid '1122334455'
Read 28 PEs from file 'upp.der'
Applying applet file: 'app.cap'...
application PE inserted into PE Sequence after securityDomain PE AID: a000000151000000
Writing 29 PEs to file 'upp_with_app.der'...
# Step #2: Create the application instance inside the application PE created in step #1:
$ ./contrib/saip-tool.py upp_with_app.der add-app-inst --output-file upp_with_app_and_instance.der \
--aid '1122334455' \
--class-aid '112233445501' \
--inst-aid '112233445501' \
--app-privileges '00' \
--app-spec-pars '00' \
--uicc-toolkit-app-spec-pars '01001505000000000000000000000000'
Read 29 PEs from file 'upp_with_app.der'
Found Load Package AID: 1122334455, adding new instance AID: 112233445501 to Application PE...
Writing 29 PEs to file 'upp_with_app_and_instance.der'...
NOTE: The parameters of the sub-commands `add-app` and `add-app-inst` are application specific. It is up to the application
developer to pick parameters that suit the application correctly. For an exact command reference see section
`saip-tool syntax`. For parameter details see `TCA eUICC Profile Package: Interoperable Format Technical Specification`,
section 8.7 and ETSI TS 102 226, section 8.2.1.3.2
Inspecting applications
-----------------------
To inspect the application PE contents of an existing profile package, sub-command `info` with parameter '--apps' can
be used. This command lists out all application and their parameters in detail. This allows an application developer
to check if the applet insertion was carried out as expected.
Example: Listing applications and their parameters
::
$ ./contrib/saip-tool.py upp_with_app_and_instance.der info --apps
Read 29 PEs from file 'upp_with_app_and_instance.der'
Application #0:
loadBlock:
loadPackageAID: '1122334455' (5 bytes)
loadBlockObject: '01000fdecaffed010204000105d07002ca440200...681080056810a00633b44104b431066800a10231' (569 bytes)
instanceList[0]:
applicationLoadPackageAID: '1122334455' (5 bytes)
classAID: '112233445501' (8 bytes)
instanceAID: '112233445501' (8 bytes)
applicationPrivileges: '00' (1 bytes)
lifeCycleState: '07' (1 bytes)
applicationSpecificParametersC9: '00' (1 bytes)
applicationParameters:
uiccToolkitApplicationSpecificParametersField: '01001505000000000000000000000000' (16 bytes)
In case further analysis with external tools or transfer of applications from one profile package to another is
necessary, the executable code in the `loadBlockObject` field can be extracted to an `.ijc` or an `.cap` file.
Example: Extracting applications from a profile package
::
$ ./contrib/saip-tool.py upp_with_app_and_instance.der extract-apps --output-dir ./apps --format ijc
Read 29 PEs from file 'upp_with_app_and_instance.der'
Writing Load Package AID: 1122334455 to file ./apps/8949449999999990023f-1122334455.ijc
Removing applications
---------------------
An application PE can be removed using sub-command `remove-app`. The user passes the load package AID as parameter. Then
`saip-tool.py` will search for the related application PE and delete it from the PE sequence.
Example: Remove an application from a profile package
::
$ ./contrib/saip-tool.py upp_with_app_and_instance.der remove-app --output-file upp_without_app.der --aid '1122334455'
Read 29 PEs from file 'upp_with_app_and_instance.der'
Found Load Package AID: 1122334455, removing related PE (id=23) from Sequence...
Removing PE application (id=23) from Sequence...
Writing 28 PEs to file 'upp_without_app.der'...
In some cases it is useful to remove only an instance from an existing application PE. This may be the case when the
an application developer wants to modify parameters of an application by removing and re-adding the instance. The
operation basically rolls the state back to step 1 explained in section :ref:`Inserting applications`
Example: Remove an application instance from an application PE
::
$ ./contrib/saip-tool.py upp_with_app_and_instance.der remove-app-inst --output-file upp_without_app.der --aid '1122334455' --inst-aid '112233445501'
Read 29 PEs from file 'upp_with_app_and_instance.der'
Found Load Package AID: 1122334455, removing instance AID: 112233445501 from Application PE...
Removing instance from Application PE...
Writing 29 PEs to file 'upp_with_app.der'...
saip-tool syntax
~~~~~~~~~~~~~~~~
.. argparse::
:module: contrib.saip-tool
:func: parser
:prog: contrib/saip-tool.py

1183
docs/shell.rst
View File

File diff suppressed because it is too large Load Diff

118
docs/sim-rest.rst Normal file
View File

@@ -0,0 +1,118 @@
sim-rest-server
===============
Sometimes there are use cases where a [remote] application will need
access to a USIM for authentication purposes. This is, for example, in
case an IMS test client needs to perform USIM based authentication
against an IMS core.
The pysim repository contains two programs: `sim-rest-server.py` and
`sim-rest-client.py` that implement a simple approach to achieve the
above:
`sim-rest-server.py` speaks to a [usually local] USIM via the PC/SC
API and provides a high-level REST API towards [local or remote]
applications that wish to perform UMTS AKA using the USIM.
`sim-rest-client.py` implements a small example client program to
illustrate how the REST API provided by `sim-rest-server.py` can be
used.
REST API Calls
--------------
POST /sim-auth-api/v1/slot/SLOT_NR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
where SLOT_NR is the integer-encoded slot number (corresponds to PC/SC
reader number). When using a single sysmoOCTSIM board, this is in the range of 0..7
Example: `/sim-auth-api/v1/slot/0` for the first slot.
Request Body
############
The request body is a JSON document, comprising of
1. the RAND and AUTN parameters as hex-encoded string
2. the application against which to authenticate (USIM, ISIM)
Example:
::
{
"rand": "bb685a4b2fc4d697b9d6a129dd09a091",
"autn": "eea7906f8210000004faf4a7df279b56"
}
HTTP Status Codes
#################
HTTP status codes are used to represent errors within the REST server
and the SIM reader hardware. They are not used to communicate protocol
level errors reported by the SIM Card. An unsuccessful authentication
will hence have a `200 OK` HTTP Status code and then encode the SIM
specific error information in the Response Body.
====== =========== ================================
Status Code Description
------ ----------- --------------------------------
200 OK Successful execution
400 Bad Request Request body is malformed
404 Not Found Specified SIM Slot doesn't exist
410 Gone No SIM card inserted in slot
====== =========== ================================
Response Body
#############
The response body is a JSON document, either
#. a successful outcome; encoding RES, CK, IK as hex-encoded string
#. a sync failure; encoding AUTS as hex-encoded string
#. errors
#. authentication error (incorrect MAC)
#. authentication error (security context not supported)
#. key freshness failure
#. unspecified card error
Example (success):
::
{
"successful_3g_authentication": {
"res": "b15379540ec93985",
"ck": "713fde72c28cbd282a4cd4565f3d6381",
"ik": "2e641727c95781f1020d319a0594f31a",
"kc": "771a2c995172ac42"
}
}
Example (re-sync case):
::
{
"synchronisation_failure": {
"auts": "dc2a591fe072c92d7c46ecfe97e5"
}
}
Concrete example using the included sysmoISIM-SJA2
--------------------------------------------------
This was tested using SIMs ending in IMSI numbers 45890...45899
The following command were executed successfully:
Slot 0
::
$ /usr/local/src/pysim/contrib/sim-rest-client.py -c 1 -n 0 -k 841EAD87BC9D974ECA1C167409357601 -o 3211CACDD64F51C3FD3013ECD9A582A0
-> {'rand': 'fb195c7873b20affa278887920b9dd57', 'autn': 'd420895a6aa2000089cd016f8d8ae67c'}
<- {'successful_3g_authentication': {'res': '131004db2ff1ce8e', 'ck': 'd42eb5aa085307903271b2422b698bad', 'ik': '485f81e6fd957fe3cad374adf12fe1ca', 'kc': '64d3f2a32f801214'}}
Slot 1
::
$ /usr/local/src/pysim/contrib/sim-rest-client.py -c 1 -n 1 -k 5C2CE9633FF9B502B519A4EACD16D9DF -o 9834D619E71A02CD76F00CC7AA34FB32
-> {'rand': '433dc5553db95588f1d8b93870930b66', 'autn': '126bafdcbe9e00000026a208da61075d'}
<- {'successful_3g_authentication': {'res': '026d7ac42d379207', 'ck': '83a90ba331f47a95c27a550b174c4a1f', 'ik': '31e1d10329ffaf0ca1684a1bf0b0a14a', 'kc': 'd15ac5b0fff73ecc'}}

179
docs/smpp-ota-tool.rst Normal file
View File

@@ -0,0 +1,179 @@
smpp-ota-tool
=============
The `smpp-ota-tool` allows users to send OTA SMS messages containing APDU scripts (RFM, RAM) via an SMPP server. The
intended audience are developers who want to test/evaluate the OTA SMS interface of a SIM/UICC/eUICC. `smpp-ota-tool`
is intended to be used as a companion tool for :ref:`pySim-smpp2sim`, however it should be usable on any other SMPP
server (such as a production SMSC of a live cellular network) as well.
From the technical perspective `smpp-ota-tool` takes the role of an SMPP ESME. It takes care of the encoding, encryption
and checksumming (signing) of the RFM/RAM OTA SMS and eventually submits it to the SMPP server. The program then waits
for a response. The response is automatically parsed and printed on stdout. This makes the program also suitable to be
called from shell scripts.
.. note:: In the following we will we will refer to `SIM` as one of the following: `SIM`, `USIM`, `ISIM`, `UICC`,
`eUICC`, `eSIM`.
Applying OTA keys
~~~~~~~~~~~~~~~~~
Depending on the `SIM` type you will receive one or more sets of keys which you can use to communicate with the `SIM`
through a secure channel protocol. When using the OTA SMS method, the SCP80 protocol is used and it therefore crucial
to use a keyset that is actually suitable for SCP80.
A keyset usually consists of three keys:
#. KIC: the key used for ciphering (encryption/decryption)
#. KID: the key used to compute a cryptographic checksum (signing)
#. KIK: the key used to encrypt/decrypt key material (key rotation, adding of new keys)
From the transport security perspective, only KIC and KID are relevant. The KIK (also referenced as "Data Encryption
Key", DEK) is only used when keys are rotated or new keys are added (see also ETSI TS 102 226, section 8.2.1.5).
When the keyset is programmed into the security domain of the `SIM`, it is tied to a specific cryptographic algorithm
(3DES, AES128 or AES256) and a so called Key Version Number (KVN). The term "Key Version Number" is misleading, since
it is actually not a version number. It is a unique identifier of a certain keyset which also identifies for which
secure channel protocol the keyset may be used. Keysets with a KVN from 1-15 (``0x01``-``0x0F``) are suitable for SCP80.
This means that it is not only important to know just the KIC/KID/KIK keys. Also the related algorithms and the KVN
numbers must be known.
.. note:: SCP80 keysets typically start counting from 1 upwards. Typical configurations use a set of 3 keysets with
KVN numbers 1-3.
Addressing an Application
~~~~~~~~~~~~~~~~~~~~~~~~~
When communicating with a specific application on a `SIM` via SCP80, it is important to address that application with
the correct parameters. The following two parameters must be known in advance:
#. TAR: The Toolkit Application Reference (TAR) number is a three byte value that uniquely addresses an application
on the `SIM`. The exact values may vary (see also ETSI TS 101 220, Table D.1).
#. MSL: The Minimum Security Level (MSL) is a bit-field that dictates which of the security measures encoded in the
SPI are mandatory (see also ETSI TS 102 225, section 5.1.1).
A practical example
~~~~~~~~~~~~~~~~~~~
.. note:: This tutorial assumes that pySim-smpp2sim is running on the local machine with its default parameters.
See also :ref:`pySim-smpp2sim`.
Let's assume that an OTA SMS shall be sent to the SIM RFM application of an sysmoISIM-SJA2. What we want to do is to
select DF.GSM and to get the select response back.
We have received the following key material from the `SIM` vendor:
::
KIC1: F09C43EE1A0391665CC9F05AF4E0BD10
KID1: 01981F4A20999F62AF99988007BAF6CA
KIK1: 8F8AEE5CDCC5D361368BC45673D99195
KIC2: 01022916E945B656FDE03F806A105FA2
KID2: D326CB69F160333CC5BD1495D448EFD6
KIK2: 08037E0590DFE049D4975FFB8652F625
KIC3: 2B22824D0D27A3A1CEEC512B312082B4
KID3: F1697766925A11F4458295590137B672
KIK3: C7EE69B2C5A1C8E160DD36A38EB517B3
Those are three keysets. The enumeration is directly equal to the KVN used. All three keysets are 3DES keys, which
means triple_des_cbc2 is the correct algorithm to use.
.. note:: The key set configuration can be confirmed by retrieving the key configuration using
`get_data key_information` from within an SCP02 session on ADF.ISD.
In this example we intend to address the SIM RFM application on the `SIM`. Which according to the manual has TAR ``B00010``
and MSL ``0x06``. When we hold ``0x06`` = ``0b00000110`` against the SPI coding chart (see also ETSI TS 102 225,
section 5.1.1). We can deduct that Ciphering and Cryptographic Checksum are mandatory.
.. note:: The MSL (see also ETSI TS 102 226, section 6.1) is assigned to an application by the `SIM` issuer. It is a
custom decision and may vary with different `SIM` types/profiles. In the case of sysmoISIM-SJS1/SJA2/SJA5 the
counter requirement has been waived to simplify lab/research type use. In productive environments, `SIM`
applications should ideally use an MSL that makes the counter mandatory.
In order to select DF.GSM (``0x7F20``) and to retrieve the select response, two APDUs are needed. The first APDU is the
select command ``A0A40000027F20`` and the second is the related get-response command ``A0C0000016``. Those APDUs will be
concatenated and are sent in a single message. The message containing the concatenated APDUs works as a script that
is received by the SIM RFM application and then executed. This method poses some limitations that have to be taken into
account when making requests like this (see also ETSI TS 102 226, section 5).
With this information we may now construct a commandline for `smpp-ota-tool.py`. We will pass the KVN as kid_idx and
kic_idx (see also ETSI TS 102 225, Table 2, fields `KIc` and `KID`). Both index values should refer to the same
keyset/KVN as keysets should not be mixed. (`smpp-ota-tool` still provides separate parameters anyway to allow testing
with invalid keyset combinations)
::
$ PYTHONPATH=./ ./contrib/smpp-ota-tool.py --kic F09C43EE1A0391665CC9F05AF4E0BD10 --kid 01981F4A20999F62AF99988107BAF6CA --kid_idx 1 --kic_idx 1 --algo-crypt triple_des_cbc2 --algo-auth triple_des_cbc2 --tar B00010 --apdu A0A40000027F20 --apdu A0C0000016
2026-02-26 17:13:56 INFO Connecting to localhost:2775...
2026-02-26 17:13:56 INFO C-APDU sending: a0a40000027f20a0c0000016...
2026-02-26 17:13:56 INFO SMS-TPDU sending: 02700000281506191515b00010da1d6cbbd0d11ce4330d844c7408340943e843f67a6d7b0674730881605fd62d...
2026-02-26 17:13:56 INFO SMS-TPDU sent, waiting for response...
2026-02-26 17:13:56 INFO SMS-TPDU received: 027100002c12b000107ddf58d1780f771638b3975759f4296cf5c31efc87a16a1b61921426baa16da1b5ba1a9951d59a39
2026-02-26 17:13:56 INFO SMS-TPDU decoded: (Container(rpl=44, rhl=18, tar=b'\xb0\x00\x10', cntr=b'\x00\x00\x00\x00\x00', pcntr=0, response_status=uEnumIntegerString.new(0, 'por_ok'), cc_rc=b'\x8f\xea\xf5.\xf4\x0e\xc2\x14', secured_data=b'\x02\x90\x00\x00\x00\xff\xff\x7f \x02\x00\x00\x00\x00\x00\t\xb1\x065\x04\x00\x83\x8a\x83\x8a'), Container(number_of_commands=2, last_status_word=u'9000', last_response_data=u'0000ffff7f2002000000000009b106350400838a838a'))
2026-02-26 17:13:56 INFO R-APDU received: 0000ffff7f2002000000000009b106350400838a838a 9000
0000ffff7f2002000000000009b106350400838a838a 9000
2026-02-26 17:13:56 INFO Disconnecting...
The result we see is the select response of DF.GSM and a status word indicating that the last command has been
processed normally.
As we can see, this mechanism now allows us to perform small administrative tasks remotely. We can read the contents of
files remotely or make changes to files. Depending on the changes we make, there may be security issues arising from
replay attacks. With the commandline above, the communication is encrypted and protected by a cryptographic checksum,
so an adversary can neither read, nor alter the message. However, an adversary could still replay an intercepted
message and the `SIM` would happily execute the contained APDUs again.
To prevent this, we may include a replay protection counter within the message. In this case, the MSL indicates that a
replay protection counter is not required. However, to extended the security of our messages, we may chose to use a
counter anyway. In the following example, we will encode a counter value of 100. We will instruct the `SIM` to make sure
that the value we send is higher than the counter value that is currently stored in the `SIM`.
To add a replay connection counter we add the commandline arguments `--cntr-req` to set the counter requirement and
`--cntr` to pass the counter value.
::
$ PYTHONPATH=./ ./contrib/smpp-ota-tool.py --kic F09C43EE1A0391665CC9F05AF4E0BD10 --kid 01981F4A20999F62AF99988107BAF6CA --kid_idx 1 --kic_idx 1 --algo-crypt triple_des_cbc2 --algo-auth triple_des_cbc2 --tar B00010 --apdu A0A40000027F20 --apdu A0C0000016 --cntr-req counter_must_be_higher --cntr 100
2026-02-26 17:16:39 INFO Connecting to localhost:2775...
2026-02-26 17:16:39 INFO C-APDU sending: a0a40000027f20a0c0000016...
2026-02-26 17:16:39 INFO SMS-TPDU sending: 02700000281516191515b000103a4f599e94f2b5dcfbbda984761b7977df6514c57a580fb4844787c436d2eade...
2026-02-26 17:16:39 INFO SMS-TPDU sent, waiting for response...
2026-02-26 17:16:39 INFO SMS-TPDU received: 027100002c12b0001049fb0315f6c6401b553867f412cefaf9355b38271178edb342a3bc9cc7e670cdc1f45eea6ffcbb39
2026-02-26 17:16:39 INFO SMS-TPDU decoded: (Container(rpl=44, rhl=18, tar=b'\xb0\x00\x10', cntr=b'\x00\x00\x00\x00d', pcntr=0, response_status=uEnumIntegerString.new(0, 'por_ok'), cc_rc=b'\xa9/\xc7\xc9\x00"\xab5', secured_data=b'\x02\x90\x00\x00\x00\xff\xff\x7f \x02\x00\x00\x00\x00\x00\t\xb1\x065\x04\x00\x83\x8a\x83\x8a'), Container(number_of_commands=2, last_status_word=u'9000', last_response_data=u'0000ffff7f2002000000000009b106350400838a838a'))
2026-02-26 17:16:39 INFO R-APDU received: 0000ffff7f2002000000000009b106350400838a838a 9000
0000ffff7f2002000000000009b106350400838a838a 9000
2026-02-26 17:16:39 INFO Disconnecting...
The `SIM` has accepted the message. The message got processed and the `SIM` has set its internal to 100. As an experiment,
we may try to re-use the counter value:
::
$ PYTHONPATH=./ ./contrib/smpp-ota-tool.py --kic F09C43EE1A0391665CC9F05AF4E0BD10 --kid 01981F4A20999F62AF99988107BAF6CA --kid_idx 1 --kic_idx 1 --algo-crypt triple_des_cbc2 --algo-auth triple_des_cbc2 --tar B00010 --apdu A0A40000027F20 --apdu A0C0000016 --cntr-req counter_must_be_higher --cntr 100
2026-02-26 17:16:43 INFO Connecting to localhost:2775...
2026-02-26 17:16:43 INFO C-APDU sending: a0a40000027f20a0c0000016...
2026-02-26 17:16:43 INFO SMS-TPDU sending: 02700000281516191515b000103a4f599e94f2b5dcfbbda984761b7977df6514c57a580fb4844787c436d2eade...
2026-02-26 17:16:43 INFO SMS-TPDU sent, waiting for response...
2026-02-26 17:16:43 INFO SMS-TPDU received: 027100000b0ab0001000000000000006
2026-02-26 17:16:43 INFO SMS-TPDU decoded: (Container(rpl=11, rhl=10, tar=b'\xb0\x00\x10', cntr=b'\x00\x00\x00\x00\x00', pcntr=0, response_status=uEnumIntegerString.new(6, 'undefined_security_error'), cc_rc=b'', secured_data=b''), None)
Traceback (most recent call last):
File "/home/user/work/git_master/pysim/./contrib/smpp-ota-tool.py", line 238, in <module>
resp, sw = smpp_handler.transceive_apdu(apdu, opts.src_addr, opts.dest_addr, opts.timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/work/git_master/pysim/./contrib/smpp-ota-tool.py", line 162, in transceive_apdu
raise ValueError("Response does not contain any last_response_data, no R-APDU received!")
ValueError: Response does not contain any last_response_data, no R-APDU received!
2026-02-26 17:16:43 INFO Disconnecting...
As we can see, the `SIM` has rejected the message with an `undefined_security_error`. The replay-protection-counter
ensures that a message can only be sent once.
.. note:: The replay-protection-counter is implemented as a 5 byte integer value (see also ETSI TS 102 225, Table 3).
When the counter has reached its maximum, it will not overflow nor can it be reset.
smpp-ota-tool syntax
~~~~~~~~~~~~~~~~~~~~
.. argparse::
:module: contrib.smpp-ota-tool
:func: option_parser
:prog: contrib/smpp-ota-tool.py

59
docs/smpp2sim.rst Normal file
View File

@@ -0,0 +1,59 @@
pySim-smpp2sim
==============
This is a program to emulate the entire communication path SMSC-CN-RAN-ME
that is usually between an OTA backend and the SIM card. This allows
to play with SIM OTA technology without using a mobile network or even
a mobile phone.
An external application can act as SMPP ESME and must encode (and
encrypt/sign) the OTA SMS and submit them via SMPP to this program, just
like it would submit it normally to a SMSC (SMS Service Centre). The
program then re-formats the SMPP-SUBMIT into a SMS DELIVER TPDU and
passes it via an ENVELOPE APDU to the SIM card that is locally inserted
into a smart card reader.
The path from SIM to external OTA application works the opposite way.
The default SMPP system_id is `test`. Likewise, the default SMPP
password is `test`
Running pySim-smpp2sim
----------------------
The command accepts the same command line arguments for smart card interface device selection as pySim-shell,
as well as a few SMPP specific arguments:
.. argparse::
:module: pySim-smpp2sim
:func: option_parser
:prog: pySim-smpp2sim.py
Example execution with sample output
------------------------------------
So for a simple system with a single PC/SC device, you would typically use something like
`./pySim-smpp2sim.py -p0` to start the program. You will see output like this at start-up
::
Using reader PCSC[HID Global OMNIKEY 3x21 Smart Card Reader [OMNIKEY 3x21 Smart Card Reader] 00 00]
INFO root: Binding Virtual SMSC to TCP Port 2775 at ::
The application has hence bound to local TCP port 2775 and expects your SMS-sending applications to send their
SMS there. Once you do, you will see log output like below:
::
WARNING smpp.twisted.protocol: SMPP connection established from ::ffff:127.0.0.1 to port 2775
INFO smpp.twisted.server: Added CommandId.bind_transceiver bind for 'test'. Active binds: CommandId.bind_transceiver: 1, CommandId.bind_transmitter: 0, CommandId.bind_receiver: 0. Max binds: 2
INFO smpp.twisted.protocol: Bind request succeeded for test. 1 active binds
And once your external program is sending SMS to the simulated SMSC, it will log something like
::
INFO root: SMS_DELIVER(MTI=0, MMS=False, LP=False, RP=False, UDHI=True, SRI=False, OA=AddressField(TON=international, NPI=unknown, 12), PID=7f, DCS=f6, SCTS=bytearray(b'"pR\x00\x00\x00\x00'), UDL=45, UD=b"\x02p\x00\x00(\x15\x16\x19\x12\x12\xb0\x00\x01'\xfa(\xa5\xba\xc6\x9d<^\x9d\xf2\xc7\x15]\xfd\xdeD\x9c\x82k#b\x15Ve0x{0\xe8\xbe]")
SMSPPDownload(DeviceIdentities({'source_dev_id': 'network', 'dest_dev_id': 'uicc'}),Address({'ton_npi': 0, 'call_number': '0123456'}),SMS_TPDU({'tpdu': '400290217ff6227052000000002d02700000281516191212b0000127fa28a5bac69d3c5e9df2c7155dfdde449c826b236215566530787b30e8be5d'}))
INFO root: ENVELOPE: d147820283818604001032548b3b400290217ff6227052000000002d02700000281516191212b0000127fa28a5bac69d3c5e9df2c7155dfdde449c826b236215566530787b30e8be5d
INFO root: SW 9000: 027100002412b000019a551bb7c28183652de0ace6170d0e563c5e949a3ba56747fe4c1dbbef16642c
.. note:: for sending OTA SMS messages :ref:`smpp-ota-tool` may be used.

58
docs/suci-keytool.rst Normal file
View File

@@ -0,0 +1,58 @@
suci-keytool
============
Subscriber concealment is an important feature of the 5G SA architecture: It avoids the many privacy
issues associated with having a permanent identifier (SUPI, traditionally the IMSI) transmitted in plain text
over the air interface. Using SUCI solves this issue not just for the air interface; it even ensures the SUPI/IMSI
is not known to the visited network (VPLMN) at all.
In principle, the SUCI mechanism works by encrypting the SUPI by asymmetric (public key) cryptography:
Only the HPLMN is in possession of the private key and hence can decrypt the SUCI to the SUPI, while
each subscriber has the public key in order to encrypt their SUPI into the SUCI. In reality, the
details are more complex, as there are ephemeral keys and cryptographic MAC involved.
In any case, in order to operate a SUCI-enabled 5G SA network, you will have to
#. generate a ECC key pair of public + private key
#. deploy the public key on your USIMs
#. deploy the private key on your 5GC, specifically the UDM function
pysim contains (in its `contrib` directory) a small utility program that can make it easy to generate
such keys: `suci-keytool.py`
Generating keys
~~~~~~~~~~~~~~~
Example: Generating a *secp256r1* ECC public key pair and storing it to `/tmp/suci.key`:
::
$ ./contrib/suci-keytool.py --key-file /tmp/suci.key generate-key --curve secp256r1
Dumping public keys
~~~~~~~~~~~~~~~~~~~
In order to store the key to SIM cards as part of `ADF.USIM/DF.5GS/EF.SUCI_Calc_Info`, you will need
a hexadecimal representation of the public key. You can achieve that using the `dump-pub-key` operation
of suci-keytool:
Example: Dumping the public key part from a previously generated key file:
::
$ ./contrib/suci-keytool.py --key-file /tmp/suci.key dump-pub-key
0473152f32523725f5175d255da2bd909de97b1d06449a9277bc629fe42112f8643e6b69aa6dce6c86714ccbe6f2e0f4f4898d102e2b3f0c18ce26626f052539bb
If you want the point-compressed representation, you can use the `--compressed` option:
::
$ ./contrib/suci-keytool.py --key-file /tmp/suci.key dump-pub-key --compressed
0373152f32523725f5175d255da2bd909de97b1d06449a9277bc629fe42112f864
suci-keytool syntax
~~~~~~~~~~~~~~~~~~~
.. argparse::
:module: contrib.suci-keytool
:func: arg_parser
:prog: contrib/suci-keytool.py

270
docs/suci-tutorial.rst Normal file
View File

@@ -0,0 +1,270 @@
Guide: Enabling 5G SUCI
=======================
SUPI/SUCI Concealment is a feature of 5G-Standalone (SA) to encrypt the
IMSI/SUPI with a network operator public key. 3GPP Specifies two different
variants for this:
* SUCI calculation *in the UE*, using key data from the SIM
* SUCI calculation *on the card itself*
pySim supports writing the 5G-specific files for *SUCI calculation in the UE* on USIM cards, assuming
that your cards contain the required files, and you have the privileges/credentials to write to them.
This is the case using sysmocom sysmoISIM-SJA2 or any flavor of sysmoISIM-SJA5.
There is no 3GPP/ETSI standard method for configuring *SUCI calculation on the card*; pySim currently
supports the vendor-specific method for the sysmoISIM-SJA5-S17).
This document describes both methods.
Technical References
~~~~~~~~~~~~~~~~~~~~
This guide covers the basic workflow of provisioning SIM cards with the 5G SUCI feature. For detailed information on the SUCI feature and file contents, the following documents are helpful:
* USIM files and structure: `3GPP TS 31.102 <https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/16.06.00_60/ts_131102v160600p.pdf>`__
* USIM tests (incl. file content examples): `3GPP TS 31.121 <https://www.etsi.org/deliver/etsi_ts/131100_131199/131121/16.01.00_60/ts_131121v160100p.pdf>`__
* Test keys for SUCI calculation: `3GPP TS 33.501 <https://www.etsi.org/deliver/etsi_ts/133500_133599/133501/16.05.00_60/ts_133501v160500p.pdf>`__
For specific information on sysmocom SIM cards, refer to
* the `sysmoISIM-SJA5 User Manual <https://sysmocom.de/manuals/sysmoisim-sja5-manual.pdf>`__ for the current
sysmoISIM-SJA5 product
* the `sysmoISIM-SJA2 User Manual <https://sysmocom.de/manuals/sysmousim-manual.pdf>`__ for the older
sysmoISIM-SJA2 product
--------------
Enabling 5G SUCI *calculated in the UE*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In short, you can enable *SUCI calculation in the UE* with these steps:
* activate USIM **Service 124**
* make sure USIM **Service 125** is disabled
* store the public keys in **EF.SUCI_Calc_Info**
* set the **Routing Indicator** (required)
If you want to disable the feature, you can just disable USIM Service 124 (and 125) in `EF.UST`.
Admin PIN
---------
The usual way to authenticate yourself to the card as the cellular
operator is to validate the so-called ADM1 (admin) PIN. This may differ
from card model/vendor to card model/vendor.
Start pySIM-shell and enter the admin PIN for your card. If you bought
the SIM card from your network operator and don’t have the admin PIN,
you cannot change SIM contents!
Launch pySIM:
::
$ ./pySim-shell.py -p 0
Using PC/SC reader interface
Autodetected card type: sysmoISIM-SJA2
Welcome to pySim-shell!
pySIM-shell (00:MF)>
Enter the ADM PIN:
::
pySIM-shell (00:MF)> verify_adm XXXXXXXX
Otherwise, write commands will fail with ``SW Mismatch: Expected 9000 and got 6982.``
Key Provisioning
----------------
::
pySIM-shell (00:MF)> select MF
pySIM-shell (00:MF)> select ADF.USIM
pySIM-shell (00:MF/ADF.USIM)> select DF.5GS
pySIM-shell (00:MF/ADF.USIM/DF.5GS)> select EF.SUCI_Calc_Info
By default, the file is present but empty:
::
pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.SUCI_Calc_Info)> read_binary_decoded
missing Protection Scheme Identifier List data object tag
9000: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff -> {}
The following JSON config defines the testfile from 3GPP TS 31.121, Section 4.9.4 with
test keys from 3GPP TS 33.501, Annex C.4. Highest priority (``0``) has a
Profile-B (``identifier: 2``) key in key slot ``1``, which means the key
with ``hnet_pubkey_identifier: 27``.
.. code:: json
{
"prot_scheme_id_list": [
{"priority": 0, "identifier": 2, "key_index": 1},
{"priority": 1, "identifier": 1, "key_index": 2},
{"priority": 2, "identifier": 0, "key_index": 0}],
"hnet_pubkey_list": [
{"hnet_pubkey_identifier": 27,
"hnet_pubkey": "0472DA71976234CE833A6907425867B82E074D44EF907DFB4B3E21C1C2256EBCD15A7DED52FCBB097A4ED250E036C7B9C8C7004C4EEDC4F068CD7BF8D3F900E3B4"},
{"hnet_pubkey_identifier": 30,
"hnet_pubkey": "5A8D38864820197C3394B92613B20B91633CBD897119273BF8E4A6F4EEC0A650"}]
}
Write the config to file (must be single-line input as for now):
::
pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.SUCI_Calc_Info)> update_binary_decoded '{ "prot_scheme_id_list": [ {"priority": 0, "identifier": 2, "key_index": 1}, {"priority": 1, "identifier": 1, "key_index": 2}, {"priority": 2, "identifier": 0, "key_index": 0}], "hnet_pubkey_list": [ {"hnet_pubkey_identifier": 27, "hnet_pubkey": "0472DA71976234CE833A6907425867B82E074D44EF907DFB4B3E21C1C2256EBCD15A7DED52FCBB097A4ED250E036C7B9C8C7004C4EEDC4F068CD7BF8D3F900E3B4"}, {"hnet_pubkey_identifier": 30, "hnet_pubkey": "5A8D38864820197C3394B92613B20B91633CBD897119273BF8E4A6F4EEC0A650"}]}'
WARNING: These are TEST KEYS with publicly known/specified private keys, and hence unsafe for live/secure
deployments! For use in production networks, you need to generate your own set[s] of keys.
Routing Indicator
-----------------
The Routing Indicator must be present for the SUCI feature. By default,
the contents of the file is **invalid** (ffffffff):
::
pySIM-shell (00:MF)> select MF
pySIM-shell (00:MF)> select ADF.USIM
pySIM-shell (00:MF/ADF.USIM)> select DF.5GS
pySIM-shell (00:MF/ADF.USIM/DF.5GS)> select EF.Routing_Indicator
pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.Routing_Indicator)> read_binary_decoded
9000: ffffffff -> {'raw': 'ffffffff'}
The Routing Indicator is a four-byte file but the actual Routing
Indicator goes into bytes 0 and 1 (the other bytes are reserved). To set
the Routing Indicator to 0x71:
::
pySIM-shell (00:MF/ADF.USIM/DF.5GS/EF.Routing_Indicator)> update_binary 17ffffff
You can also set the routing indicator to **0x0**, which is *valid* and
means “routing indicator not specified”, leaving it to the modem.
USIM Service Table
------------------
First, check out the USIM Service Table (UST):
::
pySIM-shell (00:MF)> select MF
pySIM-shell (00:MF)> select ADF.USIM
pySIM-shell (00:MF/ADF.USIM)> select EF.UST
pySIM-shell (00:MF/ADF.USIM/EF.UST)> read_binary_decoded
9000: beff9f9de73e0408400170730000002e00000000 -> [2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 25, 27, 28, 29, 33, 34, 35, 38, 39, 42, 43, 44, 45, 46, 51, 60, 71, 73, 85, 86, 87, 89, 90, 93, 94, 95, 122, 123, 124, 126]
.. list-table:: From 3GPP TS 31.102
:widths: 15 40
:header-rows: 1
* - Service No.
- Description
* - 122
- 5GS Mobility Management Information
* - 123
- 5G Security Parameters
* - 124
- Subscription identifier privacy support
* - 125
- SUCI calculation by the USIM
* - 126
- UAC Access Identities support
* - 129
- 5GS Operator PLMN List
If you’d like to enable/disable any UST service:
::
pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_deactivate 124
pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_activate 124
pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_deactivate 125
In this case, UST Service 124 is already enabled and you’re good to go. The
sysmoISIM-SJA2 does not support on-SIM calculation, so service 125 must
be disabled.
USIM Error with 5G and sysmoISIM
--------------------------------
sysmoISIM-SJA2 come 5GS-enabled. By default however, the configuration stored
in the card file-system is **not valid** for 5G networks: Service 124 is enabled,
but EF.SUCI_Calc_Info and EF.Routing_Indicator are empty files (hence
do not contain valid data).
At least for Qualcomm’s X55 modem, this results in an USIM error and the
whole modem shutting 5G down. If you don’t need SUCI concealment but the
smartphone refuses to connect to any 5G network, try to disable the UST
service 124.
sysmoISIM-SJA5 are shipped with a more forgiving default, with valid EF.Routing_Indicator
contents and disabled Service 124
SUCI calculation by the USIM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The SUCI calculation can also be performed by the USIM application on the UICC
directly. The UE then uses the GET IDENTITY command (see also 3GPP TS 31.102,
section 7.5) to retrieve a SUCI value.
The sysmoISIM-SJA5-S17 supports *SUCI calculation by the USIM*. The configuration
is not much different to the above described configuration of *SUCI calculation
in the UE*.
The main difference is how the key provisioning is done. When the SUCI
calculation is done by the USIM, then the key material is not accessed by the
UE. The specification (see also 3GPP TS 31.102, section 7.5.1.1), also does not
specify any file or file format to store the key material. This means the exact
way to perform the key provisioning is an implementation detail of the USIM
card application.
In the case of sysmoISIM-SJA5-S17, the key material for *SUCI calculation by the USIM* is stored in
`ADF.USIM/DF.SAIP/EF.SUCI_Calc_Info` (**not** in `ADF.USIM/DF.5GS/EF.SUCI_Calc_Info`!).
::
pySIM-shell (00:MF)> select MF
pySIM-shell (00:MF)> select ADF.USIM
pySIM-shell (00:MF/ADF.USIM)> select DF.SAIP
pySIM-shell (00:MF/ADF.USIM/DF.SAIP)> select EF.SUCI_Calc_Info
The file format is exactly the same as specified in 3GPP TS 31.102, section
4.4.11.8. This means the above described key provisioning procedure can be
applied without any changes, except that the file location is different.
To signal to the UE that the USIM is setup up for SUCI calculation, service
125 must be enabled in addition to service 124 (see also 3GPP TS 31.102,
section 5.3.48)
::
pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_activate 124
pySIM-shell (00:MF/ADF.USIM/EF.UST)> ust_service_activate 125
To verify that the SUCI calculation works as expected, it is possible to issue
a GET IDENTITY command using pySim-shell:
::
select ADF.USIM
get_identity
The USIM should then return a SUCI TLV Data object that looks like this:
::
SUCI TLV Data Object: 0199f90717ff021b027a2c58ce1c6b89df088a9eb4d242596dd75746bb5f3503d2cf58a7461e4fd106e205c86f76544e9d732226a4e1

64
docs/trace.rst Normal file
View File

@@ -0,0 +1,64 @@
pySim-trace
===========
pySim-trace is a utility for high-level decode of APDU protocol traces such as those obtained with
`Osmocom SIMtrace2 <https://osmocom.org/projects/simtrace2/wiki>`_ or `osmo-qcdiag <https://osmocom.org/projects/osmo-qcdiag/wiki>`_.
pySim-trace leverages the existing knowledge of pySim-shell on anything related to SIM cards,
including the structure/encoding of the various files on SIM/USIM/ISIM/HPSIM cards, and applies this
to decoding protocol traces. This means that it shows not only the name of the command (like READ
BINARY), but actually understands what the currently selected file is, and how to decode the
contents of that file.
pySim-trace also understands the parameters passed to commands and how to decode them, for example
of the AUTHENTICATE command within the USIM/ISIM/HPSIM application.
Demo
----
To get an idea how pySim-trace usage looks like, you can watch the relevant part of the 11/2022
SIMtrace2 tutorial whose `recording is freely accessible <https://media.ccc.de/v/osmodevcall-20221019-laforge-simtrace2-tutorial#t=2134>`_.
Running pySim-trace
-------------------
Running pySim-trace requires you to specify the *source* of the to-be-decoded APDUs. There are several
supported options, each with their own respective parameters (like a file name for PCAP decoding).
See the detailed command line reference below for details.
A typical execution of pySim-trace for doing live decodes of *GSMTAP (SIM APDU)* e.g. from SIMtrace2 or
osmo-qcdiag would look like this:
::
./pySim-trace.py gsmtap-udp
This binds to the default UDP port 4729 (GSMTAP) on localhost (127.0.0.1), and decodes any APDUs received
there.
pySim-trace command line reference
----------------------------------
.. argparse::
:module: pySim-trace
:func: option_parser
:prog: pySim-trace.py
Constraints
-----------
* In order to properly track the current location in the filesystem tree and other state, it is
important that the trace you're decoding includes all of the communication with the SIM, ideally
from the very start (power up).
* pySim-trace currently only supports ETSI UICC (USIM/ISIM/HPSIM) and doesn't yet support legacy GSM
SIM. This is not a fundamental technical constraint, it's just simply that nobody got around
developing and testing that part. Contributions are most welcome.

913
osmo-smdpp.py Executable file
View File

@@ -0,0 +1,913 @@
#!/usr/bin/env python3
# Early proof-of-concept towards a SM-DP+ HTTP service for GSMA consumer eSIM RSP
#
# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# asn1tools issue https://github.com/eerimoq/asn1tools/issues/194
# must be first here
import asn1tools
import asn1tools.codecs.ber
import asn1tools.codecs.der
# do not move the code
def fix_asn1_oid_decoding():
fix_asn1_schema = """
TestModule DEFINITIONS ::= BEGIN
TestOid ::= SEQUENCE {
oid OBJECT IDENTIFIER
}
END
"""
fix_asn1_asn1 = asn1tools.compile_string(fix_asn1_schema, codec='der')
fix_asn1_oid_string = '2.999.10'
fix_asn1_encoded = fix_asn1_asn1.encode('TestOid', {'oid': fix_asn1_oid_string})
fix_asn1_decoded = fix_asn1_asn1.decode('TestOid', fix_asn1_encoded)
if (fix_asn1_decoded['oid'] != fix_asn1_oid_string):
# ASN.1 OBJECT IDENTIFIER Decoding Issue:
#
# In ASN.1 BER/DER encoding, the first two arcs of an OBJECT IDENTIFIER are
# combined into a single value: (40 * arc0) + arc1. This is encoded as a base-128
# variable-length quantity (and commonly known as VLQ or base-128 encoding)
# as specified in ITU-T X.690 §8.19, it can span multiple bytes if
# the value is large.
#
# For arc0 = 0 or 1, arc1 must be in [0, 39]. For arc0 = 2, arc1 can be any non-negative integer.
# All subsequent arcs (arc2, arc3, ...) are each encoded as a separate base-128 VLQ.
#
# The decoding bug occurs when the decoder does not properly split the first
# subidentifier for arc0 = 2 and arc1 >= 40. Instead of decoding:
# - arc0 = 2
# - arc1 = (first_subidentifier - 80)
# it may incorrectly interpret the first_subidentifier as arc0 = (first_subidentifier // 40),
# arc1 = (first_subidentifier % 40), which is only valid for arc1 < 40.
#
# This patch handles it properly for all valid OBJECT IDENTIFIERs
# with large second arcs, by applying the ASN.1 rules:
# - if first_subidentifier < 40: arc0 = 0, arc1 = first_subidentifier
# - elif first_subidentifier < 80: arc0 = 1, arc1 = first_subidentifier - 40
# - else: arc0 = 2, arc1 = first_subidentifier - 80
#
# This problem is not uncommon, see for example https://github.com/randombit/botan/issues/4023
def fixed_decode_object_identifier(data, offset, end_offset):
"""Decode ASN.1 OBJECT IDENTIFIER from bytes to dotted string, fixing large second arc handling."""
def read_subidentifier(data, offset):
value = 0
while True:
b = data[offset]
value = (value << 7) | (b & 0x7F)
offset += 1
if not (b & 0x80):
break
return value, offset
subid, offset = read_subidentifier(data, offset)
if subid < 40:
first = 0
second = subid
elif subid < 80:
first = 1
second = subid - 40
else:
first = 2
second = subid - 80
arcs = [first, second]
while offset < end_offset:
subid, offset = read_subidentifier(data, offset)
arcs.append(subid)
return '.'.join(str(x) for x in arcs)
asn1tools.codecs.ber.decode_object_identifier = fixed_decode_object_identifier
asn1tools.codecs.der.decode_object_identifier = fixed_decode_object_identifier
# test our patch
asn1 = asn1tools.compile_string(fix_asn1_schema, codec='der')
decoded = asn1.decode('TestOid', fix_asn1_encoded)['oid']
assert fix_asn1_oid_string == str(decoded)
fix_asn1_oid_decoding()
from cryptography.hazmat.primitives.asymmetric.utils import encode_dss_signature # noqa: E402
from cryptography import x509 # noqa: E402
from cryptography.exceptions import InvalidSignature # noqa: E402
from cryptography.hazmat.primitives import hashes # noqa: E402
from cryptography.hazmat.primitives.asymmetric import ec, dh # noqa: E402
from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat, PrivateFormat, NoEncryption, ParameterFormat # noqa: E402
from pathlib import Path # noqa: E402
import json # noqa: E402
import sys # noqa: E402
import argparse # noqa: E402
import uuid # noqa: E402
import os # noqa: E402
import functools # noqa: E402
from typing import Optional, Dict, List # noqa: E402
from pprint import pprint as pp # noqa: E402
import base64 # noqa: E402
from base64 import b64decode # noqa: E402
from klein import Klein # noqa: E402
from twisted.web.iweb import IRequest # noqa: E402
from osmocom.utils import h2b, b2h, swap_nibbles # noqa: E402
import pySim.esim.rsp as rsp # noqa: E402
from pySim.esim import saip, PMO # noqa: E402
from pySim.esim.es8p import ProfileMetadata,UnprotectedProfilePackage,ProtectedProfilePackage,BoundProfilePackage,BspInstance # noqa: E402
from pySim.esim.x509_cert import oid, cert_policy_has_oid, cert_get_auth_key_id # noqa: E402
from pySim.esim.x509_cert import CertAndPrivkey, CertificateSet, cert_get_subject_key_id, VerifyError # noqa: E402
import logging # noqa: E402
logger = logging.getLogger(__name__)
# HACK: make this configurable
DATA_DIR = './smdpp-data'
HOSTNAME = 'testsmdpplus1.example.com' # must match certificates!
def b64encode2str(req: bytes) -> str:
"""Encode given input bytes as base64 and return result as string."""
return base64.b64encode(req).decode('ascii')
def set_headers(request: IRequest):
"""Set the request headers as mandatory by GSMA eSIM RSP."""
request.setHeader('Content-Type', 'application/json;charset=UTF-8')
request.setHeader('X-Admin-Protocol', 'gsma/rsp/v2.1.0')
def validate_request_headers(request: IRequest):
"""Validate mandatory HTTP headers according to SGP.22."""
content_type = request.getHeader('Content-Type')
if not content_type or not content_type.startswith('application/json'):
raise ApiError('1.2.1', '2.1', 'Invalid Content-Type header')
admin_protocol = request.getHeader('X-Admin-Protocol')
if admin_protocol and not admin_protocol.startswith('gsma/rsp/v'):
raise ApiError('1.2.2', '2.1', 'Unsupported X-Admin-Protocol version')
def get_eum_certificate_variant(eum_cert) -> str:
"""Determine EUM certificate variant by checking Certificate Policies extension.
Returns 'O' for old variant, or 'NEW' for Ov3/A/B/C variants."""
try:
cert_policies_ext = eum_cert.extensions.get_extension_for_oid(
x509.oid.ExtensionOID.CERTIFICATE_POLICIES
)
for policy in cert_policies_ext.value:
policy_oid = policy.policy_identifier.dotted_string
logger.debug(f"Found certificate policy: {policy_oid}")
if policy_oid == '2.23.146.1.2.1.2':
logger.debug("Detected EUM certificate variant: O (old)")
return 'O'
elif policy_oid == '2.23.146.1.2.1.0.0.0':
logger.debug("Detected EUM certificate variant: Ov3/A/B/C (new)")
return 'NEW'
except x509.ExtensionNotFound:
logger.debug("No Certificate Policies extension found")
except Exception as e:
logger.debug(f"Error checking certificate policies: {e}")
def parse_permitted_eins_from_cert(eum_cert) -> List[str]:
"""Extract permitted IINs from EUM certificate using the appropriate method
based on certificate variant (O vs Ov3/A/B/C).
Returns list of permitted IINs (basically prefixes that valid EIDs must start with)."""
# Determine certificate variant first
cert_variant = get_eum_certificate_variant(eum_cert)
permitted_iins = []
if cert_variant == 'O':
# Old variant - use nameConstraints extension
permitted_iins.extend(_parse_name_constraints_eins(eum_cert))
else:
# New variants (Ov3, A, B, C) - use GSMA permittedEins extension
permitted_iins.extend(_parse_gsma_permitted_eins(eum_cert))
unique_iins = list(set(permitted_iins))
logger.debug(f"Total unique permitted IINs found: {len(unique_iins)}")
return unique_iins
def _parse_gsma_permitted_eins(eum_cert) -> List[str]:
"""Parse the GSMA permittedEins extension using correct ASN.1 structure.
PermittedEins ::= SEQUENCE OF PrintableString
Each string contains an IIN (Issuer Identification Number) - a prefix of valid EIDs."""
permitted_iins = []
try:
permitted_eins_oid = x509.ObjectIdentifier('2.23.146.1.2.2.0') # sgp26: 2.23.146.1.2.2.0 = ASN1:SEQUENCE:permittedEins
for ext in eum_cert.extensions:
if ext.oid == permitted_eins_oid:
logger.debug(f"Found GSMA permittedEins extension: {ext.oid}")
# Get the DER-encoded extension value
ext_der = ext.value.value if hasattr(ext.value, 'value') else ext.value
if isinstance(ext_der, bytes):
try:
permitted_eins_schema = """
PermittedEins DEFINITIONS ::= BEGIN
PermittedEins ::= SEQUENCE OF PrintableString
END
"""
decoder = asn1tools.compile_string(permitted_eins_schema)
decoded_strings = decoder.decode('PermittedEins', ext_der)
for iin_string in decoded_strings:
# Each string contains an IIN -> prefix of euicc EID
iin_clean = iin_string.strip().upper()
# IINs is 8 chars per sgp22, var len according to sgp29, fortunately we don't care
if (len(iin_clean) == 8 and
all(c in '0123456789ABCDEF' for c in iin_clean) and
len(iin_clean) % 2 == 0):
permitted_iins.append(iin_clean)
logger.debug(f"Found permitted IIN (GSMA): {iin_clean}")
else:
logger.debug(f"Invalid IIN format: {iin_string} (cleaned: {iin_clean})")
except Exception as e:
logger.debug(f"Error parsing GSMA permittedEins extension: {e}")
except Exception as e:
logger.debug(f"Error accessing GSMA certificate extensions: {e}")
return permitted_iins
def _parse_name_constraints_eins(eum_cert) -> List[str]:
"""Parse permitted IINs from nameConstraints extension (variant O)."""
permitted_iins = []
try:
# Look for nameConstraints extension
name_constraints_ext = eum_cert.extensions.get_extension_for_oid(
x509.oid.ExtensionOID.NAME_CONSTRAINTS
)
name_constraints = name_constraints_ext.value
# Check permittedSubtrees for IIN constraints
if name_constraints.permitted_subtrees:
for subtree in name_constraints.permitted_subtrees:
if isinstance(subtree, x509.DirectoryName):
for attribute in subtree.value:
# IINs for O in serialNumber
if attribute.oid == x509.oid.NameOID.SERIAL_NUMBER:
serial_value = attribute.value.upper()
# sgp22 8, sgp29 var len, fortunately we don't care
if (len(serial_value) == 8 and
all(c in '0123456789ABCDEF' for c in serial_value) and
len(serial_value) % 2 == 0):
permitted_iins.append(serial_value)
logger.debug(f"Found permitted IIN (nameConstraints/DN): {serial_value}")
except x509.ExtensionNotFound:
logger.debug("No nameConstraints extension found")
except Exception as e:
logger.debug(f"Error parsing nameConstraints: {e}")
return permitted_iins
def validate_eid_range(eid: str, eum_cert) -> bool:
"""Validate that EID is within the permitted EINs of the EUM certificate."""
if not eid or len(eid) != 32:
logger.debug(f"Invalid EID format: {eid}")
return False
try:
permitted_eins = parse_permitted_eins_from_cert(eum_cert)
if not permitted_eins:
logger.debug("Warning: No permitted EINs found in EUM certificate")
return False
eid_normalized = eid.upper()
logger.debug(f"Validating EID {eid_normalized} against {len(permitted_eins)} permitted EINs")
for permitted_ein in permitted_eins:
if eid_normalized.startswith(permitted_ein):
logger.debug(f"EID {eid_normalized} matches permitted EIN {permitted_ein}")
return True
logger.debug(f"EID {eid_normalized} is not in any permitted EIN list")
return False
except Exception as e:
logger.debug(f"Error validating EID: {e}")
return False
def build_status_code(subject_code: str, reason_code: str, subject_id: Optional[str], message: Optional[str]) -> Dict:
r = {'subjectCode': subject_code, 'reasonCode': reason_code }
if subject_id:
r['subjectIdentifier'] = subject_id
if message:
r['message'] = message
return r
def build_resp_header(js: dict, status: str = 'Executed-Success', status_code_data = None) -> None:
# SGP.22 v3.0 6.5.1.4
js['header'] = {
'functionExecutionStatus': {
'status': status,
}
}
if status_code_data:
js['header']['functionExecutionStatus']['statusCodeData'] = status_code_data
def ecdsa_tr03111_to_dss(sig: bytes) -> bytes:
"""convert an ECDSA signature from BSI TR-03111 format to DER: first get long integers; then encode those."""
assert len(sig) == 64
r = int.from_bytes(sig[0:32], 'big')
s = int.from_bytes(sig[32:32*2], 'big')
return encode_dss_signature(r, s)
class ApiError(Exception):
def __init__(self, subject_code: str, reason_code: str, message: Optional[str] = None,
subject_id: Optional[str] = None):
self.status_code = build_status_code(subject_code, reason_code, subject_id, message)
def encode(self) -> str:
"""Encode the API Error into a responseHeader string."""
js = {}
build_resp_header(js, 'Failed', self.status_code)
return json.dumps(js)
class SmDppHttpServer:
app = Klein()
@staticmethod
def load_certs_from_path(path: str) -> List[x509.Certificate]:
"""Load all DER + PEM files from given directory path and return them as list of x509.Certificate
instances."""
certs = []
for dirpath, dirnames, filenames in os.walk(path):
for filename in filenames:
cert = None
if filename.endswith('.der'):
with open(os.path.join(dirpath, filename), 'rb') as f:
cert = x509.load_der_x509_certificate(f.read())
elif filename.endswith('.pem'):
with open(os.path.join(dirpath, filename), 'rb') as f:
cert = x509.load_pem_x509_certificate(f.read())
if cert:
# verify it is a CI certificate (keyCertSign + i-rspRole-ci)
if not cert_policy_has_oid(cert, oid.id_rspRole_ci):
raise ValueError("alleged CI certificate %s doesn't have CI policy" % filename)
certs.append(cert)
return certs
def ci_get_cert_for_pkid(self, ci_pkid: bytes) -> Optional[x509.Certificate]:
"""Find CI certificate for given key identifier."""
for cert in self.ci_certs:
logger.debug("cert: %s" % cert)
subject_exts = list(filter(lambda x: isinstance(x.value, x509.SubjectKeyIdentifier), cert.extensions))
logger.debug(subject_exts)
subject_pkid = subject_exts[0].value
logger.debug(subject_pkid)
if subject_pkid and subject_pkid.key_identifier == ci_pkid:
return cert
return None
def validate_certificate_chain_for_verification(self, euicc_ci_pkid_list: List[bytes]) -> bool:
"""Validate that SM-DP+ has valid certificate chains for the given CI PKIDs."""
for ci_pkid in euicc_ci_pkid_list:
ci_cert = self.ci_get_cert_for_pkid(ci_pkid)
if ci_cert:
# Check if our DPauth certificate chains to this CI
try:
cs = CertificateSet(ci_cert)
cs.verify_cert_chain(self.dp_auth.cert)
return True
except VerifyError:
continue
return False
def __init__(self, server_hostname: str, ci_certs_path: str, common_cert_path: str, use_brainpool: bool = False, in_memory: bool = False):
self.server_hostname = server_hostname
self.upp_dir = os.path.realpath(os.path.join(DATA_DIR, 'upp'))
self.ci_certs = self.load_certs_from_path(ci_certs_path)
# load DPauth cert + key
self.dp_auth = CertAndPrivkey(oid.id_rspRole_dp_auth_v2)
cert_dir = common_cert_path
if use_brainpool:
self.dp_auth.cert_from_der_file(os.path.join(cert_dir, 'DPauth', 'CERT_S_SM_DPauth_ECDSA_BRP.der'))
self.dp_auth.privkey_from_pem_file(os.path.join(cert_dir, 'DPauth', 'SK_S_SM_DPauth_ECDSA_BRP.pem'))
else:
self.dp_auth.cert_from_der_file(os.path.join(cert_dir, 'DPauth', 'CERT_S_SM_DPauth_ECDSA_NIST.der'))
self.dp_auth.privkey_from_pem_file(os.path.join(cert_dir, 'DPauth', 'SK_S_SM_DPauth_ECDSA_NIST.pem'))
# load DPpb cert + key
self.dp_pb = CertAndPrivkey(oid.id_rspRole_dp_pb_v2)
if use_brainpool:
self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_BRP.der'))
self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_BRP.pem'))
else:
self.dp_pb.cert_from_der_file(os.path.join(cert_dir, 'DPpb', 'CERT_S_SM_DPpb_ECDSA_NIST.der'))
self.dp_pb.privkey_from_pem_file(os.path.join(cert_dir, 'DPpb', 'SK_S_SM_DPpb_ECDSA_NIST.pem'))
if in_memory:
self.rss = rsp.RspSessionStore(in_memory=True)
logger.info("Using in-memory session storage")
else:
# Use different session database files for BRP and NIST to avoid file locking during concurrent runs
session_db_suffix = "BRP" if use_brainpool else "NIST"
db_path = os.path.join(DATA_DIR, f"sm-dp-sessions-{session_db_suffix}")
self.rss = rsp.RspSessionStore(filename=db_path, in_memory=False)
logger.info(f"Using file-based session storage: {db_path}")
@app.handle_errors(ApiError)
def handle_apierror(self, request: IRequest, failure):
request.setResponseCode(200)
pp(failure)
return failure.value.encode()
@staticmethod
def _ecdsa_verify(cert: x509.Certificate, signature: bytes, data: bytes) -> bool:
pubkey = cert.public_key()
dss_sig = ecdsa_tr03111_to_dss(signature)
try:
pubkey.verify(dss_sig, data, ec.ECDSA(hashes.SHA256()))
return True
except InvalidSignature:
return False
@staticmethod
def rsp_api_wrapper(func):
"""Wrapper that can be used as decorator in order to perform common REST API endpoint entry/exit
functionality, such as JSON decoding/encoding and debug-printing."""
@functools.wraps(func)
def _api_wrapper(self, request: IRequest):
validate_request_headers(request)
content = json.loads(request.content.read())
logger.debug("Rx JSON: %s" % json.dumps(content))
set_headers(request)
output = func(self, request, content)
if output == None:
return ''
build_resp_header(output)
logger.debug("Tx JSON: %s" % json.dumps(output))
return json.dumps(output)
return _api_wrapper
@app.route('/gsma/rsp2/es9plus/initiateAuthentication', methods=['POST'])
@rsp_api_wrapper
def initiateAutentication(self, request: IRequest, content: dict) -> dict:
"""See ES9+ InitiateAuthentication SGP.22 Section 5.6.1"""
# Verify that the received address matches its own SM-DP+ address, where the comparison SHALL be
# case-insensitive. Otherwise, the SM-DP+ SHALL return a status code "SM-DP+ Address - Refused".
if content['smdpAddress'] != self.server_hostname:
raise ApiError('8.8.1', '3.8', 'Invalid SM-DP+ Address')
euiccChallenge = b64decode(content['euiccChallenge'])
if len(euiccChallenge) != 16:
raise ValueError
euiccInfo1_bin = b64decode(content['euiccInfo1'])
euiccInfo1 = rsp.asn1.decode('EUICCInfo1', euiccInfo1_bin)
logger.debug("Rx euiccInfo1: %s" % euiccInfo1)
#euiccInfo1['svn']
# TODO: If euiccCiPKIdListForSigningV3 is present ...
pkid_list = euiccInfo1['euiccCiPKIdListForSigning']
if 'euiccCiPKIdListForSigningV3' in euiccInfo1:
pkid_list = pkid_list + euiccInfo1['euiccCiPKIdListForSigningV3']
# Validate that SM-DP+ supports certificate chains for verification
verification_pkid_list = euiccInfo1.get('euiccCiPKIdListForVerification', [])
if verification_pkid_list and not self.validate_certificate_chain_for_verification(verification_pkid_list):
raise ApiError('8.8.4', '3.7', 'The SM-DP+ has no CERT.DPauth.SIG which chains to one of the eSIM CA Root CA Certificate with a Public Key supported by the eUICC')
# verify it supports one of the keys indicated by euiccCiPKIdListForSigning
ci_cert = None
for x in pkid_list:
ci_cert = self.ci_get_cert_for_pkid(x)
# we already support multiple CI certificates but only one set of DPauth + DPpb keys. So we must
# make sure we choose a CI key-id which has issued both the eUICC as well as our own SM-DP side
# certs.
if ci_cert and cert_get_subject_key_id(ci_cert) == self.dp_auth.get_authority_key_identifier().key_identifier:
break
else:
ci_cert = None
if not ci_cert:
raise ApiError('8.8.2', '3.1', 'None of the proposed Public Key Identifiers is supported by the SM-DP+')
# Generate a TransactionID which is used to identify the ongoing RSP session. The TransactionID
# SHALL be unique within the scope and lifetime of each SM-DP+.
transactionId = uuid.uuid4().hex.upper()
assert not transactionId in self.rss
# Generate a serverChallenge for eUICC authentication attached to the ongoing RSP session.
serverChallenge = os.urandom(16)
# Generate a serverSigned1 data object as expected by the eUICC and described in section 5.7.13 "ES10b.AuthenticateServer". If and only if both eUICC and LPA indicate crlStaplingV3Support, the SM-DP+ SHALL indicate crlStaplingV3Used in sessionContext.
serverSigned1 = {
'transactionId': h2b(transactionId),
'euiccChallenge': euiccChallenge,
'serverAddress': self.server_hostname,
'serverChallenge': serverChallenge,
}
logger.debug("Tx serverSigned1: %s" % serverSigned1)
serverSigned1_bin = rsp.asn1.encode('ServerSigned1', serverSigned1)
logger.debug("Tx serverSigned1: %s" % rsp.asn1.decode('ServerSigned1', serverSigned1_bin))
output = {}
output['serverSigned1'] = b64encode2str(serverSigned1_bin)
# Generate a signature (serverSignature1) as described in section 5.7.13 "ES10b.AuthenticateServer" using the SK related to the selected CERT.DPauth.SIG.
# serverSignature1 SHALL be created using the private key associated to the RSP Server Certificate for authentication, and verified by the eUICC using the contained public key as described in section 2.6.9. serverSignature1 SHALL apply on serverSigned1 data object.
output['serverSignature1'] = b64encode2str(b'\x5f\x37\x40' + self.dp_auth.ecdsa_sign(serverSigned1_bin))
output['transactionId'] = transactionId
server_cert_aki = self.dp_auth.get_authority_key_identifier()
output['euiccCiPKIdToBeUsed'] = b64encode2str(b'\x04\x14' + server_cert_aki.key_identifier)
output['serverCertificate'] = b64encode2str(self.dp_auth.get_cert_as_der()) # CERT.DPauth.SIG
# FIXME: add those certificate
#output['otherCertsInChain'] = b64encode2str()
# create SessionState and store it in rss
self.rss[transactionId] = rsp.RspSessionState(transactionId, serverChallenge,
cert_get_subject_key_id(ci_cert))
return output
@app.route('/gsma/rsp2/es9plus/authenticateClient', methods=['POST'])
@rsp_api_wrapper
def authenticateClient(self, request: IRequest, content: dict) -> dict:
"""See ES9+ AuthenticateClient in SGP.22 Section 5.6.3"""
transactionId = content['transactionId']
authenticateServerResp_bin = b64decode(content['authenticateServerResponse'])
authenticateServerResp = rsp.asn1.decode('AuthenticateServerResponse', authenticateServerResp_bin)
logger.debug("Rx %s: %s" % authenticateServerResp)
if authenticateServerResp[0] == 'authenticateResponseError':
r_err = authenticateServerResp[1]
#r_err['transactionId']
#r_err['authenticateErrorCode']
raise ValueError("authenticateResponseError %s" % r_err)
r_ok = authenticateServerResp[1]
euiccSigned1 = r_ok['euiccSigned1']
euiccSigned1_bin = rsp.extract_euiccSigned1(authenticateServerResp_bin)
euiccSignature1_bin = r_ok['euiccSignature1']
euiccCertificate_dec = r_ok['euiccCertificate']
# TODO: use original data, don't re-encode?
euiccCertificate_bin = rsp.asn1.encode('Certificate', euiccCertificate_dec)
eumCertificate_dec = r_ok['eumCertificate']
eumCertificate_bin = rsp.asn1.encode('Certificate', eumCertificate_dec)
# TODO v3: otherCertsInChain
# load certificate
euicc_cert = x509.load_der_x509_certificate(euiccCertificate_bin)
eum_cert = x509.load_der_x509_certificate(eumCertificate_bin)
# Verify that the transactionId is known and relates to an ongoing RSP session. Otherwise, the SM-DP+
# SHALL return a status code "TransactionId - Unknown"
ss = self.rss.get(transactionId, None)
if ss is None:
raise ApiError('8.10.1', '3.9', 'Unknown')
ss.euicc_cert = euicc_cert
ss.eum_cert = eum_cert # TODO: do we need this in the state?
# Verify that the Root Certificate of the eUICC certificate chain corresponds to the
# euiccCiPKIdToBeUsed or TODO: euiccCiPKIdToBeUsedV3
if cert_get_auth_key_id(eum_cert) != ss.ci_cert_id:
raise ApiError('8.11.1', '3.9', 'Unknown')
# Verify the validity of the eUICC certificate chain
cs = CertificateSet(self.ci_get_cert_for_pkid(ss.ci_cert_id))
cs.add_intermediate_cert(eum_cert)
# TODO v3: otherCertsInChain
try:
cs.verify_cert_chain(euicc_cert)
except VerifyError:
raise ApiError('8.1.3', '6.1', 'Verification failed (certificate chain)')
# raise ApiError('8.1.3', '6.3', 'Expired')
# Verify euiccSignature1 over euiccSigned1 using pubkey from euiccCertificate.
# Otherwise, the SM-DP+ SHALL return a status code "eUICC - Verification failed"
if not self._ecdsa_verify(euicc_cert, euiccSignature1_bin, euiccSigned1_bin):
raise ApiError('8.1', '6.1', 'Verification failed (euiccSignature1 over euiccSigned1)')
ss.eid = ss.euicc_cert.subject.get_attributes_for_oid(x509.oid.NameOID.SERIAL_NUMBER)[0].value
logger.debug("EID (from eUICC cert): %s" % ss.eid)
# Verify EID is within permitted range of EUM certificate
if not validate_eid_range(ss.eid, eum_cert):
raise ApiError('8.1.4', '6.1', 'EID is not within the permitted range of the EUM certificate')
# Verify that the serverChallenge attached to the ongoing RSP session matches the
# serverChallenge returned by the eUICC. Otherwise, the SM-DP+ SHALL return a status code "eUICC -
# Verification failed".
if euiccSigned1['serverChallenge'] != ss.serverChallenge:
raise ApiError('8.1', '6.1', 'Verification failed (serverChallenge)')
# If ctxParams1 contains a ctxParamsForCommonAuthentication data object, the SM-DP+ Shall [...]
# TODO: We really do a very simplistic job here, this needs to be properly implemented later,
# considering all the various cases, profile state, etc.
iccid_str = None
if euiccSigned1['ctxParams1'][0] == 'ctxParamsForCommonAuthentication':
cpca = euiccSigned1['ctxParams1'][1]
matchingId = cpca.get('matchingId', None)
if not matchingId:
# TODO: check if any pending profile downloads for the EID
raise ApiError('8.2.6', '3.8', 'Refused')
if matchingId:
# look up profile based on matchingID. We simply check if a given file exists for now..
path = os.path.join(self.upp_dir, matchingId) + '.der'
# prevent directory traversal attack
if os.path.commonprefix((os.path.realpath(path),self.upp_dir)) != self.upp_dir:
raise ApiError('8.2.6', '3.8', 'Refused')
if not os.path.isfile(path) or not os.access(path, os.R_OK):
raise ApiError('8.2.6', '3.8', 'Refused')
ss.matchingId = matchingId
with open(path, 'rb') as f:
pes = saip.ProfileElementSequence.from_der(f.read())
iccid_str = b2h(pes.get_pe_for_type('header').decoded['iccid'])
else:
# there's currently no other option in the ctxParams1 choice, so this cannot happen
raise ApiError('1.3.1', '2.2', 'ctxParams1 missing mandatory ctxParamsForCommonAuthentication')
# FIXME: we actually want to perform the profile binding herr, and read the profile metadata from the profile
# Put together profileMetadata + _bin
ss.profileMetadata = ProfileMetadata(iccid_bin=h2b(swap_nibbles(iccid_str)), spn="OsmocomSPN", profile_name=matchingId)
# enable notifications for all operations
for event in ['enable', 'disable', 'delete']:
ss.profileMetadata.add_notification(event, self.server_hostname)
profileMetadata_bin = ss.profileMetadata.gen_store_metadata_request()
# Put together smdpSigned2 + _bin
smdpSigned2 = {
'transactionId': h2b(ss.transactionId),
'ccRequiredFlag': False, # whether the Confirmation Code is required
#'bppEuiccOtpk': None, # whether otPK.EUICC.ECKA already used for binding the BPP, tag '5F49'
}
smdpSigned2_bin = rsp.asn1.encode('SmdpSigned2', smdpSigned2)
ss.smdpSignature2_do = b'\x5f\x37\x40' + self.dp_pb.ecdsa_sign(smdpSigned2_bin + b'\x5f\x37\x40' + euiccSignature1_bin)
# update non-volatile state with updated ss object
self.rss[transactionId] = ss
return {
'transactionId': transactionId,
'profileMetadata': b64encode2str(profileMetadata_bin),
'smdpSigned2': b64encode2str(smdpSigned2_bin),
'smdpSignature2': b64encode2str(ss.smdpSignature2_do),
'smdpCertificate': b64encode2str(self.dp_pb.get_cert_as_der()), # CERT.DPpb.SIG
}
@app.route('/gsma/rsp2/es9plus/getBoundProfilePackage', methods=['POST'])
@rsp_api_wrapper
def getBoundProfilePackage(self, request: IRequest, content: dict) -> dict:
"""See ES9+ GetBoundProfilePackage SGP.22 Section 5.6.2"""
transactionId = content['transactionId']
# Verify that the received transactionId is known and relates to an ongoing RSP session
ss = self.rss.get(transactionId, None)
if not ss:
raise ApiError('8.10.1', '3.9', 'The RSP session identified by the TransactionID is unknown')
prepDownloadResp_bin = b64decode(content['prepareDownloadResponse'])
prepDownloadResp = rsp.asn1.decode('PrepareDownloadResponse', prepDownloadResp_bin)
logger.debug("Rx %s: %s" % prepDownloadResp)
if prepDownloadResp[0] == 'downloadResponseError':
r_err = prepDownloadResp[1]
#r_err['transactionId']
#r_err['downloadErrorCode']
raise ValueError("downloadResponseError %s" % r_err)
r_ok = prepDownloadResp[1]
# Verify the euiccSignature2 computed over euiccSigned2 and smdpSignature2 using the PK.EUICC.SIG attached to the ongoing RSP session
euiccSigned2 = r_ok['euiccSigned2']
euiccSigned2_bin = rsp.extract_euiccSigned2(prepDownloadResp_bin)
if not self._ecdsa_verify(ss.euicc_cert, r_ok['euiccSignature2'], euiccSigned2_bin + ss.smdpSignature2_do):
raise ApiError('8.1', '6.1', 'eUICC signature is invalid')
# not in spec: Verify that signed TransactionID is outer transaction ID
if h2b(transactionId) != euiccSigned2['transactionId']:
raise ApiError('8.10.1', '3.9', 'The signed transactionId != outer transactionId')
# store otPK.EUICC.ECKA in session state
ss.euicc_otpk = euiccSigned2['euiccOtpk']
logger.debug("euiccOtpk: %s" % (b2h(ss.euicc_otpk)))
# Generate a one-time ECKA key pair (ot{PK,SK}.DP.ECKA) using the curve indicated by the Key Parameter
# Reference value of CERT.DPpb.ECDDSA
logger.debug("curve = %s" % self.dp_pb.get_curve())
ss.smdp_ot = ec.generate_private_key(self.dp_pb.get_curve())
# extract the public key in (hopefully) the right format for the ES8+ interface
ss.smdp_otpk = ss.smdp_ot.public_key().public_bytes(Encoding.X962, PublicFormat.UncompressedPoint)
logger.debug("smdpOtpk: %s" % b2h(ss.smdp_otpk))
logger.debug("smdpOtsk: %s" % b2h(ss.smdp_ot.private_bytes(Encoding.DER, PrivateFormat.PKCS8, NoEncryption())))
ss.host_id = b'mahlzeit'
# Generate Session Keys using the CRT, otPK.eUICC.ECKA and otSK.DP.ECKA according to annex G
euicc_public_key = ec.EllipticCurvePublicKey.from_encoded_point(ss.smdp_ot.curve, ss.euicc_otpk)
ss.shared_secret = ss.smdp_ot.exchange(ec.ECDH(), euicc_public_key)
logger.debug("shared_secret: %s" % b2h(ss.shared_secret))
# TODO: Check if this order requires a Confirmation Code verification
# Perform actual protection + binding of profile package (or return pre-bound one)
with open(os.path.join(self.upp_dir, ss.matchingId)+'.der', 'rb') as f:
upp = UnprotectedProfilePackage.from_der(f.read(), metadata=ss.profileMetadata)
# HACK: Use empty PPP as we're still debugging the configureISDP step, and we want to avoid
# cluttering the log with stuff happening after the failure
#upp = UnprotectedProfilePackage.from_der(b'', metadata=ss.profileMetadata)
if False:
# Use random keys
bpp = BoundProfilePackage.from_upp(upp)
else:
# Use session keys
ppp = ProtectedProfilePackage.from_upp(upp, BspInstance(b'\x00'*16, b'\x11'*16, b'\x22'*16))
bpp = BoundProfilePackage.from_ppp(ppp)
# update non-volatile state with updated ss object
self.rss[transactionId] = ss
return {
'transactionId': transactionId,
'boundProfilePackage': b64encode2str(bpp.encode(ss, self.dp_pb)),
}
@app.route('/gsma/rsp2/es9plus/handleNotification', methods=['POST'])
@rsp_api_wrapper
def handleNotification(self, request: IRequest, content: dict) -> dict:
"""See ES9+ HandleNotification in SGP.22 Section 5.6.4"""
# SGP.22 Section 6.3: "A normal notification function execution status (MEP Notification)
# SHALL be indicated by the HTTP status code '204' (No Content) with an empty HTTP response body"
request.setResponseCode(204)
pendingNotification_bin = b64decode(content['pendingNotification'])
pendingNotification = rsp.asn1.decode('PendingNotification', pendingNotification_bin)
logger.debug("Rx %s: %s" % pendingNotification)
if pendingNotification[0] == 'profileInstallationResult':
profileInstallRes = pendingNotification[1]
pird = profileInstallRes['profileInstallationResultData']
transactionId = b2h(pird['transactionId'])
ss = self.rss.get(transactionId, None)
if ss is None:
logger.warning(f"Unable to find session for transactionId: {transactionId}")
return None # Will return HTTP 204 with empty body
profileInstallRes['euiccSignPIR']
# TODO: use original data, don't re-encode?
pird_bin = rsp.asn1.encode('ProfileInstallationResultData', pird)
# verify eUICC signature
if not self._ecdsa_verify(ss.euicc_cert, profileInstallRes['euiccSignPIR'], pird_bin):
raise Exception('ECDSA signature verification failed on notification')
logger.debug("Profile Installation Final Result: %s", pird['finalResult'])
# remove session state
del self.rss[transactionId]
elif pendingNotification[0] == 'otherSignedNotification':
otherSignedNotif = pendingNotification[1]
# TODO: use some kind of partially-parsed original data, don't re-encode?
euiccCertificate_bin = rsp.asn1.encode('Certificate', otherSignedNotif['euiccCertificate'])
eumCertificate_bin = rsp.asn1.encode('Certificate', otherSignedNotif['eumCertificate'])
euicc_cert = x509.load_der_x509_certificate(euiccCertificate_bin)
eum_cert = x509.load_der_x509_certificate(eumCertificate_bin)
ci_cert_id = cert_get_auth_key_id(eum_cert)
# Verify the validity of the eUICC certificate chain
cs = CertificateSet(self.ci_get_cert_for_pkid(ci_cert_id))
cs.add_intermediate_cert(eum_cert)
# TODO v3: otherCertsInChain
cs.verify_cert_chain(euicc_cert)
tbs_bin = rsp.asn1.encode('NotificationMetadata', otherSignedNotif['tbsOtherNotification'])
if not self._ecdsa_verify(euicc_cert, otherSignedNotif['euiccNotificationSignature'], tbs_bin):
raise Exception('ECDSA signature verification failed on notification')
other_notif = otherSignedNotif['tbsOtherNotification']
pmo = PMO.from_bitstring(other_notif['profileManagementOperation'])
eid = euicc_cert.subject.get_attributes_for_oid(x509.oid.NameOID.SERIAL_NUMBER)[0].value
iccid = other_notif.get('iccid', None)
if iccid:
iccid = swap_nibbles(b2h(iccid))
logger.debug("handleNotification: EID %s: %s of %s" % (eid, pmo, iccid))
else:
raise ValueError(pendingNotification)
#@app.route('/gsma/rsp3/es9plus/handleDeviceChangeRequest, methods=['POST']')
#@rsp_api_wrapper
#"""See ES9+ ConfirmDeviceChange in SGP.22 Section 5.6.6"""
# TODO: implement this
@app.route('/gsma/rsp2/es9plus/cancelSession', methods=['POST'])
@rsp_api_wrapper
def cancelSession(self, request: IRequest, content: dict) -> dict:
"""See ES9+ CancelSession in SGP.22 Section 5.6.5"""
logger.debug("Rx JSON: %s" % content)
transactionId = content['transactionId']
# Verify that the received transactionId is known and relates to an ongoing RSP session
ss = self.rss.get(transactionId, None)
if ss is None:
raise ApiError('8.10.1', '3.9', 'The RSP session identified by the transactionId is unknown')
cancelSessionResponse_bin = b64decode(content['cancelSessionResponse'])
cancelSessionResponse = rsp.asn1.decode('CancelSessionResponse', cancelSessionResponse_bin)
logger.debug("Rx %s: %s" % cancelSessionResponse)
if cancelSessionResponse[0] == 'cancelSessionResponseError':
# FIXME: print some error
return
cancelSessionResponseOk = cancelSessionResponse[1]
# TODO: use original data, don't re-encode?
ecsr = cancelSessionResponseOk['euiccCancelSessionSigned']
ecsr_bin = rsp.asn1.encode('EuiccCancelSessionSigned', ecsr)
# Verify the eUICC signature (euiccCancelSessionSignature) using the PK.EUICC.SIG attached to the ongoing RSP session
if not self._ecdsa_verify(ss.euicc_cert, cancelSessionResponseOk['euiccCancelSessionSignature'], ecsr_bin):
raise ApiError('8.1', '6.1', 'eUICC signature is invalid')
# Verify that the received smdpOid corresponds to the one in SM-DP+ CERT.DPauth.SIG
subj_alt_name = self.dp_auth.get_subject_alt_name()
if x509.ObjectIdentifier(ecsr['smdpOid']) != subj_alt_name.oid:
raise ApiError('8.8', '3.10', 'The provided SM-DP+ OID is invalid.')
if ecsr['transactionId'] != h2b(transactionId):
raise ApiError('8.10.1', '3.9', 'The signed transactionId != outer transactionId')
# TODO: 1. Notify the Operator using the function "ES2+.HandleNotification" function
# TODO: 2. Terminate the corresponding pending download process.
# TODO: 3. If required, execute the SM-DS Event Deletion procedure described in section 3.6.3.
# delete actual session data
del self.rss[transactionId]
return { 'transactionId': transactionId }
def main(argv):
parser = argparse.ArgumentParser()
parser.add_argument("-H", "--host", help="Host/IP to bind HTTP(S) to", default="localhost")
parser.add_argument("-p", "--port", help="TCP port to bind HTTP(S) to", default=443)
parser.add_argument("-c", "--certdir", help=f"cert subdir relative to {DATA_DIR}", default="certs")
parser.add_argument("-s", "--nossl", help="disable built in SSL/TLS support", action='store_true', default=False)
parser.add_argument("-v", "--verbose", help="dump more raw info", action='store_true', default=False)
parser.add_argument("-b", "--brainpool", help="Use Brainpool curves instead of NIST",
action='store_true', default=False)
parser.add_argument("-m", "--in-memory", help="Use ephermal in-memory session storage (for concurrent runs)",
action='store_true', default=False)
args = parser.parse_args()
logging.basicConfig(level=logging.DEBUG if args.verbose else logging.WARNING)
common_cert_path = os.path.join(DATA_DIR, args.certdir)
hs = SmDppHttpServer(server_hostname=HOSTNAME, ci_certs_path=os.path.join(common_cert_path, 'CertificateIssuer'), common_cert_path=common_cert_path, use_brainpool=args.brainpool)
if(args.nossl):
hs.app.run(args.host, args.port)
else:
curve_type = 'BRP' if args.brainpool else 'NIST'
cert_derpath = Path(common_cert_path) / 'DPtls' / f'CERT_S_SM_DP_TLS_{curve_type}.der'
cert_pempath = Path(common_cert_path) / 'DPtls' / f'CERT_S_SM_DP_TLS_{curve_type}.pem'
cert_skpath = Path(common_cert_path) / 'DPtls' / f'SK_S_SM_DP_TLS_{curve_type}.pem'
dhparam_path = Path(common_cert_path) / "dhparam2048.pem"
if not dhparam_path.exists():
print("Generating dh params, this takes a few seconds..")
# Generate DH parameters with 2048-bit key size and generator 2
parameters = dh.generate_parameters(generator=2, key_size=2048)
pem_data = parameters.parameter_bytes(encoding=Encoding.PEM,format=ParameterFormat.PKCS3)
with open(dhparam_path, 'wb') as file:
file.write(pem_data)
print("DH params created successfully")
if not cert_pempath.exists():
print("Translating tls server cert from DER to PEM..")
with open(cert_derpath, 'rb') as der_file:
der_cert_data = der_file.read()
cert = x509.load_der_x509_certificate(der_cert_data)
pem_cert = cert.public_bytes(Encoding.PEM) #.decode('utf-8')
with open(cert_pempath, 'wb') as pem_file:
pem_file.write(pem_cert)
SERVER_STRING = f'ssl:{args.port}:privateKey={cert_skpath}:certKey={cert_pempath}:dhParameters={dhparam_path}'
print(SERVER_STRING)
hs.app.run(host=HOSTNAME, port=args.port, endpoint_description=SERVER_STRING)
if __name__ == "__main__":
main(sys.argv)

283
pySim-prog.py
View File

@@ -25,185 +25,173 @@
#
import hashlib
from optparse import OptionParser
import argparse
import os
import random
import re
import sys
import traceback
import json
import csv
from osmocom.utils import h2b, swap_nibbles, rpad
from pySim.commands import SimCardCommands
from pySim.transport import init_reader
from pySim.cards import _cards_classes, card_detect
from pySim.utils import h2b, swap_nibbles, rpad, derive_milenage_opc, calculate_luhn, dec_iccid
from pySim.ts_51_011 import EF, EF_AD
from pySim.transport import init_reader, argparse_add_reader_args
from pySim.legacy.cards import _cards_classes, card_detect
from pySim.utils import derive_milenage_opc, calculate_luhn, dec_iccid
from pySim.ts_51_011 import EF_AD
from pySim.legacy.ts_51_011 import EF
from pySim.card_handler import *
from pySim.utils import *
from pathlib import Path
import logging
from pySim.log import PySimLogger
log = PySimLogger.get(Path(__file__).stem)
def parse_options():
parser = OptionParser(usage="usage: %prog [options]")
parser = argparse.ArgumentParser()
argparse_add_reader_args(parser)
parser.add_option("-d", "--device", dest="device", metavar="DEV",
help="Serial Device for SIM access [default: %default]",
default="/dev/ttyUSB0",
)
parser.add_option("-b", "--baud", dest="baudrate", type="int", metavar="BAUD",
help="Baudrate used for SIM access [default: %default]",
default=9600,
)
parser.add_option("-p", "--pcsc-device", dest="pcsc_dev", type='int', metavar="PCSC",
help="Which PC/SC reader number for SIM access",
default=None,
)
parser.add_option("--modem-device", dest="modem_dev", metavar="DEV",
help="Serial port of modem for Generic SIM Access (3GPP TS 27.007)",
default=None,
)
parser.add_option("--modem-baud", dest="modem_baud", type="int", metavar="BAUD",
help="Baudrate used for modem's port [default: %default]",
default=115200,
)
parser.add_option("--osmocon", dest="osmocon_sock", metavar="PATH",
help="Socket path for Calypso (e.g. Motorola C1XX) based reader (via OsmocomBB)",
default=None,
)
parser.add_option("-t", "--type", dest="type",
help="Card type (user -t list to view) [default: %default]",
parser.add_argument("-t", "--type", dest="type",
help="Card type (user -t list to view) [default: %(default)s]",
default="auto",
)
parser.add_option("-T", "--probe", dest="probe",
parser.add_argument("-T", "--probe", dest="probe",
help="Determine card type",
default=False, action="store_true"
)
parser.add_option("-a", "--pin-adm", dest="pin_adm",
parser.add_argument("-a", "--pin-adm", dest="pin_adm",
help="ADM PIN used for provisioning (overwrites default)",
)
parser.add_option("-A", "--pin-adm-hex", dest="pin_adm_hex",
parser.add_argument("-A", "--pin-adm-hex", dest="pin_adm_hex",
help="ADM PIN used for provisioning, as hex string (16 characters long",
)
parser.add_option("-e", "--erase", dest="erase", action='store_true',
help="Erase beforehand [default: %default]",
parser.add_argument("-e", "--erase", dest="erase", action='store_true',
help="Erase beforehand [default: %(default)s]",
default=False,
)
parser.add_option("-S", "--source", dest="source",
help="Data Source[default: %default]",
parser.add_argument("-S", "--source", dest="source",
help="Data Source[default: %(default)s]",
default="cmdline",
)
# if mode is "cmdline"
parser.add_option("-n", "--name", dest="name",
help="Operator name [default: %default]",
parser.add_argument("-n", "--name", dest="name",
help="Operator name [default: %(default)s]",
default="Magic",
)
parser.add_option("-c", "--country", dest="country", type="int", metavar="CC",
help="Country code [default: %default]",
parser.add_argument("-c", "--country", dest="country", type=int, metavar="CC",
help="Country code [default: %(default)s]",
default=1,
)
parser.add_option("-x", "--mcc", dest="mcc", type="string",
help="Mobile Country Code [default: %default]",
parser.add_argument("-x", "--mcc", dest="mcc",
help="Mobile Country Code [default: %(default)s]",
default="901",
)
parser.add_option("-y", "--mnc", dest="mnc", type="string",
help="Mobile Network Code [default: %default]",
parser.add_argument("-y", "--mnc", dest="mnc",
help="Mobile Network Code [default: %(default)s]",
default="55",
)
parser.add_option("--mnclen", dest="mnclen", type="choice",
help="Length of Mobile Network Code [default: %default]",
default=2,
choices=[2, 3],
parser.add_argument("--mnclen", dest="mnclen",
help="Length of Mobile Network Code [default: %(default)s]",
default="auto",
choices=["2", "3", "auto"],
)
parser.add_option("-m", "--smsc", dest="smsc",
parser.add_argument("-m", "--smsc", dest="smsc",
help="SMSC number (Start with + for international no.) [default: '00 + country code + 5555']",
)
parser.add_option("-M", "--smsp", dest="smsp",
parser.add_argument("-M", "--smsp", dest="smsp",
help="Raw SMSP content in hex [default: auto from SMSC]",
)
parser.add_option("-s", "--iccid", dest="iccid", metavar="ID",
parser.add_argument("-s", "--iccid", dest="iccid", metavar="ID",
help="Integrated Circuit Card ID",
)
parser.add_option("-i", "--imsi", dest="imsi",
parser.add_argument("-i", "--imsi", dest="imsi",
help="International Mobile Subscriber Identity",
)
parser.add_option("--msisdn", dest="msisdn",
parser.add_argument("--msisdn", dest="msisdn",
help="Mobile Subscriber Integrated Services Digital Number",
)
parser.add_option("-k", "--ki", dest="ki",
parser.add_argument("-k", "--ki", dest="ki",
help="Ki (default is to randomize)",
)
parser.add_option("-o", "--opc", dest="opc",
parser.add_argument("-o", "--opc", dest="opc",
help="OPC (default is to randomize)",
)
parser.add_option("--op", dest="op",
parser.add_argument("--op", dest="op",
help="Set OP to derive OPC from OP and KI",
)
parser.add_option("--acc", dest="acc",
parser.add_argument("--acc", dest="acc",
help="Set ACC bits (Access Control Code). not all card types are supported",
)
parser.add_option("--opmode", dest="opmode", type="choice",
parser.add_argument("--opmode", dest="opmode",
help="Set UE Operation Mode in EF.AD (Administrative Data)",
default=None,
choices=['{:02X}'.format(int(m)) for m in EF_AD.OP_MODE],
)
parser.add_option("--epdgid", dest="epdgid",
parser.add_argument("-f", "--fplmn", dest="fplmn", action="append",
help="Set Forbidden PLMN. Add multiple time for multiple FPLMNS",
)
parser.add_argument("--epdgid", dest="epdgid",
help="Set Home Evolved Packet Data Gateway (ePDG) Identifier. (Only FQDN format supported)",
)
parser.add_option("--epdgSelection", dest="epdgSelection",
parser.add_argument("--epdgSelection", dest="epdgSelection",
help="Set PLMN for ePDG Selection Information. (Only Operator Identifier FQDN format supported)",
)
parser.add_option("--pcscf", dest="pcscf",
parser.add_argument("--pcscf", dest="pcscf",
help="Set Proxy Call Session Control Function (P-CSCF) Address. (Only FQDN format supported)",
)
parser.add_option("--ims-hdomain", dest="ims_hdomain",
parser.add_argument("--ims-hdomain", dest="ims_hdomain",
help="Set IMS Home Network Domain Name in FQDN format",
)
parser.add_option("--impi", dest="impi",
parser.add_argument("--impi", dest="impi",
help="Set IMS private user identity",
)
parser.add_option("--impu", dest="impu",
parser.add_argument("--impu", dest="impu",
help="Set IMS public user identity",
)
parser.add_option("--read-imsi", dest="read_imsi", action="store_true",
parser.add_argument("--read-imsi", dest="read_imsi", action="store_true",
help="Read the IMSI from the CARD", default=False
)
parser.add_option("--read-iccid", dest="read_iccid", action="store_true",
parser.add_argument("--read-iccid", dest="read_iccid", action="store_true",
help="Read the ICCID from the CARD", default=False
)
parser.add_option("-z", "--secret", dest="secret", metavar="STR",
parser.add_argument("-z", "--secret", dest="secret", metavar="STR",
help="Secret used for ICCID/IMSI autogen",
)
parser.add_option("-j", "--num", dest="num", type=int,
parser.add_argument("-j", "--num", dest="num", type=int,
help="Card # used for ICCID/IMSI autogen",
)
parser.add_option("--batch", dest="batch_mode",
help="Enable batch mode [default: %default]",
parser.add_argument("--batch", dest="batch_mode",
help="Enable batch mode [default: %(default)s]",
default=False, action='store_true',
)
parser.add_option("--batch-state", dest="batch_state", metavar="FILE",
parser.add_argument("--batch-state", dest="batch_state", metavar="FILE",
help="Optional batch state file",
)
# if mode is "csv"
parser.add_option("--read-csv", dest="read_csv", metavar="FILE",
parser.add_argument("--read-csv", dest="read_csv", metavar="FILE",
help="Read parameters from CSV file rather than command line")
parser.add_option("--write-csv", dest="write_csv", metavar="FILE",
parser.add_argument("--write-csv", dest="write_csv", metavar="FILE",
help="Append generated parameters in CSV file",
)
parser.add_option("--write-hlr", dest="write_hlr", metavar="FILE",
parser.add_argument("--write-hlr", dest="write_hlr", metavar="FILE",
help="Append generated parameters to OpenBSC HLR sqlite3",
)
parser.add_option("--dry-run", dest="dry_run",
parser.add_argument("--dry-run", dest="dry_run",
help="Perform a 'dry run', don't actually program the card",
default=False, action="store_true")
parser.add_option("--card_handler", dest="card_handler_config", metavar="FILE",
parser.add_argument("--card_handler", dest="card_handler_config", metavar="FILE",
help="Use automatic card handling machine")
parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False)
(options, args) = parser.parse_args()
options = parser.parse_args()
if options.type == 'list':
for kls in _cards_classes:
@@ -214,15 +202,13 @@ def parse_options():
return options
if options.source == 'csv':
if (options.imsi is None) and (options.batch_mode is False) and (options.read_imsi is False) and (options.read_iccid is False):
parser.error(
"CSV mode needs either an IMSI, --read-imsi, --read-iccid or batch mode")
if (options.imsi is None) and (options.iccid is None) and (options.read_imsi is False) and (options.read_iccid is False):
parser.error("CSV mode requires one additional parameter: --read-iccid, --read-imsi, --iccid or --imsi")
if options.read_csv is None:
parser.error("CSV mode requires a CSV input file")
elif options.source == 'cmdline':
if ((options.imsi is None) or (options.iccid is None)) and (options.num is None):
parser.error(
"If either IMSI or ICCID isn't specified, num is required")
parser.error("If either IMSI or ICCID isn't specified, num is required")
else:
parser.error("Only `cmdline' and `csv' sources supported")
@@ -237,9 +223,6 @@ def parse_options():
parser.error(
"Can't give ICCID/IMSI for batch mode, need to use automatic parameters ! see --num and --secret for more information")
if args:
parser.error("Extraneous arguments")
return options
@@ -318,7 +301,14 @@ def gen_parameters(opts):
# MCC always has 3 digits
mcc = lpad(mcc, 3, "0")
# MNC must be at least 2 digits
# The MNC must be at least 2 digits long. This is also the most common case.
# The user may specify an explicit length using the --mnclen option.
if opts.mnclen != "auto":
if len(mnc) > int(opts.mnclen):
raise ValueError('mcc is longer than specified in option --mnclen')
mnc = lpad(mnc, int(opts.mnclen), "0")
else:
mnc = lpad(mnc, 2, "0")
# Digitize country code (2 or 3 digits)
@@ -346,8 +336,8 @@ def gen_parameters(opts):
# ICCID (19 digits, E.118), though some phase1 vendors use 20 :(
if opts.iccid is not None:
iccid = opts.iccid
if not _isnum(iccid, 19) and not _isnum(iccid, 20):
raise ValueError('ICCID must be 19 or 20 digits !')
if not _isnum(iccid, 18) and not _isnum(iccid, 19) and not _isnum(iccid, 20):
raise ValueError('ICCID must be 18, 19 or 20 digits !')
else:
if opts.num is None:
@@ -445,7 +435,7 @@ def gen_parameters(opts):
if not re.match('^[0-9a-fA-F]{32}$', ki):
raise ValueError('Ki needs to be 128 bits, in hex format')
else:
ki = ''.join(['%02x' % random.randrange(0, 256) for i in range(16)])
ki = os.urandom(16).hex()
# OPC (random)
if opts.opc is not None:
@@ -456,7 +446,7 @@ def gen_parameters(opts):
elif opts.op is not None:
opc = derive_milenage_opc(ki, opts.op)
else:
opc = ''.join(['%02x' % random.randrange(0, 256) for i in range(16)])
opc = os.urandom(16).hex()
pin_adm = sanitize_pin_adm(opts.pin_adm, opts.pin_adm_hex)
@@ -490,6 +480,7 @@ def gen_parameters(opts):
'impi': opts.impi,
'impu': opts.impu,
'opmode': opts.opmode,
'fplmn': opts.fplmn,
}
@@ -524,40 +515,86 @@ def write_params_csv(opts, params):
f.close()
def _read_params_csv(opts, iccid=None, imsi=None):
import csv
f = open(opts.read_csv, 'r')
def find_row_in_csv_file(csv_file_name:str, num=None, iccid=None, imsi=None):
"""
Pick a matching row in a CSV file by row number or ICCID or IMSI. When num
is not None, the search parameters iccid and imsi are ignored. When
searching for a specific ICCID or IMSI the caller must set num to None. It
is possible to search for an ICCID or an IMSI at the same time. The first
line that either contains a matching ICCID or IMSI is returned. Unused
search parameters must be set to None.
"""
f = open(csv_file_name, 'r')
cr = csv.DictReader(f)
# Make sure the CSV file contains at least the fields we are searching for
if not 'iccid' in cr.fieldnames:
raise Exception("wrong CSV file format - no field \"iccid\" or missing header!")
if not 'imsi' in cr.fieldnames:
raise Exception("wrong CSV file format - no field \"imsi\" or missing header!")
# Enforce at least one search parameter
if not num and not iccid and not imsi:
raise Exception("no CSV file search parameters!")
# Lower-case fieldnames
cr.fieldnames = [field.lower() for field in cr.fieldnames]
i = 0
if not 'iccid' in cr.fieldnames:
raise Exception("CSV file in wrong format!")
for row in cr:
if opts.num is not None and opts.read_iccid is False and opts.read_imsi is False:
if opts.num == i:
# Pick a specific row by line number (num)
if num is not None and iccid is None and imsi is None:
if num == i:
f.close()
return row
i += 1
# Pick the first row that contains the specified ICCID
if row['iccid'] == iccid:
f.close()
return row
# Pick the first row that contains the specified IMSI
if row['imsi'] == imsi:
f.close()
return row
i += 1
f.close()
print("Could not read card parameters from CSV file, no matching entry found.")
return None
def read_params_csv(opts, imsi=None, iccid=None):
row = _read_params_csv(opts, iccid=iccid, imsi=imsi)
"""
Read the card parameters from a CSV file. This function will generate the
same dictionary that gen_parameters would generate from parameters passed as
commandline arguments.
"""
row = find_row_in_csv_file(opts.read_csv, opts.num, iccid=iccid, imsi=imsi)
if row is not None:
row['mcc'] = row.get('mcc', mcc_from_imsi(row.get('imsi')))
row['mnc'] = row.get('mnc', mnc_from_imsi(row.get('imsi')))
# We cannot determine the MNC length (2 or 3 digits?) from the IMSI
# alone. In cases where the user has specified an mnclen via the
# commandline options we can use that info, otherwise we guess that
# the length is 2, which is also the most common case.
if opts.mnclen != "auto":
if opts.mnclen == "2":
row['mnc'] = row.get('mnc', mnc_from_imsi(row.get('imsi'), False))
elif opts.mnclen == "3":
row['mnc'] = row.get('mnc', mnc_from_imsi(row.get('imsi'), True))
else:
raise ValueError("invalid parameter --mnclen, must be 2 or 3 or auto")
else:
row['mnc'] = row.get('mnc', mnc_from_imsi(row.get('imsi'), False))
# NOTE: We might consider to specify a new CSV field "mnclen" in our
# CSV files for a better automatization. However, this only makes sense
# when the tools and databases we export our files from will also add
# such a field.
pin_adm = None
# We need to escape the pin_adm we get from the csv
@@ -590,6 +627,9 @@ def read_params_csv(opts, imsi=None, iccid=None):
def write_params_hlr(opts, params):
# SQLite3 OpenBSC HLR
# FIXME: The format of the osmo-hlr database has evolved, so that the code below will no longer work.
print("Warning: the database format of recent OsmoHLR versions is not compatible with pySim-prog!")
if opts.write_hlr:
import sqlite3
conn = sqlite3.connect(opts.write_hlr)
@@ -621,7 +661,7 @@ def write_params_hlr(opts, params):
conn.close()
def write_parameters(opts, params):
def write_parameters_to_csv_and_hlr(opts, params):
write_params_csv(opts, params)
write_params_hlr(opts, params)
@@ -668,13 +708,11 @@ def save_batch(opts):
fh.close()
def process_card(opts, first, ch):
def process_card(scc, opts, first, ch):
if opts.dry_run is False:
# Connect transport
ch.get(first)
if opts.dry_run is False:
# Get card
card = card_detect(opts.type, scc)
if card is None:
@@ -685,35 +723,31 @@ def process_card(opts, first, ch):
if opts.probe:
return 0
# Erase if requested
# Erase if requested (not in dry run mode!)
if opts.dry_run is False:
if opts.erase:
print("Formatting ...")
card.erase()
card.reset()
cp = None
# Generate parameters
if opts.source == 'cmdline':
cp = gen_parameters(opts)
elif opts.source == 'csv':
imsi = None
iccid = None
if opts.read_iccid:
if opts.dry_run:
# Connect transport
ch.get(False)
(res, _) = scc.read_binary(['3f00', '2fe2'], length=10)
iccid = dec_iccid(res)
elif opts.read_imsi:
if opts.dry_run:
# Connect transport
ch.get(False)
else:
iccid = opts.iccid
if opts.read_imsi:
(res, _) = scc.read_binary(EF['IMSI'])
imsi = swap_nibbles(res)[3:]
else:
imsi = opts.imsi
cp = read_params_csv(opts, imsi=imsi, iccid=iccid)
if cp is None:
print("Error reading parameters from CSV file!\n")
return 2
print_parameters(cp)
@@ -724,8 +758,8 @@ def process_card(opts, first, ch):
else:
print("Dry Run: NOT PROGRAMMING!")
# Write parameters permanently
write_parameters(opts, cp)
# Write parameters to a specified CSV file or an HLR database (not the card)
write_parameters_to_csv_and_hlr(opts, cp)
# Batch mode state update and save
if opts.num is not None:
@@ -741,10 +775,11 @@ if __name__ == '__main__':
# Parse options
opts = parse_options()
# Setup logger
PySimLogger.setup(print, {logging.WARN: "\033[33m"}, opts.verbose)
# Init card reader driver
sl = init_reader(opts)
if sl is None:
exit(1)
# Create command layer
scc = SimCardCommands(transport=sl)
@@ -770,7 +805,7 @@ if __name__ == '__main__':
while 1:
try:
rc = process_card(opts, first, ch)
rc = process_card(scc, opts, first, ch)
except (KeyboardInterrupt):
print("")
print("Terminated by user!")

58
pySim-read.py
View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
#
# Utility to display some informations about a SIM card
# Utility to display some information about a SIM card
#
#
# Copyright (C) 2009 Sylvain Munaut <tnt@246tNt.com>
@@ -25,27 +25,37 @@
import hashlib
import argparse
import os
import random
import re
import sys
from pySim.ts_51_011 import EF, DF, EF_SST_map, EF_AD
from pySim.ts_31_102 import EF_UST_map, EF_USIM_ADF_map
from pySim.ts_31_103 import EF_IST_map, EF_ISIM_ADF_map
from osmocom.utils import h2b, h2s, swap_nibbles, rpad
from pySim.ts_51_011 import EF_SST_map, EF_AD
from pySim.legacy.ts_51_011 import EF, DF
from pySim.ts_31_102 import EF_UST_map
from pySim.legacy.ts_31_102 import EF_USIM_ADF_map
from pySim.ts_31_103 import EF_IST_map
from pySim.legacy.ts_31_103 import EF_ISIM_ADF_map
from pySim.commands import SimCardCommands
from pySim.transport import init_reader, argparse_add_reader_args
from pySim.exceptions import SwMatchError
from pySim.cards import card_detect, SimCard, UsimCard, IsimCard
from pySim.utils import h2b, swap_nibbles, rpad, dec_imsi, dec_iccid, dec_msisdn
from pySim.utils import format_xplmn_w_act, dec_st
from pySim.utils import h2s, format_ePDGSelection
from pySim.legacy.cards import card_detect, SimCard, UsimCard, IsimCard
from pySim.utils import dec_imsi, dec_iccid
from pySim.legacy.utils import format_xplmn_w_act, dec_st, dec_msisdn
from pySim.ts_51_011 import EF_SMSP
option_parser = argparse.ArgumentParser(prog='pySim-read',
description='Legacy tool for reading some parts of a SIM card',
from pathlib import Path
import logging
from pySim.log import PySimLogger
log = PySimLogger.get(Path(__file__).stem)
option_parser = argparse.ArgumentParser(description='Legacy tool for reading some parts of a SIM card',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
option_parser.add_argument("--verbose", help="Enable verbose logging", action='store_true', default=False)
argparse_add_reader_args(option_parser)
def select_app(adf: str, card: SimCard):
"""Select application by its AID"""
sw = 0
@@ -70,10 +80,11 @@ if __name__ == '__main__':
# Parse options
opts = option_parser.parse_args()
# Setup logger
PySimLogger.setup(print, {logging.WARN: "\033[33m"}, opts.verbose)
# Init card reader driver
sl = init_reader(opts)
if sl is None:
exit(1)
# Create command layer
scc = SimCardCommands(transport=sl)
@@ -86,7 +97,7 @@ if __name__ == '__main__':
scc.sel_ctrl = "0004"
# Testing for Classic SIM or UICC
(res, sw) = sl.send_apdu(scc.cla_byte + "a4" + scc.sel_ctrl + "02" + "3f00")
(res, sw) = sl.send_apdu(scc.cla_byte + "a4" + scc.sel_ctrl + "02" + "3f00" + "00")
if sw == '6e00':
# Just a Classic SIM
scc.cla_byte = "a0"
@@ -139,6 +150,15 @@ if __name__ == '__main__':
(res, sw) = card.read_record('SMSP', 1)
if sw == '9000':
print("SMSP: %s" % (res,))
ef_smsp = EF_SMSP()
smsc_a = ef_smsp.decode_record_bin(h2b(res), 1).get('tp_sc_addr', {})
smsc_n = smsc_a.get('call_number', None)
if smsc_a.get('ton_npi', {}).get('type_of_number', None) == 'international' and smsc_n is not None:
smsc = '+' + smsc_n
else:
smsc = smsc_n
if smsc is not None:
print("SMSC: %s" % (smsc,))
else:
print("SMSP: Can't read, response code = %s" % (sw,))
@@ -253,6 +273,14 @@ if __name__ == '__main__':
else:
print("EHPLMN: Can't read, response code = %s" % (sw,))
# EF.FPLMN
if usim_card.file_exists(EF_USIM_ADF_map['FPLMN']):
res, sw = usim_card.read_fplmn()
if sw == '9000':
print(f'FPLMN:\n{res}')
else:
print(f'FPLMN: Can\'t read, response code = {sw}')
# EF.UST
try:
if usim_card.file_exists(EF_USIM_ADF_map['UST']):

1065
pySim-shell.py
View File

File diff suppressed because it is too large Load Diff

428
pySim-smpp2sim.py Executable file
View File

@@ -0,0 +1,428 @@
#!/usr/bin/env python3
#
# Program to emulate the entire communication path SMSC-MSC-BSC-BTS-ME
# that is usually between an OTA backend and the SIM card. This allows
# to play with SIM OTA technology without using a mobile network or even
# a mobile phone.
#
# An external application must encode (and encrypt/sign) the OTA SMS
# and submit them via SMPP to this program, just like it would submit
# it normally to a SMSC (SMS Service Centre). The program then re-formats
# the SMPP-SUBMIT into a SMS DELIVER TPDU and passes it via an ENVELOPE
# APDU to the SIM card that is locally inserted into a smart card reader.
#
# The path from SIM to external OTA application works the opposite way.
# (C) 2023-2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import argparse
import logging
import colorlog
from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, reactor, task
from twisted.cred.portal import IRealm
from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse
from twisted.cred.portal import Portal
from zope.interface import implementer
from smpp.twisted.config import SMPPServerConfig
from smpp.twisted.server import SMPPServerFactory, SMPPBindManager
from smpp.twisted.protocol import SMPPSessionStates, DataHandlerResponse
from smpp.pdu import pdu_types, operations, pdu_encoding
from pySim.sms import SMS_DELIVER, SMS_SUBMIT, AddressField
from pySim.transport import LinkBase, ProactiveHandler, argparse_add_reader_args, init_reader, ApduTracer
from pySim.commands import SimCardCommands
from pySim.cards import UiccCardBase
from pySim.exceptions import *
from pySim.cat import ProactiveCommand, SendShortMessage, SMS_TPDU, SMSPPDownload, BearerDescription
from pySim.cat import DeviceIdentities, Address, OtherAddress, UiccTransportLevel, BufferSize
from pySim.cat import ChannelStatus, ChannelData, ChannelDataLength
from pySim.utils import b2h, h2b
logger = logging.getLogger(__name__)
# MSISDNs to use when generating proactive SMS messages
SIM_MSISDN='23'
ESME_MSISDN='12'
# HACK: we need some kind of mapping table between system_id and card-reader
# or actually route based on MSISDNs
hackish_global_smpp = None
class MyApduTracer(ApduTracer):
def trace_response(self, cmd, sw, resp):
print("-> %s %s" % (cmd[:10], cmd[10:]))
print("<- %s: %s" % (sw, resp))
class TcpProtocol(protocol.Protocol):
def dataReceived(self, data):
pass
def connectionLost(self, reason):
pass
def tcp_connected_callback(p: protocol.Protocol):
"""called by twisted TCP client."""
logger.error("%s: connected!" % p)
class ProactChannel:
"""Representation of a single protective channel."""
def __init__(self, channels: 'ProactChannels', chan_nr: int):
self.channels = channels
self.chan_nr = chan_nr
self.ep = None
def close(self):
"""Close the channel."""
if self.ep:
self.ep.disconnect()
self.channels.channel_delete(self.chan_nr)
class ProactChannels:
"""Wrapper class for maintaining state of proactive channels."""
def __init__(self):
self.channels = {}
def channel_create(self) -> ProactChannel:
"""Create a new proactive channel, allocating its integer number."""
for i in range(1, 9):
if not i in self.channels:
self.channels[i] = ProactChannel(self, i)
return self.channels[i]
raise ValueError('Cannot allocate another channel: All channels active')
def channel_delete(self, chan_nr: int):
del self.channels[chan_nr]
class Proact(ProactiveHandler):
#def __init__(self, smpp_factory):
# self.smpp_factory = smpp_factory
def __init__(self):
self.channels = ProactChannels()
@staticmethod
def _find_first_element_of_type(instlist, cls):
for i in instlist:
if isinstance(i, cls):
return i
return None
"""Call-back which the pySim transport core calls whenever it receives a
proactive command from the SIM."""
def handle_SendShortMessage(self, pcmd: ProactiveCommand):
# {'smspp_download': [{'device_identities': {'source_dev_id': 'network',
# 'dest_dev_id': 'uicc'}},
# {'address': {'ton_npi': {'ext': True,
# 'type_of_number': 'international',
# 'numbering_plan_id': 'isdn_e164'},
# 'call_number': '79'}},
# {'sms_tpdu': {'tpdu': '40048111227ff6407070611535004d02700000481516011212000001fe4c0943aea42e45021c078ae06c66afc09303608874b72f58bacadb0dcf665c29349c799fbb522e61709c9baf1890015e8e8e196e36153106c8b92f95153774'}}
# ]}
"""Card requests sending a SMS. We need to pass it on to the ESME via SMPP."""
logger.info("SendShortMessage")
logger.info(pcmd)
# Relevant parts in pcmd: Address, SMS_TPDU
addr_ie = Proact._find_first_element_of_type(pcmd.children, Address)
sms_tpdu_ie = Proact._find_first_element_of_type(pcmd.children, SMS_TPDU)
raw_tpdu = sms_tpdu_ie.decoded['tpdu']
submit = SMS_SUBMIT.from_bytes(raw_tpdu)
submit.tp_da = AddressField(addr_ie.decoded['call_number'], addr_ie.decoded['ton_npi']['type_of_number'],
addr_ie.decoded['ton_npi']['numbering_plan_id'])
logger.info(submit)
self.send_sms_via_smpp(submit)
def handle_OpenChannel(self, pcmd: ProactiveCommand):
"""Card requests opening a new channel via a UDP/TCP socket."""
# {'open_channel': [{'command_details': {'command_number': 1,
# 'type_of_command': 'open_channel',
# 'command_qualifier': 3}},
# {'device_identities': {'source_dev_id': 'uicc',
# 'dest_dev_id': 'terminal'}},
# {'bearer_description': {'bearer_type': 'default',
# 'bearer_parameters': ''}},
# {'buffer_size': 1024},
# {'uicc_transport_level': {'protocol_type': 'tcp_uicc_client_remote',
# 'port_number': 32768}},
# {'other_address': {'type_of_address': 'ipv4',
# 'address': '01020304'}}
# ]}
logger.info("OpenChannel")
logger.info(pcmd)
transp_lvl_ie = Proact._find_first_element_of_type(pcmd.children, UiccTransportLevel)
other_addr_ie = Proact._find_first_element_of_type(pcmd.children, OtherAddress)
bearer_desc_ie = Proact._find_first_element_of_type(pcmd.children, BearerDescription)
buffer_size_ie = Proact._find_first_element_of_type(pcmd.children, BufferSize)
if transp_lvl_ie.decoded['protocol_type'] != 'tcp_uicc_client_remote':
raise ValueError('Unsupported protocol_type')
if other_addr_ie.decoded.get('type_of_address', None) != 'ipv4':
raise ValueError('Unsupported type_of_address')
ipv4_bytes = h2b(other_addr_ie.decoded['address'])
ipv4_str = '%u.%u.%u.%u' % (ipv4_bytes[0], ipv4_bytes[1], ipv4_bytes[2], ipv4_bytes[3])
port_nr = transp_lvl_ie.decoded['port_number']
print("%s:%u" % (ipv4_str, port_nr))
channel = self.channels.channel_create()
channel.ep = endpoints.TCP4ClientEndpoint(reactor, ipv4_str, port_nr)
channel.prot = TcpProtocol()
d = endpoints.connectProtocol(channel.ep, channel.prot)
# FIXME: why is this never called despite the client showing the inbound connection?
d.addCallback(tcp_connected_callback)
# Terminal Response example: [
# {'command_details': {'command_number': 1,
# 'type_of_command': 'open_channel',
# 'command_qualifier': 3}},
# {'device_identities': {'source_dev_id': 'terminal', 'dest_dev_id': 'uicc'}},
# {'result': {'general_result': 'performed_successfully', 'additional_information': ''}},
# {'channel_status': '8100'},
# {'bearer_description': {'bearer_type': 'default', 'bearer_parameters': ''}},
# {'buffer_size': 1024}
# ]
return self.prepare_response(pcmd) + [ChannelStatus(decoded='8100'), bearer_desc_ie, buffer_size_ie]
def handle_CloseChannel(self, pcmd: ProactiveCommand):
"""Close a channel."""
logger.info("CloseChannel")
logger.info(pcmd)
def handle_ReceiveData(self, pcmd: ProactiveCommand):
"""Receive/read data from the socket."""
# {'receive_data': [{'command_details': {'command_number': 1,
# 'type_of_command': 'receive_data',
# 'command_qualifier': 0}},
# {'device_identities': {'source_dev_id': 'uicc',
# 'dest_dev_id': 'channel_1'}},
# {'channel_data_length': 9}
# ]}
logger.info("ReceiveData")
logger.info(pcmd)
# Terminal Response example: [
# {'command_details': {'command_number': 1,
# 'type_of_command': 'receive_data',
# 'command_qualifier': 0}},
# {'device_identities': {'source_dev_id': 'terminal', 'dest_dev_id': 'uicc'}},
# {'result': {'general_result': 'performed_successfully', 'additional_information': ''}},
# {'channel_data': '16030100040e000000'},
# {'channel_data_length': 0}
# ]
return self.prepare_response(pcmd) + []
def handle_SendData(self, pcmd: ProactiveCommand):
"""Send/write data received from the SIM to the socket."""
# {'send_data': [{'command_details': {'command_number': 1,
# 'type_of_command': 'send_data',
# 'command_qualifier': 1}},
# {'device_identities': {'source_dev_id': 'uicc',
# 'dest_dev_id': 'channel_1'}},
# {'channel_data': '160301003c010000380303d0f45e12b52ce5bb522750dd037738195334c87a46a847fe2b6886cada9ea6bf00000a00ae008c008b00b0002c010000050001000101'}
# ]}
logger.info("SendData")
logger.info(pcmd)
dev_id_ie = Proact._find_first_element_of_type(pcmd.children, DeviceIdentities)
chan_data_ie = Proact._find_first_element_of_type(pcmd.children, ChannelData)
chan_str = dev_id_ie.decoded['dest_dev_id']
chan_nr = 1 # FIXME
chan = self.channels.channels.get(chan_nr, None)
# FIXME chan.prot.transport.write(h2b(chan_data_ie.decoded))
# Terminal Response example: [
# {'command_details': {'command_number': 1,
# 'type_of_command': 'send_data',
# 'command_qualifier': 1}},
# {'device_identities': {'source_dev_id': 'terminal', 'dest_dev_id': 'uicc'}},
# {'result': {'general_result': 'performed_successfully', 'additional_information': ''}},
# {'channel_data_length': 255}
# ]
return self.prepare_response(pcmd) + [ChannelDataLength(decoded=255)]
def handle_SetUpEventList(self, pcmd: ProactiveCommand):
# {'set_up_event_list': [{'command_details': {'command_number': 1,
# 'type_of_command': 'set_up_event_list',
# 'command_qualifier': 0}},
# {'device_identities': {'source_dev_id': 'uicc',
# 'dest_dev_id': 'terminal'}},
# {'event_list': ['data_available', 'channel_status']}
# ]}
logger.info("SetUpEventList")
logger.info(pcmd)
# Terminal Response example: [
# {'command_details': {'command_number': 1,
# 'type_of_command': 'set_up_event_list',
# 'command_qualifier': 0}},
# {'device_identities': {'source_dev_id': 'terminal', 'dest_dev_id': 'uicc'}},
# {'result': {'general_result': 'performed_successfully', 'additional_information': ''}}
# ]
return self.prepare_response(pcmd)
def getChannelStatus(self, pcmd: ProactiveCommand):
logger.info("GetChannelStatus")
logger.info(pcmd)
return self.prepare_response(pcmd) + []
def send_sms_via_smpp(self, submit: SMS_SUBMIT):
# while in a normal network the phone/ME would *submit* a message to the SMSC,
# we are actually emulating the SMSC itself, so we must *deliver* the message
# to the ESME
deliver = SMS_DELIVER.from_submit(submit)
deliver_smpp = deliver.to_smpp()
hackish_global_smpp.sendDataRequest(deliver_smpp)
# # obtain the connection/binding of system_id to be used for delivering MO-SMS to the ESME
# connection = smpp_server.getBoundConnections[system_id].getNextBindingForDelivery()
# connection.sendDataRequest(deliver_smpp)
def dcs_is_8bit(dcs):
if dcs == pdu_types.DataCoding(pdu_types.DataCodingScheme.DEFAULT,
pdu_types.DataCodingDefault.OCTET_UNSPECIFIED):
return True
if dcs == pdu_types.DataCoding(pdu_types.DataCodingScheme.DEFAULT,
pdu_types.DataCodingDefault.OCTET_UNSPECIFIED_COMMON):
return True
# pySim-smpp2sim.py:150:21: E1101: Instance of 'DataCodingScheme' has no 'GSM_MESSAGE_CLASS' member (no-member)
# pylint: disable=no-member
if dcs.scheme == pdu_types.DataCodingScheme.GSM_MESSAGE_CLASS and dcs.schemeData['msgCoding'] == pdu_types.DataCodingGsmMsgCoding.DATA_8BIT:
return True
else:
return False
class MyServer:
@implementer(IRealm)
class SmppRealm:
def requestAvatar(self, avatarId, mind, *interfaces):
return ('SMPP', avatarId, lambda: None)
def __init__(self, tcp_port:int = 2775, bind_ip = '::', system_id:str = 'test', password:str = 'test'):
smpp_config = SMPPServerConfig(msgHandler=self._msgHandler,
systems={system_id: {'max_bindings': 2}})
portal = Portal(self.SmppRealm())
credential_checker = InMemoryUsernamePasswordDatabaseDontUse()
credential_checker.addUser(system_id, password)
portal.registerChecker(credential_checker)
self.factory = SMPPServerFactory(smpp_config, auth_portal=portal)
logger.info('Binding Virtual SMSC to TCP Port %u at %s' % (tcp_port, bind_ip))
smppEndpoint = endpoints.TCP6ServerEndpoint(reactor, tcp_port, interface=bind_ip)
smppEndpoint.listen(self.factory)
self.tp = self.scc = self.card = None
def connect_to_card(self, tp: LinkBase):
self.tp = tp
self.scc = SimCardCommands(self.tp)
self.card = UiccCardBase(self.scc)
# this should be part of UiccCardBase, but FairewavesSIM breaks with that :/
self.scc.cla_byte = "00"
self.scc.sel_ctrl = "0004"
self.card.read_aids()
self.card.select_adf_by_aid(adf='usim')
# FIXME: create a more realistic profile than ffffff
self.scc.terminal_profile('ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff')
def _msgHandler(self, system_id, smpp, pdu):
"""Handler for incoming messages received via SMPP from ESME."""
# HACK: we need some kind of mapping table between system_id and card-reader
# or actually route based on MSISDNs
global hackish_global_smpp
hackish_global_smpp = smpp
if pdu.id == pdu_types.CommandId.submit_sm:
return self.handle_submit_sm(system_id, smpp, pdu)
else:
logger.warning('Rejecting non-SUBMIT commandID')
return pdu_types.CommandStatus.ESME_RINVCMDID
def handle_submit_sm(self, system_id, smpp, pdu):
"""SUBMIT-SM was received via SMPP from ESME. We need to deliver it to the SIM."""
# check for valid data coding scheme + PID
if not dcs_is_8bit(pdu.params['data_coding']):
logger.warning('Rejecting non-8bit DCS')
return pdu_types.CommandStatus.ESME_RINVDCS
if pdu.params['protocol_id'] != 0x7f:
logger.warning('Rejecting non-SIM PID')
return pdu_types.CommandStatus.ESME_RINVDCS
# 1) build a SMS-DELIVER (!) from the SMPP-SUBMIT
tpdu = SMS_DELIVER.from_smpp_submit(pdu)
logger.info(tpdu)
# 2) wrap into the CAT ENVELOPE for SMS-PP-Download
tpdu_ie = SMS_TPDU(decoded={'tpdu': b2h(tpdu.to_bytes())})
addr_ie = Address(decoded={'ton_npi': {'ext':False, 'type_of_number':'unknown', 'numbering_plan_id':'unknown'}, 'call_number': '0123456'})
dev_ids = DeviceIdentities(decoded={'source_dev_id': 'network', 'dest_dev_id': 'uicc'})
sms_dl = SMSPPDownload(children=[dev_ids, addr_ie, tpdu_ie])
# 3) send to the card
envelope_hex = b2h(sms_dl.to_tlv())
logger.info("ENVELOPE: %s" % envelope_hex)
(data, sw) = self.scc.envelope(envelope_hex)
logger.info("SW %s: %s" % (sw, data))
if sw in ['9200', '9300']:
# TODO send back RP-ERROR message with TP-FCS == 'SIM Application Toolkit Busy'
return pdu_types.CommandStatus.ESME_RSUBMITFAIL
elif sw == '9000' or sw[0:2] in ['6f', '62', '63'] and len(data):
# data something like 027100000e0ab000110000000000000001612f or
# 027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
# which is the user-data portion of the SMS starting with the UDH (027100)
# TODO: return the response back to the sender in an RP-ACK; PID/DCS like in CMD
deliver = operations.DeliverSM(service_type=pdu.params['service_type'],
source_addr_ton=pdu.params['dest_addr_ton'],
source_addr_npi=pdu.params['dest_addr_npi'],
source_addr=pdu.params['destination_addr'],
dest_addr_ton=pdu.params['source_addr_ton'],
dest_addr_npi=pdu.params['source_addr_npi'],
destination_addr=pdu.params['source_addr'],
esm_class=pdu.params['esm_class'],
protocol_id=pdu.params['protocol_id'],
priority_flag=pdu.params['priority_flag'],
data_coding=pdu.params['data_coding'],
short_message=h2b(data))
smpp.sendDataRequest(deliver)
return pdu_types.CommandStatus.ESME_ROK
else:
return pdu_types.CommandStatus.ESME_RSUBMITFAIL
option_parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
argparse_add_reader_args(option_parser)
smpp_group = option_parser.add_argument_group('SMPP Options')
smpp_group.add_argument('--smpp-bind-port', type=int, default=2775,
help='TCP Port to bind the SMPP socket to')
smpp_group.add_argument('--smpp-bind-ip', default='::',
help='IPv4/IPv6 address to bind the SMPP socket to')
smpp_group.add_argument('--smpp-system-id', default='test',
help='SMPP System-ID used by ESME to bind')
smpp_group.add_argument('--smpp-password', default='test',
help='SMPP Password used by ESME to bind')
if __name__ == '__main__':
log_format='%(log_color)s%(levelname)-8s%(reset)s %(name)s: %(message)s'
colorlog.basicConfig(level=logging.INFO, format = log_format)
logger = colorlog.getLogger()
opts = option_parser.parse_args()
tp = init_reader(opts, proactive_handler = Proact())
if tp is None:
exit(1)
tp.connect()
global g_ms
g_ms = MyServer(opts.smpp_bind_port, opts.smpp_bind_ip, opts.smpp_system_id, opts.smpp_password)
g_ms.connect_to_card(tp)
reactor.run()

222
pySim-trace.py Executable file
View File

@@ -0,0 +1,222 @@
#!/usr/bin/env python3
import sys
import logging, colorlog
import argparse
from pprint import pprint as pp
from pySim.apdu import *
from pySim.runtime import RuntimeState
from osmocom.utils import JsonEncoder
from pySim.cards import UiccCardBase
from pySim.commands import SimCardCommands
from pySim.profile import CardProfile
from pySim.ts_102_221 import CardProfileUICC
from pySim.ts_31_102 import CardApplicationUSIM
from pySim.ts_31_103 import CardApplicationISIM
from pySim.euicc import CardApplicationISDR, CardApplicationECASD
from pySim.transport import LinkBase
from pySim.apdu_source.gsmtap import GsmtapApduSource
from pySim.apdu_source.pyshark_rspro import PysharkRsproPcap, PysharkRsproLive
from pySim.apdu_source.pyshark_gsmtap import PysharkGsmtapPcap
from pySim.apdu_source.tca_loader_log import TcaLoaderLogApduSource
from pySim.apdu_source.stdin_hex import StdinHexApduSource
from pySim.apdu.ts_102_221 import UiccSelect, UiccStatus
log_format='%(log_color)s%(levelname)-8s%(reset)s %(name)s: %(message)s'
colorlog.basicConfig(level=logging.INFO, format = log_format)
logger = colorlog.getLogger()
# merge all of the command sets into one global set. This will override instructions,
# the one from the 'last' set in the addition below will prevail.
from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
from pySim.apdu.global_platform import ApduCommands as GpApduCommands
ApduCommands = UiccApduCommands + UsimApduCommands #+ GpApduCommands
class DummySimLink(LinkBase):
"""A dummy implementation of the LinkBase abstract base class. Currently required
as the UiccCardBase doesn't work without SimCardCommands, which in turn require
a LinkBase implementation talking to a card.
In the tracer, we don't actually talk to any card, so we simply drop everything
and claim it is successful.
The UiccCardBase / SimCardCommands should be refactored to make this obsolete later."""
def __init__(self, debug: bool = False, **kwargs):
super().__init__(**kwargs)
self._debug = debug
self._atr = h2i('3B9F96801F878031E073FE211B674A4C753034054BA9')
def __str__(self):
return "dummy"
def _send_apdu(self, pdu):
#print("DummySimLink-apdu: %s" % pdu)
return [], '9000'
def connect(self):
pass
def disconnect(self):
pass
def _reset_card(self):
return 1
def get_atr(self):
return self._atr
def wait_for_card(self):
pass
class Tracer:
def __init__(self, **kwargs):
# we assume a generic UICC profile; as all APDUs return 9000 in DummySimLink above,
# all CardProfileAddon (including SIM) will probe successful.
profile = CardProfileUICC()
profile.add_application(CardApplicationUSIM())
profile.add_application(CardApplicationISIM())
profile.add_application(CardApplicationISDR())
profile.add_application(CardApplicationECASD())
scc = SimCardCommands(transport=DummySimLink())
card = UiccCardBase(scc)
self.rs = RuntimeState(card, profile)
# APDU Decoder
self.ad = ApduDecoder(ApduCommands)
# parameters
self.suppress_status = kwargs.get('suppress_status', True)
self.suppress_select = kwargs.get('suppress_select', True)
self.show_raw_apdu = kwargs.get('show_raw_apdu', False)
self.source = kwargs.get('source', None)
def format_capdu(self, apdu: Apdu, inst: ApduCommand):
"""Output a single decoded + processed ApduCommand."""
if self.show_raw_apdu:
print(apdu)
print("%02u %-16s %-35s %-8s %s %s" % (inst.lchan_nr, inst._name, inst.path_str, inst.col_id,
inst.col_sw, json.dumps(inst.processed, cls=JsonEncoder)))
print("===============================")
def format_reset(self, apdu: CardReset):
"""Output a single decoded CardReset."""
print(apdu)
print("===============================")
def main(self):
"""Main loop of tracer: Iterates over all Apdu received from source."""
apdu_counter = 0
while True:
# obtain the next APDU from the source (blocking read)
try:
apdu = self.source.read()
apdu_counter = apdu_counter + 1
except StopIteration:
print("%i APDUs parsed, stop iteration." % apdu_counter)
return 0
if isinstance(apdu, CardReset):
self.rs.reset()
self.format_reset(apdu)
continue
# ask ApduDecoder to look-up (INS,CLA) + instantiate an ApduCommand derived
# class like 'UiccSelect'
inst = self.ad.input(apdu)
# process the APDU (may modify the RuntimeState)
inst.process(self.rs)
# Avoid cluttering the log with too much verbosity
if self.suppress_select and isinstance(inst, UiccSelect):
continue
if self.suppress_status and isinstance(inst, UiccStatus):
continue
self.format_capdu(apdu, inst)
option_parser = argparse.ArgumentParser(description='Osmocom pySim high-level SIM card trace decoder',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
global_group = option_parser.add_argument_group('General Options')
global_group.add_argument('--no-suppress-select', action='store_false', dest='suppress_select',
help="""
Don't suppress displaying SELECT APDUs. We normally suppress them as they just clutter up
the output without giving any useful information. Any subsequent READ/UPDATE/... operations
on the selected file will log the file name most recently SELECTed.""")
global_group.add_argument('--no-suppress-status', action='store_false', dest='suppress_status',
help="""
Don't suppress displaying STATUS APDUs. We normally suppress them as they don't provide any
information that was not already received in response to the most recent SEELCT.""")
global_group.add_argument('--show-raw-apdu', action='store_true', dest='show_raw_apdu',
help="""Show the raw APDU in addition to its parsed form.""")
subparsers = option_parser.add_subparsers(help='APDU Source', dest='source', required=True)
parser_gsmtap = subparsers.add_parser('gsmtap-udp', help="""
Read APDUs from live capture by receiving GSMTAP-SIM packets on specified UDP port.
Use this for live capture from SIMtrace2 or osmo-qcdiag.""")
parser_gsmtap.add_argument('-i', '--bind-ip', default='127.0.0.1',
help='Local IP address to which to bind the UDP port')
parser_gsmtap.add_argument('-p', '--bind-port', default=4729,
help='Local UDP port')
parser_gsmtap_pyshark_pcap = subparsers.add_parser('gsmtap-pyshark-pcap', help="""
Read APDUs from PCAP file containing GSMTAP (SIM APDU) communication; processed via pyshark.
Use this if you have recorded a PCAP file containing GSMTAP (SIM APDU) e.g. via tcpdump or
wireshark/tshark.""")
parser_gsmtap_pyshark_pcap.add_argument('-f', '--pcap-file', required=True,
help='Name of the PCAP[ng] file to be read')
parser_rspro_pyshark_pcap = subparsers.add_parser('rspro-pyshark-pcap', help="""
Read APDUs from PCAP file containing RSPRO (osmo-remsim) communication; processed via pyshark.
REQUIRES OSMOCOM PATCHED WIRESHARK!""")
parser_rspro_pyshark_pcap.add_argument('-f', '--pcap-file', required=True,
help='Name of the PCAP[ng] file to be read')
parser_rspro_pyshark_live = subparsers.add_parser('rspro-pyshark-live', help="""
Read APDUs from live capture of RSPRO (osmo-remsim) communication; processed via pyshark.
REQUIRES OSMOCOM PATCHED WIRESHARK!""")
parser_rspro_pyshark_live.add_argument('-i', '--interface', required=True,
help='Name of the network interface to capture on')
parser_tcaloader_log = subparsers.add_parser('tca-loader-log', help="""
Read APDUs from a TCA Loader log file.""")
parser_tcaloader_log.add_argument('-f', '--log-file', required=True,
help='Name of the log file to be read')
parser_stdin_hex = subparsers.add_parser('stdin-hex', help="""
Read APDUs as hex-string from stdin.""")
if __name__ == '__main__':
opts = option_parser.parse_args()
logger.info('Opening source %s...', opts.source)
if opts.source == 'gsmtap-udp':
s = GsmtapApduSource(opts.bind_ip, opts.bind_port)
elif opts.source == 'rspro-pyshark-pcap':
s = PysharkRsproPcap(opts.pcap_file)
elif opts.source == 'rspro-pyshark-live':
s = PysharkRsproLive(opts.interface)
elif opts.source == 'gsmtap-pyshark-pcap':
s = PysharkGsmtapPcap(opts.pcap_file)
elif opts.source == 'tca-loader-log':
s = TcaLoaderLogApduSource(opts.log_file)
elif opts.source == 'stdin-hex':
s = StdinHexApduSource()
else:
raise ValueError("unsupported source %s", opts.source)
tracer = Tracer(source=s, suppress_status=opts.suppress_status, suppress_select=opts.suppress_select,
show_raw_apdu=opts.show_raw_apdu)
logger.info('Entering main loop...')
tracer.main()

466
pySim/apdu/__init__.py Normal file
View File

@@ -0,0 +1,466 @@
"""APDU (and TPDU) parser for UICC/USIM/ISIM cards.
The File (and its classes) represent the structure / hierarchy
of the APDUs as seen in SIM/UICC/SIM/ISIM cards. The primary use case
is to perform a meaningful decode of protocol traces taken between card and UE.
The ancient wirshark dissector developed for GSMTAP generated by SIMtrace
is far too simplistic, while this decoder can utilize all of the information
we already know in pySim about the filesystem structure, file encoding, etc.
"""
# (C) 2022-2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import abc
import typing
from typing import List, Dict, Optional
from termcolor import colored
from construct import Byte
from construct import Optional as COptional
from osmocom.construct import *
from osmocom.utils import *
from pySim.runtime import RuntimeLchan, RuntimeState, lchan_nr_from_cla
from pySim.filesystem import CardADF, CardFile, TransparentEF, LinFixedEF
"""There are multiple levels of decode:
1) pure TPDU / APDU level (no filesystem state required to decode)
1a) the raw C-TPDU + R-TPDU
1b) the raw C-APDU + R-APDU
1c) the C-APDU + R-APDU split in its portions (p1/p2/lc/le/cmd/rsp)
1d) the abstract C-APDU + R-APDU (mostly p1/p2 parsing; SELECT response)
2) the decoded DATA of command/response APDU
* READ/UPDATE: requires state/context: which file is selected? how to decode it?
"""
class ApduCommandMeta(abc.ABCMeta):
"""A meta-class that we can use to set some class variables when declaring
a derived class of ApduCommand."""
def __new__(mcs, name, bases, namespace, **kwargs):
x = super().__new__(mcs, name, bases, namespace)
x._name = namespace.get('name', kwargs.get('n', None))
x._ins = namespace.get('ins', kwargs.get('ins', None))
x._cla = namespace.get('cla', kwargs.get('cla', None))
return x
BytesOrHex = typing.Union[bytes, Hexstr]
class Tpdu:
def __init__(self, cmd: BytesOrHex, rsp: Optional[BytesOrHex] = None):
if isinstance(cmd, str):
self.cmd = h2b(cmd)
else:
self.cmd = cmd
if isinstance(rsp, str):
self.rsp = h2b(rsp)
else:
self.rsp = rsp
def __str__(self):
return '%s(%02X %02X %02X %02X %02X %s %s %s)' % (type(self).__name__, self.cla, self.ins, self.p1,
self.p2, self.p3, b2h(self.cmd_data), b2h(self.rsp_data), b2h(self.sw))
@property
def cla(self) -> int:
"""Return CLA of the C-APDU Header."""
return self.cmd[0]
@property
def ins(self) -> int:
"""Return INS of the C-APDU Header."""
return self.cmd[1]
@property
def p1(self) -> int:
"""Return P1 of the C-APDU Header."""
return self.cmd[2]
@property
def p2(self) -> int:
"""Return P2 of the C-APDU Header."""
return self.cmd[3]
@property
def p3(self) -> int:
"""Return P3 of the C-APDU Header."""
return self.cmd[4]
@property
def cmd_data(self) -> int:
"""Return the DATA portion of the C-APDU"""
return self.cmd[5:]
@property
def sw(self) -> Optional[bytes]:
"""Return Status Word (SW) of the R-APDU"""
return self.rsp[-2:] if self.rsp else None
@property
def rsp_data(self) -> Optional[bytes]:
"""Return the DATA portion of the R-APDU"""
return self.rsp[:-2] if self.rsp else None
class Apdu(Tpdu):
@property
def lc(self) -> int:
"""Return Lc; Length of C-APDU body."""
return len(self.cmd_data)
@property
def lr(self) -> int:
"""Return Lr; Length of R-APDU body."""
return len(self.rsp_data)
@property
def successful(self) -> bool:
"""Was the execution of this APDU successful?"""
method = getattr(self, '_is_success', None)
if callable(method):
return method()
# default case: only 9000 is success
if self.sw == b'\x90\x00':
return True
# This is not really a generic positive APDU SW but specific to UICC/SIM
if self.sw[0] == 0x91:
return True
return False
class ApduCommand(Apdu, metaclass=ApduCommandMeta):
"""Base class from which you would derive individual commands/instructions like SELECT.
A derived class represents a decoder for a specific instruction.
An instance of such a derived class is one concrete APDU."""
# fall-back constructs if the derived class provides no override
_construct_p1 = Byte
_construct_p2 = Byte
_construct = GreedyBytes
_construct_rsp = GreedyBytes
_tlv = None
_tlv_rsp = None
def __init__(self, cmd: BytesOrHex, rsp: Optional[BytesOrHex] = None):
"""Instantiate a new ApduCommand from give cmd + resp."""
# store raw data
super().__init__(cmd, rsp)
# default to 'empty' ID column. To be set to useful values (like record number)
# by derived class {cmd_rsp}_to_dict() or process() methods
self.col_id = '-'
# fields only set by process_* methods
self.file = None
self.lchan = None
self.processed = None
# the methods below could raise exceptions and those handlers might assume cmd_{dict,resp}
self.cmd_dict = None
self.rsp_dict = None
# interpret the data
self.cmd_dict = self.cmd_to_dict()
self.rsp_dict = self.rsp_to_dict() if self.rsp else {}
@classmethod
def from_apdu(cls, apdu:Apdu, **kwargs) -> 'ApduCommand':
"""Instantiate an ApduCommand from an existing APDU."""
return cls(cmd=apdu.cmd, rsp=apdu.rsp, **kwargs)
@classmethod
def from_bytes(cls, buffer:bytes) -> 'ApduCommand':
"""Instantiate an ApduCommand from a linear byte buffer containing hdr,cmd,rsp,sw.
This is for example used when parsing GSMTAP traces that traditionally contain the
full command and response portion in one packet: "CLA INS P1 P2 P3 DATA SW" and we
now need to figure out whether the DATA part is part of the CMD or the RSP"""
apdu_case = cls.get_apdu_case(buffer)
if apdu_case in [1, 2]:
# data is part of response
return cls(buffer[:5], buffer[5:])
if apdu_case in [3, 4]:
# data is part of command
lc = buffer[4]
return cls(buffer[:5+lc], buffer[5+lc:])
raise ValueError('%s: Invalid APDU Case %u' % (cls.__name__, apdu_case))
@property
def path(self) -> List[str]:
"""Return (if known) the path as list of files to the file on which this command operates."""
if self.file:
return self.file.fully_qualified_path()
return []
@property
def path_str(self) -> str:
"""Return (if known) the path as string to the file on which this command operates."""
if self.file:
return self.file.fully_qualified_path_str()
return ''
@property
def col_sw(self) -> str:
"""Return the ansi-colorized status word. Green==OK, Red==Error"""
if self.successful:
return colored(b2h(self.sw), 'green')
return colored(b2h(self.sw), 'red')
@property
def lchan_nr(self) -> int:
"""Logical channel number over which this ApduCommand was transmitted."""
if self.lchan:
return self.lchan.lchan_nr
return lchan_nr_from_cla(self.cla)
def __str__(self) -> str:
return '%02u %s(%s): %s' % (self.lchan_nr, type(self).__name__, self.path_str, self.to_dict())
def __repr__(self) -> str:
return '%s(INS=%02x,CLA=%s)' % (self.__class__, self.ins, self.cla)
def _process_fallback(self, rs: RuntimeState):
"""Fall-back function to be called if there is no derived-class-specific
process_global or process_on_lchan method. Uses information from APDU decode."""
self.processed = {}
if 'p1' not in self.cmd_dict:
self.processed = self.to_dict()
else:
self.processed['p1'] = self.cmd_dict['p1']
self.processed['p2'] = self.cmd_dict['p2']
if 'body' in self.cmd_dict and self.cmd_dict['body']:
self.processed['cmd'] = self.cmd_dict['body']
if 'body' in self.rsp_dict and self.rsp_dict['body']:
self.processed['rsp'] = self.rsp_dict['body']
return self.processed
def process(self, rs: RuntimeState):
# if there is a global method, use that; else use process_on_lchan
method = getattr(self, 'process_global', None)
if callable(method):
self.processed = method(rs)
return self.processed
method = getattr(self, 'process_on_lchan', None)
if callable(method):
self.lchan = rs.get_lchan_by_cla(self.cla)
self.processed = method(self.lchan)
return self.processed
# if none of the two methods exist:
return self._process_fallback(rs)
@classmethod
def get_apdu_case(cls, hdr:bytes) -> int:
if hasattr(cls, '_apdu_case'):
return cls._apdu_case
method = getattr(cls, '_get_apdu_case', None)
if callable(method):
return method(hdr)
raise ValueError('%s: Class definition missing _apdu_case attribute or _get_apdu_case method' % cls.__name__)
@classmethod
def match_cla(cls, cla) -> bool:
"""Does the given CLA match the CLA list of the command?."""
if not isinstance(cla, str):
cla = '%02X' % cla
cla = cla.upper()
# see https://github.com/PyCQA/pylint/issues/7219
# pylint: disable=no-member
for cla_match in cls._cla:
cla_masked = ""
for i in range(0, 2):
if cla_match[i] == 'X':
cla_masked += 'X'
else:
cla_masked += cla[i]
if cla_masked == cla_match.upper():
return True
return False
def cmd_to_dict(self) -> Dict:
"""Convert the Command part of the APDU to a dict."""
method = getattr(self, '_decode_cmd', None)
if callable(method):
return method()
else:
return self._cmd_to_dict()
def _cmd_to_dict(self) -> Dict:
"""back-end function performing automatic decoding using _construct / _tlv."""
r = {}
method = getattr(self, '_decode_p1p2', None)
if callable(method):
r = self._decode_p1p2()
else:
r['p1'] = parse_construct(self._construct_p1, self.p1.to_bytes(1, 'big'))
r['p2'] = parse_construct(self._construct_p2, self.p2.to_bytes(1, 'big'))
r['p3'] = self.p3
if self.cmd_data:
if self._tlv:
ie = self._tlv()
ie.from_tlv(self.cmd_data)
r['body'] = ie.to_dict()
else:
r['body'] = parse_construct(self._construct, self.cmd_data)
return r
def rsp_to_dict(self) -> Dict:
"""Convert the Response part of the APDU to a dict."""
method = getattr(self, '_decode_rsp', None)
if callable(method):
return method()
else:
r = {}
if self.rsp_data:
if self._tlv_rsp:
ie = self._tlv_rsp()
ie.from_tlv(self.rsp_data)
r['body'] = ie.to_dict()
else:
r['body'] = parse_construct(self._construct_rsp, self.rsp_data)
r['sw'] = b2h(self.sw)
return r
def to_dict(self) -> Dict:
"""Convert the entire APDU to a dict."""
return {'cmd': self.cmd_dict, 'rsp': self.rsp_dict}
def to_json(self) -> str:
"""Convert the entire APDU to JSON."""
d = self.to_dict()
return json.dumps(d)
def _determine_file(self, lchan) -> CardFile:
"""Helper function for read/update commands that might use SFI instead of selected file.
Expects that the self.cmd_dict has already been populated with the 'file' member."""
if self.cmd_dict['file'] == 'currently_selected_ef':
self.file = lchan.selected_file
elif self.cmd_dict['file'] == 'sfi':
cwd = lchan.get_cwd()
self.file = cwd.lookup_file_by_sfid(self.cmd_dict['sfi'])
class ApduCommandSet:
"""A set of card instructions, typically specified within one spec."""
def __init__(self, name: str, cmds: List[ApduCommand] =[]):
self.name = name
self.cmds = {c._ins: c for c in cmds}
def __str__(self) -> str:
return self.name
def __getitem__(self, idx) -> ApduCommand:
return self.cmds[idx]
def __add__(self, other) -> 'ApduCommandSet':
if isinstance(other, ApduCommand):
if other.ins in self.cmds:
raise ValueError('%s: INS 0x%02x already defined: %s' %
(self, other.ins, self.cmds[other.ins]))
self.cmds[other.ins] = other
elif isinstance(other, ApduCommandSet):
for c in other.cmds.keys():
self.cmds[c] = other.cmds[c]
else:
raise ValueError(
'%s: Unsupported type to add operator: %s' % (self, other))
return self
def lookup(self, ins, cla=None) -> Optional[ApduCommand]:
"""look-up the command within the CommandSet."""
ins = int(ins)
if not ins in self.cmds:
return None
cmd = self.cmds[ins]
if cla and not cmd.match_cla(cla):
return None
return cmd
def parse_cmd_apdu(self, apdu: Apdu) -> ApduCommand:
"""Parse a Command-APDU. Returns an instance of an ApduCommand derived class."""
# first look-up which of our member classes match CLA + INS
a_cls = self.lookup(apdu.ins, apdu.cla)
if not a_cls:
raise ValueError('Unknown CLA=%02X INS=%02X' % (apdu.cla, apdu.ins))
# then create an instance of that class and return it
return a_cls.from_apdu(apdu)
def parse_cmd_bytes(self, buf:bytes) -> ApduCommand:
"""Parse from a buffer (simtrace style). Returns an instance of an ApduCommand derived class."""
# first look-up which of our member classes match CLA + INS
cla = buf[0]
ins = buf[1]
a_cls = self.lookup(ins, cla)
if not a_cls:
raise ValueError('Unknown CLA=%02X INS=%02X' % (cla, ins))
# then create an instance of that class and return it
return a_cls.from_bytes(buf)
class ApduHandler(abc.ABC):
@abc.abstractmethod
def input(self, cmd: bytes, rsp: bytes):
pass
class TpduFilter(ApduHandler):
"""The TpduFilter removes the T=0 specific GET_RESPONSE from the TPDU stream and
calls the ApduHandler only with the actual APDU command and response parts."""
def __init__(self, apdu_handler: ApduHandler):
self.apdu_handler = apdu_handler
self.state = 'INIT'
self.last_cmd = None
def input_tpdu(self, tpdu:Tpdu):
# handle SW=61xx / 6Cxx
if tpdu.sw[0] == 0x61 or tpdu.sw[0] == 0x6C:
self.state = 'WAIT_GET_RESPONSE'
# handle successive 61/6c responses by stupid phone/modem OS
if tpdu.ins != 0xC0:
self.last_cmd = tpdu.cmd
return None
else:
if self.last_cmd:
icmd = self.last_cmd
self.last_cmd = None
else:
icmd = tpdu.cmd
apdu = Apdu(icmd, tpdu.rsp)
if self.apdu_handler:
return self.apdu_handler.input(apdu)
return Apdu(icmd, tpdu.rsp)
def input(self, cmd: bytes, rsp: bytes):
if isinstance(cmd, str):
cmd = bytes.fromhex(cmd)
if isinstance(rsp, str):
rsp = bytes.fromhex(rsp)
tpdu = Tpdu(cmd, rsp)
return self.input_tpdu(tpdu)
class ApduDecoder(ApduHandler):
def __init__(self, cmd_set: ApduCommandSet):
self.cmd_set = cmd_set
def input(self, apdu: Apdu):
return self.cmd_set.parse_cmd_apdu(apdu)
class CardReset:
def __init__(self, atr: bytes):
self.atr = atr
def __str__(self):
if self.atr:
return '%s(%s)' % (type(self).__name__, b2h(self.atr))
return '%s' % (type(self).__name__)

82
pySim/apdu/global_platform.py Normal file
View File

@@ -0,0 +1,82 @@
# coding=utf-8
"""APDU definition/decoder of GlobalPLatform Card Spec (currently 2.1.1)
(C) 2022-2024 by Harald Welte <laforge@osmocom.org>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
from construct import FlagsEnum, Struct
from osmocom.tlv import flatten_dict_lists
from osmocom.construct import *
from pySim.apdu import ApduCommand, ApduCommandSet
from pySim.global_platform import InstallParameters
class GpDelete(ApduCommand, n='DELETE', ins=0xE4, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
class GpStoreData(ApduCommand, n='STORE DATA', ins=0xE2, cla=['8X', 'CX', 'EX']):
@classmethod
def _get_apdu_case(cls, hdr:bytes) -> int:
p1 = hdr[2]
if p1 & 0x01:
return 4
else:
return 3
class GpGetDataCA(ApduCommand, n='GET DATA', ins=0xCA, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
class GpGetDataCB(ApduCommand, n='GET DATA', ins=0xCB, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
class GpGetStatus(ApduCommand, n='GET STATUS', ins=0xF2, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
# GPCS Section 11.5.2
class GpInstall(ApduCommand, n='INSTALL', ins=0xE6, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
_construct_p1 = FlagsEnum(Byte, more_commands=0x80, for_registry_update=0x40,
for_personalization=0x20, for_extradition=0x10,
for_make_selectable=0x08, for_install=0x04, for_load=0x02)
_construct_p2 = Enum(Byte, no_info_provided=0x00, beginning_of_combined=0x01,
end_of_combined=0x03)
_construct = Struct('load_file_aid'/Prefixed(Int8ub, GreedyBytes),
'module_aid'/Prefixed(Int8ub, GreedyBytes),
'application_aid'/Prefixed(Int8ub, GreedyBytes),
'privileges'/Prefixed(Int8ub, GreedyBytes),
'install_parameters'/Prefixed(Int8ub, GreedyBytes), # TODO: InstallParameters
'install_token'/Prefixed(Int8ub, GreedyBytes))
def _decode_cmd(self):
# first use _construct* above
res = self._cmd_to_dict()
# then do TLV decode of install_parameters
ip = InstallParameters()
ip.from_tlv(res['body']['install_parameters'])
res['body']['install_parameters'] = flatten_dict_lists(ip.to_dict())
return res
class GpLoad(ApduCommand, n='LOAD', ins=0xE8, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
class GpPutKey(ApduCommand, n='PUT KEY', ins=0xD8, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
class GpSetStatus(ApduCommand, n='SET STATUS', ins=0xF0, cla=['8X', 'CX', 'EX']):
_apdu_case = 3
ApduCommands = ApduCommandSet('GlobalPlatform v2.3.1', cmds=[GpDelete, GpStoreData,
GpGetDataCA, GpGetDataCB, GpGetStatus, GpInstall,
GpLoad, GpPutKey, GpSetStatus])

531
pySim/apdu/ts_102_221.py Normal file
View File

@@ -0,0 +1,531 @@
# coding=utf-8
"""APDU definitions/decoders of ETSI TS 102 221, the core UICC spec.
(C) 2022-2024 by Harald Welte <laforge@osmocom.org>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
from typing import Optional, Dict
import logging
from construct import GreedyRange, Struct
from osmocom.utils import i2h
from osmocom.construct import *
from pySim.filesystem import *
from pySim.runtime import RuntimeLchan
from pySim.apdu import ApduCommand, ApduCommandSet
from pySim import cat
logger = logging.getLogger(__name__)
# TS 102 221 Section 11.1.1
class UiccSelect(ApduCommand, n='SELECT', ins=0xA4, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_p1 = Enum(Byte, df_ef_or_mf_by_file_id=0, child_df_of_current_df=1, parent_df_of_current_df=3,
df_name=4, path_from_mf=8, path_from_current_df=9)
_construct_p2 = BitStruct(Flag,
'app_session_control'/Enum(BitsInteger(2), activation_reset=0, termination=2),
'return'/Enum(BitsInteger(3), fcp=1, no_data=3),
'aid_control'/Enum(BitsInteger(2), first_or_only=0, last=1, next=2, previous=3))
@staticmethod
def _find_aid_substr(selectables, aid) -> Optional[CardADF]:
# full-length match
if aid in selectables:
return selectables[aid]
# sub-string match
for s in selectables.keys():
if aid[:len(s)] == s:
return selectables[s]
return None
def process_on_lchan(self, lchan: RuntimeLchan):
mode = self.cmd_dict['p1']
if mode in ['path_from_mf', 'path_from_current_df']:
# rewind to MF, if needed
if mode == 'path_from_mf':
lchan.selected_file = lchan.rs.mf
path = [self.cmd_data[i:i+2] for i in range(0, len(self.cmd_data), 2)]
for file in path:
file_hex = b2h(file)
if file_hex == '7fff': # current application
if not lchan.selected_adf:
sels = lchan.rs.mf.get_app_selectables(['ANAMES'])
# HACK: Assume USIM
logger.warning('SELECT relative to current ADF, but no ADF selected. Assuming ADF.USIM')
lchan.selected_adf = sels['ADF.USIM']
lchan.selected_file = lchan.selected_adf
#print("\tSELECT CUR_ADF %s" % lchan.selected_file)
# iterate to next element in path
continue
else:
sels = lchan.selected_file.get_selectables(['FIDS','MF','PARENT','SELF'])
if file_hex in sels:
if self.successful:
#print("\tSELECT %s" % sels[file_hex])
lchan.selected_file = sels[file_hex]
else:
#print("\tSELECT %s FAILED" % sels[file_hex])
pass
# iterate to next element in path
continue
logger.warning('SELECT UNKNOWN FID %s (%s)', file_hex, '/'.join([b2h(x) for x in path]))
elif mode == 'df_ef_or_mf_by_file_id':
if len(self.cmd_data) != 2:
raise ValueError('Expecting a 2-byte FID')
sels = lchan.selected_file.get_selectables(['FIDS','MF','PARENT','SELF'])
file_hex = b2h(self.cmd_data)
if file_hex in sels:
if self.successful:
#print("\tSELECT %s" % sels[file_hex])
lchan.selected_file = sels[file_hex]
else:
#print("\tSELECT %s FAILED" % sels[file_hex])
pass
else:
logger.warning('SELECT UNKNOWN FID %s', file_hex)
elif mode == 'df_name':
# Select by AID (can be sub-string!)
aid = b2h(self.cmd_dict['body'])
sels = lchan.rs.mf.get_app_selectables(['AIDS'])
adf = self._find_aid_substr(sels, aid)
if adf:
lchan.selected_adf = adf
lchan.selected_file = lchan.selected_adf
#print("\tSELECT AID %s" % adf)
else:
logger.warning('SELECT UNKNOWN AID %s', aid)
else:
raise ValueError('Select Mode %s not implemented' % mode)
# decode the SELECT response
if self.successful:
self.file = lchan.selected_file
if 'body' in self.rsp_dict:
# not every SELECT is asking for the FCP in response...
return lchan.selected_file.decode_select_response(b2h(self.rsp_dict['body']))
return None
# TS 102 221 Section 11.1.2
class UiccStatus(ApduCommand, n='STATUS', ins=0xF2, cla=['8X', 'CX', 'EX']):
_apdu_case = 2
_construct_p1 = Enum(Byte, no_indication=0, current_app_is_initialized=1, terminal_will_terminate_current_app=2)
_construct_p2 = Enum(Byte, response_like_select=0, response_df_name_tlv=1, response_no_data=0x0c)
def process_on_lchan(self, lchan):
if self.cmd_dict['p2'] == 'response_like_select':
return lchan.selected_file.decode_select_response(b2h(self.rsp_dict['body']))
def _decode_binary_p1p2(p1, p2) -> Dict:
ret = {}
if p1 & 0x80:
ret['file'] = 'sfi'
ret['sfi'] = p1 & 0x1f
ret['offset'] = p2
else:
ret['file'] = 'currently_selected_ef'
ret['offset'] = ((p1 & 0x7f) << 8) & p2
return ret
# TS 102 221 Section 11.1.3
class ReadBinary(ApduCommand, n='READ BINARY', ins=0xB0, cla=['0X', '4X', '6X']):
_apdu_case = 2
def _decode_p1p2(self):
return _decode_binary_p1p2(self.p1, self.p2)
def process_on_lchan(self, lchan):
self._determine_file(lchan)
if not isinstance(self.file, TransparentEF):
return b2h(self.rsp_data)
# our decoders don't work for non-zero offsets / short reads
if self.cmd_dict['offset'] != 0 or self.lr < self.file.size[0]:
return b2h(self.rsp_data)
method = getattr(self.file, 'decode_bin', None)
if self.successful and callable(method):
return method(self.rsp_data)
# TS 102 221 Section 11.1.4
class UpdateBinary(ApduCommand, n='UPDATE BINARY', ins=0xD6, cla=['0X', '4X', '6X']):
_apdu_case = 3
def _decode_p1p2(self):
return _decode_binary_p1p2(self.p1, self.p2)
def process_on_lchan(self, lchan):
self._determine_file(lchan)
if not isinstance(self.file, TransparentEF):
return b2h(self.rsp_data)
# our decoders don't work for non-zero offsets / short writes
if self.cmd_dict['offset'] != 0 or self.lc < self.file.size[0]:
return b2h(self.cmd_data)
method = getattr(self.file, 'decode_bin', None)
if self.successful and callable(method):
return method(self.cmd_data)
def _decode_record_p1p2(p1, p2):
ret = {}
ret['record_number'] = p1
if p2 >> 3 == 0:
ret['file'] = 'currently_selected_ef'
else:
ret['file'] = 'sfi'
ret['sfi'] = p2 >> 3
mode = p2 & 0x7
if mode == 2:
ret['mode'] = 'next_record'
elif mode == 3:
ret['mode'] = 'previous_record'
elif mode == 8:
ret['mode'] = 'absolute_current'
return ret
# TS 102 221 Section 11.1.5
class ReadRecord(ApduCommand, n='READ RECORD', ins=0xB2, cla=['0X', '4X', '6X']):
_apdu_case = 2
def _decode_p1p2(self):
r = _decode_record_p1p2(self.p1, self.p2)
self.col_id = '%02u' % r['record_number']
return r
def process_on_lchan(self, lchan):
self._determine_file(lchan)
if not isinstance(self.file, LinFixedEF):
return b2h(self.rsp_data)
method = getattr(self.file, 'decode_record_bin', None)
if self.successful and callable(method):
return method(self.rsp_data, self.cmd_dict['record_number'])
# TS 102 221 Section 11.1.6
class UpdateRecord(ApduCommand, n='UPDATE RECORD', ins=0xDC, cla=['0X', '4X', '6X']):
_apdu_case = 3
def _decode_p1p2(self):
r = _decode_record_p1p2(self.p1, self.p2)
self.col_id = '%02u' % r['record_number']
return r
def process_on_lchan(self, lchan):
self._determine_file(lchan)
if not isinstance(self.file, LinFixedEF):
return b2h(self.cmd_data)
method = getattr(self.file, 'decode_record_bin', None)
if self.successful and callable(method):
return method(self.cmd_data, self.cmd_dict['record_number'])
# TS 102 221 Section 11.1.7
class SearchRecord(ApduCommand, n='SEARCH RECORD', ins=0xA2, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_rsp = GreedyRange(Int8ub)
def _decode_p1p2(self):
ret = {}
sfi = self.p2 >> 3
if sfi == 0:
ret['file'] = 'currently_selected_ef'
else:
ret['file'] = 'sfi'
ret['sfi'] = sfi
mode = self.p2 & 0x7
if mode in [0x4, 0x5]:
if mode == 0x4:
ret['mode'] = 'forward_search'
else:
ret['mode'] = 'backward_search'
ret['record_number'] = self.p1
self.col_id = '%02u' % ret['record_number']
elif mode == 6:
ret['mode'] = 'enhanced_search'
# TODO: further decode
elif mode == 7:
ret['mode'] = 'proprietary_search'
return ret
def _decode_cmd(self):
ret = self._decode_p1p2()
if self.cmd_data:
if ret['mode'] == 'enhanced_search':
ret['search_indication'] = b2h(self.cmd_data[:2])
ret['search_string'] = b2h(self.cmd_data[2:])
else:
ret['search_string'] = b2h(self.cmd_data)
return ret
def process_on_lchan(self, lchan):
self._determine_file(lchan)
return self.to_dict()
# TS 102 221 Section 11.1.8
class Increase(ApduCommand, n='INCREASE', ins=0x32, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
PinConstructP2 = BitStruct('scope'/Enum(Flag, global_mf=0, specific_df_adf=1),
BitsInteger(2), 'reference_data_nr'/BitsInteger(5))
# TS 102 221 Section 11.1.9
class VerifyPin(ApduCommand, n='VERIFY PIN', ins=0x20, cla=['0X', '4X', '6X']):
_apdu_case = 3
_construct_p2 = PinConstructP2
@staticmethod
def _pin_process(apdu):
processed = {
'scope': apdu.cmd_dict['p2']['scope'],
'referenced_data_nr': apdu.cmd_dict['p2']['reference_data_nr'],
}
if apdu.lc == 0:
# this is just a question on the counters remaining
processed['mode'] = 'check_remaining_attempts'
else:
processed['pin'] = b2h(apdu.cmd_data)
if apdu.sw[0] == 0x63:
processed['remaining_attempts'] = apdu.sw[1] & 0xf
return processed
@staticmethod
def _pin_is_success(sw):
return bool(sw[0] == 0x63)
def process_on_lchan(self, _lchan: RuntimeLchan):
return VerifyPin._pin_process(self)
def _is_success(self):
return VerifyPin._pin_is_success(self.sw)
# TS 102 221 Section 11.1.10
class ChangePin(ApduCommand, n='CHANGE PIN', ins=0x24, cla=['0X', '4X', '6X']):
_apdu_case = 3
_construct_p2 = PinConstructP2
def process_on_lchan(self, _lchan: RuntimeLchan):
return VerifyPin._pin_process(self)
def _is_success(self):
return VerifyPin._pin_is_success(self.sw)
# TS 102 221 Section 11.1.11
class DisablePin(ApduCommand, n='DISABLE PIN', ins=0x26, cla=['0X', '4X', '6X']):
_apdu_case = 3
_construct_p2 = PinConstructP2
def process_on_lchan(self, _lchan: RuntimeLchan):
return VerifyPin._pin_process(self)
def _is_success(self):
return VerifyPin._pin_is_success(self.sw)
# TS 102 221 Section 11.1.12
class EnablePin(ApduCommand, n='ENABLE PIN', ins=0x28, cla=['0X', '4X', '6X']):
_apdu_case = 3
_construct_p2 = PinConstructP2
def process_on_lchan(self, _lchan: RuntimeLchan):
return VerifyPin._pin_process(self)
def _is_success(self):
return VerifyPin._pin_is_success(self.sw)
# TS 102 221 Section 11.1.13
class UnblockPin(ApduCommand, n='UNBLOCK PIN', ins=0x2C, cla=['0X', '4X', '6X']):
_apdu_case = 3
_construct_p2 = PinConstructP2
def process_on_lchan(self, _lchan: RuntimeLchan):
return VerifyPin._pin_process(self)
def _is_success(self):
return VerifyPin._pin_is_success(self.sw)
# TS 102 221 Section 11.1.14
class DeactivateFile(ApduCommand, n='DEACTIVATE FILE', ins=0x04, cla=['0X', '4X', '6X']):
_apdu_case = 1
_construct_p1 = BitStruct(BitsInteger(4),
'select_mode'/Enum(BitsInteger(4), ef_by_file_id=0,
path_from_mf=8, path_from_current_df=9))
# TS 102 221 Section 11.1.15
class ActivateFile(ApduCommand, n='ACTIVATE FILE', ins=0x44, cla=['0X', '4X', '6X']):
_apdu_case = 1
_construct_p1 = DeactivateFile._construct_p1
# TS 102 221 Section 11.1.16
auth_p2_construct = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
BitsInteger(2),
'reference_data_nr'/BitsInteger(5))
class Authenticate88(ApduCommand, n='AUTHENTICATE', ins=0x88, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_p2 = auth_p2_construct
# TS 102 221 Section 11.1.16
class Authenticate89(ApduCommand, n='AUTHENTICATE', ins=0x89, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_p2 = auth_p2_construct
# TS 102 221 Section 11.1.17
class ManageChannel(ApduCommand, n='MANAGE CHANNEL', ins=0x70, cla=['0X', '4X', '6X']):
_apdu_case = 2
_construct_p1 = Enum(Flag, open_channel=0, close_channel=1)
_construct_p2 = Struct('logical_channel_number'/Int8ub)
_construct_rsp = Struct('logical_channel_number'/Int8ub)
def process_global(self, rs):
if not self.successful:
return
mode = self.cmd_dict['p1']
if mode == 'open_channel':
created_channel_nr = self.cmd_dict['p2']['logical_channel_number']
if created_channel_nr == 0:
# auto-assignment by UICC
# pylint: disable=unsubscriptable-object
created_channel_nr = self.rsp_data[0]
manage_channel = rs.get_lchan_by_cla(self.cla)
manage_channel.add_lchan(created_channel_nr)
self.col_id = '%02u' % created_channel_nr
return {'mode': mode, 'created_channel': created_channel_nr }
if mode == 'close_channel':
closed_channel_nr = self.cmd_dict['p2']['logical_channel_number']
rs.del_lchan(closed_channel_nr)
self.col_id = '%02u' % closed_channel_nr
return {'mode': mode, 'closed_channel': closed_channel_nr }
raise ValueError('Unsupported MANAGE CHANNEL P1=%02X' % self.p1)
# TS 102 221 Section 11.1.18
class GetChallenge(ApduCommand, n='GET CHALLENGE', ins=0x84, cla=['0X', '4X', '6X']):
_apdu_case = 2
# TS 102 221 Section 11.1.19
class TerminalCapability(ApduCommand, n='TERMINAL CAPABILITY', ins=0xAA, cla=['8X', 'CX', 'EX']):
_apdu_case = 3
# TS 102 221 Section 11.1.20
class ManageSecureChannel(ApduCommand, n='MANAGE SECURE CHANNEL', ins=0x73, cla=['0X', '4X', '6X']):
@classmethod
def _get_apdu_case(cls, hdr:bytes) -> int:
p1 = hdr[2]
p2 = hdr[3]
if p1 & 0x7 == 0: # retrieve UICC Endpoints
return 2
if p1 & 0xf in [1,2,3]: # establish sa, start secure channel SA
p2_cmd = p2 >> 5
if p2_cmd in [0,2,4]: # command data
return 3
if p2_cmd in [1,3,5]: # response data
return 2
if p1 & 0xf == 4: # terminate secure channel SA
return 3
raise ValueError('%s: Unable to detect APDU case for %s' % (cls.__name__, b2h(hdr)))
# TS 102 221 Section 11.1.21
class TransactData(ApduCommand, n='TRANSACT DATA', ins=0x75, cla=['0X', '4X', '6X']):
@classmethod
def _get_apdu_case(cls, hdr:bytes) -> int:
p1 = hdr[2]
if p1 & 0x04:
return 3
return 2
# TS 102 221 Section 11.1.22
class SuspendUicc(ApduCommand, n='SUSPEND UICC', ins=0x76, cla=['80']):
_apdu_case = 4
_construct_p1 = BitStruct('rfu'/BitsInteger(7), 'mode'/Enum(Flag, suspend=0, resume=1))
# TS 102 221 Section 11.1.23
class GetIdentity(ApduCommand, n='GET IDENTITY', ins=0x78, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
_construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1), BitsInteger(7))
# TS 102 221 Section 11.1.24
class ExchangeCapabilities(ApduCommand, n='EXCHANGE CAPABILITIES', ins=0x7A, cla=['80']):
_apdu_case = 4
# TS 102 221 Section 11.2.1
class TerminalProfile(ApduCommand, n='TERMINAL PROFILE', ins=0x10, cla=['80']):
_apdu_case = 3
# TS 102 221 Section 11.2.2 / TS 102 223
class Envelope(ApduCommand, n='ENVELOPE', ins=0xC2, cla=['80']):
_apdu_case = 4
_tlv = cat.EventCollection
# TS 102 221 Section 11.2.3 / TS 102 223
class Fetch(ApduCommand, n='FETCH', ins=0x12, cla=['80']):
_apdu_case = 2
_tlv_rsp = cat.ProactiveCommand
# TS 102 221 Section 11.2.3 / TS 102 223
class TerminalResponse(ApduCommand, n='TERMINAL RESPONSE', ins=0x14, cla=['80']):
_apdu_case = 3
_tlv = cat.TerminalResponse
# TS 102 221 Section 11.3.1
class RetrieveData(ApduCommand, n='RETRIEVE DATA', ins=0xCB, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
@staticmethod
def _tlv_decode_cmd(self : ApduCommand) -> Dict:
c = {}
if self.p2 & 0xc0 == 0x80:
c['mode'] = 'first_block'
sfi = self.p2 & 0x1f
if sfi == 0:
c['file'] = 'currently_selected_ef'
else:
c['file'] = 'sfi'
c['sfi'] = sfi
c['tag'] = i2h([self.cmd_data[0]])
elif self.p2 & 0xdf == 0x00:
c['mode'] = 'next_block'
elif self.p2 & 0xdf == 0x40:
c['mode'] = 'retransmit_previous_block'
else:
logger.warning('%s: invalid P2=%02x', self, self.p2)
return c
def _decode_cmd(self):
return RetrieveData._tlv_decode_cmd(self)
def _decode_rsp(self):
# TODO: parse tag/len/val?
return b2h(self.rsp_data)
# TS 102 221 Section 11.3.2
class SetData(ApduCommand, n='SET DATA', ins=0xDB, cla=['8X', 'CX', 'EX']):
_apdu_case = 3
def _decode_cmd(self):
c = RetrieveData._tlv_decode_cmd(self)
if c['mode'] == 'first_block':
if len(self.cmd_data) == 0:
c['delete'] = True
# TODO: parse tag/len/val?
c['data'] = b2h(self.cmd_data)
return c
# TS 102 221 Section 12.1.1
class GetResponse(ApduCommand, n='GET RESPONSE', ins=0xC0, cla=['0X', '4X', '6X']):
_apdu_case = 2
ApduCommands = ApduCommandSet('TS 102 221', cmds=[UiccSelect, UiccStatus, ReadBinary, UpdateBinary, ReadRecord,
UpdateRecord, SearchRecord, Increase, VerifyPin, ChangePin, DisablePin,
EnablePin, UnblockPin, DeactivateFile, ActivateFile, Authenticate88,
Authenticate89, ManageChannel, GetChallenge, TerminalCapability,
ManageSecureChannel, TransactData, SuspendUicc, GetIdentity,
ExchangeCapabilities, TerminalProfile, Envelope, Fetch, TerminalResponse,
RetrieveData, SetData, GetResponse])

60
pySim/apdu/ts_102_222.py Normal file
View File

@@ -0,0 +1,60 @@
# coding=utf-8
"""APDU definitions/decoders of ETSI TS 102 222.
(C) 2022-2024 by Harald Welte <laforge@osmocom.org>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
import logging
from construct import Struct
from osmocom.construct import *
from pySim.apdu import ApduCommand, ApduCommandSet
from pySim.ts_102_221 import FcpTemplate
logger = logging.getLogger(__name__)
# TS 102 222 Section 6.3
class CreateFile(ApduCommand, n='CREATE FILE', ins=0xE0, cla=['0X', '4X', 'EX']):
_apdu_case = 3
_tlv = FcpTemplate
# TS 102 222 Section 6.4
class DeleteFile(ApduCommand, n='DELETE FILE', ins=0xE4, cla=['0X', '4X']):
_apdu_case = 3
_construct = Struct('file_id'/Bytes(2))
# TS 102 222 Section 6.7
class TerminateDF(ApduCommand, n='TERMINATE DF', ins=0xE6, cla=['0X', '4X']):
_apdu_case = 1
# TS 102 222 Section 6.8
class TerminateEF(ApduCommand, n='TERMINATE EF', ins=0xE8, cla=['0X', '4X']):
_apdu_case = 1
# TS 102 222 Section 6.9
class TerminateCardUsage(ApduCommand, n='TERMINATE CARD USAGE', ins=0xFE, cla=['0X', '4X']):
_apdu_case = 1
# TS 102 222 Section 6.10
class ResizeFile(ApduCommand, n='RESIZE FILE', ins=0xD4, cla=['8X', 'CX', 'EX']):
_apdu_case = 3
_construct_p1 = Enum(Byte, mode_0=0, mode_1=1)
_tlv = FcpTemplate
ApduCommands = ApduCommandSet('TS 102 222', cmds=[CreateFile, DeleteFile, TerminateDF,
TerminateEF, TerminateCardUsage, ResizeFile])

113
pySim/apdu/ts_31_102.py Normal file
View File

@@ -0,0 +1,113 @@
# -*- coding: utf-8 -*-
# without this, pylint will fail when inner classes are used
# within the 'nested' kwarg of our TlvMeta metaclass on python 3.7 :(
# pylint: disable=undefined-variable
"""
APDU commands of 3GPP TS 31.102 V16.6.0
"""
from typing import Dict
from construct import BitStruct, Enum, BitsInteger, Int8ub, this, Struct, If, Switch, Const
from construct import Optional as COptional
from osmocom.construct import *
from pySim.filesystem import *
from pySim.ts_31_102 import SUCI_TlvDataObject
from pySim.apdu import ApduCommand, ApduCommandSet
# Copyright (C) 2022 Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Mapping between USIM Service Number and its description
# TS 31.102 Section 7.1
class UsimAuthenticateEven(ApduCommand, n='AUTHENTICATE', ins=0x88, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
BitsInteger(4),
'authentication_context'/Enum(BitsInteger(3), gsm=0, umts=1,
vgcs_vbs=2, gba=4))
_cs_cmd_gsm_3g = Struct('_rand_len'/Int8ub, 'rand'/Bytes(this._rand_len),
'_autn_len'/COptional(Int8ub), 'autn'/If(this._autn_len, Bytes(this._autn_len)))
_cs_cmd_vgcs = Struct('_vsid_len'/Int8ub, 'vservice_id'/Bytes(this._vsid_len),
'_vkid_len'/Int8ub, 'vk_id'/Bytes(this._vkid_len),
'_vstk_rand_len'/Int8ub, 'vstk_rand'/Bytes(this._vstk_rand_len))
_cmd_gba_bs = Struct('_rand_len'/Int8ub, 'rand'/Bytes(this._rand_len),
'_autn_len'/Int8ub, 'autn'/Bytes(this._autn_len))
_cmd_gba_naf = Struct('_naf_id_len'/Int8ub, 'naf_id'/Bytes(this._naf_id_len),
'_impi_len'/Int8ub, 'impi'/Bytes(this._impi_len))
_cs_cmd_gba = Struct('tag'/Int8ub, 'body'/Switch(this.tag, { 0xDD: 'bootstrap'/_cmd_gba_bs,
0xDE: 'naf_derivation'/_cmd_gba_naf }))
_cs_rsp_gsm = Struct('_len_sres'/Int8ub, 'sres'/Bytes(this._len_sres),
'_len_kc'/Int8ub, 'kc'/Bytes(this._len_kc))
_rsp_3g_ok = Struct('_len_res'/Int8ub, 'res'/Bytes(this._len_res),
'_len_ck'/Int8ub, 'ck'/Bytes(this._len_ck),
'_len_ik'/Int8ub, 'ik'/Bytes(this._len_ik),
'_len_kc'/COptional(Int8ub), 'kc'/If(this._len_kc, Bytes(this._len_kc)))
_rsp_3g_sync = Struct('_len_auts'/Int8ub, 'auts'/Bytes(this._len_auts))
_cs_rsp_3g = Struct('tag'/Int8ub, 'body'/Switch(this.tag, { 0xDB: 'success'/_rsp_3g_ok,
0xDC: 'sync_fail'/_rsp_3g_sync}))
_cs_rsp_vgcs = Struct(Const(b'\xDB'), '_vstk_len'/Int8ub, 'vstk'/Bytes(this._vstk_len))
_cs_rsp_gba_naf = Struct(Const(b'\xDB'), '_ks_ext_naf_len'/Int8ub, 'ks_ext_naf'/Bytes(this._ks_ext_naf_len))
def _decode_cmd(self) -> Dict:
r = {}
r['p1'] = parse_construct(self._construct_p1, self.p1.to_bytes(1, 'big'))
r['p2'] = parse_construct(self._construct_p2, self.p2.to_bytes(1, 'big'))
auth_ctx = r['p2']['authentication_context']
if auth_ctx in ['gsm', 'umts']:
r['body'] = parse_construct(self._cs_cmd_gsm_3g, self.cmd_data)
elif auth_ctx == 'vgcs_vbs':
r['body'] = parse_construct(self._cs_cmd_vgcs, self.cmd_data)
elif auth_ctx == 'gba':
r['body'] = parse_construct(self._cs_cmd_gba, self.cmd_data)
else:
raise ValueError('Unsupported authentication_context: %s' % auth_ctx)
return r
def _decode_rsp(self) -> Dict:
r = {}
auth_ctx = self.cmd_dict['p2']['authentication_context']
if auth_ctx == 'gsm':
r['body'] = parse_construct(self._cs_rsp_gsm, self.rsp_data)
elif auth_ctx == 'umts':
r['body'] = parse_construct(self._cs_rsp_3g, self.rsp_data)
elif auth_ctx == 'vgcs_vbs':
r['body'] = parse_construct(self._cs_rsp_vgcs, self.rsp_data)
elif auth_ctx == 'gba':
if self.cmd_dict['body']['tag'] == 0xDD:
r['body'] = parse_construct(self._cs_rsp_3g, self.rsp_data)
else:
r['body'] = parse_construct(self._cs_rsp_gba_naf, self.rsp_data)
else:
raise ValueError('Unsupported authentication_context: %s' % auth_ctx)
return r
class UsimAuthenticateOdd(ApduCommand, n='AUTHENTICATE', ins=0x89, cla=['0X', '4X', '6X']):
_apdu_case = 4
_construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
BitsInteger(4),
'authentication_context'/Enum(BitsInteger(3), mbms=5, local_key=6))
# TS 31.102 Section 7.5
class UsimGetIdentity(ApduCommand, n='GET IDENTITY', ins=0x78, cla=['8X', 'CX', 'EX']):
_apdu_case = 4
_construct_p2 = BitStruct('scope'/Enum(Flag, mf=0, df_adf_specific=1),
'identity_context'/Enum(BitsInteger(7), suci=1, suci_5g_nswo=2))
_tlv_rsp = SUCI_TlvDataObject
ApduCommands = ApduCommandSet('TS 31.102', cmds=[UsimAuthenticateEven, UsimAuthenticateOdd,
UsimGetIdentity])

34
pySim/apdu_source/__init__.py Normal file
View File

@@ -0,0 +1,34 @@
import abc
import logging
from typing import Union
from pySim.apdu import Apdu, Tpdu, CardReset, TpduFilter
PacketType = Union[Apdu, Tpdu, CardReset]
logger = logging.getLogger(__name__)
class ApduSource(abc.ABC):
def __init__(self):
self.apdu_filter = TpduFilter(None)
@abc.abstractmethod
def read_packet(self) -> PacketType:
"""Read one packet from the source."""
def read(self) -> Union[Apdu, CardReset]:
"""Main function to call by the user: Blocking read, returns Apdu or CardReset."""
apdu = None
# loop until we actually have an APDU to return
while not apdu:
r = self.read_packet()
if not r:
continue
if isinstance(r, Tpdu):
apdu = self.apdu_filter.input_tpdu(r)
elif isinstance(r, Apdu):
apdu = r
elif isinstance(r, CardReset):
apdu = r
else:
raise ValueError('Unknown read_packet() return %s' % r)
return apdu

60
pySim/apdu_source/gsmtap.py Normal file
View File

@@ -0,0 +1,60 @@
# coding=utf-8
# (C) 2022 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from osmocom.gsmtap import GsmtapReceiver
from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
from pySim.apdu.ts_102_222 import ApduCommands as UiccAdmApduCommands
from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
from pySim.apdu.global_platform import ApduCommands as GpApduCommands
from . import ApduSource, PacketType, CardReset
ApduCommands = UiccApduCommands + UiccAdmApduCommands + UsimApduCommands + GpApduCommands
class GsmtapApduSource(ApduSource):
"""ApduSource for handling GSMTAP-SIM messages received via UDP, such as
those generated by simtrace2-sniff. Note that *if* you use IP loopback
and localhost addresses (which is the default), you will need to start
this source before starting simtrace2-sniff, as otherwise the latter will
claim the GSMTAP UDP port.
"""
def __init__(self, bind_ip:str='127.0.0.1', bind_port:int=4729):
"""Create a UDP socket for receiving GSMTAP-SIM messages.
Args:
bind_ip: IP address to which the socket should be bound (default: 127.0.0.1)
bind_port: UDP port number to which the socket should be bound (default: 4729)
"""
super().__init__()
self.gsmtap = GsmtapReceiver(bind_ip, bind_port)
def read_packet(self) -> PacketType:
gsmtap_msg, _addr = self.gsmtap.read_packet()
if gsmtap_msg['type'] != 'sim':
raise ValueError('Unsupported GSMTAP type %s' % gsmtap_msg['type'])
sub_type = gsmtap_msg['sub_type']
if sub_type == 'apdu':
return ApduCommands.parse_cmd_bytes(gsmtap_msg['body'])
if sub_type == 'atr':
# card has been reset
return CardReset(gsmtap_msg['body'])
if sub_type in ['pps_req', 'pps_rsp']:
# simply ignore for now
pass
else:
raise ValueError('Unsupported GSMTAP-SIM sub-type %s' % sub_type)

88
pySim/apdu_source/pyshark_gsmtap.py Normal file
View File

@@ -0,0 +1,88 @@
# coding=utf-8
# (C) 2022 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from typing import Tuple
import pyshark
from osmocom.gsmtap import GsmtapMessage
from pySim.utils import h2b
from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
from pySim.apdu.ts_102_222 import ApduCommands as UiccAdmApduCommands
from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
from pySim.apdu.global_platform import ApduCommands as GpApduCommands
from . import ApduSource, PacketType, CardReset
ApduCommands = UiccApduCommands + UiccAdmApduCommands + UsimApduCommands + GpApduCommands
logger = logging.getLogger(__name__)
class _PysharkGsmtap(ApduSource):
"""APDU Source [provider] base class for reading GSMTAP SIM APDU via tshark."""
def __init__(self, pyshark_inst):
self.pyshark = pyshark_inst
self.bank_id = None
self.bank_slot = None
self.cmd_tpdu = None
super().__init__()
def read_packet(self) -> PacketType:
p = self.pyshark.next()
return self._parse_packet(p)
def _set_or_verify_bank_slot(self, bsl: Tuple[int, int]):
"""Keep track of the bank:slot to make sure we don't mix traces of multiple cards"""
if not self.bank_id:
self.bank_id = bsl[0]
self.bank_slot = bsl[1]
else:
if self.bank_id != bsl[0] or self.bank_slot != bsl[1]:
raise ValueError('Received data for unexpected B(%u:%u)' % (bsl[0], bsl[1]))
def _parse_packet(self, p) -> PacketType:
udp_layer = p['udp']
udp_payload_hex = udp_layer.get_field('payload').replace(':','')
gsmtap = GsmtapMessage(h2b(udp_payload_hex))
gsmtap_msg = gsmtap.decode()
if gsmtap_msg['type'] != 'sim':
raise ValueError('Unsupported GSMTAP type %s' % gsmtap_msg['type'])
sub_type = gsmtap_msg['sub_type']
if sub_type == 'apdu':
return ApduCommands.parse_cmd_bytes(gsmtap_msg['body'])
if sub_type == 'atr':
# card has been reset
return CardReset(gsmtap_msg['body'])
if sub_type in ['pps_req', 'pps_rsp']:
# simply ignore for now
pass
else:
raise ValueError('Unsupported GSMTAP-SIM sub-type %s' % sub_type)
class PysharkGsmtapPcap(_PysharkGsmtap):
"""APDU Source [provider] class for reading GSMTAP from a PCAP
file via pyshark, which in turn uses tshark (part of wireshark).
"""
def __init__(self, pcap_filename):
"""
Args:
pcap_filename: File name of the pcap file to be opened
"""
pyshark_inst = pyshark.FileCapture(pcap_filename, display_filter='gsm_sim || iso7816.atr', use_json=True, keep_packets=False)
super().__init__(pyshark_inst)

158
pySim/apdu_source/pyshark_rspro.py Normal file
View File

@@ -0,0 +1,158 @@
# coding=utf-8
# (C) 2022 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import logging
from typing import Tuple
import pyshark
from pySim.utils import h2b
from pySim.apdu import Tpdu
from . import ApduSource, PacketType, CardReset
logger = logging.getLogger(__name__)
class _PysharkRspro(ApduSource):
"""APDU Source [provider] base class for reading RSPRO (osmo-remsim) via tshark."""
def __init__(self, pyshark_inst):
self.pyshark = pyshark_inst
self.bank_id = None
self.bank_slot = None
self.cmd_tpdu = None
super().__init__()
@staticmethod
def get_bank_slot(bank_slot) -> Tuple[int, int]:
"""Convert a 'bankSlot_element' field into a tuple of bank_id, slot_nr"""
bank_id = bank_slot.get_field('bankId')
slot_nr = bank_slot.get_field('slotNr')
return int(bank_id), int(slot_nr)
@staticmethod
def get_client_slot(client_slot) -> Tuple[int, int]:
"""Convert a 'clientSlot_element' field into a tuple of client_id, slot_nr"""
client_id = client_slot.get_field('clientId')
slot_nr = client_slot.get_field('slotNr')
return int(client_id), int(slot_nr)
@staticmethod
def get_pstatus(pstatus) -> Tuple[int, int, int]:
"""Convert a 'slotPhysStatus_element' field into a tuple of vcc, reset, clk"""
vccPresent = int(pstatus.get_field('vccPresent'))
resetActive = int(pstatus.get_field('resetActive'))
clkActive = int(pstatus.get_field('clkActive'))
return vccPresent, resetActive, clkActive
def read_packet(self) -> PacketType:
p = self.pyshark.next()
return self._parse_packet(p)
def _set_or_verify_bank_slot(self, bsl: Tuple[int, int]):
"""Keep track of the bank:slot to make sure we don't mix traces of multiple cards"""
if not self.bank_id:
self.bank_id = bsl[0]
self.bank_slot = bsl[1]
else:
if self.bank_id != bsl[0] or self.bank_slot != bsl[1]:
raise ValueError('Received data for unexpected B(%u:%u)' % (bsl[0], bsl[1]))
def _parse_packet(self, p) -> PacketType:
rspro_layer = p['rspro']
#print("Layer: %s" % rspro_layer)
rspro_element = rspro_layer.get_field('RsproPDU_element')
#print("Element: %s" % rspro_element)
msg_type = rspro_element.get_field('msg')
rspro_msg = rspro_element.get_field('msg_tree')
if msg_type == '12': # tpduModemToCard
modem2card = rspro_msg.get_field('tpduModemToCard_element')
#print(modem2card)
client_slot = modem2card.get_field('fromClientSlot_element')
csl = self.get_client_slot(client_slot)
bank_slot = modem2card.get_field('toBankSlot_element')
bsl = self.get_bank_slot(bank_slot)
self._set_or_verify_bank_slot(bsl)
data = modem2card.get_field('data').replace(':','')
logger.debug("C(%u:%u) -> B(%u:%u): %s", csl[0], csl[1], bsl[0], bsl[1], data)
# store the CMD portion until the RSP portion arrives later
self.cmd_tpdu = h2b(data)
elif msg_type == '13': # tpduCardToModem
card2modem = rspro_msg.get_field('tpduCardToModem_element')
#print(card2modem)
client_slot = card2modem.get_field('toClientSlot_element')
csl = self.get_client_slot(client_slot)
bank_slot = card2modem.get_field('fromBankSlot_element')
bsl = self.get_bank_slot(bank_slot)
self._set_or_verify_bank_slot(bsl)
data = card2modem.get_field('data').replace(':','')
logger.debug("C(%u:%u) <- B(%u:%u): %s", csl[0], csl[1], bsl[0], bsl[1], data)
rsp_tpdu = h2b(data)
if self.cmd_tpdu:
# combine this R-TPDU with the C-TPDU we saw earlier
r = Tpdu(self.cmd_tpdu, rsp_tpdu)
self.cmd_tpdu = False
return r
elif msg_type == '14': # clientSlotStatus
cl_slotstatus = rspro_msg.get_field('clientSlotStatusInd_element')
#print(cl_slotstatus)
client_slot = cl_slotstatus.get_field('fromClientSlot_element')
bank_slot = cl_slotstatus.get_field('toBankSlot_element')
slot_pstatus = cl_slotstatus.get_field('slotPhysStatus_element')
vccPresent, resetActive, clkActive = self.get_pstatus(slot_pstatus)
if vccPresent and clkActive and not resetActive:
logger.debug("RESET")
#TODO: extract ATR from RSPRO message and use it here
return CardReset(None)
else:
print("Unhandled msg type %s: %s" % (msg_type, rspro_msg))
class PysharkRsproPcap(_PysharkRspro):
"""APDU Source [provider] class for reading RSPRO (osmo-remsim) from a PCAP
file via pyshark, which in turn uses tshark (part of wireshark).
In order to use this, you need a wireshark patched with RSPRO support,
such as can be found at https://gitea.osmocom.org/osmocom/wireshark/src/branch/laforge/rspro
A STANDARD UPSTREAM WIRESHARK *DOES NOT WORK*.
"""
def __init__(self, pcap_filename):
"""
Args:
pcap_filename: File name of the pcap file to be opened
"""
pyshark_inst = pyshark.FileCapture(pcap_filename, display_filter='rspro', use_json=True, keep_packets=False)
super().__init__(pyshark_inst)
class PysharkRsproLive(_PysharkRspro):
"""APDU Source [provider] class for reading RSPRO (osmo-remsim) from a live capture
via pyshark, which in turn uses tshark (part of wireshark).
In order to use this, you need a wireshark patched with RSPRO support,
such as can be found at https://gitea.osmocom.org/osmocom/wireshark/src/branch/laforge/rspro
A STANDARD UPSTREAM WIRESHARK *DOES NOT WORK*.
"""
def __init__(self, interface, bpf_filter='tcp port 9999 or tcp port 9998'):
"""
Args:
interface: Network interface name to capture packets on (like "eth0")
bfp_filter: libpcap capture filter to use
"""
pyshark_inst = pyshark.LiveCapture(interface=interface, display_filter='rspro', bpf_filter=bpf_filter,
use_json=True)
super().__init__(pyshark_inst)

39
pySim/apdu_source/stdin_hex.py Normal file
View File

@@ -0,0 +1,39 @@
# coding=utf-8
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pySim.utils import h2b
from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
from pySim.apdu.ts_102_222 import ApduCommands as UiccAdmApduCommands
from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
from pySim.apdu.global_platform import ApduCommands as GpApduCommands
from . import ApduSource, PacketType, CardReset
ApduCommands = UiccApduCommands + UiccAdmApduCommands + UsimApduCommands + GpApduCommands
class StdinHexApduSource(ApduSource):
"""ApduSource for reading apdu hex-strings from stdin."""
def read_packet(self) -> PacketType:
while True:
command = input("C-APDU >")
if len(command) == 0:
continue
response = '9000'
return ApduCommands.parse_cmd_bytes(h2b(command) + h2b(response))

48
pySim/apdu_source/tca_loader_log.py Normal file
View File

@@ -0,0 +1,48 @@
# coding=utf-8
# (C) 2024 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pySim.utils import h2b
from pySim.apdu.ts_102_221 import ApduCommands as UiccApduCommands
from pySim.apdu.ts_102_222 import ApduCommands as UiccAdmApduCommands
from pySim.apdu.ts_31_102 import ApduCommands as UsimApduCommands
from pySim.apdu.global_platform import ApduCommands as GpApduCommands
from . import ApduSource, PacketType, CardReset
ApduCommands = UiccApduCommands + UiccAdmApduCommands + UsimApduCommands + GpApduCommands
class TcaLoaderLogApduSource(ApduSource):
"""ApduSource for reading log files created by TCALoader."""
def __init__(self, filename:str):
super().__init__()
self.logfile = open(filename, 'r')
def read_packet(self) -> PacketType:
command = None
response = None
for line in self.logfile:
if line.startswith('Command'):
command = line.split()[1]
print("Command: '%s'" % command)
pass
elif command and line.startswith('Response'):
response = line.split()[1]
print("Response: '%s'" % response)
return ApduCommands.parse_cmd_bytes(h2b(command) + h2b(response))
raise StopIteration

131
pySim/app.py Normal file
View File

@@ -0,0 +1,131 @@
# (C) 2021-2023 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from typing import Tuple
from pySim.transport import LinkBase
from pySim.commands import SimCardCommands
from pySim.filesystem import CardModel, CardApplication
from pySim.cards import card_detect, SimCardBase, UiccCardBase, CardBase
from pySim.runtime import RuntimeState
from pySim.profile import CardProfile
from pySim.cdma_ruim import CardProfileRUIM
from pySim.ts_102_221 import CardProfileUICC
from pySim.utils import all_subclasses
from pySim.exceptions import SwMatchError
from pySim.log import PySimLogger
log = PySimLogger.get(__name__)
# we need to import this module so that the SysmocomSJA2 sub-class of
# CardModel is created, which will add the ATR-based matching and
# calling of SysmocomSJA2.add_files. See CardModel.apply_matching_models
import pySim.sysmocom_sja2
# we need to import these modules so that the various sub-classes of
# CardProfile are created, which will be used in init_card() to iterate
# over all known CardProfile sub-classes.
import pySim.ts_31_102
import pySim.ts_31_103
import pySim.ts_31_104
import pySim.ara_m
import pySim.global_platform
import pySim.euicc
def init_card(sl: LinkBase, skip_card_init: bool = False) -> Tuple[RuntimeState, SimCardBase]:
"""
Detect card in reader and setup card profile and runtime state. This
function must be called at least once on startup. The card and runtime
state object (rs) is required for all pySim-shell commands.
"""
# Create command layer
scc = SimCardCommands(transport=sl)
# Wait up to three seconds for a card in reader and try to detect
# the card type.
log.info("Waiting for card...")
sl.wait_for_card(3)
# The user may opt to skip all card initialization. In this case only the
# most basic card profile is selected. This mode is suitable for blank
# cards that need card O/S initialization using APDU scripts first.
if skip_card_init:
return None, CardBase(scc)
generic_card = False
card = card_detect(scc)
if card is None:
log.warning("Could not detect card type - assuming a generic card type...")
card = SimCardBase(scc)
generic_card = True
profile = CardProfile.pick(scc)
if profile is None:
# It is not an unrecoverable error in case profile detection fails. It
# just means that pySim was unable to recognize the card profile. This
# may happen in particular with unprovisioned cards that do not have
# any files on them yet.
log.warning("Unsupported card type!")
return None, card
# ETSI TS 102 221, Table 9.3 specifies a default for the PIN key
# references, however card manufactures may still decide to pick an
# arbitrary key reference. In case we run on a generic card class that is
# detected as an UICC, we will pick the key reference that is officially
# specified.
if generic_card and isinstance(profile, CardProfileUICC):
card._adm_chv_num = 0x0A
log.info("Card is of type: %s", str(profile))
# FIXME: this shouldn't really be here but somewhere else/more generic.
# We cannot do it within pySim/profile.py as that would create circular
# dependencies between the individual profiles and profile.py.
if isinstance(profile, CardProfileUICC):
for app_cls in all_subclasses(CardApplication):
# skip any intermediary sub-classes such as CardApplicationSD
if hasattr(app_cls, '_' + app_cls.__name__ + '__intermediate'):
continue
profile.add_application(app_cls())
# We have chosen SimCard() above, but we now know it actually is an UICC
# so it's safe to assume it supports USIM application (which we're adding above).
# IF we don't do this, we will have a SimCard but try USIM specific commands like
# the update_ust method (see https://osmocom.org/issues/6055)
if generic_card:
card = UiccCardBase(scc)
# Create runtime state with card profile
rs = RuntimeState(card, profile)
CardModel.apply_matching_models(scc, rs)
# inform the transport that we can do context-specific SW interpretation
sl.set_sw_interpreter(rs)
# try to obtain the EID, if any
isd_r = rs.mf.applications.get(pySim.euicc.AID_ISD_R.lower(), None)
if isd_r:
rs.lchan[0].select_file(isd_r)
try:
rs.identity['EID'] = pySim.euicc.CardApplicationISDR.get_eid(scc)
except SwMatchError:
# has ISD-R but not a SGP.22/SGP.32 eUICC - maybe SGP.02?
pass
finally:
rs.reset()
return rs, card

277
pySim/ara_m.py
View File

@@ -26,27 +26,29 @@ Support for the Secure Element Access Control, specifically the ARA-M inside an
#
from construct import *
from construct import GreedyString, Struct, Enum, Int8ub, Int16ub
from construct import Optional as COptional
from pySim.construct import *
from osmocom.construct import *
from osmocom.tlv import *
from osmocom.utils import Hexstr
from pySim.filesystem import *
from pySim.tlv import *
import pySim.global_platform
# various BER-TLV encoded Data Objects (DOs)
class AidRefDO(BER_TLV_IE, tag=0x4f):
# SEID v1.1 Table 6-3
# GPD_SPE_013 v1.1 Table 6-3
_construct = HexAdapter(GreedyBytes)
class AidRefEmptyDO(BER_TLV_IE, tag=0xc0):
# SEID v1.1 Table 6-3
# GPD_SPE_013 v1.1 Table 6-3
pass
class DevAppIdRefDO(BER_TLV_IE, tag=0xc1):
# SEID v1.1 Table 6-4
# GPD_SPE_013 v1.1 Table 6-4
_construct = HexAdapter(GreedyBytes)
@@ -56,187 +58,184 @@ class PkgRefDO(BER_TLV_IE, tag=0xca):
class RefDO(BER_TLV_IE, tag=0xe1, nested=[AidRefDO, AidRefEmptyDO, DevAppIdRefDO, PkgRefDO]):
# SEID v1.1 Table 6-5
# GPD_SPE_013 v1.1 Table 6-5
pass
class ApduArDO(BER_TLV_IE, tag=0xd0):
# SEID v1.1 Table 6-8
# GPD_SPE_013 v1.1 Table 6-8
def _from_bytes(self, do: bytes):
if len(do) == 1:
if do[0] == 0x00:
self.decoded = {'generic_access_rule': 'never'}
return self.decoded
elif do[0] == 0x01:
if do[0] == 0x01:
self.decoded = {'generic_access_rule': 'always'}
return self.decoded
else:
return ValueError('Invalid 1-byte generic APDU access rule')
raise ValueError('Invalid 1-byte generic APDU access rule')
else:
if len(do) % 8:
return ValueError('Invalid non-modulo-8 length of APDU filter: %d' % len(do))
self.decoded['apdu_filter'] = []
raise ValueError('Invalid non-modulo-8 length of APDU filter: %d' % len(do))
self.decoded = {'apdu_filter': []}
offset = 0
while offset < len(do):
self.decoded['apdu_filter'] += {'header': b2h(do[offset:offset+4]),
'mask': b2h(do[offset+4:offset+8])}
self.decoded = res
return res
self.decoded['apdu_filter'] += [{'header': b2h(do[offset:offset+4]),
'mask': b2h(do[offset+4:offset+8])}]
offset += 8 # Move offset to the beginning of the next apdu_filter object
return self.decoded
def _to_bytes(self):
if 'generic_access_rule' in self.decoded:
if self.decoded['generic_access_rule'] == 'never':
return b'\x00'
elif self.decoded['generic_access_rule'] == 'always':
if self.decoded['generic_access_rule'] == 'always':
return b'\x01'
else:
return ValueError('Invalid 1-byte generic APDU access rule')
raise ValueError('Invalid 1-byte generic APDU access rule')
else:
if not 'apdu_filter' in self.decoded:
return ValueError('Invalid APDU AR DO')
raise ValueError('Invalid APDU AR DO')
filters = self.decoded['apdu_filter']
res = b''
for f in filters:
if not 'header' in f or not 'mask' in f:
return ValueError('APDU filter must contain header and mask')
raise ValueError('APDU filter must contain header and mask')
header_b = h2b(f['header'])
mask_b = h2b(f['mask'])
if len(header_b) != 4 or len(mask_b) != 4:
return ValueError('APDU filter header and mask must each be 4 bytes')
raise ValueError('APDU filter header and mask must each be 4 bytes')
res += header_b + mask_b
return res
class NfcArDO(BER_TLV_IE, tag=0xd1):
# SEID v1.1 Table 6-9
# GPD_SPE_013 v1.1 Table 6-9
_construct = Struct('nfc_event_access_rule' /
Enum(Int8ub, never=0, always=1))
class PermArDO(BER_TLV_IE, tag=0xdb):
# Android UICC Carrier Privileges specific extension, see https://source.android.com/devices/tech/config/uicc
# based on Table 6-8 of GlobalPlatform Device API Access Control v1.0
_construct = Struct('permissions'/HexAdapter(Bytes(8)))
class ArDO(BER_TLV_IE, tag=0xe3, nested=[ApduArDO, NfcArDO, PermArDO]):
# SEID v1.1 Table 6-7
# GPD_SPE_013 v1.1 Table 6-7
pass
class RefArDO(BER_TLV_IE, tag=0xe2, nested=[RefDO, ArDO]):
# SEID v1.1 Table 6-6
# GPD_SPE_013 v1.1 Table 6-6
pass
class ResponseAllRefArDO(BER_TLV_IE, tag=0xff40, nested=[RefArDO]):
# SEID v1.1 Table 4-2
# GPD_SPE_013 v1.1 Table 4-2
pass
class ResponseArDO(BER_TLV_IE, tag=0xff50, nested=[ArDO]):
# SEID v1.1 Table 4-3
# GPD_SPE_013 v1.1 Table 4-3
pass
class ResponseRefreshTagDO(BER_TLV_IE, tag=0xdf20):
# SEID v1.1 Table 4-4
# GPD_SPE_013 v1.1 Table 4-4
_construct = Struct('refresh_tag'/HexAdapter(Bytes(8)))
class DeviceInterfaceVersionDO(BER_TLV_IE, tag=0xe6):
# SEID v1.1 Table 6-12
# GPD_SPE_013 v1.1 Table 6-12
_construct = Struct('major'/Int8ub, 'minor'/Int8ub, 'patch'/Int8ub)
class DeviceConfigDO(BER_TLV_IE, tag=0xe4, nested=[DeviceInterfaceVersionDO]):
# SEID v1.1 Table 6-10
# GPD_SPE_013 v1.1 Table 6-10
pass
class ResponseDeviceConfigDO(BER_TLV_IE, tag=0xff7f, nested=[DeviceConfigDO]):
# SEID v1.1 Table 5-14
# GPD_SPE_013 v1.1 Table 5-14
pass
class AramConfigDO(BER_TLV_IE, tag=0xe5, nested=[DeviceInterfaceVersionDO]):
# SEID v1.1 Table 6-11
# GPD_SPE_013 v1.1 Table 6-11
pass
class ResponseAramConfigDO(BER_TLV_IE, tag=0xdf21, nested=[AramConfigDO]):
# SEID v1.1 Table 4-5
# GPD_SPE_013 v1.1 Table 4-5
pass
class CommandStoreRefArDO(BER_TLV_IE, tag=0xf0, nested=[RefArDO]):
# SEID v1.1 Table 5-2
# GPD_SPE_013 v1.1 Table 5-2
pass
class CommandDelete(BER_TLV_IE, tag=0xf1, nested=[AidRefDO, AidRefEmptyDO, RefDO, RefArDO]):
# SEID v1.1 Table 5-4
# GPD_SPE_013 v1.1 Table 5-4
pass
class CommandUpdateRefreshTagDO(BER_TLV_IE, tag=0xf2):
# SEID V1.1 Table 5-6
# GPD_SPE_013 V1.1 Table 5-6
pass
class CommandRegisterClientAidsDO(BER_TLV_IE, tag=0xf7, nested=[AidRefDO, AidRefEmptyDO]):
# SEID v1.1 Table 5-7
# GPD_SPE_013 v1.1 Table 5-7
pass
class CommandGet(BER_TLV_IE, tag=0xf3, nested=[AidRefDO, AidRefEmptyDO]):
# SEID v1.1 Table 5-8
# GPD_SPE_013 v1.1 Table 5-8
pass
class CommandGetAll(BER_TLV_IE, tag=0xf4):
# SEID v1.1 Table 5-9
# GPD_SPE_013 v1.1 Table 5-9
pass
class CommandGetClientAidsDO(BER_TLV_IE, tag=0xf6):
# SEID v1.1 Table 5-10
# GPD_SPE_013 v1.1 Table 5-10
pass
class CommandGetNext(BER_TLV_IE, tag=0xf5):
# SEID v1.1 Table 5-11
# GPD_SPE_013 v1.1 Table 5-11
pass
class CommandGetDeviceConfigDO(BER_TLV_IE, tag=0xf8):
# SEID v1.1 Table 5-12
# GPD_SPE_013 v1.1 Table 5-12
pass
class ResponseAracAidDO(BER_TLV_IE, tag=0xff70, nested=[AidRefDO, AidRefEmptyDO]):
# SEID v1.1 Table 5-13
# GPD_SPE_013 v1.1 Table 5-13
pass
class BlockDO(BER_TLV_IE, tag=0xe7):
# SEID v1.1 Table 6-13
# GPD_SPE_013 v1.1 Table 6-13
_construct = Struct('offset'/Int16ub, 'length'/Int8ub)
# SEID v1.1 Table 4-1
# GPD_SPE_013 v1.1 Table 4-1
class GetCommandDoCollection(TLV_IE_Collection, nested=[RefDO, DeviceConfigDO]):
pass
# SEID v1.1 Table 4-2
# GPD_SPE_013 v1.1 Table 4-2
class GetResponseDoCollection(TLV_IE_Collection, nested=[ResponseAllRefArDO, ResponseArDO,
ResponseRefreshTagDO, ResponseAramConfigDO]):
pass
# SEID v1.1 Table 5-1
# GPD_SPE_013 v1.1 Table 5-1
class StoreCommandDoCollection(TLV_IE_Collection,
nested=[BlockDO, CommandStoreRefArDO, CommandDelete,
CommandUpdateRefreshTagDO, CommandRegisterClientAidsDO,
@@ -245,7 +244,7 @@ class StoreCommandDoCollection(TLV_IE_Collection,
pass
# SEID v1.1 Section 5.1.2
# GPD_SPE_013 v1.1 Section 5.1.2
class StoreResponseDoCollection(TLV_IE_Collection,
nested=[ResponseAllRefArDO, ResponseAracAidDO, ResponseDeviceConfigDO]):
pass
@@ -259,72 +258,71 @@ class ADF_ARAM(CardADF):
files = []
self.add_files(files)
def decode_select_response(self, data_hex):
return pySim.global_platform.decode_select_response(data_hex)
@staticmethod
def xceive_apdu_tlv(tp, hdr: Hexstr, cmd_do, resp_cls, exp_sw='9000'):
def xceive_apdu_tlv(scc, hdr: Hexstr, cmd_do, resp_cls, exp_sw='9000'):
"""Transceive an APDU with the card, transparently encoding the command data from TLV
and decoding the response data tlv."""
if cmd_do:
cmd_do_enc = cmd_do.to_ie()
cmd_do_len = len(cmd_do_enc)
if cmd_do_len > 255:
return ValueError('DO > 255 bytes not supported yet')
raise ValueError('DO > 255 bytes not supported yet')
else:
cmd_do_enc = b''
cmd_do_len = 0
c_apdu = hdr + ('%02x' % cmd_do_len) + b2h(cmd_do_enc)
(data, sw) = tp.send_apdu_checksw(c_apdu, exp_sw)
(data, _sw) = scc.send_apdu_checksw(c_apdu, exp_sw)
if data:
if resp_cls:
resp_do = resp_cls()
resp_do.from_tlv(h2b(data))
return resp_do
else:
return data
else:
return None
@staticmethod
def store_data(tp, do) -> bytes:
def store_data(scc, do) -> bytes:
"""Build the Command APDU for STORE DATA."""
return ADF_ARAM.xceive_apdu_tlv(tp, '80e29000', do, StoreResponseDoCollection)
return ADF_ARAM.xceive_apdu_tlv(scc, '80e29000', do, StoreResponseDoCollection)
@staticmethod
def get_all(tp):
return ADF_ARAM.xceive_apdu_tlv(tp, '80caff40', None, GetResponseDoCollection)
def get_all(scc):
return ADF_ARAM.xceive_apdu_tlv(scc, '80caff40', None, GetResponseDoCollection)
@staticmethod
def get_config(tp, v_major=0, v_minor=0, v_patch=1):
def get_config(scc, v_major=0, v_minor=0, v_patch=1):
cmd_do = DeviceConfigDO()
cmd_do.from_dict([{'DeviceInterfaceVersionDO': {
cmd_do.from_val_dict([{'device_interface_version_do': {
'major': v_major, 'minor': v_minor, 'patch': v_patch}}])
return ADF_ARAM.xceive_apdu_tlv(tp, '80cadf21', cmd_do, ResponseAramConfigDO)
return ADF_ARAM.xceive_apdu_tlv(scc, '80cadf21', cmd_do, ResponseAramConfigDO)
@with_default_category('Application-Specific Commands')
class AddlShellCommands(CommandSet):
def __init(self):
super().__init__()
def do_aram_get_all(self, opts):
def do_aram_get_all(self, _opts):
"""GET DATA [All] on the ARA-M Applet"""
res_do = ADF_ARAM.get_all(self._cmd.card._scc._tp)
res_do = ADF_ARAM.get_all(self._cmd.lchan.scc)
if res_do:
self._cmd.poutput_json(res_do.to_dict())
def do_aram_get_config(self, opts):
"""GET DATA [Config] on the ARA-M Applet"""
res_do = ADF_ARAM.get_config(self._cmd.card._scc._tp)
def do_aram_get_config(self, _opts):
"""Perform GET DATA [Config] on the ARA-M Applet: Tell it our version and retrieve its version."""
res_do = ADF_ARAM.get_config(self._cmd.lchan.scc)
if res_do:
self._cmd.poutput_json(res_do.to_dict())
store_ref_ar_do_parse = argparse.ArgumentParser()
# REF-DO
store_ref_ar_do_parse.add_argument(
'--device-app-id', required=True, help='Identifies the specific device application that the rule appplies to. Hash of Certificate of Application Provider, or UUID. (20/32 hex bytes)')
'--device-app-id', required=True, help='Identifies the specific device application that the rule applies to. Hash of Certificate of Application Provider, or UUID. (20/32 hex bytes)')
aid_grp = store_ref_ar_do_parse.add_mutually_exclusive_group()
aid_grp.add_argument(
'--aid', help='Identifies the specific SE application for which rules are to be stored. Can be a partial AID, containing for example only the RID. (5-16 hex bytes)')
'--aid', help='Identifies the specific SE application for which rules are to be stored. Can be a partial AID, containing for example only the RID. (5-16 or 0 hex bytes)')
aid_grp.add_argument('--aid-empty', action='store_true',
help='No specific SE application, applies to all applications')
help='No specific SE application, applies to implicitly selected application (all channels)')
store_ref_ar_do_parse.add_argument(
'--pkg-ref', help='Full Android Java package name (up to 127 chars ASCII)')
# AR-DO
@@ -334,7 +332,7 @@ class ADF_ARAM(CardADF):
apdu_grp.add_argument(
'--apdu-always', action='store_true', help='APDU access is allowed')
apdu_grp.add_argument(
'--apdu-filter', help='APDU filter: 4 byte CLA/INS/P1/P2 followed by 4 byte mask (8 hex bytes)')
'--apdu-filter', help='APDU filter: multiple groups of 8 hex bytes (4 byte CLA/INS/P1/P2 followed by 4 byte mask)')
nfc_grp = store_ref_ar_do_parse.add_mutually_exclusive_group()
nfc_grp.add_argument('--nfc-always', action='store_true',
help='NFC event access is allowed')
@@ -345,51 +343,63 @@ class ADF_ARAM(CardADF):
@cmd2.with_argparser(store_ref_ar_do_parse)
def do_aram_store_ref_ar_do(self, opts):
"""Perform STORE DATA [Command-Store-REF-AR-DO] to store a new access rule."""
"""Perform STORE DATA [Command-Store-REF-AR-DO] to store a (new) access rule."""
# REF
ref_do_content = []
if opts.aid:
ref_do_content += [{'AidRefDO': opts.aid}]
if opts.aid is not None:
ref_do_content += [{'aid_ref_do': opts.aid}]
elif opts.aid_empty:
ref_do_content += [{'AidRefEmptyDO': None}]
ref_do_content += [{'DevAppIdRefDO': opts.device_app_id}]
ref_do_content += [{'aid_ref_empty_do': None}]
ref_do_content += [{'dev_app_id_ref_do': opts.device_app_id}]
if opts.pkg_ref:
ref_do_content += [{'PkgRefDO': opts.pkg_ref}]
ref_do_content += [{'pkg_ref_do': {'package_name_string': opts.pkg_ref}}]
# AR
ar_do_content = []
if opts.apdu_never:
ar_do_content += [{'ApduArDO': {'generic_access_rule': 'never'}}]
ar_do_content += [{'apdu_ar_do': {'generic_access_rule': 'never'}}]
elif opts.apdu_always:
ar_do_content += [{'ApduArDO': {'generic_access_rule': 'always'}}]
ar_do_content += [{'apdu_ar_do': {'generic_access_rule': 'always'}}]
elif opts.apdu_filter:
# TODO: multiple filters
ar_do_content += [{'ApduArDO': {'apdu_filter': [opts.apdu_filter]}}]
if len(opts.apdu_filter) % 16:
raise ValueError(f'Invalid non-modulo-16 length of APDU filter: {len(opts.apdu_filter)}')
offset = 0
apdu_filter = []
while offset < len(opts.apdu_filter):
apdu_filter += [{'header': opts.apdu_filter[offset:offset+8],
'mask': opts.apdu_filter[offset+8:offset+16]}]
offset += 16 # Move offset to the beginning of the next apdu_filter object
ar_do_content += [{'apdu_ar_do': {'apdu_filter': apdu_filter}}]
if opts.nfc_always:
ar_do_content += [{'NfcArDO': {'nfc_event_access_rule': 'always'}}]
ar_do_content += [{'nfc_ar_do': {'nfc_event_access_rule': 'always'}}]
elif opts.nfc_never:
ar_do_content += [{'NfcArDO': {'nfc_event_access_rule': 'never'}}]
ar_do_content += [{'nfc_ar_do': {'nfc_event_access_rule': 'never'}}]
if opts.android_permissions:
ar_do_content += [{'PermArDO': {'permissions': opts.android_permissions}}]
d = [{'RefArDO': [{'RefDO': ref_do_content}, {'ArDO': ar_do_content}]}]
ar_do_content += [{'perm_ar_do': {'permissions': opts.android_permissions}}]
d = [{'ref_ar_do': [{'ref_do': ref_do_content}, {'ar_do': ar_do_content}]}]
csrado = CommandStoreRefArDO()
csrado.from_dict(d)
res_do = ADF_ARAM.store_data(self._cmd.card._scc._tp, csrado)
csrado.from_val_dict(d)
res_do = ADF_ARAM.store_data(self._cmd.lchan.scc, csrado)
if res_do:
self._cmd.poutput_json(res_do.to_dict())
def do_aram_delete_all(self, opts):
def do_aram_delete_all(self, _opts):
"""Perform STORE DATA [Command-Delete[all]] to delete all access rules."""
deldo = CommandDelete()
res_do = ADF_ARAM.store_data(self._cmd.card._scc._tp, deldo)
res_do = ADF_ARAM.store_data(self._cmd.lchan.scc, deldo)
if res_do:
self._cmd.poutput_json(res_do.to_dict())
def do_aram_lock(self, opts):
"""Lock STORE DATA command to prevent unauthorized changes
(Proprietary feature that is specific to sysmocom's fork of Bertrand Martel’s ARA-M implementation.)"""
self._cmd.lchan.scc.send_apdu_checksw('80e2900001A1', '9000')
# SEAC v1.1 Section 4.1.2.2 + 5.1.2.2
sw_aram = {
'ARA-M': {
'6381': 'Rule successfully stored but an access rule already exists',
'6382': 'Rule successfully stored bu contained at least one unknown (discarded) BER-TLV',
'6382': 'Rule successfully stored but contained at least one unknown (discarded) BER-TLV',
'6581': 'Memory Problem',
'6700': 'Wrong Length in Lc',
'6981': 'DO is not supported by the ARA-M/ARA-C',
@@ -410,3 +420,84 @@ sw_aram = {
class CardApplicationARAM(CardApplication):
def __init__(self):
super().__init__('ARA-M', adf=ADF_ARAM(), sw=sw_aram)
@staticmethod
def __export_get_from_dictlist(key, dictlist):
# Data objects are organized in lists that contain dictionaries, usually there is only one dictionary per
# list item. This function goes through that list and gets the value of the first dictionary that has the
# matching key.
if dictlist is None:
return None
for d in dictlist:
if key in d:
obj = d.get(key)
if obj is None:
return ""
return obj
return None
@staticmethod
def __export_ref_ar_do_list(ref_ar_do_list):
export_str = ""
ref_do_list = CardApplicationARAM.__export_get_from_dictlist('ref_do', ref_ar_do_list.get('ref_ar_do'))
ar_do_list = CardApplicationARAM.__export_get_from_dictlist('ar_do', ref_ar_do_list.get('ref_ar_do'))
if ref_do_list and ar_do_list:
# Get ref_do parameters
aid_ref_do = CardApplicationARAM.__export_get_from_dictlist('aid_ref_do', ref_do_list)
aid_ref_empty_do = CardApplicationARAM.__export_get_from_dictlist('aid_ref_empty_do', ref_do_list)
dev_app_id_ref_do = CardApplicationARAM.__export_get_from_dictlist('dev_app_id_ref_do', ref_do_list)
pkg_ref_do = CardApplicationARAM.__export_get_from_dictlist('pkg_ref_do', ref_do_list)
# Get ar_do parameters
apdu_ar_do = CardApplicationARAM.__export_get_from_dictlist('apdu_ar_do', ar_do_list)
nfc_ar_do = CardApplicationARAM.__export_get_from_dictlist('nfc_ar_do', ar_do_list)
perm_ar_do = CardApplicationARAM.__export_get_from_dictlist('perm_ar_do', ar_do_list)
# Write command-line
export_str += "aram_store_ref_ar_do"
if aid_ref_do is not None and len(aid_ref_do) > 0:
export_str += (" --aid %s" % aid_ref_do)
elif aid_ref_do is not None:
export_str += " --aid \"\""
if aid_ref_empty_do is not None:
export_str += " --aid-empty"
if dev_app_id_ref_do:
export_str += (" --device-app-id %s" % dev_app_id_ref_do)
if apdu_ar_do and 'generic_access_rule' in apdu_ar_do:
export_str += (" --apdu-%s" % apdu_ar_do['generic_access_rule'])
elif apdu_ar_do and 'apdu_filter' in apdu_ar_do:
export_str += (" --apdu-filter ")
for apdu_filter in apdu_ar_do['apdu_filter']:
export_str += apdu_filter['header']
export_str += apdu_filter['mask']
if nfc_ar_do and 'nfc_event_access_rule' in nfc_ar_do:
export_str += (" --nfc-%s" % nfc_ar_do['nfc_event_access_rule'])
if perm_ar_do:
export_str += (" --android-permissions %s" % perm_ar_do['permissions'])
if pkg_ref_do:
export_str += (" --pkg-ref %s" % pkg_ref_do['package_name_string'])
export_str += "\n"
return export_str
@staticmethod
def export(as_json: bool, lchan):
# TODO: Add JSON output as soon as aram_store_ref_ar_do is able to process input in JSON format.
if as_json:
raise NotImplementedError("res_do encoder not yet implemented. Patches welcome.")
export_str = ""
export_str += "aram_delete_all\n"
res_do = ADF_ARAM.get_all(lchan.scc)
if not res_do:
return export_str.strip()
for res_do_dict in res_do.to_dict():
if not res_do_dict.get('response_all_ref_ar_do', False):
continue
for ref_ar_do_list in res_do_dict['response_all_ref_ar_do']:
export_str += CardApplicationARAM.__export_ref_ar_do_list(ref_ar_do_list)
return export_str.strip()

7
pySim/card_handler.py
View File

@@ -7,7 +7,7 @@ there are also automatic card feeders.
"""
#
# (C) 2019 by Sysmocom s.f.m.c. GmbH
# (C) 2019 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# This program is free software: you can redistribute it and/or modify
@@ -24,12 +24,11 @@ there are also automatic card feeders.
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from pySim.transport import LinkBase
import subprocess
import sys
import yaml
from pySim.transport import LinkBase
class CardHandlerBase:
"""Abstract base class representing a mechanism for card insertion/removal."""
@@ -97,7 +96,7 @@ class CardHandlerAuto(CardHandlerBase):
print("Card handler Config-file: " + str(config_file))
with open(config_file) as cfg:
self.cmds = yaml.load(cfg, Loader=yaml.FullLoader)
self.verbose = (self.cmds.get('verbose') == True)
self.verbose = self.cmds.get('verbose') is True
def __print_outout(self, out):
print("")

328
pySim/card_key_provider.py
View File

@@ -10,10 +10,10 @@ the need of manually entering the related card-individual data on every
operation with pySim-shell.
"""
# (C) 2021 by Sysmocom s.f.m.c. GmbH
# (C) 2021-2025 by sysmocom - s.f.m.c. GmbH
# All Rights Reserved
#
# Author: Philipp Maier
# Author: Philipp Maier, Harald Welte
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -29,96 +29,276 @@ operation with pySim-shell.
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from typing import List, Dict, Optional
from Cryptodome.Cipher import AES
from osmocom.utils import h2b, b2h
from pySim.log import PySimLogger
import os
import abc
import csv
import logging
import yaml
import argparse
log = PySimLogger.get(__name__)
card_key_providers = [] # type: List['CardKeyProvider']
class CardKeyFieldCryptor:
"""
A Card key field encryption class that may be used by Card key provider implementations to add support for
a column-based encryption to protect sensitive material (cryptographic key material, ADM keys, etc.).
The sensitive material is encrypted using a "key-encryption key", occasionally also known as "transport key"
before it is stored into a file or database (see also GSMA FS.28). The "transport key" is then used to decrypt
the key material on demand.
"""
# well-known groups of columns relate to a given functionality. This avoids having
# to specify the same transport key N number of times, if the same key is used for multiple
# fields of one group, like KIC+KID+KID of one SD.
__CRYPT_GROUPS = {
'UICC_SCP02': ['UICC_SCP02_KIC1', 'UICC_SCP02_KID1', 'UICC_SCP02_KIK1'],
'UICC_SCP03': ['UICC_SCP03_KIC1', 'UICC_SCP03_KID1', 'UICC_SCP03_KIK1'],
'SCP03_ISDR': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDR', 'SCP03_DEK_ISDR'],
'SCP03_ISDA': ['SCP03_ENC_ISDA', 'SCP03_MAC_ISDA', 'SCP03_DEK_ISDA'],
'SCP03_ECASD': ['SCP03_ENC_ECASD', 'SCP03_MAC_ECASD', 'SCP03_DEK_ECASD'],
}
__IV = b'\x23' * 16
@staticmethod
def __dict_keys_to_upper(d: dict) -> dict:
return {k.upper():v for k,v in d.items()}
@staticmethod
def __process_transport_keys(transport_keys: dict, crypt_groups: dict):
"""Apply a single transport key to multiple fields/columns, if the name is a group."""
new_dict = {}
for name, key in transport_keys.items():
if name in crypt_groups:
for field in crypt_groups[name]:
new_dict[field] = key
else:
new_dict[name] = key
return new_dict
def __init__(self, transport_keys: dict):
"""
Create new field encryptor/decryptor object and set transport keys, usually one for each column. In some cases
it is also possible to use a single key for multiple columns (see also __CRYPT_GROUPS)
Args:
transport_keys : a dict indexed by field name, whose values are hex-encoded AES keys for the
respective field (column) of the CSV. This is done so that different fields
(columns) can use different transport keys, which is strongly recommended by
GSMA FS.28
"""
self.transport_keys = self.__process_transport_keys(self.__dict_keys_to_upper(transport_keys),
self.__CRYPT_GROUPS)
for name, key in self.transport_keys.items():
log.debug("Encrypting/decrypting field %s using AES key %s" % (name, key))
def decrypt_field(self, field_name: str, encrypted_val: str) -> str:
"""
Decrypt a single field. The decryption is only applied if we have a transport key is known under the provided
field name, otherwise the field is treated as plaintext and passed through as it is.
Args:
field_name : name of the field to decrypt (used to identify which key to use)
encrypted_val : encrypted field value
Returns:
plaintext field value
"""
if not field_name.upper() in self.transport_keys:
return encrypted_val
cipher = AES.new(h2b(self.transport_keys[field_name.upper()]), AES.MODE_CBC, self.__IV)
return b2h(cipher.decrypt(h2b(encrypted_val)))
def encrypt_field(self, field_name: str, plaintext_val: str) -> str:
"""
Encrypt a single field. The encryption is only applied if we have a transport key is known under the provided
field name, otherwise the field is treated as non sensitive and passed through as it is.
Args:
field_name : name of the field to decrypt (used to identify which key to use)
encrypted_val : encrypted field value
Returns:
plaintext field value
"""
if not field_name.upper() in self.transport_keys:
return plaintext_val
cipher = AES.new(h2b(self.transport_keys[field_name.upper()]), AES.MODE_CBC, self.__IV)
return b2h(cipher.encrypt(h2b(plaintext_val)))
@staticmethod
def argparse_add_args(arg_parser: argparse.ArgumentParser):
arg_parser.add_argument('--column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append',
help='per-column AES transport key', dest='column_key')
# Depprecated argument, replaced by --column-key (see above)
arg_parser.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append',
help=argparse.SUPPRESS, dest='column_key')
@staticmethod
def transport_keys_from_opts(opts: argparse.Namespace) -> dict:
"""
Transport keys are passed via the commandline using the '--column-key' option. Each column requires a
dedicated transport key. This method can be used to extract the column keys parameters from the commandline
options into a dict that can be directly passed to the construtor with the transport_keys argument.
Args:
opts: parsed commandline options (Namespace)
"""
transport_keys = {}
for par in opts.column_key:
name, key = par.split(':')
transport_keys[name] = key
return transport_keys
class CardKeyProvider(abc.ABC):
"""Base class, not containing any concrete implementation."""
VALID_FIELD_NAMES = ['ICCID', 'ADM1',
'IMSI', 'PIN1', 'PIN2', 'PUK1', 'PUK2']
# check input parameters, but do nothing concrete yet
def _verify_get_data(self, fields: List[str] = [], key: str = 'ICCID', value: str = "") -> Dict[str, str]:
"""Verify multiple fields for identified card.
Args:
fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained
key : look-up key to identify card data, such as 'ICCID'
value : value for look-up key to identify card data
Returns:
dictionary of {field, value} strings for each requested field from 'fields'
"""
for f in fields:
if (f not in self.VALID_FIELD_NAMES):
raise ValueError("Requested field name '%s' is not a valid field name, valid field names are: %s" %
(f, str(self.VALID_FIELD_NAMES)))
if (key not in self.VALID_FIELD_NAMES):
raise ValueError("Key field name '%s' is not a valid field name, valid field names are: %s" %
(key, str(self.VALID_FIELD_NAMES)))
return {}
def get_field(self, field: str, key: str = 'ICCID', value: str = "") -> Optional[str]:
"""get a single field from CSV file using a specified key/value pair"""
fields = [field]
result = self.get(fields, key, value)
return result.get(field)
@abc.abstractmethod
def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
"""Get multiple card-individual fields for identified card.
"""
Get multiple card-individual fields for identified card. This method should not fail with an exception in
case the entry, columns or even the key column itsself is not found.
Args:
fields : list of valid field names such as 'ADM1', 'PIN1', ... which are to be obtained
key : look-up key to identify card data, such as 'ICCID'
value : value for look-up key to identify card data
Returns:
dictionary of {field, value} strings for each requested field from 'fields'
dictionary of {field : value, ...} strings for each requested field from 'fields'. In case nothing is
fond None shall be returned.
"""
@staticmethod
def argparse_add_args(arg_parser: argparse.ArgumentParser):
"""
Add the commandline arguments relevant for this card key provider.
Args:
arg_parser : argument parser group
"""
def __str__(self):
return type(self).__name__
class CardKeyProviderCsv(CardKeyProvider):
"""Card key provider implementation that allows to query against a specified CSV file"""
csv_file = None
filename = None
"""Card key provider implementation that allows to query against a specified CSV file."""
def __init__(self, filename: str):
def __init__(self, csv_filename: str, field_cryptor: CardKeyFieldCryptor):
"""
Args:
filename : file name (path) of CSV file containing card-individual key/data
csv_filename : file name (path) of CSV file containing card-individual key/data
field_cryptor : (see class CardKeyFieldCryptor)
"""
self.csv_file = open(filename, 'r')
log.info("Using CSV file as card key data source: %s" % csv_filename)
self.csv_file = open(csv_filename, 'r')
if not self.csv_file:
raise RuntimeError("Could not open CSV file '%s'" % filename)
self.filename = filename
raise RuntimeError("Could not open CSV file '%s'" % csv_filename)
self.csv_filename = csv_filename
self.crypt = field_cryptor
def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
super()._verify_get_data(fields, key, value)
self.csv_file.seek(0)
cr = csv.DictReader(self.csv_file)
if not cr:
raise RuntimeError(
"Could not open DictReader for CSV-File '%s'" % self.filename)
raise RuntimeError("Could not open DictReader for CSV-File '%s'" % self.csv_filename)
cr.fieldnames = [field.upper() for field in cr.fieldnames]
rc = {}
if key not in cr.fieldnames:
return None
return_dict = {}
for row in cr:
if row[key] == value:
for f in fields:
if f in row:
rc.update({f: row[f]})
return_dict.update({f: self.crypt.decrypt_field(f, row[f])})
else:
raise RuntimeError("CSV-File '%s' lacks column '%s'" %
(self.filename, f))
return rc
raise RuntimeError("CSV-File '%s' lacks column '%s'" % (self.csv_filename, f))
if return_dict == {}:
return None
return return_dict
@staticmethod
def argparse_add_args(arg_parser: argparse.ArgumentParser):
arg_parser.add_argument('--csv', metavar='FILE',
default="~/.osmocom/pysim/card_data.csv",
help='Read card data from CSV file')
class CardKeyProviderPgsql(CardKeyProvider):
"""Card key provider implementation that allows to query against a specified PostgreSQL database table."""
def __init__(self, config_filename: str, field_cryptor: CardKeyFieldCryptor):
"""
Args:
config_filename : file name (path) of CSV file containing card-individual key/data
field_cryptor : (see class CardKeyFieldCryptor)
"""
import psycopg2
log.info("Using SQL database as card key data source: %s" % config_filename)
with open(config_filename, "r") as cfg:
config = yaml.load(cfg, Loader=yaml.FullLoader)
log.info("Card key database name: %s" % config.get('db_name'))
db_users = config.get('db_users')
user = db_users.get('reader')
if user is None:
raise ValueError("user for role 'reader' not set up in config file.")
self.conn = psycopg2.connect(dbname=config.get('db_name'),
user=user.get('name'),
password=user.get('pass'),
host=config.get('host'))
self.tables = config.get('table_names')
log.info("Card key database tables: %s" % str(self.tables))
self.crypt = field_cryptor
def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]:
import psycopg2
from psycopg2.sql import Identifier, SQL
db_result = None
for t in self.tables:
self.conn.rollback()
cur = self.conn.cursor()
# Make sure that the database table and the key column actually exists. If not, move on to the next table
cur.execute("SELECT column_name FROM information_schema.columns where table_name = %s;", (t,))
cols_result = cur.fetchall()
if cols_result == []:
log.warning("Card Key database seems to lack table %s, check config file!" % t)
continue
if (key.lower(),) not in cols_result:
continue
# Query requested columns from database table
query = SQL("SELECT {}").format(Identifier(fields[0].lower()))
for f in fields[1:]:
query += SQL(", {}").format(Identifier(f.lower()))
query += SQL(" FROM {} WHERE {} = %s LIMIT 1;").format(Identifier(t.lower()),
Identifier(key.lower()))
cur.execute(query, (value,))
db_result = cur.fetchone()
cur.close()
if db_result:
break
if db_result is None:
return None
result = dict(zip(fields, db_result))
for k in result.keys():
result[k] = self.crypt.decrypt_field(k, result.get(k))
return result
@staticmethod
def argparse_add_args(arg_parser: argparse.ArgumentParser):
arg_parser.add_argument('--pgsql', metavar='FILE',
default="~/.osmocom/pysim/card_data_pgsql.cfg",
help='Read card data from PostgreSQL database (config file)')
def card_key_provider_register(provider: CardKeyProvider, provider_list=card_key_providers):
"""Register a new card key provider.
@@ -128,11 +308,10 @@ def card_key_provider_register(provider: CardKeyProvider, provider_list=card_key
provider_list : override the list of providers from the global default
"""
if not isinstance(provider, CardKeyProvider):
raise ValueError("provider is not a card data provier")
raise ValueError("provider is not a card data provider")
provider_list.append(provider)
def card_key_provider_get(fields, key: str, value: str, provider_list=card_key_providers) -> Dict[str, str]:
def card_key_provider_get(fields: list[str], key: str, value: str, provider_list=card_key_providers) -> Dict[str, str]:
"""Query all registered card data providers for card-individual [key] data.
Args:
@@ -143,17 +322,20 @@ def card_key_provider_get(fields, key: str, value: str, provider_list=card_key_p
Returns:
dictionary of {field, value} strings for each requested field from 'fields'
"""
key = key.upper()
fields = [f.upper() for f in fields]
for p in provider_list:
if not isinstance(p, CardKeyProvider):
raise ValueError(
"provider list contains element which is not a card data provier")
raise ValueError("Provider list contains element which is not a card data provider")
log.debug("Searching for card key data (key=%s, value=%s, provider=%s)" % (key, value, str(p)))
result = p.get(fields, key, value)
if result:
log.debug("Found card data: %s" % (str(result)))
return result
return {}
raise ValueError("Unable to find card key data (key=%s, value=%s, fields=%s)" % (key, value, str(fields)))
def card_key_provider_get_field(field: str, key: str, value: str, provider_list=card_key_providers) -> Optional[str]:
def card_key_provider_get_field(field: str, key: str, value: str, provider_list=card_key_providers) -> str:
"""Query all registered card data providers for a single field.
Args:
@@ -164,11 +346,23 @@ def card_key_provider_get_field(field: str, key: str, value: str, provider_list=
Returns:
dictionary of {field, value} strings for the requested field
"""
for p in provider_list:
if not isinstance(p, CardKeyProvider):
raise ValueError(
"provider list contains element which is not a card data provier")
result = p.get_field(field, key, value)
if result:
return result
return None
fields = [field]
result = card_key_provider_get(fields, key, value, card_key_providers)
return result.get(field.upper())
def argparse_add_card_key_provider_args(arg_parser: argparse.ArgumentParser):
"""Add card key provider commandline options to the given argument parser"""
card_key_group = arg_parser.add_argument_group('Card Key Provider Options')
CardKeyProviderCsv.argparse_add_args(card_key_group)
CardKeyProviderPgsql.argparse_add_args(card_key_group)
CardKeyFieldCryptor.argparse_add_args(card_key_group)
def init_card_key_provider(opts: argparse.Namespace):
"""Initialize card key provider depending on the user provided commandline options"""
transport_keys = CardKeyFieldCryptor.transport_keys_from_opts(opts)
card_key_field_cryptor = CardKeyFieldCryptor(transport_keys)
if os.path.isfile(os.path.expanduser(opts.csv)):
card_key_provider_register(CardKeyProviderCsv(os.path.expanduser(opts.csv), card_key_field_cryptor))
if os.path.isfile(os.path.expanduser(opts.pgsql)):
card_key_provider_register(CardKeyProviderPgsql(os.path.expanduser(opts.pgsql), card_key_field_cryptor))

1644
pySim/cards.py
View File

File diff suppressed because it is too large Load Diff

1205
pySim/cat.py
View File

File diff suppressed because it is too large Load Diff

208
pySim/cdma_ruim.py Normal file
View File

@@ -0,0 +1,208 @@
# coding=utf-8
"""R-UIM (Removable User Identity Module) card profile (see 3GPP2 C.S0023-D)
(C) 2023 by Vadim Yanitskiy <fixeria@osmocom.org>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
import enum
from construct import Bytewise, BitStruct, BitsInteger, Struct, FlagsEnum
from osmocom.utils import *
from osmocom.construct import *
from pySim.filesystem import *
from pySim.profile import CardProfile, CardProfileAddon
from pySim.ts_51_011 import CardProfileSIM
from pySim.ts_51_011 import DF_TELECOM, DF_GSM
from pySim.ts_51_011 import EF_ServiceTable
# Mapping between CDMA Service Number and its description
EF_CST_map = {
1 : 'CHV disable function',
2 : 'Abbreviated Dialing Numbers (ADN)',
3 : 'Fixed Dialing Numbers (FDN)',
4 : 'Short Message Storage (SMS)',
5 : 'HRPD',
6 : 'Enhanced Phone Book',
7 : 'Multi Media Domain (MMD)',
8 : 'SF_EUIMID-based EUIMID',
9 : 'MEID Support',
10 : 'Extension1',
11 : 'Extension2',
12 : 'SMS Parameters',
13 : 'Last Number Dialled (LND)',
14 : 'Service Category Program for BC-SMS',
15 : 'Messaging and 3GPD Extensions',
16 : 'Root Certificates',
17 : 'CDMA Home Service Provider Name',
18 : 'Service Dialing Numbers (SDN)',
19 : 'Extension3',
20 : '3GPD-SIP',
21 : 'WAP Browser',
22 : 'Java',
23 : 'Reserved for CDG',
24 : 'Reserved for CDG',
25 : 'Data Download via SMS Broadcast',
26 : 'Data Download via SMS-PP',
27 : 'Menu Selection',
28 : 'Call Control',
29 : 'Proactive R-UIM',
30 : 'AKA',
31 : 'IPv6',
32 : 'RFU',
33 : 'RFU',
34 : 'RFU',
35 : 'RFU',
36 : 'RFU',
37 : 'RFU',
38 : '3GPD-MIP',
39 : 'BCMCS',
40 : 'Multimedia Messaging Service (MMS)',
41 : 'Extension 8',
42 : 'MMS User Connectivity Parameters',
43 : 'Application Authentication',
44 : 'Group Identifier Level 1',
45 : 'Group Identifier Level 2',
46 : 'De-Personalization Control Keys',
47 : 'Cooperative Network List',
}
######################################################################
# DF.CDMA
######################################################################
class EF_SPN(TransparentEF):
'''3.4.31 CDMA Home Service Provider Name'''
_test_de_encode = [
( "010801536b796c696e6b204e57ffffffffffffffffffffffffffffffffffffffffffff",
{ 'rfu1' : 0, 'show_in_hsa' : True, 'rfu2' : 0,
'char_encoding' : 8, 'lang_ind' : 1, 'spn' : 'Skylink NW' } ),
]
def __init__(self, fid='6f41', sfid=None, name='EF.SPN',
desc='Service Provider Name', size=(35, 35), **kwargs):
super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, **kwargs)
self._construct = BitStruct(
# Byte 1: Display Condition
'rfu1'/BitsRFU(7),
'show_in_hsa'/Flag,
# Byte 2: Character Encoding
'rfu2'/BitsRFU(3),
'char_encoding'/BitsInteger(5), # see C.R1001-G
# Byte 3: Language Indicator
'lang_ind'/BitsInteger(8), # see C.R1001-G
# Bytes 4-35: Service Provider Name
'spn'/Bytewise(GsmString(32))
)
class EF_AD(TransparentEF):
'''3.4.33 Administrative Data'''
_test_de_encode = [
( "000000", { 'ms_operation_mode' : 'normal', 'additional_info' : b'\x00\x00', 'rfu' : b'' } ),
]
_test_no_pad = True
class OP_MODE(enum.IntEnum):
normal = 0x00
type_approval = 0x80
normal_and_specific_facilities = 0x01
type_approval_and_specific_facilities = 0x81
maintenance_off_line = 0x02
cell_test = 0x04
def __init__(self, fid='6f43', sfid=None, name='EF.AD',
desc='Administrative Data', size=(3, None), **kwargs):
super().__init__(fid, sfid=sfid, name=name, desc=desc, size=size, **kwargs)
self._construct = Struct(
# Byte 1: MS operation mode
'ms_operation_mode'/Enum(Byte, self.OP_MODE),
# Bytes 2-3: Additional information
'additional_info'/Bytes(2),
# Bytes 4..: RFU
'rfu'/GreedyBytesRFU,
)
class EF_SMS(LinFixedEF):
'''3.4.27 Short Messages'''
def __init__(self, fid='6f3c', sfid=None, name='EF.SMS', desc='Short messages', **kwargs):
super().__init__(fid, sfid=sfid, name=name, desc=desc, rec_len=(2, 255), **kwargs)
self._construct = Struct(
# Byte 1: Status
'status'/BitStruct(
'rfu87'/BitsRFU(2),
'protection'/Flag,
'rfu54'/BitsRFU(2),
'status'/FlagsEnum(BitsInteger(2), read=0, to_be_read=1, sent=2, to_be_sent=3),
'used'/Flag,
),
# Byte 2: Length
'length'/Int8ub,
# Bytes 3..: SMS Transport Layer Message
'tpdu'/Bytes(lambda ctx: ctx.length if ctx.status.used else 0),
)
class DF_CDMA(CardDF):
def __init__(self):
super().__init__(fid='7f25', name='DF.CDMA',
desc='CDMA related files (3GPP2 C.S0023-D)')
files = [
# TODO: lots of other files
EF_ServiceTable('6f32', None, 'EF.CST',
'CDMA Service Table', table=EF_CST_map, size=(5, 16)),
EF_SPN(),
EF_AD(),
EF_SMS(),
]
self.add_files(files)
class CardProfileRUIM(CardProfile):
'''R-UIM card profile as per 3GPP2 C.S0023-D'''
ORDER = 20
def __init__(self):
super().__init__('R-UIM', desc='CDMA R-UIM Card', cla="a0",
sel_ctrl="0000", files_in_mf=[DF_TELECOM(), DF_GSM(), DF_CDMA()])
@staticmethod
def decode_select_response(data_hex: str) -> object:
# TODO: Response parameters/data in case of DF_CDMA (section 2.6)
return CardProfileSIM.decode_select_response(data_hex)
@classmethod
def _try_match_card(cls, scc: SimCardCommands) -> None:
""" Try to access MF/DF.CDMA via 2G APDUs (3GPP TS 11.11), if this works,
the card is considered an R-UIM card for CDMA."""
cls._mf_select_test(scc, "a0", "0000", ["3f00", "7f25"])
class AddonRUIM(CardProfileAddon):
"""An Addon that can be found on on a combined SIM + RUIM or UICC + RUIM to support CDMA."""
def __init__(self):
files = [
DF_CDMA()
]
super().__init__('RUIM', desc='CDMA RUIM', files_in_mf=files)
def probe(self, card: 'CardBase') -> bool:
return card.file_exists(self.files_in_mf[0].fid)

557
pySim/commands.py
View File

@@ -5,7 +5,7 @@
#
# Copyright (C) 2009-2010 Sylvain Munaut <tnt@246tNt.com>
# Copyright (C) 2010-2021 Harald Welte <laforge@gnumonks.org>
# Copyright (C) 2010-2024 Harald Welte <laforge@gnumonks.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -21,20 +21,163 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from construct import *
from pySim.construct import LV
from pySim.utils import rpad, b2h, h2b, sw_match, bertlv_encode_len, Hexstr, h2i, str_sanitize
from typing import List, Tuple
import typing # construct also has a Union, so we do typing.Union below
from construct import Construct, Struct, Const, Select
from construct import Optional as COptional
from osmocom.construct import LV, filter_dict
from osmocom.utils import rpad, lpad, b2h, h2b, h2i, i2h, str_sanitize, Hexstr
from osmocom.tlv import bertlv_encode_len
from pySim.utils import sw_match, expand_hex, SwHexstr, ResTuple, SwMatchstr
from pySim.exceptions import SwMatchError
from pySim.transport import LinkBase
# A path can be either just a FID or a list of FID
Path = typing.Union[Hexstr, List[Hexstr]]
class SimCardCommands(object):
def __init__(self, transport):
def lchan_nr_to_cla(cla: int, lchan_nr: int) -> int:
"""Embed a logical channel number into the CLA byte."""
# TS 102 221 10.1.1 Coding of Class Byte
if lchan_nr < 4:
# standard logical channel number
if cla >> 4 in [0x0, 0xA, 0x8]:
return (cla & 0xFC) | (lchan_nr & 3)
else:
raise ValueError('Undefined how to use CLA %2X with logical channel %u' % (cla, lchan_nr))
elif lchan_nr < 16:
# extended logical channel number
if cla >> 6 in [1, 3]:
return (cla & 0xF0) | ((lchan_nr - 4) & 0x0F)
else:
raise ValueError('Undefined how to use CLA %2X with logical channel %u' % (cla, lchan_nr))
else:
raise ValueError('logical channel outside of range 0 .. 15')
def cla_with_lchan(cla_byte: Hexstr, lchan_nr: int) -> Hexstr:
"""Embed a logical channel number into the hex-string encoded CLA value."""
cla_int = h2i(cla_byte)[0]
return i2h([lchan_nr_to_cla(cla_int, lchan_nr)])
class SimCardCommands:
"""Class providing methods for various card-specific commands such as SELECT, READ BINARY, etc.
Historically one instance exists below CardBase, but with the introduction of multiple logical
channels there can be multiple instances. The lchan number will then be patched into the CLA
byte by the respective instance. """
def __init__(self, transport: LinkBase, lchan_nr: int = 0):
self._tp = transport
self.cla_byte = "a0"
self.sel_ctrl = "0000"
self.lchan_nr = lchan_nr
# invokes the setter below
self.cla_byte = "a0"
self.scp = None # Secure Channel Protocol
def fork_lchan(self, lchan_nr: int) -> 'SimCardCommands':
"""Fork a per-lchan specific SimCardCommands instance off the current instance."""
ret = SimCardCommands(transport = self._tp, lchan_nr = lchan_nr)
ret.cla_byte = self.cla_byte
ret.sel_ctrl = self.sel_ctrl
return ret
@property
def max_cmd_len(self) -> int:
"""Maximum length of the command apdu data section. Depends on secure channel protocol used."""
if self.scp:
return 255 - self.scp.overhead
else:
return 255
def send_apdu(self, pdu: Hexstr, apply_lchan:bool = True) -> ResTuple:
"""Sends an APDU and auto fetch response data
Args:
pdu : string of hexadecimal characters (ex. "A0A40000023F00")
apply_lchan : apply the currently selected lchan to the CLA byte before sending
Returns:
tuple(data, sw), where
data : string (in hex) of returned data (ex. "074F4EFFFF")
sw : string (in hex) of status word (ex. "9000")
"""
if apply_lchan:
pdu = cla_with_lchan(pdu[0:2], self.lchan_nr) + pdu[2:]
if self.scp:
return self.scp.send_apdu_wrapper(self._tp.send_apdu, pdu)
else:
return self._tp.send_apdu(pdu)
def send_apdu_checksw(self, pdu: Hexstr, sw: SwMatchstr = "9000", apply_lchan:bool = True) -> ResTuple:
"""Sends an APDU and check returned SW
Args:
pdu : string of hexadecimal characters (ex. "A0A40000023F00")
sw : string of 4 hexadecimal characters (ex. "9000"). The user may mask out certain
digits using a '?' to add some ambiguity if needed.
apply_lchan : apply the currently selected lchan to the CLA byte before sending
Returns:
tuple(data, sw), where
data : string (in hex) of returned data (ex. "074F4EFFFF")
sw : string (in hex) of status word (ex. "9000")
"""
if apply_lchan:
pdu = cla_with_lchan(pdu[0:2], self.lchan_nr) + pdu[2:]
if self.scp:
return self.scp.send_apdu_wrapper(self._tp.send_apdu_checksw, pdu, sw)
else:
return self._tp.send_apdu_checksw(pdu, sw)
def send_apdu_constr(self, cla: Hexstr, ins: Hexstr, p1: Hexstr, p2: Hexstr, cmd_constr: Construct,
cmd_data: Hexstr, resp_constr: Construct, apply_lchan:bool = True) -> Tuple[dict, SwHexstr]:
"""Build and sends an APDU using a 'construct' definition; parses response.
Args:
cla : string (in hex) ISO 7816 class byte
ins : string (in hex) ISO 7816 instruction byte
p1 : string (in hex) ISO 7116 Parameter 1 byte
p2 : string (in hex) ISO 7116 Parameter 2 byte
cmd_cosntr : defining how to generate binary APDU command data
cmd_data : command data passed to cmd_constr
resp_cosntr : defining how to decode binary APDU response data
apply_lchan : apply the currently selected lchan to the CLA byte before sending
Returns:
Tuple of (decoded_data, sw)
"""
cmd = cmd_constr.build(cmd_data) if cmd_data else b''
lc = i2h([len(cmd)]) if cmd_data else ''
le = '00' if resp_constr else ''
pdu = ''.join([cla, ins, p1, p2, lc, b2h(cmd), le])
(data, sw) = self.send_apdu(pdu, apply_lchan = apply_lchan)
if data:
# filter the resulting dict to avoid '_io' members inside
rsp = filter_dict(resp_constr.parse(h2b(data)))
else:
rsp = None
return (rsp, sw)
def send_apdu_constr_checksw(self, cla: Hexstr, ins: Hexstr, p1: Hexstr, p2: Hexstr,
cmd_constr: Construct, cmd_data: Hexstr, resp_constr: Construct,
sw_exp: SwMatchstr="9000", apply_lchan:bool = True) -> Tuple[dict, SwHexstr]:
"""Build and sends an APDU using a 'construct' definition; parses response.
Args:
cla : string (in hex) ISO 7816 class byte
ins : string (in hex) ISO 7816 instruction byte
p1 : string (in hex) ISO 7116 Parameter 1 byte
p2 : string (in hex) ISO 7116 Parameter 2 byte
cmd_cosntr : defining how to generate binary APDU command data
cmd_data : command data passed to cmd_constr
resp_cosntr : defining how to decode binary APDU response data
exp_sw : string (in hex) of status word (ex. "9000")
Returns:
Tuple of (decoded_data, sw)
"""
(rsp, sw) = self.send_apdu_constr(cla, ins, p1, p2, cmd_constr, cmd_data, resp_constr,
apply_lchan = apply_lchan)
if not sw_match(sw, sw_exp):
raise SwMatchError(sw, sw_exp.lower(), self._tp.sw_interpreter)
return (rsp, sw)
# Extract a single FCP item from TLV
def __parse_fcp(self, fcp):
def __parse_fcp(self, fcp: Hexstr):
# see also: ETSI TS 102 221, chapter 11.1.1.3.1 Response for MF,
# DF or ADF
from pytlv.TLV import TLV
@@ -53,6 +196,7 @@ class SimCardCommands(object):
# checking if the length of the remaining TLV string matches
# what we get in the length field.
# See also ETSI TS 102 221, chapter 11.1.1.3.0 Base coding.
# TODO: this likely just is normal BER-TLV ("All data objects are BER-TLV except if otherwise # defined.")
exp_tlv_len = int(fcp[2:4], 16)
if len(fcp[4:]) // 2 == exp_tlv_len:
skip = 4
@@ -60,6 +204,7 @@ class SimCardCommands(object):
exp_tlv_len = int(fcp[2:6], 16)
if len(fcp[4:]) // 2 == exp_tlv_len:
skip = 6
raise ValueError('Cannot determine length of TLV-length')
# Skip FCP tag and length
tlv = fcp[skip:]
@@ -88,11 +233,11 @@ class SimCardCommands(object):
else:
return int(r[-1][4:8], 16)
def get_atr(self) -> str:
def get_atr(self) -> Hexstr:
"""Return the ATR of the currently inserted card."""
return self._tp.get_atr()
def try_select_path(self, dir_list):
def try_select_path(self, dir_list: List[Hexstr]) -> List[ResTuple]:
""" Try to select a specified path
Args:
@@ -100,17 +245,16 @@ class SimCardCommands(object):
"""
rv = []
if type(dir_list) is not list:
if not isinstance(dir_list, list):
dir_list = [dir_list]
for i in dir_list:
data, sw = self._tp.send_apdu(
self.cla_byte + "a4" + self.sel_ctrl + "02" + i)
data, sw = self.send_apdu(self.cla_byte + "a4" + self.sel_ctrl + "02" + i + "00")
rv.append((data, sw))
if sw != '9000':
return rv
return rv
def select_path(self, dir_list):
def select_path(self, dir_list: Path) -> List[Hexstr]:
"""Execute SELECT for an entire list/path of FIDs.
Args:
@@ -120,33 +264,37 @@ class SimCardCommands(object):
list of return values (FCP in hex encoding) for each element of the path
"""
rv = []
if type(dir_list) is not list:
if not isinstance(dir_list, list):
dir_list = [dir_list]
for i in dir_list:
data, sw = self.select_file(i)
data, _sw = self.select_file(i)
rv.append(data)
return rv
def select_file(self, fid: str):
def select_file(self, fid: Hexstr) -> ResTuple:
"""Execute SELECT a given file by FID.
Args:
fid : file identifier as hex string
"""
return self._tp.send_apdu_checksw(self.cla_byte + "a4" + self.sel_ctrl + "02" + fid)
return self.send_apdu_checksw(self.cla_byte + "a4" + self.sel_ctrl + "02" + fid + "00")
def select_adf(self, aid: str):
"""Execute SELECT a given Applicaiton ADF.
def select_parent_df(self) -> ResTuple:
"""Execute SELECT to switch to the parent DF """
return self.send_apdu_checksw(self.cla_byte + "a40304")
def select_adf(self, aid: Hexstr) -> ResTuple:
"""Execute SELECT a given Application ADF.
Args:
aid : application identifier as hex string
"""
aidlen = ("0" + format(len(aid) // 2, 'x'))[-2:]
return self._tp.send_apdu_checksw(self.cla_byte + "a4" + "0404" + aidlen + aid)
return self.send_apdu_checksw(self.cla_byte + "a4" + "0404" + aidlen + aid + "00")
def read_binary(self, ef, length: int = None, offset: int = 0):
def read_binary(self, ef: Path, length: int = None, offset: int = 0) -> ResTuple:
"""Execute READD BINARY.
Args:
@@ -165,56 +313,19 @@ class SimCardCommands(object):
total_data = ''
chunk_offset = 0
while chunk_offset < length:
chunk_len = min(255, length-chunk_offset)
chunk_len = min(self.max_cmd_len, length-chunk_offset)
pdu = self.cla_byte + \
'b0%04x%02x' % (offset + chunk_offset, chunk_len)
try:
data, sw = self._tp.send_apdu_checksw(pdu)
data, sw = self.send_apdu_checksw(pdu)
except Exception as e:
raise ValueError('%s, failed to read (offset %d)' %
(str_sanitize(str(e)), offset))
e.add_note('failed to read (offset %d)' % offset)
raise e
total_data += data
chunk_offset += chunk_len
return total_data, sw
def update_binary(self, ef, data: str, offset: int = 0, verify: bool = False, conserve: bool = False):
"""Execute UPDATE BINARY.
Args:
ef : string or list of strings indicating name or path of transparent EF
data : hex string of data to be written
offset : byte offset in file from which to start writing
verify : Whether or not to verify data after write
"""
data_length = len(data) // 2
# Save write cycles by reading+comparing before write
if conserve:
data_current, sw = self.read_binary(ef, data_length, offset)
if data_current == data:
return None, sw
self.select_path(ef)
total_data = ''
chunk_offset = 0
while chunk_offset < data_length:
chunk_len = min(255, data_length - chunk_offset)
# chunk_offset is bytes, but data slicing is hex chars, so we need to multiply by 2
pdu = self.cla_byte + \
'd6%04x%02x' % (offset + chunk_offset, chunk_len) + \
data[chunk_offset*2: (chunk_offset+chunk_len)*2]
try:
chunk_data, chunk_sw = self._tp.send_apdu_checksw(pdu)
except Exception as e:
raise ValueError('%s, failed to write chunk (chunk_offset %d, chunk_len %d)' %
(str_sanitize(str(e)), chunk_offset, chunk_len))
total_data += data
chunk_offset += chunk_len
if verify:
self.verify_binary(ef, data, offset)
return total_data, chunk_sw
def verify_binary(self, ef, data: str, offset: int = 0):
def __verify_binary(self, ef, data: str, offset: int = 0):
"""Verify contents of transparent EF.
Args:
@@ -227,7 +338,56 @@ class SimCardCommands(object):
raise ValueError('Binary verification failed (expected %s, got %s)' % (
data.lower(), res[0].lower()))
def read_record(self, ef, rec_no: int):
def update_binary(self, ef: Path, data: Hexstr, offset: int = 0, verify: bool = False,
conserve: bool = False) -> ResTuple:
"""Execute UPDATE BINARY.
Args:
ef : string or list of strings indicating name or path of transparent EF
data : hex string of data to be written
offset : byte offset in file from which to start writing
verify : Whether or not to verify data after write
"""
file_len = self.binary_size(ef)
data = expand_hex(data, file_len)
data_length = len(data) // 2
# Save write cycles by reading+comparing before write
if conserve:
try:
data_current, sw = self.read_binary(ef, data_length, offset)
if data_current == data:
return None, sw
except Exception:
# cannot read data. This is not a fatal error, as reading is just done to
# conserve the amount of smart card writes. The access conditions of the file
# may well permit us to UPDATE but not permit us to READ. So let's ignore
# any such exception during READ.
pass
self.select_path(ef)
total_data = ''
chunk_offset = 0
while chunk_offset < data_length:
chunk_len = min(self.max_cmd_len, data_length - chunk_offset)
# chunk_offset is bytes, but data slicing is hex chars, so we need to multiply by 2
pdu = self.cla_byte + \
'd6%04x%02x' % (offset + chunk_offset, chunk_len) + \
data[chunk_offset*2: (chunk_offset+chunk_len)*2]
try:
chunk_data, chunk_sw = self.send_apdu_checksw(pdu)
except Exception as e:
e.add_note('failed to write chunk (chunk_offset %d, chunk_len %d)' % (chunk_offset, chunk_len))
raise e
total_data += data
chunk_offset += chunk_len
if verify:
self.__verify_binary(ef, data, offset)
return total_data, chunk_sw
def read_record(self, ef: Path, rec_no: int) -> ResTuple:
"""Execute READ RECORD.
Args:
@@ -237,50 +397,9 @@ class SimCardCommands(object):
r = self.select_path(ef)
rec_length = self.__record_len(r)
pdu = self.cla_byte + 'b2%02x04%02x' % (rec_no, rec_length)
return self._tp.send_apdu_checksw(pdu)
return self.send_apdu_checksw(pdu)
def update_record(self, ef, rec_no: int, data: str, force_len: bool = False, verify: bool = False,
conserve: bool = False):
"""Execute UPDATE RECORD.
Args:
ef : string or list of strings indicating name or path of linear fixed EF
rec_no : record number to read
data : hex string of data to be written
force_len : enforce record length by using the actual data length
verify : verify data by re-reading the record
conserve : read record and compare it with data, skip write on match
"""
res = self.select_path(ef)
if force_len:
# enforce the record length by the actual length of the given data input
rec_length = len(data) // 2
else:
# determine the record length from the select response of the file and pad
# the input data with 0xFF if necessary. In cases where the input data
# exceed we throw an exception.
rec_length = self.__record_len(res)
if (len(data) // 2 > rec_length):
raise ValueError('Data length exceeds record length (expected max %d, got %d)' % (
rec_length, len(data) // 2))
elif (len(data) // 2 < rec_length):
data = rpad(data, rec_length * 2)
# Save write cycles by reading+comparing before write
if conserve:
data_current, sw = self.read_record(ef, rec_no)
data_current = data_current[0:rec_length*2]
if data_current == data:
return None, sw
pdu = (self.cla_byte + 'dc%02x04%02x' % (rec_no, rec_length)) + data
res = self._tp.send_apdu_checksw(pdu)
if verify:
self.verify_record(ef, rec_no, data)
return res
def verify_record(self, ef, rec_no: int, data: str):
def __verify_record(self, ef: Path, rec_no: int, data: str):
"""Verify record against given data
Args:
@@ -293,7 +412,60 @@ class SimCardCommands(object):
raise ValueError('Record verification failed (expected %s, got %s)' % (
data.lower(), res[0].lower()))
def record_size(self, ef):
def update_record(self, ef: Path, rec_no: int, data: Hexstr, force_len: bool = False,
verify: bool = False, conserve: bool = False, leftpad: bool = False) -> ResTuple:
"""Execute UPDATE RECORD.
Args:
ef : string or list of strings indicating name or path of linear fixed EF
rec_no : record number to read
data : hex string of data to be written
force_len : enforce record length by using the actual data length
verify : verify data by re-reading the record
conserve : read record and compare it with data, skip write on match
leftpad : apply 0xff padding from the left instead from the right side.
"""
res = self.select_path(ef)
rec_length = self.__record_len(res)
data = expand_hex(data, rec_length)
if force_len:
# enforce the record length by the actual length of the given data input
rec_length = len(data) // 2
else:
# make sure the input data is padded to the record length using 0xFF.
# In cases where the input data exceed we throw an exception.
if len(data) // 2 > rec_length:
raise ValueError('Data length exceeds record length (expected max %d, got %d)' % (
rec_length, len(data) // 2))
elif len(data) // 2 < rec_length:
if leftpad:
data = lpad(data, rec_length * 2)
else:
data = rpad(data, rec_length * 2)
# Save write cycles by reading+comparing before write
if conserve:
try:
data_current, sw = self.read_record(ef, rec_no)
data_current = data_current[0:rec_length*2]
if data_current == data:
return None, sw
except Exception:
# cannot read data. This is not a fatal error, as reading is just done to
# conserve the amount of smart card writes. The access conditions of the file
# may well permit us to UPDATE but not permit us to READ. So let's ignore
# any such exception during READ.
pass
pdu = (self.cla_byte + 'dc%02x04%02x' % (rec_no, rec_length)) + data
res = self.send_apdu_checksw(pdu)
if verify:
self.__verify_record(ef, rec_no, data)
return res
def record_size(self, ef: Path) -> int:
"""Determine the record size of given file.
Args:
@@ -302,7 +474,7 @@ class SimCardCommands(object):
r = self.select_path(ef)
return self.__record_len(r)
def record_count(self, ef):
def record_count(self, ef: Path) -> int:
"""Determine the number of records in given file.
Args:
@@ -311,7 +483,7 @@ class SimCardCommands(object):
r = self.select_path(ef)
return self.__len(r) // self.__record_len(r)
def binary_size(self, ef):
def binary_size(self, ef: Path) -> int:
"""Determine the size of given transparent file.
Args:
@@ -321,14 +493,14 @@ class SimCardCommands(object):
return self.__len(r)
# TS 102 221 Section 11.3.1 low-level helper
def _retrieve_data(self, tag: int, first: bool = True):
def _retrieve_data(self, tag: int, first: bool = True) -> ResTuple:
if first:
pdu = '80cb008001%02x' % (tag)
pdu = '80cb008001%02x00' % (tag)
else:
pdu = '80cb000000'
return self._tp.send_apdu_checksw(pdu)
pdu = '80cb0000'
return self.send_apdu_checksw(pdu)
def retrieve_data(self, ef, tag: int):
def retrieve_data(self, ef: Path, tag: int) -> ResTuple:
"""Execute RETRIEVE DATA, see also TS 102 221 Section 11.3.1.
Args
@@ -342,23 +514,23 @@ class SimCardCommands(object):
# retrieve first block
data, sw = self._retrieve_data(tag, first=True)
total_data += data
while sw == '62f1' or sw == '62f2':
while sw in ['62f1', '62f2']:
data, sw = self._retrieve_data(tag, first=False)
total_data += data
return total_data, sw
# TS 102 221 Section 11.3.2 low-level helper
def _set_data(self, data: str, first: bool = True):
def _set_data(self, data: Hexstr, first: bool = True) -> ResTuple:
if first:
p1 = 0x80
else:
p1 = 0x00
if isinstance(data, bytes) or isinstance(data, bytearray):
if isinstance(data, (bytes, bytearray)):
data = b2h(data)
pdu = '80db00%02x%02x%s' % (p1, len(data)//2, data)
return self._tp.send_apdu_checksw(pdu)
return self.send_apdu_checksw(pdu)
def set_data(self, ef, tag: int, value: str, verify: bool = False, conserve: bool = False):
def set_data(self, ef, tag: int, value: str, verify: bool = False, conserve: bool = False) -> ResTuple:
"""Execute SET DATA.
Args
@@ -383,13 +555,13 @@ class SimCardCommands(object):
total_len = len(tlv_bin)
remaining = tlv_bin
while len(remaining) > 0:
fragment = remaining[:255]
fragment = remaining[:self.max_cmd_len]
rdata, sw = self._set_data(fragment, first=first)
first = False
remaining = remaining[255:]
remaining = remaining[self.max_cmd_len:]
return rdata, sw
def run_gsm(self, rand: str):
def run_gsm(self, rand: Hexstr) -> ResTuple:
"""Execute RUN GSM ALGORITHM.
Args:
@@ -398,21 +570,20 @@ class SimCardCommands(object):
if len(rand) != 32:
raise ValueError('Invalid rand')
self.select_path(['3f00', '7f20'])
return self._tp.send_apdu(self.cla_byte + '88000010' + rand)
return self.send_apdu_checksw('a088000010' + rand + '00', sw='9000')
def authenticate(self, rand: str, autn: str, context='3g'):
def authenticate(self, rand: Hexstr, autn: Hexstr, context: str = '3g') -> ResTuple:
"""Execute AUTHENTICATE (USIM/ISIM).
Args:
rand : 16 byte random data as hex string (RAND)
autn : 8 byte Autentication Token (AUTN)
autn : 8 byte Authentication Token (AUTN)
context : 16 byte random data ('3g' or 'gsm')
"""
# 3GPP TS 31.102 Section 7.1.2.1
AuthCmd3G = Struct('rand'/LV, 'autn'/Optional(LV))
AuthCmd3G = Struct('rand'/LV, 'autn'/COptional(LV))
AuthResp3GSyncFail = Struct(Const(b'\xDC'), 'auts'/LV)
AuthResp3GSuccess = Struct(
Const(b'\xDB'), 'res'/LV, 'ck'/LV, 'ik'/LV, 'kc'/Optional(LV))
AuthResp3GSuccess = Struct(Const(b'\xDB'), 'res'/LV, 'ck'/LV, 'ik'/LV, 'kc'/COptional(LV))
AuthResp3G = Select(AuthResp3GSyncFail, AuthResp3GSuccess)
# build parameters
cmd_data = {'rand': rand, 'autn': autn}
@@ -420,7 +591,9 @@ class SimCardCommands(object):
p2 = '81'
elif context == 'gsm':
p2 = '80'
(data, sw) = self._tp.send_apdu_constr_checksw(
else:
raise ValueError("Unsupported context '%s'" % context)
(data, sw) = self.send_apdu_constr_checksw(
self.cla_byte, '88', '00', p2, AuthCmd3G, cmd_data, AuthResp3G)
if 'auts' in data:
ret = {'synchronisation_failure': data}
@@ -428,23 +601,47 @@ class SimCardCommands(object):
ret = {'successful_3g_authentication': data}
return (ret, sw)
def status(self):
def status(self) -> ResTuple:
"""Execute a STATUS command as per TS 102 221 Section 11.1.2."""
return self._tp.send_apdu_checksw('80F20000ff')
return self.send_apdu_checksw('80F20000')
def deactivate_file(self):
def deactivate_file(self) -> ResTuple:
"""Execute DECATIVATE FILE command as per TS 102 221 Section 11.1.14."""
return self._tp.send_apdu_constr_checksw(self.cla_byte, '04', '00', '00', None, None, None)
return self.send_apdu_constr_checksw(self.cla_byte, '04', '00', '00', None, None, None)
def activate_file(self, fid):
def activate_file(self, fid: Hexstr) -> ResTuple:
"""Execute ACTIVATE FILE command as per TS 102 221 Section 11.1.15.
Args:
fid : file identifier as hex string
"""
return self._tp.send_apdu_checksw(self.cla_byte + '44000002' + fid)
return self.send_apdu_checksw(self.cla_byte + '44000002' + fid)
def manage_channel(self, mode='open', lchan_nr=0):
def create_file(self, payload: Hexstr) -> ResTuple:
"""Execute CREATE FILE command as per TS 102 222 Section 6.3"""
return self.send_apdu_checksw(self.cla_byte + 'e00000%02x%s' % (len(payload)//2, payload))
def resize_file(self, payload: Hexstr) -> ResTuple:
"""Execute RESIZE FILE command as per TS 102 222 Section 6.10"""
return self.send_apdu_checksw('80d40000%02x%s' % (len(payload)//2, payload))
def delete_file(self, fid: Hexstr) -> ResTuple:
"""Execute DELETE FILE command as per TS 102 222 Section 6.4"""
return self.send_apdu_checksw(self.cla_byte + 'e4000002' + fid)
def terminate_df(self, fid: Hexstr) -> ResTuple:
"""Execute TERMINATE DF command as per TS 102 222 Section 6.7"""
return self.send_apdu_checksw(self.cla_byte + 'e6000002' + fid)
def terminate_ef(self, fid: Hexstr) -> ResTuple:
"""Execute TERMINATE EF command as per TS 102 222 Section 6.8"""
return self.send_apdu_checksw(self.cla_byte + 'e8000002' + fid)
def terminate_card_usage(self) -> ResTuple:
"""Execute TERMINATE CARD USAGE command as per TS 102 222 Section 6.9"""
return self.send_apdu_checksw(self.cla_byte + 'fe000000')
def manage_channel(self, mode: str = 'open', lchan_nr: int =0) -> ResTuple:
"""Execute MANAGE CHANNEL command as per TS 102 221 Section 11.1.17.
Args:
@@ -455,21 +652,21 @@ class SimCardCommands(object):
p1 = 0x80
else:
p1 = 0x00
pdu = self.cla_byte + '70%02x%02x00' % (p1, lchan_nr)
return self._tp.send_apdu_checksw(pdu)
pdu = self.cla_byte + '70%02x%02x' % (p1, lchan_nr)
return self.send_apdu_checksw(pdu)
def reset_card(self):
def reset_card(self) -> Hexstr:
"""Physically reset the card"""
return self._tp.reset_card()
def _chv_process_sw(self, op_name, chv_no, pin_code, sw):
def _chv_process_sw(self, op_name: str, chv_no: int, pin_code: Hexstr, sw: SwHexstr):
if sw_match(sw, '63cx'):
raise RuntimeError('Failed to %s chv_no 0x%02X with code 0x%s, %i tries left.' %
(op_name, chv_no, b2h(pin_code).upper(), int(sw[3])))
elif (sw != '9000'):
raise SwMatchError(sw, '9000')
if sw != '9000':
raise SwMatchError(sw, '9000', self._tp.sw_interpreter)
def verify_chv(self, chv_no: int, code: str):
def verify_chv(self, chv_no: int, code: Hexstr) -> ResTuple:
"""Verify a given CHV (Card Holder Verification == PIN)
Args:
@@ -477,8 +674,7 @@ class SimCardCommands(object):
code : chv code as hex string
"""
fc = rpad(b2h(code), 16)
data, sw = self._tp.send_apdu(
self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
data, sw = self.send_apdu(self.cla_byte + '2000' + ('%02X' % chv_no) + '08' + fc)
self._chv_process_sw('verify', chv_no, code, sw)
return (data, sw)
@@ -491,12 +687,11 @@ class SimCardCommands(object):
pin_code : new chv code as hex string
"""
fc = rpad(b2h(puk_code), 16) + rpad(b2h(pin_code), 16)
data, sw = self._tp.send_apdu(
self.cla_byte + '2C00' + ('%02X' % chv_no) + '10' + fc)
data, sw = self.send_apdu(self.cla_byte + '2C00' + ('%02X' % chv_no) + '10' + fc)
self._chv_process_sw('unblock', chv_no, pin_code, sw)
return (data, sw)
def change_chv(self, chv_no: int, pin_code: str, new_pin_code: str):
def change_chv(self, chv_no: int, pin_code: Hexstr, new_pin_code: Hexstr) -> ResTuple:
"""Change a given CHV (Card Holder Verification == PIN)
Args:
@@ -505,12 +700,11 @@ class SimCardCommands(object):
new_pin_code : new chv code as hex string
"""
fc = rpad(b2h(pin_code), 16) + rpad(b2h(new_pin_code), 16)
data, sw = self._tp.send_apdu(
self.cla_byte + '2400' + ('%02X' % chv_no) + '10' + fc)
data, sw = self.send_apdu(self.cla_byte + '2400' + ('%02X' % chv_no) + '10' + fc)
self._chv_process_sw('change', chv_no, pin_code, sw)
return (data, sw)
def disable_chv(self, chv_no: int, pin_code: str):
def disable_chv(self, chv_no: int, pin_code: Hexstr) -> ResTuple:
"""Disable a given CHV (Card Holder Verification == PIN)
Args:
@@ -519,12 +713,11 @@ class SimCardCommands(object):
new_pin_code : new chv code as hex string
"""
fc = rpad(b2h(pin_code), 16)
data, sw = self._tp.send_apdu(
self.cla_byte + '2600' + ('%02X' % chv_no) + '08' + fc)
data, sw = self.send_apdu(self.cla_byte + '2600' + ('%02X' % chv_no) + '08' + fc)
self._chv_process_sw('disable', chv_no, pin_code, sw)
return (data, sw)
def enable_chv(self, chv_no: int, pin_code: str):
def enable_chv(self, chv_no: int, pin_code: Hexstr) -> ResTuple:
"""Enable a given CHV (Card Holder Verification == PIN)
Args:
@@ -532,31 +725,30 @@ class SimCardCommands(object):
pin_code : chv code as hex string
"""
fc = rpad(b2h(pin_code), 16)
data, sw = self._tp.send_apdu(
self.cla_byte + '2800' + ('%02X' % chv_no) + '08' + fc)
data, sw = self.send_apdu(self.cla_byte + '2800' + ('%02X' % chv_no) + '08' + fc)
self._chv_process_sw('enable', chv_no, pin_code, sw)
return (data, sw)
def envelope(self, payload: str):
def envelope(self, payload: Hexstr) -> ResTuple:
"""Send one ENVELOPE command to the SIM
Args:
payload : payload as hex string
"""
return self._tp.send_apdu_checksw('80c20000%02x%s' % (len(payload)//2, payload))
return self.send_apdu_checksw('80c20000%02x%s' % (len(payload)//2, payload) + "00", apply_lchan = False)
def terminal_profile(self, payload: str):
def terminal_profile(self, payload: Hexstr) -> ResTuple:
"""Send TERMINAL PROFILE to card
Args:
payload : payload as hex string
"""
data_length = len(payload) // 2
data, sw = self._tp.send_apdu(('80100000%02x' % data_length) + payload)
data, sw = self.send_apdu_checksw(('80100000%02x' % data_length) + payload, apply_lchan = False)
return (data, sw)
# ETSI TS 102 221 11.1.22
def suspend_uicc(self, min_len_secs: int = 60, max_len_secs: int = 43200):
def suspend_uicc(self, min_len_secs: int = 60, max_len_secs: int = 43200) -> Tuple[int, Hexstr, SwHexstr]:
"""Send SUSPEND UICC to the card.
Args:
@@ -566,34 +758,49 @@ class SimCardCommands(object):
def encode_duration(secs: int) -> Hexstr:
if secs >= 10*24*60*60:
return '04%02x' % (secs // (10*24*60*60))
elif secs >= 24*60*60:
if secs >= 24*60*60:
return '03%02x' % (secs // (24*60*60))
elif secs >= 60*60:
if secs >= 60*60:
return '02%02x' % (secs // (60*60))
elif secs >= 60:
if secs >= 60:
return '01%02x' % (secs // 60)
else:
return '00%02x' % secs
def decode_duration(enc: Hexstr) -> int:
time_unit = enc[:2]
length = h2i(enc[2:4])
length = h2i(enc[2:4])[0]
if time_unit == '04':
return length * 10*24*60*60
elif time_unit == '03':
if time_unit == '03':
return length * 24*60*60
elif time_unit == '02':
if time_unit == '02':
return length * 60*60
elif time_unit == '01':
if time_unit == '01':
return length * 60
elif time_unit == '00':
if time_unit == '00':
return length
else:
raise ValueError('Time unit must be 0x00..0x04')
min_dur_enc = encode_duration(min_len_secs)
max_dur_enc = encode_duration(max_len_secs)
data, sw = self._tp.send_apdu_checksw(
'8076000004' + min_dur_enc + max_dur_enc)
data, sw = self.send_apdu_checksw('8076000004' + min_dur_enc + max_dur_enc, apply_lchan = False)
negotiated_duration_secs = decode_duration(data[:4])
resume_token = data[4:]
return (negotiated_duration_secs, resume_token, sw)
# ETSI TS 102 221 11.1.22
def resume_uicc(self, token: Hexstr) -> ResTuple:
"""Send SUSPEND UICC (resume) to the card."""
if len(h2b(token)) != 8:
raise ValueError("Token must be 8 bytes long")
data, sw = self.send_apdu_checksw('8076010008' + token, apply_lchan = False)
return (data, sw)
# GPC_SPE_034 11.3
def get_data(self, tag: int, cla: int = 0x00):
data, sw = self.send_apdu_checksw('%02xca%04x00' % (cla, tag))
return (data, sw)
# TS 31.102 Section 7.5.2
def get_identity(self, context: int) -> Tuple[Hexstr, SwHexstr]:
data, sw = self.send_apdu_checksw('807800%02x00' % (context))
return (data, sw)

186
pySim/construct.py
View File

@@ -1,186 +0,0 @@
from construct.lib.containers import Container, ListContainer
from construct.core import EnumIntegerString
import typing
from construct import *
from pySim.utils import b2h, h2b, swap_nibbles
import gsm0338
"""Utility code related to the integration of the 'construct' declarative parser."""
# (C) 2021 by Harald Welte <laforge@osmocom.org>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
class HexAdapter(Adapter):
"""convert a bytes() type to a string of hex nibbles."""
def _decode(self, obj, context, path):
return b2h(obj)
def _encode(self, obj, context, path):
return h2b(obj)
class BcdAdapter(Adapter):
"""convert a bytes() type to a string of BCD nibbles."""
def _decode(self, obj, context, path):
return swap_nibbles(b2h(obj))
def _encode(self, obj, context, path):
return h2b(swap_nibbles(obj))
class Rpad(Adapter):
"""
Encoder appends padding bytes (b'\\xff') up to target size.
Decoder removes trailing padding bytes.
Parameters:
subcon: Subconstruct as defined by construct library
pattern: set padding pattern (default: b'\\xff')
"""
def __init__(self, subcon, pattern=b'\xff'):
super().__init__(subcon)
self.pattern = pattern
def _decode(self, obj, context, path):
return obj.rstrip(self.pattern)
def _encode(self, obj, context, path):
if len(obj) > self.sizeof():
raise SizeofError("Input ({}) exceeds target size ({})".format(
len(obj), self.sizeof()))
return obj + self.pattern * (self.sizeof() - len(obj))
class GsmStringAdapter(Adapter):
"""Convert GSM 03.38 encoded bytes to a string."""
def __init__(self, subcon, codec='gsm03.38', err='strict'):
super().__init__(subcon)
self.codec = codec
self.err = err
def _decode(self, obj, context, path):
return obj.decode(self.codec)
def _encode(self, obj, context, path):
return obj.encode(self.codec, self.err)
def filter_dict(d, exclude_prefix='_'):
"""filter the input dict to ensure no keys starting with 'exclude_prefix' remain."""
if not isinstance(d, dict):
return d
res = {}
for (key, value) in d.items():
if key.startswith(exclude_prefix):
continue
if type(value) is dict:
res[key] = filter_dict(value)
else:
res[key] = value
return res
def normalize_construct(c):
"""Convert a construct specific type to a related base type, mostly useful
so we can serialize it."""
# we need to include the filter_dict as we otherwise get elements like this
# in the dict: '_io': <_io.BytesIO object at 0x7fdb64e05860> which we cannot json-serialize
c = filter_dict(c)
if isinstance(c, Container) or isinstance(c, dict):
r = {k: normalize_construct(v) for (k, v) in c.items()}
elif isinstance(c, ListContainer):
r = [normalize_construct(x) for x in c]
elif isinstance(c, list):
r = [normalize_construct(x) for x in c]
elif isinstance(c, EnumIntegerString):
r = str(c)
else:
r = c
return r
def parse_construct(c, raw_bin_data: bytes, length: typing.Optional[int] = None, exclude_prefix: str = '_'):
"""Helper function to wrap around normalize_construct() and filter_dict()."""
if not length:
length = len(raw_bin_data)
parsed = c.parse(raw_bin_data, total_len=length)
return normalize_construct(parsed)
# here we collect some shared / common definitions of data types
LV = Prefixed(Int8ub, HexAdapter(GreedyBytes))
# Default value for Reserved for Future Use (RFU) bits/bytes
# See TS 31.101 Sec. "3.4 Coding Conventions"
__RFU_VALUE = 0
# Field that packs Reserved for Future Use (RFU) bit
FlagRFU = Default(Flag, __RFU_VALUE)
# Field that packs Reserved for Future Use (RFU) byte
ByteRFU = Default(Byte, __RFU_VALUE)
# Field that packs all remaining Reserved for Future Use (RFU) bytes
GreedyBytesRFU = Default(GreedyBytes, b'')
def BitsRFU(n=1):
'''
Field that packs Reserved for Future Use (RFU) bit(s)
as defined in TS 31.101 Sec. "3.4 Coding Conventions"
Use this for (currently) unused/reserved bits whose contents
should be initialized automatically but should not be cleared
in the future or when restoring read data (unlike padding).
Parameters:
n (Integer): Number of bits (default: 1)
'''
return Default(BitsInteger(n), __RFU_VALUE)
def BytesRFU(n=1):
'''
Field that packs Reserved for Future Use (RFU) byte(s)
as defined in TS 31.101 Sec. "3.4 Coding Conventions"
Use this for (currently) unused/reserved bytes whose contents
should be initialized automatically but should not be cleared
in the future or when restoring read data (unlike padding).
Parameters:
n (Integer): Number of bytes (default: 1)
'''
return Default(Bytes(n), __RFU_VALUE)
def GsmString(n):
'''
GSM 03.38 encoded byte string of fixed length n.
Encoder appends padding bytes (b'\\xff') to maintain
length. Decoder removes those trailing bytes.
Exceptions are raised for invalid characters
and length excess.
Parameters:
n (Integer): Fixed length of the encoded byte string
'''
return GsmStringAdapter(Rpad(Bytes(n), pattern=b'\xff'), codec='gsm03.38')

Some files were not shown because too many files have changed in this diff Show More