Commit Graph

97 Commits

Author SHA1 Message Date
Philipp Maier
4550574e03 card_key_provider: separate and refactor CSV column encryption
The CardKeyProviderCsv class implements a column decryption scheme
where columns are protected using a transport key. The CSV files
are enrcypted using contrib/csv-encrypt-columns.py.

The current implementation has two main problems:

- The decryption code in CardKeyProviderCsv is not specific to CSV files.
  It could be re-used in other formats, for example to decrypt columns
  (fields) red from a database. So let's split the decryption code in a
  separate class.

- The encryption code in csv-encrypt-columns.py accesses methods and
  properties in CardKeyProviderCsv. Also having the coresponding
  encryption code somewhere out of tree may be confusing. Let's improve
  the design and put encryption and decryption functions in a single
  class. Let's also make sure the encryption/decryption is covered by
  unittests.

Related: SYS#7725
Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab
2025-11-21 17:49:09 +01:00
Harald Welte
fb20b7bc58 contrib: Add a small command line script to generate StoreMetadataRequest
It's occasionally useful to be able to manually generate a
SGP.22 StoreMetadataRequest (tag BF25), so let's add a small utility
program doing exactly that.

Change-Id: I56ebd040f09dcd167b0b22148c2f1af56240b3b5
2025-11-21 11:41:29 +00:00
Oliver Smith
e5f39fbd34 Pass pylint 3.3.4 from debian trixie
************* Module osmo-smdpp
osmo-smdpp.py:657:72: E0606: Possibly using variable 'iccid_str' before assignment (possibly-used-before-assignment)

=> False-positive: code paths that don't set iccid_str raise an error, so
   this shouldn't be a problem.

************* Module pySim-smpp2sim
pySim-smpp2sim.py:427:4: E1101: Module 'twisted.internet.reactor' has no 'run' member (no-member)

=> False-positive: pylint doesn't recognize dynamically set attributes.

************* Module es9p_client
contrib/es9p_client.py:126:11: E0606: Possibly using variable 'opts' before assignment (possibly-used-before-assignment)

=> Real bug, should be "self.opts".

Related: https://stackoverflow.com/a/18712867
Change-Id: Id042ba0944b58d98d27e1222ac373c7206158a91
2025-10-02 09:06:44 +02:00
Kian-Meng Ang
4ee99c18cd Fix typos
Found via `codespell -S tests -L ist,adn,ciph,ue,ot,readd,te,oce,tye`

Change-Id: I00a72e4f479dcef88f7d1058ce53edd0129d336a
2025-09-24 17:59:17 +00:00
Oliver Smith
076fec267a requirements: set cmd2>=2.6.2,<3.0
Remove the previous workaround that set cmd2==2.4.3 in jenkins.sh. The
bug this worked around has been fixed in 2.6.2.

3.0 will break unless we use some new additional decorator.

Related: OS#6776
Change-Id: I4ba65ed486247c5670313b75f43a242d264df14b
2025-08-27 14:31:13 +02:00
Eric Wild
ccefc98160 smdpp: add proper tls support, cert generation FOR TESTING
If TLS is enabled (default) it will automagically generate missing pem files + dh params.

A faithful reproduction of the certs found in SGP.26_v1.5_Certificates_18_07_2024.zip available at
https://www.gsma.com/solutions-and-impact/technologies/esim/gsma_resources/sgp-26-test-certificate-definition-v1-5/
can be generated by running contrib/generate_certs.py. This allows adjusting the expiry dates, CA flag,
and other parameters FOR TESTING. Certs can be used by the smdpp by running
$ python -u osmo-smdpp.py -c generated

Change-Id: I84b2666422b8ff565620f3827ef4d4d7635a21be
2025-06-25 10:22:42 +02:00
Vadim Yanitskiy
004b06eab1 jenkins.sh: workaround for 'usage: build.py' in docs
Recent versions of cmd2 have changed how the 'prog' attribute is
automatically set for ArgumentParser instances.  As a result, we
are now seeing an unexpected 'build.py' artifact appearing in
the generated documentation.

Let's use an older release of cmd2, which retains the old expected
behavior.  Use it specifically for building documentation.

Change-Id: Ifbad35adc5e9d3141acfd024d7dee2a25f1cb62e
Related: https://github.com/python-cmd2/cmd2/issues/1414
Related: OS#6776
2025-05-01 02:58:56 +07:00
Philipp Maier
d5da431fd4 saip-tool: add commandline option to edit mandatory services list
Change-Id: I120b98d4b0942c26674bc1365c5711101ec95235
2025-04-16 10:35:15 +02:00
Philipp Maier
1dea0f39dc saip-tool: add features to add, remove and inspect application PEs
The PE-Application object is used to provision JAVA-card applications
into an eUICC during profile installation. Let's extend the SAIP-tool
so that we are able to add, remove and inspect applications.

Change-Id: I41db96f2f0ccc29c1725a92215ce6b17d87b76ce
2025-04-14 11:01:24 +00:00
Philipp Maier
40e795a825 saip-tool: add ProfileElement class for application PE
The application profile element has no ProfileElement class yet, so
let's create a ProfileElementApplication class and move the existing
extract-apps code into a method of ProfileElementApplication.

Change-Id: Iaa43036d388fbf1714c53cab1fc21092c4667a21
2025-03-31 12:27:24 +02:00
Philipp Maier
dc2b9574c9 saip-tool: allow removing of profile elements by type
At the moment it is only possible to remove profile elements by their identification
number. However, there may be cases where we want to remove all profile elements of
a certain type at once (e.g. when removing all applications).

Change-Id: I92f9f9d5b4382242963f1b3ded814a0d013c4808
2025-03-28 14:35:40 +01:00
Philipp Maier
2b3b2c2a3b saip-tool: add option to extact profile elements to file
In some cases it may be helpful to extract a single profile element
from the sequence to a dedicated file.

Change-Id: I77a80bfaf8970660a84fa61f7e08f404ffc4c2da
2025-03-28 14:34:55 +01:00
Philipp Maier
02a7a2139f saip-tool: add function to write PE sequence
To prevent code duplication and to make the implementation simpler,
let's add a function that takes care of writing the PE sequnece
to an output file.

Change-Id: I38733422270f5b9c18187b7f247b84bf21f9121b
2025-03-28 13:25:30 +00:00
Harald Welte
708a45bcee es2p_client: Print the activation code after confirmOrder success
Change-Id: I92608ff0cdc35b184edff0c656221644ba36f257
2024-11-25 20:29:59 +01:00
Harald Welte
1be2e9b713 contrib/suci-keytool.py: Convenience tool for SUCI key generation
This adds a small utility program that can be used for generating
keys used for SUCI in 5G SA networks, as well as for dumping them
in a format that's compatible with what is needed on the USIM.

Change-Id: I9e92bbba7f700e160ea9c58da5f23fa4c31d40c6
2024-11-25 20:29:59 +01:00
Harald Welte
73c76e02ce contrib/esim-qrcode.py: Small command line tool to encode eSIM QR codes
Change-Id: I7983de79937124cc258efd459c51f812f5fa79cb
2024-11-25 20:29:59 +01:00
Philipp Maier
7c62fc5ec4 jenkins: build docs in virtualenv as well
The build system uses a virtual environment, in which it installs
pysim and its dependencies. This is done for the integration tests,
but not when building the sphinx documentation. However, the
documentation build process also invokes pysim code to generate
documentation from the docstrings. This means we need pysim with
all its dependencies for the doc building as well.

Change-Id: I6381eeef7fa19873ca0cc330a0ab43b7ef5096e4
Related: SYS#7094
2024-10-11 15:59:58 +02:00
Oliver Smith
7b95fac022 contrib/jenkins: add SKIP_CLEAN_WORKSPACE
In order to run this script from pyosmocom's contrib/jenkins.sh script,
we want to skip the clean workspace step. Add an environment variable to
do that.

Related: OS#6570
Change-Id: Ic8dc9b85da17719195f7374d37eccb4dedba6ce8
2024-09-23 08:38:55 +02:00
Philipp Maier
f87a00c04f Add testsuite for pySim-shell with real cards
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.

Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
2024-09-20 17:53:27 +02:00
Harald Welte
a3962b2076 Migrate over to using pyosmocom
We're creating a 'pyosmocom' pypi module which contains a number of core
Osmocom libraries / interfaces that are not specific to SIM card stuff
contained here.

The main modules moved in this initial step are pySim.tlv, pySim.utils
and pySim.construct. utils is split, not all of the contents is
unrelated to SIM Cards.  The other two are moved completely.

Change-Id: I4b63e45bcb0c9ba2424dacf85e0222aee735f411
2024-09-03 21:57:47 +02:00
Harald Welte
a437d11135 contrib/jenkins.sh: Install dependencies before calling pylint
This is the only way we can make sure pylint has all required
information about imports from packages we depend upon.

Change-Id: I29582aa3d7f9ace9ce832d5b907420aaf14881fb
2024-09-03 21:56:19 +02:00
Philipp Maier
4d99c2b204 tests: move pySim-prog test and its data into a sub directory
We currently have the shell script that performs the test in the
tests directory and the related data in pysim-testdata directory.
This is confusing, let's have evrything in a dedicated sub directory

Change-Id: Ic995a7f600d164fc0be3c2eb8255dbe043429bea
Related: OS#6531
2024-09-02 17:07:21 +02:00
Philipp Maier
eb4ca1189c tests: move pySim-trace test and its data into a sub directory
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.

Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
2024-09-02 17:07:21 +02:00
Harald Welte
8ac2647004 contrib: script to generate "update" commands from diff of fsdumps
Change-Id: I08897cd353093575f98c68580afbc68b6f2f878f
2024-08-31 13:12:05 +00:00
Philipp Maier
e0241037e7 tests: move unittests into a sub directory
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.

Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
2024-08-30 05:25:20 +00:00
Harald Welte
a5e2a8dbfd contrib/saip-tool: Add 'tree' command to display filesystem tree of profile
Change-Id: I5cda7ef814648543c63938ac6a4fb9dba79379ff
2024-08-23 06:51:07 +00:00
Harald Welte
c6f8457ff1 pySim.esim.saip: maintain a parsed fileystem hierarchy
With this change, the ProfileElementSequence object will maintain a
representation of the filesystem hierarchy of the eSIM profile.  Every
file that is added by a ProfileElement will add a FsNode into that tree,
and each FsNode will point to the File object for the respective file.

This allows us to find files by their path, as well as add files by
path.

Change-Id: I2caadc24b1087855f23f3c57cdf8dabbf81757c0
2024-08-23 06:51:07 +00:00
Vadim Yanitskiy
5e2b93eb55 jenkins: use osmo-clean-workspace.sh before and after build
Related: osmo-ci.git I2409b2928b4d7ebbd6c005097d4ad7337307dd93
Change-Id: I5ebebfa27e4b0c7b2fb3aa60618a82c1bfdaa19a
Fixes: OS#6546
2024-08-21 19:04:04 +00:00
Harald Welte
01ddec2fdc contrib/saip-tool: Add command-line arguments to configure log level
Change-Id: I4257d7b76193cdaad8c8571ff49f29067e8ab8c8
2024-08-18 19:38:43 +02:00
Harald Welte
1f477495ec saip-tool: Set default log level to INFO (instead of DEBUG)
most users don't want to debug the program.

Change-Id: I54ae558cf8d87bf64cc75431cc4edcc694fa9084
2024-08-16 17:46:41 +02:00
Harald Welte
c60944a7de saip-tool: Fix TAR display for implicit TAR
Until Change-Id Ifba1048e3000829d54769b0420f5134e2f9b04e1 the TAR
output was working for implicit tar.  With said commit we fixed it
for explicit tar but broke implicit tar.

With this commit it works for both implicit and explicit TAR.

Change-Id: I76133b0e02996a138257f3fba5ceb0d2fc6fad80
2024-07-26 08:38:17 +02:00
Harald Welte
3ad3da8995 contrib/es9p_client: Add support for reporting notifications to SM-DP+
The ES9+ interface is not only used for downloading eSIM profiles, but
it is also used to report back the installation result as well as
profile management operations like enable/disable/delete.

Change-Id: Iefba7fa0471b34eae30700ed43531a515af0eb93
2024-07-17 18:05:57 +02:00
Harald Welte
9d0c2947f1 es9p_client: Move code into a class; do common steps in constructor
This is in preparation of supporting more than just 'download'

Change-Id: I5a165efcb97d9264369a9c6571cd92022cbcdfb0
2024-07-17 15:22:09 +02:00
Harald Welte
23dd13542e saip-tool: Fix output of TAR values in "print" subcommand
Change-Id: Ifba1048e3000829d54769b0420f5134e2f9b04e1
2024-07-16 15:06:57 +00:00
Harald Welte
76b3488829 saip-tool: Also dump RFM information in "info" command
example output:

Number of RFM instances: 2
RFM instanceAID: d276000005aa060200000000b00000 (-> TAR: b00000)
        MSL: 0x16
RFM instanceAID: d276000005aa060200000000b00001 (-> TAR: b00001)
        MSL: 0x16
        ADF AID: a0000000871002ff33ffff8901010100

Change-Id: I534267c7420fc5bd96eaded6078e986161729073
2024-07-10 06:51:23 +00:00
Harald Welte
37320da4ab saip-tool: Dump information about security domains from "info" command
output looks like this:

Number of security domains: 1
Security domain Instance AID: a000000151000000
        KVN=0x01, KID=0x01, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]
        KVN=0x01, KID=0x02, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]
        KVN=0x01, KID=0x03, [SdKeyComp(type=aes, mac_len=8, data=00000000000000000000000000000000)]

Change-Id: Ia25f5ca6d7e888f7032301dd2561d066a3870010
2024-07-10 06:51:23 +00:00
Harald Welte
b6532b56d2 saip-tool: Add 'extract-apps' to dump all applications from eSIM profile
This new action can be used to dump all java applications as either raw
IJC file or converted to CAP format (the usual format generated by
JavaCard toolchains).

Change-Id: I51cffa5ba3ddbea491341d678ec9249d7cf470a5
2024-06-11 08:45:27 +02:00
Harald Welte
3d70f659f3 saip-tool: Add new 'info' action to print general information
It will print something like this:

SAIP Profile Version: 2.1
Profile Type: 'GSMA Generic eUICC Test Profile'
ICCID: 8949449999999990023f
Mandatory Services: usim, isim, csim, javacard, usim-test-algorithm

NAAs: mf[1], usim[1], csim[1], isim[1]
NAA mf
NAA usim (a0000000871002ff49ff0589)
        IMSI: 001010123456063
NAA csim
NAA isim (a0000000871004ff49ff0589)

Number of applications: 0

Change-Id: I107d457c3313a766229b569453c18a8d69134bec
2024-06-10 13:39:40 +02:00
Harald Welte
f36e9fd39f es9p_client: Use a plausible TAC (copy from lpac)
Some SM-DP+ (notably Idemia) fail if the TAC is not valid.

Change-Id: I48890c4a56147410d0cd5c4e47647b8eb5ad9998
2024-06-10 13:39:40 +02:00
Harald Welte
36276e7b2a contrib/jenkins.sh: Execute pylint also on all contrib python scripts
This way we get linting coverage for sim-rest-{server,client}, eidtool,
unber and others.

Change-Id: I2d6271d493d0f6765e6a184f8ae32f8325317be2
2024-06-10 11:39:28 +00:00
Harald Welte
5341bf902f unber.py: work-around pylint reporting (possibly-used-before-assignment)
contrib/unber.py:39:22: E0606: Possibly using variable 'content' before assignment (possibly-used-before-assignment)

Change-Id: I725cd5e05e3121c853669eb4bbfe5ba51b79eb75
2024-06-10 11:39:28 +00:00
Harald Welte
3b4a673de4 add contrib/saip-tool.py
This is a tool to work with eSIM profiles in SAIP format.  It allows
to dump the contents, run constraint checkers as well as splitting
of the PE-Sequence into the individual PEs.

Change-Id: I396bcd594e0628dfc26bd90233317a77e2f91b20
2024-06-10 08:59:39 +02:00
Harald Welte
a5634c248b jenkins.sh: Include es9p_client in pylint
Change-Id: I06f6773b8b5d3dfa588617d5af81c2fddb474a3d
2024-06-09 22:49:33 +02:00
Harald Welte
50d9e2a6d8 esim.es9p: Suppress sending requestHeader on ES9+
SGP.22 states that ES9+ should not include a requestHeader

Change-Id: Ic9aa874a82241d7b26e2bcb0423961173e103020
2024-06-09 12:18:16 +02:00
Harald Welte
888c6e5647 add contrib/es9p_client: Perform ES9+ client functions like LPA+eUICC
This tool can be used to test the SM-DP+. It implements the full dance
of all HTTPs API operations to get to the downloadProfile, and will
decrypt the BPP to the UPP, which is then subsequently stored as file on
disk.

Needless to say, this will only work if you have an eUICC certificate +
private key that is compatible with the CI of your SM-DP+.

Change-Id: Idf8881e82f9835f5221c58b78ced9937cf5fb520
2024-06-09 12:18:16 +02:00
Harald Welte
1aaf978d9f CardKeyProvider: Implement support for column-based transport key encryption
It's generally a bad idea to keep [card specific] key material lying
around unencrypted in CSV files.  The industry standard solution in the
GSMA is a so-called "transport key", which encrypts the key material.

Let's introduce support for this in the CardKeyProvider (and
specifically, the CardKeyProviderCSV) and allow the user to specify
transport key material as command line options to pySim-shell.

Different transport keys can be used for different key materials, so
allow specification of keys on a CSV-column base.

The higher-level goal is to allow the CSV file not only to store
the ADM keys (like now), but also global platform key material for
establishing SCP towards various security domains in a given card.

Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff
2024-06-04 23:18:37 +02:00
Oliver Smith
6add18ea08 contrib/sim-rest-client: don't crash without args
When running without an argument, let argparse print a nice usage error:

  $ ./sim-rest-client.py
  usage: sim-rest-client.py [-h] [-H HOST] [-p PORT] [-v] [-n SLOT_NR] {auth,info} ...
  sim-rest-client.py: error: the following arguments are required: {auth,info}

Instead of:

  $ ./sim-rest-client.py
  Traceback (most recent call last):
    File "/usr/share/pysim/contrib/./sim-rest-client.py", line 185, in <module>
      main(sys.argv)
    File "/usr/share/pysim/contrib/./sim-rest-client.py", line 181, in main
      args.func(args)
      ^^^^^^^^^
  AttributeError: 'Namespace' object has no attribute 'func'

Change-Id: I92998d9b94dcfb9dcfc3da161fe5d8f45f242b78
2024-05-24 20:23:35 +00:00
Vadim Yanitskiy
a412c436b4 contrib/jenkins.sh: add 'distcheck' job to check package integrity
The idea of this new job is to catch package integrity problems,
like the missing entries in setup.py/packages[] or missing deps.

Change-Id: Ic72d58494e8fd0cab8d66ce60f7b70593b770872
Related: osmo-ci.git I9d4d9e9de2b16a4b745791f3c9c93507f43bfa6d
2024-03-21 18:39:00 +00:00
Harald Welte
95873a964e Introduce code for ES2+ API client functionality
Change-Id: Id652bb4c2df8893a824b8bb44beeafdfbb91de3f
2024-02-09 21:41:51 +01:00
Harald Welte
bc8e2e1664 contrib/jenkins.sh: include tests/*.py in pylint
Change-Id: I9c8113acf5341b198d91040710b6b10cb2b6ef38
2024-02-04 21:27:00 +01:00