130 Commits

Author SHA1 Message Date
Philipp Maier
eb4ca1189c tests: move pySim-trace test and its data into a sub directory
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.

Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
2024-09-02 17:07:21 +02:00
Philipp Maier
e0241037e7 tests: move unittests into a sub directory
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.

Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
2024-08-30 05:25:20 +00:00
Harald Welte
6d2e385acf pySim.esim.saip: Add OID comparison functions
Change-Id: Iab642e0cc6597f667ea126827ea888652f0f2689
2024-07-27 10:30:26 +02:00
Harald Welte
87b4f99a90 pySim.apdu: Get rid of HexAdapter
In the past, we always wrapped a HexAdapter around bytes-like data in
order to make sure it's printed as hex-digits.  However, now that we are
doing JSON output it's much easier to let the pySim.utils.JsonEncoder
take care of this in a generic way.

We should do a similar migration all over pySim (pySim-shell,
filesystem, etc.) - but for now only do it in the low-hanging fruit of
pySim-trace aka pySim.apdu

Change-Id: I0cde40b2db08b4db9c10c1ece9ca6fdd42aa9154
2024-07-26 06:20:46 +00:00
Harald Welte
e18586ddf0 pySim.globalplatform: Add 'http' submodule for GP Amd B RAM over HTTPS
This implements the first parts of the "GlobalPlatform Remote
Application Management over HTTP Card Specification v2.3 - Amendment B,
Versoin 1.2".  Specifically, this patch covers the TLV definitions for
the OTA message used for HTTPS session triggering.

This also adds some more unit test coverage to pySim.cat, based on
real-world data that was captured nested inside the HTTPS Administration
session triggering parameters.

Change-Id: Ia7d7bd6df41bdf1249011bad9a9a38b7669edc54
2024-07-17 18:05:57 +02:00
Harald Welte
1c2ec93164 pySim.tlv: Add COMPACT_TLV_IE TLV variant
the COMPACT-TLV variant is a TLV variant that ISO7816 uses for encoding
tag and length into a single octet. This is used (for example) in ATR
historical bytes.

Let's add support for this to our pySim TLV encoder/decoder.

Change-Id: I9e98d150b97317ae0c6be2366bdaaeaeddf8031c
2024-07-10 18:10:39 +02:00
Harald Welte
03aebf5b43 pySim.esim.saip: ProfileElement{Header,End} classes
Change-Id: I88e18c1ee4907eeac3ae5d04d7bc30d6765f91fa
2024-07-10 06:51:23 +00:00
Harald Welte
3b7e2ae2c1 pySim.saip: Add ProfileElementRFM class
Change-Id: I547e02c12345932deafa4b914fcaeaa183b69798
2024-07-10 06:51:23 +00:00
Harald Welte
2668eb6148 pySim.esim.saip: Add ProfileElementOpt{USIM,ISIM} classes
Change-Id: Iebff2e767baa19f272eeddc62d7d5b3a8f665db5
2024-07-10 06:51:23 +00:00
Harald Welte
992e60902a tests: Add ProfileElementSD and ProfileElementSSD to test_constructor_encode
Change-Id: Idc6f37b487dfa8a69ac7a50a537cfc317113d501
2024-07-10 06:51:23 +00:00
Harald Welte
292191d67a pySim.esim.saip: Add ProfileElementAKA constructor + methods
This helps us to construct an akaParameter PE from scratch.

Change-Id: I4cc42c98bf82aec085ab7f48aea4ff7efa0eae9e
2024-07-10 06:51:23 +00:00
Harald Welte
200bf6eb8b pySim.esim.saip: Meaningful defaults in PE Constructor + test
Let's make sure the constructor of ProfileElement subclasses set
meaningful defaults to the self.decoded member, so that the to_der()
method can actually encode it.   This is required when constructing
a profile from scratch, as opposed to loading an existing one from DER.

Also, add a test to verify that the encoder passes without exception;
doesn't test the generated binary data.

Change-Id: I401bca16e58461333733877ec79102a5ae7fe410
2024-07-10 06:51:23 +00:00
Harald Welte
1aa77c5d74 tests/ota_test.py: Allow stand-alone execution
Let's add a __main__ section to allow stand-alone execution via
	python3 ./tests/test_ota.py

Change-Id: Ic3940ac23c7ddc1013e21f41eae6076a11dfd4f4
2024-06-10 11:39:28 +00:00
Harald Welte
32401a54e6 pySim.ota.OtaDialectSms: Implement command decoding
So far we only implemented command encoding and response decoding.
Let's also add command decoding, which is useful for example when
decoding protocol traces.

Change-Id: Id666cea8a91a854209f3c19c1f09b512bb493c85
2024-06-10 11:39:28 +00:00
Vadim Yanitskiy
decb468092 tests: assertEquals() is deprecated, use assertEqual()
This fixes deprecation warnings printed by Python 3.11.7.

Change-Id: I1de93b0fee9e8439f7da8a3b9fd2a6974973fb4f
2024-03-02 01:36:19 +07:00
Harald Welte
318faef583 saip.personalization: include encode/decode of value; add validation method
Change-Id: Ia9fa39c25817448afb191061acd4be894300eeef
2024-02-21 09:22:40 +01:00
Harald Welte
efdf423a7f utils: Add function to verify Luhn check digits and to sanitize ICCIDs
Change-Id: I7812420cf97984dd834fca6a38c5e5ae113243cb
2024-02-18 22:30:08 +01:00
Harald Welte
e8439d9639 pylint: sms.py
pySim/sms.py:23:0: W0404: Reimport 'Flag' (imported line 23) (reimported)
pySim/sms.py:54:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:60:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:120:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:132:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:137:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:141:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:188:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:192:8: W0612: Unused variable 'flags' (unused-variable)
pySim/sms.py:209:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:228:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:236:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:279:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:306:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:311:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:319:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:339:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:347:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:373:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-nam

Change-Id: I8082a01443ef568eebda696239572f0af7b56f1b
2024-02-05 09:50:47 +01:00
Harald Welte
7156a40187 construct: Add StripTrailerAdapter
In smart cards, we every so often encounter data types that contain a
bit-mask whose length depends on whether or not there are any of the
least-significant bits are set.  So far we worked around this with
some kind of Struct('byte1', 'byte2'/COptional, 'byte3'/COptional)
approach.

Let's do thisin a generic way using the new StripTrailerAdapter.

Change-Id: I659aa7247c57c680895b0bf8412f9e477fc3587d
2024-02-05 01:39:39 +01:00
Harald Welte
cd8e16fdfe global_platform: KCV support for PUT KEY
GlobalPlatform requires the use of the KCV for DES + AES keys. Let's
implement that.

(11.8.2.3.3: "For all key types described in section B.6, the Key Check
Value shall be present.")

Change-Id: Ief168a66dee58b56f4126db12829b3a98906c8db
2024-02-04 21:27:00 +01:00
Harald Welte
57f73f8de7 make our tests pass pylint
Change-Id: If3a9f178c3f915123178efe00269fce74f6e585d
2024-02-04 21:27:00 +01:00
Harald Welte
af8826a02b Implement Global Platform SCP03
This adds an implementation of the GlobalPlatform SCP03 protocol. It has
been tested in S8 mode for C-MAC, C-ENC, R-MAC and R-ENC with AES using
128, 192 and 256 bit key lengh.  Test vectors generated while talking to
a sysmoEUICC1-C2T are included as unit tests.

Change-Id: Ibc35af5474923aed2e3bcb29c8d713b4127a160d
2024-02-04 17:56:59 +01:00
Harald Welte
13a1723c2e rename global_platform.scp02 to global_platform.scp
This is in preparation of extending it to cover SCP03 in a follow-up
patch.

Change-Id: Idc0afac6e95f89ddaf277a89f9c95607e70a471c
2024-02-04 17:56:59 +01:00
Harald Welte
41a7379a4f Introduce GlobalPlatform SCP02 implementation
This implementation of GlobalPlatform SCP02 currently only supports
C-MAC and C-ENC, but no R-MAC or R-ENC yet.

The patch also introduces the notion of having a SCP instance associated
with a SimCardCommands instance.  It also adds the establish_scp0w and
release_scp shell commands to all GlobalPlatform Security Domains.

Change-Id: I56020382b9dfe8ba0f7c1c9f71eb1a9746bc5a27
2024-02-04 17:42:30 +01:00
Harald Welte
93bdf00967 pySim.esim: Add class for parsing/encoding eSIM activation codes
Change-Id: I2256722c04b56e8d9c16a65e3cd94f6a46f4ed85
2024-01-30 21:33:41 +01:00
Harald Welte
8a39d00cc3 osmo-smdpp: Support multiple different profiles
Let's simply use the matchingId for filesystem lookup of the UPP file.

This way we can have any number of profiles by simply creating the
respeective files.

Change-Id: I0bc3a14b9fdfcc6322917dd0c69d8295de486950
2024-01-30 21:33:41 +01:00
Harald Welte
4f3d11b378 euicc: Implement EID checksum verification + computation
Change-Id: I2cb342783137ee7e4b1be3b14e9c3747316f1995
2024-01-16 19:04:19 +01:00
Harald Welte
1f7a9bd5b4 TLV: Add DGI encoding of "GP Scripting Language Annex B"
The DGI encoding is specified in Annex B of the
"GlobalPlatform Systems Scripting Language Specification v1.1.0"

which is an "archived" specification that is no longer published
by GlobalPlatform, despite it being referenced from the GlobalPlatform
Card Specification v2.3, which is the basis of the GSMA eSIM
specifications.

For some reason it was the belief of the specification authors that
yet another format of TLV encoding is needed, in addition to the BER-TLV
and COMPREHENSION-TLV used by the very same specifications.

The encoding of the tag is not really specified anywhere, but I've only
seen 16-bit examples.  The encoding of the length is specified and
implemented here accordingly.

Change-Id: Ie29ab7eb39f3165f3d695fcc1f02051338095697
2024-01-14 17:42:01 +01:00
Harald Welte
f2bcb44ccc pySim.saip.*: Support for parsing / operating on eSIM profiles
This commit introduces the capability to parse and encode
SimAlliance/TCA "Interoperable Profiles" and apply personalization
operations on them.

Change-Id: I71c252a214a634e1bd6f73472107efe2688ee6d2
2024-01-09 21:37:12 +00:00
Harald Welte
5bbb144a31 Initial proof-of-concept SM-DP+ for GSMA consumer eSIM RSP
This commit introduces

* the osmo-smdpp.py program implementing the main procedures and the
  HTTP/REST based ES9+
* python modules for ES8+ and non-volatile RSP Session State storage
* the ASN.1 source files required to parse/encode RSP
* 3GPP test certificates from SGP.26
* an unsigned profile package (UPP) of a SAIP v2.3 TS48 test profile

As I couldn't get the 'Klein' tls support to work, the SM-DP+ code
currently does not support HTTPS/TLS but plan HTTP, so you either have
to modify your LPA to use HTTP instead of HTTPS, or put a TLS proxy in
front.

I have successfully installed an eSIM profile on a test eUICC that
contains certificate/key data within the test CI defined in GSMA SGP.26

Change-Id: I6232847432dc6920cd2bd08c84d7099c29ca1c11
2024-01-09 21:37:12 +00:00
Harald Welte
242197b53d Add pySim.esim.bsp module implementing BSP (BPP Protection Protocol)
This is the protocol used for the ES8+ interface between SM-DP+ and the
eUICC in the GSMA eSIM system.

Change-Id: Ic461936f2e68e1e6f7faab33d06acf3063e261e7
2024-01-07 10:22:04 +01:00
Harald Welte
e13403b206 ts_31_102: Start to use construct for EF.SUCI_Calc_Info
We cannot fully switch to construct for all of it easily due to
the priority value and the ordering/sorting by priority implemented
in the hand-coded version.  But we can at least migrate the
encode/decode of the hnet_pubkey_list via construct.

Change-Id: I4ad5ea57bab37c2dc218e7752d538aa4cdc36ee3
2024-01-04 20:13:06 +01:00
Harald Welte
ce01f48b00 test_files: Test decoder also with ff-padded input
It's customary in the SIM card universe to right-pad data with ff bytes.
So far we only test decoders without such padding, which is unrealistic.
Let's also tests the decoders with extra 'ff' padding present.

For some files this doesn't make sense, so we add a _test_no_pad class
attribute that can be spcified to prevent this new "test with ff-padding"
from being executed for the test data of the class.

Change-Id: I7f5cbb4a6f91040fe9adef9da0a1f30f9f156dae
2023-12-29 18:51:25 +01:00
Harald Welte
bcd261583c tests_files.py: Reduce code duplication
Change-Id: Ib84a0ae35262a19fce3e688afe8e1678a4c59eba
2023-12-29 18:51:25 +01:00
Harald Welte
f6fceb8684 Implement convoluted encoding of UCS-2 as per TS 102 221 Annex A
TS 102 221 Annex A defines three variants of encoding UCS-2 characters
into byte streams in files on UICC cards: One rather simplistic one, and
two variants for optimizing memory utilization on the card.

Let's impelement a construct "Ucs2Adapter" class for this.

Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
2023-12-29 18:51:21 +01:00
Harald Welte
45626271cf global_platform: Add TLV test data for Key Information Data
Change-Id: Ib7b73cb28abea98986a66264a0779263873d7fb2
2023-12-27 15:15:58 +00:00
Harald Welte
2a36c1b921 data-driven TLV unit data test support
While we do have the _test_de_encode data driven tests for file
definitions, we don't yet have something similar for derived classes of
BER_TLV_IE. This means that TLVs used outside of the filesystem context
(for example, decoding the SELECT/STATUS response, but also eUICC and
other stuff) do not yet have test coverage.

This commit just adds the related test code, but no test data yet.

Related: OS#6317
Change-Id: Ied85f292bb57fde11dc188be84e3384dc3ff1601
2023-12-27 15:15:17 +00:00
Harald Welte
a4b9bdf238 pySim-trace_test.sh: Force termcolor to suppress color generation
on some systems, the output would otherwise contain colored status
words, which in turn mean the test otuput no longer matches the expected
output.

Change-Id: Icb700f6e85a285748e00367a398975aa5e75dec5
2023-12-23 10:38:21 +01:00
Harald Welte
542dbf6771 fix encode/decode of xPLMNwAcT
There are some pretty intricate rules about how GSM and E-UTRAN are
encoded, let's make sure we fully  support both as per 3GPP TS 31.102
Release 17.  As part of this, switch to a sorted list of access technologies,
in order to have a defined order.  This makes comparing in unit tests
much easier.  However, it also means that we need to sort the set
when printing the list of AcT in pySim-read to generate deterministic
output.

Change-Id: I398ac2a2527bd11e9c652e49fa46d6ca8d334b88
2023-12-23 08:20:42 +01:00
Harald Welte
e45168ef29 test/test_files: set maxDiff attribute
Without this the diff between expected and actual output is truncated
and one instead reads the following output:

	Diff is 844 characters long. Set self.maxDiff to None to see it.

We actually want to see the full diff to see what's not matching.

Change-Id: I6e89705061454191b6db1255de7fe549ad720800
2023-12-22 09:13:10 +01:00
Harald Welte
2822dca9ec tests: use case-insensitive compare of hex strings
Change-Id: I080f6e173fec40c27dd3ebbf252eaddf5a0e15ba
2023-12-22 09:13:10 +01:00
Philipp Maier
4af63dc760 transport: print reader device/number on init
When we initialize the reader, we currently tell only which type of
interface we are using, but we do not print the reader number or the
device path.

Let's extend the messages so that the path is printed. To prevent
problems with integration-tests, let's also add an environment variable
that we can use to detect when pySim runs inside a integration-test.

Related: OS#6210
Change-Id: Ibe296d51885b1ef5f9c9ecaf1d28da52014dcc4b
2023-10-26 15:17:07 +00:00
Philipp Maier
63e8a18883 pySim-prog_test: fix typo
Related: OS#6094
Change-Id: I6432ee3ee948fea697067fb3857cb9b83b1f8422
2023-08-01 16:10:14 +02:00
Philipp Maier
a380e4efbe pySim-trace_test: verify output of pySim-trace.py
At the moment we only verify that no exceptions occurred but the output
is not yet verfied.

Related: OS#6094
Change-Id: I3aaa779b5bd8f30936c284a80dbdcb2b0e06985c
2023-08-01 16:10:14 +02:00
Philipp Maier
7124ad1031 pySim-trace_test: fix shebang line
Related: OS#6094
Change-Id: Ib2d3a4659f5db9772ddcd9a4ae73c04fec1070fc
2023-08-01 16:01:47 +02:00
Philipp Maier
1cdcbe4f57 pysim-test: rename pysim-test.sh to pySim-prog_test.sh
We now have pySim-shell and pySim-trace. Let's give pysim-test.sh a more
distinctive name so that it is clear to which program it refers.

Related: OS#6094
Change-Id: I438f63f9580ebd3c7cc78cc5dab13c9937ac6e3a
2023-07-29 08:56:07 +00:00
Philipp Maier
ec9cdb73e7 tests: add test script for pySim-trace
pySim-trace has no test coverage yet. Let's use a script to run a
GSAMTAP pcacp through it and check that no exceptions are raised.

Related: OS#6094
Change-Id: Icfabfa7c59968021eef0399991bd05b92467d8d2
2023-07-29 08:56:07 +00:00
Harald Welte
f9e2df1296 cdma_ruim: Fix unit tests and actually enable them
As pySim.cdma_ruim was not imported by test_files.py, the unit tests
were apparently never executed and hence didn't pass.  Let's fix both
of those problems.

Change-Id: Icdf4621eb68d05a4948ae9efeb81a007d48e1bb7
2023-07-12 22:05:14 +02:00
Harald Welte
f8d2e2ba08 split pySim/legacy/{cards,utils} from pySim/{cards,utils}
There are some functions / classes which are only needed by the legacy
tools pySim-{read,prog}, bypassing our modern per-file transcoder
classes.  Let's move this code to the pySim/legacy sub-directory,
rendering pySim.legacy.* module names.

The long-term goal is to get rid of those and have all code use the
modern pySim/filesystem classes for reading/decoding/encoding/writing
any kind of data on cards.

Change-Id: Ia8cf831929730c48f90679a83d69049475cc5077
2023-07-12 22:03:59 +02:00
Harald Welte
721ba9b31f tests: Add new, data-driven OTA tests
Rather than writing one test class with associated method for each
OTA algorithm / test, let's do this in a data-driven way, where new
test cases just have to provide test data, while the code iterates over
it.

Change-Id: I8789a21fa5a4793bdabd468adc9fee3b6e633c25
2023-06-18 10:50:50 +02:00