Commit Graph

96 Commits

Author SHA1 Message Date
Harald Welte
12328c090d pySim.ts_31_102: Add support for EF.EARFCNList
This adds a construct + pyosmocore.tlv based declarative encoder/decoder
for the EF.EARFCNList file used in the context of NB-IoT in later
release USIMs.

Change-Id: I16797ca58c3ad6ebaf588d04fec011a0cbcfcef3
2024-10-25 18:09:19 +02:00
Philipp Maier
82b0f1b39a pySim-shell_test: re-enable test_list_and_rm_notif
The problems with test_list_and_rm_notif (see also change id
I7d0b6a998499d84f0eb4e24592ad43210ac54806) are now resolved, so
we can re-enable the testcase now.

Closes: SYS#7094
Change-Id: I95eb3b9c02a69653797851197e882ea9316805fc
2024-10-11 16:13:07 +02:00
Philipp Maier
7429bc0ca0 tests: sanitize all cards before running tests
Even though our tests are written in a way that they shouldn't interfere
with each other, it may happen that one testrun writes content to a file
that upsets a different testrun. The resulting problems are often
difficult to diagnose.

To minimize the problem, let's add code that can reset the cards to a
defined state. This can be done using pySim-shell's export
feature. We can generate a backup from a known good state and then play
back the backup to reset files that have been changed. Files that didn't
change will not be written thanks to the conserve_write feature of
pySim-shell.

Related: OS#4384
Change-Id: I42eaf61280968518164f2280245136fd30a603ce
2024-10-05 08:44:20 +00:00
Philipp Maier
93c89856c8 utils: move enc_msisdn and dec_msisdn to legacy/utils.py
We now have a construct based encoder/decoder for the record content
of EF.MSISDN. This means that we do not need the functions enc_msisdn
and dec_msisdn in the non-legacy code anymore. We can now move both
to legacy/utils.py.

Related: OS#5714
Change-Id: I19ec8ba14551ec282fc0cc12ae2f6d528bdfc527
2024-09-27 18:17:19 +02:00
Philipp Maier
39552464d8 ts_51_011: replace encoding of EF.MSISDN with construct model
The encoding of EF.MSISDN is currently done with enc_msisdn and
dec_msisdn from utils.py. Let's replace this with a construct
based model, similar to the one we already use with EF.ADN

Related: OS#5714
Change-Id: I647f5c63f7f87902a86c0c5d8e92fdc7f4350a5a
2024-09-27 18:17:19 +02:00
Philipp Maier
efddffe015 filesystem: pass total_len to construct of when encoding file contents
In our construct models we frequently use a context parameter "total_len",
we also pass this parameter to construct when we decode files, but we
do not pass it when we generate files. This is a problem, because when
total_len is used in the construct model, this parameter must be known
also when decoding the file.

Let's make sure that the total_len is properly determined and and passed
to construct (via pyosmocom).

Related: OS#5714
Change-Id: I1b7a51594fbc5d9fe01132c39354a2fa88d53f9b
2024-09-27 18:17:19 +02:00
Philipp Maier
d96d04718e pySim-shell_test: disable test_list_and_rm_notif
The testcase euicc.test_list_and_rm_notif fails due to a problem
with the eUICC. The eUICC reports the following error when a
delete notification attempt is made:

"delete_notification_status": "undefinedError"

Let's temporarily disable this testcase until the problem is resolved.

Change-Id: I7d0b6a998499d84f0eb4e24592ad43210ac54806
2024-09-23 13:52:51 +02:00
Philipp Maier
f87a00c04f Add testsuite for pySim-shell with real cards
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.

Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
2024-09-20 17:53:27 +02:00
Philipp Maier
d7032955c5 pySim-prog_test: add test vectors for sysmoISIM-SJA5
The sysmoISIM-SJA5 has no testvectors yet

Change-Id: Ia6684ab3ee6c85cfe7bc0ab80d34a26e3499907a
2024-09-20 17:18:50 +02:00
Philipp Maier
01a96cd8e4 pySim-prog_test: individual ICCIDs for all cards
Our test cards need to stay recognizable, so it is important that
each card has a unique ICCID. This means we must write an individual
ICCID, when we test writing the ICCID.

Related: OS#4384
Change-Id: I858a35e526e7b4868e901222d587258412779f41
2024-09-20 12:15:34 +00:00
Philipp Maier
dca641aaa2 pySim-prog_test: do not set an ICCID parameter for sysmoISIM-SJA2
The sysmoISIM-SJA2 does not support changing of the ICCID.
pySim-prog will also reject this, so let's remove the ICCID
from the parameter list.

Related: OS#4384
Change-Id: I89571f2bf7c4cec4d621c322a58687b7781b0ed2
2024-09-20 12:15:34 +00:00
Harald Welte
a3962b2076 Migrate over to using pyosmocom
We're creating a 'pyosmocom' pypi module which contains a number of core
Osmocom libraries / interfaces that are not specific to SIM card stuff
contained here.

The main modules moved in this initial step are pySim.tlv, pySim.utils
and pySim.construct. utils is split, not all of the contents is
unrelated to SIM Cards.  The other two are moved completely.

Change-Id: I4b63e45bcb0c9ba2424dacf85e0222aee735f411
2024-09-03 21:57:47 +02:00
Philipp Maier
aa182e9815 pySim-prog_test: supress stderr when probing for cards
When probing for cards, the probing might fail in case the terminal
is empty. This results into lengthy error log output that is not
of interest.

Related: OS#6532
Change-Id: I1d44f9458a05992d79b0152d3affcfeb783cccff
2024-09-03 12:38:30 +02:00
Philipp Maier
4d1f4fde4f pySim-prog_test: tolerate missing .data files
When the test detects a card, but does not find the .data faile for
it, then it fails. This can be a problem in case we want to intentionally
exclude a specific card model.

Related: OS#6532
Change-Id: Iba196ada0076385de7bffcb157a85fda0a6c1852
2024-09-03 12:38:30 +02:00
Philipp Maier
33256ddfed pySim-prog_test: tolerate empty reader slots
The test currently expects all reader slots to be populated. This means
the cards may plugged into a random order, but there may not be any empty
slots in the system at all. This requirement is easy to meet when each
card has its own single-slot USB PCSC-reader, but as soon as multislot
readers are used there may be some empty slots.

Related: OS#6532
Change-Id: I7ba1d1350b6998d65e90408184accdb212556a7c
2024-09-03 12:38:20 +02:00
Philipp Maier
4d99c2b204 tests: move pySim-prog test and its data into a sub directory
We currently have the shell script that performs the test in the
tests directory and the related data in pysim-testdata directory.
This is confusing, let's have evrything in a dedicated sub directory

Change-Id: Ic995a7f600d164fc0be3c2eb8255dbe043429bea
Related: OS#6531
2024-09-02 17:07:21 +02:00
Philipp Maier
eb4ca1189c tests: move pySim-trace test and its data into a sub directory
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.

Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
2024-09-02 17:07:21 +02:00
Philipp Maier
e0241037e7 tests: move unittests into a sub directory
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.

Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
2024-08-30 05:25:20 +00:00
Harald Welte
6d2e385acf pySim.esim.saip: Add OID comparison functions
Change-Id: Iab642e0cc6597f667ea126827ea888652f0f2689
2024-07-27 10:30:26 +02:00
Harald Welte
87b4f99a90 pySim.apdu: Get rid of HexAdapter
In the past, we always wrapped a HexAdapter around bytes-like data in
order to make sure it's printed as hex-digits.  However, now that we are
doing JSON output it's much easier to let the pySim.utils.JsonEncoder
take care of this in a generic way.

We should do a similar migration all over pySim (pySim-shell,
filesystem, etc.) - but for now only do it in the low-hanging fruit of
pySim-trace aka pySim.apdu

Change-Id: I0cde40b2db08b4db9c10c1ece9ca6fdd42aa9154
2024-07-26 06:20:46 +00:00
Harald Welte
e18586ddf0 pySim.globalplatform: Add 'http' submodule for GP Amd B RAM over HTTPS
This implements the first parts of the "GlobalPlatform Remote
Application Management over HTTP Card Specification v2.3 - Amendment B,
Versoin 1.2".  Specifically, this patch covers the TLV definitions for
the OTA message used for HTTPS session triggering.

This also adds some more unit test coverage to pySim.cat, based on
real-world data that was captured nested inside the HTTPS Administration
session triggering parameters.

Change-Id: Ia7d7bd6df41bdf1249011bad9a9a38b7669edc54
2024-07-17 18:05:57 +02:00
Harald Welte
1c2ec93164 pySim.tlv: Add COMPACT_TLV_IE TLV variant
the COMPACT-TLV variant is a TLV variant that ISO7816 uses for encoding
tag and length into a single octet. This is used (for example) in ATR
historical bytes.

Let's add support for this to our pySim TLV encoder/decoder.

Change-Id: I9e98d150b97317ae0c6be2366bdaaeaeddf8031c
2024-07-10 18:10:39 +02:00
Harald Welte
03aebf5b43 pySim.esim.saip: ProfileElement{Header,End} classes
Change-Id: I88e18c1ee4907eeac3ae5d04d7bc30d6765f91fa
2024-07-10 06:51:23 +00:00
Harald Welte
3b7e2ae2c1 pySim.saip: Add ProfileElementRFM class
Change-Id: I547e02c12345932deafa4b914fcaeaa183b69798
2024-07-10 06:51:23 +00:00
Harald Welte
2668eb6148 pySim.esim.saip: Add ProfileElementOpt{USIM,ISIM} classes
Change-Id: Iebff2e767baa19f272eeddc62d7d5b3a8f665db5
2024-07-10 06:51:23 +00:00
Harald Welte
992e60902a tests: Add ProfileElementSD and ProfileElementSSD to test_constructor_encode
Change-Id: Idc6f37b487dfa8a69ac7a50a537cfc317113d501
2024-07-10 06:51:23 +00:00
Harald Welte
292191d67a pySim.esim.saip: Add ProfileElementAKA constructor + methods
This helps us to construct an akaParameter PE from scratch.

Change-Id: I4cc42c98bf82aec085ab7f48aea4ff7efa0eae9e
2024-07-10 06:51:23 +00:00
Harald Welte
200bf6eb8b pySim.esim.saip: Meaningful defaults in PE Constructor + test
Let's make sure the constructor of ProfileElement subclasses set
meaningful defaults to the self.decoded member, so that the to_der()
method can actually encode it.   This is required when constructing
a profile from scratch, as opposed to loading an existing one from DER.

Also, add a test to verify that the encoder passes without exception;
doesn't test the generated binary data.

Change-Id: I401bca16e58461333733877ec79102a5ae7fe410
2024-07-10 06:51:23 +00:00
Harald Welte
1aa77c5d74 tests/ota_test.py: Allow stand-alone execution
Let's add a __main__ section to allow stand-alone execution via
	python3 ./tests/test_ota.py

Change-Id: Ic3940ac23c7ddc1013e21f41eae6076a11dfd4f4
2024-06-10 11:39:28 +00:00
Harald Welte
32401a54e6 pySim.ota.OtaDialectSms: Implement command decoding
So far we only implemented command encoding and response decoding.
Let's also add command decoding, which is useful for example when
decoding protocol traces.

Change-Id: Id666cea8a91a854209f3c19c1f09b512bb493c85
2024-06-10 11:39:28 +00:00
Vadim Yanitskiy
decb468092 tests: assertEquals() is deprecated, use assertEqual()
This fixes deprecation warnings printed by Python 3.11.7.

Change-Id: I1de93b0fee9e8439f7da8a3b9fd2a6974973fb4f
2024-03-02 01:36:19 +07:00
Harald Welte
318faef583 saip.personalization: include encode/decode of value; add validation method
Change-Id: Ia9fa39c25817448afb191061acd4be894300eeef
2024-02-21 09:22:40 +01:00
Harald Welte
efdf423a7f utils: Add function to verify Luhn check digits and to sanitize ICCIDs
Change-Id: I7812420cf97984dd834fca6a38c5e5ae113243cb
2024-02-18 22:30:08 +01:00
Harald Welte
e8439d9639 pylint: sms.py
pySim/sms.py:23:0: W0404: Reimport 'Flag' (imported line 23) (reimported)
pySim/sms.py:54:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:60:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:120:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:132:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:137:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:141:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:188:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:192:8: W0612: Unused variable 'flags' (unused-variable)
pySim/sms.py:209:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:228:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:236:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:279:4: C0103: Method name "fromBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:306:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:311:12: W0107: Unnecessary pass statement (unnecessary-pass)
pySim/sms.py:319:4: C0103: Method name "toBytes" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:339:4: C0103: Method name "fromSmpp" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:347:4: C0103: Method name "fromSmppSubmit" doesn't conform to snake_case naming style (invalid-name)
pySim/sms.py:373:4: C0103: Method name "toSmpp" doesn't conform to snake_case naming style (invalid-nam

Change-Id: I8082a01443ef568eebda696239572f0af7b56f1b
2024-02-05 09:50:47 +01:00
Harald Welte
7156a40187 construct: Add StripTrailerAdapter
In smart cards, we every so often encounter data types that contain a
bit-mask whose length depends on whether or not there are any of the
least-significant bits are set.  So far we worked around this with
some kind of Struct('byte1', 'byte2'/COptional, 'byte3'/COptional)
approach.

Let's do thisin a generic way using the new StripTrailerAdapter.

Change-Id: I659aa7247c57c680895b0bf8412f9e477fc3587d
2024-02-05 01:39:39 +01:00
Harald Welte
cd8e16fdfe global_platform: KCV support for PUT KEY
GlobalPlatform requires the use of the KCV for DES + AES keys. Let's
implement that.

(11.8.2.3.3: "For all key types described in section B.6, the Key Check
Value shall be present.")

Change-Id: Ief168a66dee58b56f4126db12829b3a98906c8db
2024-02-04 21:27:00 +01:00
Harald Welte
57f73f8de7 make our tests pass pylint
Change-Id: If3a9f178c3f915123178efe00269fce74f6e585d
2024-02-04 21:27:00 +01:00
Harald Welte
af8826a02b Implement Global Platform SCP03
This adds an implementation of the GlobalPlatform SCP03 protocol. It has
been tested in S8 mode for C-MAC, C-ENC, R-MAC and R-ENC with AES using
128, 192 and 256 bit key lengh.  Test vectors generated while talking to
a sysmoEUICC1-C2T are included as unit tests.

Change-Id: Ibc35af5474923aed2e3bcb29c8d713b4127a160d
2024-02-04 17:56:59 +01:00
Harald Welte
13a1723c2e rename global_platform.scp02 to global_platform.scp
This is in preparation of extending it to cover SCP03 in a follow-up
patch.

Change-Id: Idc0afac6e95f89ddaf277a89f9c95607e70a471c
2024-02-04 17:56:59 +01:00
Harald Welte
41a7379a4f Introduce GlobalPlatform SCP02 implementation
This implementation of GlobalPlatform SCP02 currently only supports
C-MAC and C-ENC, but no R-MAC or R-ENC yet.

The patch also introduces the notion of having a SCP instance associated
with a SimCardCommands instance.  It also adds the establish_scp0w and
release_scp shell commands to all GlobalPlatform Security Domains.

Change-Id: I56020382b9dfe8ba0f7c1c9f71eb1a9746bc5a27
2024-02-04 17:42:30 +01:00
Harald Welte
93bdf00967 pySim.esim: Add class for parsing/encoding eSIM activation codes
Change-Id: I2256722c04b56e8d9c16a65e3cd94f6a46f4ed85
2024-01-30 21:33:41 +01:00
Harald Welte
8a39d00cc3 osmo-smdpp: Support multiple different profiles
Let's simply use the matchingId for filesystem lookup of the UPP file.

This way we can have any number of profiles by simply creating the
respeective files.

Change-Id: I0bc3a14b9fdfcc6322917dd0c69d8295de486950
2024-01-30 21:33:41 +01:00
Harald Welte
4f3d11b378 euicc: Implement EID checksum verification + computation
Change-Id: I2cb342783137ee7e4b1be3b14e9c3747316f1995
2024-01-16 19:04:19 +01:00
Harald Welte
1f7a9bd5b4 TLV: Add DGI encoding of "GP Scripting Language Annex B"
The DGI encoding is specified in Annex B of the
"GlobalPlatform Systems Scripting Language Specification v1.1.0"

which is an "archived" specification that is no longer published
by GlobalPlatform, despite it being referenced from the GlobalPlatform
Card Specification v2.3, which is the basis of the GSMA eSIM
specifications.

For some reason it was the belief of the specification authors that
yet another format of TLV encoding is needed, in addition to the BER-TLV
and COMPREHENSION-TLV used by the very same specifications.

The encoding of the tag is not really specified anywhere, but I've only
seen 16-bit examples.  The encoding of the length is specified and
implemented here accordingly.

Change-Id: Ie29ab7eb39f3165f3d695fcc1f02051338095697
2024-01-14 17:42:01 +01:00
Harald Welte
f2bcb44ccc pySim.saip.*: Support for parsing / operating on eSIM profiles
This commit introduces the capability to parse and encode
SimAlliance/TCA "Interoperable Profiles" and apply personalization
operations on them.

Change-Id: I71c252a214a634e1bd6f73472107efe2688ee6d2
2024-01-09 21:37:12 +00:00
Harald Welte
5bbb144a31 Initial proof-of-concept SM-DP+ for GSMA consumer eSIM RSP
This commit introduces

* the osmo-smdpp.py program implementing the main procedures and the
  HTTP/REST based ES9+
* python modules for ES8+ and non-volatile RSP Session State storage
* the ASN.1 source files required to parse/encode RSP
* 3GPP test certificates from SGP.26
* an unsigned profile package (UPP) of a SAIP v2.3 TS48 test profile

As I couldn't get the 'Klein' tls support to work, the SM-DP+ code
currently does not support HTTPS/TLS but plan HTTP, so you either have
to modify your LPA to use HTTP instead of HTTPS, or put a TLS proxy in
front.

I have successfully installed an eSIM profile on a test eUICC that
contains certificate/key data within the test CI defined in GSMA SGP.26

Change-Id: I6232847432dc6920cd2bd08c84d7099c29ca1c11
2024-01-09 21:37:12 +00:00
Harald Welte
242197b53d Add pySim.esim.bsp module implementing BSP (BPP Protection Protocol)
This is the protocol used for the ES8+ interface between SM-DP+ and the
eUICC in the GSMA eSIM system.

Change-Id: Ic461936f2e68e1e6f7faab33d06acf3063e261e7
2024-01-07 10:22:04 +01:00
Harald Welte
e13403b206 ts_31_102: Start to use construct for EF.SUCI_Calc_Info
We cannot fully switch to construct for all of it easily due to
the priority value and the ordering/sorting by priority implemented
in the hand-coded version.  But we can at least migrate the
encode/decode of the hnet_pubkey_list via construct.

Change-Id: I4ad5ea57bab37c2dc218e7752d538aa4cdc36ee3
2024-01-04 20:13:06 +01:00
Harald Welte
ce01f48b00 test_files: Test decoder also with ff-padded input
It's customary in the SIM card universe to right-pad data with ff bytes.
So far we only test decoders without such padding, which is unrealistic.
Let's also tests the decoders with extra 'ff' padding present.

For some files this doesn't make sense, so we add a _test_no_pad class
attribute that can be spcified to prevent this new "test with ff-padding"
from being executed for the test data of the class.

Change-Id: I7f5cbb4a6f91040fe9adef9da0a1f30f9f156dae
2023-12-29 18:51:25 +01:00
Harald Welte
bcd261583c tests_files.py: Reduce code duplication
Change-Id: Ib84a0ae35262a19fce3e688afe8e1678a4c59eba
2023-12-29 18:51:25 +01:00