Commit Graph

1445 Commits

Author SHA1 Message Date
Oliver Smith
a8cfeb0111 docs/Makefile: make SPHINXBUILD work in venv
sphinx-build doesn't use the PYTHONPATH from the venv, unless it runs
as python3 -m sphinx.cmd.build. We need it to use the imports from
PYTHONPATH, so we can update the pyosmocom version in requirements.txt
in a patch, and this new version will be used in the jenkins job that
runs during gerrit review. Otherwise the previously installed version
(from the docker image) will be used.

Related: https://github.com/sphinx-doc/sphinx/issues/8910
Change-Id: I487e1af6a3493df5b806cc2d3d2b70bc5233b89f
2024-10-11 16:10:51 +02:00
Philipp Maier
7c62fc5ec4 jenkins: build docs in virtualenv as well
The build system uses a virtual environment, in which it installs
pysim and its dependencies. This is done for the integration tests,
but not when building the sphinx documentation. However, the
documentation build process also invokes pysim code to generate
documentation from the docstrings. This means we need pysim with
all its dependencies for the doc building as well.

Change-Id: I6381eeef7fa19873ca0cc330a0ab43b7ef5096e4
Related: SYS#7094
2024-10-11 15:59:58 +02:00
Philipp Maier
7429bc0ca0 tests: sanitize all cards before running tests
Even though our tests are written in a way that they shouldn't interfere
with each other, it may happen that one testrun writes content to a file
that upsets a different testrun. The resulting problems are often
difficult to diagnose.

To minimize the problem, let's add code that can reset the cards to a
defined state. This can be done using pySim-shell's export
feature. We can generate a backup from a known good state and then play
back the backup to reset files that have been changed. Files that didn't
change will not be written thanks to the conserve_write feature of
pySim-shell.

Related: OS#4384
Change-Id: I42eaf61280968518164f2280245136fd30a603ce
2024-10-05 08:44:20 +00:00
Philipp Maier
93c89856c8 utils: move enc_msisdn and dec_msisdn to legacy/utils.py
We now have a construct based encoder/decoder for the record content
of EF.MSISDN. This means that we do not need the functions enc_msisdn
and dec_msisdn in the non-legacy code anymore. We can now move both
to legacy/utils.py.

Related: OS#5714
Change-Id: I19ec8ba14551ec282fc0cc12ae2f6d528bdfc527
2024-09-27 18:17:19 +02:00
Philipp Maier
1f45799188 ts_102_221: se _test_de_encode instead of _test_decode in EF.DIR unittest
The unittest for EF.DIR only runs with _test_decode, but it also runs with
_test_de_encode without any problems

Related: OS#5714
Change-Id: If459073c6ff927c1cc1790d506e3979243b1fb4c
2024-09-27 18:17:19 +02:00
Philipp Maier
10ea4a0714 ts_51_011: use _test_de_encode instead of _test_decode in EF.CFIS unittest
The unittest for EF.CFIS only runs with _test_decode, but it also runs with
_test_de_encode without any problems

Related: OS#5714
Change-Id: Ib876fd799f871fe64ced2a7b64847ffd09e16ed9
2024-09-27 18:17:19 +02:00
Philipp Maier
dc2ca5d6be ts_51_011: fix unittest for EF.ADN
The unittest for EF.ADN can run with _test_de_encode. However, the original
test vectors seem to be from a card with a slightly larger record size, so
they need a bit of re-alignment

Related: OS#5714
Change-Id: I241792e66ee6167be6ddc076453344b6307d6265
2024-09-27 18:17:19 +02:00
Philipp Maier
39552464d8 ts_51_011: replace encoding of EF.MSISDN with construct model
The encoding of EF.MSISDN is currently done with enc_msisdn and
dec_msisdn from utils.py. Let's replace this with a construct
based model, similar to the one we already use with EF.ADN

Related: OS#5714
Change-Id: I647f5c63f7f87902a86c0c5d8e92fdc7f4350a5a
2024-09-27 18:17:19 +02:00
Philipp Maier
4045146f62 cosmetic: use **kwargs instead of **_kwargs
Some methods sometimes have a **_kwargs parameter, let's be consistent
and use **kwargs only.

Related: OS#5714
Change-Id: I98857cc774185e55a604eb4fbfbf62ed4bd6ded7
2024-09-27 18:17:19 +02:00
Philipp Maier
efddffe015 filesystem: pass total_len to construct of when encoding file contents
In our construct models we frequently use a context parameter "total_len",
we also pass this parameter to construct when we decode files, but we
do not pass it when we generate files. This is a problem, because when
total_len is used in the construct model, this parameter must be known
also when decoding the file.

Let's make sure that the total_len is properly determined and and passed
to construct (via pyosmocom).

Related: OS#5714
Change-Id: I1b7a51594fbc5d9fe01132c39354a2fa88d53f9b
2024-09-27 18:17:19 +02:00
Harald Welte
78c22a7d63 pySim-shell: New '-e' command line argument
Using '-e' it is possible to specify *multiple* pySim-shell commands
which shall be executed at startup.  This extends the current ability
to execute just a single command.

Example:
 ./pySim-shell.py -p0 -e 'select ADF.USIM/EF.IMSI' -e 'read_binary_decoded'

Change-Id: I74004f46105553f077c039ca0f86f75afccc7342
2024-09-23 16:13:16 +00:00
Philipp Maier
d96d04718e pySim-shell_test: disable test_list_and_rm_notif
The testcase euicc.test_list_and_rm_notif fails due to a problem
with the eUICC. The eUICC reports the following error when a
delete notification attempt is made:

"delete_notification_status": "undefinedError"

Let's temporarily disable this testcase until the problem is resolved.

Change-Id: I7d0b6a998499d84f0eb4e24592ad43210ac54806
2024-09-23 13:52:51 +02:00
Oliver Smith
7b95fac022 contrib/jenkins: add SKIP_CLEAN_WORKSPACE
In order to run this script from pyosmocom's contrib/jenkins.sh script,
we want to skip the clean workspace step. Add an environment variable to
do that.

Related: OS#6570
Change-Id: Ic8dc9b85da17719195f7374d37eccb4dedba6ce8
2024-09-23 08:38:55 +02:00
Oliver Smith
c09d4cc6b8 gitignore: add files generated with jenkins.sh
Change-Id: Iaffe04a3dfebd46efc479dc3665fd67f2c95f375
2024-09-23 08:38:55 +02:00
Philipp Maier
f87a00c04f Add testsuite for pySim-shell with real cards
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.

Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
2024-09-20 17:53:27 +02:00
Philipp Maier
d7032955c5 pySim-prog_test: add test vectors for sysmoISIM-SJA5
The sysmoISIM-SJA5 has no testvectors yet

Change-Id: Ia6684ab3ee6c85cfe7bc0ab80d34a26e3499907a
2024-09-20 17:18:50 +02:00
Philipp Maier
26ee39bebf pySim-shell: recognize ADP pins longer than 8 digits as hexadecimal
When a hexadecimal formatted ADM pin is retrieved via the
card_key_provider, it still requires the --pin-is-hex parameter so
that sanitize_pin_adm knows the correct format.

This unfortunately ruins the card_key_provider feature for all cards
that use hexadecimal pins, because the --pin-is-hex would also be
required in scripts, which makes a script either useable for cards
with hexadecimal ADM or for for cards with ASCII ADM.

To minimize the problem, let's recognize all ADM pins longer than 8
digits as hexadecimal in case --pin-is-hex is not set.

Related: OS#4348
Change-Id: Iad9398365d448946c499ce89e3cfb2c3af5d525e
2024-09-20 15:46:50 +02:00
Philipp Maier
01a96cd8e4 pySim-prog_test: individual ICCIDs for all cards
Our test cards need to stay recognizable, so it is important that
each card has a unique ICCID. This means we must write an individual
ICCID, when we test writing the ICCID.

Related: OS#4384
Change-Id: I858a35e526e7b4868e901222d587258412779f41
2024-09-20 12:15:34 +00:00
Philipp Maier
dca641aaa2 pySim-prog_test: do not set an ICCID parameter for sysmoISIM-SJA2
The sysmoISIM-SJA2 does not support changing of the ICCID.
pySim-prog will also reject this, so let's remove the ICCID
from the parameter list.

Related: OS#4384
Change-Id: I89571f2bf7c4cec4d621c322a58687b7781b0ed2
2024-09-20 12:15:34 +00:00
Philipp Maier
154e29c89a requirements: require at least construct version 2.10.70
Older construct versions seem to have problems, in particular with
evaluating COptional() correctly. With 2.10.70 no such problems
were observed.

Related: OS#5714
Change-Id: If59dc708a7194649d1f42c4cf33f6328edcb80d2
2024-09-19 18:09:33 +00:00
Philipp Maier
d5ddd04f33 pySim-shell: improve command "desc"
The "desc" command displays a string with a file description, let's also
display some size information as part of the description as well.

Related: OS#5714
Change-Id: I98e139ba2bf35df5524245cdd96f5c52cf09b986
2024-09-18 10:41:34 +02:00
Philipp Maier
6942a40909 filesystem, cosmetic: remove excess whitespace
Change-Id: I902670590ae75a5d197616ae37d8268a60125121
2024-09-18 10:41:34 +02:00
Philipp Maier
9a6425b6f2 runtime: add new API functions to get the record len and file size
We have an API function to get the number of records, let's now also
add API functions to get the record length and the overall size of
the currently selected file.

Related: OS#5714
Change-Id: Ica7811c04161d8098b40c7219ed6b939df716cfd
2024-09-17 17:59:46 +00:00
Philipp Maier
94ecf9a929 pySim-prog: rework documentation
The documentation for the classic pySim-prog application is a bit
sparse. Let's rework it so that it includes the most important
information that is required to operate pySim-prog. Let's also add
a section about how the batch mode and CSV files are used.

Related: SYS#4120
Change-Id: I1d1a65154cea7fa77428b412fcf8c7b4cba629b1
2024-09-17 15:47:30 +00:00
Philipp Maier
3eb74829df pySim-prog: fix commandline parameter check for CSV mode
The CSV mode needs one of the four additional parameters: --imsi
--iccid, --read-iccid or --read-imsi. Also this check is unrelated
to the batch mode. The CSV file parameter reading works independently
from the batch mode.

Related: SYS#4120
Change-Id: I1292afb85122ed2b7944d02ede69c928a453866f
2024-09-17 15:47:30 +00:00
Philipp Maier
3dc0496913 pySim-prog: treat --imsi and --iccid equally
When using a CSV file, we can either read the IMSI or ICCID from the
CSV file before programming. However, should also be possible to
supply both manually to identify the CSV file entry using the --imsi
or --iccid option. This currently only works for the IMSI, but not
for the ICCID.

Related: SYS#4120
Change-Id: Id3083c7794a7bd59501997f22afdc23bad3069e6
2024-09-17 15:47:30 +00:00
Philipp Maier
39e4a4b7c5 pySim-prog: add FIXME note to tell that writing hlr.db files is broken
The writing to osmo-hlr SQLITE files is broken since the SQLITE format
has evolved over time. Let's add a FIXME note to tell that this needs
fixing.

Related: SYS#4120
Change-Id: I2b23f8bb9f3c2adeb48b010834057f5b4fb1e626
2024-09-17 15:47:30 +00:00
Harald Welte
87e1ba6c18 update pyosmocom dependency to 0.0.3
0.0.3 fixes an important problem related to enabling callers of build_construct()
to pass in a total_len value in order to specify the target output size.

Change-Id: I01687bb54e65bf5cc318745df588c3d6ea14eb83
2024-09-17 17:25:43 +02:00
Harald Welte
ad3d73e734 docs: Bring osmo-smdpp documentation up to date with code
Change-Id: Ibaab1fadd5d35ecdb356bed1820074b1b0a1752e
Closes: OS#6418
2024-09-17 15:22:45 +00:00
Harald Welte
8e42a12048 docs: remove traces of modules migrated to pyosmocom
Change-Id: I2ebb17f9781c90a81e9e554bddd7a851ef51c82a
2024-09-17 15:22:45 +00:00
Harald Welte
84857accf3 pySim-shell: Detect different eUICC types and print during start-up
Change-Id: I54ea4ce663693f3951040dcc8a16bf532bf99c02
2024-09-17 15:22:45 +00:00
Harald Welte
72186cce84 pySim.profile: Further refactor card <-> profile matching
The new architecture avoids sim/ruim/uicc specific methods in
pySim.profile and instead moves the profile-specific code into the
profile; it also solves everything within the class hierarchy, no need
for global methods.

Change-Id: I3b6c44d2f5cce2513c3ec8a3ce939a242f3e4901
2024-09-17 15:22:45 +00:00
Harald Welte
5f2dfc28ff pySim/profile: Change match_with_profile from static to class method
This was suggested by vyanitskiy during gerrit patch review in
https://gerrit.osmocom.org/c/pysim/+/38049 in order to make the
upcoming eUICC CardProfiles simpler.

Change-Id: Ia7c049b31cb1c5c5bb682406d9dd7a73bcd43185
2024-09-17 15:22:45 +00:00
Philipp Maier
bd762c77ae pySim-prog: fix sourcecode formatting
Change-Id: Ie4d4ec6d1752013fa8aa39912aa600c2b4afac74
2024-09-16 12:20:43 +02:00
Philipp Maier
492379e61a pySim-prog: fix sourcecode formatting
Change-Id: I0e567a1a681891f33f170c5df50c691b20b6a978
2024-09-16 12:20:19 +02:00
Philipp Maier
7633a11239 pySim-shell: print cardinfo hexstrings in lowercase
To ensure consistency, let's print the cardinfo hexstrings in lowercase
only.

Related: OS#6531
Change-Id: Ia6a8bd0e700c7fd933fb6c1b1050ed9494462d60
2024-09-11 14:37:24 +02:00
Harald Welte
07b67439f8 pySim.euicc: Add 'get_data sgp02_eid' in ADF.ECASD of M2M eUICC
The M2M eUICC are completely different from the consumer/IoT eUICC.

Obtaining the EID works via GET DATA in the ECASD.  Let's add support
for that.

Change-Id: I6cca6f75d268229244c90b3f1f88e26c89a2b4e0
2024-09-10 20:40:16 +02:00
Harald Welte
c3fe111c0e pySim.commands: use _checksw during get_data() method
All other methods use send_apdu_checksw, just get_data()
was missing the _checksw part.

Change-Id: Ic784bf0c30b22e5e83843aa6694e2706b4b2ac48
2024-09-10 20:40:16 +02:00
Harald Welte
2fe9b6a3e9 pySim.transport: Also trace card reset events in ApduTracer
Change-Id: Ia46b65124520eb2b8015dfa3f0a135b497668b92
2024-09-10 20:37:56 +02:00
Harald Welte
241d65db12 pySim.transport: Add support for generic stdout apdu tracer
Any program using argparse_add_reader_args() will get a new
long-opt '--apdu-trace' which enables a raw APDU trace to the console.

Change-Id: I4bc3d2e023ba360f07f024d7b661a93322f87530
2024-09-07 14:43:58 +02:00
Harald Welte
bf0689a48e pySim.app: Properly reset card state after reading EID
The code had two problems:

* the RESET was only performed in the successful case, but not if
  some exceptio was raised

* the RESET was a low-level reset bypassing the RuntimeState,
  so the lchan.selected_file was stale afterwards

Fixes: Change-Id Idc2ea1d9263f39b3dff403e1535a5e6c4e88b26f

Change-Id: Ib23d3d5b58b456a25157a622c1010c81cd8b2213
2024-09-07 14:43:58 +02:00
Harald Welte
726097e51f transport: define TERMINAL RESPONSE content within ProactiveHandler
So far the core proactive handling code would always generate a positive
response, with no way for the ProactiveHandler call-back to influence
that or to include additional IEs/TLVs.

Let's change that.

Change-Id: Ic772b3383533f845689ac97ad03fcf67cf59c208
2024-09-05 11:30:53 +00:00
Harald Welte
ee9ac2f7ff suci-tutorial: fix typo s/symo/sysmo/
Change-Id: I0d3bdcf590e8dfef6deabc9967fd2f04152e1020
2024-09-04 09:54:15 +02:00
Philipp Maier
398fdd7e8c pySim-shell: use upper case letters for positional arguments
When we define positional arguments for the argument parser, we usually
use upper case letters only. However, there are some code locations that
use lower case letters. Let's translate those to capital letters to
have a consistent appeariance.

Related: OS#6531
Change-Id: Iec1ff8262bc6e9cf87c3cbf7b32fa5f753b7e574
2024-09-04 09:37:28 +02:00
Philipp Maier
639806cc5a pySim-shell: do not display 'AIDs:' when there are none
The command cardinfo also displays the AIDs of the card applications.
However, on classic GSM sim cards there are no applications. In this
case cardinfo will still display the string 'AIDs:', but it will of
course not list any AIDs under this string.

Related: OS#6531
Change-Id: Ifb111ce43fdebe85d30857dfc61ab570380b68d1
2024-09-04 09:36:44 +02:00
Philipp Maier
471162dc76 suci-tutorial: add section about SUCI calculation by the USIM
The tutorial describes how SUCI calculation in the UE is configured,
let's now add a section about SUCI calculation by the USIM.

Related: OS#6531
Change-Id: I45d47f9278b30d99ebde6891de0ba8cc74b1a0a0
2024-09-04 09:36:43 +02:00
Philipp Maier
f81331808f pySim-shell: rework startup procedure and introduce non interactive mode
When pySim-shell is used in a scripted environment, we may easily get trapped in
the pySim-shell prompt. This may happen in particular in case the script file
is not executed due to problem with the reader initialization. In such a case
pySim-shell will not exit automatically and the shellscript that was calling
pySim-shell will stall indefinetly.

To make the use of pySim-shell more reliable in scripted environments, let's
add a --noprompt option that ensures the interactive mode is never entered.
Let's also exit with an appropriate return code in case of initialization
errors, so that the calling script can know that something went wrong.

Related: OS#6531
Change-Id: I07ecb27b37e2573629981a0d032cc95cd156be7e
2024-09-04 07:15:14 +00:00
Philipp Maier
bd7c21257c commands: avoid double lchan patching, get rid of cla_byte getter+setter methods
The SimCardCommands has a cla_byte @property method, which automatically
returns the lchan patched CLA byte. We use cla_byte property to build
the UICC command APDUs inside SimCardCommands and then we hand the APDU
over to the send_apdu* methods. The cla_byte @property method as well
as the send_apdu* methods perform the lchan patching. This means the CLA
byte gets patched twice, which is technically not an issue, but can be
confusing when trying to understand the code.

To fix this, let's remove the @property methods and turn cla_byte into
a normal property again. This is also more accurate since the cla_byte
property originally was introduced to switch between UICC and classic
SIM APDU commands, which have almost identcal APDUs.

Related: OS#6531
Change-Id: I420f8a5f7ff8d9e5ef94d6519fb3716d6c7caf64
2024-09-03 21:17:28 +00:00
Harald Welte
6aabb92c38 esim.saip.templates: Fix expand_default_value_pattern for length==0
The original code treated length==0 like length==None (unspecified),
which is wrong.

Change-Id: I39fa1e2b1b9d6d1c671ea37bdbec1d6f97e8a5e7
2024-09-03 21:57:47 +02:00
Harald Welte
b22bab0b20 pySim.esim.saip.ProfileElementGFM: Initialize 'fileManagementCMD'
When constructing a ProfileElmentGFM from scratch, initialize the
decoded['fileManagementCMD'], as it is a mandatory member during
ASN.1 encode.

Change-Id: Iaae99348d36b7f0c739daf039d6ea2305b7ca9db
2024-09-03 21:57:47 +02:00