Commit Graph

1746 Commits

Author SHA1 Message Date
Neels Hofmeyr c4fc946ac8 tweak test_configurable_parameters.py: saner output composition.
Change-Id: Id3e3b46b2b3d7919a75c620803ce28d2a715008b
2026-07-15 01:11:38 +02:00
Neels Hofmeyr b0a4216f69 TuakNumberOfKeccak: remove numeric_base to hide random number sources
Change-Id: I913878e3f05ad1e39ff45da75c67582a6a1f930f
2026-07-15 01:06:31 +02:00
Neels Hofmeyr 26cd602d0d saip/batch: add blacklist to to_csv_rows()
Change-Id: I5e567e59a007cf4b5d75a4dcea5371ff4404bf24
2026-07-15 01:06:31 +02:00
Neels Hofmeyr 7593a40e64 ConfigurableParameter.get_typical_input_len: limit to 10 lines
Change-Id: Ia3d79e786f397a02bf2a8fafac5030d1198d9f76
2026-07-15 01:06:31 +02:00
Neels Hofmeyr 49b61f3c14 personalization: EF_SMSP: keep same length as found in template
Change-Id: Id24752101ae82c4986209f4103cc9cbdcce8ce1d
2026-07-15 01:06:31 +02:00
Neels Hofmeyr e0cee1f4b4 saip.batch: log parameter errors
Change-Id: I6a46b2dc9018078ab8361226d1e6b50d3b4e1aaa
2026-07-15 01:06:31 +02:00
Neels Hofmeyr f557e14984 personalization: add MncLen ConfigurableParameter
Implement as EnumParam because it only has two possible values

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
2026-07-15 01:06:30 +02:00
Neels Hofmeyr 09c41b05fd personalization: EnumParam: implement as value_map, not enum.IntEnum
EnumParam is for labels shown in the UI, not an enum to be used in
python code (what the python enum module is written for). So it is
semantically wrong to use enum.IntEnum.

- the label should be allowed to be any string, not just valid python
  identifiers. For example, a label like "SUCI-in-USIM" should be
  possible.

- we should not "leak" UI labels into the python namespace.

- the value should be allowed to be any type, so that each
  ConfigurableParameter implementation can use whichever is its internal
  native type.

History: this patch is the original version of EnumParam, which was
modified during CR to use enum.IntEnum. However, newer
ConfigurableParameters coming up don't match well:
- MncLen (labels "2" and "3")
- EuiccMandatoryServiceParam (values True and False)
- EfUstServiceParam like SuciInUsim (labels "SUCI-in-UE" and
  "SUCI-in-USIM")

So this brings back the original capability for any label string, and
any type of value.

Change-Id: I690ceccf0ec7ef7067bcaa5cec1303cdaf0f78a4
2026-07-15 01:02:14 +02:00
Neels Hofmeyr a9b1fe0ae4 esim/http_json_api.py: support text/plain response Content-Type
Allow returning text/plain Content-Types as 'data' output argument.

So far, all the esim/http_json_api functions require a JSON response.
However, a specific vendor has a list function where the request is JSON
but the response is text/plain CSV data. Allow and return in a dict.

Change-Id: Iba6e4cef1048b376050a435a900c0f395655a790
2026-07-15 01:02:14 +02:00
Neels Hofmeyr 45e2be5ab6 saip.SdKey*: remove transitional name mapping
Now, finally, all SdKey classes have a unified logical naming scheme.
Revert the transitional name mapping.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
2026-07-15 01:02:14 +02:00
Neels Hofmeyr c682421c1b saip.SdKey*: transitional name mapping
To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
2026-07-15 01:02:14 +02:00
Neels Hofmeyr 4b3b04219d ts_31_102.py: EF_SUCI_Calc_Info(TransparentEF): fix len test
while len(foo):

throws an exception when foo == None.
Instead doing

    while foo:

fixes a problem when reading in empty SUCI calc info data, e.g. from
TS48v7.0_SAIP2.3_BERTLV_SUCI_NoRAMRFM.der.

Change-Id: Ia4e2356d0241d7a6ca399ba7e8be7f27ec836104
2026-07-15 01:02:14 +02:00
Neels Hofmeyr 1f43f267ee typo 'concetenation' in personalization.py
Change-Id: I51345db014335e8a70a7437a9cad5a3e47570a95
2026-07-15 01:02:14 +02:00
Neels Hofmeyr 75c639fb6c test_configurable_parameters: test less templates
Change-Id: Ib75b6919a3acfddd99bf9baa9b6847ef731b9e67
2026-07-15 01:02:12 +02:00
Neels Hofmeyr 733394caa5 saip BatchPersonalization: call rebuild_mandatory_services()
Particular reason: when manipulating the 5G SUCI parameters, the
mandatory services get-identity, profile-a-x25519 and profile-b-p256 may
need to be reconfigured.

In general, it is a good idea to run these checks anyway.

Change-Id: I5e6eef0f1845a25cddb03af8d16c40e305bcdc1f
2026-07-15 00:58:19 +02:00
Neels Hofmeyr d2ab5c2afb saip.PES.rebuild_mandatory_services(): set 5G get-identity, profile-a-x25519, profile-b-p256
Related: SYS#8096 SYS#8037
Change-Id: Ibc29c6437c5c92e2b14938b733156536863465c1
2026-07-15 00:58:19 +02:00
Philipp Maier 25e43e1540 pySim/ara_m: move code from do_aram_ to static methods
The method do_aram_store_ref_ar_do and do_aram_delete_all, which
are part of the nested AddlShellCommands class, may be moved into
the parent class as a static method, just like the already existing
get_config method.

This makes the functionality re-usable to callers that do not use
the CMD2 API.

Change-Id: Icd1b08ec707dd939bc9e8524d7f9431aa4daae7c
Related: SYS#6959
2026-07-13 12:13:04 +02:00
Neels Hofmeyr 6e10da4c55 saip/personalization: add MncLen configurable parameter
Add a new ConfigurableParameter that represents the MNC length
(2 or 3 digits) in EF.AD (Administrative Data).

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
2026-07-06 17:24:47 +00:00
Vadim Yanitskiy 973d6eb2cc pySim.log: fix E0611: No name 'style' in module 'cmd2'
Change-Id: I191ea56f4c6e4e1916369f69fe2e1653e1d92df1
Fixes: 597f1e0 ("pySim.log, pySim-shell: fix compatibility with cmd2 >= 3.0.0")
2026-07-01 16:47:08 +07:00
Vadim Yanitskiy 597f1e0398 pySim.log, pySim-shell: fix compatibility with cmd2 >= 3.0.0
Some Linux distributions (e.g. Arch Linux) already ship cmd2 3.x.x,
which removed the style()/Fg/Bg API in favor of stylize()/Color.

Add a version guard to select the right API at runtime.
Adjust the upper bound cap in requirements.txt and setup.py.

Change-Id: Ibf2ac7847933296fb06665c87f53ed6e1f315d27
2026-06-26 02:47:45 +07:00
Vadim Yanitskiy 45d37ed959 pySim-shell: drop backwards compat quirks for cmd2 < 2.6.2
Remove version guards for cmd2 < 2.0.0 and < 2.3.0, the Cmd2Compat
and Settable2Compat wrapper classes, and the old fg/bg color API -
none of these are needed since both requirements.txt and setup.py
already mandate cmd2 >= 2.6.2.

Change-Id: Ifd1c484ab66d74323d10e946347daa637cf6f5d8
2026-06-25 22:51:03 +07:00
Harald Welte 757c7d048e setup.py: Align cmd2 minimum version with requirements.txt
As pointed out in the commit-log of Change-Id
I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6 we had different minimum
versions for cmd2 in requirements.txt vs setup.py.  Let's align that.

Change-Id: I71cee0ec3ed2abec68ec567beaab13c868721dad
2026-06-25 21:43:52 +07:00
Alexander Couzens d0e6a1b119 euicc: get_profiles_info: add additional tags
Add definitions for ProfileOwner (decoded),
Notification Configuration Info, SM-DP+ proprietary data,
Profile Policy Rules.

Change-Id: I727dbe34d87a42bb3b526bd7a8accd687d20a208
2026-06-22 17:03:04 +00:00
Alexander Couzens 980282cc12 euicc: extend get_profiles_info to retrieve all known tags
get_profiles_info only request for the default tag list, but
not all tags.
Add --all to the function to request for all known tags.

Change-Id: Ia6878519a480bd625bb1fa2567c1fd2e0e89b071
2026-06-22 17:03:04 +00:00
Neels Hofmeyr 728940efb2 saip: add numeric_base indicator to ConfigurableParameter
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr cfe2b94f67 saip SmspTpScAddr.get_values_from_pes: allow empty values
Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr 861ed0a1d8 add comment about not updating existing key_usage_qualifier
Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr b576e8fcff test_configurable_parameters.py: add tests for new parameters
For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test
2026-06-22 19:48:52 +07:00
Neels Hofmeyr 38f93d974b SmspTpScAddr: fix SMSP record length and alpha_id padding
apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field.  After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.

Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
  padding) so the re-encoded body carries 0xFF-padded alpha_id space
  and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
  alpha_id area is actually padded to that length

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
2026-06-21 06:17:54 +07:00
Neels Hofmeyr c5e7e59928 ConfigurableParameter: safer val length check
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.

Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr 98af3dd2e9 UppAudit: better indicate exception cause
Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr e9ff4f3b93 personalization: generate sdkey classes from a list
Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr ce039d69ba saip/param_source: try to not repeat random values
Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr aad92f2b73 param_source: use secrets.SystemRandom as secure random nr source
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test
2026-06-21 03:46:11 +07:00
Neels Hofmeyr 512aba8b1d param_source: use random.SystemRandom as random nr source
Python's random module uses a PRNG (Mersenne Twister) which is
utterly insecure for key generation - it was so far only used for
testing.  Replace it with random.SystemRandom(), which draws from
/dev/urandom and is suitable for generating cryptographic key material.

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test
2026-06-21 03:42:06 +07:00
Neels Hofmeyr b5ba274583 add test_param_src.py
Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr 4307cffc82 param_source: allow plugging a random implementation (for testing)
Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr bfdfcad22c personalization: add int as input type for BinaryParameter
Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr ef0a2fcb37 personalization.ConfigurableParameter: fix BytesIO() input
Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr 3974e96933 add test_configurable_parameters.py
Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).

Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.

Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.

Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr a7c762eb2e ConfigurableParameter: do not magically overwrite the 'name' attribute
The ClassVarMeta metaclass used to derive each ConfigurableParameter's
'name' attribute automatically from the Python class name (via
camel_to_snake()).  Stop doing this, for three reasons:

1) Python class names follow constraints that do not fit the naming
   commonly used in CSV files.  For example, a name like
   "5GS-SUCI-CalcInfo" starts with a digit and contains dashes,
   neither of which is permissible in a class name.

2) Python class names live in their own namespace, distinct from the
   one used to present eSIM parameters to end users.  Deriving the UI
   name from the class name couples these two namespaces together.

   Taken together, (1) and (2) mean that automatic naming both imposes
   class-name constraints on the user-visible names and merges the
   internal Python namespace with the publicly shown one - a layer
   violation from the perspective of UI design.

3) Overriding 'name' from __new__() makes manual naming impossible: a
   subclass that sets 'name = "bar"' as a class attribute would still
   end up with the value computed by the metaclass, which is
   surprising and hard to track down:

       class MySuper(metaclass=...):  # __new__ sets name = 'foo'
           ...
       class MySub(MySuper):
           name = 'bar'
       print(MySub().name)  # 'foo', not 'bar' as one would expect

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test
2026-06-21 03:29:38 +07:00
Neels Hofmeyr 710a27d6cf personalization audit: optionally audit all (unknown) SD keys
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x04..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test
2026-06-19 10:37:56 +00:00
Neels Hofmeyr 08f40db8a3 personalization: implement UppAudit and BatchAudit
Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
Jenkins: skip-card-test
2026-06-19 10:37:56 +00:00
kukutyin 4fb393e6ea fix(ts_51_011): fix lifecycle decoding
- implement proper LCS decoding per TS 102 221 / TS 31.101
- previous implementation misclassified multiple states

Related: OS#7018
Change-Id: I8a3bd820b9fbc13c025f8302d1d2eac21686c541
2026-06-08 11:54:32 +00:00
kukutyin ce5da32a75 fix(ts_51_011): apply correct access conditions length
When the access conditions are extracted from resp_bin, the wrong length
is used and only 2 bytes instead of 3 are extracted.

3GPP TS 51.011, section 9.2.1, table below "Response parameters/data
in case of an EF", clearly states that the length should be 3 bytes
(position 9-11)

Related: OS#7018
Change-Id: I410fb58c395beafba8de6d5ab4e71452f424cdf2
2026-06-08 11:54:28 +00:00
Philipp Maier 9ddd235a2c pySim/ts_51_011: rewrite comment for better understanding
The comment reads like that we were applying TS 102.221 here, but we only
mean our internal decoding format. The spec that actually matters here is
TS 51.011. Let's rephrase the comment so that this becomes more clear.

Related: OS#7018
Change-Id: Ie0184eea25f4d9f4baf9ab137c53a926edba2bf8
2026-06-03 08:46:02 +00:00
Philipp Maier 77eb30a782 unittests: add testcases for decode_select_response
A CardProfile class usually contains a static method decode_select_response.
Unfortunately those methods have no unit-test coverage yet. Let's add unit
tests for the decoders in CardProfileSIM and CardProfileUICC.

Related: OS#7018
Change-Id: Id2b5e005d7ad30d56c5c936e612600213620a0ed
2026-06-03 08:46:02 +00:00
YanTong C 2530329ae2 osmo-smdpp.py: use commonpath in transversal check
Use commonpath, as commonprefix allows accessing a sibiling directory
with the same prefix.

Change-Id: I7a42b40aa2bbcd5f0ec99f172503354c6eaa9828
2026-05-27 10:23:13 +02:00
Philipp Maier f9e4291a43 pySim/ara_m, cosmetic: swap --nfc-always and --nfc-never options
The commandline options --nfc-always and --nfc-never appear in the
opposite order when compared to --apdu-never and --apdu-always.

Let's swap the options to make the helpscreen and the code more
consistent.

Change-Id: I7289c3628b1b8dd3eec2f1c8f2132e3015422960
Related: SYS#6959
2026-05-13 15:49:20 +02:00
Philipp Maier 20538775b2 pySim/scp: migrate to pySimLogger
The module scp.py predates the existence of the pySimLogger and still
uses an individually created logger. Let's migrate to pySimLogger to
avoid unexpected effects and to be uniform with the other modules.

Related: SYS#6959
Change-Id: I5db7180f93f116dd2d99c33da264f74ea16a1a37
2026-05-08 17:54:33 +02:00