Commit Graph

1750 Commits

Author SHA1 Message Date
Neels Hofmeyr 7f87563d09 Revert "esim/http_json_api: add alternative API interface (follow up)"
This reverts commit 8b2a49aa8e.
2026-06-26 21:51:06 +02:00
Neels Hofmeyr 5d079d085f Revert "esim/http_json_api: allow URL rewriting"
This reverts commit 0634f77308.
2026-06-26 21:51:06 +02:00
Neels Hofmeyr fd8c2ba0b0 personalization: add GfmSuciRi, GfmSuciCalcInfo, and test
Change-Id: I1b69debf5992aa715171b43b30864dc152dc556f
2026-06-26 21:51:06 +02:00
Neels Hofmeyr 27dfae5efc SAIP2.1_gfmsuci.der
Change-Id: I69354754013e4a98d86a9a8abdd7a6a36eb95177
2026-06-26 21:51:06 +02:00
Neels Hofmeyr 785ebfdd06 debug log for rebuild_mandatory_services()
Change-Id: Ie53c84247a9524a81b475b3b02b5330207c15601
2026-06-26 21:51:06 +02:00
Neels Hofmeyr 0d8a43904f saip/personalization: add EuiccMandatoryServiceParam for 3 services
Even though the eUICC-Mandatory-services are always set by
saip.PES.rebuild_mandatory_services(), these new params are useful to
audit and validate the values as present in a given DER.

Change-Id: Icddeb2488c4a024c6ee5afcc1b6c8cc0e436c43c
2026-06-26 21:48:49 +02:00
Neels Hofmeyr 664d69db65 personalization: SUCI: ff-pad remaining file size
Change-Id: I9465eac9269e3f76dddd467109f547489a0feb93
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 0175b1045b saip.personalization: add SAIP 2.3 SUCI parameters
Change-Id: I3c0793b8a67bbd0c8247784bd3b5cbd265f94ec2
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 945dfb0129 tweak test_configurable_parameters.py: add iff_present flag
apply a parameter only when it exists in the template, will be useful
for suci

Change-Id: I5811ecde4c4e880bb8dbd22fffe23faafcfe36ad
2026-06-26 21:47:48 +02:00
Neels Hofmeyr f1447ea55b tweak test_configurable_parameters.py: show value found in template
Change-Id: If9039bbb8547ee24ae784a932f60cd5de6c9247b
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 7f32641008 tweak test_configurable_parameters.py: saner output composition.
Change-Id: Id3e3b46b2b3d7919a75c620803ce28d2a715008b
2026-06-26 21:47:48 +02:00
Neels Hofmeyr c01d7a70e1 TuakNumberOfKeccak: remove numeric_base to hide random number sources
Change-Id: I913878e3f05ad1e39ff45da75c67582a6a1f930f
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 8c81e70225 saip/batch: add blacklist to to_csv_rows()
Change-Id: I5e567e59a007cf4b5d75a4dcea5371ff4404bf24
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 434cc45259 ConfigurableParameter.get_typical_input_len: limit to 10 lines
Change-Id: Ia3d79e786f397a02bf2a8fafac5030d1198d9f76
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 434669e36c personalization: EF_SMSP: keep same length as found in template
Change-Id: Id24752101ae82c4986209f4103cc9cbdcce8ce1d
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 26ea0d1656 saip.batch: log parameter errors
Change-Id: I6a46b2dc9018078ab8361226d1e6b50d3b4e1aaa
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 7362525f8e personalization: add MncLen ConfigurableParameter
Implement as EnumParam because it only has two possible values

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 5e543f5566 personalization: EnumParam: implement as value_map, not enum.IntEnum
EnumParam is for labels shown in the UI, not an enum to be used in
python code (what the python enum module is written for). So it is
semantically wrong to use enum.IntEnum.

- the label should be allowed to be any string, not just valid python
  identifiers. For example, a label like "SUCI-in-USIM" should be
  possible.

- we should not "leak" UI labels into the python namespace.

- the value should be allowed to be any type, so that each
  ConfigurableParameter implementation can use whichever is its internal
  native type.

History: this patch is the original version of EnumParam, which was
modified during CR to use enum.IntEnum. However, newer
ConfigurableParameters coming up don't match well:
- MncLen (labels "2" and "3")
- EuiccMandatoryServiceParam (values True and False)
- EfUstServiceParam like SuciInUsim (labels "SUCI-in-UE" and
  "SUCI-in-USIM")

So this brings back the original capability for any label string, and
any type of value.

Change-Id: I690ceccf0ec7ef7067bcaa5cec1303cdaf0f78a4
2026-06-26 21:47:48 +02:00
Neels Hofmeyr e2ec0347f6 esim/http_json_api.py: support text/plain response Content-Type
Allow returning text/plain Content-Types as 'data' output argument.

So far, all the esim/http_json_api functions require a JSON response.
However, a specific vendor has a list function where the request is JSON
but the response is text/plain CSV data. Allow and return in a dict.

Change-Id: Iba6e4cef1048b376050a435a900c0f395655a790
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 7a1c506ba9 saip.SdKey*: remove transitional name mapping
Now, finally, all SdKey classes have a unified logical naming scheme.
Revert the transitional name mapping.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
2026-06-26 21:47:48 +02:00
Neels Hofmeyr cd7a788d6d saip.SdKey*: transitional name mapping
To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 7fcd21dd4f ts_31_102.py: EF_SUCI_Calc_Info(TransparentEF): fix len test
while len(foo):

throws an exception when foo == None.
Instead doing

    while foo:

fixes a problem when reading in empty SUCI calc info data, e.g. from
TS48v7.0_SAIP2.3_BERTLV_SUCI_NoRAMRFM.der.

Change-Id: Ia4e2356d0241d7a6ca399ba7e8be7f27ec836104
2026-06-26 21:47:48 +02:00
Neels Hofmeyr a23f06ba4c typo 'concetenation' in personalization.py
Change-Id: I51345db014335e8a70a7437a9cad5a3e47570a95
2026-06-26 21:47:48 +02:00
Neels Hofmeyr 0ed72f6ee7 test_configurable_parameters: test less templates
Change-Id: Ib75b6919a3acfddd99bf9baa9b6847ef731b9e67
2026-06-26 21:47:48 +02:00
Neels Hofmeyr c5f6ba83c2 saip BatchPersonalization: call rebuild_mandatory_services()
Particular reason: when manipulating the 5G SUCI parameters, the
mandatory services get-identity, profile-a-x25519 and profile-b-p256 may
need to be reconfigured.

In general, it is a good idea to run these checks anyway.

Change-Id: I5e6eef0f1845a25cddb03af8d16c40e305bcdc1f
2026-06-26 20:22:59 +02:00
Neels Hofmeyr d23e3fdd44 saip.PES.rebuild_mandatory_services(): set 5G get-identity, profile-a-x25519, profile-b-p256
Related: SYS#8096 SYS#8037
Change-Id: Ibc29c6437c5c92e2b14938b733156536863465c1
2026-06-26 20:21:10 +02:00
Alexander Couzens d0e6a1b119 euicc: get_profiles_info: add additional tags
Add definitions for ProfileOwner (decoded),
Notification Configuration Info, SM-DP+ proprietary data,
Profile Policy Rules.

Change-Id: I727dbe34d87a42bb3b526bd7a8accd687d20a208
2026-06-22 17:03:04 +00:00
Alexander Couzens 980282cc12 euicc: extend get_profiles_info to retrieve all known tags
get_profiles_info only request for the default tag list, but
not all tags.
Add --all to the function to request for all known tags.

Change-Id: Ia6878519a480bd625bb1fa2567c1fd2e0e89b071
2026-06-22 17:03:04 +00:00
Neels Hofmeyr 728940efb2 saip: add numeric_base indicator to ConfigurableParameter
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr cfe2b94f67 saip SmspTpScAddr.get_values_from_pes: allow empty values
Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr 861ed0a1d8 add comment about not updating existing key_usage_qualifier
Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test
2026-06-22 19:56:29 +07:00
Neels Hofmeyr b576e8fcff test_configurable_parameters.py: add tests for new parameters
For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test
2026-06-22 19:48:52 +07:00
Neels Hofmeyr 38f93d974b SmspTpScAddr: fix SMSP record length and alpha_id padding
apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field.  After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.

Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
  padding) so the re-encoded body carries 0xFF-padded alpha_id space
  and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
  alpha_id area is actually padded to that length

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
2026-06-21 06:17:54 +07:00
Neels Hofmeyr c5e7e59928 ConfigurableParameter: safer val length check
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.

Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr 98af3dd2e9 UppAudit: better indicate exception cause
Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr e9ff4f3b93 personalization: generate sdkey classes from a list
Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr ce039d69ba saip/param_source: try to not repeat random values
Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test
2026-06-21 03:46:16 +07:00
Neels Hofmeyr aad92f2b73 param_source: use secrets.SystemRandom as secure random nr source
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test
2026-06-21 03:46:11 +07:00
Neels Hofmeyr 512aba8b1d param_source: use random.SystemRandom as random nr source
Python's random module uses a PRNG (Mersenne Twister) which is
utterly insecure for key generation - it was so far only used for
testing.  Replace it with random.SystemRandom(), which draws from
/dev/urandom and is suitable for generating cryptographic key material.

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test
2026-06-21 03:42:06 +07:00
Neels Hofmeyr b5ba274583 add test_param_src.py
Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr 4307cffc82 param_source: allow plugging a random implementation (for testing)
Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr bfdfcad22c personalization: add int as input type for BinaryParameter
Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr ef0a2fcb37 personalization.ConfigurableParameter: fix BytesIO() input
Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr 3974e96933 add test_configurable_parameters.py
Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).

Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.

Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.

Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
2026-06-21 03:30:02 +07:00
Neels Hofmeyr a7c762eb2e ConfigurableParameter: do not magically overwrite the 'name' attribute
The ClassVarMeta metaclass used to derive each ConfigurableParameter's
'name' attribute automatically from the Python class name (via
camel_to_snake()).  Stop doing this, for three reasons:

1) Python class names follow constraints that do not fit the naming
   commonly used in CSV files.  For example, a name like
   "5GS-SUCI-CalcInfo" starts with a digit and contains dashes,
   neither of which is permissible in a class name.

2) Python class names live in their own namespace, distinct from the
   one used to present eSIM parameters to end users.  Deriving the UI
   name from the class name couples these two namespaces together.

   Taken together, (1) and (2) mean that automatic naming both imposes
   class-name constraints on the user-visible names and merges the
   internal Python namespace with the publicly shown one - a layer
   violation from the perspective of UI design.

3) Overriding 'name' from __new__() makes manual naming impossible: a
   subclass that sets 'name = "bar"' as a class attribute would still
   end up with the value computed by the metaclass, which is
   surprising and hard to track down:

       class MySuper(metaclass=...):  # __new__ sets name = 'foo'
           ...
       class MySub(MySuper):
           name = 'bar'
       print(MySub().name)  # 'foo', not 'bar' as one would expect

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test
2026-06-21 03:29:38 +07:00
Neels Hofmeyr 710a27d6cf personalization audit: optionally audit all (unknown) SD keys
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x04..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test
2026-06-19 10:37:56 +00:00
Neels Hofmeyr 08f40db8a3 personalization: implement UppAudit and BatchAudit
Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
Jenkins: skip-card-test
2026-06-19 10:37:56 +00:00
kukutyin 4fb393e6ea fix(ts_51_011): fix lifecycle decoding
- implement proper LCS decoding per TS 102 221 / TS 31.101
- previous implementation misclassified multiple states

Related: OS#7018
Change-Id: I8a3bd820b9fbc13c025f8302d1d2eac21686c541
2026-06-08 11:54:32 +00:00
kukutyin ce5da32a75 fix(ts_51_011): apply correct access conditions length
When the access conditions are extracted from resp_bin, the wrong length
is used and only 2 bytes instead of 3 are extracted.

3GPP TS 51.011, section 9.2.1, table below "Response parameters/data
in case of an EF", clearly states that the length should be 3 bytes
(position 9-11)

Related: OS#7018
Change-Id: I410fb58c395beafba8de6d5ab4e71452f424cdf2
2026-06-08 11:54:28 +00:00
Philipp Maier 9ddd235a2c pySim/ts_51_011: rewrite comment for better understanding
The comment reads like that we were applying TS 102.221 here, but we only
mean our internal decoding format. The spec that actually matters here is
TS 51.011. Let's rephrase the comment so that this becomes more clear.

Related: OS#7018
Change-Id: Ie0184eea25f4d9f4baf9ab137c53a926edba2bf8
2026-06-03 08:46:02 +00:00