Commit Graph

1738 Commits

Author SHA1 Message Date
Neels Hofmeyr 7d18f16e26 personalization: SUCI: ff-pad remaining file size
Change-Id: I9465eac9269e3f76dddd467109f547489a0feb93
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 00af942dbc add SUCI parameters
Change-Id: I3c0793b8a67bbd0c8247784bd3b5cbd265f94ec2
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 13530c6fe9 tweak test_configurable_parameters.py: add iff_present flag
apply a parameter only when it exists in the template, will be useful
for suci

Change-Id: I5811ecde4c4e880bb8dbd22fffe23faafcfe36ad
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 27def57be2 tweak test_configurable_parameters.py: show value found in template
Change-Id: If9039bbb8547ee24ae784a932f60cd5de6c9247b
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 2c3820c39f tweak test_configurable_parameters.py: saner output composition.
Change-Id: Id3e3b46b2b3d7919a75c620803ce28d2a715008b
2026-05-21 13:27:42 +02:00
Neels Hofmeyr b8871ed04e TuakNumberOfKeccak: remove numeric_base to hide random number sources
Change-Id: I913878e3f05ad1e39ff45da75c67582a6a1f930f
2026-05-21 13:27:42 +02:00
Neels Hofmeyr d9e0adbe1a saip/batch: add blacklist to to_csv_rows()
Change-Id: I5e567e59a007cf4b5d75a4dcea5371ff4404bf24
2026-05-21 13:27:42 +02:00
Neels Hofmeyr e04db4b7fb ConfigurableParameter.get_typical_input_len: limit to 10 lines
Change-Id: Ia3d79e786f397a02bf2a8fafac5030d1198d9f76
2026-05-21 13:27:42 +02:00
Neels Hofmeyr ac2fe707e4 xo/test_configurable_parameters
Change-Id: I15573e801a62f94f0701637562e2d64a212041ca
2026-05-21 13:27:42 +02:00
Neels Hofmeyr ec6f079a94 personalization: EF_SMSP: keep same length as found in template
Change-Id: Id24752101ae82c4986209f4103cc9cbdcce8ce1d
2026-05-21 13:27:42 +02:00
Neels Hofmeyr a9fe3e2fde personalization: fix EF_SMSP length, alpha_id padding
The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 68dd9e6140 saip.batch: log parameter errors
Change-Id: I6a46b2dc9018078ab8361226d1e6b50d3b4e1aaa
2026-05-21 13:27:42 +02:00
Neels Hofmeyr 947a0ab723 personalization: add MncLen ConfigurableParameter
Implement as EnumParam because it only has two possible values

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
2026-05-21 13:27:24 +02:00
Neels Hofmeyr e9c68df3c6 sdkeys kv40 aes
Change-Id: If5b53c840ebd1f224f9bb4706a602b415194f47b
2026-05-12 15:59:40 +02:00
Neels Hofmeyr a2ffe30542 esim/http_json_api.py: support text/plain response Content-Type
Allow returning text/plain Content-Types as 'data' output argument.

So far, all the esim/http_json_api functions require a JSON response.
However, a specific vendor has a list function where the request is JSON
but the response is text/plain CSV data. Allow and return in a dict.

Change-Id: Iba6e4cef1048b376050a435a900c0f395655a790
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 050b08157d Revert "esim/http_json_api: extend JSON API with server functionality"
This reverts commit e00c0becca.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 035c49100b Revert "esim/http_json_api: add missing apidoc"
This reverts commit 0a1c5a27d7.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 2e310ee825 Revert "http_json_api: Only require Content-Type if response body is non-empty"
This reverts commit e0a9e73267.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 41ae4db9bb Revert "esim/http_json_api: add alternative API interface"
This reverts commit f9d7c82b4d.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 6a6dacef51 Revert "esim/http_json_api: add alternative API interface (follow up)"
This reverts commit 8b2a49aa8e.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 270c148343 Revert "esim/http_json_api: allow URL rewriting"
This reverts commit 0634f77308.
2026-05-12 15:59:40 +02:00
Neels Hofmeyr a07e4de853 saip: add numeric_base indicator to ConfigurableParameter and ParamSource
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 6e2498eac0 saip SmspTpScAddr.get_values_from_pes: allow empty values
Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 1173b11ca6 SdKey KVN4X ID02: set key_usage_qual=0x48
Related: SYS#7865
Change-Id: Idc5d33a4a003801f60c95fff6931706a9aeb6692
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 1d79baca35 saip: SdKey.__doc__: update SdKey listing
Change-Id: Ib5011b0c7d76b082231744cf09077628dc4e69b7
2026-05-12 15:59:40 +02:00
Neels Hofmeyr abe4a8f9b1 esim.saip.personalization: fix TLSPSK keys
Add AES variant of TLSPSK DEK (SCP81 KVN40 key_id=0x02).

Change-Id: I713a008fd26bbfcf437e0f29717b753f058ce76a
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 429656dcc6 add comment about not updating existing key_usage_qualifier
Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 74f66fecdb test_configurable_parameters.py: add tests for new parameters
For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
2026-05-12 15:59:40 +02:00
Neels Hofmeyr bfdcdbfc23 saip: SmspTpScAddr: fix get_values_from_pes
Change-Id: I2010305340499c907bb7618c04c61e194db34814
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 87beb049a4 ConfigurableParameter: safer val length check
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
2026-05-12 15:59:40 +02:00
Neels Hofmeyr cf6c47dba2 UppAudit: better indicate exception cause
Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 10d42458a4 remove transitional name mapping
This reverts commit I974cb6c393a2ed2248a6240c2722d157e9235c33

Now, finally, all SdKey classes have a unified logical naming scheme.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 0cab988f47 transitional name mapping
To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
2026-05-12 15:59:40 +02:00
Neels Hofmeyr a259c45e7a generate sdkey classes from a list
Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
2026-05-12 15:59:40 +02:00
Neels Hofmeyr ff02cd8c78 saip SmspTpScAddr: safeguard against decoding error
Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.

The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.

So far I know only of this IndexError, so, as a workaround, catch that.

[1]
  File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
    ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
  File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
    return parse_construct(self._construct, raw_bin_data)
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
    parsed = c.parse(raw_bin_data, total_len=length, **context)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
    return self._decode(obj, context, path)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
    if r[-1] == 'f':
       ~^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
    return hexstr(super().__getitem__(val))
                  ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range

Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 54c202a7ca saip/param_source: try to not repeat random values
Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 257f8c2598 use secrets.SystemRandom as secure random nr source
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 45c88d52b3 use random.SystemRandom as random nr source (/dev/urandom)
/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
2026-05-12 15:59:40 +02:00
Neels Hofmeyr ee09e15e5d add test_param_src.py
Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 4e12d17238 param_source: allow plugging a random implementation (for testing)
Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
2026-05-12 15:59:40 +02:00
Neels Hofmeyr f18cf6d2c7 RandomHexDigitSource: rather return in string format, not bytes
Change-Id: I4e86289f6fb72cbd4cf0c90b8b49538cfab69a7f
2026-05-12 15:59:40 +02:00
Neels Hofmeyr d8768aa7bf personalization: add int as input type for BinaryParameter
Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 72556426f8 personalization.ConfigurableParameter: fix BytesIO() input
Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 8b07326265 add test_configurable_parameters.py
Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 4d23824a81 ConfigurableParameter: do not magically overwrite the 'name' attribute
Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 0a559d2c6e personalization audit: optionally audit all (unknown) SD keys
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
2026-05-12 15:59:40 +02:00
Neels Hofmeyr cbfbe4ac40 personalization: implement UppAudit and BatchAudit
Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 8017025137 param_source: allow input val expansion like '0 * 32'
Working with keys, we often generate 4, 8, 16, 32 digit wide random
values. Those then typically have default input values like

 00000000000000000000000000000000

it is hard for humans to count the number of digits. Much easier:

 00*16

Teach the ParamSource subclasses dealing with random values to
understand an expansion like this. Any expansion is carried out before
all other input value handling.

Use this expansion also in the default_value of ConfigurableParameter
subclasses that have a default_source pointing at a ParamSource that now
understand this expansion.

Related: SYS#6768
Change-Id: Ie7171c152a7b478736f8825050305606b5af5735
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 0e9583f4f7 comment in uicc.py on Security Domain Keys: add SCP81
Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
2026-05-12 15:59:40 +02:00
Neels Hofmeyr 30fa174feb personalization: fix SdKey.apply_val() implementation
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
2026-05-12 15:59:40 +02:00