Commit Graph

1730 Commits

Author SHA1 Message Date
Neels Hofmeyr
d28cf0a05e tweak test_configurable_parameters.py: add iff_present flag
apply a parameter only when it exists in the template, will be useful
for suci

Change-Id: I5811ecde4c4e880bb8dbd22fffe23faafcfe36ad
2026-04-25 06:13:02 +02:00
Neels Hofmeyr
a1a85c3214 tweak test_configurable_parameters.py: show value found in template
Change-Id: If9039bbb8547ee24ae784a932f60cd5de6c9247b
2026-04-25 06:06:37 +02:00
Neels Hofmeyr
f3760f7572 tweak test_configurable_parameters.py: saner output composition.
Change-Id: Id3e3b46b2b3d7919a75c620803ce28d2a715008b
2026-04-25 06:01:21 +02:00
Neels Hofmeyr
8b91249781 xo/test_configurable_parameters
Change-Id: I15573e801a62f94f0701637562e2d64a212041ca
2026-04-25 05:59:32 +02:00
Neels Hofmeyr
cc3f99b472 personalization: EF_SMSP: keep same length as found in template
Change-Id: Id24752101ae82c4986209f4103cc9cbdcce8ce1d
2026-04-23 23:50:12 +02:00
Neels Hofmeyr
d8c3d55c20 personalization: fix EF_SMSP length, alpha_id padding
The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
8333d6a340 saip: MncLen: fix for missing EF-AD
Change-Id: I7f5ce22d23808c91234c6a75b8a22264d3c5bc92
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
c28abecf8c saip.batch: log parameter errors
Change-Id: I6a46b2dc9018078ab8361226d1e6b50d3b4e1aaa
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
5fac39bb51 MncLen
Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
61357d223e sdkeys kv40 aes
Change-Id: If5b53c840ebd1f224f9bb4706a602b415194f47b
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
66acb109ab esim/http_json_api.py: support text/plain response Content-Type
Allow returning text/plain Content-Types as 'data' output argument.

So far, all the esim/http_json_api functions require a JSON response.
However, a specific vendor has a list function where the request is JSON
but the response is text/plain CSV data. Allow and return in a dict.

Change-Id: Iba6e4cef1048b376050a435a900c0f395655a790
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
0c0a395f00 Revert "esim/http_json_api: extend JSON API with server functionality"
This reverts commit e00c0becca.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
ab07954999 Revert "esim/http_json_api: add missing apidoc"
This reverts commit 0a1c5a27d7.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
907c29735e Revert "http_json_api: Only require Content-Type if response body is non-empty"
This reverts commit e0a9e73267.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
e6aef652b0 Revert "esim/http_json_api: add alternative API interface"
This reverts commit f9d7c82b4d.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
09fbfdc39e Revert "esim/http_json_api: add alternative API interface (follow up)"
This reverts commit 8b2a49aa8e.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
6d0b3f8b85 Revert "esim/http_json_api: allow URL rewriting"
This reverts commit 0634f77308.
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
e0ac64d501 saip: add numeric_base indicator to ConfigurableParameter and ParamSource
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
f66b6fcc5b saip SmspTpScAddr.get_values_from_pes: allow empty values
Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
6f04e5e400 SdKey KVN4X ID02: set key_usage_qual=0x48
Related: SYS#7865
Change-Id: Idc5d33a4a003801f60c95fff6931706a9aeb6692
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
3470a3e062 saip: SdKey.__doc__: update SdKey listing
Change-Id: Ib5011b0c7d76b082231744cf09077628dc4e69b7
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
696da34e81 esim.saip.personalization: fix TLSPSK keys
Add AES variant of TLSPSK DEK (SCP81 KVN40 key_id=0x02).

Change-Id: I713a008fd26bbfcf437e0f29717b753f058ce76a
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
42b8a70085 add comment about not updating existing key_usage_qualifier
Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
61264e32b8 test_configurable_parameters.py: add tests for new parameters
For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
659cb5d6c4 saip: SmspTpScAddr: fix get_values_from_pes
Change-Id: I2010305340499c907bb7618c04c61e194db34814
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
c20b979875 ConfigurableParameter: safer val length check
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
ddb5e3168e UppAudit: better indicate exception cause
Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
ebeb00b48d remove transitional name mapping
This reverts commit I974cb6c393a2ed2248a6240c2722d157e9235c33

Now, finally, all SdKey classes have a unified logical naming scheme.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
69a74eb930 transitional name mapping
To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
c00813f66f generate sdkey classes from a list
Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
0b8b59e74f saip SmspTpScAddr: safeguard against decoding error
Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.

The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.

So far I know only of this IndexError, so, as a workaround, catch that.

[1]
  File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
    ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
  File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
    return parse_construct(self._construct, raw_bin_data)
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
    parsed = c.parse(raw_bin_data, total_len=length, **context)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
    return self._decode(obj, context, path)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
    if r[-1] == 'f':
       ~^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
    return hexstr(super().__getitem__(val))
                  ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range

Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
09f2e50bd6 saip/param_source: try to not repeat random values
Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
c712ecbb0e use secrets.SystemRandom as secure random nr source
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
69218d135f use random.SystemRandom as random nr source (/dev/urandom)
/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
405ccabafd add test_param_src.py
Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
86fecce1db param_source: allow plugging a random implementation (for testing)
Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
a24dd6ac4a RandomHexDigitSource: rather return in string format, not bytes
Change-Id: I4e86289f6fb72cbd4cf0c90b8b49538cfab69a7f
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
350aa9e01f personalization: add int as input type for BinaryParameter
Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
167783ef0a personalization.ConfigurableParameter: fix BytesIO() input
Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
5dbe1d3360 add test_configurable_parameters.py
Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
ed5c032f76 ConfigurableParameter: do not magically overwrite the 'name' attribute
Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
f677a99471 personalization audit: optionally audit all (unknown) SD keys
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
566a578a63 personalization: implement UppAudit and BatchAudit
Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
475d98dcea param_source: allow input val expansion like '0 * 32'
Working with keys, we often generate 4, 8, 16, 32 digit wide random
values. Those then typically have default input values like

 00000000000000000000000000000000

it is hard for humans to count the number of digits. Much easier:

 00*16

Teach the ParamSource subclasses dealing with random values to
understand an expansion like this. Any expansion is carried out before
all other input value handling.

Use this expansion also in the default_value of ConfigurableParameter
subclasses that have a default_source pointing at a ParamSource that now
understand this expansion.

Related: SYS#6768
Change-Id: Ie7171c152a7b478736f8825050305606b5af5735
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
a0ceb319e4 comment in uicc.py on Security Domain Keys: add SCP81
Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
f091d53cff personalization: fix SdKey.apply_val() implementation
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
f45fb769e1 personalization: add get_typical_input_len() to ConfigurableParameter
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.

Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
e293227ead personalization: make AlgorithmID a new EnumParam
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.

Implement the enum value part abstractly in new EnumParam.

Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn

Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
a3a124acea personalization: indicate default ParamSource per ConfigurableParameter
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.

This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.

For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.

BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:

  my_source = MyParam.default_source.from_str( MyParam.default_value )

Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
2026-04-22 20:12:13 +02:00
Neels Hofmeyr
f4dd30dcf4 personalization: allow reading back multiple values from PES
Change-Id: Iecb68af7c216c6b9dc3add469564416b6f37f7b2
2026-04-22 20:12:13 +02:00