Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.
The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.
So far I know only of this IndexError, so, as a workaround, catch that.
[1]
File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
return parse_construct(self._construct, raw_bin_data)
File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
parsed = c.parse(raw_bin_data, total_len=length, **context)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
return self.parse_stream(io.BytesIO(data), **contextkw)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
return self._parsereport(stream, context, "(parsing)")
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
obj = self._parse(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
subobj = sc._parsereport(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
obj = self._parse(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
return self.subcon._parsereport(stream, context, path)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
obj = self._parse(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
subobj = sc._parsereport(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
obj = self._parse(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
return self.subcon._parsereport(stream, context, path)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
obj = self._parse(stream, context, path)
File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
return self._decode(obj, context, path)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
if r[-1] == 'f':
~^^^^
File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
return hexstr(super().__getitem__(val))
~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range
Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.
Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.
For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.
This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.
Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Working with keys, we often generate 4, 8, 16, 32 digit wide random
values. Those then typically have default input values like
00000000000000000000000000000000
it is hard for humans to count the number of digits. Much easier:
00*16
Teach the ParamSource subclasses dealing with random values to
understand an expansion like this. Any expansion is carried out before
all other input value handling.
Use this expansion also in the default_value of ConfigurableParameter
subclasses that have a default_source pointing at a ParamSource that now
understand this expansion.
Related: SYS#6768
Change-Id: Ie7171c152a7b478736f8825050305606b5af5735
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.
Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)
Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.
Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.
BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:
my_source = MyParam.default_source.from_str( MyParam.default_value )
Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.
For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.
Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
this fixes the following two warnings:
pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNode.walk:1: WARNING: Inline strong start-string without end-string. [docutils]
pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNodeDF.walk:1: WARNING: Inline strong start-string without end-string. [docutils]
Change-Id: Id7debf9296923b735f76623808cee68967a1ece7
For all ConfigurableParameter subclasses, provide an example_input.
This may be useful for downstream projects' user interaction, to suggest
a value or prefill an input field, as appropriate.
Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".
(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)
Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.
But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().
For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.
Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.
K and Opc use a common abstract BinaryParam.
Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240
Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.
DecimalHexParam will also be used for Pin and Adm soon.
Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods
The new features are optional, and will be heavily used by batch
personalization patches coming soon.
Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.
So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.
Details:
1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)
2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.
Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
Make sure we make use of the fill pattern when encoding file contents:
Only encode the differences to the fill pattern of the file, in order
to reduce the profile download size.
Change-Id: I61e4a5e04beba5c9092979fc546292d5ef3d7aad
This method did not work at all at the moment, likely due to API churn
over time. This change makes the following exception go away:
Traceback (most recent call last):
File "projects/git/pysim/contrib/saip-tool.py", line 473, in <module>
do_remove_naa(pes, opts)
~~~~~~~~~~~~~^^^^^^^^^^^
File "projects/git/pysim/contrib/saip-tool.py", line 203, in do_remove_naa
pes.remove_naas_of_type(naa)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/__init__.py", line 1748, in remove_naas_of_type
if template in hdr.decoded['eUICC-Mandatory-GFSTEList']:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/oid.py", line 48, in __eq__
return (self.intlist == other.intlist)
^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'intlist'
A subsequent patch should introduce unit tests to avoid such breakage in
the future.
Change-Id: I88d862d751198c3d1648ab7f11d6e6a8fdbc41c9
This adds a new check method to the pySim.esim.saip.validation.CheckBasicStructure
class, which ensures that no unused authentication algorithm related mandatory
services are indicated in the ProfileHeader.
So if a profile e.g. states in the header it requires
usim-test-algorithm, but then the actual akaParameter instances do not
actually use that algorithm, it would raise an exception.
Change-Id: Id0e1988ae1936a321d04bc7c3c3a33262c767d30
Related: SYS#7826
Some templates (e.g. for 5GS) define files which aren't completely defined.
5GS OPL5G: doesn't have a file size defined in the template,
but a record size.
Change-Id: I5ec1757d6852eb24d3662ec1c3fc88365e90a616
Define the file size early if possible.
Some templates (e.g. for 5GS) define files which aren't completely defined.
Fixes the parsing for 5GS SUCI_Calc_Info which doesn't have a file size defined.
The saip-tool will other crash when reading a 5G enabled profile:
```
Traceback (most recent call last):
File "./contrib/saip-tool.py", line 458, in <module>
pes = ProfileElementSequence.from_der(f.read())
File "pySim/esim/saip/__init__.py", line 1679, in from_der
inst.parse_der(der)
~~~~~~~~~~~~~~^^^^^
File "pySim/esim/saip/__init__.py", line 1552, in parse_der
self.pe_list.append(ProfileElement.from_der(first_tlv, pe_sequence=self))
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pySim/esim/saip/__init__.py", line 557, in from_der
inst._post_decode()
~~~~~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 668, in _post_decode
self.pe2files()
~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 655, in pe2files
file = File(k, v, template.files_by_pename.get(k, None))
File "pySim/esim/saip/__init__.py", line 133, in __init__
self.from_tuples(l)
~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 358, in from_tuples
self._body = self.file_content_from_tuples(l)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 393, in file_content_from_tuples
stream.write(self.template.expand_default_value_pattern(self.file_size))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "pySim/esim/saip/templates.py", line 123, in expand_default_value_pattern
raise ValueError("%s does not have a default length" % self)
ValueError: FileTemplate(EF.SUCI_Calc_Info) does not have a default length
```
Change-Id: I7c4a0914aef1049a416e6b091f23daab39a1dd9c
Fixes parsing of a 2.3 UICC profile.
This might be the wrong end as the spec says this is
NSI, but somehow it's working
Change-Id: I3cde1093156db274458d76e2c1c2e304d55a8466
It's a not-too-uncommon requirement to modify the SMSC address stored in
EF.SMSP. This adds a ConfigurableParameter for this purpose.
Change-Id: I6b0776c2e753e0a6d158a8cf65cb030977782ec2
We've had files2pe() for re-encoding all of the files, but let's add
a specific one for re-encoding only one of the files (such as commonly
needed during personalization)
Change-Id: I7b7f61aae6b7df6946dadf2f78fddf92995603ec
Milenage offers the capability for operators to modify the r1-r5
rotation constants as well as the c1-c5 xor-ing constants; let's
add ConfigurableParameters for that.
Change-Id: I397df6c0c708a8061e4adc0fde03a3f746bcb5b6
Related: SYS#7787
