Commit Graph

146 Commits

Author SHA1 Message Date
Neels Hofmeyr
e5f0f07835 personalization: implement reading back values from a PES
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.

Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.

Implementation detail:

Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.

For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.

Rationales for reading back values:

This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).

Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
  I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.

Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
41641979e2 personalization: add param_source.py, implement batch personalization
Implement pySim.esim.saip.personalization.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I497c60c101ea0eea980e8b1a4b1f36c0eda39002
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
6ba0713c14 personalization: discover all useful ConfigurableParameter subclasses
Discover all non-abstract subclasses of ConfigurableParameter in
ConfigurableParameter.get_all_implementations().

To be able to automatically discover all practically useful
ConfigurableParameter subclasses, introduce the is_abstract flag.
ConfigurableParameter itself sets is_abstract = True, by default
"hiding" subclasses. As soon as a subclass sets is_abstract = False, it
becomes "public". It depends on the calling code to actually implement
that decision -- this flag enables calling code to do so sanely.

For example, the BinaryParam superclass keeps is_abstract = True,
because per se it isn't capable of doing anything. The fully useful K
and Opc subclasses set is_abstract = False.

Implementation choice: I first tried to query an implicit abstract
status via abc / @abstractmethod ways, but it did not match well. A
subclass has no good implicit indicator, we need a flag instead. For
example, a superclass may provide an apply_val() implementation and
hence appear as non-abstract, but it is still not usable because a
specific 'key' member is still None, which various subclasses set.

Related: SYS#6768
Change-Id: I4970657294130b6b65d50ff19ffbb9ebab3be609
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
ed3b721669 personalization: set default values
For all ConfigurableParameter subclasses, set a default_value.

This is useful for user interaction, to prefill an input field that
indicates a valid input to modify to taste.

Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
311bd5acd4 personalization: set some typical parameter names
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".

(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)

Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
9605041660 personalization: refactor SdKey
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.

But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().

For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.

Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
ffc2cbdae6 personalization: refactor AlgorithmID, K, Opc
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.

K and Opc use a common abstract BinaryParam.

Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240

Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
5d9abb31a4 personalization: refactor Pin, Adm
Refactor Pin1, Pin2, Adm1 and Adm2 to the new ConfigurableParameter
implementation style.

Change-Id: I54aef10b6d4309398d4b779a3740a7d706d68603
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
e9ee5ae3a2 personalization: refactor Puk
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.

DecimalHexParam will also be used for Pin and Adm soon.

Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
5b5ed97890 personalization: refactor ConfigurableParameter, Iccid, Imsi
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods

The new features are optional, and will be heavily used by batch
personalization patches coming soon.

Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.

So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.

Details:

1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)

2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.

Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
2026-01-07 02:02:38 +01:00
Neels Hofmeyr
6e9625213a fix typo in doc TuakNumberOfKeccak
Change-Id: Ie6f2260d5632dea7409cffd3afa7c8d0b1986a7c
2026-01-07 00:00:21 +01:00
Harald Welte
097d565310 esim.saip: Better docstring about FsNode class
Change-Id: Id9d196e8d9b1d1b892ec50100b170d72d2c3910b
2026-01-06 21:10:29 +00:00
Harald Welte
a8ae89a041 pySim.esim.saip.ProfileElementSequence: Update type annotations
The type annotations didn't reflect reality in two cases.

Change-Id: Ib99c00a38bf009c63180b4a593d6cc796ff282d3
2026-01-06 21:10:19 +00:00
Harald Welte
0fe432fec9 pySim.esim.saip.personalization: Support for EF.SMSP personalization
It's a not-too-uncommon requirement to modify the SMSC address stored in
EF.SMSP.  This adds a ConfigurableParameter for this purpose.

Change-Id: I6b0776c2e753e0a6d158a8cf65cb030977782ec2
2025-12-23 20:57:03 +01:00
Harald Welte
c6fd1d314a esim.saip.FsProfileElement: Add file2pe() for single file conversion
We've had files2pe() for re-encoding all of the files, but let's add
a specific one for re-encoding only one of the files (such as commonly
needed during personalization)

Change-Id: I7b7f61aae6b7df6946dadf2f78fddf92995603ec
2025-12-23 20:57:01 +01:00
Harald Welte
ddbf91fc4a pySim.esim.saip.personalization: Support Milenage customization
Milenage offers the capability for operators to modify the r1-r5
rotation constants as well as the c1-c5 xor-ing constants; let's
add ConfigurableParameters for that.

Change-Id: I397df6c0c708a8061e4adc0fde03a3f746bcb5b6
Related: SYS#7787
2025-12-18 14:42:52 +01:00
Harald Welte
11dfad88e6 pySim.esim.saip: Fix compatibility with pytohn < 3.11
In python up to 3.10, the byteorder argument of the int.to_bytes()
method was mandatory, even if the length of the target byte sequence
is 1 and there factually is no byteorder.

https://docs.python.org/3.10/library/stdtypes.html#int.to_bytes
vs
https://docs.python.org/3.11/library/stdtypes.html#int.to_bytes

See also: https://discourse.osmocom.org/t/assistance-required-with-saip-pysim-tool-error-while-adding-applets-to-exiting-upp-der/2413/2

Change-Id: I8ba29d42c8d7bf0a36772cc3370eff1f6afa879f
2025-12-14 13:58:34 +01:00
Harald Welte
c07ecbae52 pySim.esim.saip: Hex representation of SecurityDomainKey
Let's print the key_usage_qualifier in hexadecimal notation (more compact)

Change-Id: Ic9a92d53d73378eafca1760dd8351215bce1157a
2025-12-09 16:23:49 +00:00
Harald Welte
52df66cd56 pySim.esim.es8p: Support non-operational ProfileMetadata
If no profileClass is given, ProfileMetadata defaults to operational.
Let's add the capability to also generate metadata for test or provisioning profiles.

Change-Id: Id55537ed03e2690c1fc9545bb3c49cfc76d8e331
2025-11-21 11:41:29 +00:00
Harald Welte
947154639c pySim.esim.saip.FsNodeADF: Fix __str__ method
It's quite common for a FsNodeADF to not have a df_name, so we need
to guard against that during stringification to avoid an exception.

Change-Id: I919d7c46575e0ebcdf3b979347a5cdd1a9feb294
2025-09-24 17:59:17 +00:00
Kian-Meng Ang
4ee99c18cd Fix typos
Found via `codespell -S tests -L ist,adn,ciph,ue,ot,readd,te,oce,tye`

Change-Id: I00a72e4f479dcef88f7d1058ce53edd0129d336a
2025-09-24 17:59:17 +00:00
Eric Wild
5d2e2ee259 bsp: fix maxpayloadsize
Change-Id: I08f544877b79681ad1f758a1ca31c292eae9f868
2025-09-24 15:04:36 +00:00
Eric Wild
6cffb31b42 memory backed ephermal session store for easy concurrent runs
Change-Id: I05bfd6ff471ccf1c8c2b5f2b748b9d4125ddd4f7
2025-08-15 13:04:02 +02:00
Eric Wild
7798ea9c5c x509 cert: fix weird cert check
Change-Id: I18beab0e1b24579724704c4141a2c457b2d4cf99
2025-08-15 13:04:02 +02:00
Eric Wild
0b1d3c85fd smdpp: less verbose by default
Those data blobs are huge.

Change-Id: I04a72b8f52417862d4dcba1f0743700dd942ef49
2025-08-15 13:04:02 +02:00
Harald Welte
e37cdbcd3e docs: Better python doc-strings for better pySim.esim manual
Change-Id: I7be6264c665a2a25105681bb5e72d0f6715bbef8
2025-05-07 10:50:47 +02:00
Philipp Maier
d5da431fd4 saip-tool: add commandline option to edit mandatory services list
Change-Id: I120b98d4b0942c26674bc1365c5711101ec95235
2025-04-16 10:35:15 +02:00
Philipp Maier
1dea0f39dc saip-tool: add features to add, remove and inspect application PEs
The PE-Application object is used to provision JAVA-card applications
into an eUICC during profile installation. Let's extend the SAIP-tool
so that we are able to add, remove and inspect applications.

Change-Id: I41db96f2f0ccc29c1725a92215ce6b17d87b76ce
2025-04-14 11:01:24 +00:00
Philipp Maier
40e795a825 saip-tool: add ProfileElement class for application PE
The application profile element has no ProfileElement class yet, so
let's create a ProfileElementApplication class and move the existing
extract-apps code into a method of ProfileElementApplication.

Change-Id: Iaa43036d388fbf1714c53cab1fc21092c4667a21
2025-03-31 12:27:24 +02:00
Philipp Maier
8da8b20f58 es8p: fix typo
Change-Id: I241efe0c7ceab190b7729a6d88101501ca37652e
2025-03-10 19:16:20 +00:00
Neels Hofmeyr
19e1330ce8 pylint: personalization.py: fix E1135: permitted_len unsupported-membership-test
pre-empt this from coming up in patch
I60ea8fd11fb438ec90ddb08b17b658cbb789c051:

E1135: Value 'self.permitted_len' doesn't support membership test (unsupported-membership-test) pickermitted_len

Change-Id: I0343f8dbbffefb4237a1cb4dd40b576f16111073
2025-03-07 21:26:54 +01:00
Harald Welte
0bb8b44ea8 esim.saip.ProfileElementUSIM: Fix IMSI decode if [only] template based
In case the fileDescriptor of EF.IMSI is purely template based and only
the file content is given in the actual profile, we must pass a template
reference to the File() constructor before we can read the IMSI.

This fixes the following exception for some profiles:
	ValueError: File(ef-imsi): No fileDescriptor found in tuple, and none set by template before

Change-Id: I14157a7b62ccd9b5b42de9b8060f2ebc5f91ebb3
2024-11-23 15:43:12 +01:00
Harald Welte
9d7caef810 esim.saip.FsProfileElement: Add create_file() method
So far we mainly created File() instances when parsing existing
profiles.  However, sometimes we want to programmatically create Files
and we should offer a convenience helper to do so, rather than asking
API users to worry about low-level details.

Change-Id: I0817819af40f3d0dc0c3d2b91039c5748dd31ee2
2024-11-22 21:02:35 +01:00
Harald Welte
9ac4ff3229 esim.saip.File: Suppress encoding attributes that are like template
The point of the SAIP template mechanism is to reduce the size of the
encoded profile.  Therefore, our encoder in the to_fileDescriptor()
method should suppress generating attributes if their value is identical
to that of the template (if any).

Change-Id: I337ee6c7e882ec711bece17b7a0def9da36b0ad7
2024-11-22 21:00:47 +01:00
Harald Welte
0f1ffd20ef esim.saip.File: Proper ARR conversion of template (into) to file (bytes)
The encoding of the access rule reference is different in FileTemplate
vs File, let's make sure we properly convert it when instantiating a
File from a FileTemplate.

Change-Id: Ibb8afb85cc0006bc5c59230ebf28b2c0c1a8a8ed
2024-11-22 20:59:19 +01:00
Harald Welte
0516e4c47a esim.saip.File: Re-compute file_size when changing body
If the API user modifies the size of the body, we need to check if we
need to re-compute the file_size attribute which is later encoded into
the fileDescriptor.  The size obviously must be large enough to fit the
body.  Let's do this implicitly by introducing a setter for File.body

Change-Id: I1a908504b845b7c90f31294faf2a6e988bdd8049
2024-11-22 20:56:58 +01:00
Harald Welte
3442333760 esim.saip: New methods for inserting ProfileElement into sequence
ProfileElements.insert_after_pe() is a convenience method to insert
a new PE after an existing one in the sequence.  This is a frequent
task as there are strict ordering requirements in the SAIP format.

Change-Id: I4424926127b4867931c2157e9340bacd2682ff0c
2024-11-22 20:49:24 +01:00
Harald Welte
5354fc22d0 [cosmetic] esim: Fix various typos in comments/messages/docs
Change-Id: I806c7a37951e72027ab9346169a3f8fe241f2c46
2024-11-22 17:04:30 +01:00
Harald Welte
93237f4407 [cosmetic] esim.saip: Fix various typos in comments/docs/messages
Change-Id: I4fc603634a0f2b53e432a77f05e811a38ba065c2
2024-11-22 16:59:26 +01:00
Harald Welte
779092b0cd esim.saip: Fix computation of file content
When generating the file content (body), we need to proceed in the
following order:

1a) If FCP contains fillPattern/repeatPattern, compute file content from those

1b) If FCP doesn't contain fillPattern/repeatPattern but template
    exists, compute file content from template

2)  Apply any fillFileConten / fillFileOffset from the SAIP File on top
    of the above

Change-Id: I822bb5fbec11a3be35910a496af7168458fd949c
Closes: OS#6642
2024-11-22 16:03:58 +01:00
Harald Welte
6046102cbb esim.saip: Compute number of records from efFileSize and record_len
If we know the efFileSize and record_len, but Fcp doesn't contain
the number of records, we can simply compute it.

Change-Id: I0cc8e7241e37ee23df00c2622422904e7ccdca77
2024-11-22 16:01:58 +01:00
Harald Welte
118624d256 pySim.esim.saip: Treat "Readable and Updateable when deactivated" flag
There's a second flag hidden in the TS 102 222 "Special File
Information"; let's parse + re-encode it properly.

Change-Id: I7644d265f746c662b64f7156b3be08a01e3a97aa
Related: OS#6643
2024-11-22 16:01:58 +01:00
Harald Welte
599845394e esim.saip: Fix parsing/generating fillPattern + repeatPattern
So far we only thought of default filling coming from a template.
However, filling can happen from the Fcp, and we need to properly parse
and [re-]encode that information.

Change-Id: Iff339cbe841112a01c9c617f43b0e69df2521b51
Related: OS#6643
2024-11-22 16:01:25 +01:00
Harald Welte
f9631fb361 pySim.esim.saip.templates: Fix DF_TELECOM FileID (7F10, not 7F11)
Change-Id: I4bc37f9d99c046cd6c6accaaf460a39eb79b660f
2024-10-20 10:14:34 +02:00
Harald Welte
6aabb92c38 esim.saip.templates: Fix expand_default_value_pattern for length==0
The original code treated length==0 like length==None (unspecified),
which is wrong.

Change-Id: I39fa1e2b1b9d6d1c671ea37bdbec1d6f97e8a5e7
2024-09-03 21:57:47 +02:00
Harald Welte
b22bab0b20 pySim.esim.saip.ProfileElementGFM: Initialize 'fileManagementCMD'
When constructing a ProfileElmentGFM from scratch, initialize the
decoded['fileManagementCMD'], as it is a mandatory member during
ASN.1 encode.

Change-Id: Iaae99348d36b7f0c739daf039d6ea2305b7ca9db
2024-09-03 21:57:47 +02:00
Harald Welte
981220641d pySim.esim.saip.File: Turn file_size into a computed property
This way, we can use file_size for both record-oriented and transparent EF

Change-Id: Ib787cabe969202073a8c10042e200f3d2c29db73
2024-09-03 21:57:47 +02:00
Harald Welte
73dd3d0637 pySim.esim.saip: Add missing initialization of File.df_name
Change-Id: Iaf596a8914850ccae584c3b78dc7711db736ac80
2024-09-03 21:57:47 +02:00
Harald Welte
65cbe48953 pySim.esim.saip: Another naming irregularity.
The choice member is called df-5gprose but the header is called
'df-5g-prose-header' (note the '-' between '5g' and 'prose'). WTF.

Change-Id: I86004ac2e18a187c26c5e470344908512d21fb9e
2024-09-03 21:57:47 +02:00
Harald Welte
52735f3685 pySim.esim.saip: Fix weird DF names
Sometimes the struct member is called like df-telecom, but in other
cases it's called df-df-saip  with a double 'df' in front.  That makes
no sense, but we have to deal with it from our constructors...

Change-Id: If5e670441f03a47fa34e97a326909b24927c12f7
2024-09-03 21:57:47 +02:00