'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.
Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)
Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.
Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.
BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:
my_source = MyParam.default_source.from_str( MyParam.default_value )
Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.
For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Implement pySim.esim.saip.personalization.BatchPersonalization,
generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.
Related: SYS#6768
Change-Id: I497c60c101ea0eea980e8b1a4b1f36c0eda39002
Discover all non-abstract subclasses of ConfigurableParameter in
ConfigurableParameter.get_all_implementations().
To be able to automatically discover all practically useful
ConfigurableParameter subclasses, introduce the is_abstract flag.
ConfigurableParameter itself sets is_abstract = True, by default
"hiding" subclasses. As soon as a subclass sets is_abstract = False, it
becomes "public". It depends on the calling code to actually implement
that decision -- this flag enables calling code to do so sanely.
For example, the BinaryParam superclass keeps is_abstract = True,
because per se it isn't capable of doing anything. The fully useful K
and Opc subclasses set is_abstract = False.
Implementation choice: I first tried to query an implicit abstract
status via abc / @abstractmethod ways, but it did not match well. A
subclass has no good implicit indicator, we need a flag instead. For
example, a superclass may provide an apply_val() implementation and
hence appear as non-abstract, but it is still not usable because a
specific 'key' member is still None, which various subclasses set.
Related: SYS#6768
Change-Id: I4970657294130b6b65d50ff19ffbb9ebab3be609
For all ConfigurableParameter subclasses, set a default_value.
This is useful for user interaction, to prefill an input field that
indicates a valid input to modify to taste.
Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".
(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)
Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.
But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().
For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.
Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.
K and Opc use a common abstract BinaryParam.
Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240
Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.
DecimalHexParam will also be used for Pin and Adm soon.
Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods
The new features are optional, and will be heavily used by batch
personalization patches coming soon.
Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.
So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.
Details:
1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)
2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.
Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
Drop the ClassVarMeta/metaclass/ABCMeta stuff -- it doesn't seem to
serve any purpose that is not similarly achieved with plain python
inheritance.
Upcoming patches will use normal inheritance a lot more.
Note that most use of the ClassVarMeta was in the SdKey subclasses, and
that these currently don't actually work. See the fix in patch
I07dfc378705eba1318e9e8652796cbde106c6a52 .
name: set a default name from the python class, as ClassVarMeta did.
Also allow setting an explicit string as name instead, per subclass
implementation (see I31f390d634e58c384589c50a33ca45d6f86d4e10).
Related: SYS#6768
Change-Id: I60ea8fd11fb438ec90ddb08b17b658cbb789c051
When trying to remove a file (e.g. DF.5G_ProSe, 5FF0),
there seems to be a case sensitive check when checking for the dict:
pySim/runtime.py: get_file_for_filename():
478 def get_file_for_filename(self, name: str):
479 """Get the related CardFile object for a specified filename."""
480 sels = self.selected_file.get_selectables()
481 return sels[name]
The dict sels contains 5ff0, but not 5FF0.
The type of argument name is str. So a case sensitive check will be used.
Change-Id: Idd0db1f4bbd3ee9eec20f5fd0f4371c2882950cd
Closes: OS#6898
The method get_field in the base class can be optimized out. This
also allows us to remove code dup in the card_key_provider_get_field
function.
Let's also fix the return code behavior. A get method in a
CardKeyProvider implementation should always return None in case
nothing is found. Also it should not crash in that case. This will
allow the card_key_provider_get function to move on to the next
CardKeyProvider. In case no CardKeyProvider yields any results, an
exception is appropriate since it is pointless to continue execution
with "None" as key material.
To make the debugging of problems easier, let's also print some debug
messages that inform the user what key/value pair and which
CardKeyProvider was queried. This will make it easier to investigate
in case an expected result was not found.
Related: SYS#7725
Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9
The method _verify_get_data was intended to be used to verify the
user input before it further processed but ended up to be a simple
check that only checks the name of the key column very basically.
Unfortunately it is difficult to generalize the check code as the
concrete implementation of those checks is highly format dependent.
With the advent of eUICCs, we now have two data formats with
different lookup keys, so a static list with valid lookup keys is
also no longer up to the task.
After all it makes not much sense to keep this method, so let's
remove it.
(From the technical perspective, the key column is not limitied to
any specif field. In theory it would even be possible to use the KI
as lookup key as well, even though it would not make sense in
practice)
Related: SYS#7725
Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9
The two properties csv_file and csv_filename are defined by the
constructor anyway, let's remove the declaration in the class body
because it is not needed.
Change-Id: Ibbe8e17b03a4ba0041c0e9990a5e9614388d9c03
let's rename the parameter filename to csv_filename to make it
more clear to what kind of file this parameter refers.
Change-Id: Id5b7c61b5e72fb205e30d2787855b2c276840a7b
It is common in CSV files that the columns have uppercase names, so we
have adopted this scheme when we started using the card_key_provider.
This also means that the API of the card_key_provider_get() and
card_key_provider_get_field() function now implicitly requires
uppercase field names like 'ICCID', 'ADM1', etc.
Unfortunately this may be unreliable, so let's convert the field
names to uppercase as soon as we receive them. This makes the API
case-insensitive and gives us the assurance that all field names
we ever work with are in uppercase.
Related: SYS#7725
Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79
The CardKeyProviderCsv class implements a column decryption scheme
where columns are protected using a transport key. The CSV files
are enrcypted using contrib/csv-encrypt-columns.py.
The current implementation has two main problems:
- The decryption code in CardKeyProviderCsv is not specific to CSV files.
It could be re-used in other formats, for example to decrypt columns
(fields) red from a database. So let's split the decryption code in a
separate class.
- The encryption code in csv-encrypt-columns.py accesses methods and
properties in CardKeyProviderCsv. Also having the coresponding
encryption code somewhere out of tree may be confusing. Let's improve
the design and put encryption and decryption functions in a single
class. Let's also make sure the encryption/decryption is covered by
unittests.
Related: SYS#7725
Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab
If no profileClass is given, ProfileMetadata defaults to operational.
Let's add the capability to also generate metadata for test or provisioning profiles.
Change-Id: Id55537ed03e2690c1fc9545bb3c49cfc76d8e331
When writing data to a transparent or linear fixed (record oriented)
and the data to write exceeds the record/file size, then the UICC will
respond with an error "6700: Checking errors - Wrong length"
In particular when the data is supplied as a JSON object and not as a
hex string, it may not be immediately obvious to the average user what
the problem actually is.
Let's check the record/file size before writing the data and raise an
exception in case the data excieeds the record/file size. Let's also
print an informative string message in case the data length is less
than the record/file size to make the user aware of unwritten bytes
at the end of a record/file.
Related: OS#6864
Change-Id: I7fa717d803ae79398d2c5daf92a7336be660c5ad
In many sub modules we still use print() to occassionally print status
messages or warnings. This technically does not hurt, but it is an unclean
solution which we should replace with something more mature.
Let's use python's built in logging framework to create a static logger
class that fits our needs. To maintain compatibility let's also make sure
that the logger class will behave like a normal print() statement when no
configuration parameters are supplied by the API user.
To illustrate how the approach can be used in sub-modules, this patch
replaces the print statements in runtime.py. The other print statements
will the be fixed in follow-up patches.
Related: OS#6864
Change-Id: I187f117e7e1ccdb2a85dfdfb18e84bd7561704eb
The function h2b expects a bytearray and must not be used on a string.
This is also true for nullstrings ('').
Related: OS#6869
Change-Id: I0e28e6ec476901bf19aa0f8640e41c74aa6e3aa2
It's quite common for a FsNodeADF to not have a df_name, so we need
to guard against that during stringification to avoid an exception.
Change-Id: I919d7c46575e0ebcdf3b979347a5cdd1a9feb294
- try to identify the CardModel by just comparing the Historical Bytes if matching by Whole ATR failed
- add decompose ATR code from pyscard-contrib
Related: OS#6837
Change-Id: Id7555e42290d232a0e0efc47e7d97575007d846f
With the introduction of using osmocom.construct.{Bytes,GreedyBytes}
in Change-Id I1c8df6350c68aa408ec96ff6cd1e405ceb1a4fbb we don't have a
need for wrapping each instance of Bytes or GreedyBytes into a
HexAdapter anymore. The osmocom.construct.{Bytes,GreedyBytes} will
automatically perform the related hex-string-to-bytes conversion if
needed - and during printing we have osmocom.utils.JsonEncoder that
makes sure to convert any bytes type to a hex-string.
Change-Id: I9c77e420c314f5e74458628dc4e767eab6d97123
The upstream construct.{Bytes,GreedyBytes} only support bytes/bytearray
input data for the encoder, while the [newly-created]
osmocom.construct.{Bytes,GreedyBytes} support alternatively hex-string input.
This is important in the context of encoding construct-based types from
JSON, where our osmocom.utils.JsonEncoder will automatically convert any
bytes to hex-string, while re-encoding those hex-strings will fail prior
to this patch.
Change-Id: I1c8df6350c68aa408ec96ff6cd1e405ceb1a4fbb
Closes: OS#6774
We've created + used osmocom.utils.JsonEncoder as an encoder class
for json.{dump,dumps} for quite some time. However, we missed to
use this decoder class from the edit_{binary,record}_decoded commands
in the pySim-shell VTY.
Change-Id: I158e028f9920d8085cd20ea022be2437c64ad700
Related: OS#6774
Use the 'r' (raw) qualifier to avoid rendering '\n' as the actual
line break in the auto-generated documentation.
Change-Id: Ie7f59685a78534eb2c43ec4bc39685d3fd264778
Recent versions of the ARA-M applet from Bertrand Martel can lock
the write access to ARA-M rules. Let's add a command for that and
some documentation.
Related: SYS#7245
Change-Id: I71581a0c9f146f9a0921093d9b53b053b4a8946c
The PE-Application object is used to provision JAVA-card applications
into an eUICC during profile installation. Let's extend the SAIP-tool
so that we are able to add, remove and inspect applications.
Change-Id: I41db96f2f0ccc29c1725a92215ce6b17d87b76ce
The pySim-smpp2sim.py program exposes two interfaces:
* SMPP server-side port, so external programs can rx/tx SMS
* APDU interface towards the SIM card
It therefore emulates the SMSC, Core Network, RAND and UE parts
that would normally be encountered in an OTA setup.
Change-Id: Ie5bae9d823bca6f6c658bd455303f63bace2258c
The application profile element has no ProfileElement class yet, so
let's create a ProfileElementApplication class and move the existing
extract-apps code into a method of ProfileElementApplication.
Change-Id: Iaa43036d388fbf1714c53cab1fc21092c4667a21
