Audit also all Security Domain KVN that we have *not* created
ConfigurableParameter subclasses for.
For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.
This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.
Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Working with keys, we often generate 4, 8, 16, 32 digit wide random
values. Those then typically have default input values like
00000000000000000000000000000000
it is hard for humans to count the number of digits. Much easier:
00*16
Teach the ParamSource subclasses dealing with random values to
understand an expansion like this. Any expansion is carried out before
all other input value handling.
Use this expansion also in the default_value of ConfigurableParameter
subclasses that have a default_source pointing at a ParamSource that now
understand this expansion.
Related: SYS#6768
Change-Id: Ie7171c152a7b478736f8825050305606b5af5735
Allow omitting some ParamSource subclassed from
ParamSource.get_all_implementations().
My previous attempts at automagically detecting abstract classes failed
conceptually, and the easiest, most explicit way is already used in
ConfigurableParameter: add an is_abstract flag.
Prep for Ie7171c152a7b478736f8825050305606b5af5735
Change-Id: Icfccdd0a8ecb5e0e9d22afa490d73c9f1849a64c
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.
Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)
Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.
Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.
BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:
my_source = MyParam.default_source.from_str( MyParam.default_value )
Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.
For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Implement pySim.esim.saip.personalization.BatchPersonalization,
generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.
Related: SYS#6768
Change-Id: I497c60c101ea0eea980e8b1a4b1f36c0eda39002
Discover all non-abstract subclasses of ConfigurableParameter in
ConfigurableParameter.get_all_implementations().
To be able to automatically discover all practically useful
ConfigurableParameter subclasses, introduce the is_abstract flag.
ConfigurableParameter itself sets is_abstract = True, by default
"hiding" subclasses. As soon as a subclass sets is_abstract = False, it
becomes "public". It depends on the calling code to actually implement
that decision -- this flag enables calling code to do so sanely.
For example, the BinaryParam superclass keeps is_abstract = True,
because per se it isn't capable of doing anything. The fully useful K
and Opc subclasses set is_abstract = False.
Implementation choice: I first tried to query an implicit abstract
status via abc / @abstractmethod ways, but it did not match well. A
subclass has no good implicit indicator, we need a flag instead. For
example, a superclass may provide an apply_val() implementation and
hence appear as non-abstract, but it is still not usable because a
specific 'key' member is still None, which various subclasses set.
Related: SYS#6768
Change-Id: I4970657294130b6b65d50ff19ffbb9ebab3be609
For all ConfigurableParameter subclasses, set a default_value.
This is useful for user interaction, to prefill an input field that
indicates a valid input to modify to taste.
Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".
(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)
Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.
But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().
For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.
Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.
K and Opc use a common abstract BinaryParam.
Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240
Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.
DecimalHexParam will also be used for Pin and Adm soon.
Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods
The new features are optional, and will be heavily used by batch
personalization patches coming soon.
Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.
So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.
Details:
1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)
2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.
Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
Drop the ClassVarMeta/metaclass/ABCMeta stuff -- it doesn't seem to
serve any purpose that is not similarly achieved with plain python
inheritance.
Upcoming patches will use normal inheritance a lot more.
Note that most use of the ClassVarMeta was in the SdKey subclasses, and
that these currently don't actually work. See the fix in patch
I07dfc378705eba1318e9e8652796cbde106c6a52 .
name: set a default name from the python class, as ClassVarMeta did.
Also allow setting an explicit string as name instead, per subclass
implementation (see I31f390d634e58c384589c50a33ca45d6f86d4e10).
Related: SYS#6768
Change-Id: I60ea8fd11fb438ec90ddb08b17b658cbb789c051
When trying to remove a file (e.g. DF.5G_ProSe, 5FF0),
there seems to be a case sensitive check when checking for the dict:
pySim/runtime.py: get_file_for_filename():
478 def get_file_for_filename(self, name: str):
479 """Get the related CardFile object for a specified filename."""
480 sels = self.selected_file.get_selectables()
481 return sels[name]
The dict sels contains 5ff0, but not 5FF0.
The type of argument name is str. So a case sensitive check will be used.
Change-Id: Idd0db1f4bbd3ee9eec20f5fd0f4371c2882950cd
Closes: OS#6898
cmd2 version 3.0 was released, with significant API changes. Limit the
dependency to below 3.0, as already reflected in requirements.txt.
Seeing but not changing the discrepancy in minimum version:
requirements.txt has >2.6.2 while setup.py has >= 1.5.0.
Related: SYS#7775 SYS#7777
Change-Id: I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6
Let's not reinvent the wheel of printing such data structures and use
the repr method provided by the respective class instead. This also
adds the missing key_usage_qualifier information to the print-out,
as well as the mac_len of the key components.
Change-Id: Iaead4a02f07130fd00bcecc43e1c843f1c221e63
The method get_field in the base class can be optimized out. This
also allows us to remove code dup in the card_key_provider_get_field
function.
Let's also fix the return code behavior. A get method in a
CardKeyProvider implementation should always return None in case
nothing is found. Also it should not crash in that case. This will
allow the card_key_provider_get function to move on to the next
CardKeyProvider. In case no CardKeyProvider yields any results, an
exception is appropriate since it is pointless to continue execution
with "None" as key material.
To make the debugging of problems easier, let's also print some debug
messages that inform the user what key/value pair and which
CardKeyProvider was queried. This will make it easier to investigate
in case an expected result was not found.
Related: SYS#7725
Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9
The method _verify_get_data was intended to be used to verify the
user input before it further processed but ended up to be a simple
check that only checks the name of the key column very basically.
Unfortunately it is difficult to generalize the check code as the
concrete implementation of those checks is highly format dependent.
With the advent of eUICCs, we now have two data formats with
different lookup keys, so a static list with valid lookup keys is
also no longer up to the task.
After all it makes not much sense to keep this method, so let's
remove it.
(From the technical perspective, the key column is not limitied to
any specif field. In theory it would even be possible to use the KI
as lookup key as well, even though it would not make sense in
practice)
Related: SYS#7725
Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9
As we plan to support other formats as data source for the Card Key
Provider soon, the more commandline options may be added and it makes
sense to group the Card Key Provider options in a dedicated group.
Let's also rename the option "--csv-column-key" to just "--column-key".
The column encryption is a generic concept and not CSV format specific.
(let's silently keep the "--csv-column-key" argument so maintain backward
compatibility)
Related: SYS#7725
Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c
The Card Key Provider is a built in mechanism of pySim-shell which
allows the user to read key material from a CSV file in order to
avoid having to lookup and enter the key material himself. The
lookup normally done by the pySim-shell commands automatically.
However, in some cases it may also be useful to be able to query the
CSV file manually in order to get certain fields displayed. Such a
command is in particular helpful to check and diagnose the CSV data
source.
Related: SYS#7725
Change-Id: I76e0f883572a029bdca65a5a6b3eef306db1c221
The two properties csv_file and csv_filename are defined by the
constructor anyway, let's remove the declaration in the class body
because it is not needed.
Change-Id: Ibbe8e17b03a4ba0041c0e9990a5e9614388d9c03
let's rename the parameter filename to csv_filename to make it
more clear to what kind of file this parameter refers.
Change-Id: Id5b7c61b5e72fb205e30d2787855b2c276840a7b
It is common in CSV files that the columns have uppercase names, so we
have adopted this scheme when we started using the card_key_provider.
This also means that the API of the card_key_provider_get() and
card_key_provider_get_field() function now implicitly requires
uppercase field names like 'ICCID', 'ADM1', etc.
Unfortunately this may be unreliable, so let's convert the field
names to uppercase as soon as we receive them. This makes the API
case-insensitive and gives us the assurance that all field names
we ever work with are in uppercase.
Related: SYS#7725
Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79
The CardKeyProviderCsv class implements a column decryption scheme
where columns are protected using a transport key. The CSV files
are enrcypted using contrib/csv-encrypt-columns.py.
The current implementation has two main problems:
- The decryption code in CardKeyProviderCsv is not specific to CSV files.
It could be re-used in other formats, for example to decrypt columns
(fields) red from a database. So let's split the decryption code in a
separate class.
- The encryption code in csv-encrypt-columns.py accesses methods and
properties in CardKeyProviderCsv. Also having the coresponding
encryption code somewhere out of tree may be confusing. Let's improve
the design and put encryption and decryption functions in a single
class. Let's also make sure the encryption/decryption is covered by
unittests.
Related: SYS#7725
Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab
The default log level of the PySimLogger is DEBUG by default. This is
to ensure that all messages are printed in an unconfigured setup.
However in pySim-Shell we care about configuring the logger, so let's
set the debug log level to INFO in startup. This will allow us to
turn debug messages on and off using the verbose switch.
Change-Id: I89315f830ce1cc2d573887de4f4cf4e19d17543b
Related: SYS#7725
It's occasionally useful to be able to manually generate a
SGP.22 StoreMetadataRequest (tag BF25), so let's add a small utility
program doing exactly that.
Change-Id: I56ebd040f09dcd167b0b22148c2f1af56240b3b5
If no profileClass is given, ProfileMetadata defaults to operational.
Let's add the capability to also generate metadata for test or provisioning profiles.
Change-Id: Id55537ed03e2690c1fc9545bb3c49cfc76d8e331
There is no unit-test for the CardKeyProviderCsv class yet. Let's add
one to ensure that the CardKeyProviderCsv class keeps working as expected.
Related: SYS#7725
Change-Id: I52519847a4c4a13a7bca49985133872b01c4aaab
The CHV commands (verify_chv, enable_chv, disable_chv, unblock_chv)
provide a --pin-nr parameter.
The --pin-nr is a decimal parameter that specifies the pin type to be
used. The exact pin type numbers are specified in ETSI TS 102.221,
Table 9.3.
Unfortunately the --pin-nr parameter is not very intuitive to use, it
it requires the user to manually lookup the numeric value. The specs
list that value as hexadecimal, so the user also has to convert it
to decimal. To make this less complicated, let's also accept
hexadecimal numbers with the --pin-nr parameter.
However, this alone does not improve the user expierience much. Let's
also add a --pin-type parameter (similar to the --adm-type parameter
of the verify_adm command) to specifiy the pin type in a human
readable form.
Change-Id: I0b58c402d95cbc4fe690e6edb214829d463e9f2c
osmo-smdpp has built-in SSL/TLS support for quite some time now. The manual does not
yet mention this feature yet.
Change-Id: I2db5ae32914386a34eab1ed7d2aff8cae82bfa9b
osmo-smdpp has built-in TLS support for some time now. Let's update
update the commandline help to be more concise.
Since the built-in SSL/TLS support is enabled by default, let's also
update the default port from 8000 to 443.
Change-Id: Ib5a069a8612beb1a9716a7514b498ec70d141178
When writing data to a transparent or linear fixed (record oriented)
and the data to write exceeds the record/file size, then the UICC will
respond with an error "6700: Checking errors - Wrong length"
In particular when the data is supplied as a JSON object and not as a
hex string, it may not be immediately obvious to the average user what
the problem actually is.
Let's check the record/file size before writing the data and raise an
exception in case the data excieeds the record/file size. Let's also
print an informative string message in case the data length is less
than the record/file size to make the user aware of unwritten bytes
at the end of a record/file.
Related: OS#6864
Change-Id: I7fa717d803ae79398d2c5daf92a7336be660c5ad
