apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field. After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.
Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
padding) so the re-encoded body carries 0xFF-padded alpha_id space
and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
alpha_id area is actually padded to that length
Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.
Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.
Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test
Python's random module uses a PRNG (Mersenne Twister) which is
utterly insecure for key generation - it was so far only used for
testing. Replace it with random.SystemRandom(), which draws from
/dev/urandom and is suitable for generating cryptographic key material.
Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test
Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).
Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.
Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.
Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.
Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
The ClassVarMeta metaclass used to derive each ConfigurableParameter's
'name' attribute automatically from the Python class name (via
camel_to_snake()). Stop doing this, for three reasons:
1) Python class names follow constraints that do not fit the naming
commonly used in CSV files. For example, a name like
"5GS-SUCI-CalcInfo" starts with a digit and contains dashes,
neither of which is permissible in a class name.
2) Python class names live in their own namespace, distinct from the
one used to present eSIM parameters to end users. Deriving the UI
name from the class name couples these two namespaces together.
Taken together, (1) and (2) mean that automatic naming both imposes
class-name constraints on the user-visible names and merges the
internal Python namespace with the publicly shown one - a layer
violation from the perspective of UI design.
3) Overriding 'name' from __new__() makes manual naming impossible: a
subclass that sets 'name = "bar"' as a class attribute would still
end up with the value computed by the metaclass, which is
surprising and hard to track down:
class MySuper(metaclass=...): # __new__ sets name = 'foo'
...
class MySub(MySuper):
name = 'bar'
print(MySub().name) # 'foo', not 'bar' as one would expect
Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.
For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x04..0x0f in an audit.
This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.
Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test
When the access conditions are extracted from resp_bin, the wrong length
is used and only 2 bytes instead of 3 are extracted.
3GPP TS 51.011, section 9.2.1, table below "Response parameters/data
in case of an EF", clearly states that the length should be 3 bytes
(position 9-11)
Related: OS#7018
Change-Id: I410fb58c395beafba8de6d5ab4e71452f424cdf2
The comment reads like that we were applying TS 102.221 here, but we only
mean our internal decoding format. The spec that actually matters here is
TS 51.011. Let's rephrase the comment so that this becomes more clear.
Related: OS#7018
Change-Id: Ie0184eea25f4d9f4baf9ab137c53a926edba2bf8
The commandline options --nfc-always and --nfc-never appear in the
opposite order when compared to --apdu-never and --apdu-always.
Let's swap the options to make the helpscreen and the code more
consistent.
Change-Id: I7289c3628b1b8dd3eec2f1c8f2132e3015422960
Related: SYS#6959
The module scp.py predates the existence of the pySimLogger and still
uses an individually created logger. Let's migrate to pySimLogger to
avoid unexpected effects and to be uniform with the other modules.
Related: SYS#6959
Change-Id: I5db7180f93f116dd2d99c33da264f74ea16a1a37
In format string we prepend when we log in verbose mode. We use %(module)s
as format string quaifier. This qualifier is replaced with the name of the
module from where the logger was called. This is mostly equal to the logger
name (__name__) we pass when we create the logger.
However, this is not the behavior we actually want. We want to log the
logger name that we passed when the logger was created. For this, we must
use %(name)s as qualifier.
Related: SYS#6959
Change-Id: I3951a70ad6ce864a7158b093cba46ae9fc1cb5bd
The contents of the --column-keys arguments are currently parsed
in init_card_key_provider. Let's add a static method in
CardKeyFieldCryptor to simplify re-usage of the CardKeyFieldCryptor
Related: SYS#6959
Change-Id: Ic955f271b1de1b1b855b21c82ed10343044e45fa
We currently create the CardKeyFieldCryptor object inside the constructor
of the concrete CardKeyProvider classes. There is currently no problem
with that, but when we create the CardKeyFieldCryptor object first and
then pass it as parameter to the constructor, we gain more flexibility
in case we want to support other CardKeyFieldCryptor variants in the
future.
Related: SYS#6959
Change-Id: If43552740aadacab9126f8a002749a9582eef8f4
in pySim-shell.py we add the commandline options for the card key
provider and do the setup accordingly. Let's put this boilerplate
code into helper functions instead, so that we can re-use it in
other pySim programs as well. Let's use pySim.transport as a
pattern.
Related: SYS#6959
Change-Id: I6d095cbb644e608f4a751a1d0749b1484cdc781d
The field 'alpha_id' is technically not an optional field, even though
the specification describes it as optional. Once the card manufacturer
decides that the field should be present, it must be always present and
vice versa.
(see code comment for a more detailed description)
Related: SYS#7765
Change-Id: I0ec99b2648b22c56f9145345e4cd8776f9217701
The legacy code found in legacy/cards.py does not use the modern
construct based encoder (pySim-read uses it). The card classes either
use their own implementation of update_smsp or use the generic method
provided by the SimCard class. The latter one is true for FairwavesSIM
and WavemobileSim.
Unfortunately the implementation found in the SimCard is wrong. It
adds padding at the end of the file instead of the beginning. This
completely messes up the contents of EF.SMSP for the cards using this
method. To fix this, let's use the leftpad feature provided by
the update_record. This will ensure a correct alignment of the file
contents.
Related: SYS#7765
Change-Id: Ie112418f1f1461762d61365d3863181ca6be7245
The current behavior we implement in the method __send_apdu_T0 is
incomplete. Some details discussed in ETSI TS 102 221,
section 7.3.1.1.4, clause 4 seem to be not fully implemented. We
may also end up sending a GET RESPONSE in other APDU cases than
case 4 (the only case that uses the GET RESPONSE command).
Related: OS#6970
Change-Id: I26f0566af0cdd61dcc97f5f502479dc76adc37cc
'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.
Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)
Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
Jenkins: skip-card-test
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.
Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
Jenkins: skip-card-test
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Jenkins: skip-card-test
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.
BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:
my_source = MyParam.default_source.from_str( MyParam.default_value )
Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Jenkins: skip-card-test
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.
For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Jenkins: skip-card-test
Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.
Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test
Previously, _test_de_encode vectors for TransRecEF subclasses were tested
via decode_record_hex()/encode_record_hex(), i.e. one record at a time,
with the decoded value being a scalar.
Switch test_de_encode_record() in TransRecEF_Test to use decode_hex() /
encode_hex() instead, so that vectors represent whole-file content
(decoded value is a list of records) -- consistent with how LinFixedEF
handles _test_de_encode. Update all existing vectors accordingly.
Change-Id: I4a9610f9ee39833cd0c90f64f89f5fbdd6f0846d
When json.loads() fails (e.g. the user made a syntax mistake), prompt
the user with "Re-open file for editing? [y]es/[n]o:" and loop back to
the editor if they answer 'y' or 'yes'. If the user declines, return
the original unmodified value so no write is attempted; the temp file
is still cleaned up by __exit__() in that case.
Change-Id: I9161b7becea0d8dfd3f5f740fbb253da2f061a1d
Related: OS#6899
A plain NamedTemporaryFile is sufficient here: we only need a single
file, not a directory to hold it. Using NamedTemporaryFile is simpler
(no subdirectory to manage) and gives us a .json suffix for free,
which editors use for syntax highlighting.
Change-Id: If3b0bd0fcc90732407dbd03b9cc883f7abeb948e
When invoking `edit_binary_decoded` or `edit_record_decoded`, the
temp file opened in the editor now contains the EF's encode/decode
test vectors as //-comment lines below the JSON content, similar to
how 'git commit' appends comments to the commit message template.
The comment block is stripped before JSON parsing on save,
so it has no effect on the written data.
The feature is implemented via a new module-level JsonEditor context
manager class that encapsulates the full edit cycle:
* write JSON + examples to a TemporaryDirectory
* invoke the editor
* read back, strip //-comments, parse and return the result
Change-Id: I5a046a9c7ba7e08a98cf643d5a26bc669539b38f
Related: OS#6900
The documentation of the getProtocol provided by pyscard says:
"Return bit mask for the protocol of connection, or None if no
protocol set. The return value is a bit mask of
CardConnection.T0_protocol, CardConnection.T1_protocol,
CardConnection.RAW_protocol, CardConnection.T15_protocol"
This suggests that the purpose of getProtocol is not to determine
which protocols are supported. Its purpose is to determine which
protocol is currently selected (either through auto selection or
through the explicit selection made by the API user). This means
we are using getProtocol wrong.
So far this was no problem, since the auto-selected protocol
should be a supported protocol anyway. However, the automatic
protocol selection may not always return a correct result (see
bug report from THD-siegfried [1]).
Let's not trust the automatic protocol selection. Instead let's
parse the ATR and make the decision based on the TD1/TD2 bytes).
[1] https://osmocom.org/issues/6952
Related: OS#6952
Change-Id: Ib119948aa68c430e42ac84daec8b9bd542db7963
When we initialize a new PySimLogger, we always call the setup method
first and then use the set_verbose and set_level method to configure
the initial log level and the initial log verbosity. However, we
initialize the PySimLogger in all our programs the same way and we
end up with the same boilerplate code every time. Let's add a keyword
parameter to the setup method where we can pass our opts.verbose (bool)
parameter so that the setup method can do the work for the main program.
In case the caller wants a different default configuration he still can
call set_verbose and set_level methods as needed.
Change-Id: I4b8ef1e203186878910c9614a1d900d5759236a8
Unfortunately we have mixed up the concept of TPDUs and APDUs in
earlier versions of pySim-shell. This lead to problems with
detecteding the APDU case properly (see also ISO/IEC 7816-3) and
also prevented us from adding support for T=1.
This problem has been fixed long time ago and all APDUs sent from
the pySim-shell code should be well formed and valid according to
ISO/IEC 7816-3.
To ensure that we continue to format APDUs correctly as APDUs (and
not TPDUs) we have added a mechanism to the LinkBase class that
would either raise an exception or print a warning if someone
mistakenly tries to send an APDU that is really a TPDU. Whether a
warning is printed or an exception is raised is controlled via the
apdu_strict member in the LinkBase class, which is false (print
warning only) by default.
The reason why we have implemneted the mechanism this way was
because we wanted to ensure that existing APDU scripts (pySim-shell
apdu command) keep working, even though when those scripts uses
APDUs which are formally invalid.
Sending a TPDU instead of an APDU via a T=0 link will still work
in almost all cases. This is also the reason why this problem
slipped through unnoticed for long time. However, there may still
be subtile problems araising from this practice. The root of the
problem is that it is impossible to distinguish between APDU case
3 and 4 when a TPDU instead of an APDU is sent. However in order
to handle a case 4 APDU correctly we must be able to distinguish
the APDU case correctly to handle the case correctly.
ETSI TS 102 221, section 7.3.1.1.4, clause 4 is very clear about
the fact that not (only) the status word (e.g. 61xx) but the
APDU case is what matters.
To complete the logic in LinkBaseTpdu and to maintain compatibility
(older APDU scripts), we must still be able to switch between the
'apdu_strict' mode and the non-strict mode. However, since
pySim-shell, pySim-prog and pySim-read internally use proper APDUs,
we may enable the 'apdu_strict' mode by default.
At the same time we will limit the effect of pySim-shell's
apdu_strict setable to the apdu command only. By doing so, the
bahviour of the apdu command is not altered. Users will still
have to enable the 'strict' mode explicitly. At the same time
all the internal functionality of pySim-shell will always use
the 'strict' mode.
Related: OS#6970
Change-Id: I9a531a825def318b28bf58291d811cf119003fab
gen_install_parameters() had contradictory logic: the outer guard
required all three arguments to be non-None/non-empty (making them
mutually inclusive), while the inner checks then treated each one
as optional.
Make each parameter independently optional (defaulting to None) and
remove the all-or-nothing check. Simplify the function body to a
straightforward single-pass construction of system_specific_params.
Change-Id: I8756fb38016cdf0527fe2e21edb44381d1dc557f
pySim-shell currently does not work on systems with Python 3.14+:
File ".../pysim/pySim/global_platform/__init__.py", line 868, in AddlShellCommands
install_cap_parser_inst_prm_g_grp = install_cap_parser_inst_prm_g.add_argument_group()
File "/usr/lib/python3.14/argparse.py", line 1794, in add_argument_group
raise ValueError('argument groups cannot be nested')
ValueError('argument groups cannot be nested')
The problem is that install_cap_parser creates a nested group inside
of mutually exclusive group. argparse never supported group nesting
properly, so it has been deprecated since Python 3.11, and eventually
got removed in Python 3.14.
Remove group nesting, adjust the usage string, and implement the
mutual exclusiveness enforcement manually in do_install_cap().
Change-Id: Idddf72d5a745345e134b23f2f01e0257d0667579
The TP-Destination Address in EF.SMSP uses the same encoding as the
TS-Service Centre Address field. However, even though the encoding
of both fields looks almost identical, it actually isn't.
The TS-Service Centre Address field encodes the length field as
octets required for the call_number + one octet for ton_npi.
(see also: 3GPP TS 24.011, section 8.2.5.2)
The TP-Destination Address uses the number of digits of the
call_number directly in the length field.
(see also: 3GPP TS 23.040, section 9.1.2.5)
Related: SYS#7765
Change-Id: I55c123c9e244e5a6e71a0348f5d476ef03e618e8
Let's add another testvector where we test what happens when we populate
none of the fields except for the tp_sc_addr.
Related: SYS#7765
Change-Id: I12b600ab17d1acfdddaffe6006095acf1a4228c9
Neels Hofmeyr