'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.
Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)
Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.
Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.
BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:
my_source = MyParam.default_source.from_str( MyParam.default_value )
Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.
For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.
Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Previously, _test_de_encode vectors for TransRecEF subclasses were tested
via decode_record_hex()/encode_record_hex(), i.e. one record at a time,
with the decoded value being a scalar.
Switch test_de_encode_record() in TransRecEF_Test to use decode_hex() /
encode_hex() instead, so that vectors represent whole-file content
(decoded value is a list of records) -- consistent with how LinFixedEF
handles _test_de_encode. Update all existing vectors accordingly.
Change-Id: I4a9610f9ee39833cd0c90f64f89f5fbdd6f0846d
When json.loads() fails (e.g. the user made a syntax mistake), prompt
the user with "Re-open file for editing? [y]es/[n]o:" and loop back to
the editor if they answer 'y' or 'yes'. If the user declines, return
the original unmodified value so no write is attempted; the temp file
is still cleaned up by __exit__() in that case.
Change-Id: I9161b7becea0d8dfd3f5f740fbb253da2f061a1d
Related: OS#6899
A plain NamedTemporaryFile is sufficient here: we only need a single
file, not a directory to hold it. Using NamedTemporaryFile is simpler
(no subdirectory to manage) and gives us a .json suffix for free,
which editors use for syntax highlighting.
Change-Id: If3b0bd0fcc90732407dbd03b9cc883f7abeb948e
When invoking `edit_binary_decoded` or `edit_record_decoded`, the
temp file opened in the editor now contains the EF's encode/decode
test vectors as //-comment lines below the JSON content, similar to
how 'git commit' appends comments to the commit message template.
The comment block is stripped before JSON parsing on save,
so it has no effect on the written data.
The feature is implemented via a new module-level JsonEditor context
manager class that encapsulates the full edit cycle:
* write JSON + examples to a TemporaryDirectory
* invoke the editor
* read back, strip //-comments, parse and return the result
Change-Id: I5a046a9c7ba7e08a98cf643d5a26bc669539b38f
Related: OS#6900
Add a Sphinx extension (docs/pysim_fs_sphinx.py) that hooks into the
builder-inited event and generates docs/filesystem.rst before Sphinx
reads any source files.
The generated page contains a hierarchical listing of all implemented
EFs and DFs, organised by application/specification (UICC/TS 102 221,
ADF.USIM/TS 31.102, ADF.ISIM/TS 31.103, SIM/TS 51.011). For each file,
the class docstring and any _test_de_encode / _test_decode vectors
are included as an encoding/decoding example table.
docs/filesystem.rst is fully generated at build time and is therefore
added to .gitignore.
Add tests/unittests/test_fs_coverage.py that walks all pySim.* modules
and verifies that every CardProfile, CardApplication, and standalone
CardDF subclass with EF/DF children is either listed in the SECTIONS
(and will appear in the docs) or explicitly EXCLUDED.
Change-Id: I06ddeefc6c11e04d7c24e116f3f39c8a6635856f
Related: OS#6316
The documentation of the getProtocol provided by pyscard says:
"Return bit mask for the protocol of connection, or None if no
protocol set. The return value is a bit mask of
CardConnection.T0_protocol, CardConnection.T1_protocol,
CardConnection.RAW_protocol, CardConnection.T15_protocol"
This suggests that the purpose of getProtocol is not to determine
which protocols are supported. Its purpose is to determine which
protocol is currently selected (either through auto selection or
through the explicit selection made by the API user). This means
we are using getProtocol wrong.
So far this was no problem, since the auto-selected protocol
should be a supported protocol anyway. However, the automatic
protocol selection may not always return a correct result (see
bug report from THD-siegfried [1]).
Let's not trust the automatic protocol selection. Instead let's
parse the ATR and make the decision based on the TD1/TD2 bytes).
[1] https://osmocom.org/issues/6952
Related: OS#6952
Change-Id: Ib119948aa68c430e42ac84daec8b9bd542db7963
The eSIM SM-DP+ server modules (`pySim.esim.es2p`, `pySim.esim.es9p`,
`pySim.esim.http_json_api`) unconditionally import optional server-side
dependencies at module level:
pySim.esim.es2p -- from klein import Klein
pySim.esim.http_json_api -- from twisted.web.server import Request
Both imports fail during a docs build if the packages are absent or
broken, causing three "autodoc: failed to import" warnings and three
missing chapters in the generated manual.
Even when klein and twisted are installed, twisted 23.10.0 (the
version pulled in transitively by smpp.twisted3's `Twisted~=23.10.0`
constraint) is incompatible with Python 3.13+ because twisted.web.http
unconditionally executes `import cgi`, a module that was removed from
the standard library in Python 3.13.
Fix: add `autodoc_mock_imports = ['klein', 'twisted']` to conf.py.
Sphinx inserts mock entries into sys.modules before each autodoc import
attempt, so the modules can be imported and documented without requiring
the real packages to be importable at build time.
Change-Id: I71650466f02a6a6d150650deed167c05d2cb6e64
sphinxarg.ext generates generic sub-headings ("Named arguments",
"Positional arguments", "Sub-commands", "General options", ...) for
every argparse command and tool. These repeat across many files and
trigger large numbers of autosectionlabel duplicate-label warnings.
Two-pronged fix:
* `autosectionlabel_maxdepth = 3` eliminates the depth-4+ warnings
(sub-headings inside each individual command block).
* `suppress_warnings` per file silences the residual depth-3 collisions
("serial reader", "decode_hex", "sub-commands", ...) that still
appear across tool documentation files.
Cross-references into these generic argparse-generated sections are not
a supported use-case, so suppressing the warnings is appropriate.
Change-Id: I9cdf2a4f6cbd435b16b90ab668205600ffd7c3b0
pySim-prog and pySim-read do not integrate the pySimLogger yet. As we
may add more debug output that should not be visible on normal use, we
should ensure that the pySimLogger is correctly set up.
Change-Id: Ia2fa535fd9ce4ffa301c3f5d6f98c1f7a4716c74
When we initialize a new PySimLogger, we always call the setup method
first and then use the set_verbose and set_level method to configure
the initial log level and the initial log verbosity. However, we
initialize the PySimLogger in all our programs the same way and we
end up with the same boilerplate code every time. Let's add a keyword
parameter to the setup method where we can pass our opts.verbose (bool)
parameter so that the setup method can do the work for the main program.
In case the caller wants a different default configuration he still can
call set_verbose and set_level methods as needed.
Change-Id: I4b8ef1e203186878910c9614a1d900d5759236a8
In pySim-read we do not have to compute any random numbers, so
we may remove random from the imports
Change-Id: Iae4ee6aafb339cc682345299b92b4ecd0bbca14e
The "ADM PIN" and "SCP02 / SCP03" sub-sections of "Field naming" used
the '~' heading character, which Sphinx resolved to level 4 - skipping
level 3 and throwing build ERRORs. As a result, both sub-sections
had no heading at all. Change both to '^' (level 3) to match the
other sub-sections in this file.
While at it, fix a typo: "consisting if" -> "consisting of".
Change-Id: Ia56efc7fadcc0fd62e87e63850b929d2f80851ba
Unfortunately we have mixed up the concept of TPDUs and APDUs in
earlier versions of pySim-shell. This lead to problems with
detecteding the APDU case properly (see also ISO/IEC 7816-3) and
also prevented us from adding support for T=1.
This problem has been fixed long time ago and all APDUs sent from
the pySim-shell code should be well formed and valid according to
ISO/IEC 7816-3.
To ensure that we continue to format APDUs correctly as APDUs (and
not TPDUs) we have added a mechanism to the LinkBase class that
would either raise an exception or print a warning if someone
mistakenly tries to send an APDU that is really a TPDU. Whether a
warning is printed or an exception is raised is controlled via the
apdu_strict member in the LinkBase class, which is false (print
warning only) by default.
The reason why we have implemneted the mechanism this way was
because we wanted to ensure that existing APDU scripts (pySim-shell
apdu command) keep working, even though when those scripts uses
APDUs which are formally invalid.
Sending a TPDU instead of an APDU via a T=0 link will still work
in almost all cases. This is also the reason why this problem
slipped through unnoticed for long time. However, there may still
be subtile problems araising from this practice. The root of the
problem is that it is impossible to distinguish between APDU case
3 and 4 when a TPDU instead of an APDU is sent. However in order
to handle a case 4 APDU correctly we must be able to distinguish
the APDU case correctly to handle the case correctly.
ETSI TS 102 221, section 7.3.1.1.4, clause 4 is very clear about
the fact that not (only) the status word (e.g. 61xx) but the
APDU case is what matters.
To complete the logic in LinkBaseTpdu and to maintain compatibility
(older APDU scripts), we must still be able to switch between the
'apdu_strict' mode and the non-strict mode. However, since
pySim-shell, pySim-prog and pySim-read internally use proper APDUs,
we may enable the 'apdu_strict' mode by default.
At the same time we will limit the effect of pySim-shell's
apdu_strict setable to the apdu command only. By doing so, the
bahviour of the apdu command is not altered. Users will still
have to enable the 'strict' mode explicitly. At the same time
all the internal functionality of pySim-shell will always use
the 'strict' mode.
Related: OS#6970
Change-Id: I9a531a825def318b28bf58291d811cf119003fab
With the increased interest in using GlobalPlatform features of
UICC and eUICCs (OTA-SMS, applets, etc.), also comes an increased
interest in how the related GlobalPlatform keys can be managed
(key rotation, adding/removing keysets from/to a Security Domain).
Unfortunately, many aspects of this topic are not immediately
obvious for the average user. Let's add a tutorial that contains
some practical examples to shine some light on the topic.
Related: SYS#7881
Change-Id: I163dfedca3df572cb8442e9a4a280e6c5b00327e
gen_install_parameters() had contradictory logic: the outer guard
required all three arguments to be non-None/non-empty (making them
mutually inclusive), while the inner checks then treated each one
as optional.
Make each parameter independently optional (defaulting to None) and
remove the all-or-nothing check. Simplify the function body to a
straightforward single-pass construction of system_specific_params.
Change-Id: I8756fb38016cdf0527fe2e21edb44381d1dc557f
pySim-shell currently does not work on systems with Python 3.14+:
File ".../pysim/pySim/global_platform/__init__.py", line 868, in AddlShellCommands
install_cap_parser_inst_prm_g_grp = install_cap_parser_inst_prm_g.add_argument_group()
File "/usr/lib/python3.14/argparse.py", line 1794, in add_argument_group
raise ValueError('argument groups cannot be nested')
ValueError('argument groups cannot be nested')
The problem is that install_cap_parser creates a nested group inside
of mutually exclusive group. argparse never supported group nesting
properly, so it has been deprecated since Python 3.11, and eventually
got removed in Python 3.14.
Remove group nesting, adjust the usage string, and implement the
mutual exclusiveness enforcement manually in do_install_cap().
Change-Id: Idddf72d5a745345e134b23f2f01e0257d0667579
The TP-Destination Address in EF.SMSP uses the same encoding as the
TS-Service Centre Address field. However, even though the encoding
of both fields looks almost identical, it actually isn't.
The TS-Service Centre Address field encodes the length field as
octets required for the call_number + one octet for ton_npi.
(see also: 3GPP TS 24.011, section 8.2.5.2)
The TP-Destination Address uses the number of digits of the
call_number directly in the length field.
(see also: 3GPP TS 23.040, section 9.1.2.5)
Related: SYS#7765
Change-Id: I55c123c9e244e5a6e71a0348f5d476ef03e618e8
Let's add another testvector where we test what happens when we populate
none of the fields except for the tp_sc_addr.
Related: SYS#7765
Change-Id: I12b600ab17d1acfdddaffe6006095acf1a4228c9
`members_by_name` is a plain dictionary. Calling it with `()` raises:
TypeError: 'dict' object is not callable
Change-Id: I7e0c09aa7303f1506fe3a025fdc3779919dd0e6c
* OtaAlgoCrypt.from_keyset() searches by `otak.algo_crypt`
but the error message prints `otak.algo_auth`. Should be
`otak.algo_crypt` instead.
* OtaAlgoAuth.__init__() checks `algo_auth` but the error message
prints `algo_crypt`. Should be `otak.algo_auth` instead.
Change-Id: Ia636fffaeadc68e3f6d5b65d477e753834c95895
* field `tp_rp` appears at bit positions 7 and 5
** bit 7 should be `tp_rp` (Reply Path)
** bit 5 should be `tp_sri` (Status Report Indication)
* field `tp_lp` is completely missing
** should be at bit position 3
Change-Id: I0274849f0fa07281b5e050af429ffda7d249f9e8
Both `do_store_data` and `store_data` have identical docstrings that
incorrectly describe the command as GET DATA. Should be "STORE DATA".
Take a chance to fix missing space between `v2.3` and `Section`.
Change-Id: I33fc80ab8ca50fadc38217b0005eec6169c8e34e
The loop builds up `response` across multiple STORE DATA blocks,
but the function returns only `data` - the response from the
*last* block. It should return the accumulated response instead.
Change-Id: I3e15c8004d1e366e8c3896e559656622f48bb1a2
The keyword argument should be `nested=`. As written `ApplicationAID`
is silently ignored - `ApplicationTemplate` will not descend into its
nested TLVs.
Change-Id: If45dbb0c9b09fe53560d109957ce339267a9f2b0
The attribute name is misspelled. The BER-TLV infrastructure looks
for `_construct`; this typo means `SupportedTlsCipherSuitesForScp81`
will never decode its content.
Change-Id: I0f637951b0eeb7eca2a8b543baa737f216a935ed
