This introduces a hierarchy of classes implementing
* ETS TS 102 225 (general command structure)
* 3GPP TS 31.115 (dialects for SMS-PP)
In this initial patch only the SMS "dialect" is supported,
but it is foreseen that USSD/SMSCB/HTTPS dialects can be
added at a later point.
Change-Id: I193ff4712c8503279c017b4b1324f0c3d38b9f84
Register a ProactiveHandler with pySim.transport and call the decoder
from pySim.cat to print a decoded version:
Example usage (exact data only works on my specific card due to the
encrpyted payload):
pySIM-shell (MF/ADF.USIM)> envelope_sms 400881214365877ff6227052000000000302700000201506393535b000118dd46f4ad6b015922f62292350d60af4af191adcbbc35cf4
FETCH: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
SendShortMessage(CommandDetails({'command_number': 1, 'type_of_command': 19, 'command_qualifier': 0}),DeviceIdentities({'source_dev_id': 'uicc', 'dest_dev_id': 'network'}),SMS_TPDU({'tpdu': '410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c'}))
SW: 9000, data: d0378103011300820281838b2c410008812143658700f621027100001c12b000119660ebdb81be189b5e4389e9e7ab2bc0954f963ad869ed7c
Change-Id: Ia4cdf06a44f46184d0da318bdf67077bc8ac9a1a
This introduces an optional argument to the LinkBase class constructor,
where the application can pass an instance of a ProactiveHandler derived
class in order to handle the proactive commands that the LinkBase is
automatically fetching whenever the card indicates so.
Change-Id: I844504e2fc1b27ce4fc7ede20b2307e698baa0f6
This adds deciding for the bulk of the TLV objects used in the
ETSI CAT (Card Application Toolkit) and 3GPP USAT (USIM Application
Toolkit) systems.
This patch just adds the definitions, but doesn't use them anywhere yet.
Change-Id: I0c66912dbc10164e040e2fec358cef13c45a66ec
The TLV_IE.from_tlv() method is part of a base class that is inherited
by more specific classes. The official way to obtain the tag is the
inherited-class-provided self._compute_tag() method, and *not* a direct
reference to the self.tag member.
This allows for some more obscure TLV parsers, such as the upcoming one
for Proactive Commands in the CAT/OTA context.
Change-Id: I0cd70e31567edc5a0584336efcb5e4282734f6dd
In commit Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5 I accidentially
broke those commands by adding argparse definitions for better
documentation. When adding the @cmd2.with_argparser decorator,
the method argument changes from the raw string to an argparse.Namespace
object.
This patch fixes the below exception:
pySIM-shell (MF/ADF.USIM)> terminal_profile ffffffff
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2129, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
File "/usr/local/lib/python3.10/dist-packages/cmd2/cmd2.py", line 2559, in onecmd
stop = func(statement)
File "/usr/local/lib/python3.10/dist-packages/cmd2/decorators.py", line 336, in cmd_wrapper
return func(*args_list, **kwargs)
File "/space/home/laforge/projects/git/pysim/pySim/ts_31_102.py", line 1274, in do_terminal_profile
(data, sw) = self._cmd.card._scc.terminal_profile(arg)
File "/space/home/laforge/projects/git/pysim/pySim/commands.py", line 583, in terminal_profile
data_length = len(payload) // 2
TypeError: object of type 'Namespace' has no len()
Change-Id: Ia861eeb2970627d3ecfd0ca73f75ca571c6885b2
Fixes: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
the interpret_sw() method was moved from RuntimeState to RuntimeLchan
in Change-Id I7aa994b625467d4e46a2edd8123240b930305360 - but the code
in pySim/exceptions.py was not adjusted accordingly.
Change-Id: I0614436c99c6a6ebc22c4dc14fb361c5f5f16686
While in the pySim-shell, it's useful to filter the currently selected
file from the choice of available files for select, this doesn't apply
for the tracing case: It's perfectly valid for the UE to SELECT the
file that's already selected right now. The operation basically
becomes equivalent to a STATUS.
Change-Id: I1a20fb3ba70426333ac34448c6cb782c51363965
We need to pass the 'PARENT' flag to get_selectables() to be able
to track SELECT on any of the parent/ancestor DF FID.
Change-Id: Ia7ac627d5edccb97160c90688d720d887fad6ec7
I didn't check the specs, but at least experience with real-world cards
(and modems) shows that it's not just permitted to select the immediate
parent DF, but all ancestors of the currently selected file.
So adjust the get_selectables() method to not just return the immediate
parent, but to recurse all the way up and report the FID of any ancestor
DF.
Change-Id: Ic9037aa9a13af6fb0c2c22b673aa4afa78575b49
In order to be able to explicitly select the MF via 3f00,
we need to pass the 'MF' to get_selectables(), so the record
is included in the list of selectable files from the current
working directory.
Change-Id: I27085896142fe547a6e93e01e63e59bbc65c8b8a
All other get_selectables() understand a flag like 'FIDS' to request
only the hexadecimal FIDs and not the file names. However, the
CardEF.get_selectables() ignored those flags and unconditionally
returned the names.
Change-Id: Icdc37cae3eecd36d167da76c30224b9d48c844fd
The Tracer implemented those options and the argparser handled it,
but we didn't ever connect the two.
Change-Id: I7d7d5fc475a8d09efdb63d3d6f1cc1de1996687b
If the modem/UE doesn't ask for the FCP to be returned, a SELECT
can exit with 9000 and no response body. Don't crash in that case.
Change-Id: I66788717bec921bc54575e60f3f81adc80584dbc
Instead of a cryptic backtrace, we now return a meaningful error like this:
{"error": {"message": "Security Status not satisfied - Card PIN enabled?", "status_word": "6982"}
Change-Id: I6dafd37dfd9fa3d52ca2c2e5ec37a6d274ba651b
Closes: OS#5606
Let's make sure even error messages are returned in JSON format.
While at it, also reduce some code duplication between the 'auth'
and 'info' route handlers by using the klein handle_errors decorator
instead of manual exception catching.
Change-Id: I1e0364e28ba7ce7451993f57c8228f9a7ade6b0e
Closes: OS#5607
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between
SIM/UICC/USIM/ISIM card and UE.
It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.
This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.
Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats. So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.
Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
Fix a bug in the pySim.sysmocom_sja2 module, where we defined unnamed
bits in BitStruct without a default value causing exceptions like this:
EXCEPTION of type 'KeyError' occurred with message: 'None'
Change-Id: Ib2da5adda4fae374ab14bb8100f338691aef719a
Closes: OS#5575
We were invoking the constructor with the description as 4th positional
argument, but that was actually the 'size' argument in this case.
Let's swap the order to be aligned with other file constructors.
Change-Id: I9acee757f096fef0d8bacbec3b52f56267cd52f6
The size should be a *tuple*. In reality we so far passed a set. The
problem with the set is that ordering is not guaranteed, and hence we
cannot assume the first and second item have meaning (minimum vs.
default record length).
Change-Id: I470f4e69c83cb2761861b3350bf8d49e31f4d957
As the documentation strings say: The size should be a *tuple*. In
reality we so far passed a set. The problem with the set is that
ordering is not guaranteed, and hence we cannot assume the first and
second item have meaning (minimum vs. default size).
While at it, use a type annotation to catch such bugs easily.
Change-Id: I553616f8c6c4aaa8f635b3d7d94e8e8f49ed5a56
Reduce all the copy+pasted '/'.join(path_list) constructs with
a method returning the formatted path string.
Change-Id: I5e9bfb425c3a3fade13ca4ccd2b891a0c21ed56d
Otherwise we have binary/bytes as values inside the dict, rather than a
hexadecimal string. That's ugly when printing without json formatting.
Change-Id: Ia3e7c4791d11bd4e3719a43d58e11e05ec986d1f
Even while we don't yet have a proper decoder, let's at least represent
the network name as hex-string
Change-Id: I4ed626699d1e4e484d4ffd04349676dadff626a0
This file existed in earlier specs like Release 3.8.0, but was removed
in later revisions. Still, there are cards around implementing that
older spec, so let's add a decoder.
Change-Id: Ic7163b2a01f64ef1223cf15b8d0813d3edf5b61a
We want to use this class in an upcoming patch for DF_MCS support,
and in order to avoid cyclic imports, EF_UServiceTable must be moved.
Change-Id: I9cd6ab795bfd92f845eb943679a3d6302f1003ce
cards can have multiple logical channels; each logical channel
has its own state of what is the current selected file + application.
Let's split the RuntimeState class into the global RuntimeState and the
per-lchan-specific RuntimeLchan class.
This code doesn't actually introduce any code that uses lchans other
than the basic logical channel (0), but just modifies the data model
to accomodate those in the future.
Change-Id: I7aa994b625467d4e46a2edd8123240b930305360
When using __str__ for a CardDF we would get "DF(DF.TELECOM)"
but when using it on CardADF we would get ADF(a0000000871002)"
instead of "ADF(ADF.USIM)". Let's fix that.
Change-Id: I5801a08bcc28cb222734af6d9ee835227f4fee69
As it is possible to select files relative to the currently selected
ADF, we should keep track of that.
Change-Id: I83c93fdcd23b1d3877644ef0bf72d330343fbbc7
We should first see if any of the files in the tree actually
require a service mapping before raising
ValueError('TODO: implement recursive service -> file mapping')
Change-Id: I9c339f0cac020e7eec7f4f840748040e5f77923d
Service 126 relates to DF.5GS/EF.UAC_AIC. As we are deactivating that
file in the script, we should also disable the related EF.UST service.
Change-Id: Id35035aaf23b2163caed3197786288c87be03cfa
The DataObject is some weird / rarely used different code than the
normal TLV encoder/decoder. It has apparently so far only been used
for decoding, without testing the encoding side, resulting in related
bugs.
Let's fix those that I encountered today, and add a test case.
Change-Id: I31370066f43c22fc3ce9e2b9ee75986a652f6fc4
This should avoid the following pylint error:
************* Module pySim.ts_31_102
pySim/ts_31_102.py:621:100: E0601: Using variable 'sw' before assignment (used-before-assignment)
Change-Id: I0bb9607cdab0e6e3cd17b4d27129a51a607bc0f2
These commands can be used to decode a user-provided hex-string,
instead of decoding the data read from the file. This is useful
for quickly manually decoding some values read from other locations,
such as e.g. copy+pasted from a eSIM profile in ASN.1 value notation.
Change-Id: I81f73bce2c26e3e5dfc7538d223bb2d2483c7fa0
The OPL has 7 bytes "LAI" as the LAI actually contains a LAC
range (so two more bytes for the end of the 16bit range).
Change-Id: I74bcf10b0a8977af0f2844044a812c5780af1706
range(0,7) in python is 0..6, and not 0..7, so we need range(0.8)
to produce the desired range covering all bits of a byte.
This resulted in services 8,16,24,... not being displayed in
the decoded output of EF.UST / EF.IST.
Change-Id: I22bbc481de342685352bf5b13d54931d3f37f9b7
The SFI TLV contanins not the raw SFI, but it contains the SFI
shifted to left by 3 bits (for some strange reason). So let's
un-shift it.
Change-Id: Ibc69b99010d2a25cbb69b6a3d1585d0cb63f1345
* don't duplicate information between .rst files and docstrings
* if there's more than a trivial single-line documentation, put it as
docstring into the python source and use ".. argparse" to pul it into
the manual
* add documentation for some commands for which it was missing
* show one level deeper in the navigation table, listing the commands
Change-Id: Ib88bb7d12faaac7d149ee1f6379bc128b83bbdd5
The FCP template provides us a lot of context, like the permissions of
a given file. Let's make it part of the 'export' output, both in raw
and in decoded form.
Change-Id: I05f17bbebd7a9b3535204b821900851a5f66e88f
Closes: OS#5457
This adds support for creating/deleting and terminating files,
as well as support for permanent card termination.
Change-Id: I5b1ffb1334afa18d62beb642268066a30deb7ea6
With this patch applied, users can directly enter commands like
select DF.GSM/EF.IMSI or
select ADF.USIM/DF.5GS/EF.5GAUTHKEYS
This feature doesn't have tabl completion, so it's mostly useful
for when you know what to select, or for use within scripts.
Change-Id: I681a132eb2df4b2aba4c2ccbdd21c6d5b88443e3
We must not only consider files in the current directory (ADF.USIM)
but also in its sub-directories. This requires us to be able to
determine the path we need to traverse between the currently selected
file (EF.UST) and the respective file in some other directory,
which is implemented via CardFile.build_select_path_to().
Change-Id: I61797fefa9dafa36a8a62c11aa2cfaeecb015740
We had service annotations only for ADF.USIM so far, but not for
the related sub-directories.
Change-Id: Iaa56a26ba53eaf18fce14845ae07a27c52a2c58a
Note: The code doesn't make use of them in any reasonable way yet!
The existing code had the following serious problems:
* when trying to update EF.SST or EF.IST, it would write to EF.UST !
* shell commands were called ust_* even for the EST/IST files
Let's introduce the proper separation between what is shared and what
is file-specific.
Change-Id: Ie55669ca37a4762fac9f71b1db528ca67056e8dd
This command performs a consistency check between the services activated
in EF.UST/EF.IST and the files that should (or should not) be
active/selectable for the given service.
Produces output like:
Checking service No 48 (inactive)
ERROR: File EF(EF.MWIS) is selectable but should not!
Checking service No 49 (active)
ERROR: File EF(EF.CFIS) is not selectable (SW=6a82) but should!
Change-Id: Iea7166959e2015eb8fa34d86036560c9e42ce4d3
We want people to use pySim-shell and should not mislead them by
having usage examples of old tools in README.md. Also, all
documentation should be in the manuals, let's try to have bits
and pieces in various places.
Change-Id: I8c07a2e0778ab95fb42be6074acb80874e681d20
This allows us [in a future patch] to perform consistency checking,
whether files exist for services not activated in EF.{UST,IST} or
vice-versa: Services are activated by files are not present or
deactivated.
Change-Id: I94bd1c3f9e977767553000077dd003423ed6dbd1
This can be populated by card profiles with the SST/IST/UST service
that is associated with the file.
Change-Id: I3b3f74b691368fa09967ecb377a9f7a6d8af7869
This can be useful when playing around with cards, for example
sending commands for which pySim-shell doesn't yet have proper support.
Change-Id: Ib504431d26ed2b6f71f77a143ff0a7fb4f5ea02e
The primary use case of the --json option is to systematically execute
all of our decoder classes in order to find bugs. As we don't have
encoders for all files yet, the output generated by 'export --json'
will in many cases not be executable as script again, unlike the normal
'export' output.
Change-Id: Idd820f8e3af70ebcbf82037b56fd2ae9655afbc5
pytlv is a nightmare of shortcomings, let's abandon it in favor of
our own meanwhile-created pySim.tlv. This has the added benefit
that unknown tags finally no longer raise exceptions.
Change-Id: Ic8e0e0ddf915949670d620630d4ceb02a9116471
Closes: OS#5414
This happens e.g. when selecting the ARA-M applet on sysmoISIM-SJA2:
pySIM-shell (MF)> select ADF.ARA-M
-> 00a4040409 a00000015141434c00
<- 9000:
Traceback (most recent call last):
File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2064, in onecmd_plus_hooks
stop = self.onecmd(statement, add_to_history=add_to_history)
File "/space/home/laforge/.local/lib/python3.9/site-packages/cmd2/cmd2.py", line 2494, in onecmd
stop = func(statement)
File "/space/home/laforge/projects/git/pysim/./pySim-shell.py", line 750, in do_select
fcp_dec = self._cmd.rs.select(path, self._cmd)
File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 1314, in select
select_resp = f.decode_select_response(data)
File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 193, in decode_select_response
return self.parent.decode_select_response(data_hex)
File "/space/home/laforge/projects/git/pysim/pySim/filesystem.py", line 378, in decode_select_response
return profile.decode_select_response(data_hex)
File "/space/home/laforge/projects/git/pysim/pySim/ts_102_221.py", line 796, in decode_select_response
t.from_tlv(h2b(resp_hex))
File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 231, in from_tlv
(rawtag, remainder) = self.__class__._parse_tag_raw(do)
File "/space/home/laforge/projects/git/pysim/pySim/tlv.py", line 258, in _parse_tag_raw
return bertlv_parse_tag_raw(do)
File "/space/home/laforge/projects/git/pysim/pySim/utils.py", line 208, in bertlv_parse_tag_raw
if binary[0] == 0xff and len(binary) == 1 or binary[0] == 0xff and binary[1] == 0xff:
IndexError: bytearray index out of range
EXCEPTION of type 'IndexError' occurred with message: 'bytearray index out of range'
Change-Id: I910e6deba27d1483dff1e986c89f1a1b2165f49b
We have a number of integers with variable-length encoding, so
add a Construct for this. Naming inspired by GreedyBytes.
Related to https://github.com/construct/construct/issues/962
Change-Id: Ic6049b74ea3705fda24855f34b4a1d5f2c9327f7
Our hand-written JSON so far is using snake_case identifiers,
while the JSON generated by the pySim.tlv classes use the class
names as keys, which LooksQuiteDifferent.
So let's auto-convert the CamelCase into something that reflects
our existing notion.
Change-Id: Id55929ef03dc48cb668e6ba7e99b6b291680a42f
We had a mixture of tab and 4space based indenting, which is a bad
idea. 4space is the standard in python, so convert all our code to
that. The result unfortuantely still shoed even more inconsistencies,
so I've decided to run autopep8 on the entire code base.
Change-Id: I4a4b1b444a2f43fab05fc5d2c8a7dd6ddecb5f07
EF.DIR can not only contain the AID + Label of TS 102 221, but can
also contain any of the DOs specified in ISO7816-4. Let's imoplement
this based on the modern pySim.tlv parser
Change-Id: I875eb49e1f0370428c2eae69af84f5483bd5b1fc
Closes: OS#5410
As we've seen in recent patches, this has been a source of bugs, so
let's be tolerant and deal with both.
Change-Id: I0a5ec2a860104ffe4524c647105a42505ac394d6
In Change-Id I6d7c1bf49a8eaf3d8e50fb12888bf3d5b46b6c55 we fixed the
filesystem code to assume the self._tlv memper is a reference to a
class, and not an instance (as this is what the majority of the code
did).
However, it seems thre wer two instances where we actually had _tlv
reference an instance. Change that to class so it's the same all over
the code base.
Change-Id: Ie4878ad6a92feafe47e375c4f5f3f198921e1e95
Harald Welte