The Card Key Provider currently only has support for CSV files
as input. Unfortunately using CSV files does not scale very well
when the card inventory is very large and continously updated.
In this case a centralized storage in the form of a database
is the more suitable approach.
This patch adds PostgreSQL support next to the existing CSV
file support. It also adds an importer tool to import existing
CSV files into the database.
Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3
Related: SYS#7725
In case pySim-shell is used directly from the git repository (not
installed via a package manager), the version command fails with an
exception because pkg_resources.get_distribution('pySim') fails.
Let's renovate the version command and migrate from pkg_resources to
importlib.resources. There are many users and developers out there who
retrieve pySim-shell directly from the git repository and not via pip3.
To accommodate for that, let's check if pySim-shell.py is located in a
git repository and if so, let's display the HEAD commit hash instead.
Since the version of the currently installed pyosmocom version also
plays a critical role, let's display the pyosmocom version as well.
Related: OS#6830
Change-Id: I2b9038f88cfcaa07894a2f09c7f5ad8a5474083d
At the moment, the help text for the --csv option shows the path to
the users home. This is due to the default value, which is dynamically
generated. Let's use a static string with "~/" and resolve the full
path later when we need it.
Related: SYS#7725
Change-Id: Ied8b1e553de8f5370369c4485a2360906c874ed2
As we plan to support other formats as data source for the Card Key
Provider soon, the more commandline options may be added and it makes
sense to group the Card Key Provider options in a dedicated group.
Let's also rename the option "--csv-column-key" to just "--column-key".
The column encryption is a generic concept and not CSV format specific.
(let's silently keep the "--csv-column-key" argument so maintain backward
compatibility)
Related: SYS#7725
Change-Id: I5093f8383551f8c9b84342ca6674c1ebdbbfc19c
The Card Key Provider is a built in mechanism of pySim-shell which
allows the user to read key material from a CSV file in order to
avoid having to lookup and enter the key material himself. The
lookup normally done by the pySim-shell commands automatically.
However, in some cases it may also be useful to be able to query the
CSV file manually in order to get certain fields displayed. Such a
command is in particular helpful to check and diagnose the CSV data
source.
Related: SYS#7725
Change-Id: I76e0f883572a029bdca65a5a6b3eef306db1c221
The default log level of the PySimLogger is DEBUG by default. This is
to ensure that all messages are printed in an unconfigured setup.
However in pySim-Shell we care about configuring the logger, so let's
set the debug log level to INFO in startup. This will allow us to
turn debug messages on and off using the verbose switch.
Change-Id: I89315f830ce1cc2d573887de4f4cf4e19d17543b
Related: SYS#7725
The CHV commands (verify_chv, enable_chv, disable_chv, unblock_chv)
provide a --pin-nr parameter.
The --pin-nr is a decimal parameter that specifies the pin type to be
used. The exact pin type numbers are specified in ETSI TS 102.221,
Table 9.3.
Unfortunately the --pin-nr parameter is not very intuitive to use, it
it requires the user to manually lookup the numeric value. The specs
list that value as hexadecimal, so the user also has to convert it
to decimal. To make this less complicated, let's also accept
hexadecimal numbers with the --pin-nr parameter.
However, this alone does not improve the user expierience much. Let's
also add a --pin-type parameter (similar to the --adm-type parameter
of the verify_adm command) to specifiy the pin type in a human
readable form.
Change-Id: I0b58c402d95cbc4fe690e6edb214829d463e9f2c
In many sub modules we still use print() to occassionally print status
messages or warnings. This technically does not hurt, but it is an unclean
solution which we should replace with something more mature.
Let's use python's built in logging framework to create a static logger
class that fits our needs. To maintain compatibility let's also make sure
that the logger class will behave like a normal print() statement when no
configuration parameters are supplied by the API user.
To illustrate how the approach can be used in sub-modules, this patch
replaces the print statements in runtime.py. The other print statements
will the be fixed in follow-up patches.
Related: OS#6864
Change-Id: I187f117e7e1ccdb2a85dfdfb18e84bd7561704eb
Startup scripts are executed using the cmd2 provided onecmd_plus_hooks
method. This method can run arbitrary commands, which also includes
the command "run_scrit" that we use to execute startup scripts.
When a script executes a quit command, or when someone issues a quit
command using the --execute-command or the command argument, then
this commands is executed. However a quit command won't actually quit
the process. All it does is to change the return code of
app.onecmd_plus_hooks (see [1]). So we must evaluate the return code
and take care of the quitting ourselves.
[1] https://cmd2.readthedocs.io/en/0.9.15/api/cmd.html#cmd2.cmd2.Cmd.onecmd_plus_hooks
Related: OS#6731
Change-Id: Ic6e9c54cdb6955d65011af3eb5a025eee5da4143
type annotations claimed the return type was Hexstr, but in reality
it was a list of integers. Let's fix that.
Change-Id: I01b247dad40ec986cf199302f8e92d16848bd499
Closes: OS#6322
Let's change the prompt from ">" to "#" when the user gains admin
privilegs using verify_adm.
Related: OS#6640
Change-Id: I957b9df7b5069b6fce5bf958c94e8ffda833c77f
When the equip method is running, all kinds of states in pySim-shell are reset.
To be sure that the card state is also reset (normally this is the case because
usually init_card is called before equip), we should send an explicit reset to
the card as well.
Related: OS#6640
Change-Id: I622a2df2c9184841f72abd18483bfbfd00b2f464
ETSI TS 102 221, section 7.3 specifies that UICCs (and eUICCs) may support two
different transport protocols: T=0 or T=1 or both. The spec also says that the
terminal must support both protocols.
This patch adds the necessary functionality to support the T=1 protocol
alongside the T=0 protocol. However, this also means that we have to sharpen
the lines between APDUs and TPDUs.
As this patch also touches the low level interface to readers it was also
manually tested with a classic serial reader. Calypso and AT command readers
were not tested.
Change-Id: I8b56d7804a2b4c392f43f8540e0b6e70001a8970
Related: OS#6367
by default pySim-shell does all kinds of probing and file selection
on startup. This is to determine the card type and to find a suitable
card profile. However, in case the card is non yet provisioned this
probing may cause a error messages and even might upset the cards
internal state. So let's have a commandline option thrugh which we
can instruct pySim-shell to skip any initialization and to give us
a prompt immediately, so that we can enter custom APDUs
Related: OS#6367
Change-Id: I1d8a57de201fe7ad7cbcbc6f72969ea8521e821d
There are situations where no card profile can be determined. In this case no
RuntimeState will be present. This is in particular the case when pySim-shell
is used on a card that is not provisioned/initialized yet. In those cases we
have to go the direct route and reset the card directly.
Related: OS#6367
Change-Id: I27bf9fdb131d8bdeba07f4dfd2b76b38f9bfdd17
The "apdu" command allows us to send custom APDUs to a card. This command is
often used in low level initialization scripts or tests. To stop the script
execution in case of an error, the command allows us to specify a status word
that must match the status word of the response. But we have no such mechanism
for the response itself. Let's add another parameter where we can pass a regex
that the response must match.
Related: OS#6367
Change-Id: I97bbcdf37bdcf00ad50a875b96940c211de7073d
Using '-e' it is possible to specify *multiple* pySim-shell commands
which shall be executed at startup. This extends the current ability
to execute just a single command.
Example:
./pySim-shell.py -p0 -e 'select ADF.USIM/EF.IMSI' -e 'read_binary_decoded'
Change-Id: I74004f46105553f077c039ca0f86f75afccc7342
When a hexadecimal formatted ADM pin is retrieved via the
card_key_provider, it still requires the --pin-is-hex parameter so
that sanitize_pin_adm knows the correct format.
This unfortunately ruins the card_key_provider feature for all cards
that use hexadecimal pins, because the --pin-is-hex would also be
required in scripts, which makes a script either useable for cards
with hexadecimal ADM or for for cards with ASCII ADM.
To minimize the problem, let's recognize all ADM pins longer than 8
digits as hexadecimal in case --pin-is-hex is not set.
Related: OS#4348
Change-Id: Iad9398365d448946c499ce89e3cfb2c3af5d525e
The "desc" command displays a string with a file description, let's also
display some size information as part of the description as well.
Related: OS#5714
Change-Id: I98e139ba2bf35df5524245cdd96f5c52cf09b986
When we define positional arguments for the argument parser, we usually
use upper case letters only. However, there are some code locations that
use lower case letters. Let's translate those to capital letters to
have a consistent appeariance.
Related: OS#6531
Change-Id: Iec1ff8262bc6e9cf87c3cbf7b32fa5f753b7e574
The command cardinfo also displays the AIDs of the card applications.
However, on classic GSM sim cards there are no applications. In this
case cardinfo will still display the string 'AIDs:', but it will of
course not list any AIDs under this string.
Related: OS#6531
Change-Id: Ifb111ce43fdebe85d30857dfc61ab570380b68d1
When pySim-shell is used in a scripted environment, we may easily get trapped in
the pySim-shell prompt. This may happen in particular in case the script file
is not executed due to problem with the reader initialization. In such a case
pySim-shell will not exit automatically and the shellscript that was calling
pySim-shell will stall indefinetly.
To make the use of pySim-shell more reliable in scripted environments, let's
add a --noprompt option that ensures the interactive mode is never entered.
Let's also exit with an appropriate return code in case of initialization
errors, so that the calling script can know that something went wrong.
Related: OS#6531
Change-Id: I07ecb27b37e2573629981a0d032cc95cd156be7e
We're creating a 'pyosmocom' pypi module which contains a number of core
Osmocom libraries / interfaces that are not specific to SIM card stuff
contained here.
The main modules moved in this initial step are pySim.tlv, pySim.utils
and pySim.construct. utils is split, not all of the contents is
unrelated to SIM Cards. The other two are moved completely.
Change-Id: I4b63e45bcb0c9ba2424dacf85e0222aee735f411
The ara_m commands use APDUs with a fix class byte (0x80). This means
that all ARA-M related features only work in the basic logical channel.
To fix this, let's compute the class byte for the current logical channel
dynamically inside the send_apdu methods of SimCardCommands. This will
fix the problem globally.
Related: OS#6531
Change-Id: Ie3e48678f178a488bfaea6cc2b9a3e18145a8d10
The basic logical channel 0 is always present. It cannot be created or
closed. Let's restrict the value range of chan_nr, so that only valid
lchan numbers can be passed.
Related: OS#6531
Change-Id: I4eebd9f15fadd18e1caeb033fda36c59446fcab8
When we define command arguments using the ArgumentParser, we sometimes
define the positional arguments first. However, since positional arguments
usually follow after the optional (--xyz) arguments, we should define the
positional arguments last.
Related: OS#6531
Change-Id: I2412eb6e7dc32ae95a575f31d4489ce210d85ea0
The CardKeyProvider support for the commands enable_chv, disable_chv,
verify_chv, change_chv and unblock_chv is broken. The reason for this
is the annotation "type=is_decimal" in the argument parser. This annotation
prevents the usage of string placeholders ("PIN1", "PUK1", etc).
Let's fix this by finding a better solution. We can also replace any
missing PIN/PUK code by checking if it is supplied or not. If not,
we query the CardKeyProvider. This also makes the usage of the *_chv
commands more uniform with the verify_adm command.
Related: OS#6531
Change-Id: I565b56ac608e801c67ca53d337bdec9efa3f3817
In the previous patch we have improved the export command. Since
the implementation of the fsdump command is very similar to the
implementation of the export command we can now apply the same
improvements to the fsdump command as well.
Change-Id: I4d2ef7b383025a5bbf122f18ecd51b7d73aaba14
Related: OS#6092
Since we now have the ability to provide export methods for all file
types in the file system (this also includes DF and ADF files), we need
to support this at shell command level as well. Let's also renovate the
walk method and the action method that does the actual exporting.
Related: OS#6092
Change-Id: I3ee661dbae5c11fec23911775f352ac13bc2c6e5
the select_parent method in RuntimeLchan currently implements a way
to escape from an application that has no filesystem support. However,
this escape route can be integrated directly into the select_file
method. This will give us the benefit that it will work transparently
in all code locations.
(This also means we can get rid of the select_parent method again)
Related: OS#6120
Change-Id: Ie6f37d13af880d24a9c7a8a95cef436b603587c7
This command exports the entire filesystem state as one JSON document,
which can be useful for storing it in a noSQL database, or for doing a
structured diff between different such dumps.
It's similar to "export", but then reasonably different to rectify a
separate command.
Change-Id: Ib179f57bc04d394efe11003ba191dca6098192d3
The function do_switch_channel method calls methods in RuntimeLchan
that should be private. There is also a code duplication in
RuntimeLchan that should be cleaned up.
Related: OS#6092
Change-Id: Ie5e5f45787abaaf032e1b49f51d447653cf2c996
The reset command resets the card using the card object. This unfortunately
leaves the RuntimeState uninformed about the event. However, the RuntimeState
class also has a reset method that resets the card and the RuntimeState. Let's
use this reset method. Also fix this method so that it ensures that the SCP is
also no longer present.
Related: OS#6092
Change-Id: I1ad29c9e7ce7d80bebc92fa173ed7a44ee4c2998
When we traverse the file system using the command "export" we will
also select all ADFs but not all ADFs may have UICC file system support.
This makes it impossible to exit those ADFs again. To exit anyway we
select an application with filesystem support first and then the parent
EF we wanted to select originally. This method may not only be useful
when traversing the filesystem, so let's put it into the RuntimeLchan
class and change it a little so that it would also work if the ADF in
question is an a sub DF.
Related: OS#6092
Change-Id: I72de51bc7519fafbcc71d829719a8af35d774342
The code that generates the filesystem export lines for the various
different file structures can be moved into the filesystem class model.
This simplifies the code since we do not need any extra logic to
distinguish between the different file structures.
Related: OS#6092
Change-Id: Icc2ee60cfc4379411744ca1033d79a1ee9cff5a6
It's generally a bad idea to keep [card specific] key material lying
around unencrypted in CSV files. The industry standard solution in the
GSMA is a so-called "transport key", which encrypts the key material.
Let's introduce support for this in the CardKeyProvider (and
specifically, the CardKeyProviderCSV) and allow the user to specify
transport key material as command line options to pySim-shell.
Different transport keys can be used for different key materials, so
allow specification of keys on a CSV-column base.
The higher-level goal is to allow the CSV file not only to store
the ADM keys (like now), but also global platform key material for
establishing SCP towards various security domains in a given card.
Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff
... and populate the RuntimeState.identity['EID'] wit it, so other
[future] parts of the system can use it.
Let's also print the EID (if available) from the 'cardinfo' shell
command.
Change-Id: Idc2ea1d9263f39b3dff403e1535a5e6c4e88b26f
In the previous patch, we've introduced a new 'identities' dict as part
of the runtime state. Let's migrate our ICCID storage into it for
consistency.
Change-Id: Ibdcf9a7c4e7e445201640bce33b768bcc4460db1
Cards where no profile was detected don't have a logical channel, and
hence must use the raw APDU at all times.
Change-Id: I08e5d190bdb4e62ee808bfd77584cb3e0b85a8ae
Fixes: Change-Id Id0c364f772c31e11e8dfa21624d8685d253220d0
The 'apdu' command so far bypassed the logical channel and also
the recently-introduced support for secure channels. Let's change
that, at least by default. If somebody wants a raw APDU without
secure / logical channel processing, they may use the --raw option.
Change-Id: Id0c364f772c31e11e8dfa21624d8685d253220d0
