It is common in CSV files that the columns have uppercase names, so we
have adopted this scheme when we started using the card_key_provider.
This also means that the API of the card_key_provider_get() and
card_key_provider_get_field() function now implicitly requires
uppercase field names like 'ICCID', 'ADM1', etc.
Unfortunately this may be unreliable, so let's convert the field
names to uppercase as soon as we receive them. This makes the API
case-insensitive and gives us the assurance that all field names
we ever work with are in uppercase.
Related: SYS#7725
Change-Id: I9d80752587e2ccff0963c10abd5a2f42f5868d79
The CardKeyProviderCsv class implements a column decryption scheme
where columns are protected using a transport key. The CSV files
are enrcypted using contrib/csv-encrypt-columns.py.
The current implementation has two main problems:
- The decryption code in CardKeyProviderCsv is not specific to CSV files.
It could be re-used in other formats, for example to decrypt columns
(fields) red from a database. So let's split the decryption code in a
separate class.
- The encryption code in csv-encrypt-columns.py accesses methods and
properties in CardKeyProviderCsv. Also having the coresponding
encryption code somewhere out of tree may be confusing. Let's improve
the design and put encryption and decryption functions in a single
class. Let's also make sure the encryption/decryption is covered by
unittests.
Related: SYS#7725
Change-Id: I180457d4938f526d227c81020e4e03c6b3a57dab
There is no unit-test for the CardKeyProviderCsv class yet. Let's add
one to ensure that the CardKeyProviderCsv class keeps working as expected.
Related: SYS#7725
Change-Id: I52519847a4c4a13a7bca49985133872b01c4aaab
In many sub modules we still use print() to occassionally print status
messages or warnings. This technically does not hurt, but it is an unclean
solution which we should replace with something more mature.
Let's use python's built in logging framework to create a static logger
class that fits our needs. To maintain compatibility let's also make sure
that the logger class will behave like a normal print() statement when no
configuration parameters are supplied by the API user.
To illustrate how the approach can be used in sub-modules, this patch
replaces the print statements in runtime.py. The other print statements
will the be fixed in follow-up patches.
Related: OS#6864
Change-Id: I187f117e7e1ccdb2a85dfdfb18e84bd7561704eb
With the introduction of using osmocom.construct.{Bytes,GreedyBytes}
in Change-Id I1c8df6350c68aa408ec96ff6cd1e405ceb1a4fbb we don't have a
need for wrapping each instance of Bytes or GreedyBytes into a
HexAdapter anymore. The osmocom.construct.{Bytes,GreedyBytes} will
automatically perform the related hex-string-to-bytes conversion if
needed - and during printing we have osmocom.utils.JsonEncoder that
makes sure to convert any bytes type to a hex-string.
Change-Id: I9c77e420c314f5e74458628dc4e767eab6d97123
Installing JAVA-card applets from a CAP file is a multi step process, which is
difficult when done manually. Fortunately it is easy to automate the process,
so let's add a dedicated command for that.
Change-Id: I6cbd37f0fad5579b20e83c27349bd5acc129e6d0
Related: OS#6679
To install JAVA-card applets we need to be able to extract the executeable
loadfile and the AIDs of the applet and the loadfile. This patch adds the
parser and related unittests.
Related: OS#6679
Change-Id: I581483ccb9d8a254fcecc995fec3c811c5cf38eb
GPD_SPE_013 Table 6-3 defines two types of AID-REF-DO objects (both
are fully independed TLV IEs with the same name). The version with
tag '4F' identifies an SE application. It may contain an AID prefix
or even be of length 0 in case the rule should apply to all SE
applications. Then there is the version with tag 'C0', which must
always have length 0 and serves a flag to apply the rule to the
implicitly selected SE application. Technically both are completely
different things, so we must also treat them separately in the
pySim-shell code.
Related: OS#6681
Change-Id: I771d5e860b12215280e3d0a8c314ce843fe0d6a2
The card initialization normally takes place automatically. Nearly all
testcases implicitly cover this code-path. However, it is also possible
to skip the card initialization and do it at some later point. This is
commonly the case for unprovisioned card that require some custom APDUs
in a basic initialization step. When this step is done one would use
the "equip" command to level up to the full featured mode. This patch
adds a testcase for this scenario
Related: OS#6367
Change-Id: I01a03fa07d8c62164453bd707c5943288ff1a972
ETSI TS 102 221, section 7.3 specifies that UICCs (and eUICCs) may support two
different transport protocols: T=0 or T=1 or both. The spec also says that the
terminal must support both protocols.
This patch adds the necessary functionality to support the T=1 protocol
alongside the T=0 protocol. However, this also means that we have to sharpen
the lines between APDUs and TPDUs.
As this patch also touches the low level interface to readers it was also
manually tested with a classic serial reader. Calypso and AT command readers
were not tested.
Change-Id: I8b56d7804a2b4c392f43f8540e0b6e70001a8970
Related: OS#6367
When a pySim-shell command is not recognized, cmd2 prints "xyz is not a
recognized command, alias, or macro." This string is a normal print out
and not an exception, but during tests, it may point out a severe problem
and therefore it should be tread like an exception.
Related: OS#6367
Change-Id: I17be6af1547b31170622e17b9cfb9c492597670d
by default pySim-shell does all kinds of probing and file selection
on startup. This is to determine the card type and to find a suitable
card profile. However, in case the card is non yet provisioned this
probing may cause a error messages and even might upset the cards
internal state. So let's have a commandline option thrugh which we
can instruct pySim-shell to skip any initialization and to give us
a prompt immediately, so that we can enter custom APDUs
Related: OS#6367
Change-Id: I1d8a57de201fe7ad7cbcbc6f72969ea8521e821d
When pySim-shell has problems starting up, it exits with an error
code. This is detected by the testsuite, but it also causes an
early exit, so that the log file content are not printed.
Change-Id: Ic0f34eda32a7c557810abcb05a84e343741fdb8a
The tests that check the establishment of a secure channel currently only test
security level 3. Also the get_data command after it only tests data reception
from the card.
Let's extend the test coverage and test the SCP establishment for security
level 1 as well. Let's also add a get_status command to make sure sending data
to the card also works (without exceptions).
Related: OS#6367
Change-Id: Idff40b414a249e532df1bdce2a8deb9b0cb9718f
When we configure the tests to display file content, we only display files that
we compare, let's also display log file contents from pySim-shell. This will
be useful in situations where we only have log output from the tests, but no
access to the file system of the test host.
Related: OS#6601
Change-Id: Ibf6f78d7e71c213c7ca1caaf21c4c890e892261e
When pySim-shell is called by a testcase, a logfile is createted. The logfile
filename contains the testcase name. However, a testcase may run pySim-shell
multiple times. In this case we overwrite the log from previous run. Let's use a
counter to generate unique file names for each run, so that we won't lose logs
from previous runs.
Related: OS#6601
Change-Id: Ib2195d9b2231f74d0a6c4fb28f4889b6c45efb1e
When we get rid of temporary files, we delete those using a wildcard,
but for the logs from pySim-shell we explicitly memorize the name
of the pySim-shell logfile and delete it later by this explicit
name. This is not necessary, let's just delete all log files present
using a wildcard.
Related: OS#6601
Change-Id: I09dc7e59d1a3dcb68f54e3a8dccb86a1bc6c9ee6
This adds a construct + pyosmocore.tlv based declarative encoder/decoder
for the EF.EARFCNList file used in the context of NB-IoT in later
release USIMs.
Change-Id: I16797ca58c3ad6ebaf588d04fec011a0cbcfcef3
The problems with test_list_and_rm_notif (see also change id
I7d0b6a998499d84f0eb4e24592ad43210ac54806) are now resolved, so
we can re-enable the testcase now.
Closes: SYS#7094
Change-Id: I95eb3b9c02a69653797851197e882ea9316805fc
Even though our tests are written in a way that they shouldn't interfere
with each other, it may happen that one testrun writes content to a file
that upsets a different testrun. The resulting problems are often
difficult to diagnose.
To minimize the problem, let's add code that can reset the cards to a
defined state. This can be done using pySim-shell's export
feature. We can generate a backup from a known good state and then play
back the backup to reset files that have been changed. Files that didn't
change will not be written thanks to the conserve_write feature of
pySim-shell.
Related: OS#4384
Change-Id: I42eaf61280968518164f2280245136fd30a603ce
We now have a construct based encoder/decoder for the record content
of EF.MSISDN. This means that we do not need the functions enc_msisdn
and dec_msisdn in the non-legacy code anymore. We can now move both
to legacy/utils.py.
Related: OS#5714
Change-Id: I19ec8ba14551ec282fc0cc12ae2f6d528bdfc527
The encoding of EF.MSISDN is currently done with enc_msisdn and
dec_msisdn from utils.py. Let's replace this with a construct
based model, similar to the one we already use with EF.ADN
Related: OS#5714
Change-Id: I647f5c63f7f87902a86c0c5d8e92fdc7f4350a5a
In our construct models we frequently use a context parameter "total_len",
we also pass this parameter to construct when we decode files, but we
do not pass it when we generate files. This is a problem, because when
total_len is used in the construct model, this parameter must be known
also when decoding the file.
Let's make sure that the total_len is properly determined and and passed
to construct (via pyosmocom).
Related: OS#5714
Change-Id: I1b7a51594fbc5d9fe01132c39354a2fa88d53f9b
The testcase euicc.test_list_and_rm_notif fails due to a problem
with the eUICC. The eUICC reports the following error when a
delete notification attempt is made:
"delete_notification_status": "undefinedError"
Let's temporarily disable this testcase until the problem is resolved.
Change-Id: I7d0b6a998499d84f0eb4e24592ad43210ac54806
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.
Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
Our test cards need to stay recognizable, so it is important that
each card has a unique ICCID. This means we must write an individual
ICCID, when we test writing the ICCID.
Related: OS#4384
Change-Id: I858a35e526e7b4868e901222d587258412779f41
The sysmoISIM-SJA2 does not support changing of the ICCID.
pySim-prog will also reject this, so let's remove the ICCID
from the parameter list.
Related: OS#4384
Change-Id: I89571f2bf7c4cec4d621c322a58687b7781b0ed2
We're creating a 'pyosmocom' pypi module which contains a number of core
Osmocom libraries / interfaces that are not specific to SIM card stuff
contained here.
The main modules moved in this initial step are pySim.tlv, pySim.utils
and pySim.construct. utils is split, not all of the contents is
unrelated to SIM Cards. The other two are moved completely.
Change-Id: I4b63e45bcb0c9ba2424dacf85e0222aee735f411
When probing for cards, the probing might fail in case the terminal
is empty. This results into lengthy error log output that is not
of interest.
Related: OS#6532
Change-Id: I1d44f9458a05992d79b0152d3affcfeb783cccff
When the test detects a card, but does not find the .data faile for
it, then it fails. This can be a problem in case we want to intentionally
exclude a specific card model.
Related: OS#6532
Change-Id: Iba196ada0076385de7bffcb157a85fda0a6c1852
The test currently expects all reader slots to be populated. This means
the cards may plugged into a random order, but there may not be any empty
slots in the system at all. This requirement is easy to meet when each
card has its own single-slot USB PCSC-reader, but as soon as multislot
readers are used there may be some empty slots.
Related: OS#6532
Change-Id: I7ba1d1350b6998d65e90408184accdb212556a7c
We currently have the shell script that performs the test in the
tests directory and the related data in pysim-testdata directory.
This is confusing, let's have evrything in a dedicated sub directory
Change-Id: Ic995a7f600d164fc0be3c2eb8255dbe043429bea
Related: OS#6531
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.
Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.
Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
In the past, we always wrapped a HexAdapter around bytes-like data in
order to make sure it's printed as hex-digits. However, now that we are
doing JSON output it's much easier to let the pySim.utils.JsonEncoder
take care of this in a generic way.
We should do a similar migration all over pySim (pySim-shell,
filesystem, etc.) - but for now only do it in the low-hanging fruit of
pySim-trace aka pySim.apdu
Change-Id: I0cde40b2db08b4db9c10c1ece9ca6fdd42aa9154
This implements the first parts of the "GlobalPlatform Remote
Application Management over HTTP Card Specification v2.3 - Amendment B,
Versoin 1.2". Specifically, this patch covers the TLV definitions for
the OTA message used for HTTPS session triggering.
This also adds some more unit test coverage to pySim.cat, based on
real-world data that was captured nested inside the HTTPS Administration
session triggering parameters.
Change-Id: Ia7d7bd6df41bdf1249011bad9a9a38b7669edc54
the COMPACT-TLV variant is a TLV variant that ISO7816 uses for encoding
tag and length into a single octet. This is used (for example) in ATR
historical bytes.
Let's add support for this to our pySim TLV encoder/decoder.
Change-Id: I9e98d150b97317ae0c6be2366bdaaeaeddf8031c
Let's make sure the constructor of ProfileElement subclasses set
meaningful defaults to the self.decoded member, so that the to_der()
method can actually encode it. This is required when constructing
a profile from scratch, as opposed to loading an existing one from DER.
Also, add a test to verify that the encoder passes without exception;
doesn't test the generated binary data.
Change-Id: I401bca16e58461333733877ec79102a5ae7fe410
So far we only implemented command encoding and response decoding.
Let's also add command decoding, which is useful for example when
decoding protocol traces.
Change-Id: Id666cea8a91a854209f3c19c1f09b512bb493c85
