This commit introduces the capability to parse and encode
SimAlliance/TCA "Interoperable Profiles" and apply personalization
operations on them.
Change-Id: I71c252a214a634e1bd6f73472107efe2688ee6d2
This commit introduces
* the osmo-smdpp.py program implementing the main procedures and the
HTTP/REST based ES9+
* python modules for ES8+ and non-volatile RSP Session State storage
* the ASN.1 source files required to parse/encode RSP
* 3GPP test certificates from SGP.26
* an unsigned profile package (UPP) of a SAIP v2.3 TS48 test profile
As I couldn't get the 'Klein' tls support to work, the SM-DP+ code
currently does not support HTTPS/TLS but plan HTTP, so you either have
to modify your LPA to use HTTP instead of HTTPS, or put a TLS proxy in
front.
I have successfully installed an eSIM profile on a test eUICC that
contains certificate/key data within the test CI defined in GSMA SGP.26
Change-Id: I6232847432dc6920cd2bd08c84d7099c29ca1c11
Contrary to {enable,disable}_profile, the delete_profile does not use
the ProfileIdentifier TLV, but directly the Iccid / IsdpAid.
Change-Id: I43e298524048703264e16cbdd0b76d82ba976985
This tool is a replacement for asn1c 'unber' program with a much more
useful/readable output:
* contains hexadecimal raw tag values
* contains hexdump of value, rather than HTML entities in pseudo-XML
Change-Id: I22c1a461ccba04c2c8caaab7ca29ea6ae76e2ea3
This is the protocol used for the ES8+ interface between SM-DP+ and the
eUICC in the GSMA eSIM system.
Change-Id: Ic461936f2e68e1e6f7faab33d06acf3063e261e7
The definitions are not used yet, as one would have to add that
dynamically based on which EF.DIR entries contain the 0x73 discretionary
template. As I don't have any cards implementing this so far, I'll skip
that part.
Change-Id: I532ff2c94021ab1b4520fe2b6988c8960319d208
In smart cards, files/records containing all-ff means they are simply
not used/initialized. Let's avoid raising exceptions when interpreting
0xff as length value and reading less bytes as value.
Change-Id: I09c3cb82063fc094eb047749996a6eceff757ea2
We cannot fully switch to construct for all of it easily due to
the priority value and the ordering/sorting by priority implemented
in the hand-coded version. But we can at least migrate the
encode/decode of the hnet_pubkey_list via construct.
Change-Id: I4ad5ea57bab37c2dc218e7752d538aa4cdc36ee3
If we are reading a file to check if we can skip the write to conserve
writes, don't treat exceptions as fatal. The file may well have the
access mode in a way that permits us to UPDATE but not to READ. Simply
fall-back to unconditional UPDATE in this case.
Change-Id: I7bffdaa7596e63c8f0ab04a3cb3ebe12f137d3a8
this has the advantage of getting the encoder for free (so far we only
had the decoder). While at it, also add some tests data for the unit
tests.
Change-Id: Ifb8caf5cd96706d7fb6b452d6552b115c0828797
It's customary in the SIM card universe to right-pad data with ff bytes.
So far we only test decoders without such padding, which is unrealistic.
Let's also tests the decoders with extra 'ff' padding present.
For some files this doesn't make sense, so we add a _test_no_pad class
attribute that can be spcified to prevent this new "test with ff-padding"
from being executed for the test data of the class.
Change-Id: I7f5cbb4a6f91040fe9adef9da0a1f30f9f156dae
The existing code used to produce an empty output in situations where a
TLV_IE_Collection would be parsed from a single TLV only with some
additional trailing padding:
>>> from pySim.utils import h2b
>>> from pySim.ts_31_102 import EF_CSGT
>>> t = EF_CSGT.Csgt_TLV_Collection()
>>> t.from_tlv(h2b('8906810300666f6fff'))
[TextCsgType(foo)]
>>> t.to_dict()
[]
This was caused by an early return (actually returning the decoded
result) but *without updating self.children*.
Change-Id: I1c84ccf698c6ff7e7f14242f9aaf7d15ac2239f4
These files are mostly related to CSG (Closed Subscriber Group)
in the context of HomeNodeB (HNB), aka femtocells.
Change-Id: Ie57963381e928e2c1da408ad46549a780056242a
Now that we have support for the UCS-2 encoding as per TS 102 221 Annex A,
we can start to make use of it from various file constructs.
As some specs say "Either 7-bit GSM or UCS-2" we also introduce
a related automatic GsmOrUcs2Adapter and GsmOrUcs2String class.
Change-Id: I4eb8aea0a13260a143e2c60fca73c3c4312fd3b2
TS 102 221 Annex A defines three variants of encoding UCS-2 characters
into byte streams in files on UICC cards: One rather simplistic one, and
two variants for optimizing memory utilization on the card.
Let's impelement a construct "Ucs2Adapter" class for this.
Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
The human representation of a PLMN is usually MCC-MNC like 262-01
or 262-001. Let's add a PlmnAdapter for use within construct, so we
can properly decode that.
Change-Id: I96f276e6dcdb54a5a3d2bcde5ee6dbaf981ed789
We used __subclasses__(), but this only returns the immediate
subclasses and not all further/nested subclasses. Instead, we must
use the pySim.utils.all_subclasses() function to really get all of them.
The hack to use the method signature of the constructor to determine if
it's an intermediate class didn't work, as even GlobbalPlatformISDR
has a optional argument for non-default AIDs. So let's introduce an
explicit class attribute for that purpose.
Change-Id: I7fb1637f8f7a149b536c4d77dac92736c526aa6c
This adds some first test data for the new unitdata driven test cases
for the TLV encoder/decoder.
It also fixes a bug in the ts_102_221.FileDescriptor decoder for BER-TLV
structured files which was found and fixed while introducing the test
data.
Related: OS#6317
Change-Id: Ief156b7e466a772c78fb632b2fa00cba2eb1eba5
While we do have the _test_de_encode data driven tests for file
definitions, we don't yet have something similar for derived classes of
BER_TLV_IE. This means that TLVs used outside of the filesystem context
(for example, decoding the SELECT/STATUS response, but also eUICC and
other stuff) do not yet have test coverage.
This commit just adds the related test code, but no test data yet.
Related: OS#6317
Change-Id: Ied85f292bb57fde11dc188be84e3384dc3ff1601
the to_dict() method generates a {class_name: value} dictionary,
for both the nested and non-nested case. However, before this patch,
the from_dict() method expects a plain list of child IE dicts
in the nested case. This is illogical.
Let's make sure from_dict always expectes a {class_name: value} dict
for both nested and non-nested situations.
Change-Id: I07e4feb3800b420d8be7aae8911f828f1da9dab8
We recently introduced a pySim.utils.build_construct() wrapper around
the raw call of the construct.build() method. So far, this wrapper
was only used from pySim.tlv, but let's also use it from
pySim.filesystem.
Basically, whenever we use parse_construct(), we should use
build_construct() as the inverse operation.
Change-Id: Ibfd61cd87edc72882aa66d6ff17861a3e918affb
on some systems, the output would otherwise contain colored status
words, which in turn mean the test otuput no longer matches the expected
output.
Change-Id: Icb700f6e85a285748e00367a398975aa5e75dec5
The context is some opaque dictionary that can be used by the
constructs; let's allow the caller of parse_construct, from_bytes,
from_tlv to specify it.
Also, when decoding a TLV_IE_Collection, pass the decode results of
existing siblings via the construct.
Change-Id: I021016aaa09cddf9d36521c1a54b468ec49ff54d
The EF.CFIS definition is not identical to EF.ADN, so we cannot recycle
the EF.ADN class to decode EF.CFIS.
Change-Id: Idcab35cbe28332e3c8612bcb90226335b48ea973
There are some pretty intricate rules about how GSM and E-UTRAN are
encoded, let's make sure we fully support both as per 3GPP TS 31.102
Release 17. As part of this, switch to a sorted list of access technologies,
in order to have a defined order. This makes comparing in unit tests
much easier. However, it also means that we need to sort the set
when printing the list of AcT in pySim-read to generate deterministic
output.
Change-Id: I398ac2a2527bd11e9c652e49fa46d6ca8d334b88
Without this the diff between expected and actual output is truncated
and one instead reads the following output:
Diff is 844 characters long. Set self.maxDiff to None to see it.
We actually want to see the full diff to see what's not matching.
Change-Id: I6e89705061454191b6db1255de7fe549ad720800
This driver description we add to the code is automatically added to the
respective user manual sections.
Change-Id: I8807bfb11f43b167f1321d556e09ec5234fff629
Let's avoid copy+pasting print statements everywhere. The instances
do already have a __str__ method for the purpose of printing their name in a
generic way.
Change-Id: I663a9ea69bf7e7aaa6502896b6a71ef692f8d844
Opening PC/SC readers by index/number is very error-prone as the order
is never deterministic in any system with multiple (hot-plugged, USB)
readers. Instead, let's offer the alternative of specifying a regular
expression to match the reader name (similar to remsim-bankd).
Change-Id: I983f19c6741904c1adf27749c9801b44a03a5d78
It's odd that the individual transport driver specifies their argparse
options but then the core transport part evaluates them individually.
This means we cannot add new options within a transport.
Let's pass the Namespace instance into the constructor of the
specific transport to improve this.
Change-Id: Ib977007dd605ec9a9c09a3d143d2c2308991a12c
In the previous commit we've stopped using those functions from modern
pySim-shell code. Hence, the only remaining user is the legacy tools,
so we can move the code to the legacy module.
Change-Id: I6f18ccb36fc33bc204c01f9ece135676510e67ec
We've recently introduced IPv{4,6}Adapter construct classes and can
switch to this instead of using the old imperative encoder/decoder
functions {enc,dec}_addr_tlv().
Aside from code cleanup, this also means we now support the IPv6 address
type in EF.PCSCF.
Change-Id: I4d01ccfe473a8a80fbee33fdcbd8a19b39da85ac
This can happen if there's a file with invalid encoding on the card,
such as a tag followed by all-ff. Let's gracefully ignore it and
return zero bytes as response.
Change-Id: Ic44557368a6034dbf4bb021ab23a57927c22def0
The metaclass uese the 'nested' attribute, while the existing code
accidentially used the 'children' attribute. The latter is used
by instances for actual child classes, while the Class/nested
attribute is for the list of classes whose instancse could be potential
children.
Change-Id: I968bd84d074dcdcec37d99be5d3d4edac9c35a0c
The length value "of course" is a hex value, don't use %02u but %02x
This fixes any eUICC command with a Lc > 10 bytes.
Change-Id: I1e1efbfb9916fc43699602cc889cf4b3d42736f2
The encoding was missing a "CHOICE" container and missed the
fact that the refreshFlag presence is mandatory for enable+disable.
Change-Id: I12e2b16b2c1b4b01dfad0d1fb485399827f25ddc
If a TLV was elementary (no nested IEs), and it had only a single
integer content whose value is 0, we erroneously encoded that as
zero-length TLV (len=0, no value part):
>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b''
>>> rf.to_tlv()
b'\x81\x00'
After this change it is correct:
>>> rf = pySim.euicc.RefreshFlag(decoded=0);
>>> rf.to_bytes()
b'\x00'
>>> rf.to_tlv()
b'\x81\x01\x00'
Change-Id: I5f4c0555cff7df9ccfc4a56da12766d1bf89122f
If we have a list of dicts, and we flatten that into a dict: Only
do that if there are no dicts with duplocate key values in the list,
as otherwise we will loose information during the transformation.
Change-Id: I7f6d03bf323a153f3172853a3ef171cbec8aece7
Closes: OS#6288
Ever since commit 30de9fd8ab in July
we are (properly) using snake_case names in the from_dict (to become
bijective with to_dict). This code was not updated by accident,
creating an exception when using the `aram_get_config`
Change-Id: If216b56b38ab17d13896074aa726278b9ba16923
Related: OS#6119
So far we implemented only one round of "Send the APDU, get SW=61xx,
call GET RESPONSE". This permitted us to receive only data up to 256
bytes.
Let's extend that to doing multiple rounds, concatenating the result.
This allows us to obtain arbitrary-length data from the card.
See Annex C.1 of ETSI TS 102 221 for examples showing multiple 61xx
iterations.
Change-Id: Ib17da655aa0b0eb203c29dc92690c81bd1300778
Closes: OS#6287
The point of this is to move generic code out of pySim-shell.py,
paving the way for more/other executables using the full power of
our class model without having to reinvent the wheel.
Change-Id: Icf557ed3064ef613ed693ce28bd3514a97a938bd
It's better for the human reader (and more obvious that it's a boolean
value) if we decode single Bits as True/False instead of 1/0.
Change-Id: Ib025f9c4551af7cf57090a0678ab0f66a6684fa4
This increases test coverage and also shows where we so far only
have decoders but no encoders yet
Change-Id: I7932bab7c81a2314c1b9477f50b82a46f24d074e
Don't even send any non-decimal PIN values to the card, but reject
them when parsing the command arguments.
Change-Id: Icec1698851471af7f76f20201dcdcfcd48ddf365
before this patch:
pySIM-shell (00:MF)> echo foo bar baz
usage: echo [-h] string
echo: error: unrecognized arguments: bar baz
after this patch:
pySIM-shell (00:MF)> echo foo bar baz
foo bar baz
Change-Id: I1369bc3aa975865e3a8a574c132e469813a9f6b9
Let's make sure we don't even bother to ask the card to verify
anything as ADM1 pin which is not either a sequence of decimal digits
or an even number of hex digits (even number of bytes).
Change-Id: I4a193a3cf63462fad73d145ab1481070ddf767ca
Let's add a proper argparser instance for the 'verify_adm' command,
avoiding situations where the user types 'verif_adm --help' and then
--help is interpreted as the PIN value, removing one more attempt from
the failed ADM1 counter.
Let's use that opportunity to improve the documentation of the command.
Change-Id: I3321fae66a11efd00c53b66c7890fce84796e658
pySim-shell output has changed over time, so some examples were
showing outdated content. Let's update those.
Change-Id: I4058719c32b61689522e90eba37253e8accb8ba5
this is far from being complete, just some basic first commands
to get the certificates and eIM configuration.
Change-Id: Ie05108e635ed9c6de10f0ba431cb1b13893f6be8
This just adds basic support for the ISD-R application and its
associated STORE DATA command which is used for the ES10x interfaces
between off-card entities and the on-card ISD-R.
Change-Id: Ieab37b083e25d3f36c20f6e9ed3e4bdfdd14a42a
Closes: OS#5637
We used to support only single-byte tags in bertlv_encode_tag,
let's fix that. The easy option is to simply call bertlv_parse_tag,
as that already supported multi-byte tags.
Change-Id: If0bd9137883c4c8b01c4dfcbb53cabeee5c1ce2b
Now that pySim-shell can switch between logical channels, let's state
the currently used logical channel in the prompt.
Change-Id: I45781a6fba205eeb4ac7f58d5cb642b7131bdd88
Related: OS#6230
We've already had the 'open_channel' and 'close_channel' commands,
which were sent to (and acknowledged by) the card. However,
those commands didn't affect the pySim-shell state, i.e. all
communication would still happen through the default channel '0'.
With this patch we introduce a 'switch_channel' command, using which
the user can determine which of the (previously opened) logical channels
shall be used by pySim-shell.
Change-Id: Ia76eb45c4925882ae6866e50b64d9610bd4d546d
Closes: OS#6230
Now that pySim-shell is aware of logical channels and issues almost
all of its APDUs on the currently selected channel, we must also make
sure that ADF selection by AID (implemented by the CardBase class)
issues the SELECT on the respective logical channel.
Before this patch, SELECT ADF by AID would always be issued on the
primary logical channel (0), irrespective of the currently active
RuntimeLchan.
Change-Id: Idf05c297e6a2e24ca539408b8912e348c0782bb4
Related: OS#6230
This new approach will "fork" separate SimCardCommands instances
for each RuntimeLchan. Higher-layer code should now always use the
RuntimeLchan.scc rather than the RuntimeState.card._scc in order to
make sure commands use the correct logical channel.
Change-Id: I13e2e871f2afc2460d9fd1cd566de42267c7d389
Related: OS#6230
Historically we always only had one instance of SimCardCommands, but
with this patch we can now have multiple instances, one for each lchan.
The SimCardCommands class is aware of the logical channel it runs on
and will patch the CLA byte accordingly.
Change-Id: Ibe5650dedc0f7681acf82018a86f83377ba81d30
Related: OS#6230
This fixes the below error during build of the documentation:
pysim/docs/shell.rst:349: ERROR: "<class 'pySim-shell.PySimCommands'>" has no attribute "apdu_cmd_parser"
Change-Id: If89b66a45ea18b5a3fc56bf77b05e679463da5a8
We already have the open channel and close_channel commands in
pySim-shell. They are sent to the card and acknowledged, respectively.
We also already do have code that can track multiple different logical
channels (the rs.lchan array). However, this is currently only used by
pySim-trace, and not by pySim-shell. Let's change that.
Change-Id: Idacee2dc57e8afe85c79bc85b259064e7f5b83a2
Related: OS#6230
While our base classes (TransparentEF / LinFixedEF) always have the
dsecription as 4th argument after "fid, sfid, name", most of the derived
file-specific classes do not share that same argument order.
As seen in the bug fixed by previous Change-Id I7f32c9fd01094620b68b0e54536ecc6cdbe67903
this can have serious consequences. Let's avoid using unnamed
(positional) arguments for the description text altogether.
Change-Id: Icfb3fd1bae038c54fa14a91aa9f75219d839968c
We were using positional arguments when instantiating instances
of classes like EF_5GS3GPPLOCI with non-default names/fids/...
However, we got the argument order wrong and were passing the
description string in the position of the file size, which causes
exceptions like the following from pySim-trace:
Traceback (most recent call last):
File "/home/laforge/projects/git/pysim/./pySim-trace.py", line 198, in <module>
tracer.main()
File "/home/laforge/projects/git/pysim/./pySim-trace.py", line 125, in main
inst.process(self.rs)
File "/home/laforge/projects/git/pysim/pySim/apdu/__init__.py", line 259, in process
self.processed = method(self.lchan)
File "/home/laforge/projects/git/pysim/pySim/apdu/ts_102_221.py", line 152, in process_on_lchan
if self.cmd_dict['offset'] != 0 or self.lr < self.file.size[0]:
TypeError: '<' not supported between instances of 'int' and 'str'
Let's use named initializers for any arguments after the usual "fid, sfid, name"
initial arguments.
Change-Id: I7f32c9fd01094620b68b0e54536ecc6cdbe67903
This adds a new operation mode for pySim-shell, where a single command
can be passed to pySim-shell, which then is executed before pySim-shell
terminates.
Example: ./pySim-shell.py -p0 export --json
Change-Id: I0ed379b23a4b1126006fd8f9e7ba2ba07fb01ada
Closes: OS#6088
Rather than having to know and explicitly list every CardApplication,
let's iterate over the __subclasses__ of the CardApplication base class.
Change-Id: Ia6918e49d73d80acfaf09506e604d4929d37f1b6
This profile has always been a hack/work-around for the situation that
a classic GSM SIM is not a UICC, and we didn't yet have the concept of
CardProfileAddons yet, so there was no way to probe and add something
to an UICC which was not an application with its own AID/ADF.
Since now we have CardProfileAddons (including one for GSM SIM),
and pySim-trace (the other user of CardProfileUICCSIM) has also switched
over to using CardProfileUICC + addons, we can remove this work-around.
Change-Id: I45cec68d72f2003123da4c3f86ed6a5a90988bd8
We have a strict "one CardProfile per card" rule. For a modern UICC
without legacy SIM support, that works great, as all applications
have AID and ADF and can hence be enumerated/detected that way.
However, in reality there are mostly UICC that have legacy SIM, GSM-R
or even CDMA support, all of which are not proper UICC applications
for historical reasons.
So instead of having hard-coded hacks in various places, let's introduce
the new concept of a CardProfileAddon. Every profile can have any
number of those. When building up the RuntimeState, we iterate over the
CardProfile addons, and probe which of those are actually on the card.
For those discovered, we add their files to the filesystem hierarchy.
Change-Id: I5866590b6d48f85eb889c9b1b8ab27936d2378b9
Harald Welte