Commit Graph

988 Commits

Author SHA1 Message Date
Harald Welte
4fe7de8568 ts_31_103: Add construct for EF.GBABP and EF.GBANL
Change-Id: Ife06f54c2443f3e048bd36f706f309843703403a
2024-01-04 21:27:39 +01:00
Harald Welte
b0c9ccba66 construct: avoid StreamError exceptions due to files containing all-ff
In smart cards, files/records containing all-ff means they are simply
not used/initialized.  Let's avoid raising exceptions when interpreting
0xff as length value and reading less bytes as value.

Change-Id: I09c3cb82063fc094eb047749996a6eceff757ea2
2024-01-04 21:20:19 +01:00
Harald Welte
e13403b206 ts_31_102: Start to use construct for EF.SUCI_Calc_Info
We cannot fully switch to construct for all of it easily due to
the priority value and the ordering/sorting by priority implemented
in the hand-coded version.  But we can at least migrate the
encode/decode of the hnet_pubkey_list via construct.

Change-Id: I4ad5ea57bab37c2dc218e7752d538aa4cdc36ee3
2024-01-04 20:13:06 +01:00
Harald Welte
9a48aea263 fileystem/tlv: remove unused imports
Change-Id: I519c7792c7fbe18be63ddc77d211f0d034afcd1f
2024-01-02 21:13:30 +01:00
Harald Welte
19d2b93d7e move SUCI sub-classes to EF_SUCI_CalcInfo
Change-Id: Iea6b176327881ff9414f4fe624e94811f9782927
2023-12-29 18:51:25 +01:00
Harald Welte
9d607978fa global_platform: Add support for more GET DATA TLVs
Example:

pySIM-shell (00:MF/ADF.ISD)> get_data extended_card_resources_info
{
    "extended_card_resources_info": [
        {
            "number_of_installed_app": 8
        },
        {
            "free_non_volatile_memory": 354504
        },
        {
            "free_volatile_memory": 10760
        }
    ]
}

Change-Id: I129e43c377b62dae1b9a88a0a2dc9663ac2a97da
2023-12-29 18:51:25 +01:00
Harald Welte
1c0a249131 commands: Ignore exceptions during READ while UPDATE
If we are reading a file to check if we can skip the write to conserve
writes, don't treat exceptions as fatal.  The file may well have the
access mode in a way that permits us to UPDATE but not to READ.  Simply
fall-back to unconditional UPDATE in this case.

Change-Id: I7bffdaa7596e63c8f0ab04a3cb3ebe12f137d3a8
2023-12-29 18:51:25 +01:00
Harald Welte
db1684df04 sysmocom_sja2: Implement EF_CHV files using construct
this has the advantage of getting the encoder for free (so far we only
had the decoder).  While at it, also add some tests data for the unit
tests.

Change-Id: Ifb8caf5cd96706d7fb6b452d6552b115c0828797
2023-12-29 18:51:25 +01:00
Harald Welte
ce01f48b00 test_files: Test decoder also with ff-padded input
It's customary in the SIM card universe to right-pad data with ff bytes.
So far we only test decoders without such padding, which is unrealistic.
Let's also tests the decoders with extra 'ff' padding present.

For some files this doesn't make sense, so we add a _test_no_pad class
attribute that can be spcified to prevent this new "test with ff-padding"
from being executed for the test data of the class.

Change-Id: I7f5cbb4a6f91040fe9adef9da0a1f30f9f156dae
2023-12-29 18:51:25 +01:00
Harald Welte
bcd261583c tests_files.py: Reduce code duplication
Change-Id: Ib84a0ae35262a19fce3e688afe8e1678a4c59eba
2023-12-29 18:51:25 +01:00
Harald Welte
69bdcf5022 Fix TLV_IE_Collection.from_tlv in certain situations
The existing code used to produce an empty output in situations where a
TLV_IE_Collection would be parsed from a single TLV only with some
additional trailing padding:

>>> from pySim.utils import h2b
>>> from pySim.ts_31_102 import EF_CSGT
>>> t = EF_CSGT.Csgt_TLV_Collection()
>>> t.from_tlv(h2b('8906810300666f6fff'))
[TextCsgType(foo)]
>>> t.to_dict()
[]

This was caused by an early return (actually returning the decoded
result) but *without updating self.children*.

Change-Id: I1c84ccf698c6ff7e7f14242f9aaf7d15ac2239f4
2023-12-29 18:51:25 +01:00
Harald Welte
a77f7e1eb9 ts_31_102: Implement decoders/encoders for EFs below DF.HNB
These files are mostly related to CSG (Closed Subscriber Group)
in the context of HomeNodeB (HNB), aka femtocells.

Change-Id: Ie57963381e928e2c1da408ad46549a780056242a
2023-12-29 18:51:25 +01:00
Harald Welte
6e6caa8b4a support UCS-2 characters in EF.MMSUP, EF.ADN, EF.SPN, EF.PNN, EF.ECC
Now that we have support for the UCS-2 encoding as per TS 102 221 Annex A,
we can start to make use of it from various file constructs.

As some specs say "Either 7-bit GSM or UCS-2" we also introduce
a related automatic GsmOrUcs2Adapter and GsmOrUcs2String class.

Change-Id: I4eb8aea0a13260a143e2c60fca73c3c4312fd3b2
2023-12-29 18:51:25 +01:00
Harald Welte
f6fceb8684 Implement convoluted encoding of UCS-2 as per TS 102 221 Annex A
TS 102 221 Annex A defines three variants of encoding UCS-2 characters
into byte streams in files on UICC cards: One rather simplistic one, and
two variants for optimizing memory utilization on the card.

Let's impelement a construct "Ucs2Adapter" class for this.

Change-Id: Ic8bc8f71079faec1bf0e538dc0dfa21403869c6d
2023-12-29 18:51:21 +01:00
Harald Welte
842fbdb15d add PlmnAdapter for decoding PLMN bcd-strings like 262f01 to 262-01
The human representation of a PLMN is usually MCC-MNC like 262-01
or 262-001.  Let's add a PlmnAdapter for use within construct, so we
can properly decode that.

Change-Id: I96f276e6dcdb54a5a3d2bcde5ee6dbaf981ed789
2023-12-28 08:08:54 +01:00
Harald Welte
dffe7af578 Fix enumeration of GlobbalPlatformISDR during card_init()
We used __subclasses__(), but this only returns the immediate
subclasses and not all further/nested subclasses.  Instead, we must
use the pySim.utils.all_subclasses() function to really get all of them.

The hack to use the method signature of the constructor to determine if
it's an intermediate class didn't work, as even GlobbalPlatformISDR
has a optional argument for non-default AIDs.  So let's introduce an
explicit class attribute for that purpose.

Change-Id: I7fb1637f8f7a149b536c4d77dac92736c526aa6c
2023-12-27 22:17:38 +01:00
Harald Welte
722c11a7e9 global_platform: Add support for key types of v2.3.1 (including AES)
Change-Id: Iae30f18435c2b0a349bfd9240b9c7cca06674534
2023-12-27 15:16:03 +00:00
Harald Welte
45626271cf global_platform: Add TLV test data for Key Information Data
Change-Id: Ib7b73cb28abea98986a66264a0779263873d7fb2
2023-12-27 15:15:58 +00:00
Harald Welte
2538dd7621 global_platform: Correctly decode Key Information Data
The list contains tuples of (key_type, key_length). Let's fix that.

Change-Id: Icf367827d62ed67afa27ee3d0ba9d5cd5bc65c99
2023-12-27 15:15:54 +00:00
Harald Welte
ee6a951774 Add TLV decoder test data
This adds some first test data for the new unitdata driven test cases
for the TLV encoder/decoder.

It also fixes a bug in the ts_102_221.FileDescriptor decoder for BER-TLV
structured files which was found and fixed while introducing the test
data.

Related: OS#6317
Change-Id: Ief156b7e466a772c78fb632b2fa00cba2eb1eba5
2023-12-27 15:15:24 +00:00
Harald Welte
2a36c1b921 data-driven TLV unit data test support
While we do have the _test_de_encode data driven tests for file
definitions, we don't yet have something similar for derived classes of
BER_TLV_IE. This means that TLVs used outside of the filesystem context
(for example, decoding the SELECT/STATUS response, but also eUICC and
other stuff) do not yet have test coverage.

This commit just adds the related test code, but no test data yet.

Related: OS#6317
Change-Id: Ied85f292bb57fde11dc188be84e3384dc3ff1601
2023-12-27 15:15:17 +00:00
Harald Welte
a9b21bdb1f tlv: Fix from_dict() symmetry
the to_dict() method generates a {class_name: value} dictionary,
for both the nested and non-nested case.  However, before this patch,
the from_dict() method expects a plain list of child IE dicts
in the nested case.  This is illogical.

Let's make sure from_dict always expectes a {class_name: value} dict
for both nested and non-nested situations.

Change-Id: I07e4feb3800b420d8be7aae8911f828f1da9dab8
2023-12-27 15:14:48 +00:00
Harald Welte
a5eb924f9e filesystem: use pySim.utils.build_construct()
We recently introduced a pySim.utils.build_construct() wrapper around
the raw call of the construct.build() method.  So far, this wrapper
was only used from pySim.tlv, but let's also use it from
pySim.filesystem.

Basically, whenever we use parse_construct(), we should use
build_construct() as the inverse operation.

Change-Id: Ibfd61cd87edc72882aa66d6ff17861a3e918affb
2023-12-23 17:49:01 +01:00
Harald Welte
a4b9bdf238 pySim-trace_test.sh: Force termcolor to suppress color generation
on some systems, the output would otherwise contain colored status
words, which in turn mean the test otuput no longer matches the expected
output.

Change-Id: Icb700f6e85a285748e00367a398975aa5e75dec5
2023-12-23 10:38:21 +01:00
Harald Welte
caef0df663 construct/tlv: Pass optional 'context' into construct decoder/encoder
The context is some opaque dictionary that can be used by the
constructs; let's allow the caller of parse_construct,  from_bytes,
from_tlv to specify it.

Also, when decoding a TLV_IE_Collection, pass the decode results of
existing siblings via the construct.

Change-Id: I021016aaa09cddf9d36521c1a54b468ec49ff54d
2023-12-23 09:15:47 +00:00
Harald Welte
188869568a docs/shell: extend the introduction part; link to video presentation
Change-Id: I77c30921f2b8c002c9dda244656c348c96b41f06
2023-12-23 09:14:59 +00:00
Harald Welte
324175f8bd additional encode/decode test data for various files
Change-Id: Ib563a2204922d2013b5f7c5abde0773051e17938
2023-12-23 08:20:42 +01:00
Harald Welte
5376251993 31.102 + 51.011: Fix encode/decode of EF.CFIS
The EF.CFIS definition is not identical to EF.ADN, so we cannot recycle
the EF.ADN class to decode EF.CFIS.

Change-Id: Idcab35cbe28332e3c8612bcb90226335b48ea973
2023-12-23 08:20:42 +01:00
Harald Welte
542dbf6771 fix encode/decode of xPLMNwAcT
There are some pretty intricate rules about how GSM and E-UTRAN are
encoded, let's make sure we fully  support both as per 3GPP TS 31.102
Release 17.  As part of this, switch to a sorted list of access technologies,
in order to have a defined order.  This makes comparing in unit tests
much easier.  However, it also means that we need to sort the set
when printing the list of AcT in pySim-read to generate deterministic
output.

Change-Id: I398ac2a2527bd11e9c652e49fa46d6ca8d334b88
2023-12-23 08:20:42 +01:00
Harald Welte
e45168ef29 test/test_files: set maxDiff attribute
Without this the diff between expected and actual output is truncated
and one instead reads the following output:

	Diff is 844 characters long. Set self.maxDiff to None to see it.

We actually want to see the full diff to see what's not matching.

Change-Id: I6e89705061454191b6db1255de7fe549ad720800
2023-12-22 09:13:10 +01:00
Harald Welte
2822dca9ec tests: use case-insensitive compare of hex strings
Change-Id: I080f6e173fec40c27dd3ebbf252eaddf5a0e15ba
2023-12-22 09:13:10 +01:00
Harald Welte
0ecbf63a02 transport: Extend the documentation for each transport driver
This driver description we add to the code is automatically added to the
respective user manual sections.

Change-Id: I8807bfb11f43b167f1321d556e09ec5234fff629
2023-12-21 12:33:12 +01:00
Harald Welte
baec4e9c81 transport: Move printing of reader number/name to generic code
Let's avoid copy+pasting print statements everywhere.  The instances
do already have a __str__ method for the purpose of printing their name in a
generic way.

Change-Id: I663a9ea69bf7e7aaa6502896b6a71ef692f8d844
2023-12-21 12:33:12 +01:00
Harald Welte
ad002797e2 transport/pcsc: Allow opening PC/SC readers by a regex of their name
Opening PC/SC readers by index/number is very error-prone as the order
is never deterministic in any system with multiple (hot-plugged, USB)
readers.  Instead, let's offer the alternative of specifying a regular
expression to match the reader name (similar to remsim-bankd).

Change-Id: I983f19c6741904c1adf27749c9801b44a03a5d78
2023-12-21 12:33:12 +01:00
Harald Welte
0f177c1d29 transport: Pass argparse.Namespace directly into transport classes
It's odd that the individual transport driver specifies their argparse
options but then the core transport part evaluates them individually.
This means we cannot add new options within a transport.

Let's pass the Namespace instance into the constructor of the
specific transport to improve this.

Change-Id: Ib977007dd605ec9a9c09a3d143d2c2308991a12c
2023-12-21 11:31:57 +00:00
Harald Welte
c108595041 move {enc,dec}_addr_tlv functions from pySim.util to pySim.legacy.util
In the previous commit we've stopped using those functions from modern
pySim-shell code.  Hence, the only remaining user is the legacy tools,
so we can move the code to the legacy module.

Change-Id: I6f18ccb36fc33bc204c01f9ece135676510e67ec
2023-12-17 10:46:31 +00:00
Harald Welte
301d6ed14a isim: Replace legacy imperative address TLV encoder/decoder with construct
We've recently introduced IPv{4,6}Adapter construct classes and can
switch to this instead of using the old imperative encoder/decoder
functions {enc,dec}_addr_tlv().

Aside from code cleanup, this also means we now support the IPv6 address
type in EF.PCSCF.

Change-Id: I4d01ccfe473a8a80fbee33fdcbd8a19b39da85ac
2023-12-17 10:46:31 +00:00
Harald Welte
b3c46135bb bertlv_parse_len: Fix input data is smaller than num length octets
This can happen if there's a file with invalid encoding on the card,
such as a tag followed by all-ff.  Let's gracefully ignore it and
return zero bytes as response.

Change-Id: Ic44557368a6034dbf4bb021ab23a57927c22def0
2023-12-17 10:46:31 +00:00
Harald Welte
6e9ae8a584 usim: Properly decode/encode IPv4 + IPv6 addresses
use normal textual representation for IPv4 and IPv6 addresses

Change-Id: I2c6c377f4502af37639e555826c85d5dcf602f9b
2023-12-17 10:46:31 +00:00
Harald Welte
478b5fe8e3 usim: ePDGId + ePDGSelection: Fix encoder/decoder + add test cases
Change-Id: Idca19b6fdabae6cc708e92c7714fa0903ea5a1ee
2023-12-17 10:46:31 +00:00
Harald Welte
cdfe1c24af usim: Add EF.ePDGSelection + EF.ePDGSelectionEm support
Change-Id: I760a394ae1eac5f1175dc9b86c11b4a60671582e
2023-12-17 10:46:31 +00:00
Harald Welte
5277b5cf2c USIM: add support for EG.ePDGIdEm (Emergency ePDG)
Change-Id: I71cb7a4b9323f57b96db2d9f12f1567eda63f742
2023-12-17 10:46:31 +00:00
Philipp Maier
a5707c7dfb filesystem: fix typo
Change-Id: I721875d302ab69340d56b33102297b56c070465f
2023-12-13 12:47:36 +01:00
Philipp Maier
82cc7cc11a runtime: refactor file selection methods select and select_file
The implementation of the methods select and select_file of class
RuntimeLchan is a bit complex. We access the card directly in several
places which makes it difficult to track the state changes. We should
clean this up so that we call self.rs.card.select_adf_by_aid/
self.scc.select_file from a single place only.

This means that the method select uses the method select_file. This
results in a much cleaner implementation. We also should take care
that the important states that we track (selected_file, selected_adf,
etc.) are updated by a single private method. Since the update always
must happen after a select _select_post is a good place to do this.

Related: OS#5418
Change-Id: I9ae213f3b078983f3e6d4c11db38fdbe504c84f2
2023-12-13 12:47:36 +01:00
Philipp Maier
14bf003dad filesystem: use sort path when selecting an application
The method build_select_path_to uses the internal file system tree model
to find the path to a given file. This works the same for applications
(ADF) as it works for normal files (EF/DF). However, an application can
be selected anytime from any location in the filesystem tree. There is
no need to select a specific path leading to that application first.
This means that if there is an ADF somewhere in the resulting
inter_path, we may clip everything before that ADF.

Related: OS#5418
Change-Id: I838a99bb47afc73b4274baecb04fff31abf7b2e2
2023-12-13 12:45:46 +01:00
Philipp Maier
174fd32f17 runtime: explain how file probing works
We use a trick to probe a file (that does not exist in the local file
model yet). Let's explain further how that works, in particular why we
do not have to upate any state if probing fails.

Change-Id: I2a8af73654251d105af8de1c17da53dfa10dc669
Related: OS#5418
2023-12-13 09:02:30 +00:00
Harald Welte
b582c3c7ea euicc: Fix TLV IE definitions for SetNickname{Req,Resp}
The metaclass uese the 'nested' attribute, while the existing code
accidentially used the 'children' attribute.  The latter is used
by instances for actual child classes, while the Class/nested
attribute is for the list of classes whose instancse could be potential
children.

Change-Id: I968bd84d074dcdcec37d99be5d3d4edac9c35a0c
2023-12-07 23:29:11 +01:00
Harald Welte
c20d442695 euicc: Fix encoding of Lc value in STORE DATA
The length value "of course" is a hex value, don't use %02u but %02x

This fixes any eUICC command with a Lc > 10 bytes.

Change-Id: I1e1efbfb9916fc43699602cc889cf4b3d42736f2
2023-12-07 22:46:40 +01:00
Harald Welte
2b6deddcdc euicc: the ICCID TLV object uses bcd-swapped-nibble encoding
Change-Id: I050f9e0fb128f3e1d472e2330b136a753794a5a1
2023-12-07 14:21:43 +01:00
Philipp Maier
5482737f31 pySim-shell: don't get trapped in applications without file system
When we traverse the file system, we may also end up selecting
applications (ADF), which do not support an USIM/ISIM like file system.
This will leave us without the ability to select the MF (or any other
file) again. The only way out is to select the ISIM or USIM application
again to get the access to the file system again.

Change-Id: Ia2fdd65f430c07acb1afdaf265d24c6928b654e0
Related: OS#5418
2023-12-07 13:21:07 +00:00