Commit Graph

276 Commits

Author SHA1 Message Date
Philipp Maier
b46cb3ffa2 utils: fix encoding of EF.MSISDN
The encoding of EF.MSISDN is a bit unstrutured. The encoder function
does not return a valid result since it lacks the parameters
Capability/Configuration2 Record Identifier and Extension5 Record
Identifier, which are mandatory but can be set to 0xFF. Also the
encoder gets its input from pySim-shell, so it should have some
more input validation, especially when the user encodes an empty
string. The encoder and decoder function also do not have unit-tests.

Since the encoder now adds the missing two bytes by isself this does
not have to be done manually anymore, so cards.py needs to be
re-aligned.

For pySim-shell.py the encoder is used from ts_51_011.py. Unfortunately
it is used wrongly there. The optional Alpha Identifier is required
here as well.

Related: OS#4963
Change-Id: Iee5369b3e3ba7fa1155facc8fa824bc60e33b55b
2021-04-23 15:52:10 +02:00
Harald Welte
977035c429 filsystem.py: Add more information to exceptions
Change-Id: Ia9449ddfaaf5f49e0a65aeeea9435141fd55fe65
2021-04-22 09:11:55 +02:00
Harald Welte
28c2431f9c Move reader related argument parser to transport module
Ideally that shared definition would be used by all programs,
rather than copy+pasting it.  Unfortunately pySim-{read,prog}
are still using optparse and first need to be converted to
argparse.

Change-Id: If77f53850e1ca65f42cf1dca3e0f460dac1b0d1a
2021-04-22 09:11:23 +02:00
Philipp Maier
fe1fb037b5 filesystem: fix wrong helpstring for update_record_decoded
the helpstring of update_record_decoded mentions hex bytes for the data
parameter, but it should be mentioned as abstract json data like in
update_binary_decoded

Change-Id: Ibae2ab49054ac5dd6fcccddd28c98d886403dac9
Related: OS#4963
2021-04-21 15:13:52 +02:00
Philipp Maier
0adabf6161 filesystem: fix wrong comment
The property rec_len is not a tuple, it is a set.

Related: OS#4963
Change-Id: I366772c62d0bb5a6400ce5b431eb94ac9248dccc
2021-04-21 15:13:52 +02:00
Philipp Maier
80ce71f58c pySim-shell: separate export summary with a headline
the export summary is printed after the log entry for the last file
without separation. This is confusing because it looks like if the
summary would refer to the last file only. Lets add a headline to make
clear that the last few lines are the "Export summary"

Change-Id: I90771e525b2b114bdb41a8e90d298ca991c09c3d
Related: OS#4963
2021-04-21 15:11:34 +02:00
Robert Falkenberg
9d16fbca4a Use construct for EF_AD in pySim-{shell, prog, read}.py, cards.py
Also serves as example for RFU (reserved for future use) fields
which should not always be reset to zero in case they have been
set on the uSIM for some reason.
See pySim/ts_51_011.py, class EF_AD.

* Add definitions for RFU {Flag, Bits, Byte, Bytes}
* Use IntEnum for OP_MODE (convenient auto completion)
* Remove obsolete definitions and imports
* Update test results for all SIMs (opmode strings are shortened)

Change-Id: I65e0a426f80a619fec38856a30e590f0e726b554
2021-04-13 11:27:37 +00:00
Harald Welte
c9cdce3e02 fix various typos all over the code
Change-Id: Ic8392a951bf94f67b51e35bed95d0e856f7a9250
2021-04-11 12:20:29 +02:00
Harald Welte
790b2709bd lots of file definitions for classic SIM and USIM
Change-Id: I91475df4a5eaca423473aaebba8c69c54c9a0c1a
2021-04-11 12:20:29 +02:00
Harald Welte
9853e247a2 ts_31_102.py: switch EF.Keys from imperative to declarative codec
Change-Id: I215a61c1e44ee2c5df70a8ca96344b4dd4d93c00
2021-04-11 12:20:29 +02:00
Harald Welte
2db5cfb525 filesystem: Introduce 'construct' support
This allows for pure declarative encoders/decoders for linear fixed
and transparent EFs.

Change-Id: Id0765ec3ddb8c044838ce47d0ff8740720a32b15
2021-04-11 12:20:29 +02:00
Harald Welte
703f933b40 pySim-shell: Add open_channel + close_channel commands
Change-Id: I53d9d7f7720eb5f10956bff74ea7ba9fd3b3bd19
2021-04-11 12:20:29 +02:00
Harald Welte
a463161ae2 pySim-shell: Adds support for DEACTIVATE FILE + ACTIVATE FILE
Change-Id: I22207dde20f991b0a22dea8f5dd695a0ec99da33
2021-04-11 12:20:29 +02:00
Harald Welte
15fae98d2e pySim-shell: Authenticate (3g) support
This adds support for AUTHENTICATE to the USIM and ISIM application,
based on the newly-introduced 'construct' encoder/decoder support.

Change-Id: Id5697463e29c3dceff98bcf80f5400f7f2bcaa6c
2021-04-11 12:20:29 +02:00
Harald Welte
e0f9ef1606 integrate 'construct' python library
'construct' is a declarative symmetric encoder/decoder for user
specified binary formats.  It should come in extremely handy in
tools like pySim.

We start the integration by adding transport methods for transceiving
APDUs with built-in encoding of the command data and decoding of the
response data.

Change-Id: Ibf457aa8b9480a8db5979defcfafd67674303f6c
2021-04-11 12:20:29 +02:00
Robert Falkenberg
d0505bdb55 WIP: Add option to set UE operation mode in EF_AD (Administrative Data)
Use ``--opmode=OPMODE`` in cmdline mode or column ``OPMODE`` in csv mode
to specify OPMODE as listed below.

Details:
The ``EF_AD`` field contains administrative data (AD).
It consists of four bytes ``B1``, ``B2``, ``B3``, ``B4``,
and optionally further bytes for future use.

Previous implementation only sets the MNC field appropriately
(located in `B4`) and sets all other bits/bytes to 0.

However, `B1` also defines the *UE operation mode* (see below).
For type approval operations, such as testing with a test uSIM,
this value could be set to `0x80` rather than `0x00`(= normal operation).
This may unlock some UE capabilities that are restricted in
normal operation mode.

Excerpt from [ETSI TS 131 102, 4.2.18](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):

```
B1 - UE operation mode:
	Coding:
	Initial value
	- '00' normal operation.
	- '80' type approval operations.
	- '01' normal operation + specific facilities.
	- '81' type approval operations + specific facilities.
	- '02' maintenance (off line).
	- '04' cell test operation.

B2 - Additional information:
	Coding:
	Reserved for future use

B3 - Additional information:
	Coding:
	- B3.b1: OFM setting (Ciphering Indicator)
	- B3.others: Reserved for future use

B4 - Length of MNC in the IMSI:
	Coding:
	- B4.b4..B4.b1: length:  '0010' (= 2) or '0011' (=3)
	- B4.others: Reserved for future use
```

**Legend:** Byte X, bit Y: BX.bY

Further reading: https://nickvsnetworking.com/usim-basics/

Change-Id: Ie9040c6b127c268878a0845ed73d0918ec6bbb08
2021-04-11 09:35:36 +00:00
Harald Welte
5e749a79ac extend JSONEncoder to serialze 'bytes' style objects as hex strings
This means we can skip a lot of code that manually converts from
bytes to hex before JSON serialization.

Change-Id: I9c9eff0556d9d196e64553b5276e162f69d0c18f
2021-04-10 18:41:15 +02:00
Harald Welte
7829d8a357 shell: Add 'apdu_trace' settable parameter for hex-dumping APDUs
Change-Id: I0c957c0b86473413f31e4bd8bc4e633fc1470222
2021-04-10 18:41:15 +02:00
Harald Welte
eb05b2f60e transport: Pass arbitrary kwargs to base-class constructor
Change-Id: I3cd5ba87cf53409ea97196d5789ed28eef072c68
2021-04-10 18:41:15 +02:00
Harald Welte
c34f9405f1 transport: Make all calls go through base class send_apdu_raw()
This allows us to add APDU tracing at one central location in the code

Change-Id: Id0593a2e6d846cc3151443f1022ae7ee030e6673
2021-04-10 18:41:15 +02:00
Harald Welte
4145d3c4af shell: add edit_{record,binary}_decoded shell commands
This allows the user to edit the file/record contents in its
JSON representation inside the standard system text editor.

Change-Id: Icf6a6e8529e7664c5645519fb4bdd55b35f34664
2021-04-10 18:41:15 +02:00
Harald Welte
fe8a74490a pySim/filesystem: Fix more Python 3.5 compatibility issues
Change-Id: I277e86a0e6b16d53d2b5b48b74915ac9a48a4211
2021-04-10 11:52:24 +02:00
Vadim Yanitskiy
a0040792bf pySim/filesystem.py: fix compatibility with Python 3.5
Change-Id: Ia4021551bcd28e6020958964f6f4e9b6dc989d94
Related: OS#5111
2021-04-10 09:48:43 +00:00
Harald Welte
850b72ad58 shell: New 'read_records' and 'read_records_decoded' commands
Those commands can be used to read all the records available in the
selected file.

Change-Id: If457b4e02bde8aa6db8cc329411f94411c100bc9
2021-04-07 16:45:12 +00:00
Harald Welte
0d4e98a2ac pySim-shell: JSONpath support for updating files/records
Change-Id: Iad09b3d878b8b58ad34cb549c80f8a6eb3149faa
2021-04-07 16:45:12 +00:00
Robert Falkenberg
75487aed90 Use zero padding for EF['ACC'] field
The ``EF_ACC`` field defines the access control class (ACC)
for a subscriber.

Without this patch, the implementation adds padding 1 towards
the most significant bits if the input is shorter than 2 bytes.

However, it should be padded with 0, otherwise additional ACCs
are allocated to the subscriber. (Probably only a single bit
shall be set to 1)

Excerpt from [ETSI TS 131 102, 4.2.15](https://www.etsi.org/deliver/etsi_ts/131100_131199/131102/04.15.00_60/ts_131102v041500p.pdf):

```
EF_ACC: Two bytes: B1, B2

B1.b8...B1.b4: high priority users (class 15...11)
B1.b3: always 0
B1.b2...B1.b2 and B2.b7...B2.b0: normal priority users (class 9...0) - to be evenly distributed across subscribers
```

**Legend:** Byte X, bit Y: BX.bY


Change-Id: I1b8dc01a6c48adad1ed8158de59b12519ed688e9
2021-04-07 16:23:11 +00:00
Philipp Maier
c98ef8a79d ts_102_221.py: fix fixup_fcp_proprietary_tlv_map()
The function fixup_fcp_proprietary_tlv_map() addes propritary TLV
tags in the range of d0 to ff to the TLV map. However, the spec defines
this range as b7 and b8 of the first tag byte set to 1. This results
in a range from c0 to ff. See also ETSI TS 102 221, section 11.1.1.4.6.0

Change-Id: I8359527c9ff303b257b181b87dc440f27735ece9
Related: OS#4963
2021-04-07 16:21:52 +00:00
Robert Falkenberg
5459536c7b SysmoISIM-SJA2: Add option to set Service Provider Name (SPN)
Same implementation as for sysmoUSIM-SJS1

Change-Id: I3a9dd2fe85126584758ea4cfa127f9cd14ab0c7d
2021-04-07 12:18:29 +02:00
Harald Welte
d7a7e17a48 utils.py: Add missing dec_plmn function.
This function is being used e.g. for ADF.USIM/EF.FPLMN entries.

The EF_PLMNsel class also already uses a function by this name, we just
haven't had any actual implementation around.

Change-Id: Iacb45c90bb6491ebb89a477a85ef1f3129b38788
2021-04-06 22:53:18 +00:00
Harald Welte
f431189a9e pySim/filesystem: Remove left-over debug print statements
the print statements in read_binary_decoded and update_binary_decoded
should have been removed a long time ago.

Change-Id: I9ccc61c426a755fae9008d0717d579fa2da0ef7c
2021-04-07 00:33:02 +02:00
Harald Welte
1748b9372c pySim-shell: Add settable parameter on JSON pretty-printing
Change-Id: Ic095c96733de2b0f359bfe067cd719d38712faff
2021-04-06 21:16:16 +02:00
Harald Welte
c9baa4d915 ts_51_011: Full encoder/decoder for EF.AD
The EF.AD class only had a partial decoder and no encoder before this
patch.

You can now do things like

pySIM-shell (MF/ADF.USIM/EF.AD)> read_binary_decoded
{
    "ms_operation_mode": "normal_and_specific_facilities",
    "specific_facilities": {
        "ofm": false
    },
    "len_of_mnc_in_imsi": 2
}
pySIM-shell (MF/ADF.USIM/EF.AD)> update_binary_decoded '{"ms_operation_mode": "normal_and_specific_facilities", "specific_facilities": {"ofm": false}, "len_of_mnc_in_imsi": 3}'

not quite all that elegant yet, but working at all.

Change-Id: Id2cb66cb26b6bd08befe9f8468b0b0773da842b1
2021-04-06 20:20:57 +02:00
Harald Welte
bcad86c08c pySim-shell: Add '--oneline' to read_{binary,record}_decoded
This allows for single-line output, rather than the default JSON
pretty-printing

Change-Id: I08e0e7b6f0d796626f4d6c3e9a2622c1ee0c210a
2021-04-06 20:20:57 +02:00
Harald Welte
4f2c546613 transport: Pass status word interpreter to exception handler
Prior to this patch, any SwMatchError raised within the 'transport'
would not be interpreted.

EXCEPTION of type 'SwMatchError' occurred with message: 'SW match failed! Expected 9000 and got 6982.'

vs (now)

EXCEPTION of type 'SwMatchError' occurred with message: 'SW match failed! Expected 9000 and got 6982: Command not allowed - Security status not satisfied'

Change-Id: I08b7f2b6bd422f7f2f36094bc8a29b187ff882a6
2021-04-04 10:54:46 +02:00
Harald Welte
522555710b utils.py: Add more type annotations
Change-Id: I50a0a07132890af0817f4ff0ce9fec53b7512522
2021-04-04 10:53:36 +02:00
Harald Welte
6e0458dda6 Move init_reader() from utils.py to transport/__init__.py
This avoids a circular dependency when introducing type annotations.

Change-Id: I168597ac14497fb188a15cb632f32452128bc1c6
2021-04-04 10:53:36 +02:00
Harald Welte
9d0f1f0cd5 card_key_provider: Documentation with sphinx / autodoc
* move existing docs to sphinx / autodoc
* add more api documentation
* improve wording on some exception strings

Change-Id: Ia41e14d643d452d92fc8d3c2fb9c4ac9021402e9
2021-04-04 10:52:34 +02:00
Harald Welte
4442b3d1c0 rename card_data to card_key_provider.
"data" is an awfully generic term.  Anything stored on a card is data.

This specific code deals with resolving key/pin material from an
external source.

Change-Id: I4c8e1be3e766f7c0565c07b39d48abf8adc375af
2021-04-04 10:36:00 +02:00
Harald Welte
90d3b970af Add type annotations and more documentation to card_data.py
Change-Id: Ia09b3ecaa582d62a97c3adac2650686dc19d5ec1
2021-04-04 10:35:18 +02:00
Harald Welte
2d4a64b43d filesystem.py: Fix type annotation of read_binary_dec()
Change-Id: I781fc0c564a318a6f9b2ec8dccf9f8865bff0e48
2021-04-03 20:34:37 +00:00
Harald Welte
236a65f02f cosmetic: fix typo in comment
Change-Id: Iac8b310a470b3ad8dee5f61342fd5acedbbd6e5d
2021-04-03 20:33:31 +00:00
Harald Welte
86fbd39738 filesystem: Fix interpret_sw() fall-through
if an application-specific interpret_sw() fails, fall back to the
profile interpret_sw().

Change-Id: I326c6002c75e2f906848784b7831ea169134dbe4
2021-04-02 21:08:30 +00:00
Harald Welte
ec7d0daf6e 51.011: Define some more files within DF_TELECOM
Add some more minimal definitions for various DF_TELECOM files

Change-Id: I155729b4d62969cde2af00fc9fb9901299fe5c25
2021-04-02 22:05:28 +02:00
Harald Welte
89e5954773 fix various file definitions
As we can notice during 'export': Some files had been defined
as LinFixed but are Transparent - and vice versa.  Let's fix those
an bring our definitions in sync with the specs.

Change-Id: I365ece7b82a1c79b3af87a79ff964d7989362789
2021-04-02 22:05:28 +02:00
Harald Welte
5ce3524f5f Fix various mistakes around the CardADF <-> CardApplication dualism
When the CardFile hierarchy talks about 'application' it means CardADF.

When the RuntimeState and CardProfile talk about 'application' they mean
a CardApplication.

Let's clarify this in the file names, and make CardADF have an optional
reference to the CardApplication, so that application specific status
word interpretation really works.

Change-Id: Ibc80a41d79dca547f14d5d84f447742e6b46d7ca
2021-04-02 21:09:42 +02:00
Harald Welte
1e45657e0f filesystem: fix various issues found by mypy
Change-Id: Ib4de80451614712bdf5377a3a5b86156008e2c42
2021-04-02 21:09:40 +02:00
Harald Welte
5a4fd52986 filesystem: Avoid GPL header showing up in sphinx autodoc
Change-Id: I1d963ae3d1511ef40d1ebcb36b0f67c40cbd6309
2021-04-02 21:09:02 +02:00
Harald Welte
ee3501fc62 Add more documentation to the classes/methods
* add type annotations in-line with PEP484
* convert existing documentation to follow the
  "Google Python Style Guide" format understood by
  the sphinx.ext.napoleon' extension
* add much more documentation all over the code base

Change-Id: I6ac88e0662cf3c56ae32d86d50b18a8b4150571a
2021-04-02 21:08:35 +02:00
Harald Welte
082d4e0956 ts_31_102: Fix decode_select_response() for DF.5GS
In Change-Id I848a766e6d00be497c7db905475e0681cce197ac we added a CardDF
instance for DF_5GS.  That DF should not have provided a
decode_select_response() method, and instead fall back to that of the
base class, which calls the method of the parent directory (ADF_USIM).

The difference is illustrated below

pySIM-shell (MF/ADF.USIM/EF.IMSI)> select DF.5GS
"622e8202782183025fc0a509800171830400018d088a01058c056611111111c60f90017083010183018183010a83010b"

vs. (with this patch):

pySIM-shell (MF/ADF.USIM)> select DF.5GS
{
    "file_descriptor": {
        "shareable": true,
        "file_type": "df",
        "structure": "no_info_given"
    },
    "file_identifier": "5FC0",
    "proprietary_info": {
        "uicc_characteristics": "71",
        "available_memory": 101640
    },
    "life_cycle_status_int": "operational_activated",
    "security_attrib_compact": "6611111111",
    "pin_status_template_do": "90017083010183018183010A83010B"
}

Change-Id: I80612711bbc8c47285a828a0759b20beea6619f1
2021-04-02 21:01:39 +02:00
Philipp Maier
46f09af11d pySim-shell: complete CHV/PIN management tools
At the moment we only have a basic version of a verify_chv commnad, but
in order to handle any CHV/PIN related situation we also need commands
to enable, disable, change and unblock CHV.

- fix verify_chv commnad: more distinct parameter names, better help
  strings, correct pin code encoding and add external source lookup
- Add unblock_chv, change_chv, enable_chv and disable_chv commands
- add/fix related functions in commands.py

Change-Id: Ic89446e6bd2021095e579fb6b20458df48ba6413
Related: OS#4963
2021-04-02 16:32:53 +02:00