to_bytes() appended the BCD filler nibble, so the next call saw one
digit more and derived a wrong length. This affects any code that encodes
an address twice, even for printing/logging.
Man, I really miss my beloved c++ const function decoration...
Fix: keep the filler in a local.
Change-Id: I81691c5a1fc5072d6d20c52d22da1eb2e180d04a
Right now APDUs that never return/fail are omitted from the trace output.
Move the print, so it actually tells me what failed..
Change-Id: I828ea36b06da36b6bb0aa073d235d4f49cbb3161
get_status() has appended a hardcoded '5c054f9f70c5cc' to the command data
field. Of the data objects in GP CS v2.3.1 Table 11-35 only the AID
search tag 4F is mandatory, the tag list is optional and not supported
in v2.1.1, where Section 9.4.2.3 defines the data field as the search
qualifier. Cards implementing that revision can reject anything else
with 6A80 as per v2.1.1 Table 9-26.
A sysmocom SJA5 does that. Its data field must be one 4F
TLV, the value is free, but nothing may precede or follow it.
So every subset returned nothing at all...
There is no need to guess: v2.1.1/v2.3.1 Section 7.4.1.3 Card Recognition
Data is "shall be present" and contains the GP version on selected SD.
Query it once, and send the tag list to cards that announce v2.2 or later.
SJA5 reports 2.1.1, sysmoEUICC reports 2.2.
Two more problems with the old list:
- A tag list is an inclusion list, old list omits tag 84, so it
suppressed the Executable Module AIDs
- It asks for tag C5 for Executable Load Files, which "may" be answered
with an error status.
Fix this by constricting or omitting the tag list depending on reported
GP version.
Change-Id: I74cd2bd47617d616bede6453397f544cde5abcb7
TS 102 221 section 7.3.1.1.4 clause 4b lets the card answer a case #4
command TPDU with a 62xx/63xx warning, upon which the terminal sends a
dummy GET RESPONSE to obtain the 61xx that announces the response length.
__send_apdu_T0() applies that unconditionally, to the status word that
terminates a GET RESPONSE...
That is not "redundant", as per GPC v2.3.1 section 11.4.3.2 table 11-38
GP GET STATUS (80 F2) answers 6310 "more data available", meaning reissue
the command as get next occurrence(s) with P2 bit 1 set (11.4.2.2 table
11-34), so for example a paginated registry
listing exchange looks like this:
84f22000024f00 => 61e4 first match, e4 bytes waiting
84c00000e4 => <page 1> 6310 more matches pending
84c0000000 => 6982 <- unsolicited, card rejects it
The card rejects the unsolicited GET RESPONSE with 6982, our SCP02
session breaks, and the next command fail with 6985. The 6310 never
reaches ADF_SD.get_status() either, so the pagination loop exits after
page 1 and prints a sliently truncated listing. Observed with a SJA5.
Fix by turning clause 4b würgaround into what it should be: a one shot
reaction to the SW returned for the command TPDU.
While at it, stop ADF_SD.get_status() from silently returning a truncated
registry: it treated every status word other than 6310 as
"nothing more to report". 6A88 is now the explicit empty result and anything
else raises, so a partial listing can no longer silently pass.
Adds unit tests for all of that so we dont break basic T0 things.
Change-Id: I10f8afa8dd5623a49a6a0e7132607b3a1fad2d8c
Adds TS 102 226 tables 5.2a/5.10a indefinite length coding,
recommended by TS 102 226 5.2.1 for RAM/RFM over HTTPS.
Noteworthy notable things to note:
- encode_expanded_cmd()
indefinite version -> inner C-APDU TLVs are still definite
- decode_expanded_resp()s returned container does not care, but
Indef has no 'number of executed commands' TLV -> number_of_commands
is the R-APDU count.
Tests are being fed with some known-good values from my sja5 sessions.
Change-Id: I4e023112e98729489ed443eec3ed5ab45c773b17
The BIP relay ( the "handset" side for SCP81) never worked: the
connect callback in handle_OpenChannel was "never called" as the fixme
says, everything else was missing.
Fixme cause: card APDU I/O is driven synchronously, proactive command loop
lives in a blocking while loop (pySim.transport.LinkBase.send_apdu_checksw)
that runs on the Twisted reactor thread. A Twisted TCP4ClientEndpoint +
connectProtocol only completes when the reactor does reactor things,
but the reactor thread is stuck in that loop for the whole proactive
session...
Fixme fix: don't fight the reactor, just drive the relay channel with a plain
old blocking socket, which fits the synchronous execution model.
Channel numbers now come from the command Device identities (channel_N ->
low nibble) instead of the hard coded chan_nr == 1.
Additionally fix two bugs found on the path to scp81 glory:
- TERMINAL RESPONSE device identities are forced to terminal->UICC per
TS 102 223 6.8.2 (prepare_response() inverts the command identities,
which for a channel-addressed BIP command yields channel_N->UICC).
- Error responses now build a valid AddlInfoBip cause, prepare_response()
hard coded empty "additional information" cannot be encoded for a
BIP error.
And some tests based on real card interactions.
Change-Id: If96c768f2e35c20ea3753e601059410121517b60
A large OTA response (for example GP GET STATUS app registry) is split by the card
into multiple SMS, each carries a TS 23.040 9.2.3.24 'concatenated short messages'
IE in its UDH. Nothing recombines them, so smpp-ota-tool currently only sees the
first incomplete part.
Add ConcatenatedSmsReassembler that accepts TP-User-Data, buffers parts
by reference number, and returns the reassembled TP-User-Data in the canonical
single-part form. Non-concatenated SMS pass through unchanged.
Both the 8-bit+16-bit references are supported.
A reserved value in the concatenation IE is not an error, these messages
are handed back as is rather than rejected, so the caller can deal with that.
Reassembly leads to a result that looks like a fat single part message the
card could have produced given infinite sms sizes, so the existing decode_resp
path is unaffected.
Feeding those parts to the ota tool needs two more fixes because the card returns
the application response as several SMS via proactive SEND SHORT MESSAGE while
the ENVELOPE SMS-PP DOWNLOAD itself contains the POR without a app R-APDU.
smpplib poll() drains everything, so message_received_handler runs on each:
- a later status-only response must not overwrite an application response
already captured, or transceive_apdu finds no last_response_data and raises
- a response that cannot be decoded is be logged and skipped rather raised
out of client.poll() which kills the tool.
Plus tests from a sja5 session.
Incomplete sets are capped (oldest evicted) so they cannot pile up.
Change-Id: I8c81097e607e0d055c4f031bbcc8a74d5c24a0e7
A multi part OTA response (full GP GET STATUS registry or some other fat
response that exceeds one SMS) is delivered as several SMS via proactive
SEND SHORT MESSAGE. The card only gives us another part if it receives a
TERMINAL RESPONSE for the previous one.
Currently smpp2sim Proact.handle_SendShortMessage relays the SMS but
returns None, so the transport falls back to prepare_response(pcmd) with
the ProactiveCommand collection (empty .children) and crashes with
'not enough values to unpack (expected 1, got 0)', dropps the SMPP link,
and never fetches the remaining parts.
Fix: make handle_SendShortMessage return a successful TERMINAL RESPONSE
built from the decoded command so the handshake proceeds.
prepare_response() is extended to handle collection via .decoded
and raises a useful error.
The send_apdu_checksw general_result='FIXME' path is now avoided for
SendShortMessage but remains a problem for other handlers that return
None.
Change-Id: Ib96ce81c4ff093b8a6fc715f79617e95a2dd8433
TS 102 226 section 5.2 "Command and Response Scripting templates"
The advantage over compact RFM/RAM commands is one C-APDU TLV per command
and one R-APDU TLV per result, so multiple commands return the response
of each one rather than only that of the last.
encode_expanded_cmd() builds the Command Scripting template
decode_expanded_resp() decodes the Response Scripting template into a
Container.
The 'truncated' key must be checked!
OtaDialect.encode_cmd()/decode_resp() now has a remote_format param for
compact and expanded formats, default stays compact,so existing code is
unaffected.
Change-Id: Idec00d16fd1a7d4a7129b2a3b6f0ef37dabcecb7
Move the code from pySim-smpp2sim.py to its own file, so it can be properly
extended.
The current file parses argv, opens a reader and starts the Twisted
reactor at import time, so nothing else can import it.
No functional changes yet, improvements follow in later commits.
Change-Id: Ifd8a15684939977d29ea83a6b669daee14484e88
SCP.overhead was so far set at construction time (SCP02: 8, SCP03:
s_mode), so the C-MAC length only.
Unfortunately sec lvl >= 3 pads the data field to the cipher block size
before encryption, so the real worst-case overhead is larger,
scc.max_cmd_len (255 - overhead) was too big, and ADF_SD.load()
used a hardcoded chunk_len=240.
Real world issue with a 286 byte CAP + SCP02 + sec lvl 3:
- 240-byte LOAD block is padded to 248,
- encrypted
- gets 8 byte C-MAC appended
-> Lc = 256
That dies with a weird "ValueError: bytes must be in range(0, 256)".
The only "fix" for that was to downgrade the seclevel.
STORE DATA has the same overflow with large max_cmd_len
(247 + padding + MAC = 256 as well).
Therefore the overhead must be properly calculated from the sec level.
While at it adjust the error in case I missed something to get a more
useful ValueError.
Change-Id: Ic208f3959a38896f64fb6ccefb24cc360a3ac3a2
how encrypt_key() pads a kcv:
len(key) % blocksize bytes
what it should do to actually do it right:
blocksize - len(key) % blocksize
so the plaintext handed to the cipher was only block aligned by luck as
long as the key length happened to be a multiple of half the block size.
And of course decrypt_key() did not invert encrypt_key() at all,
the clear text length of GP CardSpec v2.3 Table 11-70 precedes the
ENCRYPTED kcv, but it was parsed out of the DECRYPTED data, and the
length byte itself was fed to the cipher along with the cryptogram.
Fix this up with a helper and tests so it is actually usable.
Change-Id: I02b4f2ed948c31e1741e40f0226fb49757fa2570
while len(foo):
throws an exception when foo == None.
Instead doing
while foo:
fixes a problem when reading in empty SUCI calc info data, e.g. from
TS48v7.0_SAIP2.3_BERTLV_SUCI_NoRAMRFM.der.
Change-Id: Ia4e2356d0241d7a6ca399ba7e8be7f27ec836104
Jenkins: skip-card-test
Particular reason: when manipulating the 5G SUCI parameters, the
mandatory services get-identity, profile-a-x25519 and profile-b-p256 may
need to be reconfigured.
In general, it is a good idea to run these checks anyway.
Change-Id: I5e6eef0f1845a25cddb03af8d16c40e305bcdc1f
Jenkins: skip-card-test
Sadly, the EF.ARR interpretation, specifically that of the AM_DO
depends on whether the rule is for a DF or EF. As we don't know this,
allow the user to specify what kind of decode they would like, at least
in the file-specific read_arr_record + read_arr_records command.
Change-Id: I05e629e8b7dc705730d7039d2d0170ee24f8f844
When trying to use `edit_binary_decoded` with an empty file, pysim
runs into a len(None) exception, because hpkl.to_dict()['hnet_pubkey_list'] returns
None.
Can reproduced with a CCC Camp 2023 usim and editing the file.
a000ff..ff (len = 200)
Co-authored-by: Harald Welte <laforge@osmocom.org>
Change-Id: Ib8e322e65dd768bfd49e7a5620a2163f12a74ec7
The method do_aram_store_ref_ar_do and do_aram_delete_all, which
are part of the nested AddlShellCommands class, may be moved into
the parent class as a static method, just like the already existing
get_config method.
This makes the functionality re-usable to callers that do not use
the CMD2 API.
Change-Id: Icd1b08ec707dd939bc9e8524d7f9431aa4daae7c
Related: SYS#6959
Add a new ConfigurableParameter that represents the MNC length
(2 or 3 digits) in EF.AD (Administrative Data).
Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
Some Linux distributions (e.g. Arch Linux) already ship cmd2 3.x.x,
which removed the style()/Fg/Bg API in favor of stylize()/Color.
Add a version guard to select the right API at runtime.
Adjust the upper bound cap in requirements.txt and setup.py.
Change-Id: Ibf2ac7847933296fb06665c87f53ed6e1f315d27
get_profiles_info only request for the default tag list, but
not all tags.
Add --all to the function to request for all known tags.
Change-Id: Ia6878519a480bd625bb1fa2567c1fd2e0e89b071
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.
For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.
Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.
Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test
apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field. After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.
Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
padding) so the re-encoded body carries 0xFF-padded alpha_id space
and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
alpha_id area is actually padded to that length
Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.
Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.
Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test
Python's random module uses a PRNG (Mersenne Twister) which is
utterly insecure for key generation - it was so far only used for
testing. Replace it with random.SystemRandom(), which draws from
/dev/urandom and is suitable for generating cryptographic key material.
Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test
Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).
Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.
Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.
Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.
Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
The ClassVarMeta metaclass used to derive each ConfigurableParameter's
'name' attribute automatically from the Python class name (via
camel_to_snake()). Stop doing this, for three reasons:
1) Python class names follow constraints that do not fit the naming
commonly used in CSV files. For example, a name like
"5GS-SUCI-CalcInfo" starts with a digit and contains dashes,
neither of which is permissible in a class name.
2) Python class names live in their own namespace, distinct from the
one used to present eSIM parameters to end users. Deriving the UI
name from the class name couples these two namespaces together.
Taken together, (1) and (2) mean that automatic naming both imposes
class-name constraints on the user-visible names and merges the
internal Python namespace with the publicly shown one - a layer
violation from the perspective of UI design.
3) Overriding 'name' from __new__() makes manual naming impossible: a
subclass that sets 'name = "bar"' as a class attribute would still
end up with the value computed by the metaclass, which is
surprising and hard to track down:
class MySuper(metaclass=...): # __new__ sets name = 'foo'
...
class MySub(MySuper):
name = 'bar'
print(MySub().name) # 'foo', not 'bar' as one would expect
Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.
For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x04..0x0f in an audit.
This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.
Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test
When the access conditions are extracted from resp_bin, the wrong length
is used and only 2 bytes instead of 3 are extracted.
3GPP TS 51.011, section 9.2.1, table below "Response parameters/data
in case of an EF", clearly states that the length should be 3 bytes
(position 9-11)
Related: OS#7018
Change-Id: I410fb58c395beafba8de6d5ab4e71452f424cdf2
The comment reads like that we were applying TS 102.221 here, but we only
mean our internal decoding format. The spec that actually matters here is
TS 51.011. Let's rephrase the comment so that this becomes more clear.
Related: OS#7018
Change-Id: Ie0184eea25f4d9f4baf9ab137c53a926edba2bf8
The commandline options --nfc-always and --nfc-never appear in the
opposite order when compared to --apdu-never and --apdu-always.
Let's swap the options to make the helpscreen and the code more
consistent.
Change-Id: I7289c3628b1b8dd3eec2f1c8f2132e3015422960
Related: SYS#6959
The module scp.py predates the existence of the pySimLogger and still
uses an individually created logger. Let's migrate to pySimLogger to
avoid unexpected effects and to be uniform with the other modules.
Related: SYS#6959
Change-Id: I5db7180f93f116dd2d99c33da264f74ea16a1a37
Eric Wild