pyOpenSSL >= 25.0.0 makes a Context immutable once it has been used and
raises. Downgrading pyOpenSSL is not a fix either: < 25 does not import
against recent cryptography.
This server only speaks HTTP/1.1 anway so ALPN negotiation is unused
and this can be hotpatched for affected versions.
Change-Id: I5d53216f24a20625d12f0757015c19fe341303b9
GP Amendment B "Remote Application Management over HTTP" v1.2 and
ETSI TS 102 226 RFM/RAM server.
APDU command/response bodies use the Expanded Remote Application data
format.
--mode {ram,rfm} selects what we want to do, make sure to
target the right thing in the right mode, SD for RAM, AID for RFM.
This is hand-rolled on purpose:
Klein/Twisted do not help us here, have annoying headers and settings,
and don't work well at all if you want to import the file for tests and
so on. I have wasted quite some time trying to make this prettier by
re-using existing http handler parts because I thought well we import
all kinds of packages already it can't be that hard, unfortunately it is.
Change-Id: I4f243aeca65d0f33ff34e6932e3f4e156446af8b
Adds TS 102 226 tables 5.2a/5.10a indefinite length coding,
recommended by TS 102 226 5.2.1 for RAM/RFM over HTTPS.
Noteworthy notable things to note:
- encode_expanded_cmd()
indefinite version -> inner C-APDU TLVs are still definite
- decode_expanded_resp()s returned container does not care, but
Indef has no 'number of executed commands' TLV -> number_of_commands
is the R-APDU count.
Tests are being fed with some known-good values from my sja5 sessions.
Change-Id: I4e023112e98729489ed443eec3ed5ab45c773b17
The BIP relay ( the "handset" side for SCP81) never worked: the
connect callback in handle_OpenChannel was "never called" as the fixme
says, everything else was missing.
Fixme cause: card APDU I/O is driven synchronously, proactive command loop
lives in a blocking while loop (pySim.transport.LinkBase.send_apdu_checksw)
that runs on the Twisted reactor thread. A Twisted TCP4ClientEndpoint +
connectProtocol only completes when the reactor does reactor things,
but the reactor thread is stuck in that loop for the whole proactive
session...
Fixme fix: don't fight the reactor, just drive the relay channel with a plain
old blocking socket, which fits the synchronous execution model.
Channel numbers now come from the command Device identities (channel_N ->
low nibble) instead of the hard coded chan_nr == 1.
Additionally fix two bugs found on the path to scp81 glory:
- TERMINAL RESPONSE device identities are forced to terminal->UICC per
TS 102 223 6.8.2 (prepare_response() inverts the command identities,
which for a channel-addressed BIP command yields channel_N->UICC).
- Error responses now build a valid AddlInfoBip cause, prepare_response()
hard coded empty "additional information" cannot be encoded for a
BIP error.
And some tests based on real card interactions.
Change-Id: If96c768f2e35c20ea3753e601059410121517b60
A large OTA response (for example GP GET STATUS app registry) is split by the card
into multiple SMS, each carries a TS 23.040 9.2.3.24 'concatenated short messages'
IE in its UDH. Nothing recombines them, so smpp-ota-tool currently only sees the
first incomplete part.
Add ConcatenatedSmsReassembler that accepts TP-User-Data, buffers parts
by reference number, and returns the reassembled TP-User-Data in the canonical
single-part form. Non-concatenated SMS pass through unchanged.
Both the 8-bit+16-bit references are supported.
A reserved value in the concatenation IE is not an error, these messages
are handed back as is rather than rejected, so the caller can deal with that.
Reassembly leads to a result that looks like a fat single part message the
card could have produced given infinite sms sizes, so the existing decode_resp
path is unaffected.
Feeding those parts to the ota tool needs two more fixes because the card returns
the application response as several SMS via proactive SEND SHORT MESSAGE while
the ENVELOPE SMS-PP DOWNLOAD itself contains the POR without a app R-APDU.
smpplib poll() drains everything, so message_received_handler runs on each:
- a later status-only response must not overwrite an application response
already captured, or transceive_apdu finds no last_response_data and raises
- a response that cannot be decoded is be logged and skipped rather raised
out of client.poll() which kills the tool.
Plus tests from a sja5 session.
Incomplete sets are capped (oldest evicted) so they cannot pile up.
Change-Id: I8c81097e607e0d055c4f031bbcc8a74d5c24a0e7
A multi part OTA response (full GP GET STATUS registry or some other fat
response that exceeds one SMS) is delivered as several SMS via proactive
SEND SHORT MESSAGE. The card only gives us another part if it receives a
TERMINAL RESPONSE for the previous one.
Currently smpp2sim Proact.handle_SendShortMessage relays the SMS but
returns None, so the transport falls back to prepare_response(pcmd) with
the ProactiveCommand collection (empty .children) and crashes with
'not enough values to unpack (expected 1, got 0)', dropps the SMPP link,
and never fetches the remaining parts.
Fix: make handle_SendShortMessage return a successful TERMINAL RESPONSE
built from the decoded command so the handshake proceeds.
prepare_response() is extended to handle collection via .decoded
and raises a useful error.
The send_apdu_checksw general_result='FIXME' path is now avoided for
SendShortMessage but remains a problem for other handlers that return
None.
Change-Id: Ib96ce81c4ff093b8a6fc715f79617e95a2dd8433
since the very early days, pySim-shell.py has a commandline option
where the user may supply an ADM pin as commandline parameter.
(-a / --pin-adm and -A --pin-adm-hex) This was introduced to simplify
the usage of pySim-shell.py with shellscripts.
Unfortunately the code that handles those commandline options
duplicates the code of the verify_adm commmand. Fortunately it is
very easy to call pySim-shell commands directly using the CMD2
onecmd_plus_hooks method, so we can just call the verify_adm and
replace the duplicated code with that.
So far we are only able to use ADM1 pins from the commandline, since
we now practically use the verify_adm commnad, we can add another
parameter to allow the verification of ADM as well.
Related: SYS#8239
Change-Id: I7164fad757048774aa7186a84041febde75c351c
Add --format expanded flag to send C-APDUs in the expanded remote
application data format.
Each --apdu becomes its own C-APDU TLV, and the tool
logs the full per-command R-APDU list decoded from the Response Scripting
template and warns if the card reports a truncated response.
Default stays 'compact', unchanged.
Change-Id: Idad90756f85bb7e54ef720f6067bb5d6dcff0c42
TS 102 226 section 5.2 "Command and Response Scripting templates"
The advantage over compact RFM/RAM commands is one C-APDU TLV per command
and one R-APDU TLV per result, so multiple commands return the response
of each one rather than only that of the last.
encode_expanded_cmd() builds the Command Scripting template
decode_expanded_resp() decodes the Response Scripting template into a
Container.
The 'truncated' key must be checked!
OtaDialect.encode_cmd()/decode_resp() now has a remote_format param for
compact and expanded formats, default stays compact,so existing code is
unaffected.
Change-Id: Idec00d16fd1a7d4a7129b2a3b6f0ef37dabcecb7
Move the code from pySim-smpp2sim.py to its own file, so it can be properly
extended.
The current file parses argv, opens a reader and starts the Twisted
reactor at import time, so nothing else can import it.
No functional changes yet, improvements follow in later commits.
Change-Id: Ifd8a15684939977d29ea83a6b669daee14484e88
PySimLogger.setup() installs a process-global print callback.
PySimLogger_Test sets one, a helper that asserts the message equals a global
expected_message, and never removes it, so from the moment test_log runs,
every PySimLogger message emitted anywhere in the process is checked against
whatever string that global happens to hold.
Fortunately unittest discovery runs modules in sorted order, and today
the PySimLogger users that log during tests all sort before test_log, so
this only breaks as soon as I try to add tests, just like anything else
breaks as soon as I try to use it.
Change-Id: I481e2c443fe0f412380b0f1acf6da5971ffca147
SCP.overhead was so far set at construction time (SCP02: 8, SCP03:
s_mode), so the C-MAC length only.
Unfortunately sec lvl >= 3 pads the data field to the cipher block size
before encryption, so the real worst-case overhead is larger,
scc.max_cmd_len (255 - overhead) was too big, and ADF_SD.load()
used a hardcoded chunk_len=240.
Real world issue with a 286 byte CAP + SCP02 + sec lvl 3:
- 240-byte LOAD block is padded to 248,
- encrypted
- gets 8 byte C-MAC appended
-> Lc = 256
That dies with a weird "ValueError: bytes must be in range(0, 256)".
The only "fix" for that was to downgrade the seclevel.
STORE DATA has the same overflow with large max_cmd_len
(247 + padding + MAC = 256 as well).
Therefore the overhead must be properly calculated from the sec level.
While at it adjust the error in case I missed something to get a more
useful ValueError.
Change-Id: Ic208f3959a38896f64fb6ccefb24cc360a3ac3a2
how encrypt_key() pads a kcv:
len(key) % blocksize bytes
what it should do to actually do it right:
blocksize - len(key) % blocksize
so the plaintext handed to the cipher was only block aligned by luck as
long as the key length happened to be a multiple of half the block size.
And of course decrypt_key() did not invert encrypt_key() at all,
the clear text length of GP CardSpec v2.3 Table 11-70 precedes the
ENCRYPTED kcv, but it was parsed out of the DECRYPTED data, and the
length byte itself was fed to the cipher along with the cryptogram.
Fix this up with a helper and tests so it is actually usable.
Change-Id: I02b4f2ed948c31e1741e40f0226fb49757fa2570
while len(foo):
throws an exception when foo == None.
Instead doing
while foo:
fixes a problem when reading in empty SUCI calc info data, e.g. from
TS48v7.0_SAIP2.3_BERTLV_SUCI_NoRAMRFM.der.
Change-Id: Ia4e2356d0241d7a6ca399ba7e8be7f27ec836104
Jenkins: skip-card-test
The tests take long and don't need to be this many.
A patch is coming up that adds another profile to test SUCI in GFM,
this patch makes some room for that.
Change-Id: Ib75b6919a3acfddd99bf9baa9b6847ef731b9e67
Jenkins: skip-card-test
Particular reason: when manipulating the 5G SUCI parameters, the
mandatory services get-identity, profile-a-x25519 and profile-b-p256 may
need to be reconfigured.
In general, it is a good idea to run these checks anyway.
Change-Id: I5e6eef0f1845a25cddb03af8d16c40e305bcdc1f
Jenkins: skip-card-test
Sadly, the EF.ARR interpretation, specifically that of the AM_DO
depends on whether the rule is for a DF or EF. As we don't know this,
allow the user to specify what kind of decode they would like, at least
in the file-specific read_arr_record + read_arr_records command.
Change-Id: I05e629e8b7dc705730d7039d2d0170ee24f8f844
When trying to use `edit_binary_decoded` with an empty file, pysim
runs into a len(None) exception, because hpkl.to_dict()['hnet_pubkey_list'] returns
None.
Can reproduced with a CCC Camp 2023 usim and editing the file.
a000ff..ff (len = 200)
Co-authored-by: Harald Welte <laforge@osmocom.org>
Change-Id: Ib8e322e65dd768bfd49e7a5620a2163f12a74ec7
The method do_aram_store_ref_ar_do and do_aram_delete_all, which
are part of the nested AddlShellCommands class, may be moved into
the parent class as a static method, just like the already existing
get_config method.
This makes the functionality re-usable to callers that do not use
the CMD2 API.
Change-Id: Icd1b08ec707dd939bc9e8524d7f9431aa4daae7c
Related: SYS#6959
Add a new ConfigurableParameter that represents the MNC length
(2 or 3 digits) in EF.AD (Administrative Data).
Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
Some Linux distributions (e.g. Arch Linux) already ship cmd2 3.x.x,
which removed the style()/Fg/Bg API in favor of stylize()/Color.
Add a version guard to select the right API at runtime.
Adjust the upper bound cap in requirements.txt and setup.py.
Change-Id: Ibf2ac7847933296fb06665c87f53ed6e1f315d27
Remove version guards for cmd2 < 2.0.0 and < 2.3.0, the Cmd2Compat
and Settable2Compat wrapper classes, and the old fg/bg color API -
none of these are needed since both requirements.txt and setup.py
already mandate cmd2 >= 2.6.2.
Change-Id: Ifd1c484ab66d74323d10e946347daa637cf6f5d8
As pointed out in the commit-log of Change-Id
I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6 we had different minimum
versions for cmd2 in requirements.txt vs setup.py. Let's align that.
Change-Id: I71cee0ec3ed2abec68ec567beaab13c868721dad
get_profiles_info only request for the default tag list, but
not all tags.
Add --all to the function to request for all known tags.
Change-Id: Ia6878519a480bd625bb1fa2567c1fd2e0e89b071
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.
For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.
Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.
Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test
apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field. After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.
Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
padding) so the re-encoded body carries 0xFF-padded alpha_id space
and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
alpha_id area is actually padded to that length
Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.
Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
secrets.SystemRandom is defined as the most secure random source
available on the given operating system.
Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test
Python's random module uses a PRNG (Mersenne Twister) which is
utterly insecure for key generation - it was so far only used for
testing. Replace it with random.SystemRandom(), which draws from
/dev/urandom and is suitable for generating cryptographic key material.
Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test
Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).
Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.
Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.
Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.
Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
The ClassVarMeta metaclass used to derive each ConfigurableParameter's
'name' attribute automatically from the Python class name (via
camel_to_snake()). Stop doing this, for three reasons:
1) Python class names follow constraints that do not fit the naming
commonly used in CSV files. For example, a name like
"5GS-SUCI-CalcInfo" starts with a digit and contains dashes,
neither of which is permissible in a class name.
2) Python class names live in their own namespace, distinct from the
one used to present eSIM parameters to end users. Deriving the UI
name from the class name couples these two namespaces together.
Taken together, (1) and (2) mean that automatic naming both imposes
class-name constraints on the user-visible names and merges the
internal Python namespace with the publicly shown one - a layer
violation from the perspective of UI design.
3) Overriding 'name' from __new__() makes manual naming impossible: a
subclass that sets 'name = "bar"' as a class attribute would still
end up with the value computed by the metaclass, which is
surprising and hard to track down:
class MySuper(metaclass=...): # __new__ sets name = 'foo'
...
class MySub(MySuper):
name = 'bar'
print(MySub().name) # 'foo', not 'bar' as one would expect
Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test
By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.
For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x04..0x0f in an audit.
This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.
Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test
Eric Wild