pySim-smpp2sim currently claims to support every feature flag that exists,
but this immediately dies here with a SJA5:
NotImplementedError: No handler method for ProvideLocalInformation(...)
Answer anything unhandled with "performed_successfully" and log. Never
"command_beyond_terminal_capability", tho, answering that for PROVIDE LOCAL
INFORMATION refuses to open the session at all.
Change-Id: I1e55d6f88c872a04b89a3946dd840d30b329621e
Put a helper for wrapping an SMS-DELIVER TPDU in the ENVELOPE
of TS 102 223 section 7.5.1 the assembly next to SMSPPDownload,
where the IEs already live, so it can be reused by other tooling,
for example for triggering scp81 sessions.
Change-Id: Id23227eac53d697f4f13a84e087c32bf1d60f474
to_bytes() appended the BCD filler nibble, so the next call saw one
digit more and derived a wrong length. This affects any code that encodes
an address twice, even for printing/logging.
Man, I really miss my beloved c++ const function decoration...
Fix: keep the filler in a local.
Change-Id: I81691c5a1fc5072d6d20c52d22da1eb2e180d04a
Error output is currently a garbled mess, delimiters are printed with
poutput() or print(), traceback with traceback.print_exc() which goes
to stderr.
Reading stdout alone omits the report but not -->8-- lines.
Reading both streams makes the traceback appear where the two buffers
just happen to interleave, stderr is usually unbuffered but stdout is
not, so I get the error trace, APDU trace, then empty -->8-- lines.
Fix this by aligning the printing funcs.
Change-Id: I2ed37dfe241d014e4a26894cddbc9b21be1fd0c2
Right now APDUs that never return/fail are omitted from the trace output.
Move the print, so it actually tells me what failed..
Change-Id: I828ea36b06da36b6bb0aa073d235d4f49cbb3161
pySim has two APDU tracers: the StdoutApduTracer that --apdu-trace
installs in the transport for every program using pySim.transport, and
pySim-shell Cmd2ApduTracer for "set apdu_trace true", which prints with
cmd2. The command line option exists because the setting
applies too late to see the pySim startup APDUs.
The guard in init_reader() exists but is useless:
if opts.apdu_trace and not 'apdu_tracer' in kwargs:
no caller supplies a tracer that way, every tool calls init_reader()
without one?.
The current help string does not say "trace or maybe not lol" so set
the setting from the transport rather than hardcoding False, so the
option implies "set apdu_trace true", and it finally works as advertised.
Change-Id: I83469e15f4cdd67418cd11df3869f2536105dbc8
get_status() has appended a hardcoded '5c054f9f70c5cc' to the command data
field. Of the data objects in GP CS v2.3.1 Table 11-35 only the AID
search tag 4F is mandatory, the tag list is optional and not supported
in v2.1.1, where Section 9.4.2.3 defines the data field as the search
qualifier. Cards implementing that revision can reject anything else
with 6A80 as per v2.1.1 Table 9-26.
A sysmocom SJA5 does that. Its data field must be one 4F
TLV, the value is free, but nothing may precede or follow it.
So every subset returned nothing at all...
There is no need to guess: v2.1.1/v2.3.1 Section 7.4.1.3 Card Recognition
Data is "shall be present" and contains the GP version on selected SD.
Query it once, and send the tag list to cards that announce v2.2 or later.
SJA5 reports 2.1.1, sysmoEUICC reports 2.2.
Two more problems with the old list:
- A tag list is an inclusion list, old list omits tag 84, so it
suppressed the Executable Module AIDs
- It asks for tag C5 for Executable Load Files, which "may" be answered
with an error status.
Fix this by constricting or omitting the tag list depending on reported
GP version.
Change-Id: I74cd2bd47617d616bede6453397f544cde5abcb7
TS 102 221 section 7.3.1.1.4 clause 4b lets the card answer a case #4
command TPDU with a 62xx/63xx warning, upon which the terminal sends a
dummy GET RESPONSE to obtain the 61xx that announces the response length.
__send_apdu_T0() applies that unconditionally, to the status word that
terminates a GET RESPONSE...
That is not "redundant", as per GPC v2.3.1 section 11.4.3.2 table 11-38
GP GET STATUS (80 F2) answers 6310 "more data available", meaning reissue
the command as get next occurrence(s) with P2 bit 1 set (11.4.2.2 table
11-34), so for example a paginated registry
listing exchange looks like this:
84f22000024f00 => 61e4 first match, e4 bytes waiting
84c00000e4 => <page 1> 6310 more matches pending
84c0000000 => 6982 <- unsolicited, card rejects it
The card rejects the unsolicited GET RESPONSE with 6982, our SCP02
session breaks, and the next command fail with 6985. The 6310 never
reaches ADF_SD.get_status() either, so the pagination loop exits after
page 1 and prints a sliently truncated listing. Observed with a SJA5.
Fix by turning clause 4b würgaround into what it should be: a one shot
reaction to the SW returned for the command TPDU.
While at it, stop ADF_SD.get_status() from silently returning a truncated
registry: it treated every status word other than 6310 as
"nothing more to report". 6A88 is now the explicit empty result and anything
else raises, so a partial listing can no longer silently pass.
Adds unit tests for all of that so we dont break basic T0 things.
Change-Id: I10f8afa8dd5623a49a6a0e7132607b3a1fad2d8c
klein ignores host/port when endpoint_description is set, so -H has
no effect on the TLS listener, it binds every interface.
Append interface= to the string so -H means the same thing with and
without TLS.
Change-Id: I737c2e7cfa1ca7b825bd36a4a489760918031b8b
The ES9+ SM-DP+ address is used for the smdpAddress check and
for serverSigned1.serverAddress, both default to HOSTNAME.
Add --smdp-address (default: HOSTNAME) so the advertised address can carry a
port and still match what the LPA connects to when the TLS endpoint is bound
to other ports than 443. TLS certificate identity is unaffected, this is
only the ES9+ address.
SGP.22 defines both smdpAddress and serverSigned1.serverAddress as an FQDN.
Taken strictly that leaves no way to run an SM-DP+ on a port other than
443, so an address with a port is arguably outside the spec but needed
for testing.
Change-Id: I2fa822c7d5e0d5b68a6704a1779f9923505f9008
pyOpenSSL >= 25.0.0 makes a Context immutable once it has been used and
raises. Downgrading pyOpenSSL is not a fix either: < 25 does not import
against recent cryptography.
This server only speaks HTTP/1.1 anway so ALPN negotiation is unused
and this can be hotpatched for affected versions.
Change-Id: I5d53216f24a20625d12f0757015c19fe341303b9
GP Amendment B "Remote Application Management over HTTP" v1.2 and
ETSI TS 102 226 RFM/RAM server.
APDU command/response bodies use the Expanded Remote Application data
format.
--mode {ram,rfm} selects what we want to do, make sure to
target the right thing in the right mode, SD for RAM, AID for RFM.
This is hand-rolled on purpose:
Klein/Twisted do not help us here, have annoying headers and settings,
and don't work well at all if you want to import the file for tests and
so on. I have wasted quite some time trying to make this prettier by
re-using existing http handler parts because I thought well we import
all kinds of packages already it can't be that hard, unfortunately it is.
Change-Id: I4f243aeca65d0f33ff34e6932e3f4e156446af8b
Adds TS 102 226 tables 5.2a/5.10a indefinite length coding,
recommended by TS 102 226 5.2.1 for RAM/RFM over HTTPS.
Noteworthy notable things to note:
- encode_expanded_cmd()
indefinite version -> inner C-APDU TLVs are still definite
- decode_expanded_resp()s returned container does not care, but
Indef has no 'number of executed commands' TLV -> number_of_commands
is the R-APDU count.
Tests are being fed with some known-good values from my sja5 sessions.
Change-Id: I4e023112e98729489ed443eec3ed5ab45c773b17
The BIP relay ( the "handset" side for SCP81) never worked: the
connect callback in handle_OpenChannel was "never called" as the fixme
says, everything else was missing.
Fixme cause: card APDU I/O is driven synchronously, proactive command loop
lives in a blocking while loop (pySim.transport.LinkBase.send_apdu_checksw)
that runs on the Twisted reactor thread. A Twisted TCP4ClientEndpoint +
connectProtocol only completes when the reactor does reactor things,
but the reactor thread is stuck in that loop for the whole proactive
session...
Fixme fix: don't fight the reactor, just drive the relay channel with a plain
old blocking socket, which fits the synchronous execution model.
Channel numbers now come from the command Device identities (channel_N ->
low nibble) instead of the hard coded chan_nr == 1.
Additionally fix two bugs found on the path to scp81 glory:
- TERMINAL RESPONSE device identities are forced to terminal->UICC per
TS 102 223 6.8.2 (prepare_response() inverts the command identities,
which for a channel-addressed BIP command yields channel_N->UICC).
- Error responses now build a valid AddlInfoBip cause, prepare_response()
hard coded empty "additional information" cannot be encoded for a
BIP error.
And some tests based on real card interactions.
Change-Id: If96c768f2e35c20ea3753e601059410121517b60
A large OTA response (for example GP GET STATUS app registry) is split by the card
into multiple SMS, each carries a TS 23.040 9.2.3.24 'concatenated short messages'
IE in its UDH. Nothing recombines them, so smpp-ota-tool currently only sees the
first incomplete part.
Add ConcatenatedSmsReassembler that accepts TP-User-Data, buffers parts
by reference number, and returns the reassembled TP-User-Data in the canonical
single-part form. Non-concatenated SMS pass through unchanged.
Both the 8-bit+16-bit references are supported.
A reserved value in the concatenation IE is not an error, these messages
are handed back as is rather than rejected, so the caller can deal with that.
Reassembly leads to a result that looks like a fat single part message the
card could have produced given infinite sms sizes, so the existing decode_resp
path is unaffected.
Feeding those parts to the ota tool needs two more fixes because the card returns
the application response as several SMS via proactive SEND SHORT MESSAGE while
the ENVELOPE SMS-PP DOWNLOAD itself contains the POR without a app R-APDU.
smpplib poll() drains everything, so message_received_handler runs on each:
- a later status-only response must not overwrite an application response
already captured, or transceive_apdu finds no last_response_data and raises
- a response that cannot be decoded is be logged and skipped rather raised
out of client.poll() which kills the tool.
Plus tests from a sja5 session.
Incomplete sets are capped (oldest evicted) so they cannot pile up.
Change-Id: I8c81097e607e0d055c4f031bbcc8a74d5c24a0e7
A multi part OTA response (full GP GET STATUS registry or some other fat
response that exceeds one SMS) is delivered as several SMS via proactive
SEND SHORT MESSAGE. The card only gives us another part if it receives a
TERMINAL RESPONSE for the previous one.
Currently smpp2sim Proact.handle_SendShortMessage relays the SMS but
returns None, so the transport falls back to prepare_response(pcmd) with
the ProactiveCommand collection (empty .children) and crashes with
'not enough values to unpack (expected 1, got 0)', dropps the SMPP link,
and never fetches the remaining parts.
Fix: make handle_SendShortMessage return a successful TERMINAL RESPONSE
built from the decoded command so the handshake proceeds.
prepare_response() is extended to handle collection via .decoded
and raises a useful error.
The send_apdu_checksw general_result='FIXME' path is now avoided for
SendShortMessage but remains a problem for other handlers that return
None.
Change-Id: Ib96ce81c4ff093b8a6fc715f79617e95a2dd8433
since the very early days, pySim-shell.py has a commandline option
where the user may supply an ADM pin as commandline parameter.
(-a / --pin-adm and -A --pin-adm-hex) This was introduced to simplify
the usage of pySim-shell.py with shellscripts.
Unfortunately the code that handles those commandline options
duplicates the code of the verify_adm commmand. Fortunately it is
very easy to call pySim-shell commands directly using the CMD2
onecmd_plus_hooks method, so we can just call the verify_adm and
replace the duplicated code with that.
So far we are only able to use ADM1 pins from the commandline, since
we now practically use the verify_adm commnad, we can add another
parameter to allow the verification of ADM as well.
Related: SYS#8239
Change-Id: I7164fad757048774aa7186a84041febde75c351c
Add --format expanded flag to send C-APDUs in the expanded remote
application data format.
Each --apdu becomes its own C-APDU TLV, and the tool
logs the full per-command R-APDU list decoded from the Response Scripting
template and warns if the card reports a truncated response.
Default stays 'compact', unchanged.
Change-Id: Idad90756f85bb7e54ef720f6067bb5d6dcff0c42
TS 102 226 section 5.2 "Command and Response Scripting templates"
The advantage over compact RFM/RAM commands is one C-APDU TLV per command
and one R-APDU TLV per result, so multiple commands return the response
of each one rather than only that of the last.
encode_expanded_cmd() builds the Command Scripting template
decode_expanded_resp() decodes the Response Scripting template into a
Container.
The 'truncated' key must be checked!
OtaDialect.encode_cmd()/decode_resp() now has a remote_format param for
compact and expanded formats, default stays compact,so existing code is
unaffected.
Change-Id: Idec00d16fd1a7d4a7129b2a3b6f0ef37dabcecb7
Move the code from pySim-smpp2sim.py to its own file, so it can be properly
extended.
The current file parses argv, opens a reader and starts the Twisted
reactor at import time, so nothing else can import it.
No functional changes yet, improvements follow in later commits.
Change-Id: Ifd8a15684939977d29ea83a6b669daee14484e88
PySimLogger.setup() installs a process-global print callback.
PySimLogger_Test sets one, a helper that asserts the message equals a global
expected_message, and never removes it, so from the moment test_log runs,
every PySimLogger message emitted anywhere in the process is checked against
whatever string that global happens to hold.
Fortunately unittest discovery runs modules in sorted order, and today
the PySimLogger users that log during tests all sort before test_log, so
this only breaks as soon as I try to add tests, just like anything else
breaks as soon as I try to use it.
Change-Id: I481e2c443fe0f412380b0f1acf6da5971ffca147
SCP.overhead was so far set at construction time (SCP02: 8, SCP03:
s_mode), so the C-MAC length only.
Unfortunately sec lvl >= 3 pads the data field to the cipher block size
before encryption, so the real worst-case overhead is larger,
scc.max_cmd_len (255 - overhead) was too big, and ADF_SD.load()
used a hardcoded chunk_len=240.
Real world issue with a 286 byte CAP + SCP02 + sec lvl 3:
- 240-byte LOAD block is padded to 248,
- encrypted
- gets 8 byte C-MAC appended
-> Lc = 256
That dies with a weird "ValueError: bytes must be in range(0, 256)".
The only "fix" for that was to downgrade the seclevel.
STORE DATA has the same overflow with large max_cmd_len
(247 + padding + MAC = 256 as well).
Therefore the overhead must be properly calculated from the sec level.
While at it adjust the error in case I missed something to get a more
useful ValueError.
Change-Id: Ic208f3959a38896f64fb6ccefb24cc360a3ac3a2
how encrypt_key() pads a kcv:
len(key) % blocksize bytes
what it should do to actually do it right:
blocksize - len(key) % blocksize
so the plaintext handed to the cipher was only block aligned by luck as
long as the key length happened to be a multiple of half the block size.
And of course decrypt_key() did not invert encrypt_key() at all,
the clear text length of GP CardSpec v2.3 Table 11-70 precedes the
ENCRYPTED kcv, but it was parsed out of the DECRYPTED data, and the
length byte itself was fed to the cipher along with the cryptogram.
Fix this up with a helper and tests so it is actually usable.
Change-Id: I02b4f2ed948c31e1741e40f0226fb49757fa2570
while len(foo):
throws an exception when foo == None.
Instead doing
while foo:
fixes a problem when reading in empty SUCI calc info data, e.g. from
TS48v7.0_SAIP2.3_BERTLV_SUCI_NoRAMRFM.der.
Change-Id: Ia4e2356d0241d7a6ca399ba7e8be7f27ec836104
Jenkins: skip-card-test
The tests take long and don't need to be this many.
A patch is coming up that adds another profile to test SUCI in GFM,
this patch makes some room for that.
Change-Id: Ib75b6919a3acfddd99bf9baa9b6847ef731b9e67
Jenkins: skip-card-test
Particular reason: when manipulating the 5G SUCI parameters, the
mandatory services get-identity, profile-a-x25519 and profile-b-p256 may
need to be reconfigured.
In general, it is a good idea to run these checks anyway.
Change-Id: I5e6eef0f1845a25cddb03af8d16c40e305bcdc1f
Jenkins: skip-card-test
Sadly, the EF.ARR interpretation, specifically that of the AM_DO
depends on whether the rule is for a DF or EF. As we don't know this,
allow the user to specify what kind of decode they would like, at least
in the file-specific read_arr_record + read_arr_records command.
Change-Id: I05e629e8b7dc705730d7039d2d0170ee24f8f844
When trying to use `edit_binary_decoded` with an empty file, pysim
runs into a len(None) exception, because hpkl.to_dict()['hnet_pubkey_list'] returns
None.
Can reproduced with a CCC Camp 2023 usim and editing the file.
a000ff..ff (len = 200)
Co-authored-by: Harald Welte <laforge@osmocom.org>
Change-Id: Ib8e322e65dd768bfd49e7a5620a2163f12a74ec7
The method do_aram_store_ref_ar_do and do_aram_delete_all, which
are part of the nested AddlShellCommands class, may be moved into
the parent class as a static method, just like the already existing
get_config method.
This makes the functionality re-usable to callers that do not use
the CMD2 API.
Change-Id: Icd1b08ec707dd939bc9e8524d7f9431aa4daae7c
Related: SYS#6959
Add a new ConfigurableParameter that represents the MNC length
(2 or 3 digits) in EF.AD (Administrative Data).
Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test
Some Linux distributions (e.g. Arch Linux) already ship cmd2 3.x.x,
which removed the style()/Fg/Bg API in favor of stylize()/Color.
Add a version guard to select the right API at runtime.
Adjust the upper bound cap in requirements.txt and setup.py.
Change-Id: Ibf2ac7847933296fb06665c87f53ed6e1f315d27
Remove version guards for cmd2 < 2.0.0 and < 2.3.0, the Cmd2Compat
and Settable2Compat wrapper classes, and the old fg/bg color API -
none of these are needed since both requirements.txt and setup.py
already mandate cmd2 >= 2.6.2.
Change-Id: Ifd1c484ab66d74323d10e946347daa637cf6f5d8
As pointed out in the commit-log of Change-Id
I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6 we had different minimum
versions for cmd2 in requirements.txt vs setup.py. Let's align that.
Change-Id: I71cee0ec3ed2abec68ec567beaab13c868721dad
get_profiles_info only request for the default tag list, but
not all tags.
Add --all to the function to request for all known tags.
Change-Id: Ia6878519a480bd625bb1fa2567c1fd2e0e89b071
By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.
For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.
Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.
Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test
apply_val() was re-encoding the SMSP with the minimum total_len of 28,
which produces a 28-byte body with no alpha_id field. After a DER
round-trip, the profile machinery re-pads the body to the original
record length using the template's fill pattern, which may not be 0xFF.
Those non-0xFF fill bytes end up in the alpha_id area, and GSM 7-bit
decoding then fails with a KeyError when the modified profile is read
back.
Fix by:
- setting alpha_id = '' so the field is present but empty
- setting f_smsp.rec_len = 42 (28 fixed bytes + 14 bytes of alpha_id
padding) so the re-encoded body carries 0xFF-padded alpha_id space
and the efFileSize in the fileDescriptor stays consistent
- passing total_len=f_smsp.rec_len to encode_record_bin() so the
alpha_id area is actually padded to that length
Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test
validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.
Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.
Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test
Eric Wild