Make sure we make use of the fill pattern when encoding file contents:
Only encode the differences to the fill pattern of the file, in order
to reduce the profile download size.
Change-Id: I61e4a5e04beba5c9092979fc546292d5ef3d7aad
The python logger method warn is deprecated since pyton 3.3, let's us
the warning method as suggested.
Change-Id: I3a4c0ca43768198ac6011ebe79050f91c04862e5
We already have a tool to work with the ES2+ API provided by an SMDP+
(es2p_client.py) With this tool we can only make API calls towards
an SMDP+. However, SGP.22 also defines a "reverse direction" ES2+
interface through wich the SMDP+ may make API calls towards the MNO.
At the moment the only possible MNO originated API call is
ES2+handleDownloadProgressInfo. Let's add a simple tool that runs a
HTTP server to receive and log the ES2+handleDownloadProgressInfo
requests.
Related: SYS#7825
Change-Id: I95af30cebae31f7dc682617b1866f4a2dc9b760c
At the moment http_json_api only supports the client role. Let's also add
support for the server role.
This patch refactors the existing client code. This in particular means
that the following preperations have to be made:
- To use the existing JsonHttpApiFunction definitions in the client and
server the scheme has to be symetric. It already is for the most part,
but it treads the header field differently. So let's just treat the
header field like any other mandatory field and add it input_params.
(this does not affect the es9p.py code since in ES9+ the requests have
no header messages, see also SGP.22, section 6.5.1.1)
- The JsonHttpApiFunction class currently also has the code to perform
the client requests. Let's seperate that code in a JsonHttpApiClient
class to which we pass an JsonHttpApiFunction object.
- The code that does the encoding and decoding in the client role has
lots of conditions the treat the header differently. Let's do the
decisions about the header in the JsonHttpApiClient. The encoder
and decoder function should do the generic encoding and decoding
only. (however, some generic header specific conditions will remain).
The code for the server role logically mirrors the code for the client
role. We add a JsonHttpApiServer class that can be used to create
API endpoints. The API user has to pass in a call_handler through which
the application logic is defined. Above that we add an Es2pApiServer
class in es2p. In this class we implement the logic that runs the
HTTP server and receives the requests. The Es2pApiServer supports all
ES2+ functions defined by GSMA SGP.22. The user may use the provided
Es2pApiServerHandler base class to define the application logic for each
ES2+ function.
Related: SYS#7825
Change-Id: I277aa90fddb5171c4bf6c3436259aa371d30d092
The line actual_sec = func_ex_status.get('statusCodeData', None) suggests
that 'statusCodeData' may be None under normal circumstances. So let's guard
sec.update(actual_sec) so that we won't run into an exception in case
'statusCodeData' is not in func_ex_status.
Related: SYS#7825
Change-Id: I8a1a3cd5e029dba4a3aec1a64702e19b0d694ae2
This method did not work at all at the moment, likely due to API churn
over time. This change makes the following exception go away:
Traceback (most recent call last):
File "projects/git/pysim/contrib/saip-tool.py", line 473, in <module>
do_remove_naa(pes, opts)
~~~~~~~~~~~~~^^^^^^^^^^^
File "projects/git/pysim/contrib/saip-tool.py", line 203, in do_remove_naa
pes.remove_naas_of_type(naa)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/__init__.py", line 1748, in remove_naas_of_type
if template in hdr.decoded['eUICC-Mandatory-GFSTEList']:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/oid.py", line 48, in __eq__
return (self.intlist == other.intlist)
^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'intlist'
A subsequent patch should introduce unit tests to avoid such breakage in
the future.
Change-Id: I88d862d751198c3d1648ab7f11d6e6a8fdbc41c9
This adds a new check method to the pySim.esim.saip.validation.CheckBasicStructure
class, which ensures that no unused authentication algorithm related mandatory
services are indicated in the ProfileHeader.
So if a profile e.g. states in the header it requires
usim-test-algorithm, but then the actual akaParameter instances do not
actually use that algorithm, it would raise an exception.
Change-Id: Id0e1988ae1936a321d04bc7c3c3a33262c767d30
Related: SYS#7826
Some templates (e.g. for 5GS) define files which aren't completely defined.
5GS OPL5G: doesn't have a file size defined in the template,
but a record size.
Change-Id: I5ec1757d6852eb24d3662ec1c3fc88365e90a616
Define the file size early if possible.
Some templates (e.g. for 5GS) define files which aren't completely defined.
Fixes the parsing for 5GS SUCI_Calc_Info which doesn't have a file size defined.
The saip-tool will other crash when reading a 5G enabled profile:
```
Traceback (most recent call last):
File "./contrib/saip-tool.py", line 458, in <module>
pes = ProfileElementSequence.from_der(f.read())
File "pySim/esim/saip/__init__.py", line 1679, in from_der
inst.parse_der(der)
~~~~~~~~~~~~~~^^^^^
File "pySim/esim/saip/__init__.py", line 1552, in parse_der
self.pe_list.append(ProfileElement.from_der(first_tlv, pe_sequence=self))
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pySim/esim/saip/__init__.py", line 557, in from_der
inst._post_decode()
~~~~~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 668, in _post_decode
self.pe2files()
~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 655, in pe2files
file = File(k, v, template.files_by_pename.get(k, None))
File "pySim/esim/saip/__init__.py", line 133, in __init__
self.from_tuples(l)
~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 358, in from_tuples
self._body = self.file_content_from_tuples(l)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 393, in file_content_from_tuples
stream.write(self.template.expand_default_value_pattern(self.file_size))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "pySim/esim/saip/templates.py", line 123, in expand_default_value_pattern
raise ValueError("%s does not have a default length" % self)
ValueError: FileTemplate(EF.SUCI_Calc_Info) does not have a default length
```
Change-Id: I7c4a0914aef1049a416e6b091f23daab39a1dd9c
The Card Key Provider currently only has support for CSV files
as input. Unfortunately using CSV files does not scale very well
when the card inventory is very large and continously updated.
In this case a centralized storage in the form of a database
is the more suitable approach.
This patch adds PostgreSQL support next to the existing CSV
file support. It also adds an importer tool to import existing
CSV files into the database.
Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3
Related: SYS#7725
The other source files have a line break between the character encoding
qualifier line and the python comment. Let's add a line break here
as well to maintain consistency.
Change-Id: Ied6b77eede748f1ddf6fde17c9b434fa4dd1114a
Fixes parsing of a 2.3 UICC profile.
This might be the wrong end as the spec says this is
NSI, but somehow it's working
Change-Id: I3cde1093156db274458d76e2c1c2e304d55a8466
The correct abbreviated version of the company name is
"sysmocom - s.f.m.c. GmbH", i.e. lowercase and with dash.
Change-Id: Id768d2f4b78162ff83320a800e4e66f1bd324d6d
When an eUICC performs a profile installation it returns a (concatenated)
series of ASN.1 encoded strings as "simaResponse". In case the profile
installation fails for some reason the simaResponse contains diagnostic
information to diagnose why the profile installation failed.
Unfortunately there are currently no practical tools available to decode
and display the information in the simaResponse. Let's add a tool for that.
Related SYS#7617
Change-Id: Ida4c3c5446653b283a3869c0c387f328ae51e55e
In case pySim-shell is used directly from the git repository (not
installed via a package manager), the version command fails with an
exception because pkg_resources.get_distribution('pySim') fails.
Let's renovate the version command and migrate from pkg_resources to
importlib.resources. There are many users and developers out there who
retrieve pySim-shell directly from the git repository and not via pip3.
To accommodate for that, let's check if pySim-shell.py is located in a
git repository and if so, let's display the HEAD commit hash instead.
Since the version of the currently installed pyosmocom version also
plays a critical role, let's display the pyosmocom version as well.
Related: OS#6830
Change-Id: I2b9038f88cfcaa07894a2f09c7f5ad8a5474083d
At the moment, the help text for the --csv option shows the path to
the users home. This is due to the default value, which is dynamically
generated. Let's use a static string with "~/" and resolve the full
path later when we need it.
Related: SYS#7725
Change-Id: Ied8b1e553de8f5370369c4485a2360906c874ed2
There are more files where trailing digits are indicated using 'f' and
should be stripped during decode, including EF.MSISDN and EF.VGCS
This is not just a presentation issue, but actually rendered wrong data
before, see the modified test output where our "read_record_uicc.ok"
file contained "bcd_len: 7" but then only 6 BCD digits due to this bug.
Change-Id: I4571482da924a3d645caa297108279d182448d21
It's a not-too-uncommon requirement to modify the SMSC address stored in
EF.SMSP. This adds a ConfigurableParameter for this purpose.
Change-Id: I6b0776c2e753e0a6d158a8cf65cb030977782ec2
We've had files2pe() for re-encoding all of the files, but let's add
a specific one for re-encoding only one of the files (such as commonly
needed during personalization)
Change-Id: I7b7f61aae6b7df6946dadf2f78fddf92995603ec
As the input phone number ("address") might be of an odd length of
digits, let's use PaddedBcdAdapter to fix two problems:
1) strip any potential trailing f in decoding
2) fix truncation of last digit during encoding
Change-Id: I1e9865e172bc29b8a31c281106d903934e81c686
Depends: pyosmocom Ib5afb5ab5c2bc9b519dc92818fc6974f7eecba16 (0.0.12
EF.SMSP contains up to two addresses: Both are stored in a fixed-length
field of 12 octets. However, the actually used size depends on the
number of digits in the respective number. Let's compute that length
field properly
Change-Id: Idef54a3545d1a5367a1efa2f0a6f7f0c1f860105
* add another set of test data (from a real-world SIM card)
* switch from test_decode to test_de_encode as our encoder now works due
to previous commits.
Change-Id: I8d16e195641bb59b2c26072008f88434692c0cab
the PySimLogger class currently only accepts cmd2 color enum values.
This is what we need for pySim-shell.py. However, in case we want to
use the PySimLogger in stand-alone programs that do not use cmd2, this
is a bit bulky. Let's add some flexibility to PySimLogger, so that we
can specify the colors as raw ANSI strings as well.
Change-Id: I93543e19649064043ae8323f82ecd8c423d1d921
Related: SYS#7725
Some of the encoders can only generate valid output if they are told
the expected output size. This is due to variable-length fields that
depend on the size of the total record (or file). Let's always pass
the expected length to the encoder methods.
Change-Id: I88f957e49b0c88a121a266d3582e79822fa0e214
In the test_encode_file() method, we should actually test the encoder,
and not the decoder. I suppose this was a copy+paste mistake at some
point? In the LinearFixedEF_Test.test_encoder_record we were already
testing the encoder. Just TransparentEF_Test got it wrong...
Change-Id: Id23305a78ab9acd2e006f2b26b72408795844d23
There's a 3-bit RFU field that (unlike everything else in USIM/UICC)
considers '1' to be the default. Let's make sure we get that right
during encode.
Change-Id: Ibe24a07f5f73d875d2077fa55471dbfc4e90da23
On my debian unstable system with wireshark 4.6.2-3, the pyshark_gsmtap
APDU source misses to report any ATRs, as those are not part of what's
reported with the 'gsm_sim' display filter. This is due to
wireshark.git commit bcd82e2370d18e20983b378d494964d89c191cef first part
of the 4.6.0 release, which splits the ATR dissection into a separate
sub-dissector.
We cannot use the seemingly logical 'gsmtap.type == 4' instead, as old
wireshark simply bypasses any output for the gsmtap header if the SIM
sub-dissector is used.
Hence, 'gsm_sim || iso7816.atr' is something compatible with older and
newer wireshark versions.
Change-Id: I53c1c8ed58a82c37cd4be4af3890af21da839e86
Milenage offers the capability for operators to modify the r1-r5
rotation constants as well as the c1-c5 xor-ing constants; let's
add ConfigurableParameters for that.
Change-Id: I397df6c0c708a8061e4adc0fde03a3f746bcb5b6
Related: SYS#7787
TS 51.011 Section 10.5.6 refers to clause 10.5.1 (EF.ADN),
and the latter permits UCS2 in addition to 7-bit GSM alphabet.
Change-Id: If10b3d6d8b34ece02dc0350ca9ea9c3f8fbf3c9e
Otherwise we might compute float values and fail encoding like this:
> construct.core.FormatFieldError: Error in path (building) -> tp_vp_minutes
> struct '>B' error during building, given value 169.0
Change-Id: I989669434c7ddee9595ee81a0822f9966907a844
When trying to remove a file (e.g. DF.5G_ProSe, 5FF0),
there seems to be a case sensitive check when checking for the dict:
pySim/runtime.py: get_file_for_filename():
478 def get_file_for_filename(self, name: str):
479 """Get the related CardFile object for a specified filename."""
480 sels = self.selected_file.get_selectables()
481 return sels[name]
The dict sels contains 5ff0, but not 5FF0.
The type of argument name is str. So a case sensitive check will be used.
Change-Id: Idd0db1f4bbd3ee9eec20f5fd0f4371c2882950cd
Closes: OS#6898
cmd2 version 3.0 was released, with significant API changes. Limit the
dependency to below 3.0, as already reflected in requirements.txt.
Seeing but not changing the discrepancy in minimum version:
requirements.txt has >2.6.2 while setup.py has >= 1.5.0.
Related: SYS#7775 SYS#7777
Change-Id: I5186f242dbc1b770e3ab8cdca7f27d2a1029fff6
Let's not reinvent the wheel of printing such data structures and use
the repr method provided by the respective class instead. This also
adds the missing key_usage_qualifier information to the print-out,
as well as the mac_len of the key components.
Change-Id: Iaead4a02f07130fd00bcecc43e1c843f1c221e63
The method get_field in the base class can be optimized out. This
also allows us to remove code dup in the card_key_provider_get_field
function.
Let's also fix the return code behavior. A get method in a
CardKeyProvider implementation should always return None in case
nothing is found. Also it should not crash in that case. This will
allow the card_key_provider_get function to move on to the next
CardKeyProvider. In case no CardKeyProvider yields any results, an
exception is appropriate since it is pointless to continue execution
with "None" as key material.
To make the debugging of problems easier, let's also print some debug
messages that inform the user what key/value pair and which
CardKeyProvider was queried. This will make it easier to investigate
in case an expected result was not found.
Related: SYS#7725
Change-Id: I4d6367b8eb057e7b2c06c8625094d8a1e4c8eef9
The method _verify_get_data was intended to be used to verify the
user input before it further processed but ended up to be a simple
check that only checks the name of the key column very basically.
Unfortunately it is difficult to generalize the check code as the
concrete implementation of those checks is highly format dependent.
With the advent of eUICCs, we now have two data formats with
different lookup keys, so a static list with valid lookup keys is
also no longer up to the task.
After all it makes not much sense to keep this method, so let's
remove it.
(From the technical perspective, the key column is not limitied to
any specif field. In theory it would even be possible to use the KI
as lookup key as well, even though it would not make sense in
practice)
Related: SYS#7725
Change-Id: Ibf5745fb8a4f927397adff33900731524715d6a9
