When I open the .asn file in vim, pySim should not attempt to read the
vim .swp file as asn.1.
File "/home/moi/osmo-dev/src/pysim/pySim/esim/saip/__init__.py", line 45, in <module>
asn1 = compile_asn1_subdir('saip')
[...]
File "<frozen codecs>", line 325, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 21: invalid start byte
Related: OS#6937
Change-Id: I37df3fc081e51e2ed2198876c63f6e68ecc8fcd8
The class Iccid uses a BcdAdapter to encoded/decode the ICCID. This
works fine for ICCIDs that have an even (20) number of digits. In case
the digit count is odd (19), the ICCID the last digit requires padding.
Let's switch to PaddedBcdAdapter for encoding/decoding, to ensure that
odd-length ICCIDs are padded automatically.
Change-Id: I527a44ba454656a0d682ceb590eec6d9d0ac883a
Related: OS#6868
This is a follow up patch to change:
I2a5d4b59b12e08d5eae7a1215814d3a69c8921f6
- do not ignore length of kwargs
- fix role parameter (roles other than 'legacy_client' can be used now)
- use startswith instead of match
Related: SYS#7866
Change-Id: Ifae13e82d671ff09bddf771f063a388d2ab283eb
this fixes the following two warnings:
pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNode.walk:1: WARNING: Inline strong start-string without end-string. [docutils]
pySim/esim/saip/__init__.py:docstring of pySim.esim.saip.FsNodeDF.walk:1: WARNING: Inline strong start-string without end-string. [docutils]
Change-Id: Id7debf9296923b735f76623808cee68967a1ece7
While at it, also use tuples (const) instead of lists (var).
Tweaked-by: nhofmeyr@sysmocom.de (docstring, tuples)
Change-Id: Iaa6e710132e3f4c6cecc5ff786922f6c0fcfb54e
unfortunately the API changes introduced in change
I277aa90fddb5171c4bf6c3436259aa371d30d092
broke the API interface of http_json_api.py. This was taken into
account and necessary to introduce add the server functionality next
to the already existing client functionality. The changes to the API
were minimal and all code locations that use http_json_api.py
were re-aligned.
Unfortunately it was not clear at this point in time that there are
out-of-tree projects that could be affected by API changes in
http_json_api.py
To mitigate the problem this patch introduces an alternative API
interface to the JsonHttpApiFunction base class. This alternative
API interface works like the old API interface when the class is
instantiated in the original way. To make use of the revised client
the API use has to pass an additional keyword argument that defines
the role.
Related: SYS#7866
Change-Id: I2a5d4b59b12e08d5eae7a1215814d3a69c8921f6
At the moment pySim.ota codebase is not covered by any of the
integration tests (we have only normal unittests so far). To
increase the test coverage, let's add an integration test that
sends exchanges an RFM OTA-SMS with a real-world card.
However, there is no tool avaliable that can be used as an SMPP
client for pySim-smpp2sim yet. Let's use smpp_ota_apdu2.py on
laforge/ota to develop a tool that we can use to exchange SMS-TPDUs
that contain remote APDU scripts (RFM/RAM).
Finally let's use the tool we have created as a basis to create
an integration test that exchanges an SMS-TPDU with the RFM
application of a sysmoISIM-SJA5 card. The testcase shall pass
when we get the expected response from the card.
Related: OS#6868
Change-Id: If25e38be004cc1c7aeeb130431831377e78fe28d
If there is an empty body returned, such as in the case of the response
to an es9p notification, then it is of course also legal to not set the
content-type header.
This patch fixes an exception when talking to certain SM-DP+ with
es9p_client.py:
DEBUG:pySim.esim.http_json_api:HTTP RSP-STS: [204] hdr: {'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
DEBUG:pySim.esim.http_json_api:HTTP RSP: b''
{'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
<Response [204]>
Traceback (most recent call last):
File "gprojects/git/pysim/es9p/../contrib/es9p_client.py", line 315, in <module>
c.do_notification()
~~~~~~~~~~~~~~~~~^^
File "projects/git/pysim/es9p/../contrib/es9p_client.py", line 159, in do_notification
res = self.peer.call_handleNotification(data)
File "projects/git/pysim/contrib/pySim/esim/es9p.py", line 174, in call_handleNotification
return self.handleNotification.call(data)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "projects/git/pysim/contrib/pySim/esim/http_json_api.py", line 335, in call
if not response.headers.get('Content-Type').startswith(req_headers['Content-Type']):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'startswith'
Change-Id: I99e8f167b7bb869c5ff6d908ba673dac87fef71a
This introduces an "APDU source" for pySim-trace which enables the
decoding of APDUs that are copy+pasted from elsewhere, for example
APDU logs in text form created by proprietary tools, or to decode
personalization scripts or the like.
Change-Id: I5aacf13b7c27cea9efd42f01dacca61068c3aa33
For all ConfigurableParameter subclasses, provide an example_input.
This may be useful for downstream projects' user interaction, to suggest
a value or prefill an input field, as appropriate.
Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".
(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)
Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.
But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().
For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.
Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.
K and Opc use a common abstract BinaryParam.
Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240
Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.
DecimalHexParam will also be used for Pin and Adm soon.
Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods
The new features are optional, and will be heavily used by batch
personalization patches coming soon.
Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.
So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.
Details:
1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)
2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.
Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
The module still uses print to output information. Let's replace
those print calls with the more modern PySimLogger method calls.
Change-Id: I2e2ec2b84f3b84dbd8a029ae9bb64b7a96ddbde3
At the moment we use random identifiers as names when we create a
new logger for pySimLogger. Let's switch to consistently use the
module name here. For the top level modules let's use the program
name so that it will show up in the log instead of __init__.
Change-Id: I49a9beb98845f66247edd42ed548980c97a7151a
Make sure we make use of the fill pattern when encoding file contents:
Only encode the differences to the fill pattern of the file, in order
to reduce the profile download size.
Change-Id: I61e4a5e04beba5c9092979fc546292d5ef3d7aad
The python logger method warn is deprecated since pyton 3.3, let's us
the warning method as suggested.
Change-Id: I3a4c0ca43768198ac6011ebe79050f91c04862e5
At the moment http_json_api only supports the client role. Let's also add
support for the server role.
This patch refactors the existing client code. This in particular means
that the following preperations have to be made:
- To use the existing JsonHttpApiFunction definitions in the client and
server the scheme has to be symetric. It already is for the most part,
but it treads the header field differently. So let's just treat the
header field like any other mandatory field and add it input_params.
(this does not affect the es9p.py code since in ES9+ the requests have
no header messages, see also SGP.22, section 6.5.1.1)
- The JsonHttpApiFunction class currently also has the code to perform
the client requests. Let's seperate that code in a JsonHttpApiClient
class to which we pass an JsonHttpApiFunction object.
- The code that does the encoding and decoding in the client role has
lots of conditions the treat the header differently. Let's do the
decisions about the header in the JsonHttpApiClient. The encoder
and decoder function should do the generic encoding and decoding
only. (however, some generic header specific conditions will remain).
The code for the server role logically mirrors the code for the client
role. We add a JsonHttpApiServer class that can be used to create
API endpoints. The API user has to pass in a call_handler through which
the application logic is defined. Above that we add an Es2pApiServer
class in es2p. In this class we implement the logic that runs the
HTTP server and receives the requests. The Es2pApiServer supports all
ES2+ functions defined by GSMA SGP.22. The user may use the provided
Es2pApiServerHandler base class to define the application logic for each
ES2+ function.
Related: SYS#7825
Change-Id: I277aa90fddb5171c4bf6c3436259aa371d30d092
The line actual_sec = func_ex_status.get('statusCodeData', None) suggests
that 'statusCodeData' may be None under normal circumstances. So let's guard
sec.update(actual_sec) so that we won't run into an exception in case
'statusCodeData' is not in func_ex_status.
Related: SYS#7825
Change-Id: I8a1a3cd5e029dba4a3aec1a64702e19b0d694ae2
This method did not work at all at the moment, likely due to API churn
over time. This change makes the following exception go away:
Traceback (most recent call last):
File "projects/git/pysim/contrib/saip-tool.py", line 473, in <module>
do_remove_naa(pes, opts)
~~~~~~~~~~~~~^^^^^^^^^^^
File "projects/git/pysim/contrib/saip-tool.py", line 203, in do_remove_naa
pes.remove_naas_of_type(naa)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/__init__.py", line 1748, in remove_naas_of_type
if template in hdr.decoded['eUICC-Mandatory-GFSTEList']:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "projects/git/pysim/contrib/pySim/esim/saip/oid.py", line 48, in __eq__
return (self.intlist == other.intlist)
^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'intlist'
A subsequent patch should introduce unit tests to avoid such breakage in
the future.
Change-Id: I88d862d751198c3d1648ab7f11d6e6a8fdbc41c9
This adds a new check method to the pySim.esim.saip.validation.CheckBasicStructure
class, which ensures that no unused authentication algorithm related mandatory
services are indicated in the ProfileHeader.
So if a profile e.g. states in the header it requires
usim-test-algorithm, but then the actual akaParameter instances do not
actually use that algorithm, it would raise an exception.
Change-Id: Id0e1988ae1936a321d04bc7c3c3a33262c767d30
Related: SYS#7826
Some templates (e.g. for 5GS) define files which aren't completely defined.
5GS OPL5G: doesn't have a file size defined in the template,
but a record size.
Change-Id: I5ec1757d6852eb24d3662ec1c3fc88365e90a616
Define the file size early if possible.
Some templates (e.g. for 5GS) define files which aren't completely defined.
Fixes the parsing for 5GS SUCI_Calc_Info which doesn't have a file size defined.
The saip-tool will other crash when reading a 5G enabled profile:
```
Traceback (most recent call last):
File "./contrib/saip-tool.py", line 458, in <module>
pes = ProfileElementSequence.from_der(f.read())
File "pySim/esim/saip/__init__.py", line 1679, in from_der
inst.parse_der(der)
~~~~~~~~~~~~~~^^^^^
File "pySim/esim/saip/__init__.py", line 1552, in parse_der
self.pe_list.append(ProfileElement.from_der(first_tlv, pe_sequence=self))
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pySim/esim/saip/__init__.py", line 557, in from_der
inst._post_decode()
~~~~~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 668, in _post_decode
self.pe2files()
~~~~~~~~~~~~~^^
File "pySim/esim/saip/__init__.py", line 655, in pe2files
file = File(k, v, template.files_by_pename.get(k, None))
File "pySim/esim/saip/__init__.py", line 133, in __init__
self.from_tuples(l)
~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 358, in from_tuples
self._body = self.file_content_from_tuples(l)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
File "pySim/esim/saip/__init__.py", line 393, in file_content_from_tuples
stream.write(self.template.expand_default_value_pattern(self.file_size))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "pySim/esim/saip/templates.py", line 123, in expand_default_value_pattern
raise ValueError("%s does not have a default length" % self)
ValueError: FileTemplate(EF.SUCI_Calc_Info) does not have a default length
```
Change-Id: I7c4a0914aef1049a416e6b091f23daab39a1dd9c
The Card Key Provider currently only has support for CSV files
as input. Unfortunately using CSV files does not scale very well
when the card inventory is very large and continously updated.
In this case a centralized storage in the form of a database
is the more suitable approach.
This patch adds PostgreSQL support next to the existing CSV
file support. It also adds an importer tool to import existing
CSV files into the database.
Change-Id: Icba625c02a60d7e1f519b506a46bda5ded0537d3
Related: SYS#7725
The other source files have a line break between the character encoding
qualifier line and the python comment. Let's add a line break here
as well to maintain consistency.
Change-Id: Ied6b77eede748f1ddf6fde17c9b434fa4dd1114a
Fixes parsing of a 2.3 UICC profile.
This might be the wrong end as the spec says this is
NSI, but somehow it's working
Change-Id: I3cde1093156db274458d76e2c1c2e304d55a8466
The correct abbreviated version of the company name is
"sysmocom - s.f.m.c. GmbH", i.e. lowercase and with dash.
Change-Id: Id768d2f4b78162ff83320a800e4e66f1bd324d6d
There are more files where trailing digits are indicated using 'f' and
should be stripped during decode, including EF.MSISDN and EF.VGCS
This is not just a presentation issue, but actually rendered wrong data
before, see the modified test output where our "read_record_uicc.ok"
file contained "bcd_len: 7" but then only 6 BCD digits due to this bug.
Change-Id: I4571482da924a3d645caa297108279d182448d21
It's a not-too-uncommon requirement to modify the SMSC address stored in
EF.SMSP. This adds a ConfigurableParameter for this purpose.
Change-Id: I6b0776c2e753e0a6d158a8cf65cb030977782ec2
We've had files2pe() for re-encoding all of the files, but let's add
a specific one for re-encoding only one of the files (such as commonly
needed during personalization)
Change-Id: I7b7f61aae6b7df6946dadf2f78fddf92995603ec
As the input phone number ("address") might be of an odd length of
digits, let's use PaddedBcdAdapter to fix two problems:
1) strip any potential trailing f in decoding
2) fix truncation of last digit during encoding
Change-Id: I1e9865e172bc29b8a31c281106d903934e81c686
Depends: pyosmocom Ib5afb5ab5c2bc9b519dc92818fc6974f7eecba16 (0.0.12
EF.SMSP contains up to two addresses: Both are stored in a fixed-length
field of 12 octets. However, the actually used size depends on the
number of digits in the respective number. Let's compute that length
field properly
Change-Id: Idef54a3545d1a5367a1efa2f0a6f7f0c1f860105
