At the moment pySim.ota codebase is not covered by any of the
integration tests (we have only normal unittests so far). To
increase the test coverage, let's add an integration test that
sends exchanges an RFM OTA-SMS with a real-world card.
However, there is no tool avaliable that can be used as an SMPP
client for pySim-smpp2sim yet. Let's use smpp_ota_apdu2.py on
laforge/ota to develop a tool that we can use to exchange SMS-TPDUs
that contain remote APDU scripts (RFM/RAM).
Finally let's use the tool we have created as a basis to create
an integration test that exchanges an SMS-TPDU with the RFM
application of a sysmoISIM-SJA5 card. The testcase shall pass
when we get the expected response from the card.
Related: OS#6868
Change-Id: If25e38be004cc1c7aeeb130431831377e78fe28d
Remove the previous workaround that set cmd2==2.4.3 in jenkins.sh. The
bug this worked around has been fixed in 2.6.2.
3.0 will break unless we use some new additional decorator.
Related: OS#6776
Change-Id: I4ba65ed486247c5670313b75f43a242d264df14b
Recent versions of cmd2 have changed how the 'prog' attribute is
automatically set for ArgumentParser instances. As a result, we
are now seeing an unexpected 'build.py' artifact appearing in
the generated documentation.
Let's use an older release of cmd2, which retains the old expected
behavior. Use it specifically for building documentation.
Change-Id: Ifbad35adc5e9d3141acfd024d7dee2a25f1cb62e
Related: https://github.com/python-cmd2/cmd2/issues/1414
Related: OS#6776
The build system uses a virtual environment, in which it installs
pysim and its dependencies. This is done for the integration tests,
but not when building the sphinx documentation. However, the
documentation build process also invokes pysim code to generate
documentation from the docstrings. This means we need pysim with
all its dependencies for the doc building as well.
Change-Id: I6381eeef7fa19873ca0cc330a0ab43b7ef5096e4
Related: SYS#7094
In order to run this script from pyosmocom's contrib/jenkins.sh script,
we want to skip the clean workspace step. Add an environment variable to
do that.
Related: OS#6570
Change-Id: Ic8dc9b85da17719195f7374d37eccb4dedba6ce8
This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.
Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de
This is the only way we can make sure pylint has all required
information about imports from packages we depend upon.
Change-Id: I29582aa3d7f9ace9ce832d5b907420aaf14881fb
We currently have the shell script that performs the test in the
tests directory and the related data in pysim-testdata directory.
This is confusing, let's have evrything in a dedicated sub directory
Change-Id: Ic995a7f600d164fc0be3c2eb8255dbe043429bea
Related: OS#6531
We currently have the test data for pySim-trace in pysim-testdata.
This means we mix the test data with the data from our original
pySim integration tests. This is very confusing. Let's put the
test data and the testcase for pySim-trace into a dedicated
sub directory.
Change-Id: I565b4268a05c1a1334b5e7d3fbcd9ef2ef0f0c4c
Related: OS#6531
We currently mix the unit-tests with the shell script based integration
tests. Let's put them into a dedicated sub directory.
Related: OS#6531
Change-Id: I0978c5353d0d479a050bbb6e7ae5a63db5e08d24
This is a tool to work with eSIM profiles in SAIP format. It allows
to dump the contents, run constraint checkers as well as splitting
of the PE-Sequence into the individual PEs.
Change-Id: I396bcd594e0628dfc26bd90233317a77e2f91b20
The idea of this new job is to catch package integrity problems,
like the missing entries in setup.py/packages[] or missing deps.
Change-Id: Ic72d58494e8fd0cab8d66ce60f7b70593b770872
Related: osmo-ci.git I9d4d9e9de2b16a4b745791f3c9c93507f43bfa6d
We now have pySim-shell and pySim-trace. Let's give pysim-test.sh a more
distinctive name so that it is clear to which program it refers.
Related: OS#6094
Change-Id: I438f63f9580ebd3c7cc78cc5dab13c9937ac6e3a
pySim-trace has no test coverage yet. Let's use a script to run a
GSAMTAP pcacp through it and check that no exceptions are raised.
Related: OS#6094
Change-Id: Icfabfa7c59968021eef0399991bd05b92467d8d2
Split the jenkins job up in three parts, so each of them can run in
parallel, and the test part that has to run on a specific node (and
blocks it while running), finishes faster.
Don't install depends of pylint/docs jobs as they will run in docker
and the depends get installed once in the container.
Related: OS#5497
Depends: docker-playground Id5c75725d2fab46b29773fa4f637fa2d73fa7291
Depends: osmo-ci Iea4f15fd9c9f8f36cb8d638c48da000eafe746a4
Change-Id: I5245c529db729e209d78a02ab9c917a90d0e0206
-x Print commands and their arguments as they are executed
-e Exit immediately if a command exits with a non-zero status
Change-Id: I13af70ef770936bec00b050b6c4f988e53ee2833
Use multiple processes to speed up pylint. Specifying -j0 will
auto-detect the number of processors available to use.
On AMD Ryzen 7 3700X this significantly reduces the exec time:
$ time python -m pylint -j1 ... pySim *.py
real 0m12.409s
user 0m12.149s
sys 0m0.136s
$ time python -m pylint -j0 ... pySim *.py
real 0m5.541s
user 0m58.496s
sys 0m1.213s
Change-Id: I76d1696c27ddcab358526f807c4a0a7f0d4c85d4
pylint v2.15 is crashing, let's fall-back to a known to work v2.14.5.
Change-Id: Ie29be6ec6631ff2b3d8cd6b2dd9ac0ed8f505e4f
Related: https://github.com/PyCQA/pylint/issues/7375
Related: OS#5668
This introduces a new pySim.apdu module hierarchy, which contains
classes that represent TPDU/APDUs as exchanged between
SIM/UICC/USIM/ISIM card and UE.
It contains instruction level decoders for SELECT, READ BINARY and
friends, and then uses the pySim.filesystem.Runtime{Lchan,State} classes
to keep track of the currently selected EF/DF/ADF for each logical
channel, and uses the file-specific decoder classes of pySim to decode
the actual file content that is being read or written.
This provides a much more meaningful decode of protocol traces than
wireshark will ever be able to give us.
Furthermore, there's the new pySim.apdu_source set of classes which
provides "input plugins" for obtaining APDU traces in a variety of
formats. So far, GSMTAP UDP live capture and pyshark based RSPRO
live and pcap file reading are imlpemented.
Change-Id: I862d93163d495a294364168f7818641e47b18c0a
Closes: OS#5126
There is no $base in this script, and the current form renders:
make: *** /docs: No such file or directory. Stop.
Build step 'Execute shell' marked build as failure
Change-Id: Ifcf27f7497daeb285dfb364bff20d0c861c77dcb
Related: OS#5271
5.1 was the version introducing pyyaml.FullLoader which we're using,
see https://pyyaml.org/wiki/PyYAML#history
Change-Id: I0f2fa08ceeac2759218e85ad5bdce3ef951d0b74
This will replace the hand-crafted codec for EF_SPN
by a struct definition using the construct library.
Old encoders are updated and kept for API compatibility
but are not used internally anymore.
New data structures:
* Rpad(Adapter): Right-padded bytestring (0xff, adjustable)
* GsmStringAdapter(Adapter): Codec for "SMS default 7-bit
coded alphabet as defined int TS 23.038" using
the gsm0338 library.
* GsmString(n): Convenient wrapper of both above
Adjustments:
* utils: update+deprecate old dec_spn(), enc_spn()
* remove refs to deprecated functions
Change-Id: Ia1d3a3835933bac0002b7c52511481dd8094b994
The Access Mode (AM) and Security Condition (SC) DOs are incredibly
convoluted, so we need a lot of code to properly decode them.
Change-Id: If4f0725a849d41fd93de327ed00996d8179f2b0e
'construct' is a declarative symmetric encoder/decoder for user
specified binary formats. It should come in extremely handy in
tools like pySim.
We start the integration by adding transport methods for transceiving
APDUs with built-in encoding of the command data and decoding of the
response data.
Change-Id: Ibf457aa8b9480a8db5979defcfafd67674303f6c
We use a slightly modified version of sphinx-argparse
(with patch https://github.com/alex-rudakov/sphinx-argparse/pull/136 applied)
in order to generate the command reference for each shell command in the
manual.
As the upstream repository seems unmaintained for ~2 years, let's use
the osmocom 'fork' with that above-mentioned patch applied.
Change-Id: I134f267cf53c7ecbc8cbfb33a8766d63bf4a8582
As it turns out, we had this set of unit tests since 2018, but
so far they were not executed during the build verification.
Let's fix this:
* run unittest in discovery mode for all files in 'tests/' (commented out);
* rename this file, so it can be automatically detected and executed;
* properly import the API to be tested.
Currently 2 out of 16 unit tests are failing, so we need to get
them passing first and then uncomment the unittest execution.
Change-Id: I4d4facfabc75187acd5238ff4d0f26022bd58f82
pySim-prog was nice when there were only 5 parameters on a SIM that we
could program, and where the use case was pretty limited. Today, we
have SIM/USIM/ISIM cards with hundreds of files and even more parameters
to program. We cannot add a command line argument for each file to
pySim-prog.
Instead, this introduces an interactive command-line shell / REPL,
in which one can navigate the file system of the card, read and update
files both in raw format and in decoded/parsed format.
The idea is primarily inspired by Henryk Ploatz' venerable
cyberflex-shell, but implemented on a more modern basis using
the cmd2 python module.
See https://lists.osmocom.org/pipermail/simtrace/2021-January/000860.html
and https://lists.osmocom.org/pipermail/simtrace/2021-February/000864.html
for some related background.
Most code by Harald Welte. Some bug fixes by Philipp Maier
have been squashed.
Change-Id: Iad117596e922223bdc1e5b956f84844b7c577e02
Related: OS#4963
When using the batch mode of pySim-prog, the user has to insert/remove
the cards from the cardreader manually. This is fine for small batches,
but for high volume batches this method is not applicable.
This patch adds support for the integration of an automatic card handler
machine. The user can freely configure a custom commandline that is
executed when a card should be inserted or moved to a good/bad
collection bin.
Change-Id: Icfed3cad7927b92816723d75603b78e1a4b87ef1
Related: SYS#4654
This test data is used by the jenkins build verification for pySim,
and it was previously located locally on the build slave.
By moving the testdata to this repository, any contributor can
modify both the code and the expected test results simultaneously.
Change-Id: I6714b091a114035d6aab8ba750c5f2b86e438467
