Add testsuite for pySim-shell with real cards

This patch adds a comprehensive testsuite for pySim-shell. The testsuite
is based on python's unittest framework in combination with pySim-shell
scripts.

Related: OS#6531
Change-Id: Ieae1330767a6e55e62437f5f988a0d33b727b5de

tests/pySim-shell_test/README.txt

@@ -0,0 +1,107 @@
+Integration Testsuite for pySim-shell
+=====================================
+
+The testsuite presented here is build on python's unittest framework, which is
+extended via a UnittestUtils class, which can be found in utils.py. This class
+contains methods to execute pySim-shell, compare files, load file contents,
+generate pySim-shell scripts from templates, etc. It also cares about managing
+the cards and their related data (ICCIDs, ADM pins, keys, etc.)
+
+Each testcase has its own subdirectory and can be executed separately. The sub
+directory of each testcase usually contains a single python script (test.py),
+one or more pySim-shell scripts or templates and also one or more .ok files.
+The .ok files are auto-generated using a known-to-work version of pySim-shell
+and can be re-generated at any-time if required.
+
+This testsuite is designed to be executed automatically on a build sever, but
+it can also be executed on a developer workstation.
+
+Requirements
+------------
+
+The testsuite is executes pySim-shell on physical cards. This means all card
+models for which the tests were written must be physially present in PCSC
+readers.
+
+(see `config.yaml` for which card models are required)
+
+Configuration
+-------------
+
+The card models, which are present in the testsuite are set up via
+`config.yaml`. All cards are listed by their model ´name´, ´atr´ and ´iccid´
+(or eid for eUICCs). The testsuite user should not add or remove models. The
+only change that is required is to adjust the iccid and eid fields so that they
+match the values of the cards that are used for the test.
+
+Card data, such as ADM, PIN1, PIN2, etc. are configured in ´card_data.csv´. This
+file is used with the --csv parameter of pySim-shell. The format is described
+in the osmopysim-usermanual. Each card configured in ´config.yaml´ has a
+coresponding entry in ´card_data.csv´. The entries are connected via either
+the ´iccid´ or the eid as ´key´. Like with ´config.yaml´, the testsuite user
+must adjust the values, so that they match the actual cards.
+
+The file card_data.csv is also read by the testsuite, so that all contained
+fields are also available to the testcases.
+
+Data collection
+---------------
+
+On startup, the testsuite will check the presence of each card configured in
+´config.yaml´. While doing that, the PCSC reader number is determined. Then it
+will use either the ´iccid´ or the ´eid´ to fetch the card data from
+card_data.csv.
+
+All information, that is gathered during the startup procedure is collected
+in a dict that is available to the testcase. The testsuite will list all cards
+and their related information on startup.
+
+Running
+-------
+
+Testcases are implemented as python unittests and the execution of the testcases
+is no different from the execution of other python unittests.
+
+To run all tests, run the following command line from the top directory of the
+pySim repository:
+
+```
+python3 -m unittest discover ./tests/pySim-shell_test/
+```
+
+It is also possible to run a specific test only:
+```
+python3 -m unittest discover -k export_fs ./tests/pySim-shell_test
+```
+
+Regenerating .ok files
+----------------------
+
+The testsuite ships with a set of .ok files. Those files are used to compare
+the output of certain pySim-shell commands. In case the .ok files contain card
+specific data (IMSI values, ICCID values, etc.) the comparison happens in such
+a way that this data is ignored. This means a card with a different IMSI,
+ICCID, Ki etc. should work with the included .ok files. However, there may be
+changes in the code or in the file system structure that may cause a certain
+testcase to fial anyway. In this case a regeneration of the .ok files may be
+necessary.
+
+To instruct the testsuite to regenerate all .ok files, change the ´regenerate´
+field in config.yaml to True and run the testsuite once. The testsuite will then
+overwrite the .ok files with the output it got from pySim-shell. When the
+testcase execution is done, the regenerate field must be set back to False and
+the new content of the .ok files must be reviewed.
+
+Keeping temporary files
+-----------------------
+
+During development of new testcases or for debugging, it can be helpful not to
+delete all temporary files after a test run. To keep the temporary files, the
+´keepfiles´ field in config.yaml can be set to True.
+
+Printing file contents
+----------------------
+
+To locate testcase failures more easily, the user has the option to display the
+content of files that are compared or loaded from a testcase. To display the
+file contents, set the field ´print_content´ in config.yaml to true