This repository contains a csv file titled testplancsv that encompasses all tests in a human readable format, covering SPID/CIE OIDC. In order to make the dataset more comprehensible and user-friendly, we have included a clear explanation of each column:
- UID: distinctive value assigned to Uniquely Identify each record in the dataset, as
entity_under_test-message_under_test-caracteristic. It is used as a name for each test. - Input to test: describes the message or the token that must be analyzed and, if needed, its peculiarities.
- Input to Entity Under Test: information, parameters, or specific details serving as inputs for the testing process.
- Output: records the result, data or information expected by the execution of the test.
- Pattern name: contains names categorizing different recognized structures. More details in the later section.
- Message Under test: the message of the protocol affected by the test.
- Test Name: contains a descriptive name of the test.
- Description: aims to explain how the test must be accomplished.
- Entity under test: the entity undergoing evaluation. It can be either Relying Party (RP), OpenID Provider (OP), Trust Anchor (TA), Federation Authority (SA), Attribute Authority (AA) or a mixture of them.
- Requirement: outlines the specific needs, criteria or conditions for a successful compliance. This field essentially describes the reason why the test is done. This field is useful for both understanding why a test is done and for being used as a starting point for the implementation of the tests that are not present in the sources already gathered.
- Requirement Source: specifies documents, standards, or guidelines used to define the requirement.
- Profile: specify the OIDC profile considered. It can be either OIDC Core or OIDC Federation.
- Reference OAuch: contains the references associated with OAuch
- Reference OpenID Connect Conformance Profiles v3.0: contains the references associated with OpenID Connect Conformance Profiles v3.0
- Notes: provides additional information, context or background to enhance the comprehension.
- Comments: contains textual annotations or remarks.
This column assists in cataloging and distinguishing various recognized recurring structures or formats. It has been divided in Correct Input and Wrong input, depending on the expected result.
The "correct generation" type corresponds to tests where "the tester takes as input a message generated by the Entity Under Test (EUT) and checks whether a specific parameter is there and/or its value is correct". While writing the code for the automation, we figured out that those tests are equal in a conceptual point or view, but not in a strictly implementative point of view.
| Pattern name | What we have to do | Information needed by the test | Need by the Oracle |
|---|---|---|---|
| HTTP list value | Verify the value of a paramname in the message type is between a list of value | - | message type | {head/body/url} | param_name | list_of_value |
| HTTP parameter presence_1 | Verify that the paramname (i.e., Location) in the message type conatins a value (i.e., code,state,iss) | - | message type | {head/body/url} | param_name | value |
| HTTP parameter presence | Verify the presence in the message type of a paramname (i.e., code,state,iss) | - | message type | {head/body/url} | param_name |
| HTTP parameter type | Verify the type (with a regex) of a paramname (i.e., code,state,iss) in the message type | - | message type | {head/body/url} | param_name.type |
| HTTP parameter value_1 | Verify that the paramvalue of a paramname in the message type is equal to an url_encoded value | - | message type | {head/body/url} | param_name | value |
| HTTP parameter value | Verify that the paramvalue of a paramname in the message type is equal to a value | - | message type | {head/body/url} | param_name | value |
| HTTP Status | Verify in the message type the HTTP status | - | message type | {head/body/url} | HTTP_status |
| JSON in JWT parameter type | Verify the type (regex) of a paramname located in a parampath (path separated by dots) in the JSON, in header/payload of a decoded JWT (jwt_name) in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | param_path | type |
| JWE list values | Verify the value of a paramname located in a decoded JWE is between a list (at least one present). Select the message type, JWE_name interested (all the JWE with [\s\S]*) and then the param | - | message type | {head/body/url} | JWE_name | {header/payload} | param_name | list_of_value |
| JWE parameter not in value | Verify the value is not between a list (all) of a paramname located in a decoded JWE | - | message type | {head/body/url} | JWE_name | {header/payload} | param_name | list_of_value |
| JWE parameter presence | Verify the presence of a paramname located in a decoded JWE | - | message type | {head/body/url} | JWE_name | {header/payload} | param_name |
| JWE parameter values | Verify the value of a paramname located in a decoded JWE | - | message type | {head/body/url} | JWE_name | {header/payload} | param_name | value |
| JWT Check-Save to JWT_same message | Send a message type and save the value of a paramname in a JWT_name. Then verify in the same message type that a jwt_param_name in the JWT (that is located at jwt_name) has values that are inside the param_name_value saved before | message type | {head/body/url} | jwt_name | {header/payload} | param_path | saved_param | message type | {head/body/url} | jwt_name | {header/payload} | jwt_param_name |
| JWT Check-Save to JWT | Send a message type and save the value of a paramname in a JWT_name. Then verify in the second message type that a jwt_param_name in the JWT (that is located at jwt_name) has values that are inside the param_name_value saved before | message type | {head/body/url} | jwt_name | {header/payload} | param_path | saved_param | message type | {head/body/url} | jwt_name | {header/payload} | jwt_param_name |
| JWT list parameter contains | Verify that the list present in the paramvalue of a parampath located in header/payload of a decoded JWT (jwt_name) in the message type contains certain values | - | message type | {head/body/url} | jwt_name | {header/payload} | param_path | ["param_value1", "param_value2", ...] |
| JWT list parameter does not contain | Verify that the list present in the paramvalue of a parampath located in header/payload of a decoded JWT (jwt_name) in the message type does not contain certain values | - | message type | {head/body/url} | jwt_name | {header/payload} | param_path | ["param_value1", "param_value2", ...] |
| JWT list value | Verify the value of a paramname located in header/payload of a decoded JWT (jwt_name) in the message type is between a list of value | - | message type | {head/body/url} | jwt_name | {header/payload} | param_name | list_of_value |
| JWT parameter JSON presence | Verify the presence of a parampath located in header/payload of a decoded JWT (jwt_name) in the message type | - | message type| {head/body/url} | jwt_name | {header/payload} | param_path |
| JWT parameter JSON value | Verify the jsonlist_value (at least one) of a json_param_path of a param_name in a JWT (located at jwt_name) in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | json_param_path | json_list_value |
| JWT parameter not in JSON value | Verify that the jsonlist_value of a json_param_path in a JWT (located at jwt_name) in the message type is not a given value (all) | - | message type | {head/body/url} | jwt_name | {header/payload} | json_param_path | ["json_param_value", "json_param_value1", ...] |
| JWT parameter not in value | Verify that the paramvalue of a param_name located in header/payload of a decoded JWT (jwt_name) in the message type is not inside a range of values | - | message type | {head/body/url} | jwt_name | {header/payload/signature} | param_name | ["json_param_value", "json_param_value1", ...] |
| JWT parameter presence | Verify the presence of a paramname located in header/payload of a decoded JWT (jwt_name) in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | param_name |
| JWT parameter type | Verify the type (reges) in header/payload of a decoded JWT (jwt_name) in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | type |
| JWT parameter values | Verify the paramvalue of a param_name located in header/payload of a decoded JWT (jwt_name) in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | param_name | param_value |
| JWT signature check | Verify the signature of a JWT (located at jwtname) the message type | - | message type | {head/body/url} | jwt_name | JWT_public_key |
| nested JWT Check-Save to JWT | Send a message type and save the value of a paramname in a jwt_name specified with a param_path. Then verify in the message type that a param_jwt in the JWT (that is located at jwt_name) contains a jwt (jwt_nested_name) which contains a parameter (jwt_param_name) that has a value (saved_param) | message type | {head/body/url} | jwt_name | {header/payload} | param_path | saved_param | message type | {head/body/url} | jwt_name | {header/payload} | param_jwt | jwt_nested_name | {header/payload} | jwt_param_name | |
| nested JWT parameter presence | Verify the presence of a paramname in a nested JWT (jwt_nested_name) located at param_jwt of the jwt_name in the message type | - | message type | {head/body/url} | jwt_name | {header/payload} | param_jwt | jwt_nested_name | {header/payload} | param_name |
| nested JWT parameter type | Verify that the param_value of a param_name is of a certain type (regex). The param_name is located in a jwt that is located in param of a json (json_param) in another param (jwt_nested_param) of the main jwt (jwt_name) | - | message type | {head/body/url} | jwt_name | {header/payload} | json_param | jwt_nested_param | {header/payload} | param_name.type |
| nested JWT parameter values | Verify that the paramvalue of a param_name of a nested JWT (jwt_nested_name) located at param_jwt of the jwt_name in the message type is inside a list of possible values | - | message type | {head/body/url} | jwt_name | {header/payload/signature} | param_jwt | jwt_nested_name | {header/payload/signature} | param_name | ["param_value1", "param_value2", ...] |
| nested JWT signature check | Verify the signature of a nested JWT (jwtnested_name) located at param_jwt of the jwt_name in the message type | - | message type | {head/body/url} | jwt_name | {header/payload/signature} | param_jwt | jwt_nested_name | JWT_public_key |
| Param Check-Save to JWT | Send a message type and save the value of a paramname. Then verify in another message type that a jwt_param_name in the JWT (that is located at jwt_name) has the param_name_value saved before | message type | {head/body/url} | param_name | saved_param | message type | {head/body/url} | jwt_name | {header/payload} | jwt_param_name |
| Param Status | Send a message type with modified parameters then verify the HTTP status of another message type | message type | {head/body/url} | param_name | param_value | message type | {head/body/url} | HTTP_status |
| Pattern name | What we have to do | Information needed by the test | Need by the Oracle |
|---|---|---|---|
| JWT Response | Send a message type with modified parameters values in jwtname then verify in another message type the HTTP_status | message type | {head/body/url} | jwt_name | {header/payload} | param_name | param_value | JWT_private_key | message type | {head/body/url} | HTTPStatus | {head/body/url} | Error_code |
| nested JWT edit | Send a message type with modified parameters values in nested_paramname in a paramname then verify in another message type the HTTP_status | message type | {head/body/url} | param_name | {header/payload} | nested_param_name | param_value | message type | {head/body/url} | HTTPStatus | {head/body/url} | Error_code |
| Param Response | Send a message type with modified parameters values then verify in another message type the HTTP_status | message type | {head/body/url} | param_name | param_value | message type | {head/body/url} | HTTPStatus | {head/body/url} | Error_code |
| Signature JWT Response | Send a message type with modified signature of a jwtname then verify in the message type the HTTP_status | message type | {head/body/url} | jwt_name | signature_value | message type | {head/body/url} | HTTPStatus | {head/body/url} | Error_code |
All the spid-cie-oidc implementations in this repo are tested by means of spid-cie-oidc-django which features an OP, TA, and RP that will be used as testbeds to execute the tests. Your RP will have to be integrated into this environment, and the original RP can be optionally removed.
The spid-cie-oidc-django environment is run by means of a docker-compose yaml, which starts all the containers. You will need to add your RP to the compose file and set it to use a proxy that is used for testing. More details will be provided later in this guide.
To add support for an RP, you have to adhere to the folder structure of the repo as described here. To start, you can use the template folder that we created with the basic folders and files needed.
Tip: if you plan to do a pull request to this repo with support for your RP, please add to the .gitignore all the unnecessary files and, if possible, downlaod at-the-moment the sources needed to build your Docker image in the build_and_run.sh script instead of pushing them to the repo. Otherwise, you could use a submodule if your sources are in a git repo.
If you have a Docker image hosted on a Docker registry, just use it inside the docker-compose.yml file. Otherwise, if you need to build an image locally, please provide the steps to do that in the build_and_run.sh script, and use the builded container inside the docker-compose.yml file.
Either way you choose, you now need to redirect all the traffic in your container to the proxy hosted on port 8080 on the container burpsuite. To do this, there are commands available on the template's compose that install redsocks and forward the packets to/from the default OP and TA to the proxy.
Note: A debian base image is suggested in order for the default proxy forwarding to work. You can use other base images, but you will need to redirect all the outgoing traffic to localhost:8002 and localhost:8000 (OP and TA) to burpsuite:8080, which is the proxy we are using. If the proxy forwarding doesn't work, some tests will not be executed.
Warning: The port exposed by your RP can be any, but avoid 8080, 8001, 8000, and 8002. By default, it is 8005.
Do the on-boarding process
- generate your RP jwks
- register the RP on the TA
- set "name", "sub" and "jwks" with the values shown in the previous step
- set isActive to true
- create new profile on the TA
- set this RP as Descendant
- set "SPID Public SP" as Profile
- set the Federation Entity as Issuer
- after the creation, review the profiles and copy the trust marks into your RP
- complete RP on-boarding
- go to
your-rp:port/.well-known/openid-federation?format=jsonendpoint and verify trust_marks are exposed
steps taken from spid-cie-oidc-java
Your RP should now be working with the spid-cie-oidc-django OP. Try a SSO login from your RP to the OP with the credentials "user" and "oidcuser".
You should write a session for your RP. The session is a list of actions that the browser will need to do to complete authentication on your RP. This is needed to trigger all the messages that will be intercepted by the proxy.
You can start from an existing session, such as s_CIE of spid-cie-oidc-django, and change the part of your RP.
if you need more details on how to write a session, check this section of the mig-t documentation
if you plan to push your RP to the mig repo, please put the session inside the
input/mig-t/sessions/folder.
For the tests to work, you will need to edit the msg_def.json file and edit it according to your RP. This step will be automated in future releases, but it is now needed for the tests to work properly. In the template file, you will find comments "// TODO" where you need to replace your-rp-url and your-rp-port with your RP one. Following is a brief description of each one:
line 61
"contains": "your-rp-url:your-rp-port"line 156
"check regex": "/fetch\\?sub=http://your-rp-url:your-rp-port" // TODOWarning: in line 156, please escape each regex-recognized symbol in your input with \\
line 222
"check regex": "/federation_fetch_endpoint\\?iss=http://your-rp-url:your-rp-port&sub=http://trust-anchor.org:8000"line 244
"check regex": "/federation_fetch_endpoint\\?iss=http://your-rp-url:your-rp-port&sub=http://subject-aggregator.org:8004"Edit the config_testplan.json file with the right private keys used and the correct url and port of your RP.
You can now execute generate_mr.sh script, that will generate the tests inside the tests folder. This script offers two optional command-line options to customize the test generation process:
--justFill <input_path>. This flag enables the script to only populate the testplans, from the provided input_path folder, with the information extracted from the configuration file.--django. This flag includes additional information related to the Django session in the generated tests.