Added Parameterization Documentation and changed Documentation Structure

README.md

@@ -94,7 +94,7 @@ supported, once the base EDC has a reliable version.
 
 ## Getting Started
 
-The fastest way to get started is our [Getting Started Guide](./docs/getting-started/README.md)
+The fastest way to get started is our [Getting Started Guide](docs/README.md)
 which takes you through the steps of configuring and starting via
 [docker-compose.yaml](docker-compose.yaml).
 

docs/README.md

@@ -0,0 +1,135 @@
+<!-- PROJECT LOGO -->
+<br />
+<div align="center">
+  <a href="https://github.com/sovity/edc-extensions">
+    <img src="https://raw.githubusercontent.com/sovity/edc-ui/main/src/assets/images/sovity_logo.svg" alt="Logo" width="300">
+  </a>
+
+<h3 align="center">sovity Community Edition EDC:<br />Getting Started</h3>
+
+  <p align="center">
+    <a href="https://github.com/sovity/edc-extensions/issues/new?template=bug_report.md">Report Bug</a>
+    ·
+    <a href="https://github.com/sovity/edc-extensions/issues/new?template=feature_request.md">Request Feature</a>
+  </p>
+</div>
+
+## Getting Started
+
+The following example shows you how to fully configure an MDS EDC.
+
+1. Ensure the [EDC Image](../../connector) you are using in the [docker-compose.yaml](../../docker-compose.yaml) file
+   is `edc-ce-mds`.
+2. Adjust the Connector Metadata:
+   ```yaml
+   # Connector Technical Name
+   MY_EDC_NAME_KEBAB_CASE: "example-connector"
+
+   # Connector Localized Name / Title
+   MY_EDC_TITLE: "EDC Connector"
+
+   # Connector Description Text
+   MY_EDC_DESCRIPTION: "sovity Community Edition EDC Connector"
+
+   # Connector Curator
+   # The company using the EDC Connector, configuring data offers, etc.
+   MY_EDC_CURATOR_URL: "https://example.com"
+   MY_EDC_CURATOR_NAME: "Example GmbH"
+
+   # Connector Maintainer
+   # The company hosting the EDC Connector
+   MY_EDC_MAINTAINER_URL: "https://sovity.de"
+   MY_EDC_MAINTAINER_NAME: "sovity GmbH"
+   ```
+3. Generate your `.jks` file and your SKI/AKI ClientID. See [FAQ](#faq) for more information.
+4. Adjust your data space specific configuration:
+    ```yaml
+    # Required by the broker-extensions,
+    EDC_OAUTH_CLIENT_ID: '_your SKI/AKI_'
+    EDC_KEYSTORE: '_your keystore file relative to docs/getting-started/secrets_'
+    EDC_KEYSTORE_PASSWORD: '_your keystore password_'
+
+    # MDS Test Environment
+    EDC_OAUTH_TOKEN_URL: 'https://daps.test.mobility-dataspace.eu/token'
+    EDC_OAUTH_PROVIDER_JWKS_URL: 'https://daps.test.mobility-dataspace.eu/jwks.json'
+    EDC_BROKER_BASE_URL: 'https://broker.test.mobility-dataspace.eu'
+    EDC_CLEARINGHOUSE_LOG_URL: 'https://clearing.test.mobility-dataspace.eu/messages/log'
+    ```
+5. Start the EDC
+   ```shell
+   # Log-in into the Github Container Registry
+   docker login ghcr.io
+
+   # Fetches latest images
+   docker compose pull
+
+   # Runs EDC and EDC UI at localhost:11000
+   docker compose up
+   ```
+6. Visit [localhost:11000](http://localhost:11000)
+
+## OpenAPI Yaml
+
+There's an [openapi.yaml](../openapi.yaml) that includes the vanilla EDC endpoints exposed by the current control- and
+data plane APIs.
+
+## Postman Collection
+
+To further try out our EDC Backend without the UI, try our Postman Collection:
+
+1. Use Postman (https://github.com/postmanlabs)
+2. Import the collection [docs/postman_collection.json](../postman_collection.json).
+3. Depending on your configuration changes, you need to adjust variables on collection `MDV > Variables > Current Value`
+4. `api_key` needs to be aligned with `EDC_API_AUTH_KEY`
+5. To test the MDS Broker functionality, simply execute steps
+    1. `1 Create Asset`
+    2. `2 Create Policy`
+    3. `3 Create ContractDefinition`: You will see a notification about registering resource at broker, which will then
+       be reflected in the Broker's UI.
+    4. `Delete ContractDefinition`: You will see a notification about unregistering the resource at broker.
+
+## Docker Compose File Variants
+
+There are currently two maintained Docker Compose files in our root directory.
+
+| File                                                     | Description                                             |
+|----------------------------------------------------------|---------------------------------------------------------|
+| [docker-compose.yaml](../../docker-compose.yaml)         | Run a stable version.                                   |
+| [docker-compose-dev.yaml](../../docker-compose-dev.yaml) | Run the current development version, which might break. |
+
+## FAQ
+### What should the client ID entry look like?
+Example of a client-ID entry:
+
+`EDC_OAUTH_CLIENT_ID: 7X:7Y:...:B2:94:keyid:6A:2B:...:28:80`
+
+### How do you get the SKI and AKI of a p12 and how do you convert it to a jks?
+You can use a script (if you're on WSL or Linux) to generate the SKI, AKI and jks file.
+
+1. Make sure you're on Linux or on a bash console (e.g. WSL or Git Bash) and have openssl and keytool installed
+2. Navigate in the console to the resources/docs directory
+3. Run the script `./secrets/generate_ski_aki.sh [filepath].p12 [password]` and substitute [filepath] to the p12 certificate filepath and
+   [password] to the certificate password
+4. The jks file will be generated in the same folder as your p12 file and the SKI/AKI combination is printed out in the console.
+   Copy the SKI:AKI combination and use it to start the EDC (optionally also save it to your password manager).
+
+### Where should the connector certificate be stored?
+In the default configuration the connector certificate should be stored inside a `keystore.jks` in a folder `docs/getting-started/secrets` next to the docker-compose. The path and keystore name can be edited in the env-variable `EDC_KEYSTORE`.
+
+### Can I run a connector locally and consume data from an online connector?
+No, locally run connectors cannot exchange data with online connectors. A connector must have a proper URL + configuration and be accesible from the data provider via REST calls.
+
+### Can I disable the Broker- and/or ClearingHouse-Client-Extensions dynamically?
+Yes, if the two extensions are included, they can still be disabled via properties.
+The default settings can be found in `docker-compose.yaml` and can be changed there.
+````java
+# Extension Configuration
+BROKER_CLIENT_EXTENSION_ENABLED: true
+CLEARINGHOUSE_CLIENT_EXTENSION_ENABLED: true
+````
+
+## Further Documentation
+- [Manging a sovity EDC Connector via the API Wrapper Java Client Library](./api_wrapper.md)
+- [How to share parameterized HTTP data sources to expose entire APIs](./parameterized_assets.md)
+- [How Can a Pull-Data-Transfer Be Performed?](./pull-data-transfer.md)
+- [Parameterized Assets](./parameterization-documentation.md)

docs/api_wrapper.md

@@ -0,0 +1,34 @@
+Manging a sovity EDC Connector via the API Wrapper Java Client Library
+========
+
+Introduction to the sovity EDC API Wrapper
+========
+The sovity EDC API Wrapper contains several APIs, of which some are available in either our sovity EDC CE or our sovity CE EE / Connector-as-a-Servcie (CaaS). These APIs are made accessible via type-safe generated client libraries. Please note that most of these APIs are not yet complete and are under development:
+- **Use Case API**: Generic API for Use Case Applications. Its goal is to replace the Management API, so there can be stable endpoints across milestones in the auto-generated client libraries. It's still in development, so expect many new endpoints to be added here in the near future.
+- **UI API**: API endpoints for the sovity EDC UI: These endpoints might contain interesting data, that a Use Case Application might profit from, but expect these endpoints to be unstable and subject to change.
+- **Enterprise Edition API**: Special API endpoint only available in the Connector-as-a-Service (CaaS). Features such as File Storage are currently in development, but to be expected in the near future.
+
+Using the Java Client Library
+========
+This requires JDK11 or higher, and either a Gradle or Maven project.
+
+Installing The Java Client Library
+========
+Connect your Maven or Gradle Project to the Github Maven Registry
+- Maven: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry#authenticating-to-github-packages
+- Gradle: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-gradle-registry#authenticating-to-github-packages
+- This might require a Github Personal Access Token (PAT)
+Add the Java Client Library to your Maven/Gradle project: https://github.com/sovity/edc-extensions/packages/1825774
+
+Configuring The Client
+========
+- Configure the Client with either an API Key or OAuth2 Client Credentials: https://github.com/sovity/edc-extensions/tree/main/extensions/wrapper/client#usage
+- Your management API URL should look like https://your-connector-name.prod-sovity.azure.sovity.io/control/data
+
+Using The Client
+========
+Feel free to use the endpoints of the aforementioned API groups.
+Example Usage of a Use Case API Endpoint:
+```java
+KpiResult kpiResult = client.useCaseApi().kpiEndpoint();
+```

docs/data-transfer-methods.md

@@ -0,0 +1,12 @@
+Which data transfer methods are supported by the EDC-Connector?
+========
+
+The connector supports three different data transfer modes:
+
+1. HTTPData: The consumer EDC fetches the data and pushes it to the data sink.
+2. HTTPProxy: The consumer EDC fetches the data and passes it synchronously to the consumer.
+3. OutOfBand: The data source transfers the data directly to the data sink, the data is not routed through the connector.
+
+The following diagram illustrates the different transmission modes:
+
+![data-transfer-methods.png](images%2Fdata-transfer-methods.png)

docs/getting-started/README.md

@@ -1,3 +1,11 @@
+# Documentation Has Moved
+
+This documentation is outdated and will no longer be updated. For the most current and accurate information, please visit the new location.
+
+You can find the updated document at: [ReadMe](../../README.md)
+
+---
+
 <!-- PROJECT LOGO -->
 <br />
 <div align="center">

docs/getting-started/documentation/api_wrapper.md

@@ -1,3 +1,11 @@
+# Documentation Has Moved
+
+This documentation is outdated and will no longer be updated. For the most current and accurate information, please visit the new location.
+
+You can find the updated document at: [Manging a sovity EDC Connector via the API Wrapper Java Client Library](../../api_wrapper.md)
+
+---
+
 Manging a sovity EDC Connector via the API Wrapper Java Client Library
 ========
 

docs/getting-started/documentation/data-transfer-methods.md

@@ -1,3 +1,11 @@
+# Documentation Has Moved
+
+This documentation is outdated and will no longer be updated. For the most current and accurate information, please visit the new location.
+
+You can find the updated document at: [Data Transfer Methods](../../data-transfer-methods.md)
+
+---
+
 Which data transfer methods are supported by the EDC-Connector?
 ========
 

docs/getting-started/documentation/parameterized_assets.md

@@ -1,3 +1,11 @@
+# Documentation Has Moved
+
+This documentation is outdated and will no longer be updated. For the most current and accurate information, please visit the new location.
+
+You can find the updated document at: [How to share parameterized HTTP data sources to expose entire APIs](../../parameterized_assets.md)
+
+---
+
 How to share parameterized HTTP data sources to expose entire APIs
 ========
 

docs/getting-started/documentation/pull-data-transfer.md

@@ -1,3 +1,11 @@
+# Documentation Has Moved
+
+This documentation is outdated and will no longer be updated. For the most current and accurate information, please visit the new location.
+
+You can find the updated document at: [How Can a Pull-Data-Transfer Be Performed?](../../pull-data-transfer.md)
+
+---
+
 How Can a Pull-Data-Transfer Be Performed?
 ========
 
@@ -94,4 +102,4 @@ To start a pull-http-transfer using the management-API of the EDC one can send t
 }
 ```
 
-The `receiver.http.endpoint` setting is used to set the endpoint of the Pulling-Backend dynamically. Note that this setting will be renamed `https://w3id.org/edc/v0.0.1/ns/receiverHttpEndpoint` in the future `0.1.0` version of the EDC.
+The `receiver.http.endpoint` setting is used to set the endpoint of the Pulling-Backend dynamically. Note that this setting will be renamed `https://w3id.org/edc/v0.0.1/ns/receiverHttpEndpoint` in the future `0.1.0` version of the EDC.

docs/parameterization-documentation.md

@@ -0,0 +1,98 @@
+# Parameterized Assets Guide
+
+This guide will help you understand how to set up and use HTTP Data assets in our system.
+
+## What are Parameterized Assets?
+
+These are HTTP Data assets where you can change certain settings, or "parameters".
+
+## Settings You Can Change
+
+when you're choosing a HTTP Data asset in the user interface (UI), there are options you can turn on or off to let you
+change certain parts of the HTTP request. These are called "overridable" fields because you can change, or "override,"
+the default settings. Here's what each option does:
+
+### Method overridability:
+if you're using the asset and you've enabled "method overridability," you must also specify a "Custom HTTP Method."
+
+### Path overridability:
+If turned on, you can add more to the "path" of the request. The path is the part of the URL
+   that comes after the domain name.
+
+### Query Param overridability:
+When "query parameter overridability" is turned on, you can modify the parameters included in your HTTP request's URL. This gives you flexibility to customize your request.
+
+Query parameters are the part of a URL that come after a question mark ?. For example, in www.example.com/search?q=query, q=query is a query parameter.
+
+Here's how it works:
+
+When you enable query parameter overridability, our system combines two sets of parameters:
+
+1. Default parameters: These are the parameters that come with the HTTP Data asset by default.
+
+2. Your parameters: These are the extra parameters that you provide as the user, or "consumer," of the asset.
+
+The system merges these two sets of parameters together. If there are any conflicts (for example, if both sets of parameters try to set the same parameter), your parameters will take precedence and override the default ones.
+
+For example, let's say the HTTP Data asset has a default parameter of color=blue. You, as the consumer, provide a parameter of color=red. The final, merged set of parameters will be color=red because your parameter overrides the default.
+
+### Request Body overridability:
+If turned on, you can change the content of the request, also known as the "body."
+This is mainly used with methods like POST or PUT, where you need to send data to the server.
+
+The "request body" is the part of an HTTP request that contains the data you're sending to the server. This is typically used with methods like POST or PUT, where you're sending new data or updating existing data.
+
+Our system includes a feature called "request body overridability." Here's how it works:
+
+When you turn on request body overridability, you gain the ability to change the data sent in the request body.
+
+By default, the HTTP Data asset may come with a preset request body. However, with request body overridability enabled, you as the consumer can replace this preset request body with one of your own.
+
+Let's say, for example, that the HTTP Data asset has a default request body like this:
+
+```json
+{
+"name": "Default Name",
+"message": "Default Message"
+}
+```
+
+You can override this default by providing your own request body:
+
+```json
+{
+"name": "Your Name",
+"message": "Your Custom Message"
+}
+```
+
+In this example, your custom request body replaces the default one entirely.
+
+Note: you can only set your own request body if request body overridability is turned on. If it's not turned on, the system will use the default request body and you won't be able to change it.
+
+## Rules to Follow
+
+- For methods like GET, HEAD, DELETE, OPTIONS, and others, you can't have a request body.
+- For methods like POST, PUT, PATCH, you must have a request body.
+- If you break these rules or forget to choose a method, the process will finish, but no data will be sent.
+
+# Understanding Parameter Validation and Its Limitations
+
+When you're using our HTTP Data asset with parameterization enabled, it's important to understand how parameter validation works and some of the limitations you might encounter.
+
+## About Parameter Validation
+
+When you make a request with the HTTP Data asset, our system checks, or "validates," your parameters. If your parameters are invalid, you might run into some issues.
+
+For example, the system might say that your data transfer is complete even when no data has been sent or received. This happens because the validation only checks your parameters at the data plane stage of the data transfer process, and does not check them at the control plane stage.
+
+## Missing Method Issues
+
+This can also happen if you don't provide a method for your request, even though you've turned on method overridability. Without a method, your request can't be completed correctly, but the system might not catch this issue because of the validation limitations mentioned above.
+
+## About Asset Properties
+
+Our parameterization process creates special "asset properties" that let you know what kind of parameterization is enabled for your HTTP Data asset. However, these properties aren't currently shown in the user interface.
+
+This means that to know which parameterization is enabled for your asset, you'll need to note it down manually.
+