From d04d9b119d92d777781e20b219eee799febb7b7c Mon Sep 17 00:00:00 2001
From: Catarina Paralta <46568597+paralta@users.noreply.github.com>
Date: Wed, 17 Jan 2024 17:43:40 +0000
Subject: [PATCH 1/5] docs: multiple go modules rfc

---
 rfc/multiple-go-modules.md | 74 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)
 create mode 100644 rfc/multiple-go-modules.md

diff --git a/rfc/multiple-go-modules.md b/rfc/multiple-go-modules.md
new file mode 100644
index 000000000..07f48c718
--- /dev/null
+++ b/rfc/multiple-go-modules.md
@@ -0,0 +1,74 @@
+# [RFC] Split VMClarity into multiple go modules
+
+*Note: this RFC template follows HashiCrop RFC format described [here](https://works.hashicorp.com/articles/rfc-template)*
+
+|               |                                             |
+| ------------- | ------------------------------------------- |
+| **Created**   | 2024-01-17                                  |
+| **Status**    | **WIP** \| InReview \| Approved \| Obsolete |
+| **Owner**     | paralta                                     |
+| **Approvers** | vmclarity-maintainers                       |
+
+---
+
+This RFC proposes a new project structure which splits the main module into multiple modules to allow us to version modules separately and reduce dependencies.
+
+## Background
+
+VMClarity consists of a single module containing all the packages required to run VMClarity. However, there is a clear boundary between the following packages: api, cli, orchestrator, ui and uibackend.
+
+The build and deployment of these packages is already performed separately and independently.
+
+## Proposal
+
+The proposal here is to split the VMClarity repository into multiple modules:
+
+- **api**. Interface between all the services in VMClarity including the DB. Composed by API model, backend client and server.
+- **scanner** or **cli**. Responsible for running a scan in an asset and report the results back to api. Contains the logic to configure, run and manage different analysers and scanners. 
+- **orchestrator**. Responsible for managing scan configurations, scans, assets and estimations.
+- **provider**. Responsible for discovery and scan infrastructure setup for each provider. Contains logic to find assets and run scans on AWS, GCP, Azure, Docker and Kubernetes.
+- **uibackend**. Responsible for offloading the ui from data processing and filtering. Slightly coupled with ui. Composed by API model, backend client and server.
+- **utils**. Contains packages shared between modules.
+
+Each module will have its own go.mod file and each module will be versioned independently.
+
+## Implementation
+
+The scope of this RFC is not to change code logic but to change code structure. Therefore, the following table describes the path changes for each package impacted.
+
+| Module       | Current path                  | New path                      |
+| ------------ | ----------------------------- | ----------------------------- |
+| api          | pkg/apiserver                 | api/server                    |
+| api          | pkg/shared/backendclient      | api/client                    |
+| scanner      | pkg/cli                       | scanner/cli                   |
+| scanner      | pkg/shared/analyzer           | scanner/analyzer              |
+| scanner      | pkg/shared/config             | scanner/config                |
+| scanner      | pkg/shared/converter          | scanner/converter             |
+| scanner      | pkg/shared/families           | scanner/families              |
+| scanner      | pkg/shared/findingkey         | scanner/findingkey            |
+| scanner      | pkg/shared/job_manager        | scanner/jobmanager            |
+| scanner      | pkg/shared/scanner            | scanner/scanner               |
+| scanner      | pkg/shared/utils              | scanner/utils                 |
+| orchestrator | pkg/orchestrator              | orchestrator                  |
+| orchestrator | pkg/containerruntimediscovery | orchestrator/runtimediscovery |
+| uibackend    | pkg/uibackend                 | uibackend                     |
+| uibackend    | pkg/uibackend/rest            | uibackend/server              |
+| uibackend    | pkg/shared/uibackendclient    | uibackend/client              |
+| utils        | pkg/version                   | utils/version                 |
+| utils        | pkg/shared/command            | utils/command                 |
+| utils        | pkg/shared/fsutils            | utils/fsutils                 |
+| utils        | pkg/shared/log                | utils/log                     |
+| utils        | pkg/shared/manifest           | utils/manifest                |
+
+
+Furthermore, the provider could be removed from the orchestrator.
+
+| Module       | Current path                  | New path                      |
+| ------------ | ----------------------------- | ----------------------------- |
+| provider     | pkg/orchestrator/provider     | provider                      |
+
+The Dockerfiles for each package will be moved to the corresponding directory. Makefile, GitHub workflows and other files will need to be updated.
+
+## UX/UI
+
+This RFC has no user-impacting changes.

From 0cb6aabf51e0c0ca60c0bad00785ff720848c769 Mon Sep 17 00:00:00 2001
From: Catarina Paralta <46568597+paralta@users.noreply.github.com>
Date: Thu, 18 Jan 2024 10:09:24 +0000
Subject: [PATCH 2/5] docs: multiple go modules rfc review

---
 rfc/multiple-go-modules.md | 179 ++++++++++++++++++++++++++++++-------
 1 file changed, 146 insertions(+), 33 deletions(-)

diff --git a/rfc/multiple-go-modules.md b/rfc/multiple-go-modules.md
index 07f48c718..9970bd9ea 100644
--- a/rfc/multiple-go-modules.md
+++ b/rfc/multiple-go-modules.md
@@ -24,7 +24,7 @@ The build and deployment of these packages is already performed separately and i
 The proposal here is to split the VMClarity repository into multiple modules:
 
 - **api**. Interface between all the services in VMClarity including the DB. Composed by API model, backend client and server.
-- **scanner** or **cli**. Responsible for running a scan in an asset and report the results back to api. Contains the logic to configure, run and manage different analysers and scanners. 
+- **cli**. Responsible for running a scan in an asset and report the results back to api. Contains the logic to configure, run and manage different analysers and scanners. 
 - **orchestrator**. Responsible for managing scan configurations, scans, assets and estimations.
 - **provider**. Responsible for discovery and scan infrastructure setup for each provider. Contains logic to find assets and run scans on AWS, GCP, Azure, Docker and Kubernetes.
 - **uibackend**. Responsible for offloading the ui from data processing and filtering. Slightly coupled with ui. Composed by API model, backend client and server.
@@ -36,38 +36,151 @@ Each module will have its own go.mod file and each module will be versioned inde
 
 The scope of this RFC is not to change code logic but to change code structure. Therefore, the following table describes the path changes for each package impacted.
 
-| Module       | Current path                  | New path                      |
-| ------------ | ----------------------------- | ----------------------------- |
-| api          | pkg/apiserver                 | api/server                    |
-| api          | pkg/shared/backendclient      | api/client                    |
-| scanner      | pkg/cli                       | scanner/cli                   |
-| scanner      | pkg/shared/analyzer           | scanner/analyzer              |
-| scanner      | pkg/shared/config             | scanner/config                |
-| scanner      | pkg/shared/converter          | scanner/converter             |
-| scanner      | pkg/shared/families           | scanner/families              |
-| scanner      | pkg/shared/findingkey         | scanner/findingkey            |
-| scanner      | pkg/shared/job_manager        | scanner/jobmanager            |
-| scanner      | pkg/shared/scanner            | scanner/scanner               |
-| scanner      | pkg/shared/utils              | scanner/utils                 |
-| orchestrator | pkg/orchestrator              | orchestrator                  |
-| orchestrator | pkg/containerruntimediscovery | orchestrator/runtimediscovery |
-| uibackend    | pkg/uibackend                 | uibackend                     |
-| uibackend    | pkg/uibackend/rest            | uibackend/server              |
-| uibackend    | pkg/shared/uibackendclient    | uibackend/client              |
-| utils        | pkg/version                   | utils/version                 |
-| utils        | pkg/shared/command            | utils/command                 |
-| utils        | pkg/shared/fsutils            | utils/fsutils                 |
-| utils        | pkg/shared/log                | utils/log                     |
-| utils        | pkg/shared/manifest           | utils/manifest                |
-
-
-Furthermore, the provider could be removed from the orchestrator.
-
-| Module       | Current path                  | New path                      |
-| ------------ | ----------------------------- | ----------------------------- |
-| provider     | pkg/orchestrator/provider     | provider                      |
-
-The Dockerfiles for each package will be moved to the corresponding directory. Makefile, GitHub workflows and other files will need to be updated.
+| Module           | Current path                  | New path                         | Version Tag                             |
+| ---------------- | ----------------------------- | -------------------------------- | --------------------------------------- |
+| api              | api                           | api                              | api/v0.7.0                              |
+| api/server       | pkg/apiserver                 | api/server                       | api/server/v0.7.0                       |
+| api/client       | pkg/shared/backendclient      | api/client                       | api/client/v0.7.0                       |
+| cli              | pkg/cli                       | cli                              | cli/v0.7.0                              |
+| cli              | pkg/shared/analyzer           | cli/pkg/analyzer                 | cli/v0.7.0                              |
+| cli              | pkg/shared/config             | cli/pkg/config                   | cli/v0.7.0                              |
+| cli              | pkg/shared/converter          | cli/pkg/converter                | cli/v0.7.0                              |
+| cli              | pkg/shared/families           | cli/pkg/families                 | cli/v0.7.0                              |
+| cli              | pkg/shared/findingkey         | cli/pkg/findingkey               | cli/v0.7.0                              |
+| cli              | pkg/shared/job_manager        | cli/pkg/jobmanager               | cli/v0.7.0                              |
+| cli              | pkg/shared/scanner            | cli/pkg/scanner                  | cli/v0.7.0                              |
+| cli              | pkg/shared/utils              | cli/pkg/utils                    | cli/v0.7.0                              |
+| orchestrator     | pkg/orchestrator              | orchestrator                     | orchestrator/v0.7.0                     |
+| uibackend        | pkg/uibackend                 | uibackend                        | uibackend/v0.7.0                        |
+| uibackend/server | pkg/uibackend/rest            | uibackend/server                 | uibackend/server/v0.7.0                 |
+| uibackend/client | pkg/shared/uibackendclient    | uibackend/client                 | uibackend/client/v0.7.0                 |
+| utils            | pkg/version                   | utils/version                    | utils/v0.7.0                            |
+| utils            | pkg/shared/command            | utils/command                    | utils/v0.7.0                            |
+| utils            | pkg/shared/fsutils            | utils/fsutils                    | utils/v0.7.0                            |
+| utils            | pkg/shared/log                | utils/log                        | utils/v0.7.0                            |
+| utils            | pkg/shared/manifest           | utils/manifest                   | utils/v0.7.0                            |
+| provider         | pkg/orchestrator/provider     | provider                         | provider/v0.7.0                         |
+| provider         | pkg/containerruntimediscovery | provider/common/runtimediscovery | provider/common/runtimediscovery/v0.7.0 |
+
+To improve compliance with https://github.com/golang-standards/project-layout, the changes below are also proposed.
+
+| Module       | Current path                     | New path                      |
+| ------------ | -------------------------------- | ----------------------------- |
+| provider     | example_external_provider_plugin | provider/examples/external    |
+| cli          | scanner_boot_test                | cli/test/boot                 |
+
+Makefile, GitHub workflows and other files will need to be updated to point to the new paths.
+
+The VMClarity directory will have the following structure.
+
+```sh
+.
+├── Makefile
+├── api
+│   ├── client
+│   │   ├── client.cfg.yaml
+│   │   └── go.mod
+│   ├── go.mod
+│   ├── models
+│   │   └── models.cfg.yaml
+│   ├── openapi.yaml
+│   └── server
+│       ├── cmd
+│       ├── go.mod
+│       ├── pkg
+│       │   ├── common
+│       │   ├── database
+│       │   └── rest
+│       └── server.cfg.yaml
+├── cli
+│   ├── cmd
+│   ├── go.mod
+│   ├── pkg
+│   │   ├── analyzer
+│   │   ├── config
+│   │   ├── converter
+│   │   ├── families
+│   │   ├── findingkey
+│   │   ├── job_manager
+│   │   ├── presenter
+│   │   ├── scanner
+│   │   ├── state
+│   │   └── utils
+│   └── test
+│       └── boot
+├── docs
+├── e2e
+│   ├── config
+│   ├── go.mod
+│   └── testenv
+│       ├── docker
+│       ├── kubernetes
+│       ├── types
+│       └── utils
+├── img
+├── installation
+│   ├── aws
+│   ├── azure
+│   ├── docker
+│   ├── gcp
+│   └── kubernetes
+├── orchestrator
+│   ├── cmd
+│   ├── go.mod
+│   └── pkg
+│       ├── assetscanestimationwatcher
+│       ├── assetscanprocessor
+│       ├── assetscanwatcher
+│       ├── common
+│       ├── discovery
+│       ├── scanconfigwatcher
+│       ├── scanestimationwatcher
+│       └── scanwatcher
+├── provider
+│   ├── cmd
+│   ├── examples
+│   │   └── external
+│   ├── go.mod
+│   └── pkg
+│       ├── aws
+│       ├── azure
+│       ├── cloudinit
+│       ├── common
+│           └── containerruntimediscovery
+│               └── cmd
+│       ├── docker
+│       ├── external
+│       ├── gcp
+│       └── kubernetes
+├── rfc
+├── ui
+│   └── src
+├── uibackend
+│   ├── client
+│   │   └── client.cfg.yaml
+│   ├── go.mod
+│   ├── models
+│   │   └── models.cfg.yaml
+│   ├── openapi.yaml
+│   └── server
+│       ├── cmd
+│       ├── go.mod
+│       ├── pkg
+│       └── server.cfg.yaml
+└── utils
+    ├── command
+    ├── fsutils
+    ├── go.mod
+    ├── log
+    ├── manifest
+    └── version
+```
+
+# Release
+
+Each module will have a tag with the format `prefix/version` where prefix is the directory within the repository where the module is defined, more details [here](https://go.dev/wiki/Modules#publishing-a-release). For now, the same version will be used for each module even if there are no changes, this will simplify managing compatibility between modules across versions.
+
+The release GitHub workflow will be extended with an additional step to create a tag for each module. This step will only be performed after the release is successfully published.
 
 ## UX/UI
 

From b6872e9302929a9d3e2af2c10942ab6c8e432f0d Mon Sep 17 00:00:00 2001
From: Catarina Paralta <46568597+paralta@users.noreply.github.com>
Date: Fri, 19 Jan 2024 14:20:41 +0000
Subject: [PATCH 3/5] docs: multiple go modules rfc further review

---
 rfc/multiple-go-modules.md | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/rfc/multiple-go-modules.md b/rfc/multiple-go-modules.md
index 9970bd9ea..ec9f7ea8f 100644
--- a/rfc/multiple-go-modules.md
+++ b/rfc/multiple-go-modules.md
@@ -68,6 +68,7 @@ To improve compliance with https://github.com/golang-standards/project-layout, t
 | ------------ | -------------------------------- | ----------------------------- |
 | provider     | example_external_provider_plugin | provider/examples/external    |
 | cli          | scanner_boot_test                | cli/test/boot                 |
+|              | img                              | assets                        |
 
 Makefile, GitHub workflows and other files will need to be updated to point to the new paths.
 
@@ -101,7 +102,7 @@ The VMClarity directory will have the following structure.
 │   │   ├── converter
 │   │   ├── families
 │   │   ├── findingkey
-│   │   ├── job_manager
+│   │   ├── jobmanager
 │   │   ├── presenter
 │   │   ├── scanner
 │   │   ├── state
@@ -117,7 +118,7 @@ The VMClarity directory will have the following structure.
 │       ├── kubernetes
 │       ├── types
 │       └── utils
-├── img
+├── assets
 ├── installation
 │   ├── aws
 │   ├── azure

From 599ab4c079e978c411f0a2747d4d58eeac8ff06b Mon Sep 17 00:00:00 2001
From: Catarina Paralta <46568597+paralta@users.noreply.github.com>
Date: Tue, 23 Jan 2024 11:10:43 +0000
Subject: [PATCH 4/5] docs: multiple go modules rfc extend release instructions

---
 rfc/multiple-go-modules.md | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/rfc/multiple-go-modules.md b/rfc/multiple-go-modules.md
index ec9f7ea8f..ae04e52f5 100644
--- a/rfc/multiple-go-modules.md
+++ b/rfc/multiple-go-modules.md
@@ -181,7 +181,21 @@ The VMClarity directory will have the following structure.
 
 Each module will have a tag with the format `prefix/version` where prefix is the directory within the repository where the module is defined, more details [here](https://go.dev/wiki/Modules#publishing-a-release). For now, the same version will be used for each module even if there are no changes, this will simplify managing compatibility between modules across versions.
 
-The release GitHub workflow will be extended with an additional step to create a tag for each module. This step will only be performed after the release is successfully published.
+The release process will be updated to cope with tagging multiple modules. Not all steps will be automated with GitHub actions some will require new scripts in `Makefile` so a new `docs/release.md` file will be created with instructions on how to perform a release.
+
+Example of release instructions for version 0.7.0:
+
+1. [New Makefile Script] Create pull request with version bumps for all modules in repository. E.g. to bump the api module, the require section in `go.mod` files should be updated like
+    ```
+     - github.com/openclarity/vmclarity/api v0.0.0
+     + github.com/openclarity/vmclarity/api v0.7.0
+    ```
+2. [New Makefile Script] Create and push tags with v0.7.0 that points to the commit performed in the previous step. E.g. create tag for the api module with
+    ```
+    git tag -a api/v0.7.0
+    ```
+3. [GitHub Action] Current release process
+4. [Extend GitHub Action] Once a stable release is successfully published, each module will be tagged. E.g. the api module will be tagged with the tag created in step 2, `api/v0.7.0`.
 
 ## UX/UI
 

From cf7dd5701842840e93915cb991822dab0c71bfb0 Mon Sep 17 00:00:00 2001
From: Catarina Paralta <46568597+paralta@users.noreply.github.com>
Date: Tue, 23 Jan 2024 12:16:02 +0000
Subject: [PATCH 5/5] docs: multiple go modules rfc add test env module

---
 rfc/multiple-go-modules.md | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/rfc/multiple-go-modules.md b/rfc/multiple-go-modules.md
index ae04e52f5..fa50d76cd 100644
--- a/rfc/multiple-go-modules.md
+++ b/rfc/multiple-go-modules.md
@@ -61,6 +61,8 @@ The scope of this RFC is not to change code logic but to change code structure.
 | utils            | pkg/shared/manifest           | utils/manifest                   | utils/v0.7.0                            |
 | provider         | pkg/orchestrator/provider     | provider                         | provider/v0.7.0                         |
 | provider         | pkg/containerruntimediscovery | provider/common/runtimediscovery | provider/common/runtimediscovery/v0.7.0 |
+| e2e              | e2e                           | e2e                              | e2e/v0.7.0                              |
+| testenv          | e2e/testenv                   | testenv                          | testenv/v0.7.0                          |
 
 To improve compliance with https://github.com/golang-standards/project-layout, the changes below are also proposed.
 
@@ -112,12 +114,13 @@ The VMClarity directory will have the following structure.
 ├── docs
 ├── e2e
 │   ├── config
-│   ├── go.mod
-│   └── testenv
-│       ├── docker
-│       ├── kubernetes
-│       ├── types
-│       └── utils
+│   └── go.mod
+│── testenv
+│   ├── docker
+│   ├── kubernetes
+│   ├── types
+│   ├── utils
+│   └── go.mod
 ├── assets
 ├── installation
 │   ├── aws