Merge pull request #3279 from uselagoon/mkdocs

README.md

@@ -62,57 +62,54 @@ All the services that handle the API, authentication and external communication
   - [broker](https://github.com/uselagoon/lagoon/tree/main/services/broker) (the RabbitMQ message service used to communicate with Lagoon Remote)
   - [webhooks2tasks](https://github.com/uselagoon/lagoon/tree/main/services/webhooks2tasks) (the service that converts incoming webhooks to API updates)
   - [controllerhandler](https://github.com/uselagoon/lagoon/tree/main/services/controllerhandler) (the service that relays build progress from the controllers)
-  - [logs2s3](https://github.com/uselagoon/lagoon/tree/main/services/logs2s3) (the service that collects build logs and sends them to an S3-compatible back end)
+  - [actions-handler](https://github.com/uselagoon/lagoon/tree/main/services/actions-handler) (the service that to manage bulk action processing for builds and tasks)
 - Webhooks
   - [webhook-handler](https://github.com/uselagoon/lagoon/tree/main/services/webhook-handler) (the external service that Git Repositories and Registries communicate with)
   - [backup-handler](https://github.com/uselagoon/lagoon/tree/main/services/backup-handler) (the service used to collect and collate information on backups)
 - Notifications
-  - [logs2email](https://github.com/uselagoon/lagoon/tree/main/services/logs2email) (the service that pushes build notifications to a nominated email address)
-  - [logs2slack](https://github.com/uselagoon/lagoon/tree/main/services/logs2slack) (the service that pushes build notifications to a nominated Slack (or Discord) channel)
-  - [logs2rocketchat](https://github.com/uselagoon/lagoon/tree/main/services/logs2rocketchat) (the service that pushes build notifications to a nominated Rocket Chat channel)
-  - [logs2microsoftteams](https://github.com/uselagoon/lagoon/tree/main/services/logs2microsoftteams) (the service that pushes build notifications to a nominated Microsoft Teams channel)
-  - [logs2webhook](https://github.com/uselagoon/lagoon/tree/main/services/logs2webhook) (the service that pushes build notifications to a nominated webhook endpoint)
-- Other Services
-  - [ui](https://github.com/uselagoon/lagoon/tree/main/services/ui) (the main user interface and dashboard for Lagoon)
-  - [lagoon-cli](https://github.com/uselagoon/lagoon-cli) (the command-line interface for managing sites on Lagoon)
-  - [lagoon-sync](https://github.com/uselagoon/lagoon-sync) (a command-line interface for syncing databases or file assets between Lagoon environments)
-  - [drush-alias](https://github.com/uselagoon/lagoon/tree/main/services/drush-alias) (provides Drupal developers with an automated alias service for Drush)
+  - [logs2notifications](https://github.com/uselagoon/lagoon/tree/main/services/logs2notifications) (the service that pushes build notifications to a configured notification types)
 
 ### Lagoon Remote
 
 All the services that are used to provision, deploy and maintain sites hosted by Lagoon on Kubernetes live here. These services are mostly comprised of third-party tools, developed external to Lagoon itself. Installation is via a [Helm Chart](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-remote)
 
 - [Remote Controller](https://github.com/uselagoon/remote-controller) (the controllers that handle building and deploying sites onto Lagoon)
-- [build-deploy-image](https://github.com/uselagoon/build-deploy-tool) (the service that computes which services, configuration and settings to provision for Kubernetes)
+- [build-deploy-tool](https://github.com/uselagoon/build-deploy-tool) (the service that computes which services, configuration and settings to provision for Kubernetes)
 - [docker-host](https://github.com/uselagoon/lagoon/tree/main/images/docker-host) (the service that stores and caches upstream docker images for use in builds)
-- [lagoon-idler](https://github.com/amazeeio/lagoon-idler) (an optional operator that can idle non-production sites not currently in use to conserve resources)
+- [Aergia](https://github.com/amazeeio/aergia-controller) (an optional controller that can idle non-production sites not currently in use to conserve resources)
 - [Dioscuri](https://github.com/amazeeio/dioscuri) (an optional operator that provides Active/Standby functionality to Lagoon)
 - [dbaas-operator](https://github.com/amazeeio/dbaas-operator) (an optional operator that provisions databases from an underlying managed database)
 
+### Lagoon UI
+  - [ui](https://github.com/uselagoon/lagoon-ui) (the main user interface and dashboard for Lagoon, usually installed in lagoon-core, but can also be installed anywhere as a Lagoon project)
+
+### Lagoon Tools
+  - [lagoon-cli](https://github.com/uselagoon/lagoon-cli) (the command-line interface for managing sites on Lagoon)
+  - [lagoon-sync](https://github.com/uselagoon/lagoon-sync) (a command-line interface for syncing databases or file assets between Lagoon environments)
+  - [drush-alias](https://github.com/uselagoon/lagoon/tree/main/services/drush-alias) (provides Drupal developers with an automated alias service for Drush)
+
 ### Additional Services
 
 These services are usually installed alongside either Lagoon Core or Lagoon Remote to provide additional functionality to Lagoon.
 
 - Registry (required)
   - [Harbor](https://goharbor.io/) (provides image registry services to Lagoon projects)
-  - [Trivy](https://github.com/aquasecurity/trivy) (scans images for vulnerability, and can report to Lagoon)
+- Backups (optional)
+    - [k8up](https://k8up.io/) (provides a scheduled backup and prune service to environment namespaces)
 - Lagoon Logging (optional)
   - [lagoon-logging](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-logging) (utilizes [banzaicloud/logging-operator](https://github.com/banzaicloud/logging-operator) to collect and augment container&router logs from all sites, and sends them to a logs-dispatcher)
   - [logs-dispatcher](https://github.com/uselagoon/lagoon/tree/main/services/logs-dispatcher) (collects application logs from sites, as well as container&router logs from lagoon-logging, enriches them with additional metadata and sends them to a central log concentrator)
   - [lagoon-logs-concentrator](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-logs-concentrator) (collects logs from remote logs-dispatchers and sends them to Elasticsearch)
-- Open Policy Agent (optional)
-
-  - [lagoon-gatekeeper](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-gatekeeper) (centralized policy library for Lagoon)
 - Elasticsearch or Opensearch (optional)
-  - [Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/) (provides centralized log storage, search and analysis)
+  - [OpenSearch](https://opensearch.org/docs/latest/opensearch/index//) (provides centralized log storage, search and analysis)
   - [Kibana](https://opendistro.github.io/for-elasticsearch-docs/docs/kibana/) (the default user interface for Elasticsearch searching and visualization)
 - Managed databases, for use with DBaaS operator (optional)
   - MariaDB (self managed or via [Amazon RDS for MariaDB](https://aws.amazon.com/rds/mariadb/), [Azure Database for MariaDB](https://azure.microsoft.com/en-au/services/mariadb/)
   - MySQL (self managed or via [Amazon RDS for MySQL](https://aws.amazon.com/rds/mysql/), [Amazon Aurora MySQL](https://aws.amazon.com/rds/aurora/mysql-features/), [Azure Database for MySQL](https://azure.microsoft.com/en-au/services/mysql), [Cloud SQL for MySQL](https://cloud.google.com/sql/docs/mysql))
   - PostgreSQL (self managed or via [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/), [Amazon Aurora PostgreSQL](https://aws.amazon.com/rds/aurora/postgresql-features/), [Azure Database for PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql), [Cloud SQL for PostgreSQL](https://cloud.google.com/sql/docs/postgres) )
   - MongoDB (self managed, or via [Amazon DocumentDB](https://aws.amazon.com/documentdb/), [Azure Cosmos DB](https://azure.microsoft.com/en-au/services/cosmos-db/) )
-- Backups (optional)
-  - [k8up](https://k8up.io/) (provides a scheduled backup and prune service to environment namespaces)
+- Open Policy Agent (optional)
+  - [lagoon-gatekeeper](https://github.com/uselagoon/lagoon-charts/tree/main/charts/lagoon-gatekeeper) (centralized policy library for Lagoon)
 
 
 ## Testing

docs/applications/index.md

@@ -5,12 +5,14 @@ description: Applications that can be run on Lagoon
 ### A wide range of Applications, Frameworks and Languages are supported by Lagoon
 
 Lagoon broadly classifies three levels in the application stack:
+
 - **Languages** - The core building blocks of any Lagoon project, these are usually provided by Lagoon-specific images.
 - **Frameworks** - These take those base images, and add in the necessary logic, tools and packages needed to serve a website, or drive an application.
 - **Applications** - Usually built on top of Frameworks, this is the layer that content editors or developers will interact with to shape the finished product.
 
 
 When we reference any repositories for use on Lagoon, we usually refer to them in three ways:
+
 - **Templates** - These are fully-functional, cloneable starter repositories, maintained and updated regularly, ready to be extended and used with little customization.
 - **Examples** - These are fully functional repositories, maintained and updated regularly, but may require some effort to make work for your individual project.
 - **Demos** - These are repositories that have been built as a demonstration, and are usable for some of the concepts within, but aren't routinely maintained or updated.

docs/applications/node.md

@@ -4,4 +4,4 @@
 
 Lagoon provides [Node.js images](https://github.com/uselagoon/lagoon-images/tree/main/images/node) that are based on top of [the official Node Alpine images](https://hub.docker.com/_/node/).
 
-More information on how to adapt your project to run on Lagoon can be found in our [Node.js Docker Images](/docker-images/nodejs) section.
+More information on how to adapt your project to run on Lagoon can be found in our [Node.js Docker Images](../../docker-images/nodejs) section.

docs/applications/php.md

@@ -2,7 +2,7 @@
 
 ## Introduction
 
-Lagoon supports a wide-range of PHP-based applications, such as [Drupal](/drupal), Laravel, [Wordpress](/applications/wordpress), Magento and Symfony.
+Lagoon supports a wide-range of PHP-based applications, such as [Drupal](../../drupal), Laravel, [Wordpress](../wordpress), Magento and Symfony.
 
 
-More information on how to adapt your php project to run on Lagoon can be found in our [PHP-cli Docker Images](/docker-images/php-cli) and [PHP-FPM Docker Images](/docker-images/php-fpm) sections.
+More information on how to adapt your php project to run on Lagoon can be found in our [PHP-cli Docker Images](../../docker-images/php-cli) and [PHP-FPM Docker Images](../../docker-images/php-fpm) sections.

docs/applications/python.md

@@ -4,4 +4,4 @@
 
 Lagoon provides images for [Python 3.7 and above](https://github.com/uselagoon/lagoon-images/tree/main/images/python) that can be used to build web apps in a wide range of Python-based frameworks and applications.
 
-More information on how to adapt your Python project to run on Lagoon can be found in our [Python Docker Images](/docker-images/python) section.
+More information on how to adapt your Python project to run on Lagoon can be found in our [Python Docker Images](../../docker-images/python) section.

docs/contributing-to-lagoon/developing-lagoon.md

@@ -210,63 +210,6 @@ This can happen if your local resolver filters private IPs from results. You can
 
 Here are some development scenarios and useful workflows for getting things done.
 
-### Editing `kubectl-build-deploy-dind`
-
-This example shows a workflow for editing the Lagoon deploy logic.
-
-### Edit `kubectl-build-deploy-dind`
-
-In this example we want to add some functionality to the Lagoon deploy logic in the `kubectl-build-deploy-dind` image.
-
-1. Start a local KinD cluster with Lagoon installed from locally built images, and smoke-test it by running a single test suite:
-
-```bash
-make -j8 kind/test TESTS='[features-api-variables]'
-```
-
-1. Edit `images/kubectl-build-deploy-dind/build-deploy-docker-compose.sh`.
-
-```diff
---- a/images/kubectl-build-deploy-dind/build-deploy-docker-compose.sh
-+++ b/images/kubectl-build-deploy-dind/build-deploy-docker-compose.sh
-@@ -1,5 +1,7 @@
- #!/bin/bash
-
-+echo HELLO WORLD
-+
- function cronScheduleMoreOftenThan30Minutes() {
-   #takes a unexpanded cron schedule, returns 0 if it's more often that 30 minutes
-   MINUTE=$(echo $1 | (read -a ARRAY; echo ${ARRAY[0]}) )
-```
-
-1. Now rebuild the `kubectl-build-deploy-dind` image with the edits included.
-
-```bash
-rm build/kubectl-build-deploy-dind
-make -j8 build/kubectl-build-deploy-dind
-```
-
-1. Push the newly built image into the cluster registry. It will now be used for future deploys.
-
-```bash
-make kind/push-images IMAGES=kubectl-build-deploy-dind
-```
-
-1. Rerun the tests.
-
-```bash
-make kind/retest TESTS='[features-api-variables]'
-```
-
-1. See the edits have been applied.
-
-```bash
-$ kubectl -n ci-features-api-variables-control-k8s-lagoon-api-variables logs lagoon-build-lat2b | grep -A2 build-deploy-docker-compose.sh
-+ . /kubectl-build-deploy/build-deploy-docker-compose.sh
-++ echo HELLO WORLD
-HELLO WORLD
-```
-
 #### Add tests
 
 1. Repeat the first step above.

docs/installing-lagoon/requirements.md

@@ -44,7 +44,7 @@ Lagoon utilises a default 'standard' StorageClass for most workloads, and the in
 
 Lagoon also requires a storageClass called 'bulk' to be available to support persistant pod replicas (across nodes). This storageClass should support ReadWriteMany access mode and should be configured to be dynamic provisioning and expandable where possible. See https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes for more information.
 
-We have curently only included the instructions for (the now deprecated) [EFS Provisioner](./efs-provisioner.md), but most CSI drivers should also work, as will configurations with an NFS-compatible server and provisioner.
+We have curently only included the instructions for (the now deprecated) [EFS Provisioner](./efs-provisioner.md) - as the Production [EFS CSI driver](https://github.com/kubernetes-sigs/aws-efs-csi-driver) has issues with provisioning more than 120 PVCs that we are awaiting upstream possible fixes for - [here](https://github.com/kubernetes-sigs/aws-efs-csi-driver/pull/761) and [here](https://github.com/kubernetes-sigs/aws-efs-csi-driver/pull/732) - but most other providers CSI drivers should also work, as will configurations with an NFS-compatible server and provisioner.
 
 ## How much Kubernetes experience/knowledge is required?
 

docs/using-lagoon-advanced/environment-idling.md

@@ -1,8 +1,8 @@
-# Environment Idling
+# Environment Idling (optional)
 
 ## What is the Environment Idler?
 
-Lagoon automatically idles environments if they have been unused for a couple of hours. This is done in order to reduce the load on the Kubernetes clusters and improve the overall performance of production environments and development environments that are actually in use.
+Lagoon can utilise the [Aergia controller](https://github.com/amazeeio/aergia-controller), (installed in the lagoon-remote) to automatically idle environments if they have been unused for a defined period of time. This is done in order to reduce the load on the Kubernetes clusters and improve the overall performance of production environments and development environments that are actually in use.
 
 ### How does an environment get idled?
 
@@ -16,13 +16,13 @@ The Environment Idler has many different configuration capabilities. Here are th
 
 ### How does an environment get un-idled?
 
-The Lagoon Idler will automatically un-idle an environment as soon as it is visited, therefore just visiting any URL of the environment will start the environment.
+Aergia will automatically un-idle an environment as soon as it is visited, therefore just visiting any URL of the environment will start the environment. Likewise, initiating an SSH session to the environment will also restart the services.
 
 The un-idling will take a couple of seconds, as the Kubernetes cluster needs to start all containers again. During this time there will be waiting screen shown to the visitor that their environment is currently started.
 
 ### **Can I disable / prevent the Idler from idling my environment?**
 
-Yes, there is a field `autoIdle` on the project \(impacts all environments\) and environment \(if you need to target just 1 environment\), as to whether idling is allowed to take place. A value of `1` indicates the project/environment is eligible for idling. If the project is set to `0` the environments will never be idled, even if the environment is set to `0`  
+Yes, there is a field `autoIdle` on the project \(impacts all environments\) and environment \(if you need to target just 1 environment\), as to whether idling is allowed to take place. A value of `1` indicates the project/environment is eligible for idling. If the project is set to `0` the environments will never be idled, even if the environment is set to `0`
 The default is always `1`\(idling is enabled\).
 
 Talk to your Lagoon administrator if you are unsure how to set these project/environment fields.