diff --git a/README.md b/README.md index 834430c5..03bcaf3d 100644 --- a/README.md +++ b/README.md @@ -46,25 +46,27 @@ Development Environments for embedded software development. Contributors and early adopters are welcome!

-## Concept -A Development Environment is a set of tools used for a development project (e.g. build system, -debugger, toolchain, etc.). The tools are separately packed into container images, which are stored -in registries. +## The Concept in a Nutshell +A set of software tools used for a specific development project is called a Development Environment. +These tools for example can be the build system, debugger, test framework, etc... -Each Development Environment has a descriptor. A descriptor indicates which tools are -required in the project and the place their container images are stored. All descriptors are stored -in the Development Environment Catalogs. The users can browse these catalogs, and list the available -environments and their statuses. -A Development Environment can be installed from its catalog by a simple dem command, which means -that the required tool images get downloaded from the registries. +The idea is to pack the tools separately into container images, which are then can be stored in +registries. -The registries, the catalogs, and the development infrastructure form a Development Platform. +Each Development Environment has a descriptor. A descriptor, like a blueprint, indicates which tools +are required in the project, and the place their container images are stored. -The purpose of DEM is the easy management of this platform. The users can install available -Development Environments or freely create their own from the tool images available in the registries -or locally. +![Dev Env descriptor](/docs/wp-content/dev_env_descriptor.png) -![platform](docs/wp-content/platform.png) +The descriptors can be stored in the Development Environment Catalogs. The users can browse these +catalogs, and download a copy of the Development Environment descriptor to their local catalog. + +![Catalogs](/docs/wp-content/dem_catalogs.png) + +The users can install a Development Environments from their local catalog or freely create their own +based on the tools available in the registries or on their local system. + +![Dev Env installation](/docs/wp-content/dev_env_installation.png) ## Key features @@ -145,11 +147,10 @@ Now you have the Dev Env descriptor in your local catalog, but you might want to - Change the tool image for a given tool. - Set the host where the image should be placed. -You can edit it the Dev Env with: +You can edit it with: dem modify DEV_ENV_NAME - Finally, if you are ready to use it, you can initialize it with: dem install DEV_ENV_NAME diff --git a/docs/basics.md b/docs/basics.md new file mode 100644 index 00000000..2ad6e985 --- /dev/null +++ b/docs/basics.md @@ -0,0 +1,69 @@ +--- +title: The Basics +--- + +In this chapter we will cover the basic concepts of the Development Environment Manager (DEM). + +## What is a Development Environment? +A set of software tools used for a specific development project is called a **Development +Environment**. These tools for example can be the build system, debugger, test framework, etc... + +## A Container Image +To put it simply, a container image is a set of software components alongside its dependencies, +which can be run in a container. + +## A Tool Container Image +The idea is to build the tools from a Development Environment into their own respective images, so +they can run isolatedly. + +## Development Environment Descriptor +Each Development Environment has a descriptor. A descriptor, like a blueprint, indicates which tools +are required in the project, and the place their container images are stored. + +![Dev Env descriptor](wp-content/dev_env_descriptor.png) + +## Container Engine +The container engine is responsible for running the container images. The DEM uses the Docker +Container Engine to run the tool images. + +## Registry and Repository +A registry serves as a storage for the tool images, where they can be kept without occupying space on +the developer's computer. This storage enables convenient sharing of images with others, ensuring +uniform tool usage among all collaborators on the same project. +When an image is uploaded to a registry, it initiates the creation of a repository. This repository +is responsible for keeping track of the various versions of the image. + +!!! Note + + An image **repository** stores the different versions of the same image. + An image **registry** is a collection of image repositories. + +The DEM also uses registries in the background to store the tool images. To list the currently +available registries use the `dem list-reg` command. The `dem add-reg` and `dem del-reg` commands +can be used to add or delete registries. + +!!! Note + + The DEM supports the [Docker Hub](https://docs.docker.com/docker-hub/) and + [Docker Registry](https://docs.docker.com/registry/) + + If you'd like to request support for other registry types, please create a + [new descussion](https://github.com/axem-solutions/dem/discussions/categories/regsitry). + +## Development Environment Catalogs +A catalog is a collection of Development Environment descriptors. +The DEM can handle multiple catalogs. To list the currently available ones use the `dem list-cat` +command. The `dem add-cat` and `dem del-cat` commands can be used to add or delete catalogs. + +!!! Note + + axem has its own catalog, which is available by default. + +The users can browse these catalogs, and download a copy of the Development Environment descriptor +to their local catalog. + +![Catalogs](wp-content/dem_catalogs.png) + +## Development Platform +The registries, the catalogs, and the whole development infrastructure are part of the Development +Platform. \ No newline at end of file diff --git a/docs/commands.md b/docs/commands.md index 9ce6ace9..f3add486 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -38,7 +38,7 @@ Arguments: ## **`dem create DEV_ENV_NAME`** -Create a new Development Environment. +Create a new Development Environment descriptor and save it to the local descriptor storage (catalog). Running this command will open up an interactive UI on the command line. Follow the steps below to configure the new Environment. @@ -55,6 +55,8 @@ Select the required tool image and press :material-keyboard-return:. ![image select](wp-content/image_select.png) +:info: After creation, the Development Environment can be installed with the `install` command. + Arguments: `DEV_ENV_NAME` Name of the Development Environment to create. [required] @@ -65,11 +67,12 @@ Arguments: Clone a Development Environment descriptor from the catalogs. -Only the Development Environment descriptor gets cloned, the required tool images are not pulled. To -pull the required tool images, use the `install` command. +Only the Development Environment descriptor will be cloned, the required tool images won't be pulled. If a Development Environment with the same name has been already available on the host PC, the user will be asked if they want to overwrite it or not. +:info: After cloning, the Development Environment can be installed with the `install` command. + Arguments: `DEV_ENV_NAME` Name of the Development Environment, whose descriptor to clone. [required] @@ -139,10 +142,10 @@ Arguments: ## **`dem install DEV_ENV_NAME`** -Install the selected Development Environment. DEM pull all the required containerized tools (which +Install the selected Development Environment. DEM pulls all the required containerized tools (which are not yet available on the host PC) from the registry and install the Development Environment locally. If the same Development Environment is already installed, but the installation is not -complete, the missing tool images are obtained from the registry. +complete, the missing tool images get obtained from the registry. Arguments: @@ -152,9 +155,9 @@ Arguments: ## **`dem uninstall DEV_ENV_NAME`** -Uninstall the selected Development Environment. Set installed flag to False if it was True. Dem checks whether a tool image is -required or not by any of the remaining installed local Development Environments. In case the tool image is -not required anymore, the dem delete it. +Uninstall the selected Development Environment. Sets the installed flag to False. DEM checks whether +a tool image is required or not by any of the remaining installed local Development Environments. In +case the tool image is not required anymore, the DEM tries to delete it. Arguments: @@ -169,7 +172,7 @@ Assign a Development Environment to a project. If the project already has a Development Environment assigned, the user will be asked if they want to overwrite it or not. -Projects that have a Development Environment assigned can initialized with the `init` command. +Projects that have a Development Environment assigned, can be initialized with the `init` command. Arguments: @@ -184,6 +187,9 @@ working directory will be used. Initialize a project with the assigned Development Environment. +:info: After the initialization, the Development Environment can be installed with the `install` +command. + Arguments: `PROJECT_PATH` Path of the project to initialize. If not set, the current working directory will be @@ -252,6 +258,8 @@ Imports a Development Environment. The file to import only contains the Development Environment descriptor. For a successful import the DEM needs access to all the registries where the required images are stored. +:info: After the import, the Development Environment can be installed with the `install` command. + Arguments: `PATH_TO_DEV_ENV` Path of the JSON file to import. Can be an absolute path or a relative path to the @@ -347,30 +355,6 @@ List the available hosts from the config file. --- -## **`dem install DEV_ENV_NAME`** - -Install the selected Development Environment. Set installed flag to True. Dem checks which tool image is -required by the selected local Development Environments and in case the tool image is -not installed, the dem installs it. - -Arguments: - -`DEV_ENV_NAME` Name of the Development Environment to install. [required] - ---- - -## **`dem uninstall DEV_ENV_NAME`** - -Uninstall the selected Development Environment. Set installed flag to False if it was True. Dem checks whether a tool image is -required or not by any of the remaining installed local Development Environments. In case the tool image is -not required anymore, the dem delete it. - -Arguments: - -`DEV_ENV_NAME` Name of the Development Environment to uninstall. [required] - ---- - ## **`dem del-host NAME`** Delete a host from the config file. diff --git a/docs/design.md b/docs/design.md index 8a7f6fdc..56b46098 100644 --- a/docs/design.md +++ b/docs/design.md @@ -1,10 +1,10 @@ --- -title: Design +title: Design (for DEM developers) --- ## Architectural Overview -This chapter contains the technical documentation for the design. +This chapter contains the technical documentation of the design. The Development Environment Manager (DEM) provides an abstraction layer for the Development Platform, which encompasses a collection of available resources, such as registries, catalogs, diff --git a/docs/development_environments.md b/docs/development_environments.md deleted file mode 100644 index e9bdb387..00000000 --- a/docs/development_environments.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -title: Development Environments ---- - -## What is a Development Environment? -A set of software tools used for a development project is called a **Development Environment**. - -## Tool types -Currently, the following tool types are supported: - - - Build system - - Toolchain - - Debugger - - Deployer - - Test framework - - CI/CD server - -When creating a new Development Environment the user can select any of the above types and then -assign the required tools to them. - -!!! warning - - In the v0.3.0 we are going to change this behavior and let the users to define their own tool - types. - -## Tool images -To put it simply, a container image is a set of software components alongside its dependencies, -which can be run in a container. The idea is to build the tools from a Development Environment into -their own images, so they can run isolatedly. - -## Registry - -A registry can serve as a storage for tool images, where they can be kept without occupying space on -the developer's computer. This storage enables convenient sharing of images with others, ensuring -uniform tool usage among all collaborators on the same project. -When an image is uploaded to a registry, it initiates the creation of a repository. This repository -is responsible for keeping track of the various versions of the image. - -!!! Note - - An image repository stores the different versions of the same image. - An image registry is a collection of image repositories. - -The DEM also uses registries in the background to store the tool images. To list the currently -available registries use the `dem list-reg` command. The `dem add-reg` and `dem del-reg` commands -can be used to add or delete registries. - -!!! Note - - The DEM supports the [Docker Hub](https://docs.docker.com/docker-hub/) and - [Docker Registry](https://docs.docker.com/registry/) - - If you'd like to request support for other registry types, please create a - [new descussion](https://github.com/axem-solutions/dem/discussions/categories/regsitry). - -## Development Environment Catalogs -A catalog is a collection of Development Environment descriptors available to install. A descriptor -stores the location of the required tool images. The DEM can handle multiple catalogs. To list the -currently available ones use the `dem list-cat` command. The `dem add-cat` and `dem del-cat` -commands can be used to add or delete catalogs. - -!!! Note - - axem has its own catalog, which is by default available for DEM. - -## Development Platform -The registries, the catalogs, and the development infrastructure make up a Development Platform. -The users can install new Development Environments from the catalogs in a self-service manner. - -![platform](wp-content/platform.png){: .center} - -## Getting a Development Environment -There are three ways to get a Development Environment: - -- install one from a Catalog -- create a new one locally -- load an exported Development Environment descriptor - -### **Installing from a catalog** -An organization can create a Development Environment Catalog and share it with its members. This -**guarantees** that each and every member uses the **same toolset** in the **same environment**. - -![organization](wp-content/organization.png){: .center} - -The members can list the available Development Environments in the organization's catalog with the -following command: -`dem list --all --env` - -Use the `dem pull` command to install the selected Development Environment. - -1. First the DEM installs the Development Environment descriptor. -2. Then downloads the necessary tool images, which are not yet available on the host PC. - -See the [`dem pull`](commands.md#dem-pull-dev_env_name) command for more details. - -### **Creating a new Development Environment locally** -DEM provides a TUI for creating a new Development Environment that can be started with the -`dem create` command. - -See the [`dem create`](commands.md#dem-create-dev_env_name) command for more details. - -### **Load a Development Environment** -A locally already available Development Environment can be exported in JSON format with the -`dem export` command. The exported JSON describes the required tool images and the location of the -registries where they are stored. This file can be shared and on another PC can be imported with the -`dem load` command. After the DEM loaded the descriptor, it will pull the required tool images. - -See the [`dem export`](commands.md#dem-export-dev_env_name) and -[`dem load`](commands.md#dem-load-dev_env_name) commands for more details. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index a3ec38e3..e395b860 100644 --- a/docs/index.md +++ b/docs/index.md @@ -15,18 +15,12 @@ containerized Development Environments.** Learn by doing! Try our [tutorial](https://www.axemsolutions.io/tutorial/index.html) with a simple embedded project! -## Developing Embedded Software - -Writing software for other architectures than our host usually requires a lot of different tools. -The software tools used for a specific project can be grouped and they form a -**Development Environment**. To make it simpler to work with these tools, they are typically bundled -together into an IDE (Integrated Development Environment). - ## Concept ### :unlock: Loose coupling between tools -One of the biggest disadvantages of IDEs is that sometimes it is hard to use the underlying -integrated tools separately. They might depend on the IDE itself or each other, so standalone usage can be difficult and the whole IDE installation is required. +One of the biggest disadvantage of traditional IDEs is that sometimes it is hard to use the +underlying integrated tools separately. They might depend on the IDE itself or each other, so +standalone usage can be difficult and the whole IDE installation might be required. DEM's goal is to **reduce these dependencies** and provide the possibility of standalone usage. @@ -47,9 +41,8 @@ lifecycle. DEM makes the change easy by providing a way to **quickly swap tool i - to use a different version of the same tool - to use a completely different tool -Containerization ensures the **safe coexistence** of the same tools with different version numbers. -Adding a new tool is as simple as to change one, making the Development Environment very -**scalable**. +Containerization ensures the **safe coexistence** of the different versions of the same tools. +Adding a new tool is as simple as changing one, making the Development Environment **scalable**. ### :material-share: Reliable Development Environment sharing To **create software predictively and effectively**, it is crucial to have a **consistent** @@ -58,8 +51,9 @@ With DEM, you can **easily share** the same environment with every coworker. ### :rocket: Quick and reproducible setup Before starting to work on a project, setting up the required tools can be a time-consuming task. -Using DEM to **install a new Development Environment** is a **single command**. +By using DEM, the **installation of a new Development Environment** becomes very **quick** and +**simple**. If some modifications must be added to an old project no one worked on for a while, installing the -required toolset can be an exhausting task. With DEM the Development Environments can be stored in a -Catalog and can be reinstalled whenever needed. \ No newline at end of file +required toolset can be an exhausting task. With DEM the Development Environment can be assigned to +a project, so it can be **reinstalled whenever needed**. \ No newline at end of file diff --git a/docs/installation.md b/docs/installation.md index 07af93a8..d7a2ec5c 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -11,8 +11,9 @@ To use the DEM on your PC, you need to have the following tools installed: ## Installation -Use the following install script to get the latest version of DEM: - curl -fsSL 'https://raw.githubusercontent.com/axem-solutions/dem/main/install-dem.sh' | bash +Use the following install script to get the latest version of DEM: + + `curl -fsSL 'https://raw.githubusercontent.com/axem-solutions/dem/main/install-dem.sh' | bash` ### Alternative installation @@ -29,12 +30,12 @@ After installation, you can enable the autocompletion for bash and zsh shells dem --install-completion +> If the command didn't work, supply your shell type as input parameter (bash or zsh) > Note for zsh users: `compinit` must be called from your .zshrc. -## Optional: Use the source code +## Optional: Use the source code (for DEM developers) -The dem is [open source](https://github.com/axem-solutions/dem), so you can use it as a python -module. To do this, you need to add the `-m` flag to your command. +The DEM can be run as a Python module. To do this, you need to add the `-m` flag to your command. For example: @@ -49,4 +50,8 @@ First install the environment with required dependencies: Enter the virtual environment: - poetry shell \ No newline at end of file + poetry shell + +Inside the virtual environment, you can run the DEM the same way as it was an installed package: + + dem list --local --env \ No newline at end of file diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 00000000..ea778dda --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,60 @@ +--- +title: Quickstart +--- + +Now that you have the DEM installed, you might find yourself in one of the following scenarios: + +## I'd like to start with a template... + +Here at axem we'd like to create a template for every target out there. List the currently available +ones with: + + dem list --all --env + +You can clone the selected template with: + + dem clone DEV_ENV_NAME + + +## I'd like to work on a project already configured with DEM... + +In this case you only need to initialize the Dev Env with: + + dem init + +## I'd like to use a Dev Env someone shared with me... + +You can import a Dev Env descriptor JSON with: + + dem load DEV_ENV_NAME + +## I'd like to create my own Dev Env from scratch... + +Create a brand new Dev Env with the following command: + + dem create DEV_ENV_NAME + +!!! info + + We believe that a project's dependencies should be stored in the project's repository. This way, + every developer can use the same Development Environment. A Dev Env can be assigned to the + project with the `assign` command. + +At this point you have the Development Environment's blueprint - its descriptor - in your local +catalog, which you can modify to your needs. You might want to: + +- Add/remove tools. +- Change the tool image for a given tool. + +You can edit it with the Development Environment settings view: + + dem modify DEV_ENV_NAME + +Finally, if you are ready to use it, you can install it with: + + dem install DEV_ENV_NAME + +:tada: You are ready to start working with your Development Environment! + +>For more detailed instructions about the commands please refer to the +[Commands chapter](commands.md). \ No newline at end of file diff --git a/docs/wp-content/dem_catalogs.png b/docs/wp-content/dem_catalogs.png new file mode 100644 index 00000000..5e3afb17 Binary files /dev/null and b/docs/wp-content/dem_catalogs.png differ diff --git a/docs/wp-content/dev_env_descriptor.png b/docs/wp-content/dev_env_descriptor.png new file mode 100644 index 00000000..9e893548 Binary files /dev/null and b/docs/wp-content/dev_env_descriptor.png differ diff --git a/docs/wp-content/dev_env_installation.png b/docs/wp-content/dev_env_installation.png new file mode 100644 index 00000000..a963818d Binary files /dev/null and b/docs/wp-content/dev_env_installation.png differ diff --git a/docs/wp-content/platform.png b/docs/wp-content/platform.png deleted file mode 100644 index 1465f41e..00000000 Binary files a/docs/wp-content/platform.png and /dev/null differ diff --git a/mkdocs.yml b/mkdocs.yml index 865e9bdd..ac2ada90 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -19,6 +19,7 @@ theme: - navigation.instant - navigation.tracking - navigation.footer + - content.code.copy icon: repo: fontawesome/brands/github logo: wp-content/uploads/2023/03/x_w.png @@ -54,6 +55,8 @@ extra: analytics: provider: google property: G-MCWQMMXY3W + version: + provider: mike copyright: > Copyright 2023 - axem solutions - All rights reserved | @@ -78,6 +81,7 @@ markdown_extensions: nav: - 'index.md' - 'installation.md' - - 'development_environments.md' + - 'basics.md' + - 'quickstart.md' - 'commands.md' - 'design.md' \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index f99e9240..08480b9a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -39,6 +39,8 @@ optional = true [tool.poetry.group.docs.dependencies] mkdocs-material = "^9.1.7" +mike = "^2.0.0" +jinja2 = "^3.1.3" [tool.poetry.group.test]