Docs Release 2024-05-005

CHANGELOG.md

@@ -4,6 +4,84 @@ This is the Quix Cloud changelog for the current year.
 
 See [here](https://github.com/quixio/quix-docs/wiki/Docs-Releases) for the docs changelog.
 
+## 2024-05-01-cli-beta | 15 MAY 2024
+
+`NEW FEATURES`
+
+- CLI (Beta)
+    - New CLI commands
+        - `quix local pipeline up/down`:  Added the ability to run Quix pipelines locally using docker.
+        - `quix local pipeline update`:  New command to update the local pipeline with the existing applications of the project
+        - `quix local pipeline sync`:  New command to synchronize the local pipeline with Quix Cloud
+        - `quix local pipeline view`:  New command to render an image of the local pipeline
+        - `quix local ide`:  Sets up everything needed for local development and opens your local IDE
+        - `quix update`:  New command to update the version of the CLI to the latest stable version available
+    - Improvements on CLI commands
+        - `quix deployments`: Improved interactivity in all the deployments commands.
+        - Improved general user experience on all the commands using input texts and lists
+    - Deprecated commands
+        - `quix local deploy`: This command is being deprecated in favor of the new local pipeline commands. You will find same functionalities in `quix local pipeline update` and `quix local pipeline sync`.
+
+- Environments
+    - Improved environment empty states to introduce basic CLI setup commands
+    - Implemented *Code* button that allows you to
+        - Clone your repository for local development
+        - Install the Quix CLI
+        - Download the code of the branch as zip
+- Organisations
+    - New organisation overview page that allows users that are *Admins* within their organisation to see relevant organisation details.
+
+`IMPROVEMENTS`
+
+- Deployments
+    - Added the environment variable `Quix_Deployment_ReplicaName` to allow the code behave differently depending of the Replica of the deployment.
+- IDE
+    - You will now receive status updates when uploading the file such as: uploading, successful upload, and failed uploads
+- Environments
+    - Improved outdated deployments warning header to make it less UX intrusive
+- YAML
+    - Introduced pagination for commit history to improve performance
+    - When there is an error on the *sync* process, we now scroll to the relevant step in the sync process that is resulting in the error, for better visibility
+    - `DesiredStatus` property is only tracked now if it is present in the YAML.
+    - Synchronize environment automatically when we detect a new commit in the YAML that would produce 0 changes in the pipeline.
+- Topics
+    - Topic configuration is no longer mandatory in the UI and in the YAML. Now when you create a topic without user defined configuration, it will use the default configuration of the broker type.
+    - Added *Current offset* column to the *Consumer groups* tab
+- Misc
+    - Implemented loading spinners on various actionable clicks
+    - Updated set-up instructions for external repositories
+    - You should notice an improvement of speed and responsiveness to various places in the platform, e.g. creation of topics, loading of pages, and duration of sync processes
+    - Updated Quix branding for all login pages
+
+`BUG FIXES`
+
+- Deployment
+    - When deployments encounter a `runtime error` and fail to start, the logs being displayed were from the previous run (if available). This has now been fixed to show the live logs of the current failing run.
+    - Switching to *custom* offset in the *messages* tab **for the first time** displayed the first `50` messages, even if your custom offset was e.g. `45890`
+    - You were unable to download build logs more than once, unless the page was refreshed.
+    - In the deployment lineage, you had to click the *Add new* placeholder twice to open the context menu
+    - Fixed and issue where some deployments were stuck in Building state forcing the user to restart the process manually
+- Applications
+    - Files that were nested in a folder were throwing errors when trying to view them
+    - The *Run* button wasn’t visible in the online IDE when cloning some templates
+    - Commit history was sometimes failing to update the correct version of the `app.yaml` and application variables when clicking the *Latest commit* button
+    - Sync banner not updating when performing multiple updates to an application
+- Topics
+    - The empty state appeared when switching from the *waveform* tab to the *messages* tab, even if messages were present previously
+    - Context menu was available for topics that failed to create
+    - Creating a topic with a non-numeric retention breaks the input field
+    - Fixed a random bug were topics being created or updated were stuck in a progress state
+- YAML
+    - The empty state for environment variables disappeared. They’re back now.
+    - If you clicked *Edit code*, to edit the `yaml` file, there was not a way to escape edit-mode if you had not made any changes.
+    - Fixed a bug were YAML history was not retrieved correctly if some branch didn’t have a common parent (detached branches)
+    - When restoring a YAML file after being deleted was not keeping *Latest* versions of the deployments correctly
+- Environments
+    - If an environment was *out of sync*, changing the name of the environment didn’t actually change the name of the environment
+- Misc
+    - Various typing, spacing, and user interaction fixes throughout the platform
+    - Fixed a memory leak in some internal services (StreamingReader, StreamingWriter, ReplayService)
+
 ## 2024-04-05-performance-hf2 | 24 APRIL 2024
 
 `BUG FIXES`

docs/cli/cli-add-destination.md

@@ -37,10 +37,10 @@ The code just prints out the components of the message individually. You could p
 
 ## Edit requirements.txt
 
-Check the `requirements.txt` file. Make sure you are using Quix Streams greater than or equal to 2.4.1:
+Check the `requirements.txt` file. Make sure you are using Quix Streams greater than or equal to 2.4.2:
 
 ```
-quixstreams>=2.4.1
+quixstreams>=2.4.2
 ```
 
 ## Test your destination code

docs/cli/cli-add-transform.md

@@ -90,10 +90,10 @@ You can then edit the `.env` file so that the input topic is `f1-data`.
 
 ## Edit requirements.txt
 
-Check the `requirements.txt` file. Make sure you are using Quix Streams greater than or equal to 2.4.1:
+Check the `requirements.txt` file. Make sure you are using Quix Streams greater than or equal to 2.4.2:
 
 ```
-quixstreams>=2.4.1
+quixstreams>=2.4.2
 ```
 
 ## Run your transform

docs/cli/cli-build-pipeline.md

@@ -9,6 +9,28 @@ In this tutorial you'll build a simple pipeline using the Quix CLI. The steps ar
 5. **Add a destination** - you add a simple destination.
 6. **Sync to Quix Cloud** - you synchronize you CLI project with Quix Cloud.
 
+## Prerequisites
+
+### Quix CLI
+
+Make sure you have **Quix CLI installed**, if you've not already done so:
+
+```
+curl -fsSL https://github.com/quixio/quix-cli/raw/main/install.sh | sudo bash
+```
+
+For further details on installation, including instructions for Microsoft Windows, see the [install guide](https://github.com/quixio/quix-cli?tab=readme-ov-file#installation-of-quix-cli){target=_blank}.
+
+!!! tip
+
+    To update Quix CLI just run `quix update` to get the latest version of Quix CLI.
+
+### Quix Streams
+
+Make sure you install Quix Streams with `python3 -m pip install quixstreams` if you haven't already done so. 
+
+If you already have Quix Streams installed, make sure you are using the latest version with `python -m pip install quixstreams -U` or `python3 -m pip install quixstreams -U`, depending on how your system is set up.
+
 ## Next step
 
 * [Create a project](./cli-create-project.md)

docs/cli/cli-cloud-signup.md

@@ -16,10 +16,12 @@ In this step you create a project in Quix Cloud based on your Git repository.
 4. Link the Quix project to your Git repository using the **setup guide** provided for your chosen Git provider.
 5. Use PROD for your environment, and make sure the `main` branch is selected.
 
-If you need help on creating a project, you can [read the documentation](../create/create-project.md).
-
 When you've created the project, switch back to the command line.
 
+!!! tip
+
+    If you need help on creating a project, you can [read the documentation](../create/create-project.md).
+
 ## Next step
 
 * [Add a source](./cli-add-source.md)

docs/cli/cli-create-project.md

@@ -1,6 +1,6 @@
 ---
 title: Create a Git project
-description: You install the Quix CLI and create a simple project, with a pipeline consisting of one data source application, which you then sync up with Quix Cloud.
+description: You create a Git project, clone it locally, and initialize it to use Quix.
 ---
 
 # Create a Git project
@@ -11,30 +11,23 @@ From your Git repository, you create a simple project on the command line, add a
 
 ## Step 1: Create a Git repository
 
-Create a Git repo where you can store your files, for example you could use GitHub. Create a repo initialized with a `README.md` file, so it can be cloned more easily. 
+Create a Git repo with a Git provider, where you can store your project files. For example, you could use GitHub or GitLab. Create a repo initialized with a `README.md` file, so it can be cloned more easily. Add a `.gitignore` such as the standard one for Python, and a license.
 
 ## Step 2: Clone your Git repo into your local project directory
 
-For example, if your GitHub repo is named `cli-app`:
+Clone your Git repository:
 
 ```
-git clone <url-to-git>/cli-app
-cd cli-app
+git clone <url-to-git-repo>
 ```
 
-## Step 3: Install Quix CLI
+The change into your project directory. For example, if your GitHub repo is named `cli-app`:
 
 ```
-curl -fsSL https://github.com/quixio/quix-cli/raw/main/install.sh | sudo bash
+cd cli-app
 ```
 
-For further details on installation, including instructions for Microsoft Windows, see the [install guide](https://github.com/quixio/quix-cli?tab=readme-ov-file#installation-of-quix-cli){target=_blank}.
-
-!!! tip
-
-    To update Quix CLI just run `quix update` to get the latest version of Quix CLI.
-
-## Step 4: Initialize your project as a Quix project
+## Step 3: Initialize your project as a Quix project
 
 In your Git project directory, enter:
 

docs/cli/cli-github-actions.md

@@ -10,7 +10,7 @@ Sometimes you might want to invoke certain commands of the Quix CLI from GitHub
 Using the Quix CLI, you can push any changes to your code, and sync the repo to Quix, by using the following command:
 
 ```
-quix local deploy --push --sync
+quix local pipeline sync --update
 ```
 
 But sometimes, a developer may make code changes to a branch directly, without using the Quix CLI. In this case you could create a GitHub Action to make sure your repo is always synched to Quix.

docs/cli/cli-quickstart.md

@@ -0,0 +1,95 @@
+# Quix CLI Quickstart
+
+In this guide you create a simple project on the command line, test it locally, and sync it with your Quix Cloud pipeline view.
+
+## Prerequisites
+
+This guide assumes you have installed [Quix Streams](../get-started/install.md).
+
+!!! tip
+
+    Make sure you install Quix Streams with `python3 -m pip install quixstreams` if you haven't already done so. If you have Quix Streams installed, make sure you are using the latest version with `python -m pip install quixstreams -U` or `python3 -m pip install quixstreams -U`, depending on how your system is set up. 
+
+## Step 1: Install Quix CLI
+
+To install Quix CLI:
+
+```
+curl -fsSL https://github.com/quixio/quix-cli/raw/main/install.sh | sudo bash
+```
+
+For further details on installation, including instructions for Microsoft Windows, see the [install guide](https://github.com/quixio/quix-cli?tab=readme-ov-file#installation-of-quix-cli){target=_blank}.
+
+!!! tip
+
+    To update Quix CLI just run `quix update` to get the latest version of Quix CLI.
+
+## Step 2: Create a Git repository
+
+Create a Git repo in your Git provider's service, where you are going to store your project files. For example, you could create a repository on GitHub. Create a repo initialized with a `README.md` file, so it can be cloned more easily. You can add a `.gitignore` (the Python one is a good choice), and a licence as required.
+
+## Step 3: Clone your Git repo into your local project directory
+
+Copy your Git repository URL from your Git provider.
+
+Now, clone the repository using the URL. For example:
+
+```
+git clone <url-to-git-repository>
+```
+
+Change into your project directory. For example, if your project is called `cli-app`:
+
+```
+cd cli-app
+```
+
+## Step 4: Initialize your project as a Quix project
+
+In your Git project directory, enter:
+
+```
+quix local init
+```
+
+This initializes your Quix project with a `quix.yaml` file, which describes your Quix project.
+
+## Step 5: Create your application locally
+
+Now create a sample application:
+
+```
+quix local app create starter-source
+```
+
+This creates a starter source for you. You can explore the files created locally for you. 
+
+## Step 6: In Quix Cloud create a project
+
+In this step you create a project in Quix Cloud from your Git repository.
+
+1. Return to Quix Cloud.
+2. Select `Quix advanced configuration` to continue creation of your project.
+3. Select your Git provider.
+4. Link the project to your Git repository using the guide provided for your chosen Git provider.
+4. Sync Quix Cloud to your project by clicking the `Sync environment` button.
+
+## Step 7: Sync your application
+
+To sync your application, change into the `Starter transformation` directory and enter:
+
+```
+quix local pipeline sync --update
+```
+
+This updates your `quix.yaml` project file, and pushes all changes to your Git repository. It also synchronizes your Git repository with Quix Cloud.
+
+## Step 8: See your pipeline running in Quix Cloud
+
+Go to pipeline view, and see your pipeline running, with your Starter transformation.
+
+![Pipeline running](../images/starter-transform.png)
+
+## Next step
+
+* [Step through a tutorial](../cli/cli-template.md)

docs/cli/cli-reference.md

@@ -200,18 +200,12 @@ quix context broker toggle
 You can push your locally modified code to the remote repository by using:
 
 ```
-quix local deploy --push
+quix local pipeline sync --update
 ```
 
-You can optionally provide a commit message. You are also prompted if changes outside of you current application have been made. You can optionally push these changes too.
+You can optionally provide a commit message with the `-m` option. You are also prompted if changes outside of you current application have been made. You can optionally push these changes too.
 
-In order to synchronize the Git repository with the Quix Cloud environment then use:
-
-```
-quix local deploy --push --sync
-```
-
-The `--sync` option ensures the Git repository is synchronized with the Quix Cloud pipeline view.
+The `--update` option makes sure your `quix.yaml` file is updated.
 
 ## Managing permissions
 

docs/cli/cli-sync-to-cloud.md

@@ -27,10 +27,10 @@ You should be connected to the default context with the Portal URL `https://port
 To sync your application, change into the `F1 demo data` directory and enter:
 
 ```
-quix local deploy --push --sync
+quix local pipeline sync --update
 ```
 
-This updates your `quix.yaml` project file. The `--push` option pushes all changes to your Git repository (it is similar to git add, git commit, git push). You can also add a commit message with the `--commit-message` option if you want. The `--sync` option ensures your Quix environment is synched with the updated Git repository.
+The `sync` command pushes all changes to your Git repository (it is similar to git add, git commit, git push), and synchronization it with Quix Cloud. You can also add a commit message with the `--commit-message`, or `-m` option if you want. The `--update` option ensures your `quix.yaml` project file is updated as required, based on changes to your applications.
 
 !!! important
 

docs/cli/cli-template.md

@@ -83,10 +83,10 @@ You can now modify your code in any way you wish. For example, you could edit a
 Having worked on your code and tested it locally, you can now push up your code to deploy it in Quix Cloud. You can do this with:
 
 ```
-quix local deploy --push --sync
+quix local pipeline sync --update
 ```
 
-This pushes (`--push`) your modified code to the Gitea repository. The `--sync` option causes the Gitea repository to be synchronized with your Quix Cloud pipeline view.
+This updates your `quix.yaml` and pushes your modified code to the Gitea repository and syncs with Quix Cloud.
 
 In Quix Cloud you can view the pipeline and examine it in detail.
 

docs/cli/overview.md

@@ -25,5 +25,5 @@ View information about the CLI in the [GitHub repository](https://github.com/qui
 
 ## Next steps
 
-* [Step through a tutorial](./cli-build-pipeline.md)
+* [Step through the Quickstart](./cli-quickstart.md)
 * [Read the reference guide](./cli-reference.md)

docs/deploy/overview.md

@@ -64,12 +64,10 @@ If you subsequently edit your service code (application) and then click `Redeplo
 
 ## Working on the command line
 
-If you are working on the command line and want to deploy local applications, you can use the `quix local deploy --push --sync` command. This pushes locally modified code to your project repository. 
+If you are working on the command line and want to deploy local applications, you can use the `quix local pipeline sync --update` command. This modifies your `quix.yaml` file, pushes locally modified code to your project repository, and synchronizes with Quix Cloud. 
 
-The `--sync` option performs a sync between your pipeline (as viewed in Quix Cloud) and the project repository. 
+This command also prompts you to include other applications that need to be synched if required. The deployments use the settings specified in the `deployments` section of the project's `quix.yaml` file.
 
-The `deploy` command also prompts you to include other applications that need to be synched if required. The deployments use the settings specified in the `deployments` section of the project's `quix.yaml` file.
+To sync a remote (cloud-based) environment to its project repository, you can use the `quix envs sync` command. This syncs the environment in the current (default) selected context with its project repository.
 
-To sync a remote (cloud-based) environment tpo its project repository, you can use the `quix envs sync` command. This syncs the environment in the current (default) selected context with its project repository.
-
-See the [Quix reference guide](../cli/cli-reference.md) for more about working on the command line.
+See the [Quix reference guide](../cli/cli-reference.md) for more about working on the command line.