Feat/update contributing

CONTRIBUTING.md

@@ -4,6 +4,11 @@
 
 Raising [issues](https://github.com/shuttle-hq/shuttle/issues) is encouraged. We have some templates to help you get started.
 
+## Docs
+
+If you found an error in our docs, or you simply want to make them better, contributions to our [docs](https://github.com/shuttle-hq/shuttle-docs)
+are always appreciated!
+
 ## Running Locally
 You can use Docker and docker-compose to test shuttle locally during development. See the [Docker install](https://docs.docker.com/get-docker/)
 and [docker-compose install](https://docs.docker.com/compose/install/) instructions if you do not have them installed already.
@@ -13,20 +18,22 @@ You should now be ready to setup a local environment to test code changes to cor
 Build the required images with:
 
 ```bash
-$ make images
+make images
 ```
 
+> Note: The current [Makefile](https://github.com/shuttle-hq/shuttle/blob/main/Makefile) does not work on Windows systems, if you want to build the local environment on Windows you could use [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/install).
+
 The images get built with [cargo-chef](https://github.com/LukeMathWalker/cargo-chef) and therefore support incremental builds (most of the time). So they will be much faster to re-build after an incremental change in your code - should you wish to deploy it locally straight away.
 
 You can now start a local deployment of shuttle and the required containers with:
 
 ```bash
-$ make up
+make up
 ```
 
-*Note*: Other useful commands can be found within the [Makefile](https://github.com/shuttle-hq/shuttle/blob/main/Makefile).
+> Note: Other useful commands can be found within the [Makefile](https://github.com/shuttle-hq/shuttle/blob/main/Makefile).
 
-The API is now accessible on `localhost:8000` (for app proxies) and `localhost:8001` (for the control plane). When running `cargo run --bin cargo-shuttle` (in a debug build), the CLI will point itself to `localhost` for its API calls. The deployment parameters can be tweaked by changing values in the [.env](./.env) file.
+The API is now accessible on `localhost:8000` (for app proxies) and `localhost:8001` (for the control plane). When running `cargo run --bin cargo-shuttle` (in a debug build), the CLI will point itself to `localhost` for its API calls.
 
 In order to test local changes to the `shuttle-service` crate, you may want to add the below to a `.cargo/config.toml` file. (See [Overriding Dependencies](https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html) for more)
 
@@ -39,6 +46,12 @@ shuttle-shared-db = { path = "[base]/shuttle/resources/shared-db" }
 shuttle-secrets = { path = "[base]/shuttle/resources/secrets" }
 ```
 
+Prime gateway database with an admin user:
+
+```bash
+docker compose --file docker-compose.rendered.yml --project-name shuttle-dev exec gateway /usr/local/bin/service --state=/var/lib/shuttle/gateway.sqlite init --name admin --key test-key
+```
+
 Login to shuttle service in a new terminal window from the main shuttle directory:
 
 ```bash
@@ -51,18 +64,37 @@ cd into one of the examples:
 cd examples/rocket/hello-world/
 ```
 
-Deploy the example:
+Create a new project, this will start a deployer container:
 
 ```bash
 # the --manifest-path is used to locate the root of the shuttle workspace
+cargo run --manifest-path ../../../Cargo.toml --bin cargo-shuttle -- project new
+```
+
+Verify that the deployer is healthy and in the ready state:
+
+```bash
+cargo run --manifest-path ../../../Cargo.toml --bin cargo-shuttle -- project status
+```
+
+Deploy the example:
+
+```bash
 cargo run --manifest-path ../../../Cargo.toml --bin cargo-shuttle -- deploy
 ```
 
 Test if the deploy is working:
 
 ```bash
-# (the Host header should match the Host from the deploy output)
-curl --header "Host: {app}.localhost.local" localhost:8000/hello
+# the Host header should match the Host from the deploy output
+curl --header "Host: {app}.unstable.shuttleapp.rs" localhost:8000/hello
+```
+
+View logs from the current deployment:
+
+```bash
+# append `--follow` to this command for a live feed of logs
+cargo run --manifest-path ../../../Cargo.toml --bin cargo-shuttle -- logs
 ```
 
 ### Testing deployer only
@@ -96,23 +128,46 @@ export DOCKER_HOST=unix:///tmp/podman.sock
 
 shuttle can now be run locally using the steps shown earlier.
 
-*NOTE*: Testing the `gateway` with a rootless Podman does not work since Podman does not allow access to the `deployer` containers via IP address!
+> Note: Testing the `gateway` with a rootless Podman does not work since Podman does not allow access to the `deployer` containers via IP address!
 
 ## Running Tests
 
 shuttle has reasonable test coverage - and we are working on improving this
 every day. We encourage PRs to come with tests. If you're not sure about
 what a test should look like, feel free to [get in touch](https://discord.gg/H33rRDTm3p).
 
-To run the test suite - just run `make test` at the root of the repository.
+To run the unit tests for a spesific crate, from the root of the repository run:
+
+```bash
+# replace <crate-name> with the name of the crate to test, e.g. `shuttle-common`
+cargo test --package <crate-name> --all-features --lib -- --nocapture
+```
+
+To run the integration tests for a spesific crate (if it has any), from the root of the repository run:
+
+```bash
+# replace <crate-name> with the name of the crate to test, e.g. `cargo-shuttle`
+cargo test --package <crate-name> --all-features --test '*' -- --nocapture
+```
+
+To run the end-to-end tests, from the root of the repository run:
+
+```bash
+make test
+```
 
+> Note: Running all the end-to-end tests may take a long time, so it is recommended to run individual tests shipped as part of each crate in the workspace first.
 ## Committing
 
 We use the [Angular Commit Guidelines](https://github.com/angular/angular/blob/master/CONTRIBUTING.md#commit). We expect all commits to conform to these guidelines.
 
 Furthermore, commits should be squashed before being merged to master.
 
-Also, make sure your commits don't trigger any warnings from Clippy by running: `cargo clippy --tests --all-targets`. If you have a good reason to contradict Clippy, insert an #allow[] macro, so that it won't complain.
+Before committing:
+- Make sure your commits don't trigger any warnings from Clippy by running: `cargo clippy --tests --all-targets`. If you have a good reason to contradict Clippy, insert an `#[allow(clippy::<lint>)]` macro, so that it won't complain.
+- Make sure your code is correctly formatted: `cargo fmt --all --check`.
+- Make sure your `Cargo.toml`'s are sorted: `cargo sort --workspace`. This command uses the [cargo-sort crate](https://crates.io/crates/cargo-sort) to sort the `Cargo.toml` dependencies alphabetically.
+- If you've made changes to examples, make sure the above commands are ran there as well.
 
 ## Project Layout
 The folders in this repository relate to each other as follow:

e2e/README.md

@@ -4,12 +4,16 @@ This crate runs all the end-to-end tests for shuttle. These tests must run again
 Running all the end-to-end tests may take a long time, so it is recommended to run individual tests shipped as part of each crate in the workspace first.
 
 ## Running the tests
-Simply do
+In the root of the repository, run:
 
 ```bash
-$ SHUTTLE_API_KEY=test-key cargo test -- --nocapture
+make test
 ```
 
-the `--nocapture` flag helps with logging errors as they arise instead of in one block at the end.
+To run individual tests, in the root of the e2e directory run:
+
+```bash
+POSTGRES_PASSWORD=postgres APPS_FQDN=unstable.shuttleapp.rs cargo test <test name> -- --nocapture
+```
 
 The server-side logs can be accessed with `docker compose logs`.

e2e/tests/integration/axum.rs

@@ -4,8 +4,9 @@ use crate::helpers::{self, APPS_FQDN};
 
 #[test]
 fn hello_world_axum() {
-    let client = helpers::Services::new_docker("hello-world (axum)", Color::Green);
-    client.deploy("axum/hello-world");
+    let client =
+        helpers::Services::new_docker("hello-world (axum)", "axum/hello-world", Color::Green);
+    client.deploy();
 
     let request_text = client
         .get("hello")

e2e/tests/integration/helpers/mod.rs

@@ -112,10 +112,10 @@ CARGO_HOME: {}
         let admin_key = if let Ok(key) = env::var("SHUTTLE_API_KEY") {
             key
         } else {
-            "test-key".to_string()
+            "e2e-test-key".to_string()
         };
 
-        Command::new(DOCKER.as_os_str())
+        _ = Command::new(DOCKER.as_os_str())
             .args([
                 "compose",
                 "--file",
@@ -128,12 +128,11 @@ CARGO_HOME: {}
                 "--state=/var/lib/shuttle/gateway.sqlite",
                 "init",
                 "--name",
-                "admin",
+                "test",
                 "--key",
                 &admin_key,
             ])
-            .output()
-            .ensure_success("failed to create admin user on gateway");
+            .output();
     };
 }
 
@@ -214,12 +213,14 @@ pub fn spawn_and_log<D: std::fmt::Display, C: Into<Color>>(
 pub struct Services {
     api_addr: SocketAddr,
     proxy_addr: SocketAddr,
+    /// Path within the examples dir to a specific example
+    example_path: String,
     target: String,
     color: Color,
 }
 
 impl Services {
-    fn new_free<D, C>(target: D, color: C) -> Self
+    fn new_free<D, C>(target: D, example_path: D, color: C) -> Self
     where
         D: std::fmt::Display,
         C: Into<Color>,
@@ -229,16 +230,25 @@ impl Services {
             proxy_addr: "127.0.0.1:8000".parse().unwrap(),
             target: target.to_string(),
             color: color.into(),
+            example_path: example_path.to_string(),
         }
     }
 
-    pub fn new_docker<D, C>(target: D, color: C) -> Self
+    /// Initializes a a test client
+    ///
+    /// # Arguments
+    ///
+    /// * `target` - A string that describes the test target
+    /// * `example_path` - Path to a specific example within the examples dir, this is where
+    ///   `project new` and `deploy` will run
+    /// * `color` - a preferably unique `crossterm::style::Color` to distinguish test logs
+    pub fn new_docker<D, C>(target: D, example_path: D, color: C) -> Self
     where
         D: std::fmt::Display,
         C: Into<Color>,
     {
         let _ = *LOCAL_UP;
-        let service = Self::new_free(target, color);
+        let service = Self::new_free(target, example_path, color);
         service.wait_ready(Duration::from_secs(15));
 
         // Make sure provisioner is ready, else deployers will fail to start up
@@ -330,18 +340,18 @@ impl Services {
         panic!("timed out while waiting for mongodb to be ready");
     }
 
-    pub fn wait_deployer_ready(&self, project_path: &str, mut timeout: Duration) {
+    pub fn wait_deployer_ready(&self, mut timeout: Duration) {
         let mut now = SystemTime::now();
         while !timeout.is_zero() {
             let mut run = Command::new(WORKSPACE_ROOT.join("target/debug/cargo-shuttle"));
 
             if env::var("SHUTTLE_API_KEY").is_err() {
-                run.env("SHUTTLE_API_KEY", "test-key");
+                run.env("SHUTTLE_API_KEY", "e2e-test-key");
             }
 
             run.env("CARGO_HOME", CARGO_HOME.path());
             run.args(["project", "status"])
-                .current_dir(Self::get_project_path(project_path));
+                .current_dir(self.get_full_project_path());
             let stdout = run.output().unwrap().stdout;
             let stdout = String::from_utf8(stdout).unwrap();
 
@@ -358,31 +368,31 @@ impl Services {
         panic!("timed out while waiting for deployer to be ready");
     }
 
-    pub fn run_client<'s, I>(&self, args: I, project_path: &str) -> Child
+    pub fn run_client<'s, I>(&self, args: I) -> Child
     where
         I: IntoIterator<Item = &'s str>,
     {
         let mut run = Command::new(WORKSPACE_ROOT.join("target/debug/cargo-shuttle"));
 
         if env::var("SHUTTLE_API_KEY").is_err() {
-            run.env("SHUTTLE_API_KEY", "test-key");
+            run.env("SHUTTLE_API_KEY", "e2e-test-key");
         }
 
         run.env("CARGO_HOME", CARGO_HOME.path());
 
-        run.args(args)
-            .current_dir(Self::get_project_path(project_path));
+        run.args(args).current_dir(self.get_full_project_path());
         spawn_and_log(&mut run, &self.target, self.color)
     }
 
-    pub fn deploy(&self, project_path: &str) {
-        self.run_client(["project", "new"], project_path)
+    /// Starts a project and deploys a service for the example in `self.example_path`
+    pub fn deploy(&self) {
+        self.run_client(["project", "new"])
             .wait()
             .ensure_success("failed to run deploy");
 
-        self.wait_deployer_ready(project_path, Duration::from_secs(120));
+        self.wait_deployer_ready(Duration::from_secs(120));
 
-        self.run_client(["deploy", "--allow-dirty"], project_path)
+        self.run_client(["deploy", "--allow-dirty"])
             .wait()
             .ensure_success("failed to run deploy");
     }
@@ -396,7 +406,15 @@ impl Services {
         reqwest::blocking::Client::new().post(format!("http://{}/{}", self.proxy_addr, sub_path))
     }
 
-    pub fn get_project_path(project_path: &str) -> PathBuf {
-        WORKSPACE_ROOT.join("examples").join(project_path)
+    /// Gets the full path: the path within examples to a specific example appended to the workspace root
+    pub fn get_full_project_path(&self) -> PathBuf {
+        WORKSPACE_ROOT.join("examples").join(&self.example_path)
+    }
+}
+
+impl Drop for Services {
+    fn drop(&mut self) {
+        // Initiate project destruction on test completion
+        _ = self.run_client(["project", "rm"]).wait();
     }
 }

e2e/tests/integration/poem.rs

@@ -4,8 +4,9 @@ use crate::helpers::{self, APPS_FQDN};
 
 #[test]
 fn hello_world_poem() {
-    let client = helpers::Services::new_docker("hello-world (poem)", Color::Cyan);
-    client.deploy("poem/hello-world");
+    let client =
+        helpers::Services::new_docker("hello-world (poem)", "poem/hello-world", Color::Cyan);
+    client.deploy();
 
     let request_text = client
         .get("hello")
@@ -20,8 +21,8 @@ fn hello_world_poem() {
 
 #[test]
 fn postgres_poem() {
-    let client = helpers::Services::new_docker("postgres (poem)", Color::Blue);
-    client.deploy("poem/postgres");
+    let client = helpers::Services::new_docker("postgres (poem)", "poem/postgres", Color::Blue);
+    client.deploy();
 
     let add_response = client
         .post("todo")
@@ -48,8 +49,8 @@ fn postgres_poem() {
 
 #[test]
 fn mongodb_poem() {
-    let client = helpers::Services::new_docker("mongo (poem)", Color::Green);
-    client.deploy("poem/mongodb");
+    let client = helpers::Services::new_docker("mongo (poem)", "poem/mongodb", Color::Green);
+    client.deploy();
 
     // post todo and get its generated objectId
     let add_response = client