[Bug]: shuttle 0.13.0; Rocket; rocket_dyn_templates; project fails to deploy

[jhawkesworth]

What happened?

I have been unable to get a test project to deploy to shuttle when using rocket_dyn_templates feature of Rocket
https://api.rocket.rs/v0.5-rc/rocket_dyn_templates/index.html

My code is available here: https://github.com/jhawkesworth/jh-rocket-fs

If you are starting from a clean project, be aware that the default location for templates is a sub-folder called 'templates', relative to crate root. If you want to use a different folder, it appears this may only be specified in Rocket.toml, but I have had no luck getting the Rocket.toml included in the project, so the example above specifies a static folder called 'templates'.

Version

v0.13.0

Which operating systems are you seeing the problem on?

Linux

Which CPU architectures are you seeing the problem on?

x86_64

Relevant log output

2023-03-27T17:30:29.651103746Z  INFO shuttle_deployer::deployment::queue: Running tests before starting up
2023-03-27T17:30:53.995789838Z  INFO    Compiling jh-rocket-fs v0.1.0 (/opt/shuttle/shuttle-builds/jh-rocket-fs)
2023-03-27T17:30:54.352589063Z  INFO     Finished release [optimized] target(s) in 24.69s
2023-03-27T17:30:54.362394379Z  INFO      Running unittests src/main.rs (opt/shuttle/shuttle-builds/jh-rocket-fs/target/release/deps/jh_rocket_fs-37feb5346e800625)

2023-03-27T17:30:54.809098179Z  INFO Entering built state

2023-03-27T17:30:54.809293559Z  INFO Entering loading state
2023-03-27T17:30:54.813743522Z TRACE shuttle_deployer::runtime_manager: making new client
2023-03-27T17:30:54.817758620Z DEBUG shuttle_deployer::runtime_manager: Starting alpha runtime at: /opt/shuttle/shuttle-executables/52897af6-1b21-41cc-ac93-d70ed552d383
2023-03-27T17:30:56.820951261Z  INFO shuttle_proto::runtime: connecting runtime client
2023-03-27T17:30:56.821039607Z DEBUG hyper::client::connect::http: connecting to 127.0.0.1:20023
2023-03-27T17:30:56.823592293Z DEBUG hyper::client::connect::http: connected to 127.0.0.1:20023
2023-03-27T17:30:56.825994077Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-03-27T17:30:56.828661983Z  INFO shuttle_deployer::deployment::run: loading project from: /opt/shuttle/shuttle-executables/52897af6-1b21-41cc-ac93-d70ed552d383
2023-03-27T17:30:56.831528024Z DEBUG shuttle_deployer::deployment::run: loading service
2023-03-27T17:30:56.832676420Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-03-27T17:30:56.844427572Z  INFO {response="LoadResponse { success: true, message: \"\", resources: [[123, 34, 116, 121, 112, 101, 34, 58, 34, 115, 116, 97, 116, 105, 99, 102, 111, 108, 100, 101, 114, 34, 44, 34, 99, 111, 110, 102, 105, 103, 34, 58, 123, 34, 102, 111, 108, 100, 101, 114, 34, 58, 34, 116, 101, 109, 112, 108, 97, 116, 101, 115, 34, 125, 44, 34, 100, 97, 116, 97, 34, 58, 34, 47, 111, 112, 116, 47, 115, 104, 117, 116, 116, 108, 101, 47, 115, 104, 117, 116, 116, 108, 101, 45, 115, 116, 111, 114, 97, 103, 101, 47, 106, 104, 45, 114, 111, 99, 107, 101, 116, 45, 102, 115, 47, 116, 101, 109, 112, 108, 97, 116, 101, 115, 34, 125]] }"} shuttle_deployer::deployment::run: loading response

2023-03-27T17:30:56.850254256Z  INFO Entering running state
2023-03-27T17:30:56.860867805Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-03-27T17:30:56.861078379Z  INFO shuttle_deployer::deployment::run: starting service
2023-03-27T17:30:56.863048002Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-03-27T17:30:56.865635868Z  INFO {response="StartResponse { success: true }"} shuttle_deployer::deployment::run: start client response: 
2023-03-27T17:30:56.867736096Z DEBUG {response="Some(SubscribeStopResponse { reason: End, message: \"\" })"} shuttle_deployer::deployment::run: stop client response: 

2023-03-27T17:30:56.869724055Z  INFO Entering completed state
2023-03-27T17:30:56.872089769Z  INFO shuttle_deployer::deployment::run: service finished all on its own
2023-03-27T17:30:56.876720752Z DEBUG hyper::proto::h2::server: stream error: error writing a body to connection: send stream capacity unexpectedly closed

Duplicate declaration

• I have searched the issues and there are none like this.

[jonaro00]

For reference, the numbers decode into {"type":"staticfolder","config":{"folder":"templates"},"data":"/opt/shuttle/shuttle-storage/jh-rocket-fs/templates"}

[jhawkesworth]

For the record I tried with shuttle_static_folder 0.13.1 but results were the same.

[pogopaule]

Same problem here. Works with cargo shuttle run locally but cargo shuttle deploy fails.

[pogopaule]

JFYI: Problem still present with 0.14.0

...
2023-04-13T19:08:35.150379141Z  INFO Entering running state
2023-04-13T19:08:35.163242275Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-04-13T19:08:35.163594843Z  INFO shuttle_deployer::deployment::run: starting service
2023-04-13T19:08:35.166381926Z DEBUG {service.ready=true} tower::buffer::worker: processing request
2023-04-13T19:08:35.169502151Z  INFO {response="StartResponse { success: true }"} shuttle_deployer::deployment::run: start client response:
2023-04-13T19:08:35.169576671Z ERROR rocket_dyn_templates::context: Invalid template directory 'templates': No such file or directory (os error 2).
2023-04-13T19:08:35.169592745Z ERROR rocket_dyn_templates::fairing::_: Template initialization failed. Aborting launch.
2023-04-13T19:08:35.169621309Z  INFO shuttle_runtime::alpha: service stopped all on its own
2023-04-13T19:08:35.169624251Z ERROR rocket::error: Rocket failed to launch due to failing fairings:
2023-04-13T19:08:35.169625781Z  INFO rocket::error::_: Templating
2023-04-13T19:08:35.172174939Z DEBUG {response="Some(SubscribeStopResponse { reason: End, message: \"\" })"} shuttle_deployer::deployment::run: stop client response:

2023-04-13T19:08:35.175068996Z  INFO Entering completed state
2023-04-13T19:08:35.181131532Z  INFO shuttle_deployer::deployment::run: service finished all on its own

[jhawkesworth]

@pogopaule 0.14.0 also fails for me. But interestingly I don't get the same error as you. Are you able to share your code somewhere? Maybe I can help.

[pogopaule]

@jhawkesworth I am not able to reproduce the logs I posted above. I tested various things, including version changes.

Unfortunately I cannot share the code.

I'll look into other services like render, fly.io, etc. since this bug is a show stopper for my project.

P.S. An idea for a workaround I haven't tested yet: one could render the templates with the help of the handlebars crate directly and keep the content of the hbs files inside rust files...

[oddgrd]

Hey! I'm not super familiar with rocket so I may be off here, but I assume this has to do with how the Templates::fairing() discovers the static folder. Can you pass in the path that shuttle_static_folder gives you to that somehow?

[pogopaule]

@oddgrd Rocket does this via Rocket.toml. @jhawkesworth already tried it but it did not work. Same for me.

[oddgrd]

Hmmm 🤔 Did you try using this to pass in the path that you get in your shuttle main function from shuttle_static_folder? I'm afk atm but IIRC this path will be different for each deployment.

[pogopaule]

🚀 you are right @oddgrd! This works

#[shuttle_runtime::main]
async fn rocket(
   [...]
    #[shuttle_static_folder::StaticFolder] static_folder: PathBuf,
) -> shuttle_rocket::ShuttleRocket {
    [...]
    let template_dir = static_folder.to_str().unwrap();
    let figment = rocket::Config::figment().merge(("template_dir", template_dir));

    let rocket = rocket::custom(figment)
        .attach(Template::fairing())
        .mount(
    [...]

Thank you! 👍

[jhawkesworth]

@oddgrd Thank you this fix also works for my project.
I hadn't understood the path would change each time a deployment took place.
I will attempt to create an example so there is code that new users can be pointed at in future, then this issue can be closed.

[jhawkesworth]

Examples project pull request added, once it (or something better) gets merged, let's close this one.

[oddgrd]

Nice! 🥳 I remembered incorrectly, the path to the folder will be consistent, but it will not be in the same directory as your executable. I think this is a classic case of the "magic" of a framework making it difficult to figure out what went wrong, as you would expect the folder to be in the crate root.

We should add this to the docs for shuttle_static_folder I think. Clarifying that the folder will not be accessible as if it was in the root of the same directory as the executable, and the only reliable way to get the path is to use the path from #[shuttle_static_folder::StaticFolder] static_folder: PathBuf.

Thanks for the examples too, they look good! ❤️ We have a lot of examples for rocket already though, so perhaps we should do just the one with handlebars, since that shows adding the path to the rocket config as well?

[jhawkesworth]

@oddgrd Thank you for speedy review. Agree with the above. Have removed the static-folder example from my examples PR shuttle-hq/shuttle-examples#38 and opened above PR to update the static-folder documentation.