Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heat CLI project packaging and publishing #28

Merged
merged 48 commits into from
Sep 13, 2024
Merged

Heat CLI project packaging and publishing #28

merged 48 commits into from
Sep 13, 2024

Conversation

jwric
Copy link
Contributor

@jwric jwric commented Aug 19, 2024

This PR adds crate based project packaging and uploading to the Heat backend, aswell as queueing a job for the new runner service.

Closes #26

ThierryCantin-Demers and others added 30 commits June 27, 2024 18:29
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
… for now)

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
There are some problems with running with multiple config files with Heat for now.

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Which allows for specifying json config files to run a experiment training with.

Co-authored-by: Thierry Cantin-Demers <ThierryCantin-Demers@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwrichard26@gmail.com>
Experiment can now be run from crate from CLI. General cleanup of macros. Only generate macros when necessary.

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Seems to be a little untable when using multiple backends at the same time.

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Moved codegen to crate gen (mostly).

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
to prevent building heat-sdk-cli when building second cli
Organized crate generation in structs

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
…code.

Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
better encapsulation for cli context, removed most getters and setters
…cargo

pushing this as a backup for now

Co-authored-by: Thierry Cantin-Demers <ThierryCantin-Demers@users.noreply.github.com>
jwric and others added 9 commits July 26, 2024 11:31
Co-authored-by: Thierry Cantin-Demers <ThierryCantin-Demers@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
@jwric jwric self-assigned this Aug 19, 2024
@jwric jwric changed the title Heat CLI project packaging and self-hosted. Heat CLI project packaging and publishing Aug 19, 2024
@jwric jwric removed their assignment Aug 19, 2024
ThierryCantin-Demers and others added 5 commits August 19, 2024 13:42
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
Co-authored-by: Jonathan Richard <jwric@users.noreply.github.com>
@nathanielsimard
Copy link
Member

too big to review, but a demo would be cool soon @ThierryCantin-Demers

Copy link
Member

@syl20bnr syl20bnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skimmed through it and it looks fine to me.

The temp json file seems to be unused.

It is not ideal to import thousands of cargo lines of code as it is not easy to maintain and keep it in sync with upstream but the code of cargo is pretty mature already so I guess we are fine for quite some time.

Can you add a header comment that points to the original source files in github for each imported source file ?

Also can you add comments around the modifications you have made when possible with something like:

// --- Start Tracel modifications ---
...
// --- End Tracel modifications ---

@@ -0,0 +1,830 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file seems to not be used anywhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, it was the output of a cargo metadata that we forgot to remove. Removing it now.

@@ -0,0 +1,13 @@
//! All of the code in this directory is heavily inspired by the [Cargo source code](https://github.com/rust-lang/cargo)
//! and modified to fit the needs of the heat-sdk-cli project.
Copy link
Member

@syl20bnr syl20bnr Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that you thought about adding this.

Can you go more into details about the modifications you have made in this source code as well in this comment ?

Copy link
Member

@syl20bnr syl20bnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@syl20bnr syl20bnr merged commit 8efe30c into main Sep 13, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Package Burn Project
4 participants