Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚠️ Breaking Changes: Migrate model subcommand to model analyze #1061

Closed
ChenYi015 opened this issue Mar 26, 2024 · 1 comment · Fixed by #1060
Closed

⚠️ Breaking Changes: Migrate model subcommand to model analyze #1061

ChenYi015 opened this issue Mar 26, 2024 · 1 comment · Fixed by #1060

Comments

@ChenYi015
Copy link
Collaborator

ChenYi015 commented Mar 26, 2024

Currently, the arena model subcommand is used for submitting a model analysis job, with available job types being profile, evaluate, optimize and benchmark. These jobs have nothing to do with model management. To integrate some form of model registry, we first need to migrate the arena model subcommand to somewhere else, like arena model analyze.

Before migration:

$ arena model --help
submit a model analyze job.

Usage:
  arena model [flags]
  arena model [command]

Available Commands:
  benchmark   Submit a model benchmark job
  delete      Delete a model job
  evaluate    Submit a model evaluate job
  get         Get a model job
  list        List all the model jobs
  optimize    Submit a model optimize job, this is a experimental feature
  profile     Submit a model profile job

After migration:

$ arena model analyze --help                    
Submit a model analyze job. (experimental feature)

Usage:
  arena model analyze [flags]
  arena model analyze [command]

Available Commands:
  benchmark   Submit a model benchmark job
  delete      Delete a model analyze job
  evaluate    Submit a model evaluate job
  get         Get a model analyze job
  list        List all model analyze jobs
  optimize    Submit a model optimize job
  profile     Submit a model profile job
@Syulin7
Copy link
Collaborator

Syulin7 commented Mar 26, 2024

Agree, especially considering that this feature is an experimental feature. As stated in #1059, we need to support model management in Arena, so I believe it is necessary to migrate the model subcommand to analyze. Perhaps we can complete this in the next release version.

cc @cheyang @gujingit

@Syulin7 Syulin7 changed the title Migrate model subcommand to analyze ⚠️ Breaking Changes: Migrate model subcommand to analyze Mar 27, 2024
@ChenYi015 ChenYi015 changed the title ⚠️ Breaking Changes: Migrate model subcommand to analyze ⚠️ Breaking Changes: Migrate model subcommand to model analyze Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants