Skip to content

Commit

Permalink
CASMCLOUD-1220: Language linting
Browse files Browse the repository at this point in the history
  • Loading branch information
mharding-hpe authored and rustydb committed Jun 5, 2023
1 parent f5cbbd1 commit b7e8c62
Show file tree
Hide file tree
Showing 13 changed files with 108 additions and 103 deletions.
6 changes: 3 additions & 3 deletions Developing.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Using the [Cray Service Generator](https://github.com/Cray-HPE/cray-generators)
is the preferred method of integrating with the CLI. This will automatically
take your swagger file and bootstrap a new CLI module from it, creating a pull
take your Swagger file and bootstrap a new CLI module from it, creating a pull
request into the CLI along the way. The only thing you need to do after this
process is create functional tests and add them to your pull request.

Expand Down Expand Up @@ -170,7 +170,7 @@ If you find a bug in the `craycli` framework, feel free to open a bug in the
CASMCLOUD project. We'll triage it within a day or two.
[File Bug](https://github.com/Cray-HPE/craycli/issues/new)

## How to update your swagger
## How to update your Swagger

Assuming you're using the .remote option for your module: (from the root of the forked project)
Expand All @@ -185,7 +185,7 @@ Assuming you're using the .remote option for your module: (from the root of the
python -m pip install .[ci]
```
- Generate swagger.
- Generate Swagger.
```bash
nox -s swagger -- my_service_name path/to/api/file
Expand Down
155 changes: 80 additions & 75 deletions Integration.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,36 @@
# Integrating modules

Integrating an API with the CLI is a fairly easy process once you have a valid swagger file.
Integrating an API with the CLI is a fairly easy process once you have a valid Swagger file.

1. Fork this repository to make changes (Do not attempt to branch. You do not have write permissions)
1. Add your module and swagger file to the `cray/modules/` directory
1. Run the swagger parser to convert from a swagger.yaml file to a json file and validate
1. Fork this repository to make changes (Do not attempt to branch. You do not have write permissions).
1. Add your module and Swagger file to the `cray/modules/` directory.
1. Run the Swagger parser to convert from a YAML Swagger file to a JSON file and validate it.

See the [demo](demo.md) for what we showed the Compass group in Jan '19.

We've provided a generator that will create a CLI from the stash URL of your swagger file:
We've provided a generator that will create a CLI from the stash URL of your Swagger file:

pip3 install nox
nox -s generate -- [module name] [stash swagger url]
```text
pip3 install nox
nox -s generate -- [module name] [Swagger URL]
```

I.e.

nox -s generate -- uas https://github.com/Cray-HPE/uan-mgr/blob/main/api/swagger.yaml
```text
nox -s generate -- uas https://github.com/Cray-HPE/uan-mgr/blob/main/api/swagger.yaml
```

*NOTE: The generator appends `?raw&at=refs%2Fheads%2Fmaster` to the passed in URL.
You are required to use the master branch. For local development, you can pass a local
file to the generate command. However, using remote files is highly encouraged when you open your pull request.*

*NOTE: Support for anyOf/allOf/oneOf keywords in OpenAPI 3 has been added. All
*NOTE: Support for `anyOf`/`allOf`/`oneOf` keywords in OpenAPI 3 has been added. All
options under these keywords are added to the option list for the individual
command that uses them. Any validation of required options must be done on the
server side.

This will create everything required to create a CLI from your swagger file. You can now making any customizations required. The above command is idempotent and will not overwrite any customizations you've made. Make sure you write a few functional tests for your CLI to ensure it is working as you expect.
This will create everything required to create a CLI from your Swagger file. You can now making any customizations required. The above command is idempotent and will not overwrite any customizations you've made. Make sure you write a few functional tests for your CLI to ensure it is working as you expect.

### Why the manual parse to generate yet another file?

Expand All @@ -37,10 +41,9 @@ the fly for each module will be a big performance hit. The CLI needs to parse
each module to get a list of commands when it runs. We want to optimize this
as much as possible.


### Notes on user experience

We encourage all integrators to use our swagger file parser to automatically
We encourage all integrators to use our Swagger file parser to automatically
generate your CLI module. This will automatically create CLI commands based on
your REST endpoints and add options/arguments based on the payload.

Expand All @@ -52,70 +55,69 @@ That said, we understand there are scenarios where complex APIs are required or
manually creating CLI commands is needed. If this is the case for you please carefully
read the documentation below.


## Basic Example
## Basic example

An example of your cli.py will look like this:

from cray.generator import generate
from cray.core import echo


def cb(response):
""" Optional callback to do post-processing on REST responses.
Passes in a requests lib response object. It's best to omit this
unless you have specific tasks to do. The formatter will handle
formatting the result in JSON, TOML, text, or any of the other supported
formats."""
echo("Status code: {}".format(response.status_code))
return response


# Pass in the CLI group you want the commands to be added to, the directory
# where you module lives, the name of your parsed swagger file, and an
# optional callback.
cli = generate(__file__, callback=cb)

## Advanced Example

from cray.generator import generate
from cray.core import argument, option, echo
```python
from cray.generator import generate
from cray.core import echo

def cb(response):
""" Optional callback to do post-processing on REST responses.
Passes in a requests lib response object. It's best to omit this
unless you have specific tasks to do. The formatter will handle
formatting the result in JSON, TOML, text, or any of the other supported
formats."""
echo("Status code: {}".format(response.status_code))
return response

# Pass in the CLI group you want the commands to be added to, the directory
# where you module lives, the name of your parsed Swagger file, and an
# optional callback.
cli = generate(__file__, callback=cb)
```

## Advanced example

cli = generate(__file__)
```python
from cray.generator import generate
from cray.core import argument, option, echo

cli = generate(__file__)

@cli.command(name="demo")
@argument("name")
@option('--shout', is_flag=True)
def demo(name, shout):
echo("This is a verbose statement")
resp = 'Hello, {}'.format(name)
if shout:
resp = '{}!!'.format(resp.upper())
return resp
@cli.command(name="demo")
@argument("name")
@option('--shout', is_flag=True)
def demo(name, shout):
echo("This is a verbose statement")
resp = 'Hello, {}'.format(name)
if shout:
resp = '{}!!'.format(resp.upper())
return resp
```

## Guidelines

The CLI is an internal standard that controls how users interact with a cray system. It makes a few assumptions about the services it is calling. Because it is just one client and not the only possible client, services may choose to implement more sophisticated strctures than are currently supported by the CLI. The CLI will adhere to these guidelines when making calls to services.
The CLI is an internal standard that controls how users interact with a Cray system. It makes a few assumptions about the services it is calling. Because it is just one client and not the only possible client, services may choose to implement more sophisticated strctures than are currently supported by the CLI. The CLI will adhere to these guidelines when making calls to services.

### Supported CLI MIME types

```
'application/json'
'application/x-www-form-urlencoded'
'application/octet-stream'
'multipart/form-data'
```text
'application/json'
'application/x-www-form-urlencoded'
'application/octet-stream'
'multipart/form-data'
```

### Parameters

For all intents and purposes, `cray.argument` and `cray.option` acts just like
[`click.argument`](https://click.palletsprojects.com/en/7.x/arguments/) and [`click.option`](https://click.palletsprojects.com/en/7.x/options/), with one caveat. All options within
the cray cli can have a default value defined within a configuration file. When you add an
the Cray CLI can have a default value defined within a configuration file. When you add an
option to a command, the framework will first attempt to find a default value from
the configuration file before using the options `default` variable. This is why
it is so important to only use parameters on `cray.commands`, as configuration files
it is so important to only use parameters on `cray.commands`, because configuration files
are not loaded until just before the command is called. Note: default values are only
used if the option is not provided in the command.

Expand All @@ -141,7 +143,7 @@ they can quickly get out of hand. We recommend only using short names for option
- PUT/PATCH: `update`
- DELETE: `delete`

### API Tags
### API tags

craycli has the ability to parse API tags to add additional functionality
that is only for CLI interactions. The following tags are available:
Expand All @@ -159,25 +161,28 @@ everything, continue?"

Note: Tags that craycli does not recognize are ignored by the CLI.

#### Usage Example
#### Usage example

The tags need to be placed on the API calls themselves, not at the top level.

Example of **cli_danger** with the optional message:

/uais:
```yaml
/uais:
delete:
tags:
- "cli_danger$This will delete all running UAIs, Are you sure?"
summary: "Delete all UAIs on the system"
```
Example of **cli_ignore**:

/:
get:
tags:
- "cli_ignore"
```yaml
/:
get:
tags:
- "cli_ignore"
```
## Non-REST functionality
Expand All @@ -186,28 +191,28 @@ nice features like configuration files and global options, we've had to extend
some of their capabilities. This adds some caveats you need to know when
developing.
1. Always use cray provided decorators and classes, try to avoid using click directly
1. Always use Cray-provided decorators and classes, try to avoid using click directly
2. Only add arguments and options to commands, never groups[\*](#Note)
3. Be very careful when doing anything within a group function[\*](#Note)
4. Assume argument/options/globals can change up until the point your command function is called[\*](#Note)
5. Use `cray.echo` for any information that can be omitted with a `--quiet`, your command should `return` your final data, whether it is text or a dict. The CLI will automatically format a returned `dict` based on the `--format` option. The returned `dict` needs to be json serializable.
5. Use `cray.echo` for any information that can be omitted with a `--quiet`. Your command should `return` your final data, whether it is text or a dict. The CLI will automatically format a returned `dict` based on the `--format` option. The returned `dict` needs to be JSON serializable.

### Note

\* We had to make some concessions in order to have a configuration file with
We had to make some concessions in order to have a configuration file with
automatic defaults as well as global options. Since a `--configuration` option
is automatically added to each command, we don't load configuration files
until right before the command is called. It's the first parameter to be handled
for a command. Users can add default values for ANY parameter. This means
that the loading goes:

- Group level parameters loaded and group function called
- Load configuration file (based on envvar, configuration option, or default)
- All other parameters are loaded, default value goes in this order:
- Check global variables
- Check configuration file
- Check if parameter provided a default value
- Return None
that the loading happens in the following order:

1. Group level parameters loaded and group function called
2. Load configuration file (based on envvar, configuration option, or default)
3. All other parameters are loaded, default value goes in this order:
1. Check global variables
2. Check configuration file
3. Check if parameter provided a default value
4. Return `None`

As you can see, putting anything in a group level function can be called
BEFORE the configuration file is loaded and global variables are set.
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -123,19 +123,19 @@ help:
@echo ' help Show this help screen.'
@echo ' clean Remove build files.'
@echo
@echo ' image Build and publish the swagger testing image.'
@echo ' image Build and publish the Swagger testing image.'
@echo ' rpm Build a YUM/SUSE RPM.'
@echo ' all Build all production artifacts.'
@echo
@echo ' synk Runs a snyk scan.'
@echo ' synk Runs a Snyk scan.'
@echo
@echo ' prepare Prepare for making an RPM.'
@echo ' rpm_build Builds the RPM.'
@echo ' rpm_build_source Builds the SRPM.'
@echo ' rpm_package_source Creates the RPM source tarball.'
@echo
@echo ' image_login Logs into the Docker registry for pulling and publishing images.'
@echo ' image_build Builds the swagger testing image.'
@echo ' image_build Builds the Swagger testing image.'
@echo ' image_publish Builds and publishes the testing image.'
@echo ''

Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ needs of a CLI with the ability for each team to integrate their own endpoints.

The CLI provides built in configurations (stored in ~/.config/cray), built-in
authentication token stores (similar to AWS/GCP), and a built-in parser
that can take a swagger file and generate CLI commands.
that can take a Swagger file and generate CLI commands.

## Versioning

Expand Down Expand Up @@ -138,13 +138,13 @@ More information about versioning, see [version number construction][3].

## Environment variables

By default the cli looks for files in `~/.config/cray` (OS agnostic).
By default the CLI looks for files in `~/.config/cray` (OS agnostic).
If a user would like to override this, it can be configured using a `CRAY_CONFIG_DIR`
environment variable. Users can also pass command options using variables like
`CRAY_[{module name}_, {group name}_, ...]{command name}_{option name}=value`

For example, `CRAY_AUTH_LOGIN_USERNAME=ryan` can be used instead of
`cray auth login --username=ryan`. These are command specific, and the above
`cray auth login --username=ryan`. These are command-specific, and the above
environment variable will not be used for any commands other than `cray auth login`.

## Configuration files
Expand Down Expand Up @@ -173,11 +173,11 @@ configuration name to use for each command.

### Creating alternate configurations

If you have more than one system that you intend to work with, the cli can store multiple configuration files. To create a second, third, or more, use `cray init --configuration mynewconfig`
If you have more than one system that you intend to work with, the CLI can store multiple configuration files. To create a second, third, or more, use `cray init --configuration mynewconfig`

## Roadmap Items

THe Shasta CSM CLI has many opportunities for enhancements. Over the course of Phase 1 of the Open Sourcing of the CSM tooling we will be refining the roadmap and priorities for the tool. Please be sure to review this document on a periodic basis for new roadmap items, and the appropriate working group in the Cray/HPE OSS community for dicsussions of the future of the CLI and all other CSM components.
THe Shasta CSM CLI has many opportunities for enhancements. Over the course of Phase 1 of the Open Sourcing of the CSM tooling we will be refining the roadmap and priorities for the tool. Please be sure to review this document on a periodic basis for new roadmap items, and the appropriate working group in the Cray/HPE OSS community for discussions of the future of the CLI and all other CSM components.

High Priority Items:

Expand All @@ -188,7 +188,7 @@ Medium Priority Items:

- Ask the api gateway what modules should be installed.
- Advanced filters to ignore endpoints, etc.
- Run the parsers at build time, allowing to link external swagger files (git).
- Run the parsers at build time, allowing to link external Swagger files (git).
- Create a functional test generator

[1]: https://artifactory.algol60.net/artifactory/csm-rpms/hpe/unstable/sle-15sp4/craycli/
Expand Down
2 changes: 1 addition & 1 deletion Tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Hello, World!

#### Subgroups

If your cli is complex, you can split it up into different sub groups.
If your CLI is complex, you can split it up into different sub groups.

```python
import cray
Expand Down
2 changes: 1 addition & 1 deletion cray/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def cli(ctx, *args, **kwargs):
@click.pass_context
@option(
"--hostname", default=None, no_global=True,
help='Hostname of cray system.'
help='Hostname of Cray system.'
)
@option(
"--tenant", default=None, no_global=True,
Expand Down
2 changes: 1 addition & 1 deletion cray/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ def get_command(self, ctx, cmd_name):
with open(filename, encoding='utf-8') as f:
code = compile(f.read(), filename, 'exec')
# Note: We are trusting the modules to not do bad things
# Since these are cray created we can consider them safe.
# Since these are cray-created we can consider them safe.
# Using eval will improve performance and allow modules to be
# completely segregated.
eval(code, ns, ns) # pylint: disable=eval-used
Expand Down
2 changes: 1 addition & 1 deletion cray/formatting.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def _formatter(format_type):


class _NullStream:
""" NullStream used for yaml dump """
""" NullStream used for YAML dump """

def write(self, *args, **kwargs):
""" Null writer """
Expand Down
Loading

0 comments on commit b7e8c62

Please sign in to comment.