Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pre-commit configuration and workflow #164

Merged
merged 3 commits into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/precommit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Run pre-commit checks

on:
pull_request:
push:

jobs:
run-linters:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.x"

- name: Configure caching
uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: precommit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}

- name: Install pre-commit
run: |
pip install pre-commit

- name: Run linters
run: |
pre-commit run --all-files
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,5 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install tox
- name: Lint
run: tox -epep8
- name: Unit Tests
run: tox -epy3
24 changes: 24 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
repos:
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: v1.5.5
hooks:
- id: remove-tabs

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: trailing-whitespace
- id: check-merge-conflict
- id: end-of-file-fixer
- id: check-added-large-files
- id: check-case-conflict
- id: detect-private-key
- id: check-yaml
args: [--allow-multiple-documents]

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.5.2
hooks:
- id: ruff
args: [ --fix ]
- id: ruff-format
1 change: 0 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -173,4 +173,3 @@
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,3 +167,23 @@ EOF
```

`1718` is the dummy node UUID; replace it with whatever you'd like. When creating an offer for this dummy node, simply specify `resource_type` as `dummy_node` and `resource_uuid` as `1718`.

## Contributing

### Pull requests

When you submit a pull request, your changes will be validated by running a number of automatic tests.

First, we run a series of [linters] and an automatic formatter on the code to check for a variety of minor issues and ensure consistent formatting. As a developer you will want to integrate these same checks into your local development environment:

1. Install the [pre-commit] tool using your favorite package manager.
2. Run `pre-commit install` from inside this repository.

This will enable a git [`pre-commit` hook][hooks] that will run the linters and formatter whenever you commit changes locally. We are using [`ruff`][ruff] for linting and formatting; this can be integrated into many editors to provide live checks as you are writing code.

Next, we run all unit tests across all the Python versions supported by the `esi-leap` code. We expect that any changes introducing new functionality will also include appropriate unit tests to exercise those changes.

[linters]: https://en.wikipedia.org/wiki/Lint_(software)
[pre-commit]: https://pre-commit.com/
[hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
[ruff]: https://github.com/astral-sh/ruff
1 change: 0 additions & 1 deletion babel.cfg
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
[python: **.py]

18 changes: 9 additions & 9 deletions docs/esi-leap-api-ref.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ The Offer API endpoint can be reached at /v1/offers.
* The /v1/offers/\<uuid_or_name> endpoint is used to retrieve the offer with the given uuid. The response type is 'application/json'.
* The /v1/offers endpoint is used to retrieve a list of offers. This URL supports several URL variables for retrieving offers filtered by given values. The response type is 'application/json'.
* project_id: Returns all offers with given project_id.
* status: Returns all offers with given status.
* status: Returns all offers with given status.
* This value will default to returning offers with status 'available'.
* This value can be set to 'any' to return offers without filtering by status.
* resource_uuid: Returns all offers with given resource_uuid.
* resource_type: Returns all offers with given resource_type
* start_time and end_time: Passing in values for the start_time and end_time variables will return all offers with a start_time and end_time which completely span the given values. These two URL variables must be used together. Passing in only one will throw an error.
* start_time and end_time: Passing in values for the start_time and end_time variables will return all offers with a start_time and end_time which completely span the given values. These two URL variables must be used together. Passing in only one will throw an error.
* available_start_time and available_end_time: Passing in values for the available_start_time and available_end_time variables will return all offers with availabilities which completely span the given values. These two URL variables must be used together. Passing in only one will throw an error.


Expand All @@ -32,9 +32,9 @@ The Offer API endpoint can be reached at /v1/offers.
* properties:
* a json object
* This field is optional. Not setting it will default to {}.
* 'status', 'uuid', and 'availabilities' are read only.
* 'status', 'uuid', and 'availabilities' are read only.
* The response to a POST request will be the newly created offer. The response type is 'application/json'.

An example curl request is shown below.
```
curl -X POST -sH "X-Auth-Token: $token" http://localhost:7777/v1/offers -H 'Content-Type: application/json' -d '{
Expand All @@ -54,13 +54,13 @@ curl -X POST -sH "X-Auth-Token: $token" http://localhost:7777/v1/offers -H 'Con
"size_gb": 500,
"model": "YOYODYNE 1234"
},
{
{
"size_gb": 1024,
"model": "evo840 ssd"
}
]
}
}'
}'
```

##### DELETE
Expand All @@ -80,10 +80,10 @@ The lease api endpoint can be reached at /v1/leases
* The /v1/leases endpoint is used to retrieve a list of leases. This URL supports several URL variables for retrieving offers filtered by given values. The response type is 'application/json'.
* project_id: Returns all leases with given project_id.
* This value will default to the project_id of the request.
* status: Returns all offers with given status.
* status: Returns all offers with given status.
* This value will default to returning leases with status 'open'.
* This value can be set to 'any' to return leases without filtering by status.
* start_time and end_time: Passing in values for the start_time and end_time variables will return all leases with a start_time and end_time which completely span the given values. These two URL variables must be used together. Passing in only one will throw an error.
* start_time and end_time: Passing in values for the start_time and end_time variables will return all leases with a start_time and end_time which completely span the given values. These two URL variables must be used together. Passing in only one will throw an error.
* owner: Returns all leases which are related to offers with project_id 'owner'.
* view: Setting view to 'all' will return all leases in the database. This value can be used in combination with other filters.

Expand All @@ -101,7 +101,7 @@ The lease api endpoint can be reached at /v1/leases
* offer_uuid_or_name:
* A string.
* This field is required.
* 'status' and 'uuid' are read only.
* 'status' and 'uuid' are read only.
* The response to a POST request will be the newly created lease. The response type is 'application/json'.

An example curl request is shown below.
Expand Down
2 changes: 1 addition & 1 deletion docs/esi-leap-reporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ To generate a csv file with data on all lessees' usage on different nodes:
openstack esi lease list --all --long -f csv > report.csv
```

The ESI-Leap API allows users to retrieve data based on certain parameters.
The ESI-Leap API allows users to retrieve data based on certain parameters.

For example, it is possible to generate a report based on a lessee's OpenStack project_id. To get a list of all
projects type ``openstack project list`` and from there grab the desired project id and run the command:
Expand Down
8 changes: 4 additions & 4 deletions docs/esi-leap-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ An offer response has the following fields:
* deleted: an offer is no longer available for leasing. An offer is set to deleted when it is manually revoked by a user before its end_time has passed.
* "properties" is the baremetal properties of an offer.
* "availabilities" is a list of [start, end] datetime pairings representing a continuous time range in which an offer is available for leasing.
* "availabilities" is not kept in the schema and is computed when retrieving an offer.
* "availabilities" is not kept in the schema and is computed when retrieving an offer.
* "created_at", "updated_at", and "id" are only used in the schema and cannot be read or set.


Expand Down Expand Up @@ -164,11 +164,11 @@ A lease response has the following fields:


### Manager Service
An ESI-Leap manager has periodic jobs to manage offers and leases.
An ESI-Leap manager has periodic jobs to manage offers and leases.
* expire offers: out-of-date offers, i.e, the current timestamp > offer's end_time, will be updated with an 'EXPIRED' status.
* fulfill leases: if a lease's start_time <= the current timestamp and is not expired, the manager service will fulfill the resources in the leases and update the status of the leases to 'active'.
* fulfill leases: if a lease's start_time <= the current timestamp and is not expired, the manager service will fulfill the resources in the leases and update the status of the leases to 'active'.
* expire leases: same as 'expire offers', ESI-Leap will expire leases based on timestamp.
* update offers: after the manager fulfills and expires leases, it will update the relevant offers' status. The offers in a fulfilled lease should be unavailable to others. Likewise, when a lease expires, offers in the lease should be updated and be available again.
* update offers: after the manager fulfills and expires leases, it will update the relevant offers' status. The offers in a fulfilled lease should be unavailable to others. Likewise, when a lease expires, offers in the lease should be updated and be available again.

## Reporting API
ESI-Leap admin queries this API to learn about the usage of nodes given a period. The admin enters a date range to get all leases' information within that range. The results could be like this:
Expand Down
12 changes: 6 additions & 6 deletions esi_leap/api/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ def after(self, state):

def get_pecan_config():
cfg_dict = {
'app': {
'root': CONF.pecan.root,
'modules': CONF.pecan.modules,
'debug': CONF.pecan.debug,
'auth_enable': CONF.pecan.auth_enable
"app": {
"root": CONF.pecan.root,
"modules": CONF.pecan.modules,
"debug": CONF.pecan.debug,
"auth_enable": CONF.pecan.auth_enable,
}
}

Expand All @@ -54,7 +54,7 @@ def setup_app(config=None):
hooks=lambda: [ContextHook()],
debug=CONF.pecan.debug,
static_root=config.app.static_root if CONF.pecan.debug else None,
force_canonical=getattr(config.app, 'force_canonical', True),
force_canonical=getattr(config.app, "force_canonical", True),
)

if CONF.pecan.auth_enable:
Expand Down
1 change: 0 additions & 1 deletion esi_leap/api/controllers/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@


class ESILEAPBase(wtypes.Base):

def to_dict(self):
esi_leap_dict = {}
if self.fields:
Expand Down
20 changes: 9 additions & 11 deletions esi_leap/api/controllers/root.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,27 @@
from esi_leap.api.controllers.v1 import root


API_VERSION = 'v1'
API_VERSION = "v1"


class RootController(rest.RestController):

_versions = [API_VERSION]
_default_version = API_VERSION

v1 = root.Controller()

@pecan.expose(content_type='application/json')
@pecan.expose(content_type="application/json")
def index(self):
pecan.response.status_code = 300
pecan.response.content_type = 'application/json'
pecan.response.content_type = "application/json"
versions = {
'versions': [
"versions": [
{
'id': 'v1.0',
'status': 'CURRENT',
'links': [{
'href': '{0}/v1'.format(pecan.request.host_url),
'rel': 'self'
}]
"id": "v1.0",
"status": "CURRENT",
"links": [
{"href": "{0}/v1".format(pecan.request.host_url), "rel": "self"}
],
}
]
}
Expand Down
26 changes: 13 additions & 13 deletions esi_leap/api/controllers/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,11 @@ class JsonType(wtypes.UserType):
"""A simple JSON type."""

basetype = wtypes.text
name = 'json'
name = "json"

def __str__(self):
# These are the json serializable native types
return ' | '.join(map(str, (wtypes.text, int, float,
bool, list, dict, None)))
return " | ".join(map(str, (wtypes.text, int, float, bool, list, dict, None)))

@staticmethod
def validate(value):
Expand All @@ -38,7 +37,6 @@ def frombasetype(value):


class Collection(wtypes.Base):

@property
def collection(self):
return getattr(self, self._type)
Expand All @@ -52,15 +50,17 @@ def get_next(self, limit, url=None, **kwargs):
return wtypes.Unset

url = url or self._type
q_args = ''.join(['%s=%s&' % item for item in kwargs.items()])
next_args = '?%(args)slimit=%(limit)d&marker=%(marker)s' % {
'args': q_args, 'limit': limit,
'marker': getattr(self.collection[-1], 'uuid')}

next_link = '%(url)s/v1/%(resource)s%(args)s' % {
'url': url,
'resource': self._type,
'args': next_args
q_args = "".join(["%s=%s&" % item for item in kwargs.items()])
next_args = "?%(args)slimit=%(limit)d&marker=%(marker)s" % {
"args": q_args,
"limit": limit,
"marker": getattr(self.collection[-1], "uuid"),
}

next_link = "%(url)s/v1/%(resource)s%(args)s" % {
"url": url,
"resource": self._type,
"args": next_args,
}

return next_link
Expand Down
Loading
Loading