Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type annotate storage #362

Merged
merged 5 commits into from
Jun 9, 2021

Conversation

sechkova
Copy link
Contributor

Addresses #358

Description of the changes being introduced by the pull request:

Adds type annotations to storage.py.

In addition changes FilesystemBackend.get to avoid mypy complaining (correctly) about the following assignment:

  # Map our class ContextManager implementation to the function expected of the
  # securesystemslib.storage.StorageBackendInterface.get definition
  get = GetFile
securesystemslib/storage.py:224: error: Incompatible types in assignment (expression has type "Type[GetFile]", base class "StorageBackendInterface" defined the type as "Callable[[StorageBackendInterface, str], Generator[BinaryIO, None, None]]")

Please verify and check that the pull request fulfils the following requirements:

  • The code follows the Code Style Guidelines
  • Tests have been added for the bug fix or new feature
  • Docs have been added for the bug fix or new feature

@sechkova
Copy link
Contributor Author

Mentioning @joshuagl in case he can remember something that I have missed in the context manager implementation.

Copy link
Collaborator

@jku jku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me apart from that one fishy looking exception.

@@ -200,7 +200,7 @@ def get(self, filepath:str) -> Generator[BinaryIO, None, None]:
try:
file_object = open(filepath, 'rb')
yield file_object
except (FileNotFoundError, IOError):
except FileNotFoundError:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this went the wrong way and should be OSError now -- to cover things like IsADirectoryError

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a point, updated in 32ab712

@sechkova sechkova force-pushed the type-annotate-storage branch 2 times, most recently from 8fac1f3 to 32ab712 Compare June 3, 2021 16:14
Copy link
Collaborator

@joshuagl joshuagl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioning @joshuagl in case he can remember something that I have missed in the context manager implementation.

This looks great. It may have been a Python 2.7 limitation or just my not finding the right part of the docs when I implemented it without the decorator.

@@ -37,7 +38,8 @@ class StorageBackendInterface():


@abc.abstractmethod
def get(self, filepath):
@contextmanager
def get(self, filepath: str) -> Generator[BinaryIO, None, None]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if that works (and that would make sense) then I suppose it's Iterator[BinaryIO]?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would make more sense, indeed.

@@ -193,7 +195,7 @@ def __new__(cls, *args, **kwargs):


@contextmanager
def get(self, filepath):
def get(self, filepath:str) -> Generator[BinaryIO, None, None]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment on return type here.

@sechkova
Copy link
Contributor Author

sechkova commented Jun 7, 2021

Added one commit with the proposal. Checked and mypy seems to accept it.

@joshuagl
Copy link
Collaborator

joshuagl commented Jun 7, 2021

Just realised that you'll need to add storage.py to the list of files in mypy.ini

In order to enable static type checking the types of
the abstract method 'get' and its concrete implementation
have to match. Assining GetFile class to a callable leads
to type errors.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Gradually enable static type checking.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Since Python 3.3 IOError is an alias of OSError and is kept for
compatibility. No longer needed since the project supports
only bigger Python versions.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
The @contextmanager decorator returns an Iterator[BinaryIO]
which a generalisation of Generator[BinaryIO, None, None].

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
Add storage.py to the list of type annotated files
checked by mypy.

Signed-off-by: Teodora Sechkova <tsechkova@vmware.com>
@sechkova
Copy link
Contributor Author

sechkova commented Jun 7, 2021

Just realised that you'll need to add storage.py to the list of files in mypy.ini

Rebased on top of the main branch and added.

@sechkova sechkova mentioned this pull request Jun 7, 2021
Copy link
Collaborator

@joshuagl joshuagl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a nice change.

@joshuagl joshuagl merged commit 09027be into secure-systems-lab:master Jun 9, 2021
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants