Skip to content

Commit

Permalink
refactor(datasource): add file caching
Browse files Browse the repository at this point in the history
Add file caching to local file repository for retrieved files.
  • Loading branch information
wvenialbo committed Oct 25, 2024
1 parent 81db3be commit 3f02562
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion src/GOES_DL/datasource/datasource_aws.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,13 +180,20 @@ def get_file(self, file_path: str) -> bytes:
RuntimeError
If the file cannot be retrieved.
"""
local_file = self.repository.get_item(file_path)

if local_file is not None:
return local_file

folder_path: str = self._get_item_path(file_path)

try:
response = self.s3_client.get_object(
Bucket=self.bucket_name, Key=folder_path
)
return response["Body"].read()
content = response["Body"].read()
self.repository.add_item(file_path, content)
return content

except ClientError as exc:
message: str = f"Unable to retrieve the file '{file_path}': {exc}"
Expand Down

0 comments on commit 3f02562

Please sign in to comment.