Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete s3 Objects and items related to an execution #173

Merged
merged 9 commits into from
Jul 3, 2024

Conversation

caviri
Copy link
Contributor

@caviri caviri commented Jun 30, 2024

These methods allows to delete all items related to an execution including s3 objects. It updates the digital twin document removing the reference to the execution document.

@caviri caviri requested a review from sabinem June 30, 2024 08:39
Copy link
Contributor

@sabinem sabinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caviri just some small change request: the most important one: make providing the project-path optional.

odtp/cli/execution.py Outdated Show resolved Hide resolved
odtp/storage.py Outdated Show resolved Hide resolved
@caviri
Copy link
Contributor Author

caviri commented Jul 2, 2024

Hello @sabinem, I addressed your suggestions. Would you make to take another look?

@caviri caviri requested a review from sabinem July 2, 2024 04:19
@sabinem
Copy link
Contributor

sabinem commented Jul 2, 2024

@caviri I don't think you made the project-path optional. See screenshot:
Screenshot 2024-07-02 at 11 31 36

@sabinem sabinem changed the title Delete s3 Objects and items related to an execution release 0.4.0 Delete s3 Objects and items related to an execution Jul 2, 2024
@caviri caviri force-pushed the origing/feat-deletion-cli branch from dab4682 to a3da51c Compare July 2, 2024 14:05
@sabinem sabinem changed the title release 0.4.0 Delete s3 Objects and items related to an execution Delete s3 Objects and items related to an execution Jul 2, 2024
Copy link
Contributor

@sabinem sabinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caviri I approve now.

But I would still suggest that you improve a bit on the output of the command:

/app # odtp execution delete --execution-id 6683ed98b66b18232839f9d0
odtp/6683ed98b66b18232839f9d1/
odtp/6683ed98b66b18232839f9d1/odtp-output.zip

Could you please improve the output: I would rather expect a message here that the execution about what exactly has been deleted.

I left some comments in the code on this issue: the log statements write now to file for the execution commands, therefore you need to use print statements in the cli command for both exception and success cases.

Please remove the print from storage.py.

TODOs:

  • remove print in storage.py
  • add print in cli/execution.py and use log.exception instead of log.error since it gives you a traceback.

log.error(f"ERROR: Delete execution failed: {e}")
raise typer.Abort()
else:
log.info("SUCCESS: execution has been deleted")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not get this message in my tests. I got this instead:

odtp/6683af7d5923cfd2e73e613a/
odtp/6683af7d5923cfd2e73e613a/odtp-output.zip

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with the last rebase from develop and it works fine:

03/07/2024 10:42:20 AM - [storage:INFO] 140 storage.py deletePaths - Path 'odtp/66852a00d5b566c9b0c1b4cd' deleted from S3 bucket
03/07/2024 10:42:21 AM - [environment:INFO] 72 environment.py delete_folder - Folder deleted: /Users/carlosvivarrios/pro/ODTP/odtp-instance/dts/dt-example/dt-example/execution
03/07/2024 10:42:21 AM - [execution:INFO] 165 execution.py delete - SUCCESS: execution has been deleted
SUCCESS: execution has been deleted

odtp/storage.py Outdated Show resolved Hide resolved
log.error(f"ERROR: Delete execution failed: {e}")
raise typer.Abort()
else:
log.info("SUCCESS: execution has been deleted")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add print statements in the cli command, since the log statement will only go to file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rewiew @sabinem, I see the logic now.

I'll follow this, but this should be change as In the future I would like to use print statements in the CLI to pipe CLI commands (Not implemented yet).

Ideally only log.X() should go to the log file, otherwise it's a bit confusing.

odtp/cli/execution.py Outdated Show resolved Hide resolved
@caviri caviri force-pushed the origing/feat-deletion-cli branch from dd54145 to 0421d18 Compare July 3, 2024 10:36
@caviri caviri merged commit 398db44 into develop Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants