Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ Refactor exporter functionality Part1/2 (⚠️ devops) #4258

Merged

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented May 22, 2023

⚠️ DEVOPS

No intervention from devops should be required

NOTE: the WEBSERVER_EXPORTER is set as null(disabled) on all deployments. The only env var that I have added is EXPORTER_ENABLED which defaults to False. This should not be a problem.

Removed env vars:

  • EXPORTER_MAX_UPLOAD_FILE_SIZE no longer used
  • EXPORTER_DOWNLOADER_MAX_TIMEOUT_SECONDS no longer used
    Added env vars:
  • WEBSERVER_EXPORTER default to False

What do these changes do?

This is a two part series where the webserver's exporter module into will become the sdsmodule. Note: SDS = SPARC Dataset Standard

Changes:

  • code that handled the packaging of data from workspace input and output ports was removed
  • renamed a few of the internals of the package
  • restructured the package's internals
  • removed all unused code and tests

NOTE: renaming of the module of and of the env vars, together with the mypy fixes, will come with the second part

Related issue/s

How to test

DevOps Checklist

@GitHK GitHK self-assigned this May 22, 2023
@GitHK GitHK added a:webserver issue related to the webserver service changelog:♻️refactor labels May 22, 2023
@GitHK GitHK added this to the Pastel de Nata milestone May 22, 2023
@codecov
Copy link

codecov bot commented May 22, 2023

Codecov Report

Merging #4258 (b6850fd) into master (a6cfd3e) will increase coverage by 9.7%.
The diff coverage is 95.3%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #4258      +/-   ##
=========================================
+ Coverage    74.9%   84.6%    +9.7%     
=========================================
  Files         789     389     -400     
  Lines       31968   20234   -11734     
  Branches      966     173     -793     
=========================================
- Hits        23956   17127    -6829     
+ Misses       7804    3058    -4746     
+ Partials      208      49     -159     
Flag Coverage Δ
integrationtests 67.9% <87.5%> (?)
unittests 85.9% <75.9%> (+10.9%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...core_service_webserver/exporter/_formatter/_sds.py 56.3% <73.3%> (ø)
...ervice_webserver/exporter/_formatter/xlsx/utils.py 92.3% <92.3%> (ø)
...rc/simcore_service_webserver/exporter/_handlers.py 97.2% <97.2%> (ø)
...rvice_webserver/exporter/_formatter/_text_files.py 64.7% <100.0%> (ø)
...e_service_webserver/exporter/_formatter/archive.py 100.0% <100.0%> (ø)
...erver/exporter/_formatter/xlsx/code_description.py 100.0% <100.0%> (ø)
...xporter/_formatter/xlsx/core/styling_components.py 100.0% <100.0%> (ø)
...bserver/exporter/_formatter/xlsx/core/xlsx_base.py 92.3% <100.0%> (ø)
...er/exporter/_formatter/xlsx/dataset_description.py 100.0% <100.0%> (ø)
...ver/exporter/_formatter/xlsx/directory_manifest.py 100.0% <100.0%> (ø)
... and 6 more

... and 735 files with indirect coverage changes

@GitHK GitHK marked this pull request as ready for review May 23, 2023 13:12
@GitHK GitHK requested a review from elisabettai May 23, 2023 13:12
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of changes :-)
Add some hints and comments.
I will approve to avoid blocking it too long .
pls follow up on the comments
thx!

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice to delete all that stuff..

@GitHK GitHK enabled auto-merge (squash) June 1, 2023 08:17
@codeclimate
Copy link

codeclimate bot commented Jun 1, 2023

Code Climate has analyzed commit b6850fd and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1

View more on Code Climate.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 1, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 7 Code Smells

No Coverage information No Coverage information
0.1% 0.1% Duplication

@GitHK GitHK merged commit e804317 into ITISFoundation:master Jun 1, 2023
@GitHK GitHK deleted the pr-osparc-cimis-exporter-refactor-1 branch June 1, 2023 09:02
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request Jun 14, 2023
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cMIS export: disable exporting data (inputs/outputs/workspace)
4 participants