♻️ introducing structured logging (part 2) (⚠️ devops)

[matusdrobuliak66]

What do these changes do?

• Continuation on ♻️ introducing structured logging (part 1) (⚠️ devops) #4114
• Normalizing log format across the whole Python stack (excluding Director-v0 and osparc-gateway-server)
• Introducing env variable LOG_FORMAT_LOCAL_DEV_ENABLED
  • Reasoning:
  • new log format double escapes new line character to preserve it as one log message, which is then parsed back in the Graylog to be nicely human readable
  • when LOG_FORMAT_LOCAL_DEV_ENABLED is enabled during local development, log messages are not double escaped + coloring of log_level is introduced for a better experience during local development

Related issue/s

• Maintenance: revisit logging in osparc #4079
• ♻️ introducing structured logging (part 1) (⚠️ devops) #4114
• closes Normalize log messages format in all stack services #2157

How to test

DevOps Checklist

• ⚠️ LOG_FORMAT_LOCAL_DEV_ENABLED needs to be added to the template.env (ops-environmnets)
• ⚠️ LOG_FORMAT_LOCAL_DEV_ENABLED=0 needs to be added to the config file in all deployments (ops-deployment-configuration)

[codecov]

Codecov Report

Merging #4136 (e741070) into master (9ef30c9) will decrease coverage by 2.4%.
The diff coverage is 90.4%.

@@           Coverage Diff            @@
##           master   #4136     +/-   ##
========================================
- Coverage    88.3%   85.9%   -2.4%     
========================================
  Files         641     796    +155     
  Lines       29346   35587   +6241     
  Branches      602     534     -68     
========================================
+ Hits        25925   30591   +4666     
- Misses       3284    4867   +1583     
+ Partials      137     129      -8     

Flag	Coverage Δ
integrationtests	66.5% <84.6%> (-0.1%)	⬇️
unittests	82.3% <90.4%> (-1.7%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ations/src/simcore_service_invitations/web_main.py	0.0% <0.0%> (ø)
...es/web/server/src/simcore_service_webserver/cli.py	47.0% <ø> (ø)
...es/service-library/src/servicelib/logging_utils.py	73.4% <28.5%> (ø)
...ervices/storage/src/simcore_service_storage/cli.py	48.0% <50.0%> (+0.1%)	⬆️
...gent/src/simcore_service_agent/core/application.py	100.0% <100.0%> (ø)
...s/agent/src/simcore_service_agent/core/settings.py	94.1% <100.0%> (+0.1%)	⬆️
...src/simcore_service_api_server/core/application.py	96.5% <100.0%> (ø)
...er/src/simcore_service_api_server/core/settings.py	100.0% <100.0%> (ø)
...g/src/simcore_service_autoscaling/core/settings.py	100.0% <100.0%> (ø)
...utoscaling/src/simcore_service_autoscaling/main.py	100.0% <100.0%> (ø)
... and 19 more

... and 459 files with indirect coverage changes

[sanderegg]

thanks. a few comments. This will help a lot

[mrnicegyu11]

looks good from my side, thanks a lot

[pcrespov]

thx!
please double check with ops the issue I raise about .env-devel

[codeclimate]

Code Climate has analyzed commit e741070 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication