Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨Rabbitmq: use TOPIC Exchange for logs #4166

Merged

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Apr 25, 2023

What do these changes do?

  • adds simcore.services.logs.v2 RabbitMQ Exchange of type TOPIC
  • keeps simcore.services.logs RabbitMQ Exchange of type FANOUT
  • ensure backward compatibility (declaring exchanges/queues with the same name but different parameters fails) - test added in test_rabbitmq to show issue code
  • slight refactoring of publish function and of rabbit messages (going towards not using pydantic for this)

The next PR(s) will bring

  • only connect webserver if project is open
  • ensure only 1 webserver handles progress updates

Related issue/s

How to test

DevOps Checklist

@sanderegg sanderegg added the a:services-library issues on packages/service-libs label Apr 25, 2023
@sanderegg sanderegg added this to the Jelly Beans milestone Apr 25, 2023
@sanderegg sanderegg self-assigned this Apr 25, 2023
@codecov
Copy link

codecov bot commented Apr 25, 2023

Codecov Report

Merging #4166 (dcf4d42) into master (beedbd4) will increase coverage by 1.9%.
The diff coverage is 78.1%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #4166     +/-   ##
========================================
+ Coverage    82.7%   84.6%   +1.9%     
========================================
  Files         958     958             
  Lines       41560   41591     +31     
  Branches      955     955             
========================================
+ Hits        34396   35221    +825     
+ Misses       6947    6153    -794     
  Partials      217     217             
Flag Coverage Δ
integrationtests 67.2% <83.3%> (-0.1%) ⬇️
unittests 81.4% <62.5%> (+2.0%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...e_service_director_v2/modules/projects_networks.py 75.2% <0.0%> (ø)
...ice_webserver/director_v2_core_dynamic_services.py 70.1% <0.0%> (ø)
...ls-library/src/models_library/rabbitmq_messages.py 90.4% <57.8%> (-8.1%) ⬇️
...rector_v2/modules/comp_scheduler/dask_scheduler.py 88.5% <66.6%> (ø)
...ackages/service-library/src/servicelib/rabbitmq.py 94.4% <81.2%> (-1.3%) ⬇️
...s/models-library/src/models_library/basic_types.py 97.2% <100.0%> (+0.3%) ⬆️
...rc/simcore_service_autoscaling/modules/rabbitmq.py 100.0% <100.0%> (ø)
...rector_v2/modules/comp_scheduler/base_scheduler.py 91.9% <100.0%> (ø)
...modules/dynamic_sidecar/scheduler/_core/_events.py 97.2% <100.0%> (+<0.1%) ⬆️
...s/dynamic_sidecar/scheduler/_core/_events_utils.py 91.6% <100.0%> (-1.8%) ⬇️
... and 2 more

... and 72 files with indirect coverage changes

@sanderegg sanderegg closed this Apr 27, 2023
@sanderegg sanderegg force-pushed the rabbitmq/use_only_topic_queues branch from c6c5234 to beb9630 Compare April 27, 2023 07:51
@sanderegg sanderegg reopened this Apr 27, 2023
@sanderegg sanderegg force-pushed the rabbitmq/use_only_topic_queues branch from 2785bd7 to d31951b Compare April 27, 2023 14:28
@sanderegg sanderegg added a:webserver issue related to the webserver service a:director-v2 issue related with the director-v2 service a:dynamic-sidecar dynamic-sidecar service labels Apr 27, 2023
@sanderegg sanderegg marked this pull request as ready for review April 27, 2023 17:01
@sanderegg sanderegg changed the title ✨Rabbitmq: use only topic queues instead of FANOUT ✨Rabbitmq: use TOPIC Exchange for logs Apr 27, 2023
@sanderegg sanderegg force-pushed the rabbitmq/use_only_topic_queues branch from 4197f56 to 3bc7ae3 Compare April 28, 2023 09:37
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great job!

Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@sanderegg sanderegg force-pushed the rabbitmq/use_only_topic_queues branch from c881825 to dcf4d42 Compare May 4, 2023 20:27
@codeclimate
Copy link

codeclimate bot commented May 4, 2023

Code Climate has analyzed commit dcf4d42 and detected 0 issues on this pull request.

View more on Code Climate.

@sonarqubecloud
Copy link

sonarqubecloud bot commented May 4, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sanderegg sanderegg merged commit 04af546 into ITISFoundation:master May 4, 2023
@sanderegg sanderegg deleted the rabbitmq/use_only_topic_queues branch May 4, 2023 21:22
@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 30, 2023
24 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:director-v2 issue related with the director-v2 service a:dynamic-sidecar dynamic-sidecar service a:services-library issues on packages/service-libs a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants