Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance storing events in batches in Azure provider #1561

Merged
merged 5 commits into from
Jan 7, 2025

Conversation

MarcAstr0
Copy link
Collaborator

@MarcAstr0 MarcAstr0 commented Dec 26, 2024

Description

PR #1436 added the ability to store events in batches. However, there are times when one might want to store events individually due to cost or performance reasons. This PR adds the ability to enable or disable batching events via configuration. It also adds more options for the batch inserts, such as setting the size of the chunks.

Changes

  • Added the azureConfiguration property to the BoosterConfig object to set options for batch storage of events.
  • Split the storeEventsmethod into two to handle batch and non-batch scenarios.
  • Updated unit tests.
  • Updated documentation.

Checks

  • Project Builds
  • Project passes tests and checks
  • Updated documentation accordingly

@MarcAstr0 MarcAstr0 added the provider:azure Related to the Azure cloud provider label Dec 26, 2024
@MarcAstr0 MarcAstr0 requested a review from davidverdu December 26, 2024 21:36
Copy link

what-the-diff bot commented Dec 26, 2024

PR Summary

  • Introduction of Event Batching

    • A new capability has been added that allows events to be stored in batches for the Azure provider. This is an operational efficiency improvement that can help manage large volumes of data more effectively.
  • Updated Event Storage

    • The existing method of event storage has been updated to handle event storage either in batches or as individual events. This provides flexibility depending on your data volume and processing needs.
  • New Configuration Structure

    • A new configuration structure in config.ts has been introduced for Azure settings. This includes default settings for event batching and batch size which creates a more user-friendly way to manage configuration.
  • Introduction of 'AzureConfiguration' Interface

    • A new TypeScript file azure-configuration.ts has been created to define the AzureConfiguration interface and a constant for default values. This is the foundation for the new configuration structure.
  • Updated Unit Tests

    • The unit tests have been updated to test the new event batching logic, providing assurance that the new feature works as intended.
  • Added Instruction Documentation

    • A new section was added in the documentation explaining how to use the batch event storage and configure it, making it easier for users to understand and use the new feature.
  • Updates to Navigation

    • The sidebar navigation in the documentation has been updated to include the new event batching topic. This ensures that users can easily find and access this helpful new guide.

@MarcAstr0
Copy link
Collaborator Author

/integration sha=ae2952361626cfc63601e6bdcba56d6191a14ace

Copy link
Contributor

⌛ Integration tests are running...

Check their status here 👈

Copy link
Contributor

✅ Integration tests have finished successfully!

Copy link
Collaborator

@davidverdu davidverdu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MarcAstr0 MarcAstr0 merged commit 7fd8fdf into boostercloud:main Jan 7, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
provider:azure Related to the Azure cloud provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants