Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy: billing events sent to Vector and S3 should use the same idempotency key #8564

Closed
stradig opened this issue Jul 31, 2024 · 1 comment
Closed

Proxy: billing events sent to Vector and S3 should use the same idempotency key #8564

stradig opened this issue Jul 31, 2024 · 1 comment
Assignees
@cloneable
Labels
c/proxy Component: proxy

Comments

@stradig
Copy link
Contributor

stradig commented Jul 31, 2024 •
edited by Daniel-ef
Loading

The idempotency key is used to make sure that events are not recorded more than once, although they may be submitted multiple times in some cases to make sure they are not lost.

The background jobs that send the billing events to S3 and Vector are distinct. The idempotency key is generated in those jobs. We should move the generation to an earlier stage to fix that.

Relates: https://github.com/neondatabase/cloud/issues/9826
@Daniel-ef Daniel-ef changed the title Billing events sent to Vector and S3 should use the same idempotency key Proxy: billing events sent to Vector and S3 should use the same idempotency key Aug 5, 2024
@Daniel-ef Daniel-ef added the c/proxy Component: proxy label Aug 5, 2024
@cloneable cloneable self-assigned this Nov 29, 2024
@cloneable cloneable mentioned this issue Nov 29, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Nov 30, 2024
@cloneable
Merge the consumption metric pushes (#9939)
4abc8e5 
#8564

## Problem

The main and backup consumption metric pushes are completely
independent,
resulting in different event time windows and different idempotency
keys.

## Summary of changes

* Merge the push tasks, but keep chunks the same size.
awarus pushed a commit that referenced this issue Dec 5, 2024
@cloneable @awarus
Merge the consumption metric pushes (#9939)
8b7e9ed 
#8564

## Problem

The main and backup consumption metric pushes are completely
independent,
resulting in different event time windows and different idempotency
keys.

## Summary of changes

* Merge the push tasks, but keep chunks the same size.
@cloneable
Copy link
Contributor

Since today all regions have the new proxy release. The still running old releases will stop pushing to S3 within 24h, so there won't be inconsistent keys from them, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cloneable cloneable

Labels
c/proxy Component: proxy
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cloneable @Daniel-ef @stradig