CI Failure (retries) in rptest.tests.e2e_iam_role_test
#11107
Comments
twmb
changed the title
STSRoleFetchTests.test_writerptest.tests.e2e_iam_role_test
|
Nearly identical error below in same build ==================================================================================================== Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
And below in AWSRoleFetchTests.test_write |
|
At least for my PR, this is an ASAN violation: And decoding that looks like: Which is this line: redpanda/src/v/cloud_roles/probe.cc Line 34 in 09e59b9 |
|
Possibly related to our replicated metrics patch? /cc @VladLazar @dotnwat |
|
This test actual exercises the cloud roles code-paths, and implicitly the probe. While it's possible that something's wrong with the metrics patches, I'd expect that to crop up more often and in various places. I'd take a good look at the lifetime of this probe first. |
|
I'm still learning about how the probes and seastar metrics work fully, but it seems like the metrics are supposed to be unregistered when the dtor is called for the probe right? I'm not sure how the probe's lifetime could cause anything here. |
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Limit the lifetime of the metrics to the async lifetime of `refresh_credentials`. Fixes redpanda-data#11095 Fixes redpanda-data#11107 Signed-off-by: Ben Pope <ben@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Probes register metrics by capturing `this`, and it's not save to move them after that. In a lot of places this is safe because their lifetime is directly tied to a service which lives the whole program's lifetime, but any small move of that object even during initialization can break things (see redpanda-data#11155, redpanda-data#11095, redpanda-data#11107). This takes a big hammer approach to removing this foot gun by making all probes immovable and wrapping them in `std::unique_ptr`. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
https://buildkite.com/redpanda/redpanda/builds/30205#01886e06-e1ae-4b62-bd19-fe4c80a7082c
The text was updated successfully, but these errors were encountered: