-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug(?) in prometheus.CounterVec #1429
Comments
I ran into the same issue in our production code and traced it back to |
Hello! Glad to meet someone with a similar problem. Try rewriting the variable into a new variable before passing it to the Prometheus function. I can't explain why, but it seems to be starting to work. I still haven’t been able to figure out the reasons why this doesn’t work( By the way, I couldn’t find in the source code where this map could change |
Hey @Nicik, unfortunately no luck with variable passing, still the same issue. I'll try to dig up more, it would be nice if we can get some clarity on why and where the previously added metric gets updated with new label value. |
@Nicik, you are using Fiber, in Fiber http parameters, body unsafe and mutable. Set |
This saved me a lot of time! For those interested the immutable issue can affect also other aspects. Allow me to share my experience with the issue trying to be helpful to others as well. I struggled for a whole day trying to figure out what was going wrong with a bogus metric that I was collecting with this client and fiber! I have a Both copying the value or configure fiber to work with |
Hello everyone! I am faced with the problem of re-registering metrics. I have seen the last occurrence of a similar problem in #242.
Link to the repository with the sample code: https://github.com/niciki/prom_bug/tree/main
OS: MacOS 14.1.1 M1 Pro,
Golang: 1.21
client_golang: 1.18.0
I am creating the following metric:
and I plan to count the access to the method with a different, small number of different ids.
There is a problem with the following requests at the same time:
and after them I try get metrics:
and I get an error of about the following type:
I register this metric once and therefore this error caught me by surprise. I tried to make test conclusions of the variable id transmitted by passing it to the function
![image](https://private-user-images.githubusercontent.com/71276784/295279858-84fb6ae1-9092-451e-aba6-4195b90a482d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ0MzU4MTUsIm5iZiI6MTcyNDQzNTUxNSwicGF0aCI6Ii83MTI3Njc4NC8yOTUyNzk4NTgtODRmYjZhZTEtOTA5Mi00NTFlLWFiYTYtNDE5NWI5MGE0ODJkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDE3NTE1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE3ZTYzMWE4MzZkODI0MzMyMWUzYjhiYjBmMzc1YWJlOGQwMDQ3YzY5ZGE3M2UxNzVmM2JlZDY5OTEwNTMyZWQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.0lLsOFM1cigl7G77Xq29mpfCMPuKLY0nyZi8s9rPsT8)
![image](https://private-user-images.githubusercontent.com/71276784/295281214-e9d48942-880c-4e52-8ef6-6bfe2968f6a9.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ0MzU4MTUsIm5iZiI6MTcyNDQzNTUxNSwicGF0aCI6Ii83MTI3Njc4NC8yOTUyODEyMTQtZTlkNDg5NDItODgwYy00ZTUyLThlZjYtNmJmZTI5NjhmNmE5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDE3NTE1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTc1Yjc3NTk5YjcyOGFhNTZkZjA1NDU3Y2M3MWU4M2RhOGYzMzc2NDkzMTNkOGM2MGVmZTYzMDc1YmJhYTMzYjImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.dIEd7PJuToRKWvnV6VaiCv_jhcnQ2eIsbQTpECrHN6k)
sm.WithLabelValues(id).Add(1)
, thinking that I was having some kind of error, but no, everything was correct. This may be useful, but I was interested in digging into the prometheus source code and I noticed the following.I found that it looks like an array that stores hashes and label values is being changed:
it seems that the number that corresponds to its hash is changing, but I could not understand the possible reason for this behavior:
The text was updated successfully, but these errors were encountered: