Pubsub feature: get messages count waiting in subscription #527

pierre-b · 2015-05-04T15:34:57Z

Hi,

I need to know how many messages are waiting in the subscription, to pop new workers or monitor the health of my system. This is something AWS SQS does too.

Is it in the roadmap actually ?

Thank you

pierre-b · 2015-05-05T13:01:58Z

FYI: as https://code.google.com/p/cloud-pubsub/issues/detail?id=13 told me, there is a "num_undelivered_messages" endpoint to get that metric.

ryanseys · 2015-05-05T13:15:27Z

Considering it's "not accurate" I wouldn't use it yet and don't think we should support it in this library until we know it is accurate.

pierre-b · 2015-05-05T14:21:03Z

This feature could never be accurate because messages are always flowing into the subscription (depending on the publish rate), but it's crucial to know if you need to add more workers to process messages or monitor the health of your workers.

stephenplusplus · 2015-05-05T15:07:24Z

From https://cloud.google.com/monitoring/api/metrics, they list two interesting ones:

num_outstanding_messages: Current number of messages delivered but not yet acknowledged.
num_undelivered_messages: Current number of messages pending to be delivered.

Maybe we could open up a generic subscription.getMetrics('num_outstanding_msesages') and anything else from the list: topic.getMetrics('message_sizes'), as these may all be useful.

stephenplusplus · 2015-08-24T13:43:57Z

Assigned this to @callmehiphop since you had some ideas for how to implement this generically.

leibale · 2015-12-10T17:54:58Z

what about this one?

stephenplusplus · 2015-12-11T15:03:52Z

We have a few more things to implement with a higher priority before getting to this one, mostly support for the Prediction and Translate APIs, then promises.

leibale · 2015-12-12T16:03:40Z

@stephenplusplus if i will create a pull request, you will merge?

stephenplusplus · 2015-12-12T17:09:50Z

That would be great!

sahas- · 2017-03-31T00:16:02Z

Great to see you guys working on this already. If its worth, right now I use stackdriver's subscription metrics capabilities to get this information, apparently, stackdriver lags behind and the count is not accurate. Just wanted to add +1 to this request and happy to be a beta tester, if need be. Let me know.

mpcm · 2017-04-06T02:14:58Z

Adding link to the current related google doc page, for others like me that ended up here looking to learn about msg depth in a sub: https://cloud.google.com/pubsub/docs/ordering#order_of_processed_messages_matters

stephenplusplus · 2017-11-27T15:37:27Z

@callmehiphop given the large Pub/Sub rewrite that happened since this issue was opened, is this request still relevant / do you have an idea for a reliable implementation?

callmehiphop · 2017-11-27T17:28:05Z

@stephenplusplus AFAIK the re-write hasn't changed anything in regard to collecting these metrics.

stephenplusplus · 2017-12-13T03:37:03Z

This issue was moved to googleapis/nodejs-pubsub#8

* build!: Update library to use Node 12 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * dep: update gts * dep: update google-gax Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* chore: migrate to owl bot * chore: copy files from googleapis-gen fb91803ccef5d7c695139b22788b309e2197856b * chore: run the post processor

…andwritten libraries (#527) - [ ] Regenerate this pull request now. PiperOrigin-RevId: 429395631 Source-Link: googleapis/googleapis@84594b3 Source-Link: googleapis/googleapis-gen@ed74f97 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWQ3NGY5NzBmZDgyOTE0ODc0ZTZiMjdiMDQ3NjNjZmE2NmJhZmU5YiJ9

* chore: migrate to owl bot * chore: copy files from googleapis-gen fb91803ccef5d7c695139b22788b309e2197856b * chore: run the post processor

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/73563d93-aea4-4354-9013-d19800d55cda/targets

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped) | [`^12.0.0` -> `^14.0.0`](https://renovatebot.com/diffs/npm/@types%2fnode/12.20.13/14.17.0) | [![age](https://badges.renovateapi.com/packages/npm/@types%2fnode/14.17.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/@types%2fnode/14.17.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/@types%2fnode/14.17.0/compatibility-slim/12.20.13)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/@types%2fnode/14.17.0/confidence-slim/12.20.13)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: "after 9am and before 3pm" (UTC). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻️ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-tasks).

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [sinon](https://sinonjs.org/) ([source](https://github.com/sinonjs/sinon)) | [`^11.0.0` -> `^12.0.0`](https://renovatebot.com/diffs/npm/sinon/11.1.2/12.0.1) | [![age](https://badges.renovateapi.com/packages/npm/sinon/12.0.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/sinon/12.0.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/sinon/12.0.1/compatibility-slim/11.1.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/sinon/12.0.1/confidence-slim/11.1.2)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>sinonjs/sinon</summary> ### [`v12.0.1`](https://github.com/sinonjs/sinon/blob/master/CHANGES.md#1201) [Compare Source](https://github.com/sinonjs/sinon/compare/v12.0.0...v12.0.1) - [`3f598221`](https://github.com/sinonjs/sinon/commit/3f598221045904681f2b3b3ba1df617ed5e230e3) Fix issue with npm unlink for npm version > 6 (Carl-Erik Kopseng) > 'npm unlink' would implicitly unlink the current dir > until version 7, which requires an argument - [`51417a38`](https://github.com/sinonjs/sinon/commit/51417a38111eeeb7cd14338bfb762cc2df487e1b) Fix bundling of cjs module ([#2412](https://github.com/sinonjs/sinon/issues/2412)) (Julian Grinblat) > - Fix bundling of cjs module > > - Run prettier *Released by [Carl-Erik Kopseng](https://github.com/fatso83) on 2021-11-04.* #### 12.0.0 ### [`v12.0.0`](https://github.com/sinonjs/sinon/compare/v11.1.2...v12.0.0) [Compare Source](https://github.com/sinonjs/sinon/compare/v11.1.2...v12.0.0) </details> --- ### Configuration 📅 **Schedule**: "after 9am and before 3pm" (UTC). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-iot).

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [@types/sinon](https://github.com/DefinitelyTyped/DefinitelyTyped) | [`^9.0.0` -> `^10.0.0`](https://renovatebot.com/diffs/npm/@types%2fsinon/9.0.11/10.0.0) | [![age](https://badges.renovateapi.com/packages/npm/@types%2fsinon/10.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/@types%2fsinon/10.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/@types%2fsinon/10.0.0/compatibility-slim/9.0.11)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/@types%2fsinon/10.0.0/confidence-slim/9.0.11)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration :date: **Schedule**: "after 9am and before 3pm" (UTC). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-monitoring).

* updated CHANGELOG.md [ci skip] * updated package.json [ci skip] * updated samples/package.json [ci skip]

* chore(deps): upgrade gapic-generator-java to 2.4.1 PiperOrigin-RevId: 422607515 Source-Link: googleapis/googleapis@ba2ffd6 Source-Link: googleapis/googleapis-gen@73ba4ad Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNzNiYTRhZGQyMzlhNjE5ZGE1NjdmZmJkNGU1NzMwZmRkNmRlMDRkMyJ9 * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: sofisl <55454395+sofisl@users.noreply.github.com>

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/c36c6dbc-ab79-4f17-b70b-523b420b2a70/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@5747555

…ript generator. (#527) Also removing the explicit generator tag for the IAMPolicy mixin for the kms and pubsub APIS as the generator will now read it from the .yaml file. PiperOrigin-RevId: 385101839 Source-Link: googleapis/googleapis@80f4042 Source-Link: googleapis/googleapis-gen@d3509d2

stephenplusplus added the api: pubsub Issues related to the Pub/Sub API. label May 5, 2015

stephenplusplus added the enhancement label May 5, 2015

stephenplusplus assigned callmehiphop Aug 24, 2015

stephenplusplus mentioned this issue Nov 3, 2015

Query usage stats #924

Closed

landrito added priority: p2 Moderately-important priority. Fix may not be included in next release. Status: Acknowledged labels Mar 7, 2017

Danack mentioned this issue Aug 16, 2017

Monitoring pubsub queues without extension googleapis/google-cloud-php#635

Closed

stephenplusplus mentioned this issue Dec 13, 2017

get messages count waiting in subscription googleapis/nodejs-pubsub#8

Closed

stephenplusplus closed this as completed Dec 13, 2017

stephenplusplus removed priority: p2 Moderately-important priority. Fix may not be included in next release. type: enhancement labels Dec 13, 2017

sofisl mentioned this issue Sep 16, 2022

migrate code from googleapis/nodejs-os-login #3366

Closed

4 tasks

sofisl pushed a commit that referenced this issue Sep 27, 2022

chore: migrate to owl bot (#527)

0649123

* chore: migrate to owl bot * chore: copy files from googleapis-gen fb91803ccef5d7c695139b22788b309e2197856b * chore: run the post processor

sofisl mentioned this issue Sep 27, 2022

migrate code from googleapis/nodejs-video-intelligence #3374

Closed

4 tasks

This was referenced Oct 11, 2022

chore: release main #3393

Closed

chore: release main #3394

Closed

chore: release main #3397

Closed

sofisl mentioned this issue Oct 12, 2022

migrate code from googleapis/nodejs-bigquery-data-transfer #3399

Merged

4 tasks

This was referenced Oct 12, 2022

chore: release main #3400

Closed

chore: release main #3403

Closed

chore: release main #3407

Closed

sofisl pushed a commit that referenced this issue Nov 9, 2022

chore: migrate to owl bot (#527)

1e6ed45

* chore: migrate to owl bot * chore: copy files from googleapis-gen fb91803ccef5d7c695139b22788b309e2197856b * chore: run the post processor

This was referenced Nov 9, 2022

migrate code from googleapis/nodejs-video-intelligence #3507

Closed

migrate code from googleapis/nodejs-video-intelligence #3519

Merged

migrate code from googleapis/nodejs-translate #3520

Merged

sofisl mentioned this issue Nov 10, 2022

migrate code from googleapis/nodejs-tasks #3530

Merged

4 tasks

This was referenced Nov 11, 2022

migrate code from googleapis/nodejs-iot #3544

Merged

migrate code from googleapis/nodejs-monitoring #3545

Merged

sofisl mentioned this issue Nov 11, 2022

migrate code from googleapis/nodejs-dialogflow #3546

Merged

4 tasks

sofisl pushed a commit that referenced this issue Nov 11, 2022

chore: release 1.2.0 (#527)

5636667

* updated CHANGELOG.md [ci skip] * updated package.json [ci skip] * updated samples/package.json [ci skip]

sofisl mentioned this issue Nov 11, 2022

migrate code from googleapis/nodejs-kms #3601

Merged

4 tasks

sofisl mentioned this issue Jan 10, 2023

migrate code from googleapis/nodejs-dlp #3837

Merged

4 tasks

sofisl mentioned this issue Sep 13, 2023

migrate code from googleapis/nodejs-automl #4655

Closed

4 tasks

sofisl mentioned this issue Sep 14, 2023

migrate code from googleapis/nodejs-automl #4656

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pubsub feature: get messages count waiting in subscription #527

Pubsub feature: get messages count waiting in subscription #527

pierre-b commented May 4, 2015

pierre-b commented May 5, 2015

ryanseys commented May 5, 2015

pierre-b commented May 5, 2015

stephenplusplus commented May 5, 2015

stephenplusplus commented Aug 24, 2015

leibale commented Dec 10, 2015

stephenplusplus commented Dec 11, 2015

leibale commented Dec 12, 2015

stephenplusplus commented Dec 12, 2015

sahas- commented Mar 31, 2017

mpcm commented Apr 6, 2017 •

edited

Loading

stephenplusplus commented Nov 27, 2017

callmehiphop commented Nov 27, 2017

stephenplusplus commented Dec 13, 2017

Pubsub feature: get messages count waiting in subscription #527

Pubsub feature: get messages count waiting in subscription #527

Comments

pierre-b commented May 4, 2015

pierre-b commented May 5, 2015

ryanseys commented May 5, 2015

pierre-b commented May 5, 2015

stephenplusplus commented May 5, 2015

stephenplusplus commented Aug 24, 2015

leibale commented Dec 10, 2015

stephenplusplus commented Dec 11, 2015

leibale commented Dec 12, 2015

stephenplusplus commented Dec 12, 2015

sahas- commented Mar 31, 2017

mpcm commented Apr 6, 2017 • edited Loading

stephenplusplus commented Nov 27, 2017

callmehiphop commented Nov 27, 2017

stephenplusplus commented Dec 13, 2017

mpcm commented Apr 6, 2017 •

edited

Loading