Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

We got the error of All keys in the pipeline should belong to the same slots allocation group when enableAutoPipelining set true #1248

Closed
yunnysunny opened this issue Dec 5, 2020 · 3 comments
Closed

We got the error of All keys in the pipeline should belong to the same slots allocation group when enableAutoPipelining set true #1248

yunnysunny opened this issue Dec 5, 2020 · 3 comments
Labels
need detail need reproduce released

Comments

@yunnysunny
Copy link

yunnysunny commented Dec 5, 2020 •
edited
Loading

The error stack is below:

    at Pipeline.exec (/myapp/node_modules/ioredis/built/pipeline.js:250:29)
    at executeAutoPipeline (/myapp/node_modules/ioredis/built/autoPipelining.js:41:14)
    at /myapp/node_modules/ioredis/built/autoPipelining.js:59:13
    at tryCatcher (/myapp/node_modules/standard-as-callback/built/utils.js:11:23)
    at /myapp/node_modules/standard-as-callback/built/index.js:19:49
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

When update our version to 4.19.2 and set enableAutoPipelining true, the error come up. It make us annoying. And then we disabled the enableAutoPipelining, the error dismissing.

We also met with an uncaught exception that make the process exited:

TypeError: Cannot read property 'Symbol(callbacks)' of undefined
    at Immediate.executeAutoPipeline (/myapp/node_modules/ioredis/built/autoPipelining.js:39:31)
    at processImmediate (internal/timers.js:441:21)

It increased our determination to disable enableAutoPipelining.
@simoneb
Copy link

simoneb commented Feb 11, 2021

Can you please provide a reproduction for this issue? A code example would work. Thanks

@Paul-Isache
Copy link

Checked on latest version of ioredis ,4.22.0 down to 4.19.0, with 2 master and 3 slaves, and I am not able to reproduce it.

kemmerer pushed a commit to kemmerer/ioredis that referenced this issue Apr 13, 2021
@mkemmerer
fix(cluster): autopipeline when prefix is used
7494495 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

This may fix redis#1264 and redis#1248.
@kemmerer kemmerer mentioned this issue Apr 13, 2021
Closed
TysonAndre pushed a commit to TysonAndre/ioredis that referenced this issue Jul 27, 2021
@mkemmerer @TysonAndre
fix(cluster): autopipeline when prefix is used
ec6e154 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files

This may fix redis#1264 and redis#1248.
@TysonAndre TysonAndre mentioned this issue Jul 27, 2021
Merged
TysonAndre pushed a commit to TysonAndre/ioredis that referenced this issue Jul 27, 2021
@mkemmerer @TysonAndre
fix(cluster): autopipeline when prefix is used
941d5e5 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files

This may fix redis#1264 and redis#1248.
@TysonAndre TysonAndre mentioned this issue Jul 28, 2021
Closed
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): autopipeline when keyPrefix is used
5e067d6 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): autopipeline when keyPrefix is used
07486a1 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): autopipeline when keyPrefix is used
dffd026 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): autopipeline when keyPrefix is used
2e10f8a 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): autopipeline when keyPrefix is used
e1cab95 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 28, 2021
@TysonAndre
fix(cluster): fix autopipeline with keyPrefix or arg array
32d5125 
Previously the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Similarly, `args[0]` can be a (possibly empty) array which the Command
constructor would flatten. Properly compute the first flattened key when
autopipelining for a Redis.Cluster instance.

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 31, 2021
@TysonAndre
fix(cluster): fix autopipeline with keyPrefix or arg array
36f6387 
Previously, the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Similarly, `args[0]` can be a (possibly empty) array which the Command
constructor would flatten. Properly compute the first flattened key when
autopipelining for a Redis.Cluster instance.

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.

Fixes redis#1392
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Jul 31, 2021
@TysonAndre
fix(cluster): fix autopipeline with keyPrefix or arg array
f116d48 
Previously, the building of pipelines ignored the key prefix.
It was possible that two keys, foo and bar, might be set into
the same pipeline. However, after being prefixed by a configured
"keyPrefix" value, they may no longer belong to the same
pipeline.

This led to the error:
"All keys in the pipeline should belong to the same slots
allocation group"

Similarly, `args[0]` can be a (possibly empty) array which the Command
constructor would flatten. Properly compute the first flattened key when
autopipelining for a Redis.Cluster instance.

Amended version of https://github.com/luin/ioredis/pull/1335/files - see comments on that PR

This may fix redis#1264 and redis#1248.

Fixes redis#1392
@luin luin closed this as completed in d7477aa Aug 1, 2021
ioredis-robot pushed a commit that referenced this issue Aug 1, 2021
@semantic-release-bot
chore(release): 4.27.7 [skip ci]
a13eddc 
## [4.27.7](v4.27.6...v4.27.7) (2021-08-01)

### Bug Fixes

* **cluster:** fix autopipeline with keyPrefix or arg array ([#1391](#1391)) ([d7477aa](d7477aa)), closes [#1264](#1264) [#1248](#1248) [#1392](#1392)
@ioredis-robot
Copy link
Collaborator

🎉 This issue has been resolved in version 4.27.7 🎉

The release is available on:

Your semantic-release bot 📦🚀

TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Aug 27, 2021
@TysonAndre
Fix undefined property warning in executeAutoPipeline
4bd4011 
Seen in an uncaughtException handler.

```
err_type=uncaughtException, exiting cause=TypeError: Cannot read property 'Symbol(callbacks)' of undefined
    at Immediate.executeAutoPipeline (.../node_modules/ioredis/built/autoPipelining.js:34:31)
    at Shim.applySegment (.../node_modules/newrelic/lib/shim/shim.js:1430:20)
    at Immediate.wrapper (.../node_modules/newrelic/lib/shim/shim.js:2097:17)
    at processImmediate (internal/timers.js:463:21)
```

A similar error was among the errors reported in
redis#1248

I suspect the async nature of the delete call may cause it to be
deleted.
TysonAndre added a commit to TysonAndre/ioredis that referenced this issue Aug 27, 2021
@TysonAndre
Fix undefined property warning in executeAutoPipeline
a8986c5 
Seen in an uncaughtException handler.

```
err_type=uncaughtException, exiting cause=TypeError: Cannot read property 'Symbol(callbacks)' of undefined
    at Immediate.executeAutoPipeline (.../node_modules/ioredis/built/autoPipelining.js:34:31)
    at Shim.applySegment (.../node_modules/newrelic/lib/shim/shim.js:1430:20)
    at Immediate.wrapper (.../node_modules/newrelic/lib/shim/shim.js:2097:17)
    at processImmediate (internal/timers.js:463:21)
```

A similar error was among the errors reported in
redis#1248

I suspect the process.exec might call the callback synchronously in the
same tick in some cases, immediately deleting the autopipeline from
running pipelines?
@TysonAndre TysonAndre mentioned this issue Aug 27, 2021
Merged
luin pushed a commit that referenced this issue Aug 30, 2021
@TysonAndre
fix: Fix undefined property warning in executeAutoPipeline (#1425)
f898672 
Seen in an uncaughtException handler.

```
err_type=uncaughtException, exiting cause=TypeError: Cannot read property 'Symbol(callbacks)' of undefined
    at Immediate.executeAutoPipeline (.../node_modules/ioredis/built/autoPipelining.js:34:31)
    at Shim.applySegment (.../node_modules/newrelic/lib/shim/shim.js:1430:20)
    at Immediate.wrapper (.../node_modules/newrelic/lib/shim/shim.js:2097:17)
    at processImmediate (internal/timers.js:463:21)
```

A similar error was among the errors reported in
#1248

I suspect the process.exec might call the callback synchronously in the
same tick in some cases, immediately deleting the autopipeline from
running pipelines?
Bastian added a commit to Bastian/bstats-backend that referenced this issue Sep 4, 2022
@Bastian
Update to latest 4.x version of ioredis
2ca0cee 
Fixes issue with auto pipelining.

See redis/ioredis#1248
janus-dev87 added a commit to janus-dev87/ioredis-work that referenced this issue Mar 1, 2024
@janus-dev87
chore(release): 4.27.7 [skip ci]
a8ed140 
## [4.27.7](redis/ioredis@v4.27.6...v4.27.7) (2021-08-01)

### Bug Fixes

* **cluster:** fix autopipeline with keyPrefix or arg array ([#1391](redis/ioredis#1391)) ([d7477aa](redis/ioredis@d7477aa)), closes [#1264](redis/ioredis#1264) [#1248](redis/ioredis#1248) [#1392](redis/ioredis#1392)
janus-dev87 added a commit to janus-dev87/ioredis-work that referenced this issue Mar 1, 2024
@janus-dev87
fix: Fix undefined property warning in executeAutoPipeline (#1425)
3f85bcb 
Seen in an uncaughtException handler.

```
err_type=uncaughtException, exiting cause=TypeError: Cannot read property 'Symbol(callbacks)' of undefined
    at Immediate.executeAutoPipeline (.../node_modules/ioredis/built/autoPipelining.js:34:31)
    at Shim.applySegment (.../node_modules/newrelic/lib/shim/shim.js:1430:20)
    at Immediate.wrapper (.../node_modules/newrelic/lib/shim/shim.js:2097:17)
    at processImmediate (internal/timers.js:463:21)
```

A similar error was among the errors reported in
redis/ioredis#1248

I suspect the process.exec might call the callback synchronously in the
same tick in some cases, immediately deleting the autopipeline from
running pipelines?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
need detail need reproduce released
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@simoneb @luin @yunnysunny @Paul-Isache @ioredis-robot