Upcoming release of RQ 1.0

[selwin]

Hey there, just want to give a heads up about the next major RQ release, v1.0. I plan on releasing this new version in the second half of Feb (although this may change due to work commitments).

This release will have a few backward incompatible changes, the biggest one is the introduction of FailedJobRegistry that replaces FailedQueue. I have documented the new registry here.

The branch for the upcoming release is here: https://github.com/rq/rq/tree/v1.0
You may also find the changelog helpful: https://github.com/rq/rq/blob/v1.0/CHANGES.md

Let me know if there's any questions. Thanks!

[jeffery9876]

May I request to focus on better security before version 1.0?

#175

[selwin]

Care to elaborate more on the specific features you’re after? Best, Selwin

…

On 4 Feb 2019 18.34 +0700, Jeffery Li ***@***.***>, wrote: May I request to focus on better security before version 1.0? #175 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jeffery9876]

Hi I’ve attached the link to issue no.175, but maybe you missed it.

…

On Mon, 4 Feb 2019 at 12:38, Selwin Ong ***@***.***> wrote: Care to elaborate more on the specific features you’re after? Best, Selwin On 4 Feb 2019 18.34 +0700, Jeffery Li ***@***.***>, wrote: > May I request to focus on better security before version 1.0? > #175 > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#172 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABJy1V9vWtEz0309_DRJna0J44jDLkXfks5vKCnEgaJpZM4aO0LE> .

[eoranged]

May I request to focus on better security before version 1.0?

#175

@JefferyLondon to avoid confusion: RQ Dashboard and RQ itself are separate project.
Hereby RQ 1.0 release is being discussed.

[mitakuye]

RQ 1.0 Released. Most obvious breaking change is the removal of get_failed_queue, as this will break rq-dashboard right away during imports

https://github.com/rq/rq/blob/master/CHANGES.md

[eoranged]

I've been a bit busy recently, so I didn't have time to review It in depth before.
Since there is a separate registry for each queue now, It will require change in API, but for now I'm thinking about implementing quick and ugly workaround by collecting list of failed jobs from all registries, which will keep backward compatibility in all APIs.
This will make 0.5.0 version of rq_dashboard, which will support both old and new rq versions. Next in 1.0 version APIs will be changed and support for earlier rq versions (0.x) will be dropped.

[eoranged]

Added quirky support for 1.0 in #182 with some caveats.
Works on my local dummy setup, but It will be nice If someone can test It in their test environments.
Please don't use It on production yet, cause there definitely will be dragons bugs.

[eoranged]

Released 0.5.0 with these changes.
It works good on my synthetic scenarios, but I haven't tested in real-life apps yet, so cannot confirm that every scenario works as expected.

[amir20]

I am getting a different error but seems related to this

.direnv/python-venv-3.7.3/lib/python3.7/site-packages/rq_dashboard/web.py:45: in <module>
    from .compat import get_failed_queue, get_all_queues
E     File ".../Workspace/clashstats/.direnv/python-venv-3.7.3/lib/python3.7/site-packages/rq_dashboard/compat.py", line 15
E       def __init__(self, default_timeout=None, connection=None, async=True):
E                                                                     ^
E   SyntaxError: invalid syntax

[eoranged]

@amir20 thanks for reporting this. Fixed in 0.5.1 release.
Also added tests, so we won't face It during future changes.

[amir20]

Thanks! That errors seems to have gone away. But now there is a new error and I can't make sense of it. It seems to be in rq-dashboard though.

During handling of the above exception, another exception occurred:
[0] web_1             |
[0] web_1             | Traceback (most recent call last):
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
[0] web_1             |     self.handle_request(listener, req, client, addr)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
[0] web_1             |     respiter = self.wsgi(environ, resp.start_response)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2309, in __call__
[0] web_1             |     return self.wsgi_app(environ, start_response)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2295, in wsgi_app
[0] web_1             |     response = self.handle_exception(e)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1741, in handle_exception
[0] web_1             |     reraise(exc_type, exc_value, tb)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
[0] web_1             |     raise value
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
[0] web_1             |     response = self.full_dispatch_request()
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
[0] web_1             |     rv = self.handle_user_exception(e)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
[0] web_1             |     reraise(exc_type, exc_value, tb)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
[0] web_1             |     raise value
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
[0] web_1             |     rv = self.dispatch_request()
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
[0] web_1             |     return self.view_functions[rule.endpoint](**req.view_args)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/rq_dashboard/web.py", line 162, in overview
[0] web_1             |     failed = get_failed_queue()
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/rq_dashboard/compat.py", line 89, in get_failed_queue
[0] web_1             |     return FailedQueue()
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/rq_dashboard/compat.py", line 22, in __init__
[0] web_1             |     self._registries = (q.failed_job_registry for q in Queue.all())
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/rq/queue.py", line 41, in all
[0] web_1             |     for rq_key in connection.smembers(cls.redis_queues_keys)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 1916, in smembers
[0] web_1             |     return self.execute_command('SMEMBERS', name)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 772, in execute_command
[0] web_1             |     connection = pool.get_connection(command_name, **options)
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/redis/connection.py", line 994, in get_connection
[0] web_1             |     connection.connect()
[0] web_1             |   File "/usr/local/lib/python3.7/site-packages/redis/connection.py", line 497, in connect
[0] web_1             |     raise ConnectionError(self._error_message(e))
[0] web_1             | redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Cannot assign requested address.

I am running redis and flask with docker-compose. When I rollback the changes, it works fine. Is there something obvious that I am missing?

[amir20]

I have updated all my python packages except rq and rq-dashboard and this problem no longer exists. So it is either rq or rq-dashboard throwing the above error.

[jonozzz]

@amir20 have you tried this https://stackoverflow.com/questions/54965291/error-99-connecting-to-localhost6379-cannot-assign-requested-address

[amir20]

I do not use localhost as the address. Since it’s docker network I can refer to it with hostname.

I suspect something else is going on. With older version of rq and rq-dashboard it works. When I upgrade, I get that error.

[eoranged]

@amir20 sorry for delay. I misread your previous message and considered the matter resolved.
As for your issue, could you please try version from master? Problems you describing might be caused by issues with handling legacy configuration parameters.

To install rq_dashboard from master you can use pip install git+https://github.com/eoranged/rq-dashboard.git@master#egg=rq_dashboard

If this won't help, please create separate issue with more details:

• python version
• rq_dashboard version
• how do you run It? (via CLI or by embeding into Flask app)
• how do you pass Redis configuration?

[amir20]

It's ok. I just haven't updated rq-dashboard because it had been broken. I can give master a try next weekend during the holidays. I'll report back. Honesty, I haven't done anything else different that just simple web app with rq. So I am wondering why only I see it. I'll have to investigate a little and maybe even provide a demo of it being broken.

[amir20]

Update: 0.5.2 seems to have fixed it. Not sure what changed'

[eoranged]

@amir20 0.5.2 includes additional backward compatibility changes. You should see warnings about deprecated configuration options right now.

[amir20]

I don't see any warnings. What should I be looking for? I do see other warnings related to flask-cache. So I know it's working.

…

On Sat, Jul 13, 2019 at 10:34 AM Vladimir Protasov ***@***.***> wrote: @amir20 <https://github.com/amir20> 0.5.2 includes additional backward compatibility changes. You should see warnings about deprecated configuration options right now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#172>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAB7UO5HBTVIDVJWVUKPQ2TP7IGZJANCNFSM4GR3ILCA> .

-- Amir Raminfar* / *amirraminfar.com

[eoranged]

These warnings: https://github.com/eoranged/rq-dashboard/blob/master/rq_dashboard/legacy_config.py#L16