Update celery to 4.1.0 #278

pyup-bot · 2017-07-24T18:15:28Z

There's a new version of celery available.
You are currently using 3.1.23. I have updated it to 4.1.0

These links might come in handy: PyPI | Changelog | Homepage | Docs

Changelog

4.1.0

=====
:release-date: 2017-07-25 00:00 PM PST
:release-by: Omer Katz

Configuration: CELERY_SEND_EVENTS instead of CELERYD_SEND_EVENTS for 3.1.x compatibility (3997)

Contributed by abhinav nilaratna.

App: Restore behavior so Broadcast queues work. (3934)

Contributed by Patrick Cloke.

Sphinx: Make appstr use standard format (4134) (4139)

Contributed by Preston Moore.

App: Make id, name always accessible from logging.Formatter via extra (3994)

Contributed by Yoichi NAKAYAMA.

Worker: Add worker_shutting_down signal (3998)

Contributed by Daniel Huang.

PyPy: Support PyPy version 5.8.0 (4128)

Contributed by Omer Katz.

Results: Elasticsearch: Fix serializing keys (3924)

Contributed by :github_user:staticfox.

Canvas: Deserialize all tasks in a chain (4015)

Contributed by :github_user:fcoelho.

Systemd: Recover loglevel for ExecStart in systemd config (4023)

Contributed by Yoichi NAKAYAMA.

Sphinx: Use the Sphinx add_directive_to_domain API. (4037)

Contributed by Patrick Cloke.

App: Pass properties to before_task_publish signal (4035)

Contributed by Javier Domingo Cansino.

Results: Add SSL option for Redis backends (3831)

Contributed by Chris Kuehl.

Beat: celery.schedule.crontab: fix reduce (3826) (3827)

Contributed by Taylor C. Richberger.

State: Fix celery issues when using flower REST API

Contributed by Thierry RAMORASOAVINA.

Results: Elasticsearch: Fix serializing document id.

Contributed by Acey9.

Beat: Make shallow copy of schedules dictionary

Contributed by Brian May.

Beat: Populate heap when periodic tasks are changed

Contributed by Wojciech Żywno.

Task: Allow class methods to define tasks (3952)

Contributed by georgepsarakis.

Platforms: Always return boolean value when checking if signal is supported (3962).

Contributed by Jian Yu.

Canvas: Avoid duplicating chains in chords (3779)

Contributed by Ryan Hiebert.

Canvas: Lookup task only if list has items (3847)

Contributed by Marc Gibbons.

Results: Allow unicode message for exception raised in task (3903)

Contributed by George Psarakis.

Python3: Support for Python 3.6 (3904, 3903, 3736)

Contributed by Jon Dufresne, George Psarakis, Asif Saifuddin Auvi, Omer Katz.

App: Fix retried tasks with expirations (3790)

Contributed by Brendan MacDonell.

Fixes items format route in docs (3875)

Contributed by Slam.

Utils: Fix maybe_make_aware (3850)

Contributed by Taylor C. Richberger.

Task: Fix task ETA issues when timezone is defined in configuration (3867)

Contributed by George Psarakis.

Concurrency: Consumer does not shutdown properly when embedded in gevent application (3746)

Contributed by Arcadiy Ivanov.

Canvas: Fix 3725: Task replaced with group does not complete (3731)

Contributed by Morgan Doocy.

Task: Correct order in chains with replaced tasks (3730)

Contributed by Morgan Doocy.

Result: Enable synchronous execution of sub-tasks (3696)

Contributed by shalev67.

Task: Fix request context for blocking task apply (added hostname) (3716)

Contributed by Marat Sharafutdinov.

Utils: Fix task argument handling (3678) (3693)

Contributed by Roman Sichny.

Beat: Provide a transparent method to update the Scheduler heap (3721)

Contributed by Alejandro Pernin.

Beat: Specify default value for pidfile option of celery beat. (3722)

Contributed by Arnaud Rocher.

Results: Elasticsearch: Stop generating a new field every time when a new result is being put (3708)

Contributed by Mike Chen.

Requirements

Now depends on :ref:Kombu 4.1.0 <kombu:version-4.1.0>.

Results: Elasticsearch now reuses fields when new results are added.

Contributed by Mike Chen.

Results: Fixed MongoDB integration when using binary encodings
(Issue 3575).

Contributed by Andrew de Quincey.

Worker: Making missing *args and **kwargs in Task protocol 1
return empty value in protocol 2 (Issue 3687).

Contributed by Roman Sichny.

App: Fixed :exc:TypeError in AMQP when using deprecated signal
(Issue 3707).

Contributed by :github_user:michael-k.

Beat: Added a transparent method to update the scheduler heap.

Contributed by Alejandro Pernin.

Task: Fixed handling of tasks with keyword arguments on Python 3
(Issue 3657).

Contributed by Roman Sichny.

Task: Fixed request context for blocking task apply by adding missing
hostname attribute.

Contributed by Marat Sharafutdinov.

Task: Added option to run subtasks synchronously with
disable_sync_subtasks argument.

Contributed by :github_user:shalev67.

App: Fixed chaining of replaced tasks (Issue 3726).

Contributed by Morgan Doocy.

Canvas: Fixed bug where replaced tasks with groups were not completing
(Issue 3725).

Contributed by Morgan Doocy.

Worker: Fixed problem where consumer does not shutdown properly when
embedded in a gevent application (Issue 3745).

Contributed by Arcadiy Ivanov.

Results: Added support for using AWS DynamoDB as a result backend (3736).

Contributed by George Psarakis.

Testing: Added caching on pip installs.

Contributed by :github_user:orf.

Worker: Prevent consuming queue before ready on startup (Issue 3620).

Contributed by Alan Hamlett.

App: Fixed task ETA issues when timezone is defined in configuration
(Issue 3753).

Contributed by George Psarakis.

Utils: maybe_make_aware should not modify datetime when it is
already timezone-aware (Issue 3849).

Contributed by Taylor C. Richberger.

App: Fixed retrying tasks with expirations (Issue 3734).

Contributed by Brendan MacDonell.

Results: Allow unicode message for exceptions raised in task
(Issue 3858).

Contributed by :github_user:staticfox.

Canvas: Fixed :exc:IndexError raised when chord has an empty header.

Contributed by Marc Gibbons.

Canvas: Avoid duplicating chains in chords (Issue 3771).

Contributed by Ryan Hiebert and George Psarakis.

Utils: Allow class methods to define tasks (Issue 3863).

Contributed by George Psarakis.

Beat: Populate heap when periodic tasks are changed.

Contributed by :github_user:wzywno and Brian May.

Results: Added support for Elasticsearch backend options settings.

Contributed by :github_user:Acey9.

Events: Ensure Task.as_dict() works when not all information about
task is available.

Contributed by :github_user:tramora.

Schedules: Fixed pickled crontab schedules to restore properly (Issue 3826).

Contributed by Taylor C. Richberger.

Results: Added SSL option for redis backends (Issue 3830).

Contributed by Chris Kuehl.

Documentation and examples improvements by:

Bruno Alla

Jamie Alessio

Vivek Anand

Peter Bittner

Kalle Bronsen

Jon Dufresne

James Michael DuPont

Sergey Fursov

Samuel Dion-Girardeau

Daniel Hahler

Mike Helmick

Marc Hörsken

Christopher Hoskin

Daniel Huang

Primož Kerin

Michal Kuffa

Simon Legner

Anthony Lukach

Ed Morley

Jay McGrath

Rico Moorman

Viraj Navkal

Ross Patterson

Dmytro Petruk

Luke Plant

Eric Poelke

Salvatore Rinchiera

Arnaud Rocher

Kirill Romanov

Simon Schmidt

Tamer Sherif

YuLun Shih

Ask Solem

Tom 'Biwaa' Riat

Arthur Vigil

Joey Wilhelm

Jian Yu

YuLun Shih

Arthur Vigil

Joey Wilhelm

:github_user:baixuexue123

:github_user:bronsen

:github_user:michael-k

:github_user:orf

:github_user:3lnc

4.0.2

=====
:release-date: 2016-12-15 03:31 P.M PST
:release-by: Ask Solem

Now depends on :mod:amqp 2.1.4

This new version takes advantage of TCP Keepalive settings on Linux,
making it better at detecting closed connections, also in failover
conditions.

Redis: Priority was reversed so, e.g. priority 0 became priority 9.

.. _version-4.0.1:

4.0.1

=====
:release-date: 2016-12-07 06:00 P.M PST
:release-by: Ask Solem

Now depends on :mod:amqp 2.1.3

This new version takes advantage of the new TCP_USER_TIMEOUT socket option
on Linux.

Producer: Fixed performance degradation when default exchange specified
(Issue 651).

QPid: Switch to using getattr in qpid.Transport.del (Issue 658)

Contributed by Patrick Creech.

QPid: Now uses monotonic time for timeouts.

MongoDB: Fixed compatibility with Python 3 (Issue 661).

Consumer: __exit__ now skips cancelling consumer if connection-related
error raised (Issue 670).

MongoDB: Removes use of natural sort (Issue 638).

Contributed by Anton Chaporgin.

Fixed wrong keyword argument channel error (Issue 652).

Contributed by Toomore Chiang.

Safe argument to urllib.quote must be bytes on Python 2.x (Issue 645).

Documentation improvments by:

Carlos Edo

Cemre Mengu

.. _version-4.0:

4.0

===
:release-date: 2016-10-28 16:45 P.M UTC
:release-by: Ask Solem

Now depends on :mod:amqp 2.0.

The new py-amqp version have been refactored for better performance,
using modern Python socket conventions, and API consistency.

No longer depends on :mod:anyjson.

Kombu will now only choose between :pypi:simplejson and the built-in
:mod:json.

Using the latest version of simplejson is recommended:

.. code-block:: console

   $ pip install -U simplejson

Removed transports that are no longer supported in this version:

Django ORM transport

SQLAlchemy ORM transport

Beanstalk transport

ZeroMQ transport

amqplib transport (use pyamqp).

API Changes

Signature of :class:kombu.Message now takes body as first argment.

   It used to be ``Message(channel, body=body, **kw)``, but now it&#39;s
   ``Message(body, channel=channel, **kw)``.

   This is unlikey to affect you, as the Kombu API does not have
   users instantiate messages manually.

New SQS transport

Donated by NextDoor, with additional contributions from mdk.

.. note::

   ``kombu[sqs]`` now depends on :pypi:`pycurl`.

New Consul transport.

Contributed by Wido den Hollander.

New etcd transport.

Contributed by Stephen Milner.

New Qpid transport.

It was introduced as an experimental transport in Kombu 3.0, but is now
mature enough to be fully supported.

Created and maintained by Brian Bouterse.

Redis: Priority 0 is now lowest, 9 is highest.
(backward incompatible)

This to match how priorities in AMQP works.

Fix contributed by Alex Koshelev.

Redis: Support for Sentinel

You can point the connection to a list of sentinel URLs like:

.. code-block:: text

   sentinel://0.0.0.0:26379;sentinel://0.0.0.0:26380/...

where each sentinel is separated by a ;. Multiple sentinels are handled
by :class:kombu.Connection constructor, and placed in the alternative
list of servers to connect to in case of connection failure.

Contributed by Sergey Azovskov, and Lorenzo Mancini

RabbitMQ Queue Extensions

New arguments have been added to :class:kombu.Queue that lets
you directly and conveniently configure the RabbitMQ queue extensions.

Queue(expires=20.0)

   Set queue expiry time in float seconds.

   See :attr:`kombu.Queue.expires`.

Queue(message_ttl=30.0)

   Set queue message time-to-live float seconds.

   See :attr:`kombu.Queue.message_ttl`.

Queue(max_length=1000)

   Set queue max length (number of messages) as int.

   See :attr:`kombu.Queue.max_length`.

Queue(max_length_bytes=1000)

   Set queue max length (message size total in bytes) as int.

   See :attr:`kombu.Queue.max_length_bytes`.

Queue(max_priority=10)

   Declare queue to be a priority queue that routes messages
   based on the ``priority`` field of the message.

   See :attr:`kombu.Queue.max_priority`.

RabbitMQ: Message.ack now supports the multiple argument.

If multiple is set to True, then all messages received before
the message being acked will also be acknowledged.

amqps:// can now be specified to require SSL (Issue 610).

Consumer.cancel_by_queue is now constant time.

Connection.ensure* now raises :exc:kombu.exceptions.OperationalError.

Things that can be retried are now reraised as
:exc:kombu.exceptions.OperationalError.

Redis: Fixed SSL support.

Contributed by Robert Kolba.

New Queue.consumer_arguments can be used for the ability to
set consumer priority via x-priority.

See https://www.rabbitmq.com/consumer-priority.html

Example:

.. code-block:: python

   Queue(
       &#39;qname&#39;,
       exchange=Exchange(&#39;exchange&#39;),
       routing_key=&#39;qname&#39;,
       consumer_arguments={&#39;x-priority&#39;: 3},
   )

Queue/Exchange: no_declare option added (also enabled for
internal amq. exchanges) (Issue 565).

JSON serializer now calls obj.__json__ for unsupported types.

This means you can now define a __json__ method for custom
types that can be reduced down to a built-in json type.

Example:

.. code-block:: python

   class Person:
       first_name = None
       last_name = None
       address = None

       def __json__(self):
           return {
               &#39;first_name&#39;: self.first_name,
               &#39;last_name&#39;: self.last_name,
               &#39;address&#39;: self.address,
           }

JSON serializer now handles datetimes, Django promise, UUID and Decimal.

Beanstalk: Priority 0 is now lowest, 9 is highest.
(backward incompatible)

This to match how priorities in AMQP works.

Fix contributed by Alex Koshelev.

Redis: now supports SSL using the ssl argument to
:class:~kombu.Connection.

Redis: Fanout exchanges are no longer visible between vhosts,
and fanout messages can be filtered by patterns.
(backward incompatible)

It was possible to enable this mode previously using the
fanout_prefix, and fanout_patterns
transport options, but now these are enabled by default.

If you want to mix and match producers/consumers running different
versions you need to configure your kombu 3.x clients to also enable
these options:

.. code-block:: pycon

   &gt;&gt;&gt; Connection(transport_options={
       &#39;fanout_prefix&#39;: True,
       &#39;fanout_patterns&#39;: True,
   })

Pidbox: Mailbox new arguments: TTL and expiry.

Mailbox now supports new arguments for controlling
message TTLs and queue expiry, both for the mailbox
queue and for reply queues.

queue_expires (float/int seconds).

queue_ttl (float/int seconds).

reply_queue_expires (float/int seconds).

reply_queue_ttl (float/int seconds).

All take seconds in int/float.

Contributed by Alan Justino.

Exchange.delivery_mode now defaults to :const:None, and the default
is instead set by Producer.publish.

:class:~kombu.Consumer now supports a new prefetch_count argument,
which if provided will force the consumer to set an initial prefetch count
just before starting.

Virtual transports now stores priority as a property, not in
delivery_info, to be compatible with AMQP.

reply_to argument to Producer.publish can now be
:class:~kombu.Queue instance.

Connection: There's now a new method
Connection.supports_exchange_type(type) that can be used to check if the
current transport supports a specific exchange type.

SQS: Consumers can now read json messages not sent by Kombu.

Contributed by Juan Carlos Ferrer.

SQS: Will now log the access key used when authentication fails.

Contributed by Hank John.

Added new :class:kombu.mixins.ConsumerProducerMixin for consumers that
will also publish messages on a separate connection.

Messages: Now have a more descriptive repr.

Contributed by Joshua Harlow.

Async: HTTP client based on curl.

Async: Now uses poll instead of select where available.

MongoDB: Now supports priorities

Contributed by Alex Koshelev.

Virtual transports now supports multiple queue bindings.

Contributed by Federico Ficarelli.

Virtual transports now supports the anon exchange.

If when publishing a message, the exchange argument is set to '' (empty
string), the routing_key will be regarded as the destination queue.

This will bypass the routing table compeltely, and just deliver the
message to the queue name specified in the routing key.

Zookeeper: Transport now uses the built-in suport in kazoo to handle
failover when using a list of server names.

Contributed by Joshua Harlow.

ConsumerMixin.run now passes keyword arguments to .consume.

Deprecations and removals

The deprecated method Consumer.add_queue_from_dict has been removed.

Use instead:

.. code-block:: python

   consumer.add_queue(Queue.from_dict(queue_name, **options))

The deprecated function kombu.serialization.encode has been removed.

Use :func:kombu.serialization.dumps instead.

The deprecated function kombu.serialization.decode has been removed.

Use :func:kombu.serialization.loads instead.

Removed module kombu.syn

detect_environment has been moved to kombu.utils.compat

.. _version-3.0.37:

3.5.0.3

Adds Process._authkey alias to .authkey for 2.7 compat.

Remove superfluous else clause from max_memory_per_child_check.

Document and test all supported Python versions.

Extend 'Process' to be compatible with < Py3.5.

Use a properly initialized logger in pool.py error logging.

_trywaitkill can now kill a whole process group if the worker process declares itself as a group leader.

Fix cpython issue 14881 (See http://bugs.python.org/issue14881).

Fix for a crash on windows.

Fix messaging in case of worker exceeds max memory.

3.5.0.2

max_memory_per_child was measured in kilobytes on Linux, but bytes on
*BSD/MacOS, it's now always kilobytes.

Windows: Adds support for max_memory_per_child, but requires the
psutil package to be installed.

Fixed bug in ForkingPickler.loadbuf, where it tried to pass
a BytesIO instance directly to pickle.loads on Python 2.7.

3.5.0.1

Connection: Properly handle EINTR (Issue 191).

Fixed bug with missing CreateProcess for Windows on Python 2.7.

Adds Process._counter for compatibility with Python <3.5.

3.5.0.0

No longer supports Python 2.6

You need Python 2.7 or later to use this version of billiard.

Merged changes from CPython 3.5

3.3.0.20

Pool: Timeouts will attempt to send SIGKILL, but this signal
does not exist on Windows. Replaced with SIGTERM.

3.3.0.19

Pool: Exceptions in user timeout callbacks are now logged instead
of crashing the pool.

Contributed by Pierre Fersing.

Pool: Exit codes in errors were improperly being represented as signals.

Pool: .map. and .imap now working again.

Now builds on FreeBSD 10.

Contributed by Michael Fladischer.

3.3.0.18

Now compiles on GNU/kFreeBSD

Contributed by Michael Fladischer.

Pool: AF_PIPE address fixed so that it works on recent Windows versions
in combination with Python 2.7.7.

Fix contributed by Joshua Tacoma.

Pool: Fix for Supervisor object has no attribute _children error.

Fix contributed by Andres Riancho.

Pool: Fixed bug with human_status(None).

Pool: shrink did not work properly if asked to remove more than 1 process.

3.3.0.17

Fixes SemLock on Python 3.4 (Issue 107) when using
forking_enable(False).

Pool: Include more useful exitcode information when processes exit.

3.3.0.16

Previous release was missing the billiard.py3 package from MANIFEST
so the installation would not work on Python 3.

3.3.0.15

Pool: Fixed "cannot join process not started" error.

Now uses billiard.py2 and billiard.py3 specific packages that are installed
depending on the python version used.

This way the installation will not import version specific modules (and
possibly crash).

3.3.0.14

Fixed problem with our backwards compatible bytes wrapper
(Issue 103).

No longer expects frozen applications to have a valid __file__
attribute.

Fix contributed by George Sibble.

3.3.0.13

Fixes compatability with Python < 2.7.6

No longer attempts to handle SIGBUS

Contributed by Vishal Vatsa.

Non-thread based pool now only handles signals:

SIGHUP, SIGQUIT, SIGTERM, SIGUSR1,
SIGUSR2.

setup.py: Only show compilation warning for build related commands.

3.3.0.12

Fixed installation for Python 3.

Contributed by Rickert Mulder.

Pool: Fixed bug with maxtasksperchild.

Fix contributed by Ionel Cristian Maries.

Pool: Fixed bug in maintain_pool.

3.3.0.11

Fixed Unicode error when installing the distribution (Issue 89).

Daemonic processes are now allowed to have children.

But note that it will not be possible to automatically
terminate them when the process exits.

See discussion at celery/celery#1709

Pool: Would not always be able to detect that a process exited.

3.3.0.10

Windows: Fixed problem with missing WAITABANDONED_0

Fix contributed by Matthias Wagner

Windows: PipeConnection can now be inherited.

Fix contributed by Matthias Wagner

3.3.0.9

Temporary workaround for Celery maxtasksperchild issue.

Fix contributed by Ionel Cristian Maries.

3.3.0.8

Now also sets multiprocessing.current_process for compatibility
with loggings processName field.

3.3.0.7

Fixed compatibility with PyPy 2.1 + 2.2.

Fixed problem in pypy detection.

Fix contributed by Tin Tvrtkovic.

Now uses ctypes.find_library instead of hardcoded path to find
the macOS CoreServices framework.

Fix contributed by Moritz Kassner.

3.3.0.6

Now works without C extension again.

New _billiard.read(fd, buffer, [len, ]) function
implements os.read with buffer support (new buffer API)

New pure-python implementation of Connection.send_offset.

3.3.0.5

All platforms except for Windows/PyPy/Jython now requires the C extension.

3.3.0.4

Fixed problem with Python3 and setblocking.

3.3.0.3

Now works on Windows again.

3.3.0.2

ApplyResult.terminate() may be set to signify that the job
must not be executed. It can be used in combination with
Pool.terminate_job.

Pipe/_SimpleQueue: Now supports rnonblock/wnonblock arguments
to set the read or write end of the pipe to be nonblocking.

Pool: Log message included exception info but exception happened
in another process so the resulting traceback was wrong.

Pool: Worker process can now prepare results before they are sent
back to the main process (using Worker.prepare_result).

3.3.0.1

Pool: New correlation_id argument to apply_async can be
used to set a related id for the ApplyResult object returned:

>>> r = pool.apply_async(target, args, kwargs, correlation_id='foo')
>>> r.correlation_id
'foo'

Pool: New callback on_process_exit is called when a pool
process exits, with signature (pid, exitcode).

Contributed by Daniel M. Taub.

Pool: Improved the too many restarts detection.

3.3.0.0

Dual code base now runs on Python 2.6+ and Python 3.

No longer compatible with Python 2.5

Includes many changes from multiprocessing in 3.4.

Now uses time.monotonic when available, also including
fallback implementations for Linux and macOS.

No longer cleans up after receiving SIGILL, SIGSEGV or SIGFPE

Contributed by Kevin Blackham

Finalize and register_after_fork is now aliases to multiprocessing.

It's better to import these from multiprocessing directly now
so that there aren't multiple registries.

New billiard.queues._SimpleQueue that does not use semaphores.

Pool: Can now be extended to support using multiple IPC queues.

Pool: Can now use async I/O to write to pool IPC queues.

Pool: New Worker.on_loop_stop handler can be used to add actions
at pool worker process shutdown.

Note that, like all finalization handlers, there is no guarantee that
this will be executed.

Contributed by dmtaub.

Got merge conflicts? Close this PR and delete the branch. I'll create a new PR for you.

Happy merging! 🤖

codecov-io · 2017-07-24T18:37:59Z

Codecov Report

Merging #278 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #278   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           5      5           
  Lines          64     64           
=====================================
  Hits           64     64

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 104a0ef...f4a48b6. Read the comment docs.

Update celery from 3.1.23 to 4.1.0

f4a48b6

pyup-bot mentioned this pull request Jul 24, 2017

Update celery to 4.0.2 #134

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update celery to 4.1.0 #278

Update celery to 4.1.0 #278

pyup-bot commented Jul 24, 2017

codecov-io commented Jul 24, 2017

Update celery to 4.1.0 #278

Are you sure you want to change the base?

Update celery to 4.1.0 #278

Conversation

pyup-bot commented Jul 24, 2017

Changelog

4.1.0

4.0.2

4.0.1

4.0

Deprecations and removals

3.5.0.3

3.5.0.2

3.5.0.1

3.5.0.0

3.3.0.20

3.3.0.19

3.3.0.18

3.3.0.17

3.3.0.16

3.3.0.15

3.3.0.14

3.3.0.13

3.3.0.12

3.3.0.11

3.3.0.10

3.3.0.9

3.3.0.8

3.3.0.7

3.3.0.6

3.3.0.5

3.3.0.4

3.3.0.3

3.3.0.2

3.3.0.1

3.3.0.0

codecov-io commented Jul 24, 2017

Codecov Report