Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shrink_disk_usage for sqlite3 base queues #182

Merged
merged 1 commit into from
Apr 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,11 @@ Close the console, and then recreate the queue:
'str2'
>>>

New functions:
*Available since v0.8.0*

- ``shrink_disk_usage`` perform a ``VACUUM`` against the sqlite, and rebuild the database file, this usually takes long time and frees a lot of disk space after ``get()``


Example usage of SQLite3 based ``UniqueQ``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
2 changes: 1 addition & 1 deletion persistqueue/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# coding=utf-8
__author__ = 'Peter Wang'
__license__ = 'BSD'
__version__ = '0.8.0-alpha0'
__version__ = '0.8.0-beta0'

from .exceptions import Empty, Full # noqa
from .queue import Queue # noqa
Expand Down
5 changes: 0 additions & 5 deletions persistqueue/sqlackqueue.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,11 +167,6 @@ def clear_acked_data(
)
return sql, AckStatus.acked

@sqlbase.with_conditional_transaction
def shrink_disk_usage(self):
sql = """VACUUM"""
return sql, ()

@property
def _sql_mark_ack_status(self):
return self._SQL_MARK_ACK_UPDATE.format(
Expand Down
5 changes: 5 additions & 0 deletions persistqueue/sqlbase.py
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,11 @@ def queue(self):
datarows.append(item)
return datarows

@with_conditional_transaction
def shrink_disk_usage(self):
sql = """VACUUM"""
return sql, ()

@property
def size(self):
return self.total
Expand Down
1 change: 1 addition & 0 deletions persistqueue/tests/test_sqlackqueue.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def test_open_close_1000(self):
# assert adding another one still works
q.put('foobar')
data = q.get()
q.shrink_disk_usage()
self.assertEqual('foobar', data)

def test_random_read_write(self):
Expand Down
1 change: 1 addition & 0 deletions persistqueue/tests/test_sqlqueue.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def test_open_close_1000(self):
# assert adding another one still works
q.put('foobar')
data = q.get()
q.shrink_disk_usage()
self.assertEqual('foobar', data)

def test_random_read_write(self):
Expand Down