-
Notifications
You must be signed in to change notification settings - Fork 240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQL cleanup thread or per requests #211
Conversation
Added app command option. Abstracted so that new session interfaces only have to implement the delete_session method. |
src/flask_session/sessions.py
Outdated
|
||
def _cleanup_n_requests(self) -> None: | ||
"""Delete expired sessions approximately every N requests.""" | ||
if self.cleanup_n_seconds or ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this should run if self.cleanup_n_seconds
is truthy. If that is specified, the clean up will run multiple times:
- before every request
- in the cleanup threads (one per uWSGI worker)
src/flask_session/sessions.py
Outdated
): | ||
self._delete_expired_sessions() | ||
|
||
def _start_cleanup_thread(self, cleanup_n_seconds: int) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As mentioned in the discord discussion, this will start one thread per uWSGI worker.
I would personally not implement this at all and use the cleanup per requests instead. It's not really essential for the clean up to run exactly every N seconds and I would lean towards having fewer connections to the database.
src/flask_session/sessions.py
Outdated
).delete(synchronize_session=False) | ||
self.db.session.commit() | ||
self.app.logger.info("Deleted expired sessions") | ||
except Exception as e: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need a db.session.rollback()
when handling the exception?
def _delete_expired_sessions(self) -> None: | ||
try: | ||
self.db.session.query(self.sql_session_model).filter( | ||
self.sql_session_model.expiry <= datetime.utcnow() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both here and in the session creation I would use the database's now()
function. This would prevent issues due to time synchronization between multiple servers.
Lint and docs fix
…in /services/flask (#75) Updates the requirements on [flask-session](https://github.com/pallets-eco/flask-session) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pallets-eco/flask-session/releases">flask-session's releases</a>.</em></p> <blockquote> <h2>0.8.0</h2> <p>Add DynamodDB backend and other minor fixes.</p> <p>Full release notes: <a href="https://flask-session.readthedocs.io/en/latest/changes.html#id1">https://flask-session.readthedocs.io/en/latest/changes.html#id1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pallets-eco/flask-session/blob/development/CHANGES.rst">flask-session's changelog</a>.</em></p> <blockquote> <h2>0.8.0 - 2024-03-26</h2> <p>Added</p> <pre><code>- Add DynamoDB session interface (`[#214](pallets-eco/flask-session#214) <https://github.com/pallets-eco/flask-session/pull/214>`_). - Add ability to install client libraries for backends using optional dependencies (extras) (`[#228](pallets-eco/flask-session#228) <https://github.com/pallets-eco/flask-session/pull/228>`_). <p>Fixed</p> <pre><code>- Include prematurely removed ``cachelib`` dependency. Will be removed in 1.0.0 to be an optional dependency (`[#223](pallets-eco/flask-session#223) &lt;https://github.com/pallets-eco/flask-session/issues/223&gt;`_). 0.7.0 - 2024-03-18 ------------------ Changed </code></pre> <ul> <li>Access session interfaces via subfolder, for example <code>flask_session.redis.RedisSessionInterface</code> (<code>2bc7df &lt;pallets-eco/flask-session@2bc7df1be7b8929e55cb25f13845caf0503630d8&gt;</code>_).</li> <li>Deprecate <code>pickle</code> in favor of <code>msgspec</code>, which is configured with <code>SESSION_SERIALIZATION_FORMAT</code> to choose between <code>'json'</code> and <code>'msgpack'</code>. All sessions will convert to msgspec upon first interaction with 0.7.0. Pickle is still available to read existing sessions, but will be removed in 1.0.0. (<code>c7f8ce &lt;pallets-eco/flask-session@c7f8ced0e1532dea87850d34b3328a3fcb769988&gt;</code><em>, <code>c7f8ce &lt;pallets-eco/flask-session@c7f8ced0e1532dea87850d34b3328a3fcb769988&gt;</code></em>)</li> <li>Deprecate <code>SESSION_USE_SIGNER</code> (<code>a5dba7 &lt;pallets-eco/flask-session@a5dba7022f806c8fb4412d0428b69dd4a077e4a7&gt;</code>_).</li> <li>Deprecate :class:<code>flask_session.filesystem.FileSystemSessionInterface</code> in favor of the broader :class:<code>flask_session.cachelib.CacheLibSessionInterface</code> (<code>2bc7df &lt;pallets-eco/flask-session@2bc7df1be7b8929e55cb25f13845caf0503630d8&gt;</code>_).</li> </ul> <p>Added</p> <pre><code>- Add time-to-live expiration for MongoDB (`9acee3 &lt;https://github.com/pallets-eco/flask-session/commit/9acee3c5fb7072476f3feea923529d19d5e855c3&gt;`_). - Add retry for SQL based storage (`[#211](pallets-eco/flask-session#211) &lt;https://github.com/pallets-eco/flask-session/pull/211&gt;`_). - Add ``flask session_cleanup`` command and alternatively, ``SESSION_CLEANUP_N_REQUESTS`` for SQLAlchemy or future non-TTL backends (`[#211](pallets-eco/flask-session#211) &lt;https://github.com/pallets-eco/flask-session/pull/211&gt;`_). - Add type hints (`7d7d58 &lt;https://github.com/pallets-eco/flask-session/commit/7d7d58ce371553da39095a421445cf639a62bd5f&gt;`_). - Add logo and additional documentation. - Add vary cookie header when session modified or accessed as per flask's built-in session (`7ab698 &lt;https://github.com/pallets-eco/flask-session/commit/7ab6980c8ba15912df13dd1e78242803e8104dd6&gt;`_). - Add regenerate method to session interface to mitigate fixation (`[#27](pallets-eco/flask-session#27) &lt;https://github.com/pallets-eco/flask-session/pull/27&gt;`_, `[#39](pallets-eco/flask-session#39) &lt;https://github.com/pallets-eco/flask-session/issues/39&gt;`_)(`80df63 &lt;https://github.com/pallets-eco/flask-session/commit/80df635ffd466fa7798f6031be5469b4d5dae069&gt;`_). Removed </code></pre> <ul> <li>Remove null session in favour of relevant exception messages (<code>[#107](pallets-eco/flask-session#107) &lt;pallets-eco/flask-session#107;, <code>[#182](pallets-eco/flask-session#182) &lt;pallets-eco/flask-session#182 &lt;pallets-eco/flask-session@d7ed1c6e7eb3904888b72f0d6c006db1b9b60795&gt;</code>_).</li> <li>Drop support for Python 3.7 which is end-of-life and precludes use of msgspec (<code>bd7e5b &lt;pallets-eco/flask-session@bd7e5b0bbfc10cdfa9c83b859593c69cc4381571&gt;</code>_).</li> </ul> <p>Fixed</p> <pre><code>- Prevent session identifier reuse on storage miss (`[#76](pallets-eco/flask-session#76) &lt;https://github.com/pallets-eco/flask-session/pull/76&gt;`_). - Abstraction to improve consistency between backends. - Enforce ``PERMANENT_SESSION_LIFETIME`` as expiration consistently for all backends (`[#81](pallets-eco/flask-session#81) &lt;https://github.com/pallets-eco/flask-session/issues/81&gt;`_)(`86895b &lt;https://github.com/pallets-eco/flask-session/commit/86895b523203ca67c9f87416bdbf028852dcb357&gt;`_). - Specifically include backend session interfaces in public API and document usage (`[#210](pallets-eco/flask-session#210) &lt;https://github.com/pallets-eco/flask-session/issues/210&gt;`_). - Fix non-permanent sessions not updating expiry (`[#221](pallets-eco/flask-session#221) &lt;https://github.com/pallets-eco/flask-session/issues/221&gt;`_). 0.6.0 - 2024-01-16 ------------------ &lt;/tr&gt;&lt;/table&gt; </code></pre> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="pallets-eco/flask-session@65c05eae0e6d679047394b39a8d6e2bce8195ac3"><code>65c05ea</code></a> Prep release</li> <li><a href="pallets-eco/flask-session@475cc038376d4d5c2949edf7751a61ed64b7f33c"><code>475cc03</code></a> Merge branch 'erik' into development</li> <li><a href="pallets-eco/flask-session@1af5cb223bcb4bb556cf8b49e803700ac332ab89"><code>1af5cb2</code></a> Remind the correct PR branch</li> <li><a href="pallets-eco/flask-session@b7a219bcac97e74276d8e419e6174aea2c4ffdad"><code>b7a219b</code></a> Reduce documentation and remove filesystem extra</li> <li><a href="pallets-eco/flask-session@6c9a698b61ead380988ba4b69b0f161de47e1886"><code>6c9a698</code></a> Fix docs announcement overflow</li> <li><a href="pallets-eco/flask-session@f8f5ad15f787c5b30b663f84883ad2c8108b8314"><code>f8f5ad1</code></a> implemented DynamoDBSessionInterface and tests. (<a href="https://github.com/pallets-eco/flask-session/issues/214">#214</a>)</li> <li><a href="pallets-eco/flask-session@5a3413b9b1d67a4116404f23022b58326a60fbdf"><code>5a3413b</code></a> Add changelog and contributor</li> <li><a href="pallets-eco/flask-session@17369528b45d9b5a59003ec006ede35ced40f694"><code>1736952</code></a> Fix docs requirements</li> <li><a href="pallets-eco/flask-session@a92914ed1196acc2eab4d562996ac944413fb19c"><code>a92914e</code></a> Add docs</li> <li><a href="pallets-eco/flask-session@ab9a756a18de87033cf6d3b2d86c6dc46e006800"><code>ab9a756</code></a> Tidy up dynamo db, add docker and requirements</li> <li>Additional commits viewable in <a href="pallets-eco/flask-session@0.6.1rc1...0.8.0">compare view</a></li> </ul> </details> <br /> </code></pre> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…in /services/flask (#75) Updates the requirements on [flask-session](https://github.com/pallets-eco/flask-session) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pallets-eco/flask-session/releases">flask-session's releases</a>.</em></p> <blockquote> <h2>0.8.0</h2> <p>Add DynamodDB backend and other minor fixes.</p> <p>Full release notes: <a href="https://flask-session.readthedocs.io/en/latest/changes.html#id1">https://flask-session.readthedocs.io/en/latest/changes.html#id1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pallets-eco/flask-session/blob/development/CHANGES.rst">flask-session's changelog</a>.</em></p> <blockquote> <h2>0.8.0 - 2024-03-26</h2> <p>Added</p> <pre><code>- Add DynamoDB session interface (`[#214](pallets-eco/flask-session#214) <https://github.com/pallets-eco/flask-session/pull/214>`_). - Add ability to install client libraries for backends using optional dependencies (extras) (`[#228](pallets-eco/flask-session#228) <https://github.com/pallets-eco/flask-session/pull/228>`_). <p>Fixed</p> <pre><code>- Include prematurely removed ``cachelib`` dependency. Will be removed in 1.0.0 to be an optional dependency (`[#223](pallets-eco/flask-session#223) &lt;https://github.com/pallets-eco/flask-session/issues/223&gt;`_). 0.7.0 - 2024-03-18 ------------------ Changed </code></pre> <ul> <li>Access session interfaces via subfolder, for example <code>flask_session.redis.RedisSessionInterface</code> (<code>2bc7df &lt;pallets-eco/flask-session@2bc7df1be7b8929e55cb25f13845caf0503630d8&gt;</code>_).</li> <li>Deprecate <code>pickle</code> in favor of <code>msgspec</code>, which is configured with <code>SESSION_SERIALIZATION_FORMAT</code> to choose between <code>'json'</code> and <code>'msgpack'</code>. All sessions will convert to msgspec upon first interaction with 0.7.0. Pickle is still available to read existing sessions, but will be removed in 1.0.0. (<code>c7f8ce &lt;pallets-eco/flask-session@c7f8ced0e1532dea87850d34b3328a3fcb769988&gt;</code><em>, <code>c7f8ce &lt;pallets-eco/flask-session@c7f8ced0e1532dea87850d34b3328a3fcb769988&gt;</code></em>)</li> <li>Deprecate <code>SESSION_USE_SIGNER</code> (<code>a5dba7 &lt;pallets-eco/flask-session@a5dba7022f806c8fb4412d0428b69dd4a077e4a7&gt;</code>_).</li> <li>Deprecate :class:<code>flask_session.filesystem.FileSystemSessionInterface</code> in favor of the broader :class:<code>flask_session.cachelib.CacheLibSessionInterface</code> (<code>2bc7df &lt;pallets-eco/flask-session@2bc7df1be7b8929e55cb25f13845caf0503630d8&gt;</code>_).</li> </ul> <p>Added</p> <pre><code>- Add time-to-live expiration for MongoDB (`9acee3 &lt;https://github.com/pallets-eco/flask-session/commit/9acee3c5fb7072476f3feea923529d19d5e855c3&gt;`_). - Add retry for SQL based storage (`[#211](pallets-eco/flask-session#211) &lt;https://github.com/pallets-eco/flask-session/pull/211&gt;`_). - Add ``flask session_cleanup`` command and alternatively, ``SESSION_CLEANUP_N_REQUESTS`` for SQLAlchemy or future non-TTL backends (`[#211](pallets-eco/flask-session#211) &lt;https://github.com/pallets-eco/flask-session/pull/211&gt;`_). - Add type hints (`7d7d58 &lt;https://github.com/pallets-eco/flask-session/commit/7d7d58ce371553da39095a421445cf639a62bd5f&gt;`_). - Add logo and additional documentation. - Add vary cookie header when session modified or accessed as per flask's built-in session (`7ab698 &lt;https://github.com/pallets-eco/flask-session/commit/7ab6980c8ba15912df13dd1e78242803e8104dd6&gt;`_). - Add regenerate method to session interface to mitigate fixation (`[#27](pallets-eco/flask-session#27) &lt;https://github.com/pallets-eco/flask-session/pull/27&gt;`_, `[#39](pallets-eco/flask-session#39) &lt;https://github.com/pallets-eco/flask-session/issues/39&gt;`_)(`80df63 &lt;https://github.com/pallets-eco/flask-session/commit/80df635ffd466fa7798f6031be5469b4d5dae069&gt;`_). Removed </code></pre> <ul> <li>Remove null session in favour of relevant exception messages (<code>[#107](pallets-eco/flask-session#107) &lt;pallets-eco/flask-session#107;, <code>[#182](pallets-eco/flask-session#182) &lt;pallets-eco/flask-session#182 &lt;pallets-eco/flask-session@d7ed1c6e7eb3904888b72f0d6c006db1b9b60795&gt;</code>_).</li> <li>Drop support for Python 3.7 which is end-of-life and precludes use of msgspec (<code>bd7e5b &lt;pallets-eco/flask-session@bd7e5b0bbfc10cdfa9c83b859593c69cc4381571&gt;</code>_).</li> </ul> <p>Fixed</p> <pre><code>- Prevent session identifier reuse on storage miss (`[#76](pallets-eco/flask-session#76) &lt;https://github.com/pallets-eco/flask-session/pull/76&gt;`_). - Abstraction to improve consistency between backends. - Enforce ``PERMANENT_SESSION_LIFETIME`` as expiration consistently for all backends (`[#81](pallets-eco/flask-session#81) &lt;https://github.com/pallets-eco/flask-session/issues/81&gt;`_)(`86895b &lt;https://github.com/pallets-eco/flask-session/commit/86895b523203ca67c9f87416bdbf028852dcb357&gt;`_). - Specifically include backend session interfaces in public API and document usage (`[#210](pallets-eco/flask-session#210) &lt;https://github.com/pallets-eco/flask-session/issues/210&gt;`_). - Fix non-permanent sessions not updating expiry (`[#221](pallets-eco/flask-session#221) &lt;https://github.com/pallets-eco/flask-session/issues/221&gt;`_). 0.6.0 - 2024-01-16 ------------------ &lt;/tr&gt;&lt;/table&gt; </code></pre> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="pallets-eco/flask-session@65c05eae0e6d679047394b39a8d6e2bce8195ac3"><code>65c05ea</code></a> Prep release</li> <li><a href="pallets-eco/flask-session@475cc038376d4d5c2949edf7751a61ed64b7f33c"><code>475cc03</code></a> Merge branch 'erik' into development</li> <li><a href="pallets-eco/flask-session@1af5cb223bcb4bb556cf8b49e803700ac332ab89"><code>1af5cb2</code></a> Remind the correct PR branch</li> <li><a href="pallets-eco/flask-session@b7a219bcac97e74276d8e419e6174aea2c4ffdad"><code>b7a219b</code></a> Reduce documentation and remove filesystem extra</li> <li><a href="pallets-eco/flask-session@6c9a698b61ead380988ba4b69b0f161de47e1886"><code>6c9a698</code></a> Fix docs announcement overflow</li> <li><a href="pallets-eco/flask-session@f8f5ad15f787c5b30b663f84883ad2c8108b8314"><code>f8f5ad1</code></a> implemented DynamoDBSessionInterface and tests. (<a href="https://github.com/pallets-eco/flask-session/issues/214">#214</a>)</li> <li><a href="pallets-eco/flask-session@5a3413b9b1d67a4116404f23022b58326a60fbdf"><code>5a3413b</code></a> Add changelog and contributor</li> <li><a href="pallets-eco/flask-session@17369528b45d9b5a59003ec006ede35ced40f694"><code>1736952</code></a> Fix docs requirements</li> <li><a href="pallets-eco/flask-session@a92914ed1196acc2eab4d562996ac944413fb19c"><code>a92914e</code></a> Add docs</li> <li><a href="pallets-eco/flask-session@ab9a756a18de87033cf6d3b2d86c6dc46e006800"><code>ab9a756</code></a> Tidy up dynamo db, add docker and requirements</li> <li>Additional commits viewable in <a href="pallets-eco/flask-session@0.6.1rc1...0.8.0">compare view</a></li> </ul> </details> <br /> </code></pre> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Have added option for either a background thread or per-request cleanup for SQL. Initially I thought this is out of scope for flask-session, but thought it might be worth to try. There have been some PRs for per-request cleanup but I thought this would have less desirable so also tried the threads option. It may be that we implement only that one.