Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implemented DynamoDBSessionInterface and tests. #214

Merged
merged 13 commits into from
Mar 24, 2024
4 changes: 4 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ jobs:
image: mongo
ports:
- 27017:27017
dynamodb:
image: amazon/dynamodb-local
ports:
- 8000:8000
steps:
- uses: actions/checkout@v4
- uses: supercharge/redis-github-action@1.5.0
Expand Down
7 changes: 5 additions & 2 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
0.7.2 - 2024-03-21
0.8.0
------------------

Added
~~~~~~~
- Add DynamoDB session interface (`#214 <https://github.com/pallets-eco/flask-session/pull/214>`_).

Fixed
~~~~~
- Include prematurely removed ``cachelib`` dependency. Will be removed in 1.0.0 to be an optional dependency (`#223 <https://github.com/pallets-eco/flask-session/issues/223>`_).
- Note 0.7.1 was not released due to a publishing error.


0.7.0 - 2024-03-18
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## Contributors

- [necat1](https://github.com/necat1)
- [nebolax](https://github.com/nebolax)
- [Taragolis](https://github.com/Taragolis)
- [Lxstr](https://github.com/Lxstr)
Expand Down
16 changes: 12 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
version: '3.8'

services:
dynamodb-local:
image: "amazon/dynamodb-local:latest"
container_name: dynamodb-local
ports:
- "8000:8000"
environment:
- AWS_ACCESS_KEY_ID=dummy
- AWS_SECRET_ACCESS_KEY=dummy
- AWS_DEFAULT_REGION=us-west-2

mongo:
image: mongo:latest
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
ports:
- "27017:27017"
volumes:
Expand All @@ -26,4 +33,5 @@ services:
volumes:
postgres_data:
mongo_data:
redis_data:
redis_data:
dynamodb_data:
3 changes: 2 additions & 1 deletion docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ Anything documented here is part of the public API that Flask-Session provides,
.. autoclass:: flask_session.filesystem.FileSystemSessionInterface
.. autoclass:: flask_session.cachelib.CacheLibSessionInterface
.. autoclass:: flask_session.mongodb.MongoDBSessionInterface
.. autoclass:: flask_session.sqlalchemy.SqlAlchemySessionInterface
.. autoclass:: flask_session.sqlalchemy.SqlAlchemySessionInterface
.. autoclass:: flask_session.dynamodb.DynamoDBSessionInterface
15 changes: 15 additions & 0 deletions docs/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,21 @@ SqlAlchemy

Default: ``None``

Dynamodb
~~~~~~~~~~~~~~~~~~~~~~~

.. py:data:: SESSION_DYNAMODB
A ``boto3.resource`` instance.
Default: Instance connected to ``'localhost:8000'``
.. py:data:: SESSION_DYNAMODB_TABLE_NAME
The name of the table you want to use.
Default: ``'Sessions'``
.. deprecated:: 0.7.0

``SESSION_FILE_DIR``, ``SESSION_FILE_THRESHOLD``, ``SESSION_FILE_MODE``. Use ``SESSION_CACHELIB`` instead.
Expand Down
2 changes: 2 additions & 0 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ Flask-Session has an increasing number of directly supported storage and client
- pymongo_
* - SQL Alchemy
- flask-sqlalchemy_
* - DynamoDB
- boto3_

Other libraries may work if they use the same commands as the ones listed above.

Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -85,5 +85,7 @@ dev-dependencies = [
"sphinx>=7.1.2",
"furo>=2024.1.29",
"sphinx-favicon>=1.0.1",
"boto3>=1.34.68",
"mypy_boto3_dynamodb>=1.34.67",
"pymemcache>=4.0.0",
]
2 changes: 2 additions & 0 deletions requirements/dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,6 @@ redis
pymemcache
Flask-SQLAlchemy
pymongo
boto3
mypy_boto3_dynamodb

4 changes: 3 additions & 1 deletion requirements/docs.in
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,6 @@ redis
cachelib
pymongo
flask_sqlalchemy
pymemcache
pymemcache
boto3
mypy_boto3_dynamodb
26 changes: 24 additions & 2 deletions requirements/docs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ beautifulsoup4==4.12.3
# via furo
blinker==1.7.0
# via flask
boto3==1.34.69
# via -r requirements/docs.in
botocore==1.34.69
# via
# boto3
# s3transfer
cachelib==0.12.0
# via -r requirements/docs.in
certifi==2023.5.7
Expand Down Expand Up @@ -40,10 +46,16 @@ jinja2==3.1.2
# via
# flask
# sphinx
jmespath==1.0.1
# via
# boto3
# botocore
markupsafe==2.1.2
# via
# jinja2
# werkzeug
mypy-boto3-dynamodb==1.34.67
# via -r requirements/docs.in
packaging==23.1
# via sphinx
pygments==2.15.1
Expand All @@ -54,10 +66,16 @@ pymemcache==4.0.0
# via -r requirements/docs.in
pymongo==4.6.2
# via -r requirements/docs.in
python-dateutil==2.9.0.post0
# via botocore
redis==5.0.1
# via -r requirements/docs.in
requests==2.30.0
# via sphinx
s3transfer==0.10.1
# via boto3
six==1.16.0
# via python-dateutil
snowballstemmer==2.2.0
# via sphinx
soupsieve==2.5
Expand Down Expand Up @@ -87,8 +105,12 @@ sphinxcontrib-serializinghtml==1.1.5
sqlalchemy==2.0.27
# via flask-sqlalchemy
typing-extensions==4.10.0
# via sqlalchemy
# via
# mypy-boto3-dynamodb
# sqlalchemy
urllib3==2.0.2
# via requests
# via
# botocore
# requests
werkzeug==3.0.1
# via flask
15 changes: 15 additions & 0 deletions src/flask_session/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,12 @@ def _get_interface(self, app):
"SESSION_CLEANUP_N_REQUESTS", Defaults.SESSION_CLEANUP_N_REQUESTS
)

# DynamoDB settings
SESSION_DYNAMODB = config.get("SESSION_DYNAMODB", Defaults.SESSION_DYNAMODB)
SESSION_DYNAMODB_TABLE = config.get(
"SESSION_DYNAMODB_TABLE", Defaults.SESSION_DYNAMODB_TABLE
)

common_params = {
"app": app,
"key_prefix": SESSION_KEY_PREFIX,
Expand Down Expand Up @@ -165,6 +171,15 @@ def _get_interface(self, app):
bind_key=SESSION_SQLALCHEMY_BIND_KEY,
cleanup_n_requests=SESSION_CLEANUP_N_REQUESTS,
)
elif SESSION_TYPE == "dynamodb":
from .dynamodb import DynamoDBSessionInterface

session_interface = DynamoDBSessionInterface(
**common_params,
client=SESSION_DYNAMODB,
table_name=SESSION_DYNAMODB_TABLE,
)

else:
raise ValueError(f"Unrecognized value for SESSION_TYPE: {SESSION_TYPE}")

Expand Down
4 changes: 4 additions & 0 deletions src/flask_session/defaults.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,7 @@ class Defaults:
SESSION_SQLALCHEMY_SEQUENCE = None
SESSION_SQLALCHEMY_SCHEMA = None
SESSION_SQLALCHEMY_BIND_KEY = None

# DynamoDB settings
SESSION_DYNAMODB = None
SESSION_DYNAMODB_TABLE = "Sessions"
1 change: 1 addition & 0 deletions src/flask_session/dynamodb/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .dynamodb import DynamoDBSession, DynamoDBSessionInterface # noqa: F401
126 changes: 126 additions & 0 deletions src/flask_session/dynamodb/dynamodb.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
import warnings
from datetime import datetime
from datetime import timedelta as TimeDelta
from decimal import Decimal
from typing import Optional

import boto3
from mypy_boto3_dynamodb.service_resource import DynamoDBServiceResource
from flask import Flask
from itsdangerous import want_bytes

from ..base import ServerSideSession, ServerSideSessionInterface
from ..defaults import Defaults


class DynamoDBSession(ServerSideSession):
pass


class DynamoDBSessionInterface(ServerSideSessionInterface):
"""A Session interface that uses dynamodb as backend. (`boto3` required)

:param client: A ``DynamoDBServiceResource`` instance.
:param key_prefix: A prefix that is added to all DynamoDB store keys.
:param use_signer: Whether to sign the session id cookie or not.
:param permanent: Whether to use permanent session or not.
:param sid_length: The length of the generated session id in bytes.
:param table_name: DynamoDB table name to store the session.

.. versionadded:: 0.6
The `sid_length` parameter was added.

.. versionadded:: 0.2
The `use_signer` parameter was added.
"""

session_class = DynamoDBSession

def __init__(
self,
app: Flask,
client: Optional[DynamoDBServiceResource] = Defaults.SESSION_DYNAMODB,
key_prefix: str = Defaults.SESSION_KEY_PREFIX,
use_signer: bool = Defaults.SESSION_USE_SIGNER,
permanent: bool = Defaults.SESSION_PERMANENT,
sid_length: int = Defaults.SESSION_ID_LENGTH,
serialization_format: str = Defaults.SESSION_SERIALIZATION_FORMAT,
table_name: str = Defaults.SESSION_DYNAMODB_TABLE,
):

if client is None:
warnings.warn(
"No valid DynamoDBServiceResource instance provided, attempting to create a new instance on localhost:8000.",
RuntimeWarning,
stacklevel=1,
)
client = boto3.resource(
"dynamodb",
endpoint_url="http://localhost:8000",
region_name="us-west-2",
aws_access_key_id="dummy",
aws_secret_access_key="dummy",
)

try:
client.create_table(
AttributeDefinitions=[
{"AttributeName": "id", "AttributeType": "S"},
],
TableName=table_name,
KeySchema=[
{"AttributeName": "id", "KeyType": "HASH"},
],
BillingMode="PAY_PER_REQUEST",
)
client.meta.client.get_waiter("table_exists").wait(TableName=table_name)
client.meta.client.update_time_to_live(
TableName=self.table_name,
TimeToLiveSpecification={
"Enabled": True,
"AttributeName": "expiration",
},
)
except (AttributeError, client.meta.client.exceptions.ResourceInUseException):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there may be a problem here:

Scenario:

  • Running inside a real environment, trying to use existing table.
  • The instance or whatever is running this code doesn't have dynamodb:CreateTable or dynamodb:UpdateTimeToLive or dynamodb:Describe/GetTable permissions
  • I'd expect an access denied error in this case, because it doesn't even get to the point of checking if the table exists etc.
  • This except probably doesn't handle that.

This means we'd have to give whichever resource is running this permission to use the CreateTable etc. APIs even though we want to use an existing table. Not ideal from a security perspective.

(Purely theoretical at this point, but I'm fairly certain this is what would happen)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, seems more reasonable to check if the table exists before the try, is that what you would suggest? Could you add a PR to development?

Copy link
Contributor

@MauriceBrg MauriceBrg Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a security perspective I'd prefer that the implementation can work with minimal permissions, in this case:

  • dynamodb:DeleteItem
  • dynamodb:GetItem
  • dynamodb:UpdateItem

This would be sufficient for read and write access to a properly configured table.

All this table and TTL management is something that we usually like to handle through Infrastructure as Code. Many companies also require additional configuration regarding backups, etc.

Having the option of doing that through flask-session is nice for convenience, but IMO a bit too much magic that happens by default.

I'd prefer an create_table flag or something like that, which is should be disabled by default.
If we find a good approach that balances different requirements I may be able to add a PR.


Got distracted and forgot the middle part :D

I'd prefer to handle the non-existence of a table when we try to write to it/read from it. The ask for forgiveness instead of approval approach - that requires fewer API calls and AWS permissions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean. I think the first step would be to factor out the creation into a single method on each interface. Similar to what is done here https://github.com/kroketio/quart-session/blob/master/quart_session/sessions.py.This could then be easily subclassed to skip the method (maybe like 3 lines of code).

The second step would be adding the config setting for all interfaces, which I am less sure of (adding more settings) unless there is a bit of demand or discussing with the team.

I think if the first was done neatly it would be merged quickly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm currently looking into this, but I'm not super familiar with the codebase yet.

My approach would be to:

  1. Refactor DynamoDBSessionInterface to have the create table logic as a separate function
  2. Create a subclass ExistingDynamoDBTableSessionInterface that skips the create logic
  3. Add a SESSION_TYPE existing_dynamodb_table to flask_session/__init__.py that uses 2)
  4. Tests, obviously

This would result in there being two DynamoDB varieties for the SESSION_TYPE.
That would make this implementation different from the others.

I'm considering another optional flag to the constructor of DynamoDBSessionInterface, something like table_already_exists (Default: False), which would skip the create function.
This would probably require another DynamoDB parameter, e.g. SESSION_DYNAMODB_TABLE_ALREADY_EXISTS.

Not sure what is a better approach, any thoughts @Lxstr ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify a bit

Step 1 is good.

This is already done for postgresql (impending PR) but would also need to be done for SQLAlchemy and the base class

Steps 2 and 3 can be instead adding the flag SESSION_TABLE_EXISTS

Base class should look something like:

class ServerSideSessionInterface(FlaskSessionInterface, ABC):
    ...

    def __init__(... session_table_exists: Optional[bool] = Defaults.SESSION_TABLE_EXISTS,):
        ...
        if not session_table_exists:
            self._create_schema_and_table()

    def _create_schema_and_table():
        pass

Tests would be great

Does that make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good to me, I've added a variation of that as a PR: #237

# TTL already exists, or table already exists
pass

self.client = client
self.store = client.Table(table_name)
super().__init__(
app,
key_prefix,
use_signer,
permanent,
sid_length,
serialization_format,
)

def _retrieve_session_data(self, store_id: str) -> Optional[dict]:
# Get the saved session (document) from the database
document = self.store.get_item(Key={"id": store_id}).get("Item")
if document:
serialized_session_data = want_bytes(document.get("val").value)
return self.serializer.decode(serialized_session_data)
return None

def _delete_session(self, store_id: str) -> None:
self.store.delete_item(Key={"id": store_id})

def _upsert_session(
self, session_lifetime: TimeDelta, session: ServerSideSession, store_id: str
) -> None:
storage_expiration_datetime = datetime.utcnow() + session_lifetime
# Serialize the session data
serialized_session_data = self.serializer.encode(session)

self.store.update_item(
Key={
"id": store_id,
},
UpdateExpression="SET val = :value, expiration = :exp",
ExpressionAttributeValues={
":value": serialized_session_data,
":exp": Decimal(storage_expiration_datetime.timestamp()),
},
)
Loading
Loading