Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Bumps Flask Caching to fix RCE vulnerability #25090

Merged
merged 3 commits into from
Aug 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 43 additions & 45 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -578,6 +578,7 @@ npm ci
```

Note that Superset uses [Scarf](https://docs.scarf.sh) to capture telemetry/analytics about versions being installed, including the `scarf-js` npm package. As noted elsewhere in this documentation, Scarf gathers aggregated stats for the sake of security/release strategy, and does not capture/retain PII. [You can read here](https://docs.scarf.sh/package-analytics/) about the package, and various means to opt out of it, but one easy way to opt out is to add this setting in `superset-frontent/package.json`:

```json
// your-package/package.json
{
Expand All @@ -598,10 +599,13 @@ There are three types of assets you can build:
3. `npm run build-instrumented`: instrumented application code for collecting code coverage from Cypress tests

If this type of error comes while building assets(i.e using above commands):

```bash
Error: You must provide the URL of lib/mappings.wasm by calling SourceMapConsumer.initialize
```

Then put this:

```bash
export NODE_OPTIONS=--no-experimental-fetch
```
Expand Down Expand Up @@ -925,28 +929,22 @@ For debugging locally using VSCode, you can configure a launch configuration fil

```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Flask",
"type": "python",
"request": "launch",
"module": "flask",
"env": {
"FLASK_APP": "superset",
"SUPERSET_ENV": "development"
},
"args": [
"run",
"-p 8088",
"--with-threads",
"--reload",
"--debugger"
],
"jinja": true,
"justMyCode": true
}
]
"version": "0.2.0",
"configurations": [
{
"name": "Python: Flask",
"type": "python",
"request": "launch",
"module": "flask",
"env": {
"FLASK_APP": "superset",
"SUPERSET_ENV": "development"
},
"args": ["run", "-p 8088", "--with-threads", "--reload", "--debugger"],
"jinja": true,
"justMyCode": true
}
]
}
```

Expand Down Expand Up @@ -1031,24 +1029,24 @@ You are now ready to attach a debugger to the process. Using VSCode you can conf

```json
{
"version": "0.2.0",
"configurations": [
"version": "0.2.0",
"configurations": [
{
"name": "Attach to Superset App in Docker Container",
"type": "python",
"request": "attach",
"connect": {
"host": "127.0.0.1",
"port": 5678
},
"pathMappings": [
{
"name": "Attach to Superset App in Docker Container",
"type": "python",
"request": "attach",
"connect": {
"host": "127.0.0.1",
"port": 5678
},
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/app"
}
]
},
]
"localRoot": "${workspaceFolder}",
"remoteRoot": "/app"
}
]
}
]
}
```

Expand Down Expand Up @@ -1349,7 +1347,7 @@ To do this, you'll need to:
but perfect for testing (stores cache in `/tmp`)

```python
from cachelib.file import FileSystemCache
from flask_caching.backends.filesystemcache import FileSystemCache
RESULTS_BACKEND = FileSystemCache('/tmp/sqllab')
```

Expand Down Expand Up @@ -1415,11 +1413,11 @@ Note not all fields are correctly categorized. The fields vary based on visualiz

### Time

| Field | Type | Notes |
| ------------------ | -------- | ------------------------------------- |
| `granularity_sqla` | _string_ | The SQLA **Time Column** widget |
| `time_grain_sqla` | _string_ | The SQLA **Time Grain** widget |
| `time_range` | _string_ | The **Time range** widget |
| Field | Type | Notes |
| ------------------ | -------- | ------------------------------- |
| `granularity_sqla` | _string_ | The SQLA **Time Column** widget |
| `time_grain_sqla` | _string_ | The SQLA **Time Grain** widget |
| `time_range` | _string_ | The **Time range** widget |

### GROUP BY

Expand Down
2 changes: 1 addition & 1 deletion docker/pythonpath_dev/superset_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
import logging
import os

from cachelib.file import FileSystemCache
from celery.schedules import crontab
from flask_caching.backends.filesystemcache import FileSystemCache

logger = logging.getLogger()

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/installation/async-queries-celery.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ celery --app=superset.tasks.celery_app:app beat
```

To setup a result backend, you need to pass an instance of a derivative of from
cachelib.base.BaseCache to the RESULTS_BACKEND configuration key in your superset_config.py. You can
from flask_caching.backends.base import BaseCache to the RESULTS_BACKEND configuration key in your superset_config.py. You can
use Memcached, Redis, S3 (https://pypi.python.org/pypi/s3werkzeugcache), memory or the file system
(in a single server-type setup or for testing), or to write your own caching interface. Your
`superset_config.py` may look something like:
Expand All @@ -79,7 +79,7 @@ S3_CACHE_KEY_PREFIX = 'sql_lab_result'
RESULTS_BACKEND = S3Cache(S3_CACHE_BUCKET, S3_CACHE_KEY_PREFIX)

# On Redis
from cachelib.redis import RedisCache
from flask_caching.backends.rediscache import RedisCache
RESULTS_BACKEND = RedisCache(
host='localhost', port=6379, key_prefix='superset_results')
```
Expand Down
2 changes: 1 addition & 1 deletion helm/superset/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ maintainers:
- name: craig-rueda
email: craig@craigrueda.com
url: https://github.com/craig-rueda
version: 0.10.5
version: 0.10.6
dependencies:
- name: postgresql
version: 12.1.6
Expand Down
2 changes: 1 addition & 1 deletion helm/superset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ NOTE: This file is generated by helm-docs: https://github.com/norwoodj/helm-docs

# superset

![Version: 0.10.5](https://img.shields.io/badge/Version-0.10.5-informational?style=flat-square)
![Version: 0.10.6](https://img.shields.io/badge/Version-0.10.6-informational?style=flat-square)

Apache Superset is a modern, enterprise-ready business intelligence web application

Expand Down
2 changes: 1 addition & 1 deletion helm/superset/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Create chart name and version as used by the chart label.

{{- define "superset-config" }}
import os
from cachelib.redis import RedisCache
from flask_caching.backends.rediscache import RedisCache

def env(key, default=None):
return os.getenv(key, default)
Expand Down
6 changes: 3 additions & 3 deletions requirements/base.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ bottleneck==1.3.7
# via pandas
brotli==1.0.9
# via flask-compress
cachelib==0.4.1
# via apache-superset
cachelib==0.6.0
# via flask-caching
Comment on lines +34 to +35
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm tempted to bump this one to a more recent one, but let's do that when we bump to Flask-Caching>=2

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's safer to do that when bumping Flask-Caching 👍🏼

celery==5.2.2
# via apache-superset
certifi==2023.7.22
Expand Down Expand Up @@ -100,7 +100,7 @@ flask-appbuilder==4.3.6
# via apache-superset
flask-babel==1.0.0
# via flask-appbuilder
flask-caching==1.10.1
flask-caching==1.11.1
# via apache-superset
flask-compress==1.13
# via apache-superset
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ combine_as_imports = true
include_trailing_comma = true
line_length = 88
known_first_party = superset
known_third_party =alembic,apispec,backoff,cachelib,celery,click,colorama,cron_descriptor,croniter,cryptography,dateutil,deprecation,flask,flask_appbuilder,flask_babel,flask_caching,flask_compress,flask_jwt_extended,flask_login,flask_migrate,flask_sqlalchemy,flask_talisman,flask_testing,flask_wtf,freezegun,geohash,geopy,holidays,humanize,isodate,jinja2,jwt,markdown,markupsafe,marshmallow,msgpack,nh3,numpy,pandas,parameterized,parsedatetime,pgsanity,pkg_resources,polyline,prison,progress,pyarrow,sqlalchemy_bigquery,pyhive,pyparsing,pytest,pytest_mock,pytz,redis,requests,selenium,setuptools,shillelagh,simplejson,slack,sqlalchemy,sqlalchemy_utils,sqlparse,typing_extensions,urllib3,werkzeug,wtforms,wtforms_json,yaml
known_third_party =alembic,apispec,backoff,celery,click,colorama,cron_descriptor,croniter,cryptography,dateutil,deprecation,flask,flask_appbuilder,flask_babel,flask_caching,flask_compress,flask_jwt_extended,flask_login,flask_migrate,flask_sqlalchemy,flask_talisman,flask_testing,flask_wtf,freezegun,geohash,geopy,holidays,humanize,isodate,jinja2,jwt,markdown,markupsafe,marshmallow,msgpack,nh3,numpy,pandas,parameterized,parsedatetime,pgsanity,pkg_resources,polyline,prison,progress,pyarrow,sqlalchemy_bigquery,pyhive,pyparsing,pytest,pytest_mock,pytz,redis,requests,selenium,setuptools,shillelagh,simplejson,slack,sqlalchemy,sqlalchemy_utils,sqlparse,typing_extensions,urllib3,werkzeug,wtforms,wtforms_json,yaml
multi_line_output = 3
order_by_type = false

Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ def get_git_sha() -> str:
},
install_requires=[
"backoff>=1.8.0",
"cachelib>=0.4.1,<0.5",
"celery>=5.2.2, <6.0.0",
"click>=8.0.3",
"click-option-group",
Expand All @@ -86,7 +85,7 @@ def get_git_sha() -> str:
"deprecation>=2.1.0, <2.2.0",
"flask>=2.2.5, <3.0.0",
"flask-appbuilder>=4.3.6, <5.0.0",
"flask-caching>=1.10.1, <2.0",
"flask-caching>=1.11.1, <2.0",
"flask-compress>=1.13, <2.0",
"flask-talisman>=1.0.0, <2.0",
"flask-login>=0.6.0, < 1.0",
Expand Down
2 changes: 1 addition & 1 deletion superset/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@
from typing import Any, Callable, Literal, TYPE_CHECKING, TypedDict

import pkg_resources
from cachelib.base import BaseCache
from celery.schedules import crontab
from flask import Blueprint
from flask_appbuilder.security.manager import AUTH_DB
from flask_caching.backends.base import BaseCache
from pandas import Series
from pandas._libs.parsers import STR_NA_VALUES # pylint: disable=no-name-in-module
from sqlalchemy.orm.query import Query
Expand Down
2 changes: 1 addition & 1 deletion superset/extensions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@
from typing import Any, Callable, Optional

import celery
from cachelib.base import BaseCache
from flask import Flask
from flask_appbuilder import AppBuilder, SQLA
from flask_caching.backends.base import BaseCache
from flask_migrate import Migrate
from flask_talisman import Talisman
from flask_wtf.csrf import CSRFProtect
Expand Down
5 changes: 5 additions & 0 deletions tests/integration_tests/cachekeys/api_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@
SupersetTestCase,
post_assert_metric,
)
from tests.integration_tests.fixtures.birth_names_dashboard import (
load_birth_names_dashboard_with_slices,
load_birth_names_data,
)


@pytest.fixture
Expand Down Expand Up @@ -95,6 +99,7 @@ def test_invalidate_cache_bad_request(invalidate):
assert rv.status_code == 400


@pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
def test_invalidate_existing_caches(invalidate):
schema = get_example_default_schema() or ""
bn = SupersetTestCase.get_birth_names_dataset()
Expand Down
1 change: 1 addition & 0 deletions tests/integration_tests/core_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ def test_cache_logging(self):
self.get_json_resp(f"/superset/warm_up_cache?slice_id={slc.id}")
ck = db.session.query(CacheKey).order_by(CacheKey.id.desc()).first()
assert ck.datasource_uid == f"{slc.table.id}__table"
db.session.delete(ck)
app.config["STORE_CACHE_KEYS_IN_METADATA_DB"] = store_cache_keys

def test_redirect_invalid(self):
Expand Down
Loading