Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.3.1 #320

Merged
merged 30 commits into from
Jul 29, 2024
Merged

1.3.1 #320

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5ab9053
update certego-ui
drosetti Nov 27, 2023
3e568f5
Bump slack-sdk from 3.21.3 to 3.26.1 in /requirements (#251)
dependabot[bot] Dec 29, 2023
8d66c96
Bump django from 4.1.11 to 4.1.13 in /requirements (#239)
dependabot[bot] Dec 29, 2023
9b2cd1c
Bump library/nginx from 1.25.2-alpine to 1.25.3-alpine in /docker (#237)
dependabot[bot] Dec 29, 2023
09946bf
Bump psycopg2-binary from 2.9.7 to 2.9.9 in /requirements (#230)
dependabot[bot] Dec 29, 2023
1fd8a59
Bump uwsgi from 2.0.22 to 2.0.24 in /requirements (#266)
dependabot[bot] May 29, 2024
be3d249
Bump library/nginx from 1.25.3-alpine to 1.25.4-alpine in /docker (#269)
dependabot[bot] May 29, 2024
6793fdc
Bump certego-saas from 0.7.0 to 0.7.8 in /requirements (#255)
dependabot[bot] May 29, 2024
e75525e
updated frontend dependencies
drosetti May 29, 2024
371113d
updated use-axios
drosetti May 29, 2024
eee92a8
Bump express from 4.18.1 to 4.19.2 in /frontend (#277)
dependabot[bot] May 29, 2024
48f9d65
Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /frontend (#276)
dependabot[bot] May 29, 2024
5c43f3c
Bump follow-redirects from 1.15.1 to 1.15.6 in /frontend (#274)
dependabot[bot] May 29, 2024
118a7e4
Bump @adobe/css-tools from 4.1.0 to 4.3.2 in /frontend (#249)
dependabot[bot] May 29, 2024
7abe5e3
Bump semver from 5.7.1 to 5.7.2 in /frontend (#204)
dependabot[bot] May 29, 2024
6e2e04f
Bump word-wrap from 1.2.3 to 1.2.5 in /.github/configurations/node_li…
dependabot[bot] May 29, 2024
3f6235b
Bump tough-cookie from 4.1.2 to 4.1.4 in /frontend (#288)
dependabot[bot] May 29, 2024
a63a450
Bump word-wrap from 1.2.3 to 1.2.5 in /frontend (#287)
dependabot[bot] May 29, 2024
930aef0
reverted axios version to incompatibility with other dependencies
drosetti May 29, 2024
5336cd1
Bump postcss from 8.4.24 to 8.4.38 in /.github/configurations/node_li…
dependabot[bot] May 29, 2024
01b8110
Bump ejs from 3.1.8 to 3.1.10 in /frontend (#284)
dependabot[bot] May 29, 2024
90759f5
Honeytest (#319)
mlodic Jul 24, 2024
ecd153e
Bump slack-sdk from 3.26.1 to 3.29.0 in /requirements (#306)
dependabot[bot] Jul 24, 2024
41b96a5
Bump uwsgitop from 0.11 to 0.12 in /requirements (#296)
dependabot[bot] Jul 24, 2024
19f8ee7
Bump library/nginx from 1.25.4-alpine to 1.27.0-alpine in /docker (#303)
dependabot[bot] Jul 24, 2024
8ed0f84
general dependencies upgrades and watchman fix (#279)
mlodic Jul 24, 2024
6bcb350
Bump certego-saas from 0.7.8 to 0.7.10 in /requirements (#297)
dependabot[bot] Jul 24, 2024
78510b2
dump 1.3.1
mlodic Jul 24, 2024
b716707
fixed CSV/txt output
mlodic Jul 24, 2024
447ddb8
removed racaptcha (#322)
drosetti Jul 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/configurations/node_linters/eslint/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 11 additions & 11 deletions .github/configurations/node_linters/stylelint/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def feed_type_validation(feed_type):
feed_choices.extend([hp.name.lower() for hp in generalHoneypots]) # FEEDS

if feed_type not in feed_choices:
logger.info(f"Feed type {feed_type} not in feed_choises {feed_choices}")
logger.info(f"Feed type {feed_type} not in feed_choices {feed_choices}")
raise serializers.ValidationError("Invalid feed_type")
return feed_type

Expand Down
39 changes: 21 additions & 18 deletions api/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,7 @@ def write(self, value):
def feeds(request, feed_type, attack_type, age, format_):
logger.info(f"request /api/feeds with params: feed type: {feed_type}, attack_type: {attack_type}, Age: {age}, format: {format_}")
iocs_queryset = get_queryset(request, feed_type, attack_type, age, format_)
resp_data = feeds_response(request, iocs_queryset, feed_type, format_)
return Response(resp_data, status=status.HTTP_200_OK)
return feeds_response(request, iocs_queryset, feed_type, format_)


@api_view([GET])
Expand All @@ -52,7 +51,7 @@ def feeds_pagination(request):
paginator = CustomPageNumberPagination()
iocs_queryset = get_queryset(request, params["feed_type"], params["attack_type"], params["age"], "json")
iocs = paginator.paginate_queryset(iocs_queryset, request)
resp_data = feeds_response(request, iocs, params["feed_type"], "json")
resp_data = feeds_response(request, iocs, params["feed_type"], "json", dict_only=True)
return paginator.get_paginated_response(resp_data)


Expand Down Expand Up @@ -113,7 +112,7 @@ def get_queryset(request, feed_type, attack_type, age, format_):
return iocs


def feeds_response(request, iocs, feed_type, format_):
def feeds_response(request, iocs, feed_type, format_, dict_only=False):
logger.info(f"Format feeds in: {format_}")
license_text = (
f"# These feeds are generated by The Honeynet Project" f" once every 10 minutes and are protected" f" by the following license: {FEEDS_LICENSE}"
Expand Down Expand Up @@ -167,20 +166,24 @@ def feeds_response(request, iocs, feed_type, format_):
serializer_item.is_valid(raise_exception=True)
json_list.append(serializer_item.data)

# check if sorting the results by feed_type
ordering = request.query_params.get("ordering")
sorted_list = []
if ordering == "feed_type":
sorted_list = sorted(json_list, key=lambda k: k["feed_type"])
elif ordering == "-feed_type":
sorted_list = sorted(json_list, key=lambda k: k["feed_type"], reverse=True)

if sorted_list:
logger.info("Return feeds sorted by feed_type field")
json_list = sorted_list

logger.info(f"Number of feeds returned: {len(json_list)}")
return {"license": FEEDS_LICENSE, "iocs": json_list}
# check if sorting the results by feed_type
ordering = request.query_params.get("ordering")
sorted_list = []
if ordering == "feed_type":
sorted_list = sorted(json_list, key=lambda k: k["feed_type"])
elif ordering == "-feed_type":
sorted_list = sorted(json_list, key=lambda k: k["feed_type"], reverse=True)

if sorted_list:
logger.info("Return feeds sorted by feed_type field")
json_list = sorted_list

logger.info(f"Number of feeds returned: {len(json_list)}")
resp_data = {"license": FEEDS_LICENSE, "iocs": json_list}
if dict_only:
return resp_data
else:
return Response(resp_data, status=status.HTTP_200_OK)


# The Doc does not work as intended. We should refactor this by correctly leveraging DRF
Expand Down
1 change: 0 additions & 1 deletion authentication/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ class Meta:
is_active = rfs.BooleanField(default=False, read_only=True)

def validate_profile(self, profile):

logger.info(f"{profile}")

self._profile_serializer = UserProfileSerializer(data=profile)
Expand Down
22 changes: 5 additions & 17 deletions authentication/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import rest_email_auth.views
from certego_saas.apps.auth import views as certego_views
from certego_saas.apps.auth.backend import CookieTokenAuthentication
from certego_saas.ext.mixins import RecaptchaV2Mixin
from certego_saas.ext.throttling import POSTUserRateThrottle
from django.conf import settings
from django.contrib.auth import get_user_model, login
Expand All @@ -27,13 +26,13 @@
User: AUTH_USER_MODEL = get_user_model()


class PasswordResetRequestView(rest_email_auth.views.PasswordResetRequestView, RecaptchaV2Mixin):
class PasswordResetRequestView(rest_email_auth.views.PasswordResetRequestView):
authentication_classes: List = []
permission_classes: List = []
throttle_classes: List = [POSTUserRateThrottle]


class PasswordResetView(rest_email_auth.views.PasswordResetView, RecaptchaV2Mixin):
class PasswordResetView(rest_email_auth.views.PasswordResetView):
authentication_classes: List = []
permission_classes: List = []
throttle_classes: List = [POSTUserRateThrottle]
Expand All @@ -46,14 +45,14 @@ class EmailVerificationView(rest_email_auth.views.EmailVerificationView):
serializer_class = EmailVerificationSerializer


class RegistrationView(rest_email_auth.views.RegistrationView, RecaptchaV2Mixin):
class RegistrationView(rest_email_auth.views.RegistrationView):
authentication_classes: List = []
permission_classes: List = []
throttle_classes: List = [POSTUserRateThrottle]
serializer_class = RegistrationSerializer


class ResendVerificationView(rest_email_auth.views.ResendVerificationView, RecaptchaV2Mixin):
class ResendVerificationView(rest_email_auth.views.ResendVerificationView):
authentication_classes: List = []
permission_classes: List = []
throttle_classes: List = [POSTUserRateThrottle]
Expand Down Expand Up @@ -93,31 +92,20 @@ def checkConfiguration(request):
if not variable:
errors["SMTP backend"] = "configuration required"

# if you are in production environment
if settings.STAGE_PRODUCTION:
# recaptcha key
if settings.DRF_RECAPTCHA_SECRET_KEY == "fake":
errors["RECAPTCHA_SECRET_KEY"] = "required"

logger.info(f"Configuration errors: {errors}")
if errors:
return Response(status=status.HTTP_501_NOT_IMPLEMENTED)
return Response(status=status.HTTP_200_OK)


class LoginView(certego_views.LoginView, RecaptchaV2Mixin):
class LoginView(certego_views.LoginView):
@staticmethod
def validate_and_return_user(request):
serializer = LoginSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
return serializer.validated_data["user"]

def post(self, request, *args, **kwargs):
try:
self.get_serializer() # for RecaptchaV2Mixin
except AssertionError:
# it will raise this bcz `serializer_class` is not defined
pass
response = super().post(request, *args, **kwargs)
uname = request.user.username
logger.info(f"LoginView: received request from '{uname}'.")
Expand Down
2 changes: 1 addition & 1 deletion docker/.version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
REACT_APP_GREEDYBEAR_VERSION="1.3.0"
REACT_APP_GREEDYBEAR_VERSION="1.3.1"
2 changes: 1 addition & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ RUN npm install
RUN PUBLIC_URL=/static/reactapp/ npm run build

# Stage 2: Backend
FROM python:3.9.16-alpine3.17
FROM python:3.11.9-alpine3.20

ENV PYTHONUNBUFFERED 1
ENV DJANGO_SETTINGS_MODULE greedybear.settings
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile_nginx
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM library/nginx:1.25.2-alpine
FROM library/nginx:1.27.0-alpine
RUN mkdir -p /var/cache/nginx /var/cache/nginx/feeds
RUN apk update && apk upgrade && apk add bash
ENV NGINX_LOG_DIR /var/log/nginx
Expand Down
3 changes: 1 addition & 2 deletions docker/default.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ services:
volumes:
- ../configuration/uwsgi/greedybear.ini:/etc/uwsgi/sites/greedybear.ini
- generic_logs:/var/log/greedybear
- ../frontend/public/env.js:/var/www/reactapp/env.js # this is needed because we have to mount the file from the server to the image BEFORE staticfiles
- static_content:/opt/deploy/greedybear/static
entrypoint:
- ./docker/entrypoint_uwsgi.sh
Expand Down Expand Up @@ -47,7 +46,7 @@ services:
- uwsgi

rabbitmq:
image: library/rabbitmq:3.9-alpine
image: library/rabbitmq:3.12-alpine
container_name: greedybear_rabbitmq
volumes:
- ../configuration/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
Expand Down
6 changes: 0 additions & 6 deletions docker/env_file_template
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,3 @@ MOCK_CONNECTIONS=False

# True for public deployment, False for internal deployment
PUBLIC_DEPLOYMENT=False

# drf-recaptcha
## localhost, internal deployments
RECAPTCHA_SECRET_KEY_GB_LOCAL=""
## public deployment
RECAPTCHA_SECRET_KEY_GB_PUBLIC=""
10 changes: 10 additions & 0 deletions docker/logrotate/application
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/var/lib/docker/volumes/*_generic_logs/_data/*/*.log {
daily
missingok
rotate 2
compress
delaycompress
notifempty
create 0644 www-data www-data
copytruncate
}
10 changes: 10 additions & 0 deletions docker/logrotate/docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/var/log/docker/*.log {
daily
missingok
rotate 2
compress
delaycompress
notifempty
create 0655 syslog adm
copytruncate
}
19 changes: 19 additions & 0 deletions docker/logrotate/nginx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/var/lib/docker/volumes/*_nginx_logs/_data/*.log {
daily
missingok
rotate 8
compress
delaycompress
notifempty
create 0644 www-data www-data
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi \
endscript
postrotate
container_id=$(docker ps -f name=greedybear_nginx --quiet)
docker exec $container_id killall -USR1 nginx
endscript
}
4 changes: 2 additions & 2 deletions docker/watchman_install.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
#!/bin/bash

# This script can be disabled during development using REPO_DOWNLOADER_ENABLED=true env variable
if [ "WATCHMAN" = "false" ]; then echo "Skipping WATCHMAN installation because we are not in test mode"; exit 0; fi
if [ "$WATCHMAN" = "false" ]; then echo "Skipping WATCHMAN installation because we are not in test mode"; exit 0; fi

apk add gcc linux-headers build-base
pip3 install --compile -r requirements/django-server-requirements.txt

# install Watchman to enhance performance on the Django development Server
# https://docs.djangoproject.com/en/3.2/ref/django-admin/#runserver
cd /tmp
wget https://github.com/facebook/watchman/releases/download/v2022.03.21.00/watchman-v2022.03.21.00-linux.zip
wget https://github.com/facebook/watchman/releases/download/v2024.04.15.00/watchman-v2024.04.15.00-linux.zip
unzip watchman-*-linux.zip
cd watchman-*-linux/
mkdir -p /usr/local/{bin,lib} /usr/local/var/run/watchman
Expand Down
12 changes: 0 additions & 12 deletions docs/source/Installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ cd docker/
cp env_file_template env_file
cp env_file_postgres_template env_file_postgres
cd ..
cp frontend/public/env_template.js frontend/public/env.js
```

Now you can start by building the image using docker-compose and run the project.
Expand Down Expand Up @@ -62,17 +61,6 @@ In the `env_file`, configure different variables as explained below.
* `SLACK_TOKEN`: Slack token of your Slack application that will be used to send/receive notifications
* `DEFAULT_SLACK_CHANNEL`: ID of the Slack channel you want to post the message to

#### Recaptcha configuration
The Registration Page and the Login Page contain a Recaptcha form from Google. By default, that Recaptcha is not configured and is not shown.
If your intention is to publish GreedyBear as a Service you should configure different variables as explained below.

In the `frontend/public/env.js` set the variable:
* `RECAPTCHA_SITEKEY`: Recaptcha Key for your site

In the `docker/env_file` set the variables:
* `RECAPTCHA_SECRET_KEY_GB_LOCAL`: your recaptcha secret key internal deployment
* `RECAPTCHA_SECRET_KEY_GB_PUBLIC`: your recaptcha secret key for public deployment

In that case, you would need to [re-build](/Installation.md#update-and-rebuild) the application to have the changes properly reflected.


Expand Down
4 changes: 0 additions & 4 deletions docs/source/Usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@ If you have GreedyBear deployed on an AWS instance you can use the SES service.

In a development environment the emails that would be sent are written to the standard output.

### Recaptcha configuration
The Registration Page contains a Recaptcha form from Google. By default, that Recaptcha is not configured and is not shown.
If your intention is to publish GreedyBear as a Service you should [configure the Recaptcha](/Installation.md#recaptcha-configuration).

### Amazon SES

If you like, you could use Amazon SES for sending automated emails.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/schema.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
openapi: 3.0.3
info:
title: GreedyBear API specification
version: 1.3.0
version: 1.3.1
paths:
/api/apiaccess:
get:
Expand Down
13 changes: 9 additions & 4 deletions frontend/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -182,10 +182,15 @@ module.exports = {
// transform: undefined,

// An array of regexp pattern strings that are matched against all source file paths, matched files will skip transformation
// transformIgnorePatterns: [
// "/node_modules/",
// "\\.pnp\\.[^\\/]+$"
// ],
/* node_modules can need to be transformated in this case we have to exclude them from the regex
Simply add node_modules name to the list
*/
transformIgnorePatterns: [
`/node_modules/(?!(${[
"axios"
].join("|")})/)`,
"\\.pnp\\.[^\\/]+$",
],

// An array of regexp pattern strings that are matched against all modules before the module loader will automatically return a mock for them
// unmockedModulePathPatterns: undefined,
Expand Down
Loading
Loading