Very slow response from the server when signup/login #5230
-
|
I'm using django-allauth to handle user signup/login. The current setup works, but the response from the server is incredibly slow (>15s, other types of responses are usually <500ms). In base.py (I used cookiecutter-django to setup everything), the relevant settings are LOGIN_REDIRECT_URL = "myapp:home"
LOGIN_URL = "account_login"
ACCOUNT_SIGNUP_REDIRECT_URL = reverse_lazy("myapp:accept-terms-of-service")
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = "mandatory"
ACCOUNT_ADAPTER = "project.users.adapters.AccountAdapter"In adapters.py, class AccountAdapter(DefaultAccountAdapter):
def is_open_for_signup(self, request: HttpRequest) -> bool:
return getattr(settings, "ACCOUNT_ALLOW_REGISTRATION", True)
def send_mail(self, template_prefix, email, context):
mailing_thread = threading.Thread(
target=super().send_mail,
args=(template_prefix, email, context),
)
mailing_thread.start()In views.py class CustomLoginView(LoginView):
def form_valid(self, form):
response = super().form_valid(form)
user_id = self.request.user.id
if not UserProfile.objects.filter(user_id=user_id).exists():
return redirect("myapp:accept-terms-of-service")
return response
def home(request):
return render(request, "myapp/home.html")The idea is to send a verification email to the user when he signs up. And when he first signs in, he will need to go to the process of accepting the legal documents. The responses of two requests are very slow: 1. When the user is signing up, there is a significant delay before the server makes a response, though ultimately the user receives the correct email. 2. When the user first logs in, there is a significant delay before showing the accept terms page. I have tried to use both django celery and threads to send the email, but the delay doesn't change at all (BTW all of them works, just very slow). So it's probably not a blocking problem. I am not sure if this is related to how boto3 is configured because it seems that AWS SES is ultimately sending the right email. But the following appears in the terminal each time when a user signs up: |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 10 replies
-
|
[Note: cross-posted to pennersr/django-allauth#3978 and anymail/django-anymail#386.] |
Beta Was this translation helpful? Give feedback.
-
|
Are you using the debug toolbar? It can be slow with many queries. Attach a profiler and you will find the slowiness. |
Beta Was this translation helpful? Give feedback.
-
|
The duration is before the query right? |
Beta Was this translation helpful? Give feedback.
-
|
Oh, that's right. So in this case, the executions of these queries are actually very fast. It is something else that is causing the delay. How can I get more details on what's happening between these queries? This is the logging config that I use: LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"verbose": {
"format": "%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s",
},
},
"handlers": {
"console": {
"level": "DEBUG",
"class": "logging.StreamHandler",
"formatter": "verbose",
},
"file": {
"level": "DEBUG",
"class": "logging.FileHandler",
"filename": "logs/django.log",
"formatter": "verbose",
},
},
"root": {"level": "DEBUG", "handlers": ["console", "file"]},
"loggers": {
"": {
"handlers": ["file", "console"],
"level": "DEBUG",
"propagate": True,
},
"django": {
"handlers": ["file", "console"],
"level": "DEBUG",
"propagate": False,
},
"django.request": {
"handlers": ["file", "console"],
"level": "DEBUG",
"propagate": False,
},
"django.db.backends": {
"handlers": ["file", "console"],
"level": "DEBUG",
"propagate": False,
},
},
} |
Beta Was this translation helpful? Give feedback.
-
|
If you think SES is causing it change the mail provider or use a smtp account. Another option is to use the production configuration on you dev machine. If you use sentry you can measure the performance of your app. Set the profiling and tracing params as described here: https://docs.sentry.io/platforms/python/integrations/django/#configure |
Beta Was this translation helpful? Give feedback.
-
|
(One of the cross-posted threads concluded SES was not the issue.) You mentioned (in a cross-post) you're using django-silk. That includes a SQL Inspection view where you can sort queries by execution time, which might help identify the problem. (But note that django-silk stores its measurements in your DB, so if you're running into DB performance issues django-silk might be making it worse—or could even be the root cause.) There are several reasons your production performance might differ from dev. Are you on similar same hardware? Does your dev database have as much data as production? Is your UserProfile.user_id a ForeignKey or OneToOneField (or otherwise indexed)? If not, that query in your CustomLoginView is going to get slower and slower the more users you add. Or it might be unrelated to the DB. One of the cross-posted threads asked whether you're doing anything else during signup that would involve a network call—such as fetching profile info/photos from auth providers. Have you looked into this? Have you checked your production error logs? When your mailing_thread raises an unhandled exception where does that error end up? (You're not handling it, and networks are never 100% reliable, so this is a "when" not an "if".) Any other errors messages around the time the delays occur? |
Beta Was this translation helpful? Give feedback.
-
Thanks for the suggestion. I'll give sentry a shot. |
Beta Was this translation helpful? Give feedback.
-
|
@medmunds @foarsitter The problem is due to RedisCache. I used ssh tunnels to test the local dev version of the same project on the server, and the slow page response issues never came up. After that, in my CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": env("REDIS_URL"),
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
# Mimicing memcache behavior.
# https://github.com/jazzband/django-redis#memcached-exceptions-behavior
"IGNORE_EXCEPTIONS": True,
},
},
}with the following CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "",
},
}and using the production setting, the problems went away. My understanding is that in production, it is preferable to use RedisCache rather than MemCache, but I still haven't figured out why in this case Redis is causing trouble. |
Beta Was this translation helpful? Give feedback.
-
|
I dont know how IGNORE_EXCEPTIONS exactly behaves but I think it hides the exception behind all this. Probably your redis url is invalid. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it is indeed a Redis URL issue. Part of the reason is that I develop using local settings and in production, IGNORE_EXCEPTIONS is set to be True by cookiecutter-django. Therefore there're no debug messages. I'd suggest that IGNORE_EXCEPTIONS should be set to be False by default for developers to discover what's wrong easily. |
Beta Was this translation helpful? Give feedback.
-
|
Big thanks to @medmunds and @foarsitter. I appreciate your help. |
Beta Was this translation helpful? Give feedback.
I dont know how IGNORE_EXCEPTIONS exactly behaves but I think it hides the exception behind all this.
Probably your redis url is invalid.