harden connections to redis

[rkuo-danswer]

Description

Fixes DAN-764.
Sets several retry and keepalive mechanisms across our own redis pools and celery's connections to redis.

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Accepted Risk

[Any know risks or failure modes to point out to reviewers]

Related Issue(s)

[If applicable, link to the issue(s) this PR addresses]

Checklist:

• All of the automated tests pass
• All PR comments are addressed and marked resolved
• If there are migrations, they have been rebased to latest main
• If there are new dependencies, they are added to the requirements
• If there are new environment variables, they are added to all of the deployment methods
• If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
• Docker images build and basic functionalities work
• Author has done a final read through of the PR right before merge

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 4, 2024 0:31am

[greptile-apps]

PR Summary

This PR focuses on improving Redis connection handling and Celery configurations in the Danswer application, aiming to enhance reliability and stability, particularly in production environments.

• Introduced new Redis health check and connection pool configurations in app_configs.py
• Added Redis socket keepalive options in constants.py to maintain persistent connections
• Enhanced redis_pool.py with configurable health checks and keepalive settings for Redis connections
• Updated Celery configuration in celeryconfig.py with retry mechanisms and improved Redis connection handling
• Added CELERY_BROKER_POOL_LIMIT environment variable to Docker Compose files for better Redis connection management

_{6 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings}