Welcome to the exciting world of our Django project! This repository combines the power of Django and Celery to deliver a robust and efficient backend solution. Below, you'll find all the information you need to get started, run migrations, and utilize the project effectively.
- Introduction
- Prerequisites
- Installation
- Configuration
- Running the Project
- Dockerizing Project Execution
- Running Migrations
- Environment Variables
- Troubleshooting
- Contributing
- License
Our project combines Django, a powerful web framework, and Celery, a distributed task queue system. This combination allows for seamless handling of background tasks, ensuring a responsive user experience and efficient resource utilization.
Before you begin, make sure you have the following installed:
- project-specific dependencies (see
requirements.txt
)
-
Clone this repository:
git clone https://github.com/openchatai/OpenChat.git cd dj_backend_server
-
Create a virtual environment:
python3 -m venv venv source venv/bin/activate
-
Install dependencies:
pip install -r requirements.txt
If database migrations fail to run for any reason, you can use the following command to re-run the migrations within the Docker container:
make force_migrate
To address migration issues within the Docker container, a new Makefile target named force_migrate
has been introduced. This target allows you to forcefully re-run the migrations, which can be useful in resolving migration-related problems.
The Makefile now includes logic to dynamically select the appropriate Docker Compose file based on the underlying system architecture. This decision is made between two options: docker-compose.yaml
for non-Linux environments and docker-compose.linux.yaml
for Linux systems. This ensures that the correct Docker Compose configuration is utilized according to the specific environment in use.
To run migrations inside the Docker container, you can use the following command:
make migrate
## Configuration
Before running the project, you need to configure your environment variables. Rename the `.env.example` file to `.env` and fill in the necessary values for your environment.
## Running the Project
To start the Django development server along with Celery:
```bash
python manage.py runserver
To start the Celery worker:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
export DISABLE_SPRING=true
celery -A dj_backend_server worker --loglevel=info
To apply database migrations:
python manage.py migrate
Certainly, here's the rephrased content with a consistent and user-friendly style:
Here's an improved version of your markdown:
This section provides a detailed breakdown of the environment variables defined within the .env
file. These variables are crucial for configuring your application and services.
OPENAI_API_TYPE
: Specifies the type of API, which can be set to either 'azure' or 'openai'.OPENAI_API_BASE
: The base URL for the OpenAI API.OPENAI_API_KEY
: Your unique OpenAI API key.OPENAI_API_VERSION
: The version of the OpenAI API in use.OPENAI_EMBEDDING_MODEL_NAME
: The name of the specific embedding model being utilized.OPENAI_DEPLOYMENT_NAME
: The designated deployment name.OPENAI_COMPLETION_MODEL
: The exact completion model in use, e.g., 'gpt-3.5-turbo'.EMBEDDING_PROVIDER
: The chosen provider for embeddings, typically set to 'openai'.STORE
: The vector store option, which can be either 'PINECONE' or 'QDRANT'.PINECONE_API_KEY
: API key for Pinecone, if applicable.PINECONE_ENV
: Pinecone environment identifier, if used.VECTOR_STORE_INDEX_NAME
: The name assigned to the vector store index, if applicable.QDRANT_URL
: The URL for Qdrant, if it's part of your setup.
CELERY_BROKER_URL
: Redis broker URL for Celery (e.g.,redis://localhost:6379/0
).CELERY_RESULT_BACKEND
: Redis backend for Celery (e.g.,redis://localhost:6379/0
).
DATABASE_NAME
: Name of the database.DATABASE_USER
: Username for database access.DATABASE_PASSWORD
: Password for database access.DATABASE_HOST
: Hostname of the database, usually 'localhost' in this context.DATABASE_PORT
: Port number for the database connection (e.g.,3306
).
ALLOWED_HOSTS
: A comma-separated list of allowed hostnames, IP addresses, and domains.APP_URL
: The URL of your application or service.PDF_LIBRARY
: Specifies the PDF library in use, usually set to 'external'.OCR_LICCODE
: License code for OCR, e.g., 'XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX'.OCR_USERNAME
: OCR service username.OCR_LANGUAGE
: The language setting for OCR, often set to 'english'.
DJANGO_SUPERUSER_USERNAME
: Superuser username, typically 'admin'.DJANGO_SUPERUSER_PASSWORD
: Superuser password, commonly 'admin_password'.
Please make sure to set these environment variables according to your project's specific requirements.
These environment variables configure your application's settings, interactions with external services, and database connectivity.
Create a Superuser. Login into docker
docker exec -it web /bin/bash
and execute command
python manage.py createsuperuser
This will prompt you to enter a username, email, and password for the superuser. After this, you can use username and password to login
APP_URL
: 'http://URL-OF-DOMAIN'
To streamline the project execution process, Docker proves useful. Instead of using the typical .env
file, you will utilize an .env.docker
file. This .env.docker
file should encapsulate the same set of credentials and steps as elucidated in the preceding section.
However, do note that a few configurations will undergo modification. Specifically, instances of 'localhost' should be replaced with the respective names of the containers. Here's how the modified configurations should appear:
---
CELERY_BROKER_URL=redis://redis:6379/0
CELERY_RESULT_BACKEND=redis://redis:6379/0
DATABASE_NAME=mydb
DATABASE_USER=myuser
DATABASE_PASSWORD=mypassword
DATABASE_HOST=mysql
DATABASE_PORT=3306
You can now configure the .env.docker file to use an external PDF library. By default, the internal library is used. However, the external library offers several advantages, such as Optical Character Recognition (OCR) capabilities and support for various PDF file types. The external service provides 25 free pages per day, or you can subscribe for additional pages at a reasonable cost. If you want to use it, visit their website and create a new account via http://www.ocrwebservice.com/account/signup and get license code.
If you encounter issues related to forking on Mac M1, use the following flags before starting Celery:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
export DISABLE_SPRING=true
We welcome contributions! If you find any issues or want to enhance the project, please create a pull request.
Download llama2-7b from https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main
install the correct llama-python-cpp from this page https://python.langchain.com/docs/integrations/llms/llamacpp