add docs

README.md

@@ -19,24 +19,26 @@ A simple GUI app to synchronize recorded audio with video lip movements using th
    cd Lipsync
    ```
 
-3. Install Poetry and the required packages for the backend (assuming you have Python already installed):
+3. Make sure you put the necessary model files into backend/wav2lip-hq/checkpoints as described in the [wav2lip-hq directory](https://github.com/Markfryazino/wav2lip-hq)
+
+4. Install Poetry and the required packages for the backend (assuming you have Python already installed):
    ```bash
    cd backend
    curl -sSL https://install.python-poetry.org | python3 -
    poetry install
    ```
 
-4. Run the backend app:
+5. Run the backend app:
    ```bash
    poetry run app.py
    ```
 
-5. Install the required packages for the frontend (assuming you have npm already installed)
+6. Install the required packages for the frontend (assuming you have npm already installed)
    ```bash
    cd ../frontend
    npm install
    ```
-6. Run the frontend app:
+7. Run the frontend app:
    ```bash
    npm run dev
    ```

backend/docker/Dockerfile.backend

@@ -4,12 +4,15 @@ FROM pytorch/pytorch:1.8.1-cuda11.1-cudnn8-runtime
 # Set the working directory in the container
 WORKDIR /backend
 
+# Install backend dependencies
+RUN pip install poetry
+COPY backend .
+RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi
+
 # Install system dependencies and clean up
 RUN apt-get update && apt-get install -y git wget && \
     rm -rf /var/lib/apt/lists/*
 
-# Clone the Wav2Lip-HQ repository
-RUN git clone https://github.com/Markfryazino/wav2lip-hq.git
 
 # Change to the Wav2Lip-HQ directory
 WORKDIR /backend/wav2lip-hq
@@ -33,12 +36,6 @@ RUN chmod +x download_models.py && \
 # Create directories for input and output videos
 RUN mkdir -p videos results
 
-# Install backend dependencies
-RUN pip3 install poetry
-COPY backend /backend
-WORKDIR /backend
-RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi
-
 # Set the entrypoint to the Flask app
 ENTRYPOINT ["python", "app.py"]
 

docs/api.md

@@ -0,0 +1,103 @@
+# Backend API Documentation
+
+This documentation provides an overview of the Flask API implemented in the Lipsync Backend application. The API allows users to merge audio and video files and retrieve uploaded files.
+
+## Routes
+
+### `/uploads/<filename>`
+
+- **HTTP Method**: GET
+- **Description**: This route allows users to retrieve uploaded files by specifying the filename.
+- **Parameters**:
+  - `filename` (str): The name of the file to retrieve.
+- **Example Request**:
+  ```
+  GET /uploads/example.mp4
+  ```
+- **Example Response**:
+  - The requested file will be sent as a response.
+
+### `/api/merge`
+
+- **HTTP Method**: POST
+- **Description**: This route is used for merging audio and video files.
+- **Parameters**:
+  - `audio` (file): The audio file to be merged.
+  - `video` (file): The video file to be merged.
+- **Example Request**:
+  ```
+  POST /api/merge
+  ```
+  - Request Body:
+    - `audio` (multipart/form-data): The audio file to be merged.
+    - `video` (multipart/form-data): The video file to be merged.
+- **Example Response**:
+  - **Success Response (HTTP 200 OK)**:
+    - Content-Type: application/json
+    - Body:
+      ```json
+      {
+        "videoPath": "/uploads/merged_video.mp4"
+      }
+      ```
+    - Description: If the merge operation is successful, the API returns the path to the merged video file.
+  - **Error Responses**:
+    - HTTP 400 Bad Request:
+      - Content-Type: application/json
+      - Body:
+        ```json
+        {
+          "error": "Missing audio or video file"
+        }
+        ```
+      - Description: Returned when either the audio or video file is missing in the request.
+    - HTTP 400 Bad Request:
+      - Content-Type: application/json
+      - Body:
+        ```json
+        {
+          "error": "No selected file"
+        }
+        ```
+      - Description: Returned when no files are selected in the request.
+    - HTTP 400 Bad Request:
+      - Content-Type: application/json
+      - Body:
+        ```json
+        {
+          "error": "File type not allowed"
+        }
+        ```
+      - Description: Returned when the file type is not allowed for merging.
+    - HTTP 404 Not Found:
+      - Content-Type: application/json
+      - Body:
+        ```json
+        {
+          "error": "Output file not found."
+        }
+        ```
+      - Description: Returned when the output file is not found after the merge operation.
+    - HTTP 500 Internal Server Error:
+      - Content-Type: application/json
+      - Body:
+        ```json
+        {
+          "error": "<error_message>"
+        }
+        ```
+      - Description: Returned when an internal server error occurs during the merge operation.
+
+## `app.py`
+
+This script initializes the Flask application and defines a simple endpoint, used to determine if the backend is working:
+
+- **Endpoint**: `/`
+- **HTTP Method**: GET
+- **Description**: This endpoint returns a JSON response with a welcome message.
+- **Example Response**:
+  ```json
+  {
+    "message": "Lipsync Backend API"
+  }
+  ```

docs/design_requirements.md

@@ -0,0 +1,47 @@
+# Overview
+
+In this technical assessment, you will build a full-stack application that includes a single-page front-end application using React and a backend API using Python and Flask. The application aims to demonstrate your ability to work with various libraries and frameworks.
+
+## Objective
+
+Create an application that allows a user to perform three types of recordings:
+1. A regular video recording used for training the Wav2Lip model.
+2. A video-only recording.
+3. An audio-only recording.
+
+After these recordings are completed, your application should use a Flask backend API to combine the audio-only and video-only recordings into a single video using the Wav2lip-hq library.
+
+**Note:** You can use additional libraries or packages to complete this assessment.
+
+## Requirements
+
+### Frontend:
+- **Setup React App:** Create a new React application using Create React App or your preferred setup.
+- **User Interface:** Develop a simple user interface with three buttons, each corresponding to the three types of recordings mentioned above.
+- **Recording:** Use the React-media-recorder library to handle video and audio recording.
+- **Sending Data:** Once each recording is finished, send the data to the Flask backend for processing. Use POST requests to upload the files.
+- **Receiving Data:** Upon the backend completing processing, receive and display the final combined video on the frontend.
+
+### Backend:
+- **Setup Flask App:** Create a new Flask application with at least one API endpoint to handle the file upload and processing.
+- **File Upload:** Receive the uploaded files from the frontend, and save them in a specific directory.
+- **Processing:** Use the Wav2lip-hq library to combine the audio-only and video-only files into a single video.
+- **Response:** Once the video is generated, send a response back to the frontend to confirm the process completion, including the final video.
+
+### Bonus:
+- Implement a progress bar during video processing.
+- Use Docker for the application deployment.
+
+## Submission
+
+Your submission should include:
+- All source code used to implement the full-stack application.
+- A README file that explains how to run and test your service. Please also include basic API documentation.
+
+## Evaluation Criteria
+
+We'll evaluate your solution based on:
+- **Correctness:** Does the solution accomplish the stated objectives?
+- **Code quality:** Is the code organized, and can it be easily understood?
+- **Testing:** Is the code adequately tested?
+- **Documentation:** Is the service and its API well-documented?