Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker #34

Merged
merged 5 commits into from
Oct 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"recommendations": [
"svelte.svelte-vscode",
"ms-python.vscode-pylance",
"ms-python.python",
"charliermarsh.ruff"
]
}
14 changes: 12 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
{
"python.defaultInterpreterPath": "backend/.venv/bin/python",
"autoDocstring.docstringFormat": "google",
"editor.wordWrapColumn": 100,
"eslint.validate": ["javascript", "javascriptreact", "svelte"],
"python.testing.pytestEnabled": true,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"python.analysis.typeCheckingMode": "basic",
"editor.formatOnSave": true,
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter"
"editor.defaultFormatter": "charliermarsh.ruff"
},
"python.formatting.provider": "none"
"[json]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
}
}
98 changes: 36 additions & 62 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,55 @@

By the time you finish this guide, you will be able to run the JS frontend [Svelte](https://kit.svelte.dev/) and the Python backend [FastAPI](https://fastapi.tiangolo.com/).

> **Note**
> We use bash for everything and **NOT** Powershell
Note that we use `docker-compose` as defined in the `docker-compose.yml`.

## Installation

### Backend
> **Important**
> You must have [Docker Desktop](https://www.docker.com/products/docker-desktop/) GUI installed and the `docker-compose` bash command.

> **Warning**
You can run everything by doing

```bash
make
```

and navigate to [http://0.0.0.0:5173](http://0.0.0.0:5173) webserver.

That's it. This will spin up a docker container with the backend, database, and frontend servers running.

To turn it off do

```bash
make down
```
Check out the [`Makefile`](./Makefile) for more shortcuts like this.

If you want intellisense/autocomplete in VSCode, continue to the rest of the installation. Otherwise you are done. Totally optional, but I'd do it for autocomplete.

### Backend

> **Important**
> Before you do anything, make sure you have [Python](https://www.python.org/downloads/) installed.

Then install the [Poetry Python Package Manager](https://python-poetry.org/) by doing

```bash
cd backend
pip3 install poetry
```

Then install the packages listed under [`pyproject.toml`](./backend/pyproject.toml) by doing

```bash
poetry config virtualenvs.in-project true # required for the .venv to get created
poetry install
```

**🥳 You're done with the backend installation!**

### Frontend

> **Warning**
> **Important**
> Before you do anything, make sure you first have [Node](https://nodejs.org/en/download) installed.

If you haven't already, also install [Yarn](https://classic.yarnpkg.com/lang/en/docs/install/#mac-stable) globally as the package manager for node by doing
Expand All @@ -47,36 +68,14 @@ yarn install

**🥳 You're done with the frontend installation!**

## Running the Development Servers

It's very simple! You need two terminals open:

On the first terminal, run the backend

```bash
cd backend
poetry run dev
```

which will run the [ `server.py` ](./backend/src/server.py) HTTP server on localhost port 8000.

Then, in the other terminal, run the frontend development server

```bash
cd frontend
yarn dev --open
```

which should open up your browser to the user interface at [http://localhost:5173/](http://localhost:5173/).

**🥳 You now should be up an running!**

In the future, from the root directory you can use make commands to quickly run things instead of having to `cd` to different places and run different commands. Check out [`Makefile`](./Makefile).

But don't use it unless you understand what it's running.

## Navigating the codebase

> **Warning**
> Ignore the build script in the video, they are outdated
> We use docker now through `make`
> Additionally the host has changed from `localhost` to `0.0.0.0`

I'll explain top-down how to understand the structure of this repository. First of all, you want to mainly pay attention to the [`frontend`](./frontend/) then the [`backend`](./backend/) code. I'll explain:

### Frontend
Expand Down Expand Up @@ -114,38 +113,13 @@ Check out the FastAPI docs for more info.

Once we created the HTTP endpoints in the backend, we can call
```bash
cd frontend
yarn openapi
make api
```

which generates frontend bindings/functions we can use in the frontend. See the [`+page.svelte`](./frontend/src/routes/page.svelte) for an example usage with the `Backend` object (generated from the command above). You'll see in the network tab that it just makes a `fetch` call to the backend.

## FAQ

### How to add JavaScript packages to the frontend?

```bash
cd frontend
yarn add npm_package_name
```

for example if I wanted to install [D3](https://www.npmjs.com/package/d3):

```bash
cd frontend
yarn add d3
```
### Database

### How to add Python packages to the backend?
We use a PostgreSQL database, but this can be swapped out later by changing the [`db.py`](./backend/src/db.py).

```bash
cd backend
poetry add pypi_package_name
```

for example if I wanted to install [PyTorch](https://pypi.org/project/torch/):

```bash
cd backend
poetry add torch
```
The starting scheme is here in [`init.sql`](./backend/init.sql) for the initial tables to be created.
33 changes: 24 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
install-front:
cd frontend; yarn install
all: up

install-back:
cd backend; poetry install
up:
docker-compose up -d

front:
cd frontend; yarn dev --open
down:
docker-compose down

back:
cd backend; poetry run dev
restart:
make down
make up

api: # the backend server needs to be running too
hard-restart:
make down
docker-compose up --build -d

api: # generates the api for the frontend to call the backend
cd frontend; yarn openapi

install:
pip3 intall poetry
poetry config virtualenvs.in-project true
@echo "Install Docker Desktop https://www.docker.com/products/docker-desktop/"
@echo "and make sure `docker-compose` is available as a bash command"

restart-venv:
cd backend; poetry config virtualenvs.in-project true;
cd backend; poetry env list; poetry env remove --all;
cd backend; poetry install
26 changes: 2 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,16 @@

**A user friendly system to discover insights into venom protein data.**

<!-- ## Live Deployments
- [Venome Docs](https://venome.vercel.app) contains the live deployment of [`docs`](./docs/README.md) -->

## Structure

- [`docker-compose.yml`](./docker-compose.yml) contains the Docker container build
- [`frontend`](./frontend/README.md) contains the user interface in Svelte/JS
- [`backend`](./backend/README.md) contains the HTTP server and Database
<!-- - [`docs`](./docs/README.md) contains the documentation website generator (starlight/astro)
- [`wip`](/wip/README.md) contains 🚧work-in-progress🚧 documents -->


## Quick start

For an in depth guide of the installation and running, go to [`DEVELOPMENT.md`](./DEVELOPMENT.md).

If you have everything nicely installed, then you can quickly do:

### Frontend

```bash
cd frontend
yarn install
yarn dev
```

### Backend

```bash
cd backend
poetry install
poetry run dev
```
Go to [`DEVELOPMENT.md`](./DEVELOPMENT.md) to get up and running.

## Resources

Expand Down
1 change: 1 addition & 0 deletions backend/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.venv
3 changes: 2 additions & 1 deletion backend/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
__pycache__/
__pycache__/
.venv
13 changes: 13 additions & 0 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM python:3.11

WORKDIR /app

COPY . /app/

RUN pip install poetry

RUN poetry config virtualenvs.in-project true

RUN poetry install

EXPOSE 8000
13 changes: 5 additions & 8 deletions backend/README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
# Backend

## Development

First install the python libraries
## Add packages

```bash
cd backend
pip3 install -r requirements.txt
poetry add py_package_name
```

Then, run the HTTP server by
Note you'll need to rebuild the docker container after adding a new package

```bash
python3 server.py
```
make hard-restart
```
7 changes: 7 additions & 0 deletions backend/init.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
-- protein entries (basic) has name and id for a start
-- TODO: add more info for the protein entries (like PDB, other protein info, markdown text)
-- NOTE: text is like varchar but without a length limit
CREATE TABLE protein_entries (
id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
name text NOT NULL UNIQUE
);
50 changes: 49 additions & 1 deletion backend/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ python = ">=3.9,<3.13"
numpy = "^1.26.1"
uvicorn = "^0.23.2"
fastapi = "^0.104.0"
psycopg = "^3.1.12"
psycopg-pool = "^3.1.8"


[build-system]
Expand All @@ -19,4 +21,4 @@ build-backend = "poetry.core.masonry.api"


[tool.poetry.scripts]
dev = "src.server:main" # run src/server.py at main()
dev = "src.server:poetry_run_dev"
Loading