docker compose setup #162

LawyZheng · 2024-04-07T03:01:32Z

build a docker image
support docker compose setup

.github/workflows/ci.yml

wintonzheng

LGTM overall. Thanks for building this!

wintonzheng · 2024-04-07T08:43:47Z

Dockerfile

+COPY --from=requirements-stage /tmp/requirements.txt /app/requirements.txt
+RUN pip install --no-cache-dir --upgrade -r requirements.txt
+RUN pip install --no-cache-dir streamlit
+# Use following command, if you have network problem


what are these comments? can we delete these comments?

wintonzheng · 2024-04-07T09:01:49Z

docker-compose.yml

+      - BROWSER_TYPE=chromium-headful
+      - ENABLE_OPENAI=true
+      - OPENAI_API_KEY=<your_openai_key>
+      # If you want to use other LLM provide, like azure or anthropic


Suggested change

# If you want to use other LLM provide, like azure or anthropic

# If you want to use other LLM provider, like azure and anthropic:

wintonzheng · 2024-04-07T09:02:26Z

docker-compose.yml

+      - ./videos:/data/videos
+      - ./har:/data/har
+      - ./.streamlit:/app/.streamlit
+    environment:


can we point it to the .env file?

If .env pointed, users need to run the docker-compose together with .env.
In this way, rookie users could run the single docker-compose.yml file.
And advanced users could point it to the .env by themselves.

Besides, if we change the settings in env, we are supposed to restart/recreate the container.
If we have an isolated env setting file, I think it's easy to forget to restart/recreate.
I think load from .env could be more useful if we want to share the same env between multiple containers, instead of running a single container

ykeremy · 2024-04-08T01:30:23Z

docker-compose.yml

+    depends_on:
+      - postgres


This wouldn't work well because this way it just waits for the container to be up. Instead, we should be waiting for postgres to be ready.

This eventually works out but until skyvern is up, the container will restart multiple times since it'll die without a healthy DB.

Great, I have noticed it already.
But I don't have any idea how to make skyvern check the DB health hook in docker-compose.
Any good idea?

So I just let skyvern restart on failure. haha

Yeah it works but if someone checks out the logs or runs without -d it could get confusing 😄

Can you check this out and see if it works? I did something similar before, if this doesn't work we can hop on a call later?

https://stackoverflow.com/a/55835081

README.md

Co-authored-by: Kerem Yilmaz <kerem@skyvern.com>

ykeremy · 2024-04-08T01:32:19Z

README.md

@@ -101,6 +101,16 @@ Note: Our setup script does these two for you, but they are here for reference.
    ```
 1. Navigate to `http://localhost:8501` in your browser to start using the UI

+## Docker Compose setup


We should test the docker, (maybe even ask people in DC to test it out and provide feedback) and when it's all good this should be the primary way to start using Skyvern.

ykeremy · 2024-04-08T05:24:01Z

@LawyZheng the Skyvern container waits the postgres to actually be ready now!! 🎉

Stearmlit can't connect to the server though:

ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: /api/v1/internal/tasks?page=1&page_size=15 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff5e1d8ad0>: Failed to establish a new connection: [Errno 111] Connection refused'))

LMK if you need help debugging.

LawyZheng · 2024-04-08T05:58:39Z

@ykeremy you'd better delete or move away the old secrets.toml in folder .streamlit.
If you are trying to use the old DB, you need to verify some things:

the connect string in docker-compose is correct.
need change the the host in secrets.toml to skyvern from 127.0.0.1/ or you can map a port 8000 to the local machine in the skyvern container

LawyZheng · 2024-04-08T06:01:28Z

My default docker-compose is prepared for uses who haven't do any setups yet.
And only streamlit port is exposed to the local machine network. postgres and skyvern are both in the docker network without exported.

ykeremy · 2024-04-08T06:15:25Z

Removing the old secrets.toml was the solution 🎉

There will be users who already set something up in their locals that would like to start using the dockerized solution. Let's think about how we should let people know about stuff like this. I'll add this to our discussions topics in tomorrow's meeting.

andr0s · 2024-04-09T17:00:04Z

btw this seems to be working, i've just tested it :) Thanks

LawyZheng added 6 commits April 5, 2024 17:06

add docker compose setup

a81fb77

use original repo

8922790

set default openai to true in docker compose

8c4aa93

add gitchub ci for docker image build

4ab03ab

change build image ci

956a4b2

fix github ci error

d06c7b9

wintonzheng reviewed Apr 7, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

wintonzheng reviewed Apr 7, 2024

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

LawyZheng added 11 commits April 7, 2024 12:28

fix github ci error

bfb3290

Update ci.yml

2fa001e

Update ci.yml

365f337

seperate building workflow

ae13502

Update build-docker-image.yml

32f9895

Update build-docker-image.yml

dec5d43

Update build-docker-image.yml

d2a2a96

make ci workflow callable

bee15ae

call ci workflow in build image

b11bde3

change github workflow to release

1dc3681

update docker compose and README

4643773

LawyZheng requested review from ykeremy and wintonzheng April 7, 2024 07:52

wintonzheng reviewed Apr 7, 2024

View reviewed changes

deleting useless comment

a6395df

ykeremy reviewed Apr 8, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

Update README.md

71ad0c2

Co-authored-by: Kerem Yilmaz <kerem@skyvern.com>

ykeremy reviewed Apr 8, 2024

View reviewed changes

resolve docker compose depend on issue

77c31d1

LawyZheng merged commit c0b4510 into main Apr 11, 2024
2 checks passed

LawyZheng deleted the feature/support-docker-setup branch April 11, 2024 00:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker compose setup #162

docker compose setup #162

LawyZheng commented Apr 7, 2024

wintonzheng left a comment

wintonzheng Apr 7, 2024

wintonzheng Apr 7, 2024

wintonzheng Apr 7, 2024

LawyZheng Apr 7, 2024

LawyZheng Apr 7, 2024 •

edited

Loading

ykeremy Apr 8, 2024

LawyZheng Apr 8, 2024

LawyZheng Apr 8, 2024

ykeremy Apr 8, 2024

LawyZheng Apr 8, 2024

ykeremy Apr 8, 2024

ykeremy commented Apr 8, 2024

LawyZheng commented Apr 8, 2024

LawyZheng commented Apr 8, 2024

ykeremy commented Apr 8, 2024

andr0s commented Apr 9, 2024

	# If you want to use other LLM provide, like azure or anthropic
	# If you want to use other LLM provider, like azure and anthropic:

docker compose setup #162

docker compose setup #162

Conversation

LawyZheng commented Apr 7, 2024

wintonzheng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LawyZheng Apr 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ykeremy commented Apr 8, 2024

LawyZheng commented Apr 8, 2024

LawyZheng commented Apr 8, 2024

ykeremy commented Apr 8, 2024

andr0s commented Apr 9, 2024

LawyZheng Apr 7, 2024 •

edited

Loading