As IDE, we use Visual Studio Code.
For developing both Python and TypeScript code,
use Docker image janjones/awe-gradient
.
The development Docker image can be built either
-
locally,
cd gradient docker build -t janjones/awe-gradient .
-
by executing GitHub Action Gradient Docker Image which also pushes the image to Docker Hub as
janjones/awe-gradient
.
Remote environments described below usually assume
existence of janjones/awe-gradient
on Docker Hub.
To develop locally, either reproduce your environment
according to gradient/Dockerfile
1,
or develop inside a Docker image as described below.
-
Install Docker Desktop.
-
Download the Docker image (or build it locally as described above).
docker pull janjones/awe-gradient
-
Start the Docker image.
# Go up to repository root. cd .. # (Optional) Remove persistent volume to start fresh. docker volume rm awe # Run the container with persistent storage in a named volume. docker run --rm -it -v awe:/storage -p 22:22 janjones/awe-gradient
-
Now SSH can be used as below. Password is displayed in Docker output.
ssh root@localhost
-
Continue with VS Code SSH steps.
If CPU is enough (e.g., to process data or experiment with a simple model), Gitpod is configured and can be used.
When GPU is needed, Gradient.run cloud can be used. Since it does not support direct SSH connections, a little trick is needed2.
-
Create and start Gradient notebook. In advanced options
- ensure Git repository is not set,
- select the development Docker image (
janjones/awe-gradient:latest
) as a custom container, and - set the command to
/run.sh
.
-
Install huproxy locally and configure SSH as shown below. Replace
<JUPYTER_LAB_ID>
by ID from URL that opens in Gradient when clicking on "Open in Jupyter Lab" button. As SSH password, use token from that URL. Alternatively, look into terminal output in Gradient web environment after the notebook starts, both the token and public URL should appear there.Host 127.0.0.1 User root ProxyCommand <FULL_PATH_TO_huproxyclient.exe> wss://<JUPYTER_LAB_ID>.paperspacegradient.com/proxy/%h/%p
-
Continue with VS Code SSH steps.
-
Connect VS Code via Remote SSH and open directory
/storage/awe/src
. -
If this is the first time, clone the repository, configure Git and initialize VSCode:
git clone https://github.com/jjonescz/awe . ./sh/configure.sh ./gradient/install.sh
Footnotes
-
Note that all Python package versions are frozen in
awe/requirements.txt
. ↩ -
This trick is based on directing SSH over WebSockets which might stop working if Gradient.run infrastructure is changed in a non-trivial way. ↩