-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
72ba398
commit 6c5e2f6
Showing
9 changed files
with
427 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# About | ||
|
||
GLADOS, the Automated Science Pipeline, started when Dr. Jason Yoder, a CSSE Professor at Rose-Hulman Institute of Technology, found himself building the same interface for many different computational research projects over his years of research. Finding a natural need to make this repeated process easier to set up, the project began in 2021 as a senior capstone project, with continuous iterations done by different senior capstone project teams each academic year. | ||
|
||
As of 2024, the project has reached basic usability, allowing for multiple experiments to be ran on the system in parallel, without requiring large changes to the original code. | ||
|
||
## FAQ | ||
|
||
* Why 'GLADOS'? | ||
|
||
GLADOS stands for 'General Learning and Automatic Discovery for Operationalizing Science', which is a mouthful, but has a memorable acronym, taking inspiration from the Portal series. | ||
|
||
* What technology does this project run on? | ||
|
||
GLADOS uses React and Next.js on the frontend, Python on the backend, and MongoDB for data storage, with Firebase for account management. The production server currently runs on a single node Kubernetes cluster within the Rose-Hulman firewall, though older versions primarily used Docker containers. | ||
|
||
* Future goals? | ||
|
||
In the future, GLADOS hopes to expand the Kubernetes cluster, wider experiment support, and more options for data aggregation, among other bells and whistles. This is currently being handed off to next year's senior capstone project team. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,19 +1,21 @@ | ||
# Welcome to GLADOS | ||
|
||
## For Users | ||
GLADOS is an Automated Science Pipeline, allowing for computational research experiments to get aggregated results from multiple experiment runs easily. Currently developed by seniors at Rose-Hulman Institute of Technology, the goal is to develop a system that makes it simple to run experiments and gather relevant data, through a low cost, open-source, community-built project. | ||
|
||
When you're on the Rose-Hulman internal network, you can access the deployed project at <http://glados-lb.csse.rose-hulman.edu>. | ||
|
||
**To get a local copy of the system running on your own computer**: Start by following the [User guide](tutorial/installation.md). | ||
## Usage | ||
|
||
**To run an experiment on a running copy of the system**: Follow the [Running an Experiment guide](tutorial/usage.md). | ||
**To run an experiment on a running copy of the system**, [get started here](tutorial/usage.md). | ||
|
||
## For Devs | ||
**To get a local copy of the system running on your own computer**, start by following [installation steps](tutorial/installation.md). | ||
|
||
To set up a new machine for development, follow the [Dev guide](tutorial/development_installation.md). | ||
## Development | ||
|
||
To set up a new machine for development, follow the [developer installation guide](tutorial/developer_installation.md). | ||
|
||
To get familiar with the components of the system, check out the [Repository Guide](repository_guide.md). | ||
|
||
This wiki also contains docs from the [old repo](https://github.com/GLADOSPipeline/Monorepo/wiki) prefixed with `Old`. | ||
<!-- This wiki also contains docs from the [old repo](https://github.com/GLADOSPipeline/Monorepo/wiki) prefixed with `Old`. | ||
These files are no longer up to date, and are planned to be removed, but are kept for developer reference. | ||
This message will be removed once the files have been removed. | ||
This message will be removed once the files have been removed. --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
# Developer Installation | ||
|
||
This guide contains a short tutorial on getting a local development environment up and running. | ||
|
||
!!! warning | ||
|
||
As of March 2024, we have shifted towards running the system on Kubernetes, rather than Docker. These steps still work with the current codebase, but as we fully migrate the codebase to our Kubernetes version, these steps will need to be updated accordingly. | ||
|
||
!!! note | ||
|
||
These steps mainly concern developing on a Windows computer, with Git Bash. Steps on other OS's, or with WSL terminal, will have the same steps, although the specific steps on installation, as well as configuration, may differ. | ||
|
||
## Dependencies | ||
|
||
If you have already followed the [Installation guide](installation.md), [skip ahead](developer_installation.md#tool-installation). If not, continue reading. | ||
|
||
## Required Tools | ||
|
||
### Git | ||
|
||
Git is a code version management tool that integrates with Github, where our codebase is stored. | ||
|
||
If you don't already have it installed, you can download it on the [git website](https://git-scm.com/downloads). | ||
|
||
### Docker | ||
|
||
Docker is a utility that manages small virtual machine "containers." | ||
|
||
To install it, follow the instructions on [Docker's documentation](https://docs.docker.com/get-docker/). | ||
|
||
You might encounter a permissions error on Windows | ||
mentioning not being part of the docker-users group. | ||
Directions to fix that can be found [here](https://icij.gitbook.io/datashare/faq-errors/you-are-not-allowed-to-use-docker-you-must-be-in-the-docker-users-group-.-what-should-i-do). | ||
|
||
#### WSL Error | ||
|
||
To get Docker to work properly, you will need to install WSL. | ||
This may fail for Windows. In this case, check that virtualization is enabled in the BIOS. | ||
|
||
## Retrieve the Code | ||
|
||
In a location of your choice in a drive with at least 1 GB of free space, | ||
use Git to clone the repo. | ||
|
||
Your command will probably look like this: | ||
|
||
```bash | ||
git clone --recurse-submodules https://github.com/AutomatingSciencePipeline/Monorepo.git | ||
``` | ||
|
||
Or this if you have [ssh keys](https://docs.github.com/en/authentication/connecting-to-github-with-ssh) set up (Recommended): | ||
|
||
```bash | ||
git clone --recurse-submodules git@github.com:AutomatingSciencePipeline/Monorepo.git | ||
``` | ||
|
||
The `--recurse-submodules` can be omitted if you don't want to download a copy of the docs as well. | ||
|
||
If cloning isn't working for some reason, | ||
you can download a zip from [here](https://github.com/AutomatingSciencePipeline/Monorepo/archive/refs/heads/main.zip) instead. | ||
Note that, if you download a zip, Windows or antivirus programs may block you from running the code, | ||
in which case you will have to navigate various security dialogs to unblock it. | ||
It may also not have the correct line endings (CRLF vs LF) for your system. | ||
|
||
## Get the .env file(s) | ||
|
||
The system keeps track of private keys (strings that act as passwords to various services) in a `.env` file. | ||
For security reasons, these cannot be publicly distributed with the rest of the code. | ||
|
||
You can get a copy of our `.env` file from our private Discord in the [#env-files channel](https://discord.com/channels/1017208818989539368/1042935101601873970). Note that you must already be a member of the Discord to open this link; contact us if you need access. | ||
|
||
The file should be named exactly `.env` and placed in the root folder of the repository. | ||
|
||
#### Alternative: Set up your own .env file | ||
|
||
You can create your own `.env` file, if you want to set up your own GLADOS system. There is an `.example.env` file in the repository that has sample fields to fill in. During this process, you will need to set up a Firebase project for account management. Note that the frontend code has the Firebase public key in it, which does not need to be secure, but will need to be replaced when hosting your own GLADOS system. | ||
|
||
## Tool Installation | ||
|
||
We include a `dev-install.sh` script that assists in the process of installing everything needed to develop on the system. This setup process includes: | ||
|
||
* Installing Docker | ||
* Installing Python 3.8 via pyenv | ||
* Creating pipenvs for both the backend and documentation packages, containing project dependencies | ||
* Installing Node.JS via npm, with project dependencies | ||
|
||
Of course, it is recommended to look through the scripts themselves, before running it, for a sanity check. | ||
|
||
To run the script, run the following command with a Git Bash terminal in the root folder of the repository: | ||
|
||
```sh | ||
bash dev-install.sh | ||
``` | ||
|
||
This process may require you to close and reopen the terminal, uninstall programs you may have, or restart your computer. You may need to run the script multiple times, as a result. | ||
|
||
## IDE Setup | ||
|
||
We strongly suggest using Visual Studio Code (VSCode) as your IDE. | ||
Numerous configuration files and utilities, such as suggested extensions, are already included with the repo. | ||
|
||
It can be downloaded [here](https://code.visualstudio.com/download). | ||
|
||
After installing VSCode, you should install the recommended extensions after opening the repository in VSCode. A prompt may appear after opening the project, but if not, you can also go to the Extensions view, filter by '@recommended', and install the extensions under 'Workspace Recommendations'. | ||
|
||
Configure the python interpreter via Ctrl+Shift+P > `Python: Select Interpreter` > `Use Python from 'python.defaultInterpreterPath' setting` | ||
|
||
### Check Linter Setup | ||
|
||
In order to test the frontend linter, we'll introduce a syntax error and make sure that it reports it as a problem. You can undo this change after you're done. | ||
|
||
* Open `/apps/frontend/pages/_app.tsx` | ||
* Add the line `var unused = 'hi';` as shown in the screenshot below and confirm that the same 3 errors are reported when you hover over the symbol `unused`. | ||
|
||
![Intentional Error](https://i.imgur.com/6at100I.png) | ||
|
||
If the linter doesn't detect 3 errors, make sure the recommended extensions are installed, and check the VSCode "Output" View, select ESLint from the dropdown, and check for any errors. | ||
|
||
You may run into CI issues, with build failures, without a working linter. | ||
|
||
You are now fully set up to develop on the system! | ||
|
||
## Quick Launching with Docker | ||
|
||
You can quickly build and launch all service containers via VSCode Tasks. | ||
|
||
Ctrl+Shift+P > `Tasks: Run Task` > `DEVELOPMENT Docker Compose Up (Attached)` | ||
|
||
If you don't want to use VSCode tasks to start the commands, look at the `.vscode/tasks.json` file to see what commands are run behind the scenes. | ||
|
||
This supports enable hot reloading for the backend and frontend. | ||
Once the docker containers are built (this will take a while for the first run) you should start seeing log messages from the containers. | ||
|
||
The Docker Desktop application will tell you what port the frontend is accessible from, but it should match the port specified in your `.env` file. | ||
|
||
![Docker Desktop screenshot](https://i.imgur.com/W8zJPks.png) | ||
|
||
Click on the link shown in the screenshot and it should open a page to the frontend in your browser. | ||
Once you try to connect, the frontend will start to build for hot reloading. | ||
|
||
The frontend's hot reloading is really slow at first but gets better after its first builds are done. It may take up to **a few minutes** to turn on for the first time. | ||
While it is building, your browser will be sitting on a blank page still trying to connect. This is normal. | ||
|
||
After it finishes loading, you should be taken to the homepage. | ||
|
||
## Extras | ||
|
||
!!! note | ||
|
||
These are optional resources that you may consider using, although we have found a simpler setup to work better. | ||
|
||
### Remote Development Extensions | ||
|
||
The Remote Development extension pack for VS Code allows you to connect your IDE to connect to remote servers, or containers, and develop from them as if they were your local computer. | ||
|
||
It can be installed with [this quick install link](vscode:extension/ms-vscode-remote.vscode-remote-extensionpack) | ||
or downloaded via its [marketplace page](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack). |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.