Skip to content

Commit

Permalink
more pages added, already deployed
Browse files Browse the repository at this point in the history
  • Loading branch information
AlanZhang2002 committed Apr 30, 2024
1 parent 72ba398 commit 6c5e2f6
Show file tree
Hide file tree
Showing 9 changed files with 427 additions and 28 deletions.
19 changes: 19 additions & 0 deletions docs/docs/about.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# About

GLADOS, the Automated Science Pipeline, started when Dr. Jason Yoder, a CSSE Professor at Rose-Hulman Institute of Technology, found himself building the same interface for many different computational research projects over his years of research. Finding a natural need to make this repeated process easier to set up, the project began in 2021 as a senior capstone project, with continuous iterations done by different senior capstone project teams each academic year.

As of 2024, the project has reached basic usability, allowing for multiple experiments to be ran on the system in parallel, without requiring large changes to the original code.

## FAQ

* Why 'GLADOS'?

GLADOS stands for 'General Learning and Automatic Discovery for Operationalizing Science', which is a mouthful, but has a memorable acronym, taking inspiration from the Portal series.

* What technology does this project run on?

GLADOS uses React and Next.js on the frontend, Python on the backend, and MongoDB for data storage, with Firebase for account management. The production server currently runs on a single node Kubernetes cluster within the Rose-Hulman firewall, though older versions primarily used Docker containers.

* Future goals?

In the future, GLADOS hopes to expand the Kubernetes cluster, wider experiment support, and more options for data aggregation, among other bells and whistles. This is currently being handed off to next year's senior capstone project team.
14 changes: 9 additions & 5 deletions docs/docs/documentation/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,27 @@ Documentation is powered by [MkDocs](https://www.mkdocs.org/) with [MkDocs Mater
## Setup
For developers, a MkDocs setup is included as part of the dev install script. When working with the documentation locally, remember to activate the venv:

```
```sh
source docs/.venv/Scripts/activate
```

Otherwise, mkdocs commands will likely not work.

To exit the venv, while in the venv, run

```
```sh
deactivate
```

!!! note

For any mkdocs commands, you must be in the outermost `docs` folder for it to find your files correctly.

## Updating
Updating documentation is as simple as updating Markdown files, all contained within the `docs` folder in the Monorepo.

To preview your changes, this will build the site locally, with automatic updates as you make changes:
```
```sh
mkdocs serve
```

Expand All @@ -30,14 +34,14 @@ mkdocs serve
With Github Actions, updates to the `main` branch will automatically build and deploy the static site to the `gh-pages` branch.

To deploy manually, run
```
```sh
mkdocs gh-deploy --force
```

This updates the `gh-pages` branch with the static site. Keep in mind, if there are unpublished changes, it will display those changes as well, so beware!
<!-- TODO: maybe put this warning in a warning block -->
To preview what files will be generated and published, this generates the static website files under `docs/site/`:
```
```sh
mkdocs build
```

Expand Down
16 changes: 9 additions & 7 deletions docs/docs/index.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Welcome to GLADOS

## For Users
GLADOS is an Automated Science Pipeline, allowing for computational research experiments to get aggregated results from multiple experiment runs easily. Currently developed by seniors at Rose-Hulman Institute of Technology, the goal is to develop a system that makes it simple to run experiments and gather relevant data, through a low cost, open-source, community-built project.

When you're on the Rose-Hulman internal network, you can access the deployed project at <http://glados-lb.csse.rose-hulman.edu>.

**To get a local copy of the system running on your own computer**: Start by following the [User guide](tutorial/installation.md).
## Usage

**To run an experiment on a running copy of the system**: Follow the [Running an Experiment guide](tutorial/usage.md).
**To run an experiment on a running copy of the system**, [get started here](tutorial/usage.md).

## For Devs
**To get a local copy of the system running on your own computer**, start by following [installation steps](tutorial/installation.md).

To set up a new machine for development, follow the [Dev guide](tutorial/development_installation.md).
## Development

To set up a new machine for development, follow the [developer installation guide](tutorial/developer_installation.md).

To get familiar with the components of the system, check out the [Repository Guide](repository_guide.md).

This wiki also contains docs from the [old repo](https://github.com/GLADOSPipeline/Monorepo/wiki) prefixed with `Old`.
<!-- This wiki also contains docs from the [old repo](https://github.com/GLADOSPipeline/Monorepo/wiki) prefixed with `Old`.
These files are no longer up to date, and are planned to be removed, but are kept for developer reference.
This message will be removed once the files have been removed.
This message will be removed once the files have been removed. -->
6 changes: 4 additions & 2 deletions docs/docs/repository_guide.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# This page is currently being migrated, and consists of old information

# Repository Guide

!!! warning

This document is out of date, originating from the 2022-23 team, and is currently being migrated. There are some broken links, and some unfinished content.

In order to keep the entire codebase in sync as it is developed,
we have opted to organize every service into a singular monorepo.

Expand Down
157 changes: 157 additions & 0 deletions docs/docs/tutorial/developer_installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# Developer Installation

This guide contains a short tutorial on getting a local development environment up and running.

!!! warning

As of March 2024, we have shifted towards running the system on Kubernetes, rather than Docker. These steps still work with the current codebase, but as we fully migrate the codebase to our Kubernetes version, these steps will need to be updated accordingly.

!!! note

These steps mainly concern developing on a Windows computer, with Git Bash. Steps on other OS's, or with WSL terminal, will have the same steps, although the specific steps on installation, as well as configuration, may differ.

## Dependencies

If you have already followed the [Installation guide](installation.md), [skip ahead](developer_installation.md#tool-installation). If not, continue reading.

## Required Tools

### Git

Git is a code version management tool that integrates with Github, where our codebase is stored.

If you don't already have it installed, you can download it on the [git website](https://git-scm.com/downloads).

### Docker

Docker is a utility that manages small virtual machine "containers."

To install it, follow the instructions on [Docker's documentation](https://docs.docker.com/get-docker/).

You might encounter a permissions error on Windows
mentioning not being part of the docker-users group.
Directions to fix that can be found [here](https://icij.gitbook.io/datashare/faq-errors/you-are-not-allowed-to-use-docker-you-must-be-in-the-docker-users-group-.-what-should-i-do).

#### WSL Error

To get Docker to work properly, you will need to install WSL.
This may fail for Windows. In this case, check that virtualization is enabled in the BIOS.

## Retrieve the Code

In a location of your choice in a drive with at least 1 GB of free space,
use Git to clone the repo.

Your command will probably look like this:

```bash
git clone --recurse-submodules https://github.com/AutomatingSciencePipeline/Monorepo.git
```

Or this if you have [ssh keys](https://docs.github.com/en/authentication/connecting-to-github-with-ssh) set up (Recommended):

```bash
git clone --recurse-submodules git@github.com:AutomatingSciencePipeline/Monorepo.git
```

The `--recurse-submodules` can be omitted if you don't want to download a copy of the docs as well.

If cloning isn't working for some reason,
you can download a zip from [here](https://github.com/AutomatingSciencePipeline/Monorepo/archive/refs/heads/main.zip) instead.
Note that, if you download a zip, Windows or antivirus programs may block you from running the code,
in which case you will have to navigate various security dialogs to unblock it.
It may also not have the correct line endings (CRLF vs LF) for your system.

## Get the .env file(s)

The system keeps track of private keys (strings that act as passwords to various services) in a `.env` file.
For security reasons, these cannot be publicly distributed with the rest of the code.

You can get a copy of our `.env` file from our private Discord in the [#env-files channel](https://discord.com/channels/1017208818989539368/1042935101601873970). Note that you must already be a member of the Discord to open this link; contact us if you need access.

The file should be named exactly `.env` and placed in the root folder of the repository.

#### Alternative: Set up your own .env file

You can create your own `.env` file, if you want to set up your own GLADOS system. There is an `.example.env` file in the repository that has sample fields to fill in. During this process, you will need to set up a Firebase project for account management. Note that the frontend code has the Firebase public key in it, which does not need to be secure, but will need to be replaced when hosting your own GLADOS system.

## Tool Installation

We include a `dev-install.sh` script that assists in the process of installing everything needed to develop on the system. This setup process includes:

* Installing Docker
* Installing Python 3.8 via pyenv
* Creating pipenvs for both the backend and documentation packages, containing project dependencies
* Installing Node.JS via npm, with project dependencies

Of course, it is recommended to look through the scripts themselves, before running it, for a sanity check.

To run the script, run the following command with a Git Bash terminal in the root folder of the repository:

```sh
bash dev-install.sh
```

This process may require you to close and reopen the terminal, uninstall programs you may have, or restart your computer. You may need to run the script multiple times, as a result.

## IDE Setup

We strongly suggest using Visual Studio Code (VSCode) as your IDE.
Numerous configuration files and utilities, such as suggested extensions, are already included with the repo.

It can be downloaded [here](https://code.visualstudio.com/download).

After installing VSCode, you should install the recommended extensions after opening the repository in VSCode. A prompt may appear after opening the project, but if not, you can also go to the Extensions view, filter by '@recommended', and install the extensions under 'Workspace Recommendations'.

Configure the python interpreter via Ctrl+Shift+P > `Python: Select Interpreter` > `Use Python from 'python.defaultInterpreterPath' setting`

### Check Linter Setup

In order to test the frontend linter, we'll introduce a syntax error and make sure that it reports it as a problem. You can undo this change after you're done.

* Open `/apps/frontend/pages/_app.tsx`
* Add the line `var unused = 'hi';` as shown in the screenshot below and confirm that the same 3 errors are reported when you hover over the symbol `unused`.

![Intentional Error](https://i.imgur.com/6at100I.png)

If the linter doesn't detect 3 errors, make sure the recommended extensions are installed, and check the VSCode "Output" View, select ESLint from the dropdown, and check for any errors.

You may run into CI issues, with build failures, without a working linter.

You are now fully set up to develop on the system!

## Quick Launching with Docker

You can quickly build and launch all service containers via VSCode Tasks.

Ctrl+Shift+P > `Tasks: Run Task` > `DEVELOPMENT Docker Compose Up (Attached)`

If you don't want to use VSCode tasks to start the commands, look at the `.vscode/tasks.json` file to see what commands are run behind the scenes.

This supports enable hot reloading for the backend and frontend.
Once the docker containers are built (this will take a while for the first run) you should start seeing log messages from the containers.

The Docker Desktop application will tell you what port the frontend is accessible from, but it should match the port specified in your `.env` file.

![Docker Desktop screenshot](https://i.imgur.com/W8zJPks.png)

Click on the link shown in the screenshot and it should open a page to the frontend in your browser.
Once you try to connect, the frontend will start to build for hot reloading.

The frontend's hot reloading is really slow at first but gets better after its first builds are done. It may take up to **a few minutes** to turn on for the first time.
While it is building, your browser will be sitting on a blank page still trying to connect. This is normal.

After it finishes loading, you should be taken to the homepage.

## Extras

!!! note

These are optional resources that you may consider using, although we have found a simpler setup to work better.

### Remote Development Extensions

The Remote Development extension pack for VS Code allows you to connect your IDE to connect to remote servers, or containers, and develop from them as if they were your local computer.

It can be installed with [this quick install link](vscode:extension/ms-vscode-remote.vscode-remote-extensionpack)
or downloaded via its [marketplace page](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack).
Empty file.
2 changes: 0 additions & 2 deletions docs/docs/tutorial/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ Directions to fix that can be found [here](https://icij.gitbook.io/datashare/faq
To get Docker to work properly, you will need to install WSL.
This may fail for Windows. In this case, check that virtualization is enabled in the BIOS.



## Retrieve the Code

In a location of your choice in a drive with at least 1 GB of free space,
Expand Down
Loading

0 comments on commit 6c5e2f6

Please sign in to comment.