-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add weave demos Signed-off-by: vsoch <vsoch@users.noreply.github.com>
- Loading branch information
Showing
7 changed files
with
4,424 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
ARG tag=jammy | ||
FROM ubuntu:${tag} | ||
|
||
# docker build -t demos . | ||
# command without flux: maestro run ball_bounce_suite.yaml --autoyes --pgen pgen.py | ||
# workdir: /workflow | ||
|
||
ENV DEBIAN_FRONTEND=noninteractive | ||
RUN apt-get update | ||
RUN apt-get -y install \ | ||
bwa \ | ||
git \ | ||
samtools \ | ||
bcftools \ | ||
curl \ | ||
libbz2-dev \ | ||
liblzma-dev \ | ||
libcurl4-openssl-dev \ | ||
libncurses5-dev \ | ||
python3-pip \ | ||
&& apt-get clean \ | ||
&& apt-get autoremove \ | ||
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* | ||
|
||
# This is needed for the notebook render | ||
RUN git clone https://github.com/LLNL/PyDV /tmp/pydv && \ | ||
cd /tmp/pydv/ && \ | ||
python3 -m pip install . | ||
|
||
# Wrappers to ensure we source the mamba environment! | ||
RUN git clone --depth 1 https://github.com/LLNL/weave-demos /workflow | ||
WORKDIR /workflow/CZ/ball_bounce | ||
COPY ./scripts/setup.sh /setup.sh | ||
COPY ./ball_bounce_suite_flux.yaml ./ball_bounce_suite_flux.yaml | ||
RUN /bin/bash /setup.sh && python3 -m pip install -r requirements.txt && \ | ||
# The workflow expects to find python | ||
ln -s /usr/bin/python3 /usr/bin/python && \ | ||
python3 -m pip install jut && \ | ||
python3 -m pip install typing-extensions --upgrade && \ | ||
python3 -m pip install ipympl && \ | ||
python3 -m pip install pydv | ||
|
||
# Temporary fix for | ||
# https://github.com/LLNL/PyDV/issues/275 | ||
COPY ./scripts/pydvpy.py /usr/local/lib/python3.8/dist-packages/pydv/pydvpy.py | ||
COPY ./scripts/visualization.ipynb ./visualization.ipynb | ||
EXPOSE 8888 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Weave Demos | ||
|
||
This is an example container where you can build (optional) and run | ||
the [weave demos](https://github.com/LLNL/weave-demos), specifically | ||
to simualte bouncing a ball. | ||
|
||
```bash | ||
$ docker build -t demos . | ||
``` | ||
|
||
Then shell inside | ||
|
||
```bash | ||
$ docker run --entrypoint bash -it demos | ||
``` | ||
|
||
## Manual Run | ||
|
||
To run the tutorial as is (without Flux) do: | ||
|
||
```bash | ||
# The "y" says yes to prompts | ||
$ maestro run ball_bounce_suite.yaml --pgen pgen.py -y | ||
``` | ||
|
||
Output will appear in "output" and you can dig into the subdirectory to find logs: | ||
|
||
```bash | ||
$ cat output/ball-bounce_20230315-035713/logs/ball-bounce.log | ||
``` | ||
```console | ||
... | ||
2023-03-15 03:57:22,708 - maestrowf.datastructures.core.executiongraph:_execute_record:584 - INFO - Attempting submission of 'run-ball-bounce_BOX_SIDE_LENGTH.100.GRAVITY.0.5.GROUP_ID.8d0688.RUN_ID.7.X_POS_INITIAL.45.X_VEL_INITIAL.-2.Y_POS_INITIAL.37.Y_VEL_INITIAL.8.Z_POS_INITIAL.19.Z_VEL_INITIAL.-1' (attempt 1 of 1)... | ||
2023-03-15 03:57:22,775 - maestrowf.interfaces.script.localscriptadapter:submit:151 - INFO - Execution returned status OK. | ||
``` | ||
The last line should say it returned OK and shouldn't say FAILED! I found the output directories (length!) a little overwhelming to browse, so I installed and used tree: | ||
|
||
```bash | ||
$ apt-get update && apt-get install -y tree | ||
$ tree output/*/run-ball-bounce/ | ||
``` | ||
```console | ||
└── BOX_SIDE_LENGTH.100.GRAVITY.0.5.GROUP_ID.8d0688.RUN_ID.9.X_POS_INITIAL.45.X_VEL_INITIAL.7.Y_POS_INITIAL.37.Y_VEL_INITIAL.-6.Z_POS_INITIAL.19.Z_VEL_INITIAL.-2 | ||
├── output.dsv | ||
├── run-ball-bounce_BOX_SIDE_LENGTH.100.GRAVITY.0.5.GROUP_ID.8d0688.RUN_ID.9.X_POS_INITIAL.45.X_VEL_INITIAL.7.Y_POS_INITIAL.37.Y_VEL_INITIAL.-6.Z_POS_INITIAL.19.Z_VEL_INITIAL.-2.33.err | ||
├── run-ball-bounce_BOX_SIDE_LENGTH.100.GRAVITY.0.5.GROUP_ID.8d0688.RUN_ID.9.X_POS_INITIAL.45.X_VEL_INITIAL.7.Y_POS_INITIAL.37.Y_VEL_INITIAL.-6.Z_POS_INITIAL.19.Z_VEL_INITIAL.-2.33.out | ||
└── run-ball-bounce_BOX_SIDE_LENGTH.100.GRAVITY.0.5.GROUP_ID.8d0688.RUN_ID.9.X_POS_INITIAL.45.X_VEL_INITIAL.7.Y_POS_INITIAL.37.Y_VEL_INITIAL.-6.Z_POS_INITIAL.19.Z_VEL_INITIAL.-2.sh | ||
``` | ||
It looks like each directory has a script, and output, and an error file. I found the output/error to be empty, but the `output.dsv` is some kind of matric of numbers, and the submit script | ||
(the shell script) is just a single line: | ||
|
||
``` | ||
#!/bin/bash | ||
python /workflow/ball_bounce/./ball_bounce.py output.dsv 45 37 19 -3 3 1 0.5 100 8d0688 2 | ||
``` | ||
|
||
This is probably what they mean when they say running a single simulation. That command runs one simulation. For | ||
fun, I tried this on my own: | ||
|
||
```bash | ||
$ mkdir test | ||
$ cd test | ||
$ python /workflow/ball_bounce/./ball_bounce.py output.dsv 45 37 19 -3 3 1 0.5 100 8d0688 2 | ||
``` | ||
|
||
It generated an output.dsv almost immediately. We could likely adjust the simulation parameters for the calculation. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
description: | ||
name: ball-bounce | ||
description: A workflow that simulates a ball bouncing in a box over several input sets. | ||
|
||
env: | ||
variables: | ||
OUTPUT_PATH: ./output | ||
SIM_SCRIPT_PATH: ./ball_bounce.py | ||
PROCESS_SCRIPT_PATH: ./dsv_to_sina.py | ||
OUTPUT_DS_PATH: ./output.sqlite | ||
LAUNCHER: flux submit | ||
|
||
batch: | ||
type : flux | ||
|
||
study: | ||
- name: run-ball-bounce | ||
description: Run a family of simulations of a ball in a box. | ||
run: | ||
# Note that flux submit before (should be?) $(LAUNCHER) but I didn't see that worked - I think likely it needs a different submit command / setup | ||
cmd: | | ||
$(LAUNCHER) python $(SPECROOT)/$(SIM_SCRIPT_PATH) output.dsv $(X_POS_INITIAL) $(Y_POS_INITIAL) $(Z_POS_INITIAL) $(X_VEL_INITIAL) $(Y_VEL_INITIAL) $(Z_VEL_INITIAL) $(GRAVITY) $(BOX_SIDE_LENGTH) $(GROUP_ID) $(RUN_ID) | ||
- name: ingest-ball-bounce | ||
description: Ingest the outputs from the previous step | ||
run: | ||
cmd: | | ||
$(LAUNCHER) python $(SPECROOT)/$(PROCESS_SCRIPT_PATH) ../run-ball-bounce $(SPECROOT)/$(OUTPUT_DS_PATH) | ||
depends: [run-ball-bounce_*] | ||
|
||
# This notebook has bugs - but when fixed this will work | ||
# - name: plot-ball-bounce | ||
# description: Generate the notebook that shows something interesting! | ||
# run: | ||
# I hard coded the path because I was lazy - not ideal | ||
# cmd: | | ||
# $(LAUNCHER) jupyter nbconvert --to notebook --execute /workflow/ball_bounce/visualization.ipynb --output=/workflow/ball_bounce/render.ipynb --ExecutePreprocessor.timeout=-1 | ||
# depends: [ingest-ball-bounce] | ||
|
Oops, something went wrong.