Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][ENH] add Dockerfile for python version og GLM single #46

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
#
# vscode files
.vscode/

# test related
tests/outputs/

# data folder
# data folders
data/

# example output folders
Expand All @@ -13,6 +15,9 @@ figures/
report.html
##*.png

# pyenv
Pipfile

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down Expand Up @@ -54,6 +59,7 @@ pip-delete-this-directory.txt
htmlcov/
.tox/
.coverage
coverage_html
.coverage.*
.cache
nosetests.xml
Expand Down Expand Up @@ -153,4 +159,3 @@ codegen/

# Octave session info
octave-workspace

7 changes: 7 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0
hooks:
- id: check-yaml
56 changes: 56 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# CONTRIBUTING

Information for anyone who would like to contribute to this repository.

## Repository map

```bash
├── .git
├── .github
│ └── workflows # Github continuous integration set up
├── examples
│ ├── data
│ ├── example1outputs
│ ├── example2outputs
├── glmsingle # Python implementation
│ ├── cod
│ ├── design
│ ├── gmm
│ ├── hrf
│ ├── ols
│ ├── ssq
│ └── utils
├── matlab # Matlab implementation
│ ├── examples
│ ├── fracridge
│ └── utilities
└── tests # Python and Matlab tests
└── data

```

## Generic

### Makefile

### pre-commit

## Matlab

### Style guide

### Tests

#### Demos

### Continuous integration

## Python

### Style guide

### Tests

#### Demos

### Continuous integration
127 changes: 127 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Generated by: Neurodocker version 0.7.0+0.gdc97516.dirty
# Latest release: Neurodocker version 0.8.0
# Timestamp: 2022/04/11 12:55:39 UTC
#
# Thank you for using Neurodocker. If you discover any issues
# or ways to improve this software, please submit an issue or
# pull request on our GitHub repository:
#
# https://github.com/ReproNim/neurodocker
Comment on lines +1 to +9
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is automatically generated via the Makefile


FROM python:3.9.12-slim-buster

USER root

ARG DEBIAN_FRONTEND="noninteractive"

ENV LANG="en_US.UTF-8" \
LC_ALL="en_US.UTF-8" \
ND_ENTRYPOINT="/neurodocker/startup.sh"
RUN export ND_ENTRYPOINT="/neurodocker/startup.sh" \
&& apt-get update -qq \
&& apt-get install -y -q --no-install-recommends \
apt-utils \
bzip2 \
ca-certificates \
curl \
locales \
unzip \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen \
&& dpkg-reconfigure --frontend=noninteractive locales \
&& update-locale LANG="en_US.UTF-8" \
&& chmod 777 /opt && chmod a+s /opt \
&& mkdir -p /neurodocker \
&& if [ ! -f "$ND_ENTRYPOINT" ]; then \
echo '#!/usr/bin/env bash' >> "$ND_ENTRYPOINT" \
&& echo 'set -e' >> "$ND_ENTRYPOINT" \
&& echo 'export USER="${USER:=`whoami`}"' >> "$ND_ENTRYPOINT" \
&& echo 'if [ -n "$1" ]; then "$@"; else /usr/bin/env bash; fi' >> "$ND_ENTRYPOINT"; \
fi \
&& chmod -R 777 /neurodocker && chmod a+s /neurodocker

ENTRYPOINT ["/neurodocker/startup.sh"]

RUN apt-get update -qq \
&& apt-get install -y -q --no-install-recommends \
build-essential \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

COPY ["glmsingle", "setup.py", "MANIFEST.in", "README.md", "requirements.txt", "/glmsingle/"]

WORKDIR /glmsingle

RUN pip install .

RUN pip install jupyterlab

EXPOSE 8888

RUN test "$(getent passwd neuro)" || useradd --no-user-group --create-home --shell /bin/bash neuro
USER neuro

WORKDIR /home/neuro

COPY ["examples", "/home/neuro/examples"]

RUN echo '{ \
\n "pkg_manager": "apt", \
\n "instructions": [ \
\n [ \
\n "base", \
\n "python:3.9.12-slim-buster" \
\n ], \
\n [ \
\n "install", \
\n [ \
\n "build-essential" \
\n ] \
\n ], \
\n [ \
\n "copy", \
\n [ \
\n "glmsingle", \
\n "setup.py", \
\n "MANIFEST.in", \
\n "README.md", \
\n "requirements.txt", \
\n "/glmsingle/" \
\n ] \
\n ], \
\n [ \
\n "workdir", \
\n "/glmsingle" \
\n ], \
\n [ \
\n "run", \
\n "pip install ." \
\n ], \
\n [ \
\n "run", \
\n "pip install jupyterlab" \
\n ], \
\n [ \
\n "expose", \
\n [ \
\n "8888" \
\n ] \
\n ], \
\n [ \
\n "user", \
\n "neuro" \
\n ], \
\n [ \
\n "workdir", \
\n "/home/neuro" \
\n ], \
\n [ \
\n "copy", \
\n [ \
\n "examples", \
\n "/home/neuro/examples" \
\n ] \
\n ] \
\n ] \
\n}' > /neurodocker/neurodocker_specs.json
160 changes: 160 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
.DEFAULT_GOAL := help

BROWSER := python -c "$$BROWSER_PYSCRIPT"

# TODO make more general to use the local matlab version
MATLAB = /usr/local/MATLAB/R2017a/bin/matlab
MATLAB_ARG = -nodisplay -nosplash -nodesktop

define BROWSER_PYSCRIPT
import os, webbrowser, sys

from urllib.request import pathname2url

webbrowser.open("file://" + pathname2url(os.path.abspath(sys.argv[1])))
endef
export BROWSER_PYSCRIPT

# determines what "make help" will show
define PRINT_HELP_PYSCRIPT
import re, sys

for line in sys.stdin:
match = re.match(r'^([a-zA-Z_-]+):.*?## (.*)$$', line)
if match:
target, help = match.groups()
print("%-20s %s" % (target, help))
endef
export PRINT_HELP_PYSCRIPT

################################################################################
# GENERIC
.PHONY: help clean clean-test lint install_dev

help:
@python -c "$$PRINT_HELP_PYSCRIPT" < $(MAKEFILE_LIST)
clean: clean-build clean-pyc clean-test ## remove all build, test, coverage artifacts

clean-test: ## remove test and coverage artifacts
rm -rf .tox/
rm -rf .coverage
rm -rf htmlcov/
rm -rf .pytest_cache
rm -rf tests/data
rm -rf test/outputs

install_dev: ## install for both matlab and python developpers
pip install -e .
pip install -r requirements_dev.txt

lint: lint/black lint/flake8 lint/miss_hit ## check style

test: test-matlab test-python

tests/data/nsdcoreexampledataset.mat:
mkdir tests/data
curl -fsSL --retry 5 -o "tests/data/nsdcoreexampledataset.mat" https://osf.io/k89b2/download

################################################################################

################################################################################
# MATLAB

.PHONY: lint/miss_hit

lint/miss_hit: ## lint and checks matlab code
mh_style --fix tests && mh_metric --ci tests && mh_lint tests

test-matlab: tests/data/nsdcoreexampledataset.mat
$(MATLAB) $(MATLAB_ARG) -r "run_tests; exit()"

coverage-matlab: test-matlab
$(BROWSER) coverage_html/index.html

################################################################################

################################################################################
# PYTHON

.PHONY: clean-build clean-pyc coverage-python install lint/flake8 lint/black

clean-build: ## remove build artifacts
rm -fr build/
rm -fr dist/
rm -fr .eggs/
find . -name '*.egg-info' -exec rm -fr {} +
find . -name '*.egg' -exec rm -f {} +

clean-pyc: ## remove Python file artifacts
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -fr {} +

lint/flake8: ## check style with flake8
flake8 tests
lint/black: ## check style with black
black tests

test-python: tests/data/nsdcoreexampledataset.mat ## run tests quickly with the default Python
pytest

test-notebooks:
pytest --nbmake --nbmake-timeout=3000 "./examples"
test-all: ## run tests on every Python version with tox
tox

coverage-python: ## check code coverage quickly with the default Python
coverage run --source glmsingle -m pytest
coverage report -m
coverage html
$(BROWSER) htmlcov/index.html

install: clean ## install the package to the active Python's site-packages
python setup.py install

################################################################################

################################################################################
# DOCKER

.PHONY: Dockerfile glmsingle.def
Dockerfile:
docker run --rm repronim/neurodocker:0.7.0 generate docker \
--base python:3.9.12-slim-buster \
--pkg-manager apt \
--install "build-essential" \
--copy glmsingle setup.py MANIFEST.in README.md requirements.txt /glmsingle/ \
--workdir /glmsingle \
--run "pip install ." \
--run "pip install jupyterlab" \
--expose 8888 \
--user neuro \
--workdir /home/neuro \
--copy examples /home/neuro/examples > Dockerfile
Comment on lines +123 to +134
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the neurodocker command to build the Dockerfile for GLMsingle


docker_build: Dockerfile
docker build . -t glmsingle:latest

docker_run: docker_build
mkdir -p $$PWD/tmp
docker run -it --rm \
--publish 8888:8888 \
--volume $$PWD/tmp:/home/neuro/tmp \
glmsingle:latest \
jupyter-lab --no-browser --ip 0.0.0.0

glmsingle.def:
docker run --rm repronim/neurodocker:0.7.0 generate singularity \
--base python:3.9.12-slim-buster \
--pkg-manager apt \
--install "build-essential" \
--copy glmsingle setup.py MANIFEST.in README.md requirements.txt /home/glmsingle/ \
--workdir /glmsingle \
--run "pip install ." \
--user neuro \
--workdir /home/neuro \
--copy examples /home/neuro/examples > glmsingle.def

singularity_build: glmsingle.def
sudo singularity build glmsingle.sif glmsingle.def
Loading