Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: pcds-3.3.5 and CI cleanup #140

Merged
merged 48 commits into from
Sep 22, 2020
Merged
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
3a484eb
MNT: update deps, update matplotlib, add py-spy
ZLLentz Sep 18, 2020
7c67bbc
CI: comment out the builds that don't even work to save travis cycles
ZLLentz Sep 18, 2020
fc369cf
MNT: unpin matplotlib for now, clearly a deep incompat
ZLLentz Sep 18, 2020
4d10def
MNT: pick up new pcdsutils version
ZLLentz Sep 18, 2020
1a9d556
MNT: try not pinning bluesky
ZLLentz Sep 18, 2020
1c11b1e
MNT: update hutch-python pin
ZLLentz Sep 18, 2020
b0cf71f
MNT: unpin tornado, bluesky fixed the thing a while back
ZLLentz Sep 18, 2020
fd37761
ENH: first pass at debug helper script
ZLLentz Sep 19, 2020
48df515
MNT: allow conda activate
ZLLentz Sep 19, 2020
077746c
MNT: pick yes
ZLLentz Sep 19, 2020
cdf5c82
ENH: add ending message for debugger
ZLLentz Sep 21, 2020
46fc007
ENH: add shorter env creation script
ZLLentz Sep 21, 2020
03f35cc
MNT: execute permissions
ZLLentz Sep 21, 2020
520d44f
ENH: improve output for debug build.
ZLLentz Sep 21, 2020
e4a2afe
MNT: remove py-spy for now, bump various tags
ZLLentz Sep 21, 2020
0ad975e
CI: try to get CI to work for all builds again.
ZLLentz Sep 21, 2020
12d0a3a
ENH: updated pcds to 3.3.5
ZLLentz Sep 21, 2020
84b2082
CI: rework ci config
ZLLentz Sep 21, 2020
7e2c137
CI: some iteration
ZLLentz Sep 21, 2020
9e62e93
CI: try to fully expand to desired test set
ZLLentz Sep 21, 2020
71d8051
CI: add timeouts in, don't want to wait for failed solves
ZLLentz Sep 21, 2020
0b7cfb6
CI: put condarc into the workspace
ZLLentz Sep 21, 2020
9e367a1
MNT: add some more debug help/fixes to debug script
ZLLentz Sep 21, 2020
0432d5b
CI: set up pythonpath for dev repos
ZLLentz Sep 21, 2020
2d52973
CI: conda info -a
ZLLentz Sep 21, 2020
0778824
MNT: cd up to get correct dir picked up by workspace
ZLLentz Sep 22, 2020
8edd28d
MNT: find the extra tests better
ZLLentz Sep 22, 2020
26a4830
CI: cd up to fix workspaces
ZLLentz Sep 22, 2020
6e90578
CI: simplify
ZLLentz Sep 22, 2020
00ff203
CI: check if the python script error happens in shell
ZLLentz Sep 22, 2020
ab4d899
CI: try a thing
ZLLentz Sep 22, 2020
aa255cd
MNT: grab the error output
ZLLentz Sep 22, 2020
3ab3770
MNT: add conda info to script for debug
ZLLentz Sep 22, 2020
84660bc
CI: add which python for debug
ZLLentz Sep 22, 2020
0858bc8
CI: try updating tags in base env
ZLLentz Sep 22, 2020
2e9e385
CI: run update env from scripts dir
ZLLentz Sep 22, 2020
bf297c0
CI: fail faster
ZLLentz Sep 22, 2020
17a5f2e
CI: enable fast-finish
ZLLentz Sep 22, 2020
e8d4ec3
CI: rearrange tests to optimize fast finish
ZLLentz Sep 22, 2020
d61402c
CI: minor cleanup
ZLLentz Sep 22, 2020
ac9403b
CI: more cleanup
ZLLentz Sep 22, 2020
a08a13f
CI: set yes flag, split python update to second line
ZLLentz Sep 22, 2020
7251fed
CI: add some debug info
ZLLentz Sep 22, 2020
f74157b
CI: typo
ZLLentz Sep 22, 2020
43ed14a
MNT: add debug message
ZLLentz Sep 22, 2020
4ea0a00
MNT: fix default arg overwriting input
ZLLentz Sep 22, 2020
b0c4191
CI: debug command
ZLLentz Sep 22, 2020
4fd8df1
CI: try another way
ZLLentz Sep 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
313 changes: 213 additions & 100 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -10,107 +10,220 @@ addons:
- herbstluftwm
- libxkbcommon-x11-0

matrix:
jobs:
fast_finish: true
include:
- name: Latest Env
env:
- VARIANT=yaml
- PY_VER=3.6
- name: py3.6 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.6
- name: py3.6 Master Branch
env:
- VARIANT=dev
- PY_VER=3.6
- name: py3.7 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.7
- name: py3.7 Master Branch
env:
- VARIANT=dev
- PY_VER=3.7
- name: py3.8 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.8
- name: py3.8 Master Branch
env:
- VARIANT=dev
- PY_VER=3.8

allow_failures:
- name: py3.6 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.6
- name: py3.6 Master Branch
env:
- VARIANT=dev
- PY_VER=3.6
- name: py3.7 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.7
- name: py3.7 Master Branch
env:
- VARIANT=dev
- PY_VER=3.7
- name: py3.8 Newest Tags
env:
- VARIANT=tag
- PY_VER=3.8
- name: py3.8 Master Branch
env:
- VARIANT=dev
- PY_VER=3.8

install:
- wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
- bash miniconda.sh -b -p $HOME/miniconda
- source "$HOME/miniconda/etc/profile.d/conda.sh"
- conda activate
- hash -r
# Use our condarc
- cp condarc ~/.condarc
- conda config --set always_yes yes --set changeps1 no
- conda install conda-build anaconda-client packaging
# Reboot conda after updating conda to avoid subtle path bugs
- conda deactivate
- conda activate
# Useful for debugging any issues with conda
- conda info -a
- |
if [ "$VARIANT" = "yaml" ]; then
# Create the environment from yaml
conda env create -q -n pcds-test -f envs/pcds/env.yaml
else
if [ "$VARIANT" = "dev" ]; then
# Add the dev channel to be first channel
conda config --add channels pcds-dev
fi
pushd scripts
python update_tags.py pcds
timeout 10m ./create_base_env.sh test pcds "$PY_VER"
popd
fi
- conda activate pcds-test
# Check out the tests for all of our packages
- |
if [ "$VARIANT" = "dev" ]; then
python scripts/test_setup.py pcds
else
python scripts/test_setup.py pcds --tag
fi
# Show us which test scripts exist
- readlink -f */run_tests.py
- name: "Python 3.6 Next Env"
- name: "Python 3.6 Dev Env"
- name: "Python 3.7 Next Env"
- name: "Python 3.8 Next Env"
- name: "Python 3.6 Next Env Unit Tests"
- name: "Python 3.6 Master Branch Unit Tests"
- name: "Python 3.7 Next Env Unit Tests"
- name: "Python 3.8 Next Env Unit Tests"

include:
- stage: "Main Build"
name: "Create Base Env from YAML"
workspaces:
create:
name: conda_base
paths:
- miniconda
install:
- wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
- bash miniconda.sh -b -p miniconda
- source miniconda/etc/profile.d/conda.sh
- conda activate
- hash -r
# Use our condarc
- cp condarc miniconda/.condarc
- conda config --set always_yes yes --set changeps1 no
- conda install conda-build anaconda-client packaging
# Reboot conda after updating conda to avoid subtle path bugs
- conda deactivate
- conda activate
# Useful for debugging any issues with conda
- conda info -a
- which python
script:
# Always create the environment from yaml
- conda env create -q -n pcds-test -f envs/pcds/env.yaml

- stage: "Main Test"
name: "Current Env Unit Tests"
workspaces:
use:
- conda_base
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate pcds-test
- conda info -a
- which python
- cd scripts
- python test_setup.py pcds --tag
before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
script:
- ./run_all_tests.sh pcds

- stage: "Extra Builds"
name: "Python 3.6 Next Env"
workspaces:
use: conda_base
create:
name: py36
paths:
- miniconda
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate
- python scripts/update_tags.py pcds --debug
- conda info -a
- which python
script:
- pushd scripts
- timeout 10m ./update_env.sh pcds-test pcds 3.6 && popd
- pwd

- stage: "Extra Builds"
name: "Python 3.6 Dev Env"
workspaces:
use: conda_base
create:
name: py36-dev
paths:
- miniconda
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate
- conda config --add channels pcds-dev
- python scripts/update_tags.py pcds --debug
- conda info -a
- which python
script:
- pushd scripts
- timeout 10m ./update_env.sh pcds-test pcds 3.6 && popd
- pwd

- stage: "Extra Builds"
name: "Python 3.7 Next Env"
workspaces:
use: conda_base
create:
name: py37
paths:
- miniconda
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate
- python scripts/update_tags.py pcds --debug
- conda info -a
- which python
script:
- pushd scripts
- timeout 10m ./update_env.sh pcds-test pcds 3.7 && popd
- pwd

- stage: "Extra Builds"
name: "Python 3.8 Next Env"
workspaces:
use: conda_base
create:
name: py38
paths:
- miniconda
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate
- python scripts/update_tags.py pcds --debug
- conda info -a
- which python
script:
- pushd scripts
- timeout 10m ./update_env.sh pcds-test pcds 3.8 && popd
- pwd

- stage: "Extra Tests"
name: "Python 3.6 Next Env Unit Tests"
workspaces:
use:
- py36
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate pcds-test
- conda info -a
- which python
- cd scripts
- python test_setup.py pcds --tag
before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
script:
- ./run_all_tests.sh pcds

- stage: "Extra Tests"
name: "Python 3.6 Master Branch Unit Tests"
workspaces:
use:
- py36-dev
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate pcds-test
- conda info -a
- which python
- cd scripts
- python test_setup.py pcds
- |
for repo in *
do
export PYTHONPATH="$PYTHONPATH:$(readlink -f $repo)"
done
before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
script:
- ./run_all_tests.sh pcds

before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
- stage: "Extra Tests"
name: "Python 3.7 Next Env Unit Tests"
workspaces:
use:
- py37
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate pcds-test
- conda info -a
- which python
- cd scripts
- python test_setup.py pcds --tag
before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
script:
- ./run_all_tests.sh pcds

script:
- scripts/run_all_tests.sh pcds
- stage: "Extra Tests"
name: "Python 3.8 Next Env Unit Tests"
workspaces:
use:
- py38
install:
- source miniconda/etc/profile.d/conda.sh
- conda activate pcds-test
- conda info -a
- which python
- cd scripts
- python test_setup.py pcds --tag
before_script:
# Run windows manager
- "herbstluftwm &"
- sleep 1
script:
- ./run_all_tests.sh pcds
25 changes: 12 additions & 13 deletions envs/pcds/conda-packages.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
anaconda-client
bluesky=1.6.1
bluesky>=1.6.6
bokeh>=1.0.1
caproto>=0.6.0
coloredlogs
@@ -14,37 +14,37 @@ elog>=0.1.2
epicscorelibs=7.0.3.99.1.0
flake8
h5py
happi>=1.7.1
happi>=1.7.2
holoviews>=1.10.9
hutch-python>=1.4.0
hutch-python>=1.5.0
hxrsnd>=0.1.7
ipython>=7.1
jupyter
lightpath>=0.6.5
line_profiler
lucid>=0.7.0
lucid>=0.7.1
lxml
mysqlclient
numpy>=1.14
opencv
ophyd>=1.5.2
ophyd>=1.5.3
pandas
panel
papermill
pcaspy
pcdsdaq>=2.2.6
pcdsdevices>=2.10.0
pcdsutils>=0.3.0
pcdswidgets>=0.4.1
pcdsdaq>=2.2.7
pcdsdevices>=2.11.0
pcdsutils>=0.3.1
pcdswidgets>=0.4.2
periodictable
pmgr>=1.1.2
pre-commit
psdm_qs_cli>=0.3.1
pswalker>=1.0.5
pyaudio
pyca>=3.1.1
pydm>=1.10.3
pyepics>=3.4.2
pydm>=1.10.4
pyepics>=3.4.3
pyfiglet
pymongo
pytables
@@ -53,7 +53,7 @@ pytest-benchmark
pytest-qt
pytest-timeout
python-graphviz
pytmc>=2.7.1
pytmc>=2.7.5
qdarkstyle
scikit-image
scikit-learn
@@ -63,7 +63,6 @@ simplejson
sphinx
sphinx_rtd_theme
timechart>=1.2.3
tornado<5.0.0
transfocate>=0.3.3
typhos>=1.1.1
versioneer
Loading