Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add devcontainer integrations #451

Merged
merged 18 commits into from
Mar 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .devcontainer/apt-deps.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
git
less
htop
nano
ssh
42 changes: 42 additions & 0 deletions .devcontainer/conda-forge/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
{
"name": "Miniforge (default-channel=conda-forge)",
"image": "condaforge/miniforge3:latest",

// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
"mounts": [
"source=${localWorkspaceFolder}/../conda,target=/workspaces/conda,type=bind,consistency=cached",
"source=${localWorkspaceFolder}/../mamba,target=/workspaces/mamba,type=bind,consistency=cached"
],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_create.sh",
// Use 'postStartCommand' to run commands after the container is started.
"postStartCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_start.sh",

// Configure tool-specific properties.
"customizations": {
"vscode": {
"settings": {
jaimergp marked this conversation as resolved.
Show resolved Hide resolved
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
},
"extensions": [
jaimergp marked this conversation as resolved.
Show resolved Hide resolved
"charliermarsh.ruff",
"eamodio.gitlens",
"ms-toolsai.jupyter"
]
}
}

// Adjust to connect as non-root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root",

}
42 changes: 42 additions & 0 deletions .devcontainer/defaults/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// For format details, see https://aka.ms/devcontainer.json
{
"name": "Miniconda (default-channel=defaults)",
"image": "continuumio/miniconda3:latest",

// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
"mounts": [
"source=${localWorkspaceFolder}/../conda,target=/workspaces/conda,type=bind,consistency=cached",
"source=${localWorkspaceFolder}/../mamba,target=/workspaces/mamba,type=bind,consistency=cached"
],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_create.sh",
// Use 'postStartCommand' to run commands after the container is started.
"postStartCommand": "bash /workspaces/conda-libmamba-solver/.devcontainer/post_start.sh",

// Configure tool-specific properties.
"customizations": {
"vscode": {
jaimergp marked this conversation as resolved.
Show resolved Hide resolved
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
},
"extensions": [
"charliermarsh.ruff",
"eamodio.gitlens",
"ms-toolsai.jupyter"
]
}
}

// Adjust to connect as non-root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root",

}
41 changes: 41 additions & 0 deletions .devcontainer/post_create.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash

# This script assumes we are running in a Miniconda container where:
# - /opt/conda is the Miniconda or Miniforge installation directory
# - https://github.com/conda/conda is mounted at /workspaces/conda
# - https://github.com/conda/conda-libmamba-solver is mounted at
# /workspaces/conda-libmamba-solver
jaimergp marked this conversation as resolved.
Show resolved Hide resolved
# - https://github.com/mamba-org/mamba is (optionally) mounted at
# /workspaces/mamba

set -euo pipefail

HERE=$(dirname $0)
BASE_CONDA=${BASE_CONDA:-/opt/conda}
SRC_CONDA=${SRC_CONDA:-/workspaces/conda}
SRC_CONDA_LIBMAMBA_SOLVER=${SRC_CONDA_LIBMAMBA_SOLVER:-/workspaces/conda-libmamba-solver}

if which apt-get > /dev/null; then
echo "Installing system dependencies"
apt-get update
DEBIAN_FRONTEND=noninteractive xargs -a "$HERE/apt-deps.txt" apt-get install -y
fi


if [ ! -f "$SRC_CONDA/pyproject.toml" ]; then
echo "https://github.com/conda/conda not found! Please clone or mount to $SRC_CONDA"
exit 1
fi

# Clear history to avoid unneeded conflicts
echo "Clearing base history..."
echo '' > "$BASE_CONDA/conda-meta/history"

echo "Installing dev & test dependencies..."
"$BASE_CONDA/bin/conda" install -n base --yes --quiet \
--file="$SRC_CONDA/tests/requirements.txt" \
--file="$SRC_CONDA/tests/requirements-ci.txt" \
--file="$SRC_CONDA/tests/requirements-Linux.txt" \
--file="$SRC_CONDA/tests/requirements-s3.txt" \
--file="$SRC_CONDA_LIBMAMBA_SOLVER/dev/requirements.txt" \
--file="$SRC_CONDA_LIBMAMBA_SOLVER/tests/requirements.txt"
76 changes: 76 additions & 0 deletions .devcontainer/post_start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/bin/bash

# This script assumes we are running in a Miniconda container where:
# - /opt/conda is the Miniconda or Miniforge installation directory
# - https://github.com/conda/conda is mounted at /workspaces/conda
# - https://github.com/conda/conda-libmamba-solver is mounted at
# /workspaces/conda-libmamba-solver
jaimergp marked this conversation as resolved.
Show resolved Hide resolved
# - https://github.com/mamba-org/mamba is (optionally) mounted at
# /workspaces/mamba

set -euo pipefail

BASE_CONDA=${BASE_CONDA:-/opt/conda}
SRC_CONDA=${SRC_CONDA:-/workspaces/conda}
SRC_CONDA_LIBMAMBA_SOLVER=${SRC_CONDA_LIBMAMBA_SOLVER:-/workspaces/conda-libmamba-solver}
SRC_MAMBA=${SRC_MAMBA:-/workspaces/mamba}

cat >> ~/.bashrc <<EOF
function develop-mamba() (
# Runs in a subshell to avoid polluting the current shell
set -euo pipefail
if ! conda config --show channels | grep -q conda-forge; then
echo "Miniconda not compatible with develop-mamba"
exit 1
fi
if [ ! -f "$SRC_MAMBA/mamba/setup.py" ]; then
echo "Mamba 1.x not found at $SRC_MAMBA"
exit 1
fi
# Install mamba dev dependencies only once:
if [ ! -f ~/.mamba-develop-installed ]; then
# remove "sel(win)" in environment yaml hack since conda does not understand
# libmamba specific specs
echo "Installing mamba 1.x in dev mode..."
sed '/sel(.*)/d' "$SRC_MAMBA/mamba/environment-dev.yml" > /tmp/mamba-environment-dev.yml
CONDA_QUIET=1 "$BASE_CONDA/condabin/conda" env update -p "$BASE_CONDA" \
--file /tmp/mamba-environment-dev.yml
"$BASE_CONDA/condabin/conda" install make -yq # missing in mamba's dev env
# Clean build directory to avoid issues with stale build files
test -f "$SRC_MAMBA/build/CMakeCache.txt" && rm -rf "$SRC_MAMBA/build"
fi
# Compile
cd "$SRC_MAMBA"
"$BASE_CONDA/bin/cmake" -B build/ \
-DBUILD_LIBMAMBA=ON \
-DBUILD_SHARED=ON \
-DCMAKE_INSTALL_PREFIX="$BASE_CONDA" \
-DCMAKE_PREFIX_PATH="$BASE_CONDA" \
-DBUILD_LIBMAMBAPY=ON
"$BASE_CONDA/bin/cmake" --build build/ -j\${NPROC:-2}
if [ ! -f ~/.mamba-develop-installed ]; then
"$BASE_CONDA/condabin/conda" remove -p "$BASE_CONDA" -yq --force libmambapy libmamba
fi
make install -C build/
cd -
"$BASE_CONDA/bin/pip" install -e "$SRC_MAMBA/libmambapy/" --no-deps
test -f "$BASE_CONDA/conda-meta/mamba-"*".json" && "$BASE_CONDA/bin/pip" install -e "$SRC_MAMBA/mamba/" --no-deps
touch ~/.mamba-develop-installed || true
)
EOF

cd "$SRC_CONDA"
echo "Initializing conda in dev mode..."
"$BASE_CONDA/bin/python" -m conda init --dev bash
cd -

echo "Installing conda-libmamba-solver in dev mode..."
"$BASE_CONDA/bin/python" -m pip install -e "$SRC_CONDA_LIBMAMBA_SOLVER" --no-deps

set -x
conda list -p "$BASE_CONDA"
conda info
conda config --show-sources
set +x
test -f "$SRC_MAMBA/mamba/setup.py" \
&& echo "Mamba mounted at $SRC_MAMBA; source ~/.bashrc and run develop-mamba() for dev-install"
1 change: 1 addition & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -662,6 +662,7 @@ jobs:
steps:
- name: Determine Success
uses: re-actors/alls-green@v1.2.2
id: alls-green # CONDA-LIBMAMBA-SOLVER CHANGE
with:
# permit jobs to be skipped if there are no code changes (see changes job)
allowed-skips: ${{ toJSON(needs) }}
Expand Down
59 changes: 0 additions & 59 deletions dev/linux/bashrc.sh

This file was deleted.

38 changes: 0 additions & 38 deletions dev/linux/upstream_integration.sh

This file was deleted.

38 changes: 0 additions & 38 deletions dev/linux/upstream_unit.sh

This file was deleted.

3 changes: 2 additions & 1 deletion docs/dev/implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ both within the `conda_libmamba_solver` package itself, and as a `conda` plugin.

## Repository structure

* `.devcontainer`: Configuration for DevContainer development workflows.
* `.github/workflows/`: CI pipelines to run unit and upstream tests, as well as linting and performance benchmarks. Some extra workflows might be added by the `conda/infra` settings.
* `conda_libmamba_solver/`: The Python package. Check sections below for details.
* `recipe/`: The conda-build recipe used for the PR build previews. It should be kept in sync with `conda-forge` and `defaults`.
* `dev/`: Supporting scripts and configuration files to set up development environments.
* `dev/`: Supporting configuration files to set up development environments.
* `docs/`: Documentation sources.
* `tests/`: pytest testing infrastructure.
* `pyproject.toml`: Project metadata. See below for details.
Expand Down
Loading
Loading