Skip to content

Commit

Permalink
feat(template): Adding azure-pipelines to template (#317)
Browse files Browse the repository at this point in the history
  • Loading branch information
Anavelyz authored Oct 25, 2024
1 parent d9497ac commit d8c990a
Show file tree
Hide file tree
Showing 12 changed files with 22,859 additions and 13 deletions.
22 changes: 16 additions & 6 deletions docs/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The structure of the guide is as follows:
- [Governance document](#governance-document)
- [Roadmap document](#roadmap-document)
- [Version control](#version-control)
- [Continuos Integration](#continuos-integration)

## Notes about the text-based user interface (TUI)

Expand Down Expand Up @@ -985,9 +986,18 @@ quickly identify and fix problems. By automating repetitive tasks, CI also saves
time and effort. With SciCookie you can integrate tools like GitHub and/or
CircleCI into your project.

- GitHub Actions (**github_actions**): This option allows you to incorporate
GitHub Actions into your project. As a native CI/CD platform integrated with
GitHub, it offers flexible workflows and an easy setup process.
- CircleCI (**circleci**): This option enables you to integrate CircleCI into
your project. Known for its speed and reliability, CircleCI is a cloud-based
CI/CD platform that streamlines the development process.
- [GitHub Actions](https://github.com/features/actions) (**github_actions**):
This option allows you to incorporate GitHub Actions into your project. As a
native CI/CD platform integrated with GitHub, it offers flexible workflows and
an easy setup process.
- [CircleCI](https://circleci.com/) (**circleci**): This option enables you to
integrate CircleCI into your project. Known for its speed and reliability,
CircleCI is a cloud-based CI/CD platform that streamlines the development
process.
- [Azure Pipelines](https://azure.microsoft.com/products/devops/pipelines/)
(**azure_pipelines**): With this option, you can integrate Azure Pipelines
into your into your project. It is a robust CI/CD platform that enables
automation of testing and deployments across multiple environments and
supports multiple languages and platforms. It provides continuous integration
with GitHub, Azure Repos and other version control systems, facilitating an
agile and efficient workflow.
3 changes: 2 additions & 1 deletion src/scicookie/cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,5 +80,6 @@
"git_https_upstream": "",
"git_main_branch": "main",
"use_github_actions": "no",
"use_circleci": "no"
"use_circleci": "no",
"use_azure_pipelines": "no"
}
3 changes: 3 additions & 0 deletions src/scicookie/hooks/post_gen_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
USE_HYPOTHESIS = {{ cookiecutter.use_hypothesis == "yes" }}
USE_GITHUB_ACTIONS = {{ cookiecutter.use_github_actions == "yes" }}
USE_CIRCLECI = {{ cookiecutter.use_circleci == "yes" }}
USE_AZURE = {{ cookiecutter.use_azure_pipelines == "yes" }}
{% if cookiecutter.code_of_conduct == "contributor-covenant" -%}
COC_PATH = PROJECT_DIRECTORY / 'coc' / 'CONTRIBUTOR_COVENANT.md'
{%- elif cookiecutter.code_of_conduct == "citizen-code-of-conduct" -%}
Expand Down Expand Up @@ -327,6 +328,8 @@ def clean_up_ci():
remove_dir(".circleci")
if not USE_GITHUB_ACTIONS:
remove_dir(".github")
if not USE_AZURE:
remove_project_file("azure-pipelines.yml")

def http2ssh(url):
url = url.replace("https://", "git@")
Expand Down
1 change: 1 addition & 0 deletions src/scicookie/profiles/base.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -298,4 +298,5 @@ continuos_integration:
choices:
- github_actions
- circleci
- azure_pipelines
visible: true
11 changes: 8 additions & 3 deletions src/scicookie/{{cookiecutter.project_slug}}/.circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
steps:
- checkout
- run:
name: Create environment
name: Create environment and install dependencies
command: |
conda env create -f conda/dev.yaml
conda init bash
Expand All @@ -38,11 +38,14 @@ jobs:
python -m pip install -r requirements.txt
{%- endif %}
{%- if cookiecutter.build_system == "poetry" %}
poetry config virtualenvs.create false;
poetry install
{%- elif cookiecutter.build_system == "flit" %}
flit install
{%- elif cookiecutter.build_system == "pdm" %}
pdm install
{%- elif cookiecutter.build_system == "pixi" %}
pixi install
{%- else %}
pip install .
{%- endif %}
Expand All @@ -53,10 +56,12 @@ jobs:
{%- if cookiecutter.use_makim == "yes" %}
makim tests.unit
makim tests.linter
{%- endif %}
{%- if cookiecutter.use_make == "yes" %}
{%- elif cookiecutter.use_make == "yes" %}
make test
make lint
{%- else %}
# add your command for running tests here
# add your command for running the linter here
{%- endif %}
{%- if cookiecutter.use_conda == "yes" %}
- save_cache:
Expand Down
83 changes: 83 additions & 0 deletions src/scicookie/{{cookiecutter.project_slug}}/azure-pipelines.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Python package
# Create and test a Python package on multiple Python versions.
# Add steps that analyze code, save the dist with the build record, publish to a PyPI-compatible index, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/python

trigger:
- main

pool:
vmImage: ubuntu-latest # Adjust if needed

steps:
{%- if cookiecutter.use_conda == "yes" %}
- script: |
conda env create -f conda/dev.yaml
conda init
source ~/.bashrc
source /usr/share/miniconda/bin/activate
conda activate {{ cookiecutter.project_slug }}
{%- if cookiecutter.build_system == "poetry" %}
poetry install
{%- elif cookiecutter.build_system == "flit" %}
flit install
{%- elif cookiecutter.build_system == "pdm" %}
pdm install
{%- else %}
pip install .
{%- endif %}
{%- if cookiecutter.use_pre_commit == "yes" %}
pre-commit install
{%- endif %}
displayName: "Install dependencies"
{%- else %}
- task: UsePythonVersion@0
inputs:
versionSpec: "3.10" # Adjust if needed

- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
{%- if cookiecutter.build_system == "poetry" %}
poetry config virtualenvs.create false;
poetry install
{%- elif cookiecutter.build_system == "flit" %}
flit install
{%- elif cookiecutter.build_system == "pdm" %}
pdm install
{%- elif cookiecutter.build_system == "pixi" %}
pixi install
{%- else %}
pip install .
{%- endif %}
{%- if cookiecutter.use_pre_commit == "yes" %}
pre-commit install
{%- endif %}
displayName: "Install dependencies"
{%- endif %}
- script: |
{%- if cookiecutter.use_conda == "yes" %}
source /usr/share/miniconda/bin/activate
conda activate {{ cookiecutter.project_slug }}
{%- endif %}
{%- if cookiecutter.use_makim == "yes" %}
makim tests.unit
{%- elif cookiecutter.use_make == "yes" %}
make test
{%- else %}
# add your command for running tests here
{%- endif %}
displayName: "Tests"
- script: |
{%- if cookiecutter.use_conda == "yes" %}
source /usr/share/miniconda/bin/activate
conda activate {{ cookiecutter.project_slug }}
{%- endif %}
{%- if cookiecutter.use_makim == "yes" %}
makim tests.linter
{%- elif cookiecutter.use_make == "yes" %}
make lint
{%- else %}
# add your command for running the linter here
{%- endif %}
displayName: "Run style checks"
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@ dependencies:
- pip:
# distlib issue
- paginate==0.5.6
{%- endif %}
{%- endif %}
1 change: 1 addition & 0 deletions tests/profiles/test-depends-on.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -307,4 +307,5 @@ continuos_integration:
choices:
- github_actions
- circleci
- azure_pipelines
visible: true
1 change: 1 addition & 0 deletions tests/profiles/test-main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -303,4 +303,5 @@ continuos_integration:
choices:
- github_actions
- circleci
- azure_pipelines
visible: true
32 changes: 32 additions & 0 deletions tests/scripts/check_azure_pipelines.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
"""Test Azure Pipelines YAML files."""

import json
import sys

import yaml

from jsonschema import validate

print(sys.argv)
args = 3
if len(sys.argv) != args:
raise Exception(
"Usage: python test_azure_pipelines.py <schema_file> <yaml_file>"
)

schema_file = sys.argv[1]
yaml_file = sys.argv[2]

with open(schema_file, "r") as f:
schema = json.load(f)

with open(yaml_file, "r") as f:
yaml_data = yaml.safe_load(f)


validate(instance=yaml_data, schema=schema)
print(
"The file {} is valid according to the schema {}.".format(
yaml_file, schema_file
)
)
16 changes: 14 additions & 2 deletions tests/smoke/ci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@ set -e

SMOKE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# run the complete smoke tests
# Run the complete smoke tests
. ${SMOKE_DIR}/base.sh "use_circleci=yes use_conda=yes"
. ${SMOKE_DIR}/base.sh "use_circleci=yes use_pyenv=yes"
. ${SMOKE_DIR}/base.sh "use_azure_pipelines=yes use_conda=yes"
. ${SMOKE_DIR}/base.sh "use_azure_pipelines=yes use_pyenv=yes"

# check the CI files from template
# Check the CI files from template
. ${SMOKE_DIR}/base-template.sh "use_circleci=yes use_conda=yes"
pushd /tmp/osl/osl-python-package
circleci config validate
Expand All @@ -17,3 +19,13 @@ popd
pushd /tmp/osl/osl-python-package
circleci config validate
popd

. ${SMOKE_DIR}/base-template.sh "use_azure_pipelines=yes use_conda=yes"
pushd /tmp/osl/osl-python-package
python "${SMOKE_DIR}/../scripts/check_azure_pipelines.py" ${SMOKE_DIR}/schemas/azure-pipelines.json azure-pipelines.yml
popd

. ${SMOKE_DIR}/base-template.sh "use_azure_pipelines=yes use_pyenv=yes"
pushd /tmp/osl/osl-python-package
python "${SMOKE_DIR}/../scripts/check_azure_pipelines.py" ${SMOKE_DIR}/schemas/azure-pipelines.json azure-pipelines.yml
popd
Loading

0 comments on commit d8c990a

Please sign in to comment.