Skip to content

Commit

Permalink
Fixes for Viash 0.9 (#27)
Browse files Browse the repository at this point in the history
* Update action.yml

* Update jq query

* update for viash 0.9

* add matrix to output of ns list

* update docs

* fix description

* add more actions

* update action

* update readme

* remove functionality

* remove variable

* update action

* update

* fix action

* fix action

* fix ci

* pin versions

* fix action

* fix ci

* fix ci

* fix ci

* fix ci

* fix ci

* fix ci

* fix ci

* add matrix to output

* fix ci

* fix action

* fix action

* fix action

* improve readability

* fix action

* clean up action

* move update docker engine

* remove viash hub deploy

* arguments `components` and `workflows` were removed in favour for the `target_dir` argument.

* uniformize documentation pages

* add changelog

* update docs

* update changelog

* update mentioned viash versions to v6

* fix platform description

* fix indentation

* update readme

* simplify readme

* fix check-concurrent-pr

* fix fallback

* add paste

* fix flag

* add entry to changelog
  • Loading branch information
rcannood authored Jun 28, 2024
1 parent 2e6acdb commit 9706ce0
Show file tree
Hide file tree
Showing 43 changed files with 1,702 additions and 585 deletions.
33 changes: 31 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,32 @@
# viash-actions v6.0.0

## Breaking changes

* `update-docker-engine` was moved to `project/update-docker-engine` (PR #27).

* `pro/build-nextflow-params`: Arguments `components` and `workflows` were removed in favour for the `target_dir` argument (PR #27).

* `pro/build-nextflow-schemas`: Arguments `components` and `workflows` were removed in favour for the `target_dir` argument (PR #27).

* Removed `viash-hub/deploy`: Viash Hub pulls source code from GitHub and builds Viash packages accordingly, as opposed to pushing deployments using GitHub Actions (PR #27).

## New functionality

* `project/build-target`: Created an action for building a target directory using `viash ns build` and generating a matrix of the components for which Docker images need to be built and pushed (PR #27).

* `project/check-concurrent-pr`: Created an action for checking if a push event is concurrent with a pull request (PR #27).

## Major changes

* All actions were updated to work with Viash 0.9 and later (PR #27).

## Documentation

* Updated all READMEs to be more consistent (PR #27).

* Added a Makefile for generating READMEs more efficiently (PR #27).


# viash-actions v5.3.1

## Bug fixes
Expand All @@ -20,13 +49,13 @@

## Minor changes

* `project/detect-changed-components`: bump `tj-actions/changed-files` to `42.0.2` (PR #24)
* `project/detect-changed-components`: bump `tj-actions/changed-files` to `42.0.2` (PR #24).

# viash-actions v5.1.0

## New functionality

* `pro/build-nextflow-schemas`: Add optional flag to allow dataset input for nf-tower (PR #23)
* `pro/build-nextflow-schemas`: Add optional flag to allow dataset input for nf-tower (PR #23).

# viash-actions v5.0.0

Expand Down
15 changes: 15 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Set up some variables for consistency
QMD_FILES := $(shell find . -name "README.qmd") # Find all README.qmd files
MD_FILES := $(patsubst %.qmd,%.md,$(QMD_FILES)) # Corresponding MD filenames

# Default target: build all MD files
all: $(MD_FILES)

README.md: README.qmd
quarto render $<
%/README.md: %/README.qmd %/action.yml
quarto render $<

# Clean up generated MD files
clean:
rm -f $(MD_FILES)
55 changes: 41 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,50 @@
# Github Actions for Viash

This repository stores [Github Actions](https://github.com/features/actions) useful for building and publishing [Viash](https://viash.io) components:

1. [viash-io/viash-actions/setup](setup) - Install Viash
2. [viash-io/viash-actions/ns-list](ns-list) - List Viash components in a repository
3. [viash-io/viash-actions/ns-build](ns-build) - Build a namespace from many viash config files
This repository stores Github Actions useful for building and publishing
[Viash](https://viash.io) components:

There are also a few actions that are commonly used in Viash projects:
1. [`ns-build`](ns-build) - viash ns build
2. [`ns-list`](ns-list) - viash ns list
3. [`setup`](setup) - Setup Viash

1. [viash-io/viash-actions/project/cache-and-sync-s3](project/cache-and-sync-s3) - Sync and cache an S3 bucket
2. [viash-io/viash_actions/project/detect-changed-components](project/detect-changed-components) - Detect changed Viash components
There are also some actions that are commonly used in Viash projects:

We recommend using `v3` for your actions.
1. [`project/build-target`](project/build-target) - Build target
directory
2. [`project/detect-changed-components`](project/detect-changed-components) -
Detect components with changed files
3. [`project/is-pr`](project/is-pr) - Is PR
4. [`project/sync-and-cache-s3`](project/sync-and-cache-s3) - Sync and
cache an S3 bucket
5. [`project/update-docker-engine`](project/update-docker-engine) -
Update Docker Engine

## Release Management
Finally, there are some [Viash
Pro](https://www.data-intuitive.com/services/viashpro.html) actions:

1. [`pro/build-nextflow-params`](pro/build-nextflow-params) - Build
parameter yaml
2. [`pro/build-nextflow-schemas`](pro/build-nextflow-schemas) - Build
nf-tower schemas
3. [`pro/generate-documentation-qmd`](pro/generate-documentation-qmd) -
Build and publish qmd

This repository uses [GitHub's recommended release management for actions](https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-release-management-for-actions):
We recommend using `v6` for your actions.

## Release Management

* GitHub releases with tags are used for updates on the actions.
* Semantic versioning is used, with major, minor and possibly patch release.
* Major versions (such as `v1`) will always point to the last minor or patch release for this major version. (when `v1.0.2` is out, `v1` will point to this update to). This means using `viash-io/viash-actions/setup@v1` in your workflow file will automatically get the updated versions. Using `viash-io/viash-actions/setup@v1.0.2` will pin a specific release.
* Major version changes (`v1` to `v2`) will often come with breaking changes, and workflows might require manual updates.
This repository uses [GitHub’s recommended release management for
actions](https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-release-management-for-actions):

- GitHub releases with tags are used for updates on the actions.
- Semantic versioning is used, with major, minor and possibly patch
release.
- Major versions (such as `v1`) will always point to the last minor or
patch release for this major version. (when `v1.0.2` is out, `v1` will
point to this update to). This means using
`viash-io/viash-actions/setup@v1` in your workflow file will
automatically get the updated versions. Using
`viash-io/viash-actions/setup@v1.0.2` will pin a specific release.
- Major version changes (`v1` to `v2`) will often come with breaking
changes, and workflows might require manual updates.
54 changes: 54 additions & 0 deletions README.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: Github Actions for Viash
format: gfm
---

```{r}
#| include: false
library(tidyverse)
repo_name <- "viash-io/viash-actions"
latest_stable_version <- "v6"
actions <- tibble(
file = list.files(".", full.names = TRUE, recursive = TRUE, pattern = "action.yml"),
path = gsub("^\\./(.*)/action.yml$", "\\1", file),
action = map(file, yaml::read_yaml),
action_name = map_chr(action, "name"),
namespace = ifelse(grepl("/", path), gsub("/.*", "", path), ""),
str = paste0("1. [`", path, "`](", path, ") - ", action_name, "\n")
)
```


This repository stores Github Actions useful for building and publishing [Viash](https://viash.io) components:

```{r output="asis"}
#| echo: false
cat(actions %>% filter(namespace == "") %>% pull(str) %>% paste(collapse = ""))
```

There are also some actions that are commonly used in Viash projects:

```{r output="asis"}
#| echo: false
cat(actions %>% filter(namespace == "project") %>% pull(str) %>% paste(collapse = ""))
```

Finally, there are some [Viash Pro](https://www.data-intuitive.com/services/viashpro.html) actions:

```{r output="asis"}
#| echo: false
cat(actions %>% filter(namespace == "pro") %>% pull(str) %>% paste(collapse = ""))
```

We recommend using ``r latest_stable_version`` for your actions.

## Release Management

This repository uses [GitHub's recommended release management for actions](https://docs.github.com/en/actions/creating-actions/about-custom-actions#using-release-management-for-actions):

* GitHub releases with tags are used for updates on the actions.
* Semantic versioning is used, with major, minor and possibly patch release.
* Major versions (such as `v1`) will always point to the last minor or patch release for this major version. (when `v1.0.2` is out, `v1` will point to this update to). This means using `viash-io/viash-actions/setup@v1` in your workflow file will automatically get the updated versions. Using `viash-io/viash-actions/setup@v1.0.2` will pin a specific release.
* Major version changes (`v1` to `v2`) will often come with breaking changes, and workflows might require manual updates.
62 changes: 42 additions & 20 deletions ns-build/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
ns-build
================


# ns-build

<!--
DO NOT EDIT THIS FILE MANUALLY!
This README was generated by running `make`
-->

Build a namespace from many viash config files.

Expand All @@ -11,7 +17,7 @@ each of the arguments.

We recommend using a Linux or MacOS runner if possible.

### Inputs available
### Inputs

- `query`: - *optional*. Filter which components get selected by
component and namespace name. Can be a regex. Example:
Expand All @@ -22,39 +28,55 @@ We recommend using a Linux or MacOS runner if possible.
component name. Can be a regex. Example: “^component1”.
- `src`: - *optional*. A source directory containing viash config files,
possibly structured in a hierarchical folder structure. Default: src/.
- `platform`: - *optional*. Acts as a regular expression to filter the
platform ids specified in the found config files. If this is not
provided, all platforms will be used. If no platforms are defined in a
config, the native platform will be used. In addition, the path to a
platform yaml file can also be specified.
- `platform`: - *optional*. (viash \< 0.9.0) Acts as a regular
expression to filter the platform ids specified in the found config
files. If this is not provided, all platforms will be used. If no
platforms are defined in a config, the native platform will be used.
In addition, the path to a platform yaml file can also be specified.
- `runner`: - *optional*. (viash \>= 0.9.0) Acts as a regular expression
to filter the runner ids specified in the found config files. If this
is not provided, all runners will be used. If no runners are defined
in a config, the executable runner will be used.
- `engine`: - *optional*. (viash \>= 0.9.0) Acts as a regular expression
to filter the engine ids specified in the found config files. If this
is not provided, all engines will be used. If no engines are defined
in a config, the native engine will be used.
- `parallel`: - *optional*. Whether or not to run the process in
parallel.
- `config_mod`: - *optional*. Modify a viash config at runtime using
dynamic config modding.
config_mod.
- `target`: - *optional*. A target directory to build the executables
into. Default: target/.
- `setup`: - *optional*. Which setup strategy for creating the container
to use \[Docker Platform only\].
- `colorize`: - *optional*. Specify whether the console output should be
colorized. If not specified, we attempt to detect this automatically.
Possible values are: “true”, “false”, “auto”.
- `loglevel`: - *optional*. Specify the log level in us. Possible values
are: “error”, “warn”, “info”, “debug”, “trace”.
- `setup`: - *optional*. Which docker_setup_strategy for creating the
container to use \[Docker Engine only\].

### Outputs

- `output`: The output of ‘viash ns build’.

## Examples

``` yaml
name: demo of viash ns build
name: Demo of ns-build

on:
branch: main
push:

jobs:
viash-linux:
runs-on: ubuntu-latest
steps:
- uses: viash-io/viash-actions/setup@v3
- name: Check out repository
uses: actions/checkout@v3

- uses: viash-io/viash-actions/ns-build@v3
viash-macos:
runs-on: macos-latest
steps:
- uses: viash-io/viash-actions/setup@v3
- name: Install Viash
uses: viash-io/viash-actions/setup@v6

- uses: viash-io/viash-actions/ns-build@v3
- name: viash ns build
uses: viash-io/viash-actions/ns-build@v6
```
40 changes: 28 additions & 12 deletions ns-build/README.qmd
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
---
title: ns-build
format: gfm
---

```{r}
#| include: false
library(tidyverse)
# action <- yaml::read_yaml("ns-build/action.yml")
action <- yaml::read_yaml("action.yml")
action_name <- basename(getwd())
```

# `r action_name`

<!--
DO NOT EDIT THIS FILE MANUALLY!
This README was generated by running `make`
-->

`r action$description`

## Usage
Expand All @@ -18,7 +24,7 @@ This action will run `viash ns build`. See the reference documentation on [ns bu

We recommend using a Linux or MacOS runner if possible.

### Inputs available
### Inputs

```{r}
#| echo: false
Expand All @@ -30,26 +36,36 @@ lines <- map_chr(names(action$inputs), function(name) {
knitr::asis_output(paste0(lines, collapse = "\n"))
```

### Outputs

```{r}
#| echo: false
lines <- map_chr(names(action$outputs), function(name) {
output <- action$outputs[[name]]
glue::glue("* `{name}`: {output$description}")
})
knitr::asis_output(paste0(lines, collapse = "\n"))
```

## Examples

```yaml
name: demo of viash ns build
name: Demo of `r action_name`

on:
branch: main
push:

jobs:
viash-linux:
runs-on: ubuntu-latest
steps:
- uses: viash-io/viash-actions/setup@v3
- name: Check out repository
uses: actions/checkout@v3

- uses: viash-io/viash-actions/ns-build@v3
viash-macos:
runs-on: macos-latest
steps:
- uses: viash-io/viash-actions/setup@v3
- name: Install Viash
uses: viash-io/viash-actions/setup@v6

- uses: viash-io/viash-actions/ns-build@v3
- name: `r action$name`
uses: viash-io/viash-actions/ns-build@v6
```
Loading

0 comments on commit 9706ce0

Please sign in to comment.