-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conda Bump version #273
Conda Bump version #273
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested on a few repository suggested in the description using the previous tag and running the suggested commands.
In addition to the various inline comment, the primary issue for repositories with multiple packages like conda-raw2ometiff
or conda-bftools
is that the transformation works for one of the packages but the other package is completely overwritten with the same YAML.
A minor formatting issue is that the YAML file is writtent with different indentation style, making the diff longer than it should be.
scc/git.py
Outdated
return | ||
|
||
if args.repo: | ||
url = self.GITHUB_URL + args.repo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we actually enforce this at the metadata layer? i.e. have some key that would always refer to the source GitHub repository so that this would not need to be pass at runtime?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went with the current status of the repositories. Having a common template will certainly simplify the handling
The BF conda file does not follow any of the conventions used in the other repositories, leading to the indentation style difference |
Ha, the change I made to fix the writing introduced a bug for repositories with multiple meta files. Thanks for checking that. |
This is unrelated to the style of the diff --git a/meta.yaml b/meta.yaml
index e38e08d..787dca5 100644
--- a/meta.yaml
+++ b/meta.yaml
@@ -1,13 +1,14 @@
{% set name = "omero-py" %}
-{% set version = "5.9.1" %}
+{% set version = 5.10.0 %}
package:
name: "{{ name|lower }}"
version: "{{ version }}{{ environ.get('VERSION_SUFFIX', '') }}"
source:
- url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
- sha256: 173b09411b45e4f728b54fd8203c246e2e114d0cc75fa083c5eecee76456cb21
+ url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version
+ }}.tar.gz
+ sha256: b9e8d71ca56131367045e976c490e6c0913525e0331a504d639a6d10f771e42d
build:
# Can't use noarch because Windows requirements are different
@@ -16,46 +17,46 @@ build:
number: 1
script: "{{ PYTHON }} -m pip install . --no-deps --ignore-installed -vv "
script_env:
- - VERSION_SUFFIX
+ - VERSION_SUFFIX
entry_points:
- - omero = omero.main:main
+ - omero = omero.main:main
requirements:
host:
- - pip
- - python
+ - pip
+ - python
run:
- - appdirs
- - future
- - numpy
- - pillow
- - python
- - pywin32 # [win]
- - requests
- - pyyaml
- - zeroc-ice36-python
+ - appdirs
+ - future
+ - numpy
+ - pillow
+ - python
+ - pywin32 # [win]
+ - requests
+ - pyyaml
+ - zeroc-ice36-python
test:
commands:
- - omero --help
- - omero version
+ - omero --help
+ - omero version
imports:
- - omero
- - omero.api
- - omero.clients
- - omero.cmd
- - omero.constants
- - omero.fs
- - omero.gateway
- - omero.grid
- - omero.install
- - omero.metadatastore
- - omero.model
- - omero.plugins
- - omero.romio
- - omero.sys
- - omero.util
- - omero_ext
+ - omero
+ - omero.api
+ - omero.clients
+ - omero.cmd
+ - omero.constants
+ - omero.fs
+ - omero.gateway
+ - omero.grid
+ - omero.install
+ - omero.metadatastore
+ - omero.model
+ - omero.plugins
+ - omero.romio
+ - omero.sys
+ - omero.util
+ - omero_ext
about:
home: https://www.openmicroscopy.org/
@@ -67,4 +68,4 @@ about:
extra:
recipe-maintainers:
- - ome
+ - ome |
aha, my file must have been already "correctly" formatted during my tests |
The package |
👍 for YAML consistency across all files and hacing it enforced by the writing tool. I was just surprised by the indentation of the block list item mostly because other places like the Ansible roles use |
All the points have now been fixed (I think). Comment #273 (comment), the code could be removed after changing the logic in the yaml files across the various repositories |
Retested on the various repository
diff --git a/meta.yaml b/meta.yaml
index 081bff7..787dca5 100644
--- a/meta.yaml
+++ b/meta.yaml
@@ -1,12 +1,13 @@
{% set name = "omero-py" %}
-{% set version = "5.10.0" %}
+{% set version = 5.10.0 %}
package:
name: "{{ name|lower }}"
version: "{{ version }}{{ environ.get('VERSION_SUFFIX', '') }}"
source:
- url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
+ url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version
+ }}.tar.gz
sha256: b9e8d71ca56131367045e976c490e6c0913525e0331a504d639a6d10f771e42d
build:
diff --git a/bftools-libs/meta.yaml b/bftools-libs/meta.yaml
index 568c893..9b30479 100644
--- a/bftools-libs/meta.yaml
+++ b/bftools-libs/meta.yaml
@@ -1,6 +1,6 @@
package:
name: "bftools-libs"
- version: "6.6.0"
+ version: "6.7.0"
build:
number: 0
@@ -8,7 +8,7 @@ build:
source:
url: https://downloads.openmicroscopy.org/bio-formats/{{ PKG_VERSION }}/artifacts/bftools.zip
- sha256: d13553047ac552327107e029a077e32b48a7e0f7e028cf4c04d81539f2f56a21
+ sha256: 31a4200f7c5271bd11640743244b15ecb5166214
extra:
recipe-maintainers:
diff --git a/bftools/meta.yaml b/bftools/meta.yaml
index 1801170..8f18f73 100644
--- a/bftools/meta.yaml
+++ b/bftools/meta.yaml
@@ -1,13 +1,13 @@
package:
name: "bftools"
- version: "6.6.0"
+ version: "6.7.0"
build:
number: 0
source:
url: https://downloads.openmicroscopy.org/bio-formats/{{ PKG_VERSION }}/artifacts/bftools.zip
- sha256: d13553047ac552327107e029a077e32b48a7e0f7e028cf4c04d81539f2f56a21
+ sha256: 31a4200f7c5271bd11640743244b15ecb5166214
requirements:
run:
All non-whitespace diff are related to the formatting style discussed earlier. Two outstanding issue:
|
I agree. As this PR is trying to handle all these cases, the two options I can think of would be:
Is there a shortlist of the divergences that need to be reviewed? |
Differences
We may also want to add the repo i.e. ome/omero-py for example so we do not have to pass the parameter The problems reported for |
@sbesson the problems described in #273 (comment) should now be fixed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re-executed the last commit against the various repositories
conda-omero-py
:git checkout 5.9.1-0 & scc bump-version-conda && git diff -w 5.10.0-0
diff --git a/meta.yaml b/meta.yaml
index 081bff7..fb5658c 100644
--- a/meta.yaml
+++ b/meta.yaml
@@ -6,7 +6,8 @@ package:
version: "{{ version }}{{ environ.get('VERSION_SUFFIX', '') }}"
source:
- url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
+ url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version
+ }}.tar.gz
sha256: b9e8d71ca56131367045e976c490e6c0913525e0331a504d639a6d10f771e42d
build:
conda-bftools
:git checkout 6.6.0 && scc bump-version-conda --repo ome/bioformats && git diff -w 6.6.0
diff --git a/bftools-libs/meta.yaml b/bftools-libs/meta.yaml
index 568c893..e02b1da 100644
--- a/bftools-libs/meta.yaml
+++ b/bftools-libs/meta.yaml
@@ -1,6 +1,6 @@
package:
name: "bftools-libs"
- version: "6.6.0"
+ version: "6.7.0"
build:
number: 0
@@ -8,7 +8,7 @@ build:
source:
url: https://downloads.openmicroscopy.org/bio-formats/{{ PKG_VERSION }}/artifacts/bftools.zip
- sha256: d13553047ac552327107e029a077e32b48a7e0f7e028cf4c04d81539f2f56a21
+ sha256: 7c013b23529d61280e04c3dd312401d7b71fb5892f5e7870f87d37bab5f3988f
extra:
recipe-maintainers:
diff --git a/bftools/meta.yaml b/bftools/meta.yaml
index 1801170..05b5ad9 100644
--- a/bftools/meta.yaml
+++ b/bftools/meta.yaml
@@ -1,13 +1,13 @@
package:
name: "bftools"
- version: "6.6.0"
+ version: "6.7.0"
build:
number: 0
source:
url: https://downloads.openmicroscopy.org/bio-formats/{{ PKG_VERSION }}/artifacts/bftools.zip
- sha256: d13553047ac552327107e029a077e32b48a7e0f7e028cf4c04d81539f2f56a21
+ sha256: 7c013b23529d61280e04c3dd312401d7b71fb5892f5e7870f87d37bab5f3988f
requirements:
run:
conda-bioformats2raw
:git checkout 0.2.6-0 && scc bump-version-conda --repo glencoesoftware/bioformats2raw && git diff 0.3.0-0 -w
diff --git a/bioformats2raw-libs/meta.yaml b/bioformats2raw-libs/meta.yaml
index a6448a9..ed80323 100644
--- a/bioformats2raw-libs/meta.yaml
+++ b/bioformats2raw-libs/meta.yaml
@@ -10,7 +10,8 @@ build:
noarch: generic
source:
- url: https://github.com/glencoesoftware/bioformats2raw/releases/download/v{{ version }}/bioformats2raw-{{ version }}.zip
+ url: https://github.com/glencoesoftware/bioformats2raw/releases/download/v{{ version
+ }}/bioformats2raw-{{ version }}.zip
sha256: dbc74eee15b4ce46170aac67de9a1f2849b89b8d84ab55792140d30cd206f8e4
# Leave the OS specific dependencies to the wrapper
diff --git a/bioformats2raw/meta.yaml b/bioformats2raw/meta.yaml
index 4498ea3..a09ad69 100644
--- a/bioformats2raw/meta.yaml
+++ b/bioformats2raw/meta.yaml
@@ -9,7 +9,8 @@ build:
number: 0
source:
- url: https://github.com/glencoesoftware/{{ name }}/releases/download/v{{ version }}/{{ name }}-{{ version }}.zip
+ url: https://github.com/glencoesoftware/{{ name }}/releases/download/v{{ version
+ }}/{{ name }}-{{ version }}.zip
sha256: dbc74eee15b4ce46170aac67de9a1f2849b89b8d84ab55792140d30cd206f8e4
patches:
- windows-bat-classpath-wildcard.patch # [win]
diff --git a/bioformats2raw/windows-bat-classpath-wildcard.patch b/bioformats2raw/windows-bat-classpath-wildcard.patch
index 887c6ba..bdd806a 100644
--- a/bioformats2raw/windows-bat-classpath-wildcard.patch
+++ b/bioformats2raw/windows-bat-classpath-wildcard.patch
@@ -1,10 +1,10 @@
---- bin/bioformats2raw.bat.orig 2021-08-02 13:34:52.000000000 +0100
-+++ bin/bioformats2raw.bat 2021-08-02 13:35:10.000000000 +0100
+--- bin/bioformats2raw.bat.orig 2020-10-28 10:10:16.000000000 +0000^M
++++ bin/bioformats2raw.bat 2020-10-28 10:10:32.000000000 +0000^M
....
Also executed the scc bump-conda-version
from the HEAD of each repository and the only changes where whitespaces (indentation) and the split of url:
into multilines.
Functionally this works as expected and I am happy to get this merged to move into the next step.
From the discussion above,a potential set of actions:
- run the command on each repository, open a PR with the formatting changes, probably make a
build
release as well to confirm it works? - test the command is a no-op when no change after this commit has been made
- unify
version
,name
andsha256
- store the GitHub information in the
meta.yml
. I would suggesthome:
is the natural place to store this - https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#about-section
Coming back to #273, should we wait until the repositories have been cleaned up and the logic reduced before making a release of this new command?
We already use the |
Actually I am going to push another commit using the |
👍 A separate issue is that these terms are fairly undefined. And at least I feel defining https://ww.openmicroscopy.org/ as the home of https://anaconda.org/ome/bioformats2raw is a bit of a stretch for instance. |
With 75e59d2, it is no longer necessary to use |
Yes, this should be changed. |
Co-authored-by: Sébastien Besson <seb.besson@gmail.com>
Co-authored-by: Sébastien Besson <seb.besson@gmail.com>
Co-authored-by: Sébastien Besson <seb.besson@gmail.com>
Co-authored-by: Sébastien Besson <seb.besson@gmail.com>
Briefly discussed with @jburel. The only addition I would suggest would be to have some logic to also reset the |
The fact that the changes were made even if the version was the same has now be fixed i.e. running scc on the latest conda-omero-py should stop at the version check |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested the fail-fast for the version check
(base) sbesson@ls30630:conda-omero-py ((5.10.0-0)) $ /opt/ome/scc/venv/bin/scc bump-version-conda && echo success
2021-10-04 09:22:00,211 [scc.bump-ver] INFO no new version
success
(base) sbesson@ls30630:conda-raw2ometiff ((0.3.0-0)) $ /opt/ome/scc/venv/bin/scc bump-version-conda && echo success
2021-10-04 09:22:19,777 [scc.bump-ver] INFO no new version
success
Also tested the build number reset using omero-py
(base) sbesson@ls30630:conda-omero-py ((5.10.0-0)) $ git checkout 5.9.0-0
Previous HEAD position was 1637920 Merge pull request #17 from jburel/bump510
HEAD is now at d029c15 Merge pull request #16 from jburel/bump_version
(base) sbesson@ls30630:conda-omero-py ((5.9.0-0)) $ /opt/ome/scc/venv/bin/scc bump-version-conda
(base) sbesson@ls30630:conda-omero-py ((fbe4420...)) $ git diff 5.10.0-0
diff --git a/meta.yaml b/meta.yaml
index 081bff7..5576ea5 100644
--- a/meta.yaml
+++ b/meta.yaml
@@ -6,56 +6,57 @@ package:
version: "{{ version }}{{ environ.get('VERSION_SUFFIX', '') }}"
source:
- url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version }}.tar.gz
+ url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/{{ name }}-{{ version
+ }}.tar.gz
sha256: b9e8d71ca56131367045e976c490e6c0913525e0331a504d639a6d10f771e42d
build:
# Can't use noarch because Windows requirements are different
# If we remove pywin32 from OMERO.py we could switch to noarch
# noarch: python
- number: 1
+ number: 0
script: "{{ PYTHON }} -m pip install . --no-deps --ignore-installed -vv "
script_env:
- - VERSION_SUFFIX
+ - VERSION_SUFFIX
entry_points:
- - omero = omero.main:main
+ - omero = omero.main:main
requirements:
host:
- - pip
- - python
+ - pip
+ - python
run:
- - appdirs
- - future
- - numpy
- - pillow
- - python
- - pywin32 # [win]
- - requests
- - pyyaml
- - zeroc-ice36-python
+ - appdirs
+ - future
+ - numpy
+ - pillow
+ - python
+ - pywin32 # [win]
+ - requests
+ - pyyaml
+ - zeroc-ice36-python
test:
commands:
- - omero --help
- - omero version
+ - omero --help
+ - omero version
imports:
- - omero
- - omero.api
- - omero.clients
- - omero.cmd
- - omero.constants
- - omero.fs
- - omero.gateway
- - omero.grid
- - omero.install
- - omero.metadatastore
- - omero.model
- - omero.plugins
- - omero.romio
- - omero.sys
- - omero.util
- - omero_ext
+ - omero
+ - omero.api
+ - omero
+ - omero.api
+ - omero.clients
+ - omero.cmd
+ - omero.constants
+ - omero.fs
+ - omero.gateway
+ - omero.grid
+ - omero.install
+ - omero.metadatastore
+ - omero.model
+ - omero.plugins
+ - omero.romio
+ - omero.sys
+ - omero.util
+ - omero_ext
about:
home: https://www.openmicroscopy.org/
@@ -67,4 +68,4 @@ about:
extra:
recipe-maintainers:
Nothing to add from my side. Since the command no longer reformats the YAML recipes if the last release version is identical, should this happen manually for conda-omero-py
, conda-raw2ometiff
and conda-bioformats2raw
or as part of the next release of each component?
The first step:
|
No more commits. |
This PR introduces a new command
scc bump-version-conda
to update theversion
andsha256
in themeta.yaml
files in the conda-* repositories. The command will be run in a GitHub action using a scheduled job in the conda-* repositories i.e. similar to a bot. If changes are detected, themeta
files are modified, committed and a PR is created. Note that the methodbump-version-conda
does not create the PR, this done via GitHub action in order to reduce the boiler plate required to create a PRWhat does this PR do?
git ls-remote
. rc, dv, m* are not consideredTo test this PR:
Test 1:
conda-omero-py
:scc bump-version-conda
Test 2:
conda-bftools
:scc bump-version-conda
https://downloads.openmicroscopy.org/bio-formats/6.7.0/artifacts/bftools.zip.sha1
Test 3:
scc bump-version-conda
Workflow in action jburel/conda-omero-py#4
cc @joshmoore @sbesson