Move hera to submodule #371

reds-heig · 2020-03-18T10:12:19Z

Reference issues/PRs

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Description

Previously hera submodule (bottleneck and wasserstein implementations) was not a submodule but sources were hard-coded.
This PR removes old hera sources with new up-to-date repository maintained from the author itself.

Screenshots (if appropriate)

Any other comments?

Checklist

I have read the guidelines for contributing.
My code follows the code style of this project. I used flake8 to check my Python changes.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed. I used pytest to check this on Python tests.

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

CLAassistant · 2020-03-18T10:12:38Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

MonkeyBreaker · 2020-03-18T11:18:06Z

After looking into the issue of why pipelines are currently failing, I stumbled onto a similar issue reported recently to azure-pipelines.

The problem is that the copy step fails when encountering some symlinks, unfortunately, introducing hera repository, makes this issue appear, for the following files:

gtda/externals/hera/matching/tests/test_bifiltration_1.txt
gtda/externals/hera/matching/tests/test_bifiltration_full_triangle_rene.txt

I'll contact the maintainer of hera to fix this issue, but at the meantime we have 2 solutions:

wait till the maintainer fixes this issue
As recommended in issue , using a simple bash script to copy the file instead

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

In Wasserstein, by default epsilon_common_ratio is equal 5, and it can be equal 0, which our bindings did not allow Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Add method to retrieve what wasserstein consider to be infinity as an input Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

MonkeyBreaker · 2020-03-19T06:58:16Z

After looking into the issue of why pipelines are currently failing, I stumbled onto a similar issue reported recently to azure-pipelines.

The problem is that the copy step fails when encountering some symlinks, unfortunately, introducing hera repository, makes this issue appear, for the following files:
* `gtda/externals/hera/matching/tests/test_bifiltration_1.txt`

* `gtda/externals/hera/matching/tests/test_bifiltration_full_triangle_rene.txt `
I'll contact the maintainer of hera to fix this issue, but at the meantime we have 2 solutions:
* wait till the maintainer fixes this issue

* As recommended in [issue ](https://github.com/Microsoft/azure-pipelines-tasks/issues/9046), using a simple bash script to copy the file instead

This has now been resolved

gtda/externals/bindings/wasserstein_bindings.cpp

gtda/externals/python/tests/test_bottleneck.py

gtda/externals/python/tests/test_wasserstein.py

ulupo · 2020-03-19T07:34:17Z

@MonkeyBreaker thanks for this, and for the new tests!

MonkeyBreaker · 2020-03-19T07:36:10Z

I'll perform the changes this afternoon, I'm currently busy on another topic, but it'll be resolved today.

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

@wreise

* Add temporary patch to azure-pipelines.yml (#369) Get around Azure DevOps issues by following a suggestion by Sean Law: https://twitter.com/seanmylaw/status/1239332444865773568 * Remove metatransformers (#368) Signed-off-by: Guillaume Tauzin <guillaumetauzin.ut@gmail.com> * Extend plotting functionalities to more transformers and other modules (#362) * start plotting for images * Add the image plotting file * Add an imagePlotMixin subclassing the ImagePlot for theimage submodule, add persistenceImage plots * Add plot_persistence_image to the package * Add plotting functionalities to filtrations in the image module and forbetticurves * Preprocessing covered by plots * Add basic plotting functions for different vectorisation methods * Incorporate plotting for diagram representations and images/preprocessing * Change the docstring for ImagePlotterMixin and add fail on 3d images * Correct the samplingsin features, and titles * Small wording fix * Change dosctrings for plot in diagrams/features * Fix typo in variable name * Change plotting of heatkernel to the same as persistenceimage, use samplings * Cast images to floats, so as they can be always plotted * Add a Homology and PointCloud Mixin * Use the new PlotMixins * Add todo in Forgetdimensions, in relation to plotting * Add plotting to point_cloudsrescaling, correctimports * Wording changes * Correct n_values to n_bins and add 's' to n_homologydimensionS * Correct spelling mistakes * Change the name of ...PlotterMixin to ...Plotter * Make SlidingWindowEmbedding a PointCloudPlotter * Fix typoin import * Fix linting * Correct the way the distance matrices are displayed, and add the SlidingWindow.plot method * Change behaviour of transform_plot to return the 0-th sample * Docstring fixes * Create diagrams/representations.py, and migrate amplitude to features.py This change reflects the new documentation structure and the difference now drawn there between feature creation for diagrams (scalars) and vector representations/vectorizations. These changes are mostly invisible to the user. * Remove unused identity function * Rename HomologyPlotter as DiagramPlotter * Add homology_dimensions kwarg to DiagramPlotter, revise docstring * Small docstring fixes * Try adding triggers in the yaml * Remove excessive blank line * Revert "Change behaviour of transform_plot to return the 0-th sample" This reverts commit 2fa0209. * Fix docstring for PlotterMixin * Further docstring refinements in `PlotterMixin` * Remove `DiagramPlotter`, differentiate between implementations of `plot` Also make plot methods static in classes in `gtda.homology` and in ForgetDimensino * Correctly import PlotterMixin * Correct the plotting in the lorentz notebook * Remove PointCloudPlotter and refine plot methods for point clouds * Thorough refactoring of plotting API - Eliminate some top-level functions for diagram representations, leave only betti_curves and betti_surfaces - Remove ImagePlotter mixin and plotting/_plot.py, create top-level plot_heatmap function - Rename plotting/diagrams.py to plotting/diagrams_representations.py - Rename plotting/image.py to plotting/images.py - Rename plotting/homology.py to plotting/persistence_digrams.py - Fix problem with Binarizer coming from now-removed normalize parameter - Refactor notebooks to use plotting API - MIscellaneous small changes elsewhere The philosophy behind the file changes in gtda/plotting is that file names now reflect data structures and do not try to follow the names of subpackages like diagrams, image, etc. * Grammar * Fix default parameter for homology_dimension_ix in HeatKernel * Change plot_diagrams to allow ForgetDimension to work * Allow _subdiagrams to better handle infs of different kind * Add inline comment for future selves * Write check_diagrams docstring and make it convert to float, revert previous changes to _subdiagrams * Pass copy=False to astype in check_diagrams Co-authored-by: Guillaume Tauzin <guillaume.tauzin@epfl.ch> Co-authored-by: Guillaume Tauzin <guillaumetauzin.ut@gmail.com> Co-authored-by: Umberto Lupo <46537483+ulupo@users.noreply.github.com> Co-authored-by: Umberto <u.lupo@l2f.ch> * Glossary entry: distance, inner products & kernels (#370) * Update P landscapes * Add distances, inner products and kernels glossary entry * Update after W's comments Signed-off-by: ammedmar <ammedmar@gmail.com> Co-authored-by: ammedmar <anibal@medina-mardones.com> * Add reference for images submodules (#372) * Add reference for images submodules Signed-off-by: Guillaume Tauzin <guillaumetauzin.ut@gmail.com> * Move hera to submodule (#371) * delete hera repository * Add hera as submodule * Remove unnecessary branch for submodule * Add parameter max_bids_per_round when computing wasserstein * Fix an inconsistency with wasserstein implementation In Wasserstein, by default epsilon_common_ratio is equal 5, and it can be equal 0, which our bindings did not allow * Add test for python bindings of wasserstein Add method to retrieve what wasserstein consider to be infinity as an input * Add test for python bindings of bottleneck * Add fixes done by author after I contact him Signed-off-by: julian <julian.burellaperez@heig-vd.ch> * Miscellaneous refinements to GitHub pages (#363) * Remove code fetching logo as it is now stored in doc/images * Fix logo in examples directory * Partial rewriting of contributing page * Remove instructions for building documentation * Simplify path in Makefile * Remove documentation job in azure-pipelines.yml * Add @wreise to authors in doc/conf.py * Remove paper citation from FAQ * Rename Glossary to Theory Glossary * Linting in images subpackage * Add PlotterMixin to base.rst * Update FAQ * Improve README with pointers to GH pages * Implement @wreise's review suggestions * Fix links to svgs * Remove remaining mentions of plotting.py * Rename basic to tutorials and advanced to examples * Fixes in jupyter notebooks in examples folder * Add plotting in docs * Fixes in glossary * Correct the VBox display * Add TODO instead of empty reference in Cech docstring * Fix references in docstrings * Reintroduce scrollZoom in mapper_quickstart * Improve landing page and move Lewis' blog post to notebooks/tutorials.rst Co-authored-by: Wojciech Reise <reisewojtus@gmail.com> * Pcs different sizes (#318) * Add a check_point_clouds fct in utils/validation, and apply it throughout classes in ``homology/simplicial.py`` * Adapt and add tests * Make the output of ImageToPointCloud variable in size, and adjust the tests (modify the expected values) * Add mapper and utils to global __init__ * Revise use of check_array throughout * Linting and code clarity improvements throughout * Fix some incorrect links to GH pages * Revert to trivial slicing in transform_plot method of PlotterMixin, to cover case of list input Co-authored-by: Umberto <u.lupo@l2f.ch> * Release of v0.2.0 (#375) * Bump version to 0.2.0, add release notes * Improve LaTeX rendering in mapper_quickstart.ipynb * Add @rth as a contributor and code author for v0.2.0 (#376) * Add @rth to code authors and list of contributors for v0.2.0 * [WIP] Add twine check to CI, attempt fix in README (#379) * Add twine check to azure-pipelines.yml * Fix formatting and duplicate issues in README.rst * Tests for plotting functions and methods, other increases in test coverage (#384) * Remove images subpackage from code omitted from coverage * Add tests for Filtering class * Add tests in diagrams subpackage, linting - Rename test_features -> test_features_representations - Create parametrized dummy tests for fit_transform_plot method of transformers in representations.py - Improve structure and range of tests in test_preprocessing in a similar way - Create dummy tests of shape for BettiCurve and PersistenceLandscape - Linting and code style changes elsewhere * Add fit_transform_plot test for GraphGeodesicDistance * Add fit_transform_plot tests in gtda.homology * Fix typo in voids_on_the_plane notebook * Add fit_transform_plot tests for images subpackage * Add fit_transform_plot tests for point_clouds subpackage * Add test for SlidingWindow.plot * Linting * Exclude gtda.utils._doc and gtda.utils.intervals from coveragerc * Implement @wreise's suggestions - Rename some test functions - Implement a solution to the automatic tab opening problem, based on setting the default value of plotly.io.renderers in test files * Upload coverage report only from Azure macOS job * Exclude test folders from coverage reports * Vectorization2 (#378) * Update P landscapes * Add distances, inner products and kernels glossary entry Signed-off-by: ammedmar <anibal@medina-mardones.com> * Fix azure on windows (#388) * Update ccache * Update variable used to indicate boost headers location Official answer from boost devs: https://discourse.cmake.org/t/findboost-difference-between-boost-include-dir-and-boost-include-dirs/855 * Add install boost step in Azure CI, which downloads boost Signed-off-by: julian <julian.burellaperez@heig-vd.ch> * Allow more general input to ripser when metric='precomputed', improve/refactor check_point_clouds and add tests (#386) * Remove boolean mask on infinite values in VietorisRipsPersistence * Refactoring of check_point_clouds - Rename distance_matrix kwarg to distance_matrices - Add checks for square input when distance_matrices is set to True - More comprehensive/friendly error messages and warnings * Increase generality of ripser function when metric == 'precomputed' Sklearn's pairwise_distances is no longer called when metric='precomputed'. This is because pairwise_distances checks for positivity and finite input in that case, but this is unnecessarily restrictive for ripser. pairwise_distances also checks for square input so this is no longer checked in the case metric='precomputed'. But the ripser function is not meant for use in isolation. It is meant to be called by a VietorisRipsPersistence instance, and input validation can be offloaded to that class. In particular, 9bab872 introduces checks for square input. * Add tests for check_point_clouds * Refresh docstrings for VietorisRipsPersistence - Mention "adjacency matrices of weighted undirected graphs" as well as distance matrices - Mention compatibility with filtration in docstrings for fit and transform * Add citing entry to README.rst and doc/faq.rst (#389) * Add citation section in README * Add citation info to FAQ page in docs Signed-off-by: Guillaume Tauzin <guillaumetauzin.ut@gmail.com> Co-authored-by: Umberto Lupo <46537483+ulupo@users.noreply.github.com> * Fix markdown subsection formatting (#390) * Fix citing giotto-tda section * Fix Important link section * Prepare release 0.2.1 (#391) * Make the docs and push to a test branch in gtda-docs * Fix indentation Co-authored-by: Umberto Lupo <46537483+ulupo@users.noreply.github.com> Co-authored-by: Guillaume Tauzin <guillaumetauzin.ut@gmail.com> Co-authored-by: Guillaume Tauzin <guillaume.tauzin@epfl.ch> Co-authored-by: Umberto <u.lupo@l2f.ch> Co-authored-by: Anibal M. Medina-Mardones <ammedmar@gmail.com> Co-authored-by: ammedmar <anibal@medina-mardones.com> Co-authored-by: REDS institute <reds-heig@users.noreply.github.com>

julian added 7 commits March 18, 2020 10:50

delete hera repository

cfcb3e3

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Add hera as submodule

8360cbe

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Fix compilation issues with new hera repository

92bc458

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Update ccaches

b35ee09

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Remove unnecessary branch for submodule

05da94a

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Fix wrong target for bottleneck when including headers

fd0826d

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Update include path from relatives to system

ad48e36

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

reds-heig requested review from MonkeyBreaker and ulupo March 18, 2020 10:12

Fix issue when compiling on Windows due to ssize_t is POSIX only

1b17ab6

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

julian added 5 commits March 18, 2020 19:56

Add parameter max_bids_per_round when computing wasserstein

1a591d8

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Fix an inconsistency with wasserstein implementation

6ced76e

In Wasserstein, by default epsilon_common_ratio is equal 5, and it can be equal 0, which our bindings did not allow Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Add test for python bindings of wasserstein

4088dad

Add method to retrieve what wasserstein consider to be infinity as an input Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Add test for python bindings of bottleneck

010179a

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

Add fixes done by author after I contact him

d0cce91

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

ulupo requested changes Mar 19, 2020

View reviewed changes

MonkeyBreaker approved these changes Mar 19, 2020

View reviewed changes

Clean up after code review

b47cbb8

Signed-off-by: julian <julian.burellaperez@heig-vd.ch>

ulupo approved these changes Mar 20, 2020

View reviewed changes

ulupo merged commit 24bd879 into giotto-ai:master Mar 20, 2020

ulupo mentioned this pull request Mar 22, 2020

Adds WitnessPersistence #155

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move hera to submodule #371

Move hera to submodule #371

reds-heig commented Mar 18, 2020 •

edited by MonkeyBreaker

Loading

CLAassistant commented Mar 18, 2020

MonkeyBreaker commented Mar 18, 2020

MonkeyBreaker commented Mar 19, 2020

ulupo commented Mar 19, 2020

MonkeyBreaker commented Mar 19, 2020

Move hera to submodule #371

Move hera to submodule #371

Conversation

reds-heig commented Mar 18, 2020 • edited by MonkeyBreaker Loading

CLAassistant commented Mar 18, 2020

MonkeyBreaker commented Mar 18, 2020

MonkeyBreaker commented Mar 19, 2020

ulupo commented Mar 19, 2020

MonkeyBreaker commented Mar 19, 2020

reds-heig commented Mar 18, 2020 •

edited by MonkeyBreaker

Loading