v0.5.0

v0.5.0

Updates / New Features

CI

• Updated notebooks CI workflow to include notebook data caching.

Documentation

• Added text discussing black box methods to introduction.rst.

• Added a section to introduction.rst that describes the links between saliency algorithms and implementations.

• Edited all text.

• Update top-level README.md file to have more useful content.

• Update misc. doc on local SonarQube scanning.

Examples

• Add example notebook for saliency on Atari deep RL agent, including updates
  on top of the original work to normalize saliency maps and conform to our
  API standards.

• Add example demonstrating saliency map generation for COCO formatted
  serialized detections.

• Updated examples to all use a common data sub-directory when downloading or
  saving generated data.

Implementations

• Add SquaredDifferenceScoring implementation of the GenerateClassifierConfidenceSaliency
  interface that uses squared difference.

• Add RandomGrid implementation of PerturbImage. This generates masks
  of randomly occluded cells with a given size in pixels.

Utilities

• Add gen_coco_sal function to compute saliency maps for detections in a
  kwcoco dataset, with accompanying cli script sal-on-coco-dets which
  does this on a COCO formatted json file and writes saliency maps to disk.

• Add multi-threaded functionality to occlude_image_batch utility.

Containerization

• Added Dockerfile and compose file that create base xaitk_saliency image.

Fixes

Build

• Fix incorrect specification of actually-optional papermill in relation to
  its intended inclusion in the example_deps extra.

• Update patch version of Pillow transitive dependency locked in the
  poetry.lock file to address CVE-2021-23437.

• Update the developer dependency and locked version of ipython to address a
  security vulnerability.

Implementations

• Fix incorrect cosine similarity computation and mask inversion in implementation of
  DRISEScoring detector saliency.

Examples

• Updated example Jupyter notebooks with more consistent dependency checks and
  also fixed minor header formatting issues.

Tests

• Fix deprecation warnings around the use of numpy.random.random_integers.

Utilities

• Fix xaitk_saliency.utils.detection.format_detection to not upcast the
  data type output when objectness is None.

• Fix xaitk_saliency.utils.masking.weight_regions_by_scalar to not upcast
  the data type output when inv_masks is True.

• Update xaitk_saliency.utils.masking.weight_regions_by_scalar to not use
  fully vectorized operation which significantly improves efficiency.

v0.4.0

This minor release expands our documentation and examples pool.
We additionally provide an the D-RISE implementation for the
GenerateDetectorProposalSaliency interface.

Updates / New Features

CI

• Added workflow for test running some example notebooks.

• Update CodeCov action used to version 2.

Documentation

• Added text discussing white box methods to introduction.rst.

• Added some review process documentation.

• Add initial FAQ documentation file.

• Add background material for saliency maps to introduction.rst.

• Added API docs section, which includes descriptions of all interfaces.

• Added content to the CONTRIBUTING.md file on:

  • including notes here for added updates, features and fixes

  • Jupyter notebook CI workflow inclusion

• Add implementations section.

• Update example Jupyter notebooks to work with Google Colab.

Examples

• Add example notebook using classifier-based interfaces and implementations
  with scikit-learn on the MNIST dataset.

• Edited notebook examples.

Implementations

• Add DRISEScoring implementation of the GenerateDetectorProposalSaliency
  interface using detection output and associated occlusion masks.

• Add SlidingRadial implementation of the PerturbImage interface that
  slides radial occlusion areas across an image.

Tests

• Removed use of unittest.TestCase as it is not utilized directly in any way
  that PyTest does not provide.

Utilities

• Add type annotation, documentation and unit-tests for using image matrices as
  the fill option instead of just a solid color.

• Add format_detection helper function to form the input for
  GenerateDetectorProposalSaliency from separated components.

• Add example notebook showing the use of SlidingRadial perturbation and
  the use of occlude_image_batch with blurred-image alpha blending.

Fixes

Implementations

• Fixed ValueError messages raised in the SimilarityScoring
  implementation. Added unittests to check the raising and message content.

v0.3.1

v0.3.1

This patch release focuses on removing some listed dependencies that are in
reality unused.

Fixes

CI

• Enable the running of CI for branches and PRs targeting release branches.

Package

• Remove required dependencies that are in fact not used.

v0.3.0

v0.3.0

This minor release provides updates to example notebooks, interface naming
updating for consistency, as well as added a "high level" interface for visual
saliency map generation from black-box classifiers.

Updates / New Features

Documentation

• Add "SuperPixelSaliency" notebook to demonstrate use of arbitrary perturbation
  masks based on superpixels for saliency map generation.

• Add "VIAME_OcclusionSaliency" notebook demonstrating integration with VIAME
  toolkit based on saliency map generation for a fish classification task.

• Add "covid_classification" notebook demonstrating integration with MONAI
  based on saliency map generation for a COVID-19 X-ray classification task.

• Updated notebook demonstration for SimilarityScoring usage to better track
  the notebook structures across the repo.

• Introduce class naming philosophy in the CONTRIBUTING.md file.

• Updated notebooks to make use of GenerateImageClassifierBlackboxSaliency
  interface/implementations where appropriate.

• Updated wording of the SuperPixelSaliency.ipynb notebook as to cover a
  use-case where the GenerateImageClassifierBlackboxSaliency interface API
  is not appropriate, i.e. already have masks computed.

• Added support for doc building on Windows by adding a platform check so that "make.bat" is called for and not "make html".

Interfaces

• Update PerturbImage to only output perturbation masks, dropping physical
  image perturbation output. Since this output generation was the same across
  all known implementations, that part has been split out into a utility
  function.

• Added support for positive and negative saliency values as output by the
  saliency map generation interfaces.

• Updated PerturbImage interface to take in numpy.ndarray as the image
  data structure.

• Added new, higher-level GenerateImageClassifierBlackboxSaliency interface
  for transforming an image and black-box classifier into visual saliency maps.

• Renamed ImageClassifierSaliencyMapGenerator interface to be
  GenerateClassifierConfidenceSaliency.

• Renamed ImageSimilaritySaliencyMapGenerator interface to be
  GenerateDescriptorSimilaritySaliency.

• Renamed ImageDetectionSaliencyMapGenerator interface to be
  GenerateDetectorProposalSaliency.

Implementations

• Add RISEScoring implementation, with the ability to also compute a
  de-biased form of RISE with an optional input parameter.

• Add RISEStack implementation of the GenerateImageClassifierBlackboxSaliency
  interface as a simple way to invoke the combination of RISE component
  algorithms.

• Add SlidingWindowStack implementation of the GenerateImageClassifierBlackboxSaliency
  interface as a simple way to invoke the combination of the Sliding Window
  perturbation method and the occlusion-based based scoring method.

Misc.

• Update locked dependency versions to latest defined by abstract requirements.

Utils

• Masking

  • Added utility functions for occluded image generation that was previously
    duplicated across PerturbImage implementations. Added both batch and
    streaming versions of this utility.

Fixes

Documentation

• Fixed misspelled "miscellaneous" file.

Implementations

• Fix saliency map normalization in both OcclusionScoring as well as
  SimilarityScoring to disallow cross-class pollution in the norm.

Misc.

• Fixed up module naming inconsistencies.

v0.2.0

v0.2.0

This initial alpha release introduces the xaitk-saliency toolkit for computing
visual saliency heat-maps for input imagery over based on black-box operations.

Updates / New Features

CI

• Added properties file for SonarQube scans.

• Add CodeCov integration.

Documentation

• Updated the "Occlusion Saliency" notebook to flow smoother and include
  un-commentable RISE perturbation algorithm option. The narrative has
  been more explicitly tuned to follow an "application" narrative.

• Add miscellaneous documentation on how to run a local SonarQube scan and
  experimental documentation on setting up scanning as a CI workflow job.

Interfaces

• Add new interfaces in accordance to the v0.2 API draft.

  • Added to doc-strings to expand on detail around saliency heatmap return
    value range and meaning.

  • Updated image perturbation interface to function in a streaming iterator
    fashion instead of in-bulk as a means of performance optimization as well
    as to allow it to function on larger image sizes and larger perturbation
    quantities at the same time.

• Removed old interface classes "ImageSaliencyMapGenerator", "SaliencyBlackbox"
  and "ImageSaliencyAugmenter".

Implementations

• Add new occlusion based classifier scoring in accordance to the v0.2 API draft for ImageClassifierSaliencyMapGenerator.

• Add new RISE based perturbation algorithm in accordance to the v0.2 API draft for PerturbImage

• Add new similarity based scoring algorithm in accordance to the v0.2 API draft for ImageSimilaritySaliencyMapGenerator

• Remove old "stub" implementations in transitioning to the new API draft

  • Removed "LogitImageSaliencyAugmenter" implementation class.

  • Removed "LogitImageSaliencyMapGenerator" implementation class.

  • Removed old RISE implementation classes.

  • Removed old SBSM implementation classes.

Fixes

• Update Read the Docs documentation link in README

• Address various "code smells" as reported by SonarQube/SonarCloud.