NRTK EXPLORER

NRTK Explorer is a web application for exploring image datasets. It provides insights of a image dataset in COCO format and it evaluate image transformation and perturbation resilience of object recognition DL models. It is built using trame by the kitware team.

Features

• Explore image datasets in COCO format.
• Apply parametrized image degradation (such as blur) to the images.
• Benchmark dataset resilience with a differential PCA|UMAP analysis on the embeddings of the images and their transformations.
• Evaluate object detection DL models in both the source images and its transformations.
• When possible it will attempt to utilize the user GPU as much as possible to speedup its computations.

Installing

Install it from pypi:

pip install nrtk-explorer

Usage

Explore Hugging Face hosted dataset:

nrtk-explorer --dataset rafaelpadilla/coco2017

Compare inference results for Hugging Face hosted models:

nrtk-explorer --dataset cppe-5 --models qubvel-hf/detr-resnet-50-finetuned-10k-cppe5 ashaduzzaman/detr_finetuned_cppe5

2 COCO format datasets are available at: https://github.com/vicentebolea/nrtk_explorer_datasets/

git clone https://github.com/vicentebolea/nrtk_explorer_datasets.git
nrtk-explorer --dataset ./nrtk_explorer_datasets/coco-od-2017/mini_val2017.json ./nrtk_explorer_datasets/OIRDS_v1_0/oirds.json

CLI flags and options

• --dataset specify the path to a COCO dataset JSON file, a Hugging Face dataset repository name, or a directory loadable by the Dataset library. You can specify multiple datasets using a space as the separator. Example: nrtk-explorer --dataset ../foo-dir/coco.json cppe-5
• --download Cache Hugging Face Hub datasets locally instead of streaming them. When datasets are streamed, nrtk-explorer limits the number of loaded images.
• --models specify the Hugging Face Hub object detection repository name or a directory loadable by the Transformers library. Load multiple models using space as the separator.
  Example: nrtk-explorer --models hustvl/yolos-tiny facebook/detr-resnet-50
• -h|--help show the help for the command line options. nrtk-explorer inherits the trame command line options and flags.

Contribute to NRTK_EXPLORER

git clone https://github.com/Kitware/nrtk-explorer.git
cd nrtk-explorer
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -e '.[dev]'
pytest .

For more details on setting up a development environment see DEVELOPMENT docs.

Create release

1. Merge main to release with a merge commit.
2. Run "Create Release" workflow with workflow from release branch.
3. Merge release to main with a merge commit.