Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial release #28

Merged
merged 138 commits into from
Dec 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
0429ff4
initial commit of initial release branch
Xu-Justin Oct 9, 2022
b7abc06
add test
Xu-Justin Oct 10, 2022
cff1435
rename tests folder, add test, add coveragerc
Xu-Justin Oct 10, 2022
5125555
disable warning for couldn't parse file
Xu-Justin Oct 10, 2022
4cd943b
add run_test.sh
Xu-Justin Oct 10, 2022
55e8318
update run_test.sh
Xu-Justin Oct 10, 2022
48590c4
requirements.txt
Xu-Justin Oct 12, 2022
1d47d18
Create python-app.yml
Xu-Justin Oct 12, 2022
b4e68b2
Merge branch 'initial_release' of github.com:Xu-Justin/patchmentation…
Xu-Justin Oct 12, 2022
bc9c91d
add requirements
Xu-Justin Oct 12, 2022
f9f2457
this build should failed
Xu-Justin Oct 12, 2022
b4f8030
this build should failed
Xu-Justin Oct 12, 2022
db79f04
this tests have warnings
Xu-Justin Oct 12, 2022
2425317
cleaning up
Xu-Justin Oct 12, 2022
2933442
Merge pull request #2 from Xu-Justin/feature/automation-test
Xu-Justin Oct 12, 2022
2e6110c
add coverage report
Xu-Justin Oct 12, 2022
b6254ee
add image credits
Xu-Justin Oct 12, 2022
616fb63
add functional to convert BGR2RGB
Xu-Justin Oct 13, 2022
f71ef20
Update README.md
Xu-Justin Oct 13, 2022
c613fcc
add visualization with streamlit
Xu-Justin Oct 13, 2022
b7db34b
add streamlit extra component
Xu-Justin Oct 15, 2022
7c3633e
add draw patch
Xu-Justin Oct 15, 2022
d724330
Merge pull request #4 from Xu-Justin/feature/dev-streamlit
Xu-Justin Oct 15, 2022
5447479
restructure
Xu-Justin Oct 15, 2022
ca3a806
Merge pull request #5 from Xu-Justin/restructure
Xu-Justin Oct 15, 2022
cbea10b
add docker
Xu-Justin Oct 16, 2022
996cc75
update docker ci branch
Xu-Justin Oct 16, 2022
b17d001
update docker ci context
Xu-Justin Oct 16, 2022
84e79ae
Merge pull request #7 from Xu-Justin/feature/docker-env
Xu-Justin Oct 16, 2022
c145223
restructure tests
Xu-Justin Oct 18, 2022
a21d224
add PASCAL VOC classes.txt
Xu-Justin Oct 18, 2022
2c48068
add some hint
Xu-Justin Oct 18, 2022
998db00
fix test utils sys path
Xu-Justin Oct 18, 2022
ad83a7f
add comparator, filter, transform
Xu-Justin Oct 18, 2022
392976f
cleaning up
Xu-Justin Oct 20, 2022
dd28d91
add check grayscale
Xu-Justin Oct 20, 2022
6799eeb
add filter and transform
Xu-Justin Oct 20, 2022
3529f97
add tests filter, transform, comparator
Xu-Justin Oct 20, 2022
53684e8
rename test_utils folder
Xu-Justin Oct 20, 2022
6a53110
add dummy filter and transform to streamlit, will be fix at another pr
Xu-Justin Oct 20, 2022
f141f4a
Merge pull request #8 from Xu-Justin/feature/patch-transformation
Xu-Justin Oct 20, 2022
7cb57e5
support image array with classes filtering
Xu-Justin Oct 22, 2022
2cf78c4
add methods to patch
Xu-Justin Oct 22, 2022
1146cf6
add converter
Xu-Justin Oct 22, 2022
632a1e7
fix patch augmentation and modify filter
Xu-Justin Oct 22, 2022
9ab86f7
add streamlit
Xu-Justin Oct 22, 2022
894fe56
Merge pull request #10 from Xu-Justin/feature/streamlit
Xu-Justin Oct 22, 2022
8b87904
Update validator.py
Xu-Justin Oct 25, 2022
0ac0509
add check_image_bbox to validator dataset
Xu-Justin Oct 25, 2022
560d72e
Update validator.py
Xu-Justin Oct 25, 2022
b362b44
fix validator
Xu-Justin Oct 25, 2022
07a1da6
add kwargs to validate method
Xu-Justin Oct 25, 2022
d32d95a
Merge pull request #11 from Xu-Justin/fix/validator
Xu-Justin Oct 25, 2022
6dfce11
Coco_Added
Jonathantho1 Oct 27, 2022
aad5122
Update test_loader.py
Xu-Justin Oct 27, 2022
5ea7faf
add coco loader
Xu-Justin Oct 27, 2022
f809836
fix loader coco
Xu-Justin Oct 27, 2022
5c27194
fix loader coco
Xu-Justin Oct 27, 2022
09ecdba
fix loader coco
Xu-Justin Oct 27, 2022
540a7b8
fix loader coco
Xu-Justin Oct 27, 2022
d90f19c
reformat coco annotations
Xu-Justin Oct 27, 2022
4db51ca
remove script.py
Jonathantho1 Nov 2, 2022
7898281
Update loader.py
Jonathantho1 Nov 2, 2022
6ba493b
Merge pull request #14 from Xu-Justin/feature/loader_coco
Xu-Justin Nov 2, 2022
05c6d00
add image mask to support image transparency
Xu-Justin Nov 3, 2022
2fbbbcc
fix validate pixel
Xu-Justin Nov 3, 2022
df8ffc2
Merge pull request #15 from Xu-Justin/feature/mask
Xu-Justin Nov 3, 2022
5ab1a0b
add converted patch2image
Xu-Justin Nov 4, 2022
27dbc5e
create mask with true value
Xu-Justin Nov 4, 2022
42876c7
add additional color converter
Xu-Justin Nov 4, 2022
a46072c
change transform input from np.ndarray to Image
Xu-Justin Nov 4, 2022
7a559f9
Merge pull request #16 from Xu-Justin/feature/refactor
Xu-Justin Nov 4, 2022
981798e
add get_mask
Xu-Justin Nov 5, 2022
5307802
add gaussian kernel 2d
Xu-Justin Nov 5, 2022
9bd0055
add scipy
Xu-Justin Nov 5, 2022
c5e34b9
add soft edge
Xu-Justin Nov 5, 2022
cd5a6c5
add hard edge
Xu-Justin Nov 5, 2022
1a13353
add overlay image
Xu-Justin Nov 7, 2022
21c9e8d
rework visibility suppression as visibility thresholding
Xu-Justin Nov 7, 2022
6277764
add refresh button
Xu-Justin Nov 7, 2022
41d1f27
fix softedge and hardedge
Xu-Justin Nov 7, 2022
8e1fbdf
change visibility threshold comparator to gte
Xu-Justin Nov 7, 2022
207eb08
rework patch augmentation
Xu-Justin Nov 7, 2022
51185e8
remove place_image_array and visibility_suppression
Xu-Justin Nov 7, 2022
d76ab04
Merge pull request #17 from Xu-Justin/feature/edge-blending
Xu-Justin Nov 7, 2022
5da9a37
add negative patching
Xu-Justin Nov 7, 2022
6cf6441
add filter by aspect ratio
Xu-Justin Nov 7, 2022
29b67d7
fix negative patch produce patch with zero area
Xu-Justin Nov 7, 2022
e96cb1d
change default value for filter by aspect ratio
Xu-Justin Nov 7, 2022
061d783
Merge pull request #18 from Xu-Justin/feature/negative-patching
Xu-Justin Nov 7, 2022
845ea1d
add background patching
Xu-Justin Nov 8, 2022
5a18bfc
Merge pull request #19 from Xu-Justin/feature/background-patching
Xu-Justin Nov 8, 2022
fbd9b30
add weighted random 2d
Xu-Justin Nov 9, 2022
c556ec1
add patch distribution
Xu-Justin Nov 9, 2022
722a93e
Merge pull request #20 from Xu-Justin/feature/patch-distribution
Xu-Justin Nov 9, 2022
7a9a5b5
fix negative patch
Xu-Justin Nov 9, 2022
5887b67
add test
Xu-Justin Nov 9, 2022
c1f70e6
Merge pull request #21 from Xu-Justin/feature/fix-negative-patch
Xu-Justin Nov 9, 2022
10b44b8
refactor requirements
Xu-Justin Nov 17, 2022
5e03ada
add deployment
Xu-Justin Nov 17, 2022
eb40851
add workflow
Xu-Justin Nov 17, 2022
46192ed
add workflow
Xu-Justin Nov 17, 2022
8b6677a
add workflow
Xu-Justin Nov 17, 2022
058f48c
add workflow
Xu-Justin Nov 17, 2022
bd26865
add workflow
Xu-Justin Nov 17, 2022
95dbf7c
add workflow
Xu-Justin Nov 17, 2022
82c5cb7
add minimal README
Xu-Justin Nov 17, 2022
53c9c71
add deploy script
Xu-Justin Nov 17, 2022
67f8988
v0.0.2
Xu-Justin Nov 17, 2022
7668dc1
refactor requirements
Xu-Justin Nov 17, 2022
873459a
update Dockerfile
Xu-Justin Nov 17, 2022
3f00823
update dockerfile
Xu-Justin Nov 17, 2022
4ecec5a
Merge pull request #22 from Xu-Justin/feature/test-pypi-deployment
Xu-Justin Nov 17, 2022
b47ec73
add pascal voc loader
Xu-Justin Nov 24, 2022
09ea0c2
Merge pull request #23 from Xu-Justin/feature/pascal-voc-loader
Xu-Justin Nov 24, 2022
e8b49f0
add pascal voc 2007 dataset
Xu-Justin Nov 26, 2022
c4773c2
Merge pull request #24 from Xu-Justin/feature/pascal-voc-2007
Xu-Justin Nov 26, 2022
efd448f
refactor collections
Xu-Justin Dec 4, 2022
f250c90
add OverflowBBox
Xu-Justin Dec 4, 2022
863d97a
fix test_dataset_generate_classes
Xu-Justin Dec 4, 2022
a7689e8
refactor collections
Xu-Justin Dec 4, 2022
faa1392
remove validator
Xu-Justin Dec 4, 2022
8175a02
add imread caching
Xu-Justin Dec 4, 2022
a048047
fix image_patch image_array
Xu-Justin Dec 4, 2022
953d26d
add dataset caching
Xu-Justin Dec 4, 2022
9d5cf35
remove duplicate load_yolo_dataset
Xu-Justin Dec 5, 2022
e9eb0a6
add test
Xu-Justin Dec 5, 2022
3bbcddd
update .coveragerc
Xu-Justin Dec 5, 2022
514cb91
fix test not throw error
Xu-Justin Dec 5, 2022
1a7ce19
Merge pull request #25 from Xu-Justin/feature/refactor
Xu-Justin Dec 5, 2022
0132e13
add max_n_patches
Xu-Justin Dec 5, 2022
196ed22
optimize hard edge mask
Xu-Justin Dec 5, 2022
0b73bff
add softedge test
Xu-Justin Dec 5, 2022
285062d
max_n_patches
Xu-Justin Dec 5, 2022
6530c2c
Merge pull request #26 from Xu-Justin/feature/max_n_patches
Xu-Justin Dec 5, 2022
5e69a29
bump version to 0.1.0
Xu-Justin Dec 5, 2022
31ea380
remove cache after deploy
Xu-Justin Dec 5, 2022
eb510a4
Merge pull request #27 from Xu-Justin/release/0.1.0
Xu-Justin Dec 5, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[run]
omit =
*/site-packages/*
*/dist-packages/*
*/distutils/*
tests/*
patchmentation/dataset.py

disable_warnings = couldnt-parse
32 changes: 32 additions & 0 deletions .github/workflows/docker-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: docker ci

on:
push:
branches:
- '**docker**'
pull_request:
branches:
- '**'

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v3
with:
push: true
tags: jstnxu/patchmentation:env
42 changes: 42 additions & 0 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Python application

on:
push:
branches:
- '**'
pull_request:
branches:
- '**'

permissions:
contents: read

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
requirements=requirements/requirements-main.txt
if [ -f $requirements ]; then pip install -r $requirements; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Run test
run: |
pytest -v
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
*__pycache__*
.pytest_cache/*

.coverage
htmlcov/*

# development tools
dev.ipynb
.ipynb_checkpoints

# deployment
build/*
dist/*
*.egg-info/
14 changes: 14 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
FROM python:3.10.8-slim-bullseye

RUN apt-get update -qq && \
apt-get install -y zip unzip htop screen libgl1-mesa-glx libglib2.0-0 libsm6 libxrender1 libxext6 && \
rm -rf /var/cache/apk/*

COPY requirements/* /requirements/
RUN pip --no-cache-dir install -r /requirements/requirements-main.txt
RUN pip --no-cache-dir install -r /requirements/requirements-test.txt
RUN pip --no-cache-dir install -r /requirements/requirements-dev.txt
RUN pip --no-cache-dir install -r /requirements/requirements-streamlit.txt

WORKDIR /workspace
CMD ["bash"]
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
# patchmentation
# patchmentation

A python library to perform patch augmentation.

## Installation

```bash
pip install patchmentation
```

THIS README will be updated soon.
2 changes: 2 additions & 0 deletions Streamlit/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
from . import section
from . import page
30 changes: 30 additions & 0 deletions Streamlit/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import page

import streamlit as st
from streamlit_option_menu import option_menu

def main():
MENU_DATASET = 'Dataset'
MENU_PATCHMENTATION = 'Patchmentation'

with st.sidebar:
menu = option_menu(
'Main Menu',
[
MENU_DATASET,
MENU_PATCHMENTATION
],
default_index=0
)

if menu == MENU_DATASET:
page.dataset()

if menu == MENU_PATCHMENTATION:
page.patchmentation()

if __name__ == '__main__':
main()
36 changes: 36 additions & 0 deletions Streamlit/page.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import section
from patchmentation import patch_augmentation

import streamlit as st
import random

def dataset(key: str = 'page-dataset'):
dataset = section.dataset(key=f'{key}-dataset')
section.display_dataset(dataset, key=f'{key}-display_dataset')

def patchmentation(key: str = 'page-patchmentation'):
dataset = section.dataset(key=f'{key}-dataset')
shuffle = section.input_shuffle(key=f'{key}-shuffle')
negative_patches = section.negative_patch(dataset, key=f'{key}-negative-patches')
background_image = section.background_image(key=f'{key}-background_image')
actions = section.input_actions(key=f'{key}-input_actions')
conf = section.patchmentation_configuration(key=f'{key}-patchmentation_conf')

if dataset is None:
st.error(f'ERROR: Invalid dataset')
return

patches = negative_patches
for image_patch in dataset.image_patches:
for patch in image_patch.patches:
patches.append(patch)

if shuffle:
random.shuffle(patches)

result_image_patch = patch_augmentation(patches, background_image, conf.get('visibility_threshold', 0.5), actions, max_n_patches=conf.get('max_n_patches', 100))
section.display_image_patch(result_image_patch, key=f'{key}-display_result_image_patch')
section.refresh_button(key=f'{key}-refresh')
Loading