Work in progress.
Personal project - working on in spare time only.
Unsupervised segmentation of video frames with the goal of detection bounds of a go-kart track
Building on technique described in Kim, Kanezaki & Tanaka 2020. Original implementation available here.
with Poetry and Poe the Poet:
poetry install
poetry shell # drop into poetry shell for ease of using poe
poe force-cuda11 # installs pytorch for cuda11
poe add-lightning # install pytorch-lightning and lightning-bolts without attempting to reinstall pytorch cpu
keeping an eye on python-poetry/poetry#2613 for making this smoother.
Add to PYTHONPATH
(for running scripts)
export PYTHONPATH=$(pwd)/src:$PYTHONPATH
- move TODOs to issues...
- lightningdataloader/dataset
- single image
- mp4
- review why crossentropy between cluster labels and features works
- evaluation code - want to check whether things improve
- should be able to eval loop through to get overall score
- hacky evaluate of demo.py (importlib, sys.argv)
- able to substitute MyNet for UNet
- use predictions of previous frame (or some window) to calc loss? should have similar predictions
- probably don't want to run these in the same batch though as too similar? want to shuffle...
- maybe run simultaneously?
- scribbles?
- validation during training?
- correct pixel metric? some overall % pixel assigment correct in image
intuition behind similarity loss fn (torch.nn.CrossEntropyLoss
between torch.argmax(output)
and output
):
encourages model to concretely choose a particular class for each pixel