-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single mantis directory #1061
Single mantis directory #1061
Conversation
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a small comment - I think I confused you/I was also confused when I commented on the issue about the naming thing.
I don't think mask_prefix
is actually needed. The masks are generated using generate_and_save_pixel_cluster_masks
, which doesn't even have an option for a prefix, only a suffix. So the way the code is, there would be no situation where the masks have a prefix. I was originally thinking we could add the prefix when the masks are copied over into the mantis directory, however, after looking at this code, I don't think it's actually needed - I think new_mask_suffix
is enough. So like users who want to can use the new_mask_suffix
, can make it like population_pixel_mask_whatevertheywant.tiff
in the mantis folder. So like I think we can jut remove mask_prefix
and leave everything else as is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing to add on top of Candace's comments.
I think we should have a prefix so that all the masks are next to one another in alphabetical order when opening files in Mantis. Makes it easier to find the one you're looking for |
They all start with "population_" then with the suffix after - which will make them all easy to find I think? Since you're opening one FOV in mantis at a time anyways, you can just look for all the files that start with "population" |
Currently they all have a built in prefix of "population". So masks by default are |
Okay perfect |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
* single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix
* single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix
* added plotting notebook * updated readmed * git merge issues * git merge fixes v2 - docs/landing.md * type * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * fixed clipped plots in the notebooks. * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Added continuous variable segmentation plots * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * strange merge issue * removed branch filters on CI, so if you branch off of a branch and run GHA, the CI workflow should work * changed ci.yml, filter push on main branch only * python 3.9 doesn't support | for Union type overloading * added fovs parameter for continuous variable plots * updated notebook to include plots * updated cell clustering notebooks to add erosion (default to True), cleaned up some notebook outputs * fixed style imports * re-added test_generate_summary_stats. Was accidently deleted in a refactoring step. * newline at eof fiber_segmentation_test.py * seaborn-paper -> seaborn-v0_8-paper * added cell to view all plotting styles i mask gen nb * pycodestyle --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com>
* plot_cluster Next Release v0.6.4 (#1012) * 0.6.4 * updated environment.yml * Update README.md * updated windows setup docs * added repo verison in git clone * file fix Deepcell upload loop cleaning (#1023) * loop check for successful deepcell upload * update test zip paths * helper functions pls * zip input, upload, and extract per batch * cleaning * docstring indent * fov -> fovs file name * remove overwrite warning * new dir for each test call * fix bad logic warning testing * remove arg from docstring * add helper function test * batch_num start at 1 * add previously processed warning * set timeouts and scrap retry * correct timeout errors and update tests * timeout 5 mins * add to unprocessed list after loop closes * continue loop after extraction * break instead of continue * skipped processing print fixes * adjust print statements * 3 seconds before second zip call update * updated readthedocs python version * updated plot_utils docstring * docstrings v2 * use GS [1,1] if cbar_visible is False, otherwise use GS [1, 2] for plot + colorbar * added cbar_visible T/F for tests * pycodestyle * added numpy list to cmap / norm function * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * Mass plotting notebook (#1052) * added plotting notebook * updated readmed * git merge issues * git merge fixes v2 - docs/landing.md * type * Remove cells that don't have any pixel clusters expressed prior to Pixie (#1051) * Drop cells without any SOM cluster expressions * Add testing for dropping non-expressed cells * Include `img_sub_folder` as parameter in Mantis calls (#1050) * Ensure mantis can read the image sub folder specified * Test on example dataset * Default img_sub_folder to None in create_mantis_dir (standard used elsewhere) * Doc fix and updated example dataset * Update segmentation labels again * Clarify img_sub_folder (mostly for refreshing tests) * Force .github/get_example_dataset.py to use updated revision * Update the file names to look for * fixed clipped plots in the notebooks. * Update HuggingFace version (#1053) * v0.1.10 (#1057) * Added continuous variable segmentation plots * Pin to scikit-image to v0.19.3 (#1060) * pin to 0.19.3 * requirement less than 0.19.3 * Make sure overwrite functionality for pixel clustering propagates into helper functions (#1058) * Propagate overwrite functionality for pixel clustering * Ensure re-normalization prevented on overwrite for pixel SOM re-assignment * normalize_data should be the opposite of overwrite * Single mantis directory (#1061) * single mantis dir, no recopying * post clustering mask suffix * generic clustering adjustments * mask prefix * remove mask prefix * strange merge issue * removed branch filters on CI, so if you branch off of a branch and run GHA, the CI workflow should work * changed ci.yml, filter push on main branch only * python 3.9 doesn't support | for Union type overloading * added fovs parameter for continuous variable plots * updated notebook to include plots * updated cell clustering notebooks to add erosion (default to True), cleaned up some notebook outputs * fixed style imports * re-added test_generate_summary_stats. Was accidently deleted in a refactoring step. * newline at eof fiber_segmentation_test.py * seaborn-paper -> seaborn-v0_8-paper * added cell to view all plotting styles i mask gen nb * pycodestyle --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com> * default ero * added erode=True for generate_cluster_mask --------- Co-authored-by: alex-l-kong <31424707+alex-l-kong@users.noreply.github.com> Co-authored-by: camisowers <38049893+camisowers@users.noreply.github.com>
If you haven't already, please read through our contributing guidelines before opening your PR
What is the purpose of this PR?
Closes #669 and closes #1059. Creates a single top level mantis directory for notebooks 2, 3, and 4, to avoid duplicate files.
How did you implement your changes
_post_clustering_cell_mask
instead of just_cell_mask
to avoid overwriting notebook 3 cell cluster output files._generic_cell_mask
.Remaining issues