Check for both FOV and summary file processing before skipping a processing step during pixel clustering.

[efrancis28]

In many of the steps for pixel clustering, the script will check to see if FOVs have been processed in the feather file (ex. 'cluster_pixels') or if a single summary file has been made (ex. ;train_pixel_som') before proceeding to run that particular step. However, if your script crashes when rendering the summary files that come out of each step these cannot be easily reproduced because the 'check' will see that the FOVs have been processed and think that step is completed. For example, I have been dealing with occasional OS-related issues when trying to do consensus clustering. My individual FOV feather files will have the metacluster information appended, but the script will crash before producing the summary csv generated by 'compute_pixel_cluster_channel_avg,' which is called from 'pixel_consensus_cluster'. If I try to re-run the consensus clustering step, it will not rerun because it 'sees' that all FOVs have been processed. In this case I would have to make a call from within the Jupyter notebook directly to 'pixel_cluster_utils.compute_pixel_cluster_channel_avg.' This is thematic across all of the steps of pixel clustering. It seems to me that there are two solutions:

1. Check that the FOV-level AND summary file outputs have been completed from each step and enable the scrip to 'pick up where it left off' if both or either are missing.
2. For the relevant functions, separate the FOV processing from the generation of the summary files in the Jupyter notebook with two different functions. Right now both are generated in a single function call but the same function call will only check for the existence of one or another to indicate if that step has been completed.

[ngreenwald]

I agree. Have you talked to Alex to see if you can figure out why its still crashing? I thought we had sorted out most of these issues.

Would you say that the majority of the issues have from the summarizing steps? If so, I think it makes sense to separate those into their own cell.

[alex-l-kong]

@ngreenwald the OS-related issues Erin was experiencing were related to having initially processed these FOVs using the multiprocess library, which caused various FOV discrepancies we had to handle manually in many cases (this won't be an issue moving forward since multiprocess is turned off by default). Generating the summary files shouldn't otherwise fail as they're pretty straightforward operations, however I agree it's a good idea to separate out the calls as an additional fail-safe and to prevent pixel_consensus_cluster from getting too bloated.

[ngreenwald]

Got it, sounds good. Have you updated the MIBIAN to confirm that Erin isn't getting these errors anymore?

[alex-l-kong]

@ngreenwald yeah MIBIAN is updated and these errors shouldn't be appearing anymore.