Question about nuclei segmentation

[chenyhvvvv]

When I run through your pipeline, there is always an error of
Traceback (most recent call last):
File "/import/home2/yhchenmath/Code_raw/BIDCell-main/run.py", line 3, in
model.run_pipeline()
File "/import/home2/yhchenmath/Code_raw/BIDCell-main/bidcell/BIDCellModel.py", line 40, in run_pipeline
self.preprocess()
File "/import/home2/yhchenmath/Code_raw/BIDCell-main/bidcell/BIDCellModel.py", line 61, in preprocess
segment_nuclei(self.config)
File "/import/home2/yhchenmath/Code_raw/BIDCell-main/bidcell/processing/nuclei_segmentation.py", line 120, in segment_nuclei
patch_nuclei = segment_dapi(patch_resized, config.nuclei.diameter, config.nuclei.use_cpu)
File "/import/home2/yhchenmath/Code_raw/BIDCell-main/bidcell/processing/nuclei_segmentation.py", line 26, in segment_dapi
mask, _, _, _ = model.eval(img, diameter=diameter, channels=channels)
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/models.py", line 188, in eval
masks, flows, styles = self.cp.eval(x, channels=channels,
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/models.py", line 410, in eval
masks, styles, dP, cellprob, p = self._run_cp(
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/models.py", line 517, in _run_cp
outputs = dynamics.resize_and_compute_masks(
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/dynamics.py", line 779, in resize_and_compute_masks
mask, p = compute_masks(dP, cellprob, p=p, niter=niter,
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/dynamics.py", line 830, in compute_masks
mask = get_masks(p, iscell=cp_mask)
File "/home/yhchenmath/anaconda3/envs/bidcell/lib/python3.10/site-packages/cellpose/dynamics.py", line 746, in get_masks
M0 = M[tuple(pflows)]
IndexError: index -2147483628 is out of bounds for axis 0 with size 5140

The dapi file is download directly from the 10x website. What's wrong with it?

[xhelenfu]

Hi, may I know the name/link to the dataset and file name of the DAPI image you are using? I can take a look at this issue

[chenyhvvvv]

Oh. This because the cuda version and I have solved it. And I have a question about the paper Fig.2 (b). Your method uses the cellpose nuclei segmentation result to get the number of batch size. So I wonder why the segmented cell number of BIDCell is larger than the cellpose(nuclei)? And for the Xenium breastcancer1, the segmented cell number of 10X nuclei and the 10X have such a gap? Since in nucleus_boundaries.parquet of the original data both have 160000+ cell id.

[chenyhvvvv]

And can you provide the scRNA reference and Markere gene or tell me how to find them on stereo-seq mouse brain? I wonder if they are from Allen mouse brain too. Thanks a lot!