ENH: Nosetest for wavelets on different ROIs #348
Conversation
…of wavelet filter on different ROIs of the same image.
| if not column.startswith("general"): | ||
| featList = dfFeatures[column].tolist() | ||
| maxDiff = max(abs(a) - abs(b) for a, b in itertools.combinations(featList, 2)) | ||
| assert maxDiff == 0 |
There was a problem hiding this comment.
Asserting equals 0 would not account for machine precision errors. For general feature tests in PyRadiomics we allow a maximum difference of 3% compared to baseline. See also here.
There was a problem hiding this comment.
Moreover, it might be a good idea to work with a csv-defined baseline. This will ensure that your calculating the same values each time. By recalculating your baseline here, it might be possible that some systematic errors are not caught. I.e. if for some reason, the filter adds 1 to each result, it would not be caught here.
|
|
||
| features = [] | ||
| for extractor in extractors: | ||
| for data in self.testData: |
There was a problem hiding this comment.
These two loops sort of specify different parameters of testing. In PyRadiomics we split these out as well.
This is acchieved by the @parameterized.expand(<generatorFunction>()), where <generatorFunction> is a function that utilizes the yield statement (i.e. is a python generator). Inside that function are the loops to define your testcases, and the yield statement returns an equal number of arguments as are consumed by your test function.
See here for an example.
There was a problem hiding this comment.
The loops don't specify different test cases/scenarios. It is just one case which compares 4 different ROI variations for computational equality. Parameterized creates independent test cases which wouldn't help. Only thing I could do is create parameterized test cases that check always two pairs of my 4 variations. I don't see the benefit though :/
There was a problem hiding this comment.
Also, we could think about parameterizing the features, so one case for each feature. However, the actual intention is to check the wavelet filter for equality, using the feature values is just a means to make it easier. To implement the intention more correctly I could re-write this to actually get the wavelet filtered images and compare the pixels under the mask for equality ... but I am not sure if it's worth the effort since this does the job.
|
|
||
| dfFeatures = pd.DataFrame(features) | ||
| for column in dfFeatures: | ||
| if not column.startswith("general"): |
There was a problem hiding this comment.
@michaelschwier, you can prevent the addition of the general info columns by calling addProvenance(False) on each extractor when you instantiate it in getTestExtractor()
Add a test to check if precropping and different size input images still produce the same output for the wavelet filter. This test uses 2 new test cases that have been added to the 1.0.0 release of PyRadiomics and enabled for download by `getTestCase()`. Additionally ensures temporal consistency by using a baseline file that defines the 1D-array of voxels in the ROI for each wavelet level (This array can be obtained by slicing the image array with a boolean cast of the mask array). The baseline is generated using the large (64x64x64) image without pre cropping. The function to generate this baseline is added to `add_baseline.py`. Finally, apply some style changes to the other test files to make them more consistent with pyradiomics style. Supersedes AIM-Harvard#348.
|
Superseded by #387 |
Add a test to check if precropping and different size input images still produce the same output for the wavelet filter. This test uses 2 new test cases that have been added to the 1.0.0 release of PyRadiomics and enabled for download by `getTestCase()`. Additionally ensures temporal consistency by using a baseline file that defines the 1D-array of voxels in the ROI for each wavelet level (This array can be obtained by slicing the image array with a boolean cast of the mask array). The baseline is generated using the large (64x64x64) image without pre cropping. The function to generate this baseline is added to `add_baseline.py`. Finally, apply some style changes to the other test files to make them more consistent with pyradiomics style. Supersedes AIM-Harvard#348.
Follow up on PR #346.
This test check equality of features extracted after wavelet filtering on the same image but different ROIs (with padding large enough such that wavelet filter doesn't run into the image border for pixels under the mask).
Test data is a small and simple generated image containing a checkered sphere and noise around.