Add adaptive threshold algorithm using mean method

[miralshah365]

Description

New threshold adaptive method added.

This method allows the user to set threshold values according to neighbors. The threshold value is found by calculating the mean of all the neighbors including pixel itself. This mean value is then treated as a threshold and the binary threshold is performed.

References

• Implements [gsoc2019] Implement adaptive mean thresholding #315

Tasklist

• Add test case(s) to be added as part of [gsoc2019] Add tests for adaptive thresholding #353
• Ensure all CI builds pass (Add adaptive threshold algorithm using mean method #341 (comment))
• Review and approve

[mloskot]

@miralshah365 First round of review that only addresses the run-time failure you mentioned on Gitter.

[mloskot]

@miralshah365 I noticed a memory leak

[mloskot]

@miralshah365
I've been playing with the algorithm against this little funny sample of 16x16 pixel greyscale gradient background with two 'barely visible rectangles.
You can download it here .

OpenCV

The 16x16 sample above passed through the adaptive mean threshold by OpenCV:

import cv2
import numpy as np
import matplotlib.pyplot as plt
path = 'D:\\workshop\\opencv\\'
path_original = path + "rectangles.png"

img = cv2.imread(path_original, 0)

ret, th1 = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)
th2 = cv2.adaptiveThreshold(
    img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 11, 0)
th3 = cv2.adaptiveThreshold(
    img, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 0)

titles = ['Input', 'Global Thresholding (t=127)',
          'Adaptive Mean', 'Adaptive Gaussian']
images = [img, th1, th2, th3]
for i in range(4):
    plt.subplot(2, 2, i+1), plt.imshow(images[i], 'gray')
    plt.title(titles[i])
    plt.xticks([]), plt.yticks([])
plt.show()

identifies both rectangles:

GIL

Using threshold_adaptive:

std::string image_dir = "d:\\workshop\\opencv\\";
{
    gray8_image_t img;
    read_image(image_dir + "rectangles.png", img, png_tag{});
    gray8_image_t img_out(img.dimensions());

    boost::gil::threshold_adaptive(const_view(img), view(img_out), 11);
    write_view(image_dir + "out-threshold-adaptive.png", view(img_out), png_tag{});
}

identifies only one, the upper rectangle:

Here is dump of the convoluted view generated with this

write_view("d:\\workshop\\opencv\\out-threshold-adaptive-convoluted-view.png", temp_view, png_tag{});

Unfortunately, I have no way to generate equivalent for OpenCV.

Conclusion

There still may be some fine tuning of the algorithm necessary.
At least, it may be worth to test it a bit more, also with your impl. of convolution.

[miralshah365]

That's because of kernel size and image size are too close and in such cases what scheme we chose for the border is really important. OpenCV's default scheme is this while GIL extends the border with zeros

I created the same type of image but in a larger size (3000x3000 pixels)

and tests on this image:

OpenCV output:

Boost.GIL output:

Somehow they are inverted I'll investigate that behavior.

[mloskot]

@miralshah365 Here is comparison of the rectangles.png passed through the threshold_adaptive(kernel_size=11) and convolve called with all the supported options.
Shortly, convolve_option_extend_constant gives most similar output to the OpenCV.

---

LHS - threshold_adaptive output image, RHS - convolve result image.

---

Interestingly, OpenCV for ADAPTIVE_THRESH_MEAN_C uses BORDER_REPLICATE|BORDER_ISOLATED (see imgproc/src/thresh.cpp:1666-1668).

@miralshah365 This OpenCV's boundary mode is quite alike the convolve_option_extend_constant which also replicates boundary values, right? For the mean filter, the replication does make sense, doesn't it?

[mloskot]

(Related to my earlier comment #341 (comment))

@miralshah365 From your yesterday comment on Gitter:

• correct output is what OpenCV produces with C=0
• correct output is white rectangles on black background

I don't think there is a single correct output because for such sharp edge cases (i.e. dark gray features on light gray background), thresholding result does depend on value of `C1:

From adaptive threshold description at http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm

if the threshold employed is not the mean, but (mean-C), where C is a constant.
Using this statistic, all pixels which exist in a uniform neighborhood (e.g. along the margins) are set to background.

Assuming that OpenCV defaults to C=5, one may argue that 'correct' output is black rectangles on white background. This is what OpenCV produces from our 3000x3000 test image with the two rectangles for any C>0. So, since 'correct' depends on algorithm parameters, I wouldn't seek for a single correct answer.

OpenCV:

• C > 0: black rectangles on white background
• C = 0: white rectangles on black background

GIL:

• px1 >= px2 ? max_value : 0: black rectangles on white background
• px1 > px2 ? max_value : 0: white rectangles on black background

In other words, equivalent threshold tests are:

• black rectangles

  px1 >= px2      ? max_value : 0
  px1 > (px2 - 5) ? max_value : 0
  

• white rectangles

  px1 > px2       ? max_value : 0
  px1 > (px2 - 0) ? max_value : 0
  

So, the major difference between behaviour of OpenCV and GIL that I can see is this flexibility due to the extra constant parameter. This is a common approach, see also https://imagej.net/Auto_Local_Threshold for the Mean formula it uses.

@miralshah365 Your algorithm could also allow such parametrization.

Alternative mean test using offset

Let's an alternative threshold test using C constant as mean offset:

std::abs(px1 - px2) < C ? max_value : 0

Although I don't think it is common approach and rather fine-tuned/specific to cetain applications, its properties are interesting, from http://www.roborealm.com/help/Adaptive_Threshold.php:

This helps to ensure stability of the white pixels by ensuring that they white pixels are higher than the mean by X amount.
A low offset value will cause some pixels to vibrate between black and white if they are near the intensity edge.

If you try it out for our test case, you should see that it produces thicker and gradual outline of each rectangle which closely reflects the mean values near the edges of rectangles (produced by the convolve with the mean kernel), which are gradual as well:

[mloskot]

@miralshah365 Good job! Please, merge whenever you like (assuming CI-s are green, of course).

---

More tests will come as part of separate task #353 along with the fine-tuning of the algorithm itself, if we discover and agree as necessary.

[mloskot]

@miralshah365 Travis CI for boostorg/ must be having big queue, so we may be waiting 12, 24, 48 ... hours, unfortunately. However, I see that your private Travis CI has built this branch green: https://travis-ci.org/miralshah365/gil/builds/563401955, so we should be safe to assume it's all good.

If you want, I can merge this PR without waiting for the Boost's Travis CI. Shall I do it?