fix bug in deflate with windowBits == 8

[MylesBorins]

This patch comes from the optipng project and is licensesd under the
zlib license. I am unsure the original author but it may be @ctruta

This patch applies to zlib-1.1.3. It fixes a problem with 256-byte
sliding windows, that exists in all the zlib versions up to 1.1.3,
and it is proposed to be included in the future zlib-1.1.4 release
The problem occurs because MIN_LOOKAHEAD in deflate.c is usually
bigger than 256, and this confuses the longest_match() function.

The validity of the patch is enabled by this spec correction. The
particular implementation of the reference library limits the
distance codes to 2^windowBits - MIN_LOOKAHEAD.
When windowBits =9, all the distance codes are no bigger than
250, so the CINFO flag in the zlib stream can safely be 0.

The patch also extends the functionality of deflateInit(), so that
other (smaller) window sizes are also accepted. This way, the
library becomes more extensible, allowing future additions of
newer strategies optimized for speed and/or low memory
requirements, such as the Z_RLE strategy. If a certain value of
windowBits is not directly supported by the implementation, it can
be adjusted to the nearest supported value: valid zlib streams are
still produced.

Refs: http://optipng.sourceforge.net/pngtech/zlib-256win-bug.html

[jcoglan]

@MylesBorins What's the reason the check for windowBits < 8 has been removed here?

[MylesBorins]

Because it is no longer an error

[jcoglan]

I'm probably missing something but I thought that bug was just in windowBits=8. Does fixing the 256-byte window bug mean all values of windowBits from 0 to 15 are now valid?

[MylesBorins]

in retrospect that is what is in the original patch mentioned in the refernce

[peter-gribanov]

I think more correctly like this:

    if (memLevel < 1 || memLevel > MAX_MEM_LEVEL || method != Z_DEFLATED ||
-        windowBits < 8 || windowBits > 15 || level < 0 || level > 9 ||
+        windowBits < 9 || windowBits > 15 || level < 0 || level > 9 ||
-        strategy < 0 || strategy > Z_FIXED || (windowBits == 8 && wrap != 1)) {
+        strategy < 0 || strategy > Z_FIXED) {
        return Z_STREAM_ERROR;
    }
-    if (windowBits == 8) windowBits = 9;  /* until 256-byte window bug fixed */

[peter-gribanov]

As far as i can understand, a 256 byte window is not supported at all #171. Therefore, it is more correct to write like this:

-    s->w_bits = (uInt)windowBits;
+    s->w_bits = ((uInt)windowBits >= 9) ? (uInt)windowBits : 9;

[peter-gribanov]

As far as i can understand, a 256 byte window is not supported at all #171. Therefore, it is more correct to write like this:

-        uInt header = (Z_DEFLATED + ((s->w_bits-8)<<4)) << 8;
+        uInt optimized_cinfo = (s->w_bits > 9) ? s->w_bits - 8 : 0;
+        uInt header = (Z_DEFLATED + (optimized_cinfo<<4)) << 8;