[security] CVE-2019-9674: Zip Bomb vulnerability

[krnick]

BPO	36260
Nosy	@jaraco, @vstinner, @tiran, @serhiy-storchaka, @18z, @tirkarthi, @krnick, @sidra-asa
PRs	bpo-36260: Add pitfalls to zipfile module documentation #13378 [3.8] bpo-36260: Add pitfalls to zipfile module documentation (GH-13378) #15976

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-11.15:04:49.599>
created_at = <Date 2019-03-11.07:16:58.724>
labels = ['type-security', '3.8', '3.7', 'library']
title = '[security] CVE-2019-9674: Zip Bomb vulnerability'
updated_at = <Date 2020-02-10.07:59:41.779>
user = 'https://github.com/krnick'

bugs.python.org fields:

activity = <Date 2020-02-10.07:59:41.779>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-11.15:04:49.599>
closer = 'jaraco'
components = ['Library (Lib)']
creation = <Date 2019-03-11.07:16:58.724>
creator = 'krnick'
dependencies = []
files = []
hgrepos = []
issue_num = 36260
keywords = ['patch']
message_count = 19.0
messages = ['337650', '337651', '337652', '337835', '339061', '339062', '339083', '339084', '339087', '339316', '339329', '339406', '339408', '339587', '341256', '342693', '351921', '351964', '361673']
nosy_count = 8.0
nosy_names = ['jaraco', 'vstinner', 'christian.heimes', 'serhiy.storchaka', '18z', 'xtreak', 'krnick', 'Victor Kung']
pr_nums = ['13378', '15976']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue36260'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[krnick]

Dear Python Community,

We’ve found a vulnerability in cpython Lib and already received a cve number (CVE-2019-9674)
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-9674

We also have a patch for this vulnerability, please tell us what to do next.
Since we don’t want to uncover the vulnerability before it get fixed.

JUN-WEI SONG

[18z]

Dear community,

I am one of the discoverer of this vulnerability, please tell us what to do next :D

Kunyu Chen

[tirkarthi]

You can find the process to report security vulnerabilities at https://www.python.org/news/security/ . Please email the details to security@python.org and who will analyze the report before public disclosure.

[18z]

Thank you Karthikeyan Singaravelan.
We're working on it :D

Kunyu Chen

[tiran]

Issue bpo-36462 contains more information. The reporter claims that the zipfile module is inherent insecure because it does not provide any heuristics to make zipbomb attacks harder.

I'm -1 to implement such a heuristic. The zipfile module is a low level module and should not limit extraction by defaykt. Instead we should improve documentation and maybe implement some method that simplifies detection of zipbomb attacks. I'm thinking about a method that returns total count of files, total compressed size and total uncompressed size.

[serhiy-storchaka]

All these are simple one-liners:

    len(zf.infolist())
    sum(zi.compress_size for zi in zf.infolist())
    sum(zi.file_size for zi in zf.infolist())

[krnick]

Thank you python community, these two issues are indeed the same problem.

I also think that it is good to make a related document to reduce such problems.

[krnick]

Thank you python community, these two issues are indeed the same problem.

I also think that it is good to make a related document to reduce such problems.

[18z]

Thank you for the responses.

I agree with Christian Heimes.

It's indeed better to improve the documentation rather than directly implement the heuristic.

[krnick]

Hello Python community,

With Christian Heimes’ suggestion, we manipulate appropriate warning to inform users that they may encounter zip bomb issues when using the zipfile module.

The warning we would like to add in the zipfile documentation is shown below :

https://github.com/python/cpython/blob/3.7/Doc/library/zipfile.rst

.. warning::

Never extract files from untrusted sources without prior 
inspection. It is possible that the file may contain zip bomb 
issues such as 42.zip. The zip bomb will usually be a small file 
before decompression, but once it is decompressed, it will 
exhaust system resources.

You can protect your system by limiting system resources, limiting compression ratio (zip bombs are usually quite high), and checking for nested zip files.

We are also pleasure to provide a patch to enhance the zipfile module to provide basic information.

In zipfile.py

https://github.com/python/cpython/blob/master/Lib/zipfile.py

Inside the ZipFile class :

def filecount(self):                                                                                         
    """Return total count of files in the archive."""                                                        
    return len(self.filelist)                                                                                
                                                                                                                 
def total_compressed_size(self):                                                                             
    """Return total compressed size in the archive."""                                                       
    return sum([data.compress_size for data in self.filelist])                                               
                                                                                                                 
def total_uncompressed_size(self):                                                                           
    """Return total uncompressed size in the archive."""                                                     
    return sum([data.file_size for data in self.filelist])

[serhiy-storchaka]

I am against such trivial methods in ZipFile. Its interface is already complicate. The advantage of Python is that you do not need tons of methods for every possible query -- you can just combine few Python features into a one-line expression.

As for the documentation change, it could be useful to add more general note about possible pitfalls. What happen when interrupt extracting or adding to the archive, what happen when extract into existing tree or overwrite an existing file, what happen when the file system does not support some file names, what happen when extract to case-insensitive file system, what happen when extract encrypted file with wrong password, etc. We do not have to tell the user what he should not do, just to warn about the possible consequences.

[sidra-asa]

Hello Python community,

I’m curious why the patch or pitfall prevention in ZipFile
are not suggested. I have no idea if everyone read documentation in detail. It seems straightforward to add the methods in ZipFile with well documented rather than just warn in documentation. Any comment would be appreciated.

Victor Kung

[tiran]

The suggested approach is merely a heuristic that reduces the impact of a zipbomb. An attacker can circumvent the heuristic. In best case scenario, the approach just increases the cost factor for a successful DoS. For example an attacker may have to upload 10 larger zip files instead of one smaller zip file to fill up the disk space of a server.

The correct approach is to always verify all data from untrusted sources. It's the 101 of application security.

[sidra-asa]

I see.

@christian Heimes Thank you for the response.

[krnick]

Thank you very much for your reply.

Based on discussions above, consensuses are improving the zipfile documentation.

And we (JUN-WEI SONG & KunYu Chen) would like to work on this.

With opinions of Serhiy Storchaka, Christian Heimes and the ideas we have, possible pitfalls are listed below.

1. From file itself: 

    Decompression may fail due to an incorrect password, an 
    incorrect CRC checksum, an incorrect PKZIP format, an 
    unsupported compression method, or an unsupported decryption.

2. File system: 

    Each file system has different limitations such as allowable 
    characters in directory entries, the max length of file name, 
    the max length of path name, the max size of single file, the 
    max number of files, the max number of files in a single 
    directory, etc. Decompression will fail as long as these 
    limitations are exceeded.

 3. Operating system: 

    The lack of memory or disk space would lead to decompression 
    failed (see also Zip Bomb). 

 4. Interrupt: 

    Users should be careful in interrupting the process of 
    decompression, such as control-C or killing the process during 
    decompression, which may result in incomplete decompression of 
    the archive.

5. Different default behaviors: 

    Users should figure out different default extraction behaviors, 
    such as when extracting into the existing tree, it will 
    overwriting an existing file without asking, or  when in a 
    case-insensitive file system, it keeps only one file when 
    extracting an archive which contains many files that have the 
    same name but different case. 

Please let us know if anything’s missing.

[krnick]

Dear friends,

We moved a little bit forward to improve the writing. :)

[jaraco]

New changeset 3ba51d5 by Jason R. Coombs (JunWei Song) in branch 'master':
bpo-36260: Add pitfalls to zipfile module documentation (bpo-13378)
3ba51d5

[jaraco]

New changeset c5a6723 by Jason R. Coombs (Miss Islington (bot)) in branch '3.8':
bpo-36260: Add pitfalls to zipfile module documentation (GH-13378) (GH-15976)
c5a6723

[vstinner]

I marked bpo-39341 as a duplicate of this issue: "[security] zipfile: ZIP Bomb vulnerability, don't check announced uncompressed size".