Deflate64 zip-files not supported by dynamic-sidecar powered services

[mrnicegyu11]

Long story short

A user created a large zip-file using the windows explorer native capabilities. Appareantly, the algorithm used for the zip-file is ""Deflate64". The file is connected to a jupyter notebook. In the jupyter, the folder-structure of the zip-file is present in the inputs but the actual contents are missing.
The reason is that the unzipping routine does not support the "deflate64" algorithm: https://stackoverflow.com/questions/12809651/opening-zipfile-of-unsupported-compression-type-silently-returns-empty-filestrea

Expected behaviour

Either: The file is successfully unzipped
Or: A verbose and clear error is presented to the user, suggesting she may use a different zipping algorithm

Actual behaviour

No error, folder structure of zip-file is present but contents are missing

Steps to reproduce

Upload a deflate64 zip file to osparc using the filepicker, connect it to a jupyter

Your environment

osparc dalco prod

Logs from the jupyter

transfer completed in 93.63766647316515s
all_dwi.zip: data size: 9791.67063999176MB, transfer rate 104.56978488243162MB/s
completed download [(Port(display_order=1.0, label='input_files_1', description='Any input files. One or serveral files compressed in a zip will be downloaded in an inputs folder.', property_type='data:*/*', file_to_key_map=None, unit=None, key='input_1', widget=None, default_value=None, value=PortLink(node_uuid='a3a528d4-4dcd-46ba-af48-407e29a8175f', output='outFile')), PosixPath('/tmp/simcorefiles/input_1/all_dwi.zip')), (Port(display_order=2.0, label='input_files_2', description='Any input files. One or serveral files compressed in a zip will be downloaded in an inputs folder.', property_type='data:*/*', file_to_key_map=None, unit=None, key='input_2', widget=None, default_value=None, value=None), None), (Port(display_order=3.0, label='input_files_3', description='Any input files. One or serveral files compressed in a zip will be downloaded in an inputs folder.', property_type='data:*/*', file_to_key_map=None, unit=None, key='input_3', widget=None, default_value=None, value=None), None), (Port(display_order=4.0, label='input_files_4', description='Any input files. One or serveral files compressed in a zip will be downloaded in an inputs folder.', property_type='data:*/*', file_to_key_map=None, unit=None, key='input_4', widget=None, default_value=None, value=None), None)]
creating directory /home/jovyan/work/inputs/input_1
unzipping /tmp/simcorefiles/input_1/all_dwi.zip
Unexpected problem when processing retrieve call
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/conda/lib/python3.7/site-packages/servicelib/archiving_utils.py", line 50, in _zipfile_single_file_extract_worker
    with zf.open(name=file_in_archive) as zip_fp:
  File "/opt/conda/lib/python3.7/zipfile.py", line 1560, in open
    return ZipExtFile(zef_file, mode, zinfo, pwd, True)
  File "/opt/conda/lib/python3.7/zipfile.py", line 809, in __init__
    self._decompressor = _get_decompressor(self._compress_type)
  File "/opt/conda/lib/python3.7/zipfile.py", line 722, in _get_decompressor
    raise NotImplementedError("compression type %d (%s)" % (compress_type, descr))
NotImplementedError: compression type 9 (deflate64)
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/jupyter_commons/handlers/retrieve.py", line 37, in post
    transfered_size = await _input_retriever.download_data(ports)
  File "/opt/conda/lib/python3.7/site-packages/jupyter_commons/handlers/_input_retriever.py", line 149, in download_data
    archive_to_extract=downloaded_file, destination_folder=dest_path
  File "/opt/conda/lib/python3.7/site-packages/servicelib/archiving_utils.py", line 108, in unarchive_dir
    extracted_paths: List[Path] = await asyncio.gather(*tasks)
NotImplementedError: compression type 9 (deflate64)

[mrnicegyu11]

This hints at when windows decides to use deflate64 instead of deflate https://superuser.com/questions/1249267/how-does-windows-decide-whether-to-use-deflate-or-deflate64-for-zipping-files#comment1973099_1249267

[GitHK]

TLDR: do not use deflate64, generate the archive with a different zipping capable software.

---

Due to legal reasons python is not able to implement deflate64 since is proprietary. The only possibility here is to use https://pypi.org/project/zipfile-deflate64/

[mrnicegyu11]

Well this amounts to saying we dont support windows-explorer created zip-files :D Fine by me, but very unintuitive for windows users

[GitHK]

Please provide me with an example file generated on windows. Zip 2 empty text files and use defalte64. Make sure the error pops up.
Will try to add support for this.