Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OverflowError in archiveinfo.py #41

Closed
fdellwing opened this issue Dec 16, 2019 · 8 comments · Fixed by #42
Closed

OverflowError in archiveinfo.py #41

fdellwing opened this issue Dec 16, 2019 · 8 comments · Fixed by #42
Assignees
@miurahr
Labels
bug Something isn't working
Milestone
v0.5: compression

Comments

@fdellwing
Copy link 

Traceback (most recent call last):
  File "main.py", line 112, in <module>
    archive.close()
  File "/home/fade/PycharmProjects/mbPatchCreator/venv/lib/python3.7/site-packages/py7zr/py7zr.py", line 764, in close
    self._write_archive()
  File "/home/fade/PycharmProjects/mbPatchCreator/venv/lib/python3.7/site-packages/py7zr/py7zr.py", line 579, in _write_archive
    encoded=self.encoded_header_mode)
  File "/home/fade/PycharmProjects/mbPatchCreator/venv/lib/python3.7/site-packages/py7zr/archiveinfo.py", line 977, in write
    self.files_info.write(file)
  File "/home/fade/PycharmProjects/mbPatchCreator/venv/lib/python3.7/site-packages/py7zr/archiveinfo.py", line 854, in write
    self._write_times(file, Property.CREATION_TIME, 'creationtime')
  File "/home/fade/PycharmProjects/mbPatchCreator/venv/lib/python3.7/site-packages/py7zr/archiveinfo.py", line 775, in _write_times
    write_byte(fp, (num_defined * 8 + 2).to_bytes(1, byteorder='little'))
OverflowError: int too big to convert

The biggest file is a 7.7mb .war application. In sum the uncompressed files are 9.6mb.
@miurahr
Copy link
Owner

miurahr commented Dec 16, 2019 •
edited
Loading

Thank you for the report.
Which version do you use? Is 0.5b1?

@miurahr miurahr self-assigned this Dec 16, 2019
@miurahr miurahr added the bug Something isn't working label Dec 16, 2019
@miurahr miurahr added this to the 7z compression(v0.5) milestone Dec 16, 2019
@miurahr
Copy link
Owner

miurahr commented Dec 16, 2019 •
edited
Loading

Gotcha. It should be UINT64 but code try to write as a BYTE.

@miurahr
Copy link
Owner

miurahr commented Dec 18, 2019

Here is another problem in write_uint64() for larger values.

@miurahr miurahr mentioned this issue Dec 19, 2019
Merged
miurahr added a commit that referenced this issue Dec 19, 2019
@miurahr
Fix archive bug (fixes #41) (#42)
e8c611e 
* Refactoring test suites
* Add compression tests
* Fix archive logics
- Update write_uint64()
- Use write_uint64() for size field
* fix writing of attributes

Signed-off-by: Hiroshi Miura <miurahr@linux.com>
@miurahr miurahr reopened this Dec 19, 2019
@miurahr
Copy link
Owner

miurahr commented Dec 19, 2019

Now released v0.5b2 with fixes bugs found here.
Cloud you try again with v0.5b2 and feedback a result with your case?

@fdellwing
Copy link
Author

fdellwing commented Dec 20, 2019 •
edited
Loading

It does run through now, but it produces no valid 7z archiv:

7z l mbPatchCreator/patch251.7z 

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,12 CPUs Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz (906EA),ASM,AES-NI)

Scanning the drive for archives:
1 file, 8222762 bytes (8031 KiB)

Listing archive: mbPatchCreator/patch251.7z


ERROR: mbPatchCreator/patch251.7z : mbPatchCreator/patch251.7z
Open ERROR: Can not open the file as [7z] archive


ERRORS:
Unsupported feature
WARNINGS:
Unsupported feature


Errors: 1

The files seem to be present though:

binwalk mbPatchCreator/patch251.7z 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             7-zip archive data, version 0.4
348579        0x551A3         Zip archive data, at least v2.0 to extract, compressed size: 457, uncompressed size: 808, name: <redacted>
349077        0x55395         Zip archive data, at least v2.0 to extract, compressed size: 11093, uncompressed size: 109297, name: <redacted>
360213        0x57F15         Zip archive data, at least v2.0 to extract, compressed size: 427, uncompressed size: 819, name: <redacted>
360730        0x5811A         Zip archive data, at least v2.0 to extract, compressed size: 2176, uncompressed size: 5442, name: <redacted>
362995        0x589F3         Zip archive data, at least v2.0 to extract, compressed size: 587, uncompressed size: 1240, name: <redacted>
363687        0x58CA7         Zip archive data, at least v2.0 to extract, compressed size: 577, uncompressed size: 1231, name: <redacted>
364365        0x58F4D         Zip archive data, at least v2.0 to extract, compressed size: 484, uncompressed size: 857, name: <redacted>
364947        0x59193         Zip archive data, at least v2.0 to extract, compressed size: 877, uncompressed size: 2129, name: <redacted>
365927        0x59567         Zip archive data, at least v2.0 to extract, compressed size: 357, uncompressed size: 647, name: <redacted>
366395        0x5973B         Zip archive data, at least v2.0 to extract, compressed size: 646, uncompressed size: 1426, name: <redacted>
551096        0x868B8         Zip archive data, at least v2.0 to extract, compressed size: 690, uncompressed size: 1255, name: <redacted>
551843        0x86BA3         Zip archive data, at least v2.0 to extract, compressed size: 852, uncompressed size: 1973, name: <redacted>
552740        0x86F24         Zip archive data, at least v2.0 to extract, compressed size: 109486, uncompressed size: 125551, name: <redacted>
662296        0xA1B18         Zip archive data, at least v2.0 to extract, compressed size: 33824, uncompressed size: 36998, name: <redacted>
696193        0xA9F81         Zip archive data, at least v2.0 to extract, compressed size: 29464, uncompressed size: 31413, name: <redacted>
725726        0xB12DE         Zip archive data, at least v2.0 to extract, compressed size: 24867, uncompressed size: 27075, name: <redacted>
750658        0xB7442         Zip archive data, at least v2.0 to extract, compressed size: 272101, uncompressed size: 291526, name: <redacted>
1022829       0xF9B6D         Zip archive data, at least v2.0 to extract, compressed size: 40767, uncompressed size: 43033, name: <redacted>
1063652       0x103AE4        Zip archive data, at least v2.0 to extract, compressed size: 3961, uncompressed size: 4969, name: <redacted>
1067682       0x104AA2        Zip archive data, at least v2.0 to extract, compressed size: 245276, uncompressed size: 270750, name: <redacted>
1313037       0x14090D        Zip archive data, at least v2.0 to extract, compressed size: 5602, uncompressed size: 7453, name: <redacted>
1318700       0x141F2C        Zip archive data, at least v2.0 to extract, compressed size: 377872, uncompressed size: 427729, name: <redacted>
1696654       0x19E38E        Zip archive data, at least v2.0 to extract, compressed size: 4880, uncompressed size: 7028, name: <redacted>
1701606       0x19F6E6        Zip archive data, at least v2.0 to extract, compressed size: 145067, uncompressed size: 163575, name: <redacted>
1846746       0x1C2DDA        Zip archive data, at least v2.0 to extract, compressed size: 25597, uncompressed size: 29257, name: <redacted>
1872404       0x1C9214        Zip archive data, at least v2.0 to extract, compressed size: 60313, uncompressed size: 67758, name: <redacted>
1932778       0x1D7DEA        Zip archive data, at least v2.0 to extract, compressed size: 684146, uncompressed size: 765648, name: <redacted>
2617030       0x27EEC6        Zip archive data, at least v2.0 to extract, compressed size: 112421, uncompressed size: 120603, name: <redacted>
2729519       0x29A62F        Zip archive data, at least v2.0 to extract, compressed size: 18832, uncompressed size: 21255, name: <redacted>
2748420       0x29F004        Zip archive data, at least v2.0 to extract, compressed size: 464767, uncompressed size: 466793, name: <redacted>
3213268       0x3107D4        Zip archive data, at least v2.0 to extract, compressed size: 92043, uncompressed size: 105134, name: <redacted>
3516517       0x35A865        Zip archive data, at least v2.0 to extract, compressed size: 76224, uncompressed size: 94557, name: <redacted>
3592813       0x36D26D        Zip archive data, at least v2.0 to extract, compressed size: 56394, uncompressed size: 65012, name: <redacted>
3649273       0x37AEF9        Zip archive data, at least v2.0 to extract, compressed size: 631847, uncompressed size: 710492, name: <redacted>
4281205       0x415375        Zip archive data, at least v2.0 to extract, compressed size: 50183, uncompressed size: 59594, name: <redacted>
4331460       0x4217C4        Zip archive data, at least v2.0 to extract, compressed size: 58052, uncompressed size: 62983, name: <redacted>
4389575       0x42FAC7        Zip archive data, at least v2.0 to extract, compressed size: 792555, uncompressed size: 890168, name: <redacted>
5182232       0x4F1318        Zip archive data, at least v2.0 to extract, compressed size: 2643, uncompressed size: 4467, name: <redacted>
5184936       0x4F1DA8        Zip archive data, at least v2.0 to extract, compressed size: 4400, uncompressed size: 7239, name: <redacted>
5189433       0x4F2F39        Zip archive data, at least v2.0 to extract, compressed size: 407453, uncompressed size: 467060, name: <redacted>
5596971       0x55672B        Zip archive data, at least v2.0 to extract, compressed size: 1796, uncompressed size: 2497, name: <redacted>
5598827       0x556E6B        Zip archive data, at least v2.0 to extract, compressed size: 12788, uncompressed size: 16064, name: <redacted>
5611680       0x55A0A0        Zip archive data, at least v2.0 to extract, compressed size: 4012, uncompressed size: 6674, name: <redacted>
5615781       0x55B0A5        Zip archive data, at least v2.0 to extract, compressed size: 149683, uncompressed size: 151881, name: <redacted>
5765528       0x57F998        Zip archive data, at least v2.0 to extract, compressed size: 15573, uncompressed size: 18037, name: <redacted>
5781170       0x5836B2        Zip archive data, at least v2.0 to extract, compressed size: 16437, uncompressed size: 18323, name: <redacted>
5797672       0x587728        Zip archive data, at least v2.0 to extract, compressed size: 3074, uncompressed size: 5112, name: <redacted>
5800817       0x588371        Zip archive data, at least v2.0 to extract, compressed size: 6150, uncompressed size: 8695, name: <redacted>
5807035       0x589BBB        Zip archive data, at least v2.0 to extract, compressed size: 4267, uncompressed size: 5848, name: <redacted>
5811362       0x58ACA2        Zip archive data, at least v2.0 to extract, compressed size: 514180, uncompressed size: 605774, name: <redacted>
6500505       0x633099        Zip archive data, at least v2.0 to extract, compressed size: 1431, uncompressed size: 3827, name: <redacted>
6502007       0x633677        Zip archive data, at least v2.0 to extract, compressed size: 4986, uncompressed size: 17645, name: <redacted>
6507065       0x634A39        Zip archive data, at least v2.0 to extract, compressed size: 7029, uncompressed size: 33787, name: <redacted>
6514160       0x6365F0        Zip archive data, at least v2.0 to extract, compressed size: 1561, uncompressed size: 4338, name: <redacted>
6515796       0x636C54        Zip archive data, at least v2.0 to extract, compressed size: 8141, uncompressed size: 36428, name: <redacted>
6524004       0x638C64        Zip archive data, at least v2.0 to extract, compressed size: 1710, uncompressed size: 4946, name: <redacted>
6525787       0x63935B        Zip archive data, at least v2.0 to extract, compressed size: 903, uncompressed size: 1985, name: <redacted>
6526762       0x63972A        Zip archive data, at least v2.0 to extract, compressed size: 283559, uncompressed size: 283500, name: <redacted>
6810399       0x67EB1F        Zip archive data, at least v2.0 to extract, compressed size: 281213, uncompressed size: 281212, name: <redacted>
7091685       0x6C35E5        Zip archive data, at least v2.0 to extract, compressed size: 269876, uncompressed size: 269832, name: <redacted>

@miurahr
Copy link
Owner

miurahr commented Dec 22, 2019

How can I reproduce your case?
Could you mind to put a minimum test case, pytest function and data, which can be bundled here in GPL?

@miurahr
Copy link
Owner

miurahr commented Dec 22, 2019

Overflow error happened on reporter environment are gone.
I'd like to close here and ask @fdellwing to open a new issue ticket for another problem.

@fdellwing
Copy link
Author

I'm in xmas vacation, will open new issue after I get back to work.

@fdellwing fdellwing mentioned this issue Jan 7, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@miurahr miurahr

Labels
bug Something isn't working
Projects
None yet
Milestone
v0.5: compression
Development

Successfully merging a pull request may close this issue.

Fix archive bug (fixes #41) miurahr/py7zr
2 participants
@miurahr @fdellwing