Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log debug messages to file in tests #1892

Merged
merged 14 commits into from
Oct 28, 2024
Merged

Log debug messages to file in tests #1892

merged 14 commits into from
Oct 28, 2024

Conversation

TheTechromancer
Copy link
Collaborator

@TheTechromancer TheTechromancer commented Oct 26, 2024

Distro test todos:

  • wpscan:
    • ubuntu 20.04: No package matching 'ruby-rubygems' is available
    • fedora: Failed to find required executable \"gem\" in paths: /github/home/.bbot/tools:/tmp/.bbot_test/tools:/github/home/.local/bin:/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/bin:/github/home/.pyenv/shims:/github/home/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  • gowitness:
    • fedora: Setup hard-failed for gowitness: Failed to set up Google chrome. Please install manually or try again with --force-deps.
  • httpx:
    • alpine: Failed to find handler for \"/root/.ansible/tmp/ansible-tmp-1729952156.4725027-16210-212368468090700/httpx_1.2.5_linux_amd64jkxcktp4.zip\". Make sure the required command to extract the file is installed.\nCommand \"/bin/tar\" detected as tar type None. GNU tar required.\nUnable to find required 'zipinfo' binary in the path.\nCommand \"unzip -Z\" could not handle archive: /usr/bin/unzip: unrecognized option: Z\nBusyBox v1.36.1 (2024-06-10 07:11:47 UTC) multi-call binary.\n\nUsage: unzip [-lnojpqK] FILE[.zip] [FILE]... [-x FILE]... [-d DIR]\n\nExtract FILEs from ZIP archive\n\n\t-l\tList contents (with -q for short form)\n\t-n\tNever overwrite files (default: ask)\n\t-o\tOverwrite\n\t-j\tDo not restore paths\n\t-p\tWrite to stdout\n\t-t\tTest\n\t-q\tQuiet\n\t-K\tDo not clear SUID bit\n\t-x FILE\tExclude FILEs\n\t-d DIR\tExtract into DIR\n
  • unstructured:
    • gentoo:
2024-10-26T14:47:54.1856517Z   File "/github/home/.pyenv/versions/3.11.10/lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker
2024-10-26T14:47:54.1856682Z     r = call_item.fn(*call_item.args, **call_item.kwargs)
2024-10-26T14:47:54.1856806Z         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1857090Z   File "/__w/bbot/bbot/bbot/modules/unstructured.py", line 141, in extract_text
2024-10-26T14:47:54.1857220Z     elements = partition(filename=file_path)
2024-10-26T14:47:54.1857325Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1858051Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured/partition/auto.py", line 336, in partition
2024-10-26T14:47:54.1858215Z     partition_pdf = partitioner_loader.get(file_type)
2024-10-26T14:47:54.1858324Z                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1859004Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured/partition/auto.py", line 525, in get
2024-10-26T14:47:54.1859221Z     self._partitioners[file_type] = self._load_partitioner(file_type)
2024-10-26T14:47:54.1859341Z                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1860084Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured/partition/auto.py", line 543, in _load_partitioner
2024-10-26T14:47:54.1860364Z     partitioner_module = importlib.import_module(file_type.partitioner_module_qname)
2024-10-26T14:47:54.1860494Z                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1860891Z   File "/github/home/.pyenv/versions/3.11.10/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-10-26T14:47:54.1861079Z     return _bootstrap._gcd_import(name[level:], package, level)
2024-10-26T14:47:54.1861201Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1861401Z   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-10-26T14:47:54.1861607Z   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-10-26T14:47:54.1861842Z   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2024-10-26T14:47:54.1862047Z   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2024-10-26T14:47:54.1862268Z   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2024-10-26T14:47:54.1862508Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-10-26T14:47:54.1863216Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured/partition/pdf.py", line 57, in <module>
2024-10-26T14:47:54.1863455Z     from unstructured.partition.pdf_image.analysis.layout_dump import (
2024-10-26T14:47:54.1864354Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured/partition/pdf_image/analysis/layout_dump.py", line 8, in <module>
2024-10-26T14:47:54.1864587Z     from unstructured_inference.inference.layout import DocumentLayout
2024-10-26T14:47:54.1865393Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured_inference/inference/layout.py", line 15, in <module>
2024-10-26T14:47:54.1865717Z     from unstructured_inference.inference.layoutelement import LayoutElement, LayoutElements
2024-10-26T14:47:54.1866491Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/unstructured_inference/inference/layoutelement.py", line 7, in <module>
2024-10-26T14:47:54.1866770Z     from layoutparser.elements.layout import TextBlock
2024-10-26T14:47:54.1867483Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/layoutparser/elements/__init__.py", line 16, in <module>
2024-10-26T14:47:54.1867589Z     from .layout_elements import (
2024-10-26T14:47:54.1868339Z   File "/github/home/.cache/pypoetry/virtualenvs/bbot-f4v6Ab49-py3.11/lib/python3.11/site-packages/layoutparser/elements/layout_elements.py", line 25, in <module>
2024-10-26T14:47:54.1868581Z     from cv2 import getPerspectiveTransform as _getPerspectiveTransform
2024-10-26T14:47:54.1868866Z ImportError: libGL.so.1: cannot open shared object file: No such file or directory
2024-10-26T14:47:54.1868941Z """
2024-10-26T14:47:54.1868948Z 
2024-10-26T14:47:54.1869173Z The above exception was the direct cause of the following exception:
2024-10-26T14:47:54.1869180Z 
2024-10-26T14:47:54.1869297Z Traceback (most recent call last):
2024-10-26T14:47:54.1869525Z   File "/__w/bbot/bbot/bbot/scanner/scanner.py", line 1277, in _acatch
2024-10-26T14:47:54.1869605Z     yield
2024-10-26T14:47:54.1869811Z   File "/__w/bbot/bbot/bbot/modules/base.py", line 689, in _worker
2024-10-26T14:47:54.1869913Z     await self.handle_event(event)
2024-10-26T14:47:54.1870185Z   File "/__w/bbot/bbot/bbot/modules/unstructured.py", line 87, in handle_event
2024-10-26T14:47:54.1870449Z     content = await self.scan.helpers.run_in_executor_mp(extract_text, file_path)
2024-10-26T14:47:54.1870585Z               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-26T14:47:54.1870870Z ImportError: libGL.so.1: cannot open shared object file: No such file or directory

@TheTechromancer TheTechromancer changed the base branch from stable to dev October 26, 2024 13:56
Copy link

codecov bot commented Oct 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93%. Comparing base (84cc1a2) to head (8a8d33e).
Report is 15 commits behind head on dev.

Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #1892   +/-   ##
=====================================
- Coverage     93%     93%   -0%     
=====================================
  Files        356     356           
  Lines      27508   27516    +8     
=====================================
- Hits       25380   25376    -4     
- Misses      2128    2140   +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TheTechromancer TheTechromancer self-assigned this Oct 27, 2024
@TheTechromancer TheTechromancer merged commit 5360050 into dev Oct 28, 2024
13 checks passed
@TheTechromancer TheTechromancer deleted the better-logging branch October 28, 2024 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants