Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Print a full traceback on convert exception #272

Open
mgorny opened this issue Dec 17, 2024 · 3 comments
Open

Print a full traceback on convert exception #272

mgorny opened this issue Dec 17, 2024 · 3 comments
Labels
enhancement New feature or request source::user-request Requests that come from the community and library users.
Milestone
Maintenance: Bugs...

Comments

@mgorny
Copy link

mgorny commented Dec 17, 2024

Description
Currently, if a recipe fails to parse, we often get an unclear exception message (e.g. in #229), or with onednn-feedstock:

$ conda-recipe-manager  convert --debug recipe/meta.yaml  
EXCEPTION: An exception occurred while parsing the recipe file
list index out of range
0 errors and 0 warnings were found.

Could we have it print the complete traceback instead? Possibly conditionally to --debug, or using some other flag.

Additional context
Reproduced using 0.4.0 and e2bf2cb, on top of conda-forge/onednn-feedstock@15fa3b7. I'll file a separate bug for the parsing problem, once I debug it a bit myself.

conda list 
# packages in environment at /home/mgorny/miniforge3/envs/crm:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               h04ea711_2    conda-forge
attrs                     24.3.0             pyh71513ae_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_1    conda-forge
brotli                    1.1.0                hb9d3cd8_2    conda-forge
brotli-bin                1.1.0                hb9d3cd8_2    conda-forge
brotli-python             1.1.0           py312h2ec8cdc_2    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.34.4               hb9d3cd8_0    conda-forge
ca-certificates           2024.12.14           hbcca054_0    conda-forge
cairo                     1.18.2               h3394656_1    conda-forge
certifi                   2024.12.14         pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h06ac9bb_0    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_1    conda-forge
click                     8.1.7           unix_pyh707e725_1    conda-forge
colorama                  0.4.6              pyhd8ed1ab_1    conda-forge
conda                     24.11.1         py312h7900ff3_0    conda-forge
conda-libmamba-solver     24.11.1            pyhd8ed1ab_0    conda-forge
conda-package-handling    2.4.0              pyha770c72_1    conda-forge
conda-package-streaming   0.11.0             pyhd8ed1ab_0    conda-forge
conda-recipe-manager      0.4.0              pyhd8ed1ab_0    conda-forge
contourpy                 1.3.1           py312h68727a3_0    conda-forge
cpp-expected              1.1.0                hf52228f_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_1    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
fmt                       11.0.2               h434a139_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_3    conda-forge
fontconfig                2.15.0               h7e30c49_1    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.55.3          py312h178313f_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
frozendict                2.4.6           py312h66e93f0_0    conda-forge
gdk-pixbuf                2.42.12              hb9ae30d_0    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_1    conda-forge
gitpython                 3.1.43             pyhff2d567_1    conda-forge
graphite2                 1.3.13            h59595ed_1003    conda-forge
graphviz                  12.0.0               hba01fac_0    conda-forge
gtk2                      2.24.33              h6470451_5    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h2                        4.1.0              pyhd8ed1ab_1    conda-forge
harfbuzz                  9.0.0                hda332d3_1    conda-forge
hpack                     4.0.0              pyhd8ed1ab_1    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_1    conda-forge
icu                       75.1                 he02047a_0    conda-forge
idna                      3.10               pyhd8ed1ab_1    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_1    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_1    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_1    conda-forge
jsonpointer               3.0.0           py312h7900ff3_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_1    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.7           py312h68727a3_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.43                 h712a8e2_2    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libarchive                3.7.7                h4585015_3    conda-forge
libblas                   3.9.0           25_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hb9d3cd8_2    conda-forge
libbrotlidec              1.1.0                hb9d3cd8_2    conda-forge
libbrotlienc              1.1.0                hb9d3cd8_2    conda-forge
libcblas                  3.9.0           25_linux64_openblas    conda-forge
libcurl                   8.11.1               h332b0f4_0    conda-forge
libdeflate                1.23                 h4ddbbb0_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.4                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgd                     2.3.3               hd3e95f3_10    conda-forge
libgfortran               14.2.0               h69a702a_1    conda-forge
libgfortran5              14.2.0               hd5240d6_1    conda-forge
libglib                   2.82.2               h2ff4ddf_0    conda-forge
libgomp                   14.2.0               h77fa898_1    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           25_linux64_openblas    conda-forge
liblzma                   5.6.3                hb9d3cd8_1    conda-forge
libmamba                  2.0.5                hf72d635_0    conda-forge
libmambapy                2.0.5           py312hf3f0a4e_0    conda-forge
libnghttp2                1.64.0               h161d5f1_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.28          pthreads_h94d23a6_1    conda-forge
libpng                    1.6.44               hadc24fc_0    conda-forge
librsvg                   2.58.4               hc0ffecb_0    conda-forge
libsolv                   0.7.30               h3509ff9_0    conda-forge
libsqlite                 3.47.2               hee588c1_0    conda-forge
libssh2                   1.11.1               hf672d98_0    conda-forge
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.2.0               h4852527_1    conda-forge
libtiff                   4.7.0                hd9ff511_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.17.0               h8a09558_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.13.5               h8d12d68_1    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
lz4-c                     1.10.0               h5888daf_1    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
markupsafe                3.0.2           py312h178313f_1    conda-forge
matplotlib-base           3.10.0          py312hd3ec401_0    conda-forge
menuinst                  2.2.0           py312h7900ff3_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
networkx                  3.4.2              pyh267e887_2    conda-forge
nlohmann_json             3.11.3               he02047a_1    conda-forge
numpy                     2.2.0           py312h7e784f5_0    conda-forge
openjpeg                  2.5.3                h5fbd93e_0    conda-forge
openssl                   3.4.0                hb9d3cd8_0    conda-forge
packaging                 24.2               pyhd8ed1ab_2    conda-forge
pango                     1.54.0               h3a902e7_3    conda-forge
pcre2                     10.44                hba22ea6_2    conda-forge
pillow                    11.0.0          py312h7b63e92_0    conda-forge
pip                       24.3.1             pyh145f28c_1    conda-forge
pixman                    0.44.2               h29eaf8c_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_2    conda-forge
platformdirs              4.3.6              pyhd8ed1ab_1    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_1    conda-forge
pthread-stubs             0.4               hb9d3cd8_1002    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py312h66e93f0_2    conda-forge
pycparser                 2.22               pyh29332c3_1    conda-forge
pygraphviz                1.14            py312h011e53f_0    conda-forge
pyparsing                 3.2.0              pyhd8ed1ab_2    conda-forge
pysocks                   1.7.1              pyha55dd90_7    conda-forge
python                    3.12.8          h9e4cc4f_1_cpython    conda-forge
python-dateutil           2.9.0.post0        pyhff2d567_1    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pyyaml                    6.0.2           py312h66e93f0_1    conda-forge
qhull                     2020.2               h434a139_5    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_1    conda-forge
reproc                    14.2.5.post0         hb9d3cd8_0    conda-forge
reproc-cpp                14.2.5.post0         h5888daf_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_1    conda-forge
rpds-py                   0.22.3          py312h12e396e_0    conda-forge
ruamel.yaml               0.18.6          py312h66e93f0_1    conda-forge
ruamel.yaml.clib          0.2.8           py312h66e93f0_1    conda-forge
setuptools                75.6.0             pyhff2d567_1    conda-forge
simdjson                  3.11.3               h84d6215_0    conda-forge
six                       1.17.0             pyhd8ed1ab_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
spdlog                    1.14.1               hed91bc2_1    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tqdm                      4.67.1             pyhd8ed1ab_0    conda-forge
truststore                0.10.0             pyhd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_1    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
unicodedata2              15.1.0          py312h66e93f0_1    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_1    conda-forge
xorg-libice               1.1.2                hb9d3cd8_0    conda-forge
xorg-libsm                1.2.5                he73a12e_0    conda-forge
xorg-libx11               1.8.10               h4f16b4b_1    conda-forge
xorg-libxau               1.0.12               hb9d3cd8_0    conda-forge
xorg-libxdmcp             1.1.5                hb9d3cd8_0    conda-forge
xorg-libxext              1.3.6                hb9d3cd8_0    conda-forge
xorg-libxrender           0.9.12               hb9d3cd8_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.8.0                h59595ed_0    conda-forge
zipp                      3.21.0             pyhd8ed1ab_1    conda-forge
zstandard                 0.23.0          py312hef9b889_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge
@mgorny mgorny added enhancement New feature or request source::user-request Requests that come from the community and library users. to sort Needs additional investigation labels Dec 17, 2024
@schuylermartin45 schuylermartin45 removed the to sort Needs additional investigation label Dec 17, 2024
@schuylermartin45
Copy link
Collaborator

There currently is a --debug flag but that more or less is for developers of crm convert. It dumps information about the parse tree.

I'm not super thrilled about how I went about logging in crm convert. I think we can retroactively take some lessons from crm bump-recipe and improve the logging dramatically. crm bump-recipe introduced a -v that can be applied globally (i.e. crm -v bump-recipe) to produce verbose logging. But so far only bump-recipe` supports it.

I have gotten this feedback in a few different ways already and I think it is probably time for a change. The tricky bit is I don't want to introduce a ton of logging code to the library components of conda-recipe-manager. I'll have to have a think on this but would love to see what we could do to improve this for users.

@mgorny
Copy link
Author

mgorny commented Dec 17, 2024

Thanks! My point is basically getting some hint on what could be going wrong. The onednn-feedstock case was relatively simple, but e.g. pytorch-cpu-feedstock (especially in the version from conda-forge/pytorch-cpu-feedstock#298) is a true nightmare (and on top of that, that is definitely a package that would truly benefit from rattler-build) :-).

@schuylermartin45
Copy link
Collaborator

Totally understand and completely agree on that point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request source::user-request Requests that come from the community and library users.
Projects
Conda Recipe Manager
Status: Ready
Milestone
Maintenance: Bugs & User Issues
Development

No branches or pull requests
2 participants
@mgorny @schuylermartin45