Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandoc installed and used is higher than the one supported by Quarto #58

Closed
1 task done
tuncbkose opened this issue Sep 2, 2024 · 7 comments · Fixed by #59
Closed
1 task done

Pandoc installed and used is higher than the one supported by Quarto #58

tuncbkose opened this issue Sep 2, 2024 · 7 comments · Fixed by #59
Labels
bug Something isn't working

Comments

@tuncbkose
Copy link

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

I've described at quarto-dev/quarto-cli#10682 an issue I've been having with the conda-forge version of quarto.

Issue

With using quarto render file.qmd --to pdf on

# Test

![](file.png)

when file.png exists in the directory, LaTeX compilation fails with

ERROR: 
compilation failed- error
Undefined control sequence.
l.227 \pandocbounded
                    {\includegraphics[keepaspectratio]{

It looks to me like the \pandocbounded function added to pandoc on 3.2.1 is not defined in some step of the rendering.

Error is given on

Quarto 1.5.55
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.1: OK
      Dart Sass version 1.58.3: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.5.55
      Path: /.../micromamba/envs/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/bin
      Version: 2021

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.5 (Conda)
      Path: /.../micromamba/envs/quarto/bin/python
      Jupyter: 5.7.2
      Kernels: python3, julia-1.10

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

but with downgrading via mamba install "pandoc<3.2" quarto everything is ok:

Quarto 1.4.557
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.12: OK
      Dart Sass version 1.58.3: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.557
      Path: /.../micromamba/envs/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/bin
      Version: 2021

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.5 (Conda)
      Path: /.../micromamba/envs/quarto/bin/python
      Jupyter: 5.7.2
      Kernels: python3, julia-1.10

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

Extra

The developers over there also highlighted that the feedstock should be following dependencies outlined in here though perhaps that is a separate issue.

Installed packages

_libgcc_mutex                  0.1             conda_forge         conda-forge
  _openmp_mutex                  4.5             2_gnu               conda-forge
  anyio                          4.4.0           pyhd8ed1ab_0        conda-forge
  argon2-cffi                    23.1.0          pyhd8ed1ab_0        conda-forge
  argon2-cffi-bindings           21.2.0          py312h98912ed_4     conda-forge
  arrow                          1.3.0           pyhd8ed1ab_0        conda-forge
  asttokens                      2.4.1           pyhd8ed1ab_0        conda-forge
  async-lru                      2.0.4           pyhd8ed1ab_0        conda-forge
  attrs                          24.2.0          pyh71513ae_0        conda-forge
  babel                          2.14.0          pyhd8ed1ab_0        conda-forge
  beautifulsoup4                 4.12.3          pyha770c72_0        conda-forge
  bleach                         6.1.0           pyhd8ed1ab_0        conda-forge
  brotli-python                  1.1.0           py312h30efb56_1     conda-forge
  bzip2                          1.0.8           h4bc722e_7          conda-forge
  ca-certificates                2024.7.4        hbcca054_0          conda-forge
  cached-property                1.5.2           hd8ed1ab_1          conda-forge
  cached_property                1.5.2           pyha770c72_1        conda-forge
  certifi                        2024.7.4        pyhd8ed1ab_0        conda-forge
  cffi                           1.17.0          py312h1671c18_0     conda-forge
  charset-normalizer             3.3.2           pyhd8ed1ab_0        conda-forge
  click                          8.1.7           unix_pyh707e725_0   conda-forge
  comm                           0.2.2           pyhd8ed1ab_0        conda-forge
  dart-sass                      1.58.3          ha770c72_1          conda-forge
  debugpy                        1.8.5           py312hca68cad_0     conda-forge
  decorator                      5.1.1           pyhd8ed1ab_0        conda-forge
  defusedxml                     0.7.1           pyhd8ed1ab_0        conda-forge
  deno                           1.41.0          hfc7925d_0          conda-forge
  deno-dom                       0.1.35          hd9586b0_1          conda-forge
  entrypoints                    0.4             pyhd8ed1ab_0        conda-forge
  esbuild                        0.19.2          ha770c72_0          conda-forge
  exceptiongroup                 1.2.2           pyhd8ed1ab_0        conda-forge
  executing                      2.0.1           pyhd8ed1ab_0        conda-forge
  fqdn                           1.5.1           pyhd8ed1ab_0        conda-forge
  greenlet                       3.0.3           py312h30efb56_0     conda-forge
  h11                            0.14.0          pyhd8ed1ab_0        conda-forge
  h2                             4.1.0           pyhd8ed1ab_0        conda-forge
  hpack                          4.0.0           pyh9f0ad1d_0        conda-forge
  httpcore                       1.0.5           pyhd8ed1ab_0        conda-forge
  httpx                          0.27.0          pyhd8ed1ab_0        conda-forge
  hyperframe                     6.0.1           pyhd8ed1ab_0        conda-forge
  idna                           3.7             pyhd8ed1ab_0        conda-forge
  importlib-metadata             8.2.0           pyha770c72_0        conda-forge
  importlib_metadata             8.2.0           hd8ed1ab_0          conda-forge
  importlib_resources            6.4.0           pyhd8ed1ab_0        conda-forge
  ipykernel                      6.29.5          pyh3099207_0        conda-forge
  ipython                        8.26.0          pyh707e725_0        conda-forge
  ipywidgets                     8.1.3           pyhd8ed1ab_0        conda-forge
  isoduration                    20.11.0         pyhd8ed1ab_0        conda-forge
  jedi                           0.19.1          pyhd8ed1ab_0        conda-forge
  jinja2                         3.1.4           pyhd8ed1ab_0        conda-forge
  json5                          0.9.25          pyhd8ed1ab_0        conda-forge
  jsonpointer                    3.0.0           py312h7900ff3_0     conda-forge
  jsonschema                     4.23.0          pyhd8ed1ab_0        conda-forge
  jsonschema-specifications      2023.12.1       pyhd8ed1ab_0        conda-forge
  jsonschema-with-format-nongpl  4.23.0          hd8ed1ab_0          conda-forge
  jupyter                        1.0.0           pyhd8ed1ab_10       conda-forge
  jupyter-cache                  1.0.0           pyhd8ed1ab_0        conda-forge
  jupyter-lsp                    2.2.5           pyhd8ed1ab_0        conda-forge
  jupyter_client                 8.6.2           pyhd8ed1ab_0        conda-forge
  jupyter_console                6.6.3           pyhd8ed1ab_0        conda-forge
  jupyter_core                   5.7.2           py312h7900ff3_0     conda-forge
  jupyter_events                 0.10.0          pyhd8ed1ab_0        conda-forge
  jupyter_server                 2.14.2          pyhd8ed1ab_0        conda-forge
  jupyter_server_terminals       0.5.3           pyhd8ed1ab_0        conda-forge
  jupyterlab                     4.2.4           pyhd8ed1ab_0        conda-forge
  jupyterlab_pygments            0.3.0           pyhd8ed1ab_1        conda-forge
  jupyterlab_server              2.27.3          pyhd8ed1ab_0        conda-forge
  jupyterlab_widgets             3.0.11          pyhd8ed1ab_0        conda-forge
  keyutils                       1.6.1           h166bdaf_0          conda-forge
  krb5                           1.21.3          h659f571_0          conda-forge
  ld_impl_linux-64               2.40            hf3520f5_7          conda-forge
  libedit                        3.1.20191231    he28a2e2_2          conda-forge
  libexpat                       2.6.2           h59595ed_0          conda-forge
  libffi                         3.4.2           h7f98852_5          conda-forge
  libgcc-ng                      14.1.0          h77fa898_0          conda-forge
  libgomp                        14.1.0          h77fa898_0          conda-forge
  libnsl                         2.0.1           hd590300_0          conda-forge
  libsodium                      1.0.18          h36c2ea0_1          conda-forge
  libsqlite                      3.46.0          hde9e2c9_0          conda-forge
  libstdcxx-ng                   14.1.0          hc0a3c3a_0          conda-forge
  libuuid                        2.38.1          h0b41bf4_0          conda-forge
  libxcrypt                      4.4.36          hd590300_1          conda-forge
  libzlib                        1.3.1           h4ab18f5_1          conda-forge
  markupsafe                     2.1.5           py312h98912ed_0     conda-forge
  matplotlib-inline              0.1.7           pyhd8ed1ab_0        conda-forge
  mistune                        3.0.2           pyhd8ed1ab_0        conda-forge
  nbclient                       0.10.0          pyhd8ed1ab_0        conda-forge
  nbconvert                      7.16.4          hd8ed1ab_1          conda-forge
  nbconvert-core                 7.16.4          pyhd8ed1ab_1        conda-forge
  nbconvert-pandoc               7.16.4          hd8ed1ab_1          conda-forge
  nbformat                       5.10.4          pyhd8ed1ab_0        conda-forge
  ncurses                        6.5             h59595ed_0          conda-forge
  nest-asyncio                   1.6.0           pyhd8ed1ab_0        conda-forge
  notebook                       7.2.1           pyhd8ed1ab_0        conda-forge
  notebook-shim                  0.2.4           pyhd8ed1ab_0        conda-forge
  openssl                        3.3.1           h4bc722e_2          conda-forge
  overrides                      7.7.0           pyhd8ed1ab_0        conda-forge
  packaging                      24.1            pyhd8ed1ab_0        conda-forge
  pandoc                         3.2.1           ha770c72_0          conda-forge
  pandocfilters                  1.5.0           pyhd8ed1ab_0        conda-forge
  parso                          0.8.4           pyhd8ed1ab_0        conda-forge
  pexpect                        4.9.0           pyhd8ed1ab_0        conda-forge
  pickleshare                    0.7.5           py_1003             conda-forge
  pip                            24.2            pyhd8ed1ab_0        conda-forge
  pkgutil-resolve-name           1.3.10          pyhd8ed1ab_1        conda-forge
  platformdirs                   4.2.2           pyhd8ed1ab_0        conda-forge
  prometheus_client              0.20.0          pyhd8ed1ab_0        conda-forge
  prompt-toolkit                 3.0.47          pyha770c72_0        conda-forge
  prompt_toolkit                 3.0.47          hd8ed1ab_0          conda-forge
  psutil                         6.0.0           py312h9a8786e_0     conda-forge
  ptyprocess                     0.7.0           pyhd3deb0d_0        conda-forge
  pure_eval                      0.2.3           pyhd8ed1ab_0        conda-forge
  pycparser                      2.22            pyhd8ed1ab_0        conda-forge
  pygments                       2.18.0          pyhd8ed1ab_0        conda-forge
  pysocks                        1.7.1           pyha2e5f31_6        conda-forge
  python                         3.12.5          h2ad013b_0_cpython  conda-forge
  python-dateutil                2.9.0           pyhd8ed1ab_0        conda-forge
  python-fastjsonschema          2.20.0          pyhd8ed1ab_0        conda-forge
  python-json-logger             2.0.7           pyhd8ed1ab_0        conda-forge
  python_abi                     3.12            4_cp312             conda-forge
  pytz                           2024.1          pyhd8ed1ab_0        conda-forge
  pyyaml                         6.0.2           py312h41a817b_0     conda-forge
  pyzmq                          26.1.0          py312h7ab5c7e_0     conda-forge
  qtconsole-base                 5.5.2           pyha770c72_0        conda-forge
  qtpy                           2.4.1           pyhd8ed1ab_0        conda-forge
  quarto                         1.5.55          ha770c72_0          conda-forge
  readline                       8.2             h8228510_1          conda-forge
  referencing                    0.35.1          pyhd8ed1ab_0        conda-forge
  requests                       2.32.3          pyhd8ed1ab_0        conda-forge
  rfc3339-validator              0.1.4           pyhd8ed1ab_0        conda-forge
  rfc3986-validator              0.1.1           pyh9f0ad1d_0        conda-forge
  rpds-py                        0.20.0          py312hf008fa9_0     conda-forge
  send2trash                     1.8.3           pyh0d859eb_0        conda-forge
  setuptools                     72.1.0          pyhd8ed1ab_0        conda-forge
  six                            1.16.0          pyh6c4a22f_0        conda-forge
  sniffio                        1.3.1           pyhd8ed1ab_0        conda-forge
  soupsieve                      2.5             pyhd8ed1ab_1        conda-forge
  sqlalchemy                     2.0.32          py312h41a817b_0     conda-forge
  stack_data                     0.6.2           pyhd8ed1ab_0        conda-forge
  tabulate                       0.9.0           pyhd8ed1ab_1        conda-forge
  terminado                      0.18.1          pyh0d859eb_0        conda-forge
  tinycss2                       1.3.0           pyhd8ed1ab_0        conda-forge
  tk                             8.6.13          noxft_h4845f30_101  conda-forge
  tomli                          2.0.1           pyhd8ed1ab_0        conda-forge
  tornado                        6.4.1           py312h9a8786e_0     conda-forge
  traitlets                      5.14.3          pyhd8ed1ab_0        conda-forge
  types-python-dateutil          2.9.0.20240316  pyhd8ed1ab_0        conda-forge
  typing-extensions              4.12.2          hd8ed1ab_0          conda-forge
  typing_extensions              4.12.2          pyha770c72_0        conda-forge
  typing_utils                   0.1.0           pyhd8ed1ab_0        conda-forge
  typst                          0.11.0          he8a937b_0          conda-forge
  tzdata                         2024a           h0c530f3_0          conda-forge
  uri-template                   1.3.0           pyhd8ed1ab_0        conda-forge
  urllib3                        2.2.2           pyhd8ed1ab_1        conda-forge
  wcwidth                        0.2.13          pyhd8ed1ab_0        conda-forge
  webcolors                      24.6.0          pyhd8ed1ab_0        conda-forge
  webencodings                   0.5.1           pyhd8ed1ab_2        conda-forge
  websocket-client               1.8.0           pyhd8ed1ab_0        conda-forge
  wheel                          0.44.0          pyhd8ed1ab_0        conda-forge
  widgetsnbextension             4.0.11          pyhd8ed1ab_0        conda-forge
  xz                             5.2.6           h166bdaf_0          conda-forge
  yaml                           0.2.5           h7f98852_2          conda-forge
  zeromq                         4.3.5           h75354e8_4          conda-forge
  zipp                           3.19.2          pyhd8ed1ab_0        conda-forge
  zstandard                      0.23.0          py312h3483029_0     conda-forge
  zstd                           1.5.6           ha6fb4c9_0          conda-forge

Environment info

libmamba version : 1.5.8
     micromamba version : 1.5.8
           curl version : libcurl/8.6.0 OpenSSL/3.2.1 zlib/1.2.13 zstd/1.5.5 libssh2/1.11.0 nghttp2/1.58.0
     libarchive version : libarchive 3.7.2 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.5
       envs directories : /.../micromamba/envs
          package cache : /.../micromamba/pkgs
                          /.../.mamba/pkgs
            environment : quarto (active)
           env location : /.../micromamba/envs/quarto
      user config files : /.../.mambarc
 populated config files : /.../.condarc
       virtual packages : __unix=0=0
                          __linux=6.8.0=0
                          __glibc=2.35=0
                          __archspec=1=x86_64-v4
               channels : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/nodefaults/linux-64
                          https://conda.anaconda.org/nodefaults/noarch
       base environment : /.../micromamba
               platform : linux-64
@tuncbkose tuncbkose added the bug Something isn't working label Sep 2, 2024
@cderv
Copy link

cderv commented Sep 2, 2024

Let me details the problem here.

Quarto is supposed to work and use Pandoc 3.2 as defined in configuration file
https://github.com/quarto-dev/quarto-cli/blob/295111c10fe3339125b8d928a4630a3b1085744e/configuration#L14

and conda-forge read this

{% set pandoc_version = load_file_regex(load_file='configuration', regex_pattern='export PANDOC=(\S+)') %}

So it may consider it as a usual version spec, and probably thinking that Pandoc 3.2.1 is an ok version. But it is not. Quarto expect this 3.2 version only.

Bundling for conda-forge requires a pinned version of dependency so that the right one is used. Hopefully this is possible.

Confirmation that 3.2.1 is the one installed.

See log for conda install quarto mentionning pandoc 3.2.1

The following NEW packages will be INSTALLED:

  dart-sass          conda-forge/win-64::dart-sass-1.58.3-h57928b3_1
  deno               conda-forge/win-64::deno-1.41.0-h1f5608b_0
  deno-dom           conda-forge/win-64::deno-dom-0.1.35-h8b8d39b_1
  esbuild            conda-forge/win-64::esbuild-0.23.1-h57928b3_0
  libexpat           conda-forge/win-64::libexpat-2.6.2-h63175ca_0
  libsqlite          conda-forge/win-64::libsqlite-3.46.0-h2466b09_0
  libzlib            conda-forge/win-64::libzlib-1.2.13-h2466b09_6
  pandoc             conda-forge/win-64::pandoc-3.2.1-h57928b3_0
  python_abi         conda-forge/win-64::python_abi-3.12-5_cp312
  quarto             conda-forge/win-64::quarto-1.5.55-h57928b3_0
  typst              conda-forge/win-64::typst-0.11.0-h975169c_0
  ucrt               conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
  vc14_runtime       conda-forge/win-64::vc14_runtime-14.40.33810-hcc2c482_20

whereas Quarto is configuring with 3.2

@cderv cderv changed the title \pandocbounded error when rendering pdf Pandoc installed and used is higher than the one supported by Quarto Sep 2, 2024
@mfisher87
Copy link
Member

mfisher87 commented Sep 5, 2024

I believe the build dependency version is set correctly here to pandoc 3.2:

- pandoc {{ pandoc_version.group(1) if pandoc_version else "" }}

But then here we set the runtime dependency to ... I think.... pandoc >=3.2.1,<3.2.2.0a0

- {{ pin_compatible("pandoc", max_pin="x.x.x") }}

from conda_build.jinja_context import pin_compatible
from conda_build.metadata import MetaData

metadata = MetaData.fromdict({
    'requirements': {'build': ['pandoc 3.2']},
    'package': {'name': 'foo', 'version': '0.1.0'},
})

print(pin_compatible(metadata, "pandoc", max_pin="x.x.x"))

If I understand correctly, the runtime specification we want is "exactly 3.2, not 3.2.1 or anything else". I don't think it's possible to generate the specification we want with pin_compatible. I think we need to change the runtime to dependency to exactly match the build dependency, i.e. pandoc {{ pandoc_version.group(1) if pandoc_version else "" }}.

Does this sound correct to everyone?

@mfisher87
Copy link
Member

@cderv based on this comment by you in the upstream repo, do you think we need to apply the same change to every runtime dependency whose version we read from configuration?

It is probably what is happening in conda feedstock: it is not correctly converting this value to a specific version for it to be used during conda installation. It is up to the tool using quarto's configuration file to do the right thing.

@cderv
Copy link

cderv commented Sep 5, 2024

If I understand correctly, the runtime specification we want is "exactly 3.2, not 3.2.1 or anything else"

yes exactly.

do you think we need to apply the same change to every runtime dependency whose version we read from configuration

Quarto is built and tested in main repo against the exact version of the dependency. There is no warranty that it will work with other version, including newer. Which is proven by the Pandoc bug of a new version.

So yes I think we need to make sure that the configuration are exact match.

We do this for now.

requirements:
build:
- deno {{ deno_version.group(1) if deno_version else "" }}
- deno-dom {{ deno_dom_version.group(1) if deno_dom_version else "" }}
- esbuild
- pandoc {{ pandoc_version.group(1) if pandoc_version else "" }}
- typst {{ typst_version.group(1) if typst_version else "" }}
run:
# Deno, deno-dom and pandoc have proven to be fickle
# regarding Quarto's source code. The pinning here
# is not so much that there are binary compatibility
# issues, but just that these dependencies are only
# known to work in a pretty narrow range.
- {{ pin_compatible("deno", max_pin="x.x.x") }}
- {{ pin_compatible("pandoc", max_pin="x.x.x") }}
# we vendor deno-dom JS dependencies, so it is important
# that the version at runtime matches the one at build time
- {{ pin_compatible("deno-dom", max_pin="x.x.x") }}
- typst {{ typst_version.group(1) if typst_version else "" }}
- esbuild
- dart-sass

I don't know enough about conda build process but why do we have different version for build and run ?
Is there a way to specific exact match ?

@mfisher87
Copy link
Member

why do we have different version for build and run ?

I'm not sure, many of those choices were made before I started as a maintainer! Perhaps the exact match wasn't available on conda-forge.

Is there a way to specific exact match ?

We are currently doing that with typst; I think that's what we should do with pandoc as well. Should we also apply this change to deno and deno-dom?

@cderv
Copy link

cderv commented Sep 5, 2024

Somehow for deno and deno_dom is works as is it seems. correct version seems to be installed. Maybe because it don't have the a.b(.c) version problem.

@mfisher87
Copy link
Member

That certainly sounds plausible!

For now, let's ensure that the pandoc runtime pin exactly matches the build pin, which is read exactly from configuration. I'll try and open a PR after work today. We may want to pull the broken versions. But I think we should figure out what's going on with #53 too before asking the admins to pull broken packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants