Navigating errors when producing PDFs in ARM Linux containers

[jtkiley]

I spent a few hours wrangling PDF issues on linux-arm64 (Docker devcontainer on macOS). Ultimately, I did get it to work, and here's what I found.

1. The quarto install tinytex platform error does not provide any direction other than installing manually. A link to https://yihui.org/tinytex/#installation would be helpful here.
2. Freshly installed, quarto needs to retrieve multiple packages on first run, and that retrieval fails more often than not (up to six retries needed in my experience). Those error messages about database versions aren't all that helpful, when retrying repeatedly is the fix.
3. For automation, it seems better to use Texlive from package repositories, because it very consistently works and avoids the tlmgr updates that cause the problem above. However, be aware of the issue below.

From #909:

xelatex is indeed the default (but also I believe comes with all versions of tinytex and all downloads of texlive so isn't normally something that should require dynamic download).

This is not the case for Debian packages, and a lot of containers are Debian images. When using a Debian Texlive install, quarto produces a badly misleading error about updating Texlive, which ends up being wasted time, because xelatex missing is the real problem.

This works on Debian:

sudo apt-get update
sudo apt-get install texlive texlive-xetex

Suggestions:

1. (Workaround) Add the install link to the platform error.
2. (Workaround) It's probably more of a Tinytex issue, but handle retries for package installation to overcome the unreliability.
3. (Best fix) Package Tinytex for arm64. Support ARM operating systems rstudio/tinytex-releases#37
4. (Also) Handle the case of missing xelatex better, particularly by not suggesting Texlive upgrades in cases where it's not the problem.

Tasks

Preview Give feedback

1. Handle better xelatex missing on non TinyTeX installation #7681
   enhancement latex +2
2. Add the install link to the platform error.

[cderv]

Thanks for the feedback

The quarto install tinytex platform error does not provide any direction other than installing manually. A link to yihui.org/tinytex/#installation would be helpful here.

I did add the link

Freshly installed, quarto needs to retrieve multiple packages on first run, and that retrieval fails more often than not (up to six retries needed in my experience). Those error messages about database versions aren't all that helpful, when retrying repeatedly is the fix.

When using quarto install tinytex, we are installing in Quarto the bundle TinyTeX (https://github.com/rstudio/tinytex-releases/#releases) which has more packages already bundled that other bundle. If you installed manually following https://yihui.org/tinytex/#installation, you need to set TINYTEX_INSTALLER=TinyTeX so that the same bundle is installed.
Otherwise, the default is TINYTEX_INSTALLER=TinyTeX-1 which has less bundled package version from CTAN, which could end up in some packages installation needed on first run.

Could it be the case ?

Otherwise, I'll be happy to know which packages are missing. We tried to included in TinyTeX bundle the minimum required by Quarto. We may be missing some.

For automation, it seems better to use Texlive from package repositories, because it very consistently works and avoids the tlmgr updates that cause the problem above.

You mean not using TinyTeX, is that it ? You recommend using TexLive directly from the OS package manager ?

Just curious because I believe even when you do that, as long as Auto installation is not opt-out when using Quarto engine, it will still try to update and install missing tools. (https://quarto.org/docs/output-formats/pdf-engine.html#quarto-pdf-engine)

We are aware of a "problem" where Quarto will install by default the monthly release, and we should have a switch to install the daily TinyTeX release instead (which is fresher)

(Also) Handle the case of missing xelatex better, particularly by not suggesting Texlive upgrades in cases where it's not the problem.

We should definitely improve that - thanks !

Though this will probably be the case only when TinyTeX is not used, as I think we do make sure in TinyTeX that xetex is included.

[jtkiley]

@cderv Thanks so much for this.

When using quarto install tinytex, we are installing in Quarto the bundle TinyTeX (https://github.com/rstudio/tinytex-releases/#releases) which has more packages already bundled that other bundle. If you installed manually following https://yihui.org/tinytex/#installation, you need to set TINYTEX_INSTALLER=TinyTeX so that the same bundle is installed.
Otherwise, the default is TINYTEX_INSTALLER=TinyTeX-1 which has less bundled package version from CTAN, which could end up in some packages installation needed on first run.

Could it be the case ?

Otherwise, I'll be happy to know which packages are missing. We tried to included in TinyTeX bundle the minimum required by Quarto. We may be missing some.

I experimented with this a bit and found that the yihui.org script has this check:

if [ $OSNAME != 'Linux' -o $(uname -m) != 'x86_64' -o "$OSTYPE" != 'linux-gnu' ]; then
    TINYTEX_INSTALLER="installer-unix"
fi

In linux-arm64 containers, $(uname -m) is aarch64, so it's using the installer-unix variant. I looked for this after noticing that the environment variable didn't change the number of packages installed.

In my test document (a few pages from a real project), I'm missing these three packages:

• scrartcl.cls
• caption.sty
• bookmark.sty

You mean not using TinyTeX, is that it ? You recommend using TexLive directly from the OS package manager ?

Just curious because I believe even when you do that, as long as Auto installation is not opt-out when using Quarto engine, it will still try to update and install missing tools. (https://quarto.org/docs/output-formats/pdf-engine.html#quarto-pdf-engine)

We are aware of a "problem" where Quarto will install by default the monthly release, and we should have a switch to install the daily TinyTeX release instead (which is fresher)

Ideally, TinyTex would be the best way of doing this, but it fails enough that it's hard to use it for automation. It occasionally fails when the script runs, leaving a broken install, and a subsequent run of the script complains about a texlive directory already being found (so, it can't just be brute forced by trying repeatedly). At a higher frequency, the tlmgr updates by quarto fail with the message about the local database being newer than the package repository. In my testing today, it was less frequent than last time I did it, but I still saw it.

Using Texlive from the Debian package repo works every time (at the cost of transfer/disk space). That's appealing for automation, where no one is watching. It's also appealing for training, where you want your devcontainer.json to just work every time.

Given where these errors happen, I think TinyTex is likely to be the best option once there's full ARM Linux support in releases, removing the need for the script-based install.

[cderv]

Yes the script is from here: https://github.com/rstudio/tinytex/blob/main/tools/install-bin-unix.sh

For aarch64, it will no install a bundle because we don't have one yet (rstudio/tinytex-releases#37)
So it will try to install from source: https://github.com/rstudio/tinytex/blob/3136eefb8386a31ccf6f3cfdd35ce837efb213d1/tools/install-bin-unix.sh#L52-L61

but I wonder if we have a problem there - not sure the installation goes as plan... 🤔

Given where these errors happen, I think TinyTex is likely to be the best option once there's full ARM Linux support in releases, removing the need for the script-based install.

Agreed. I'll see how I can add build for ARM in our Github workflow. Any contribution welcome: We build eveything in https://github.com/rstudio/tinytex/blob/main/.github/workflows/build.yaml