Cache for LaTeX package locations

[PHPirates]

Fixes #1424

Summary of additions and changes

• Cache locations in memory (as String)
• Multiple calls to kpsewhich may still occur while adding to the cache concurrently

How to test this pull request

Have lots of usepackages, I guess

[stenwessel]

Great! This seems to work on my end, but I would like to wait for @HannahSchellekens to perform the actual benchmarks :)

[HannahSchellekens]

Sure. will get to it soon. First need to fix my own stuff

[HannahSchellekens]

It does work. Sort of.

Of course you've got a bigger startup time when opening the IDE. I have a period of ~10 seconds of typing where it is slow. Afterward it is silky smooth.

However, whenever there is a package that is not yet installed, I do get a MikTeX UAC prompt. Which I think is highly undesirable.

This is definitely an improvement though.

[PHPirates]

@HannahSchellekens Any ideas how we could detect that you have your UAC set to paranoid? Doesn't it go away if you click 'yes' once?

[HannahSchellekens]

@HannahSchellekens Any ideas how we could detect that you have your UAC set to paranoid? Doesn't it go away if you click 'yes' once?

They do not disappear until you install the package, either by compiling or install by the package manager. This means that you will get approximately 3-5 UAC prompts every second you type for every non included package. A large document of mine produces 13 of such prompts when typing a single letter.

I have no clue if it can be detected. I assume it cannot.

[stenwessel]

What exactly is the cause of the UAC prompt? Does kpsewhich attempt (on itself) to install the package?

[HannahSchellekens]

Did a little test in the terminal, amsmath installed, 12many not installed:

It seems that when it is not present, it prompts for an install.

Log:

2020-05-20 12:47:29,798+0200 FATAL kpsewhich - Source: Libraries\MiKTeX\PackageManager\PackageInstaller.cpp
2020-05-20 12:47:29,798+0200 FATAL kpsewhich - Line: 1441
2020-05-20 12:47:29,816+0200 INFO  kpsewhich - finishing with exit code 1
2020-05-20 12:49:26,755+0200 INFO  kpsewhich - starting with command line: "C:\Program Files\MiKTeX 2.9\miktex\bin\x64\kpsewhich.exe" 12many.sty
2020-05-20 12:49:26,906+0200 INFO  kpsewhich - installing package 12many triggered by tex\latex\12many\12many.sty
2020-05-20 12:49:30,697+0200 FATAL kpsewhich - Cannot start MiKTeX package manager.
2020-05-20 12:49:30,697+0200 FATAL kpsewhich - Info: hr="The operation was canceled by the user.

[stenwessel]

I think this is highly undesirable. Anyone with the 'no prompt' package install option enabled in MiKTeX will result in silently installing the package when only opening a document with many uninstalled packages included. I am searching if it is possible to disable auto-install for kpsewhich, but I am not sure if it is possible (also, the fact that it attempts an install on miktex is not documented anywhere)

[PHPirates]

That is strange, it must be one of the hidden MiKTeX features, as kpsewhich on TeX Live certainly does not do that.

You may also already have found https://tex.stackexchange.com/questions/493692/package-detection-without-installation-in-miktex
I doubt that is a good idea. The other option is simply disabling the file reference feature for packages on Windows, I guess.

[stenwessel]

I did not yet come across the SE answer. Any elaborate official documentation I can find on kpsewhich is from TeXLive which indeed does not have this issue (I suppose all packages are already included in the distribution). I am currently looking through the miktex kpsewhich source code (or what it seems to be) to see where exactly the install is triggered and if there is any undocumented way of preventing this.

[stenwessel]

My conclusion from the miktex source:

Miktex does not provide an actual kpsewhich implementation, but instead emulates its behavior as it already has its own file locating system, implemented as Session::FindFile. We want to trigger the file database search (referred to as ls-R in the TeX community) to locate the actual location of the file. Here is the only point where the invocation of the package installer is skipped. However, this only happens when at least one file is found (which is not the behavior we need). The function IsMpmFile function checks whether it is a virtual entry in the database that refers to a package in the repository, but that is not installed on the system of the user.

I think that there would be no way of avoiding this when using kpsewhich with miktex. I will at least submit an issue to the miktex project. But for now we need to find an alternative file location method for Windows (this comes painfully close to implementing the LaTeX SDK we have been talking about for years 😄 ).

[stenwessel]

Tagging the issue MiKTeX/miktex#548 here for reference (we'll see how it unfolds).

[stenwessel]

Well, that was fast! kpsewhich --miktex-disable-installer should be the solution for miktex then :)

[HannahSchellekens]

Can confirm this works!

[HannahSchellekens]

UAC prompts are gone and usability concerns are resolved.

Performance is still not there yet, but that requires some more investigation.