Kpsewhich times out on Windows when using texlive-full

[FrankMilesFrms]

Type of JetBrains IDE (IntelliJ, PyCharm, etc.) and version

IntelliJ IDEA 2024.1.4 (Ultimate Edition)

Operating System

Windows 10

TeXiFy IDEA version

0.9.7

What I did (steps to reproduce)

I install livetex in D:/softwares/latex.

When I created a new project, Idea displays an underscore error.

In fact, the compiler knows where the 'article' is and outputs the result correctly, but it also prompts an error.

Minimal example to reproduce the problem

\documentclass{article}

\begin{document}
    
\end{document}

Expected behavior

There should be no error message in this article

Actual behavior

(if applicable) The full stacktrace of the exception thrown

bibtex main
This is BibTeX, Version 0.99d (TeX Live 2024)
The top-level auxiliary file: main.aux
I found no \citation commands---while reading file main.aux
I found no \bibdata command---while reading file main.aux
I found no \bibstyle command---while reading file main.aux
(There were 3 error messages)

进程已结束，退出代码为 2
(The process has ended and the exit code is 2)

[PHPirates]

D:/softwares/latex

Is this directory added to your PATH environment variable?
Did you change anything in the run configuration to make sure pdflatex is found?

[FrankMilesFrms]

D：/softwares/latex

Yes, It has been added to user environment variables and system environment variables.(also include SumatraPDF)

[PHPirates]

Thanks, so if you run kpsewhich article.sty in a command prompt then you get the correct path?
If you added the correct installation paths to your PATH in the Windows environment variables settings, then filling in the path to the compiler and sumatra is not necessary in the run configuration.

[FrankMilesFrms]

Thanks, so if you run kpsewhich article.sty in a command prompt then you get the correct path?
If you added the correct installation paths to your PATH in the Windows environment variables settings, then filling in the path to the compiler and sumatra is not necessary in the run configuration.

Yes, I get the correct path. But the problem is still there, whether I fill it or not.

[FrankMilesFrms]

If I fill in the absolute path of article, idea will not prompt an error. But whether it's an absolute path or not doesn't seem to affect the final result.

[PHPirates]

Thanks for the screenshots, certainly something weird is going on. Apparently IntelliJ cannot run system commands properly, even though the command prompt can.

How did you install IntelliJ?
Could you go to Help > Diagnostic Tools > Debug Log Settings and add #nl.hannahsten.texifyidea.util.Log then reproduce the issue and upload idea.log here? (help > show log)

[FrankMilesFrms]

Thanks for the screenshots, certainly something weird is going on. Apparently IntelliJ cannot run system commands properly, even though the command prompt can.

How did you install IntelliJ? Could you go to Help > Diagnostic Tools > Debug Log Settings and add #nl.hannahsten.texifyidea.util.Log then reproduce the issue and upload idea.log here? (help > show log)

I install IDEA in the path "D:\Softwares\JetBrains\IntelliJ IDEA Ultimate\bin"
idea.log

[PHPirates]

Thanks, the issue is that the command to find the package location times out. If you run this command, how long does it take, or does it hang?

kpsewhich -expand-path .;C:/Users/Hello/.texlive2024/texmf-config/tex/kpsewhich//;C:/Users/Hello/.texlive2024/texmf-var/tex/kpsewhich//;C:/Users/Hello/texmf/tex/kpsewhich//;!!D:/Softwares/TeXLive/texmf-local/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/kpsewhich//;C:/Users/Hello/.texlive2024/texmf-config/tex/generic//;C:/Users/Hello/.texlive2024/texmf-var/tex/generic//;C:/Users/Hello/texmf/tex/generic//;!!D:/Softwares/TeXLive/texmf-local/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/generic//;C:/Users/Hello/.texlive2024/texmf-config/tex/latex//;C:/Users/Hello/.texlive2024/texmf-var/tex/latex//;C:/Users/Hello/texmf/tex/latex//;!!D:/Softwares/TeXLive/texmf-local/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/latex//;C:/Users/Hello/.texlive2024/texmf-config/tex///;C:/Users/Hello/.texlive2024/texmf-var/tex///;C:/Users/Hello/texmf/tex///;!!D:/Softwares/TeXLive/texmf-local/tex///;!!D:/Softwares/TeXLive/2024/texmf-config/tex///;!!D:/Softwares/TeXLive/2024/texmf-var/tex///;!!D:/Softwares/TeXLive/2024/texmf-dist/tex///;.;C:/Users/Hello/.texlive2024/texmf-config/bibtex/bib//;C:/Users/Hello/.texlive2024/texmf-var/bibtex/bib//;C:/Users/Hello/texmf/bibtex/bib//;!!D:/Softwares/TeXLive/texmf-local/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-config/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-var/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-dist/bibtex/bib//

Interesting is that most paths point to a texmf tree in C:/Users/Hello/

FYI These paths are coming from

kpsewhich -show-path=tex

[FrankMilesFrms]

Thanks, the issue is that the command to find the package location times out. If you run this command, how long does it take, or does it hang?

kpsewhich -expand-path .;C:/Users/Hello/.texlive2024/texmf-config/tex/kpsewhich//;C:/Users/Hello/.texlive2024/texmf-var/tex/kpsewhich//;C:/Users/Hello/texmf/tex/kpsewhich//;!!D:/Softwares/TeXLive/texmf-local/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/kpsewhich//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/kpsewhich//;C:/Users/Hello/.texlive2024/texmf-config/tex/generic//;C:/Users/Hello/.texlive2024/texmf-var/tex/generic//;C:/Users/Hello/texmf/tex/generic//;!!D:/Softwares/TeXLive/texmf-local/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/generic//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/generic//;C:/Users/Hello/.texlive2024/texmf-config/tex/latex//;C:/Users/Hello/.texlive2024/texmf-var/tex/latex//;C:/Users/Hello/texmf/tex/latex//;!!D:/Softwares/TeXLive/texmf-local/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-config/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-var/tex/latex//;!!D:/Softwares/TeXLive/2024/texmf-dist/tex/latex//;C:/Users/Hello/.texlive2024/texmf-config/tex///;C:/Users/Hello/.texlive2024/texmf-var/tex///;C:/Users/Hello/texmf/tex///;!!D:/Softwares/TeXLive/texmf-local/tex///;!!D:/Softwares/TeXLive/2024/texmf-config/tex///;!!D:/Softwares/TeXLive/2024/texmf-var/tex///;!!D:/Softwares/TeXLive/2024/texmf-dist/tex///;.;C:/Users/Hello/.texlive2024/texmf-config/bibtex/bib//;C:/Users/Hello/.texlive2024/texmf-var/bibtex/bib//;C:/Users/Hello/texmf/bibtex/bib//;!!D:/Softwares/TeXLive/texmf-local/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-config/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-var/bibtex/bib//;!!D:/Softwares/TeXLive/2024/texmf-dist/bibtex/bib//

Interesting is that most paths point to a texmf tree in C:/Users/Hello/

FYI These paths are coming from

kpsewhich -show-path=tex

The results are displayed very quickly when running in cmd.

[FrankMilesFrms]

tex

No 'tex' folder in 'C:\Users\Hello.texlive2024'
C:\Users\Hello.texlive2024 is only 64kb

No texmf-var folder , Only exist 'texmf-config'

[snbudzisz]

Hi. Idk if this is what's causing yours, but I had the same issue recently. IntelliJ compiled everything just fine, even though it underlined article and didn't think it could find the article class. Other packages were not underlined.

I reinstalled MiKTeX (not sure if that was necessary), and the issue was still there.
Then I checked Windows's environment variables. The MiKTeX installation added two values to the Path variable, including %MiKTeX% and a bin file location (yellow highlight in photo).

But it didn't make a variable for the %MiKTeX% value it added to the path. So I added it as a new variable, using the same bin file location in the first photo.

Then I relaunched IntelliJ (may have restarted computer, too? can't remember) and the issue is gone now.

I know you're using TeXLive, and not MiKTeX, but my issue was nearly identical, and we are both using Windows. What's funny is that, even though IntelliJ told me it couldn't find the article class, it was still finding other packages.
And I wouldn't have expected the undefined %MiKTeX% to have caused the problem if the bin file was added to the path directly, but making the new variable above did fix it.

[FrankMilesFrms]

Hi. Idk if this is what's causing yours, but I had the same issue recently. IntelliJ compiled everything just fine, even though it underlined article and didn't think it could find the article class. Other packages were not underlined.

I reinstalled MiKTeX (not sure if that was necessary), and the issue was still there. Then I checked Windows's environment variables. The MiKTeX installation added two values to the Path variable, including %MiKTeX% and a bin file location (yellow highlight in photo).

But it didn't make a variable for the %MiKTeX% value it added to the path. So I added it as a new variable, using the same bin file location in the first photo.

Then I relaunched IntelliJ (may have restarted computer, too? can't remember) and the issue is gone now.

I know you're using TeXLive, and not MiKTeX, but my issue was nearly identical, and we are both using Windows. What's funny is that, even though IntelliJ told me it couldn't find the article class, it was still finding other packages. And I wouldn't have expected the undefined %MiKTeX% to have caused the problem if the bin file was added to the path directly, but making the new variable above did fix it.

Thanks for replying, but it doesn't work.
When I reboot my computer, the error is still there, nothing has changed.

[PHPirates]

The results are displayed very quickly when running in cmd.

Thanks, that is indeed very strange. The current timeout is 3 seconds, and somehow intellij cannot execute the command within 3 seconds but you can.
I increased the timeout here to 60 seconds, could you try installing this version, then reproduce the problem and upload idea.log again?
https://plugins.jetbrains.com/plugin/download?rel=true&updateId=617381

[FrankMilesFrms]

The results are displayed very quickly when running in cmd.

Thanks, that is indeed very strange. The current timeout is 3 seconds, and somehow intellij cannot execute the command within 3 seconds but you can. I increased the timeout here to 60 seconds, could you try installing this version, then reproduce the problem and upload idea.log again? https://plugins.jetbrains.com/plugin/download?rel=true&updateId=617381

OK

[FrankMilesFrms]

The results are displayed very quickly when running in cmd.

Thanks, that is indeed very strange. The current timeout is 3 seconds, and somehow intellij cannot execute the command within 3 seconds but you can. I increased the timeout here to 60 seconds, could you try installing this version, then reproduce the problem and upload idea.log again? https://plugins.jetbrains.com/plugin/download?rel=true&updateId=617381

I reinstalled and tested it, and the good news is the problem is fixed. In addition, when Idea loading project, it loses response for up to 60 seconds.

idea.log

[PHPirates]

Thanks, something is clearly not working properly because the -expand-path command still times out, while other kpsewhich commands only take half a second.
I'm a bit confused why it can find the packages now, because the command timed out (blocking the UI for 60 seconds) so it shouldn't find anything.
What you can at least try, is get rid of the references to C:\Users\Hello, not sure where that is defined, maybe in some environment variable like TEXMFHOME?

[FrankMilesFrms]

Thanks, something is clearly not working properly because the command still times out, while other kpsewhich commands only take half a second. I'm a bit confused why it can find the packages now, because the command timed out (blocking the UI for 60 seconds) so it shouldn't find anything. What you can at least try, is get rid of the references to , not sure where that is defined, maybe in some environment variable like TEXMFHOME?-expand-path``C:\Users\Hello

First I removed all references from the Environment Variables and then re-installed the TeXiFy plugin you just provided, this time there also was no message that 'article' could not be found.

In the end, running kpsewhich -expand-path C:\Users\Hello will only yield c:/Users/Hello

[PHPirates]

Sorry I don't quite understand. What environment variables do you have set now, and is the issues still there if you use the stable 0.9.7 version of TeXiFy?

I still have no idea what could have caused the timeouts.

[FrankMilesFrms]

Sorry I don't quite understand. What environment variables do you have set now, and is the issues still there if you use the stable 0.9.7 version of TeXiFy?

I still have no idea what could have caused the timeouts.

I removed D:\Softwares\TeXLive\2024\bin\

The issue is still there when I use the stable 0.9.7 version of TeXiFy

[tristankretzer]

I have the same problem (Windows 10, IntelliJ IDEA 2024.2.4 (Ultimate Edition), TeXiFy IDEA 0.9.7, TeX Live 2023 full with sources and docs).

I was able to attach a debugger and can confirm the problem occurs due to running in the timeout when expanding the paths. Manually running kpsewhich -expand-path [...] showed that the execution time on my system is consistently more than 4 seconds.

As to why this happens: I'm pretty sure it is due to me having installed the full scheme. Some timings I measured on a SATA SSD:

Scheme	Execution time in seconds
minimal	< 0.05
basic	~0.17
small	~0.31
medium	~0.75
full	~4.5

Probably the easiest way to fix this is to add an optional parameter timeout to runCommand() and pass it to runCommandWithExitCode(). If you like, I can open a pull request to add this change.

[PHPirates]

Thanks a lot for the research! Maybe the problem is specific to TeX Live full scheme on Windows only, as I don't see any problems with the full scheme on Linux. I will try to reproduce this. [Edit] On my Windows the command kpsewhich -expand-path $(kpsewhich -show-path=tex) takes 2.7 seconds, so within timeout of 3 seconds but only just

I don't quite understand why kpsewhich could take so long, but if it's only 5 seconds we could indeed change the timeout (maybe only for Windows?) and make sure it runs in the background so it doesn't block the UI. It is consistent at least with the observation above (#3653 (comment)) that the problem was fixed with increased timeout.

A pull request would be appreciated! Then we can finetune from there.

[tristankretzer]

I created the pull request #3727.

It is a rather simple (hot)fix due to another discovery I had when trying to fix this: It seems that the command runner does not recognize the command has stopped, i.e. it runs forever if you don't kill it manually as it happens in nl/hannahsten/texifyidea/util/SystemEnvironment.kt:116. This happens only with the path expansion, not the two commands before.

Are you able to verify on your side if the path expansion does not stop on itself, i.e. runs in the timeout and is killed? (Just to check if it is a me problem or a problem in general before trying to debug it.)

[tristankretzer]

So, I did some more digging and was able to independently (from the plugin) reproduce the behavior, i.e. Process.waitFor() blocking forever if no timeout is set. Looking in the documentation of Process leads to the probable issue:

Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the process may cause the process to block, or even deadlock.
(https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/Process.html)

Reading the process' output before calling Process.waitFor() does not lead to locking forever, i.e. problem solved.

I'll try to think about a solution and will update the pull request if I find one.

[PHPirates]

Yes, now I can reproduce the problem exactly as you describe: if I put 3 or 30 seconds as timeout, in both cases the command will run for the 3 or 30 seconds, even though the full output is already available much earlier - and when the output stream is read before the waitFor, this issue does not occur. Well found!
I was definitely not thinking about buffer sizes when writing that code. Also explains why it especially happens with texlive full on Windows, I guess