Should support filename longer than 260 characters

[6ziv]

Well, I can see that it is a limitation from Microsoft Windows.
But MSDN suggests that this limitation may be removed.
Enable Long Paths in Windows 10, Version 1607, and Later
And there are cases we have to use long paths (when handling some really complicated 3rd-party projects).
Maybe ninja could only raise a warning, but do not stop the build, like cmake does?

[jhasse]

Duplicate of #1514.

[jhasse]

#1514 was removed by GitHub because the user who created it was banned or something. No idea why they won't simply remove his posts only ...

Anyway, I will reopen this one then. #1514 can still be accessed via the GitHub API: https://api.github.com/repos/ninja-build/ninja/issues/1514/comments

[jhasse]

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

[LDong-Arm]

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

Hi @jhasse,

From the page you linked, the user needs to edit the registry to enable long path support, and also the application needs to declare it in its own manifest (pasted from that page):

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Presumably adding this to ninja.manifest should work? Or how would this impact compatibility with old Windows?

Thanks.

[LDong-Arm]

Oh #1939 already does that actually.

[emmenlau]

So, what are the next steps for this issue?

[parbo]

RealDiskInterFace uses posix file functions like fopen, and also the *A versions of win32 calls, while the long paths page says only *W versions are affected. Another option might be to prepend \\?\, but this only works for absolute paths.

[parbo]

I made a PR: #2289

[vbaderks]

Hi @parbo

FYI: It seems that the Windows Universal C CRT only calls CreateFileW. It performs internally a codepage\utf-8 conversion to a wchar_t before calling CreateFileW:
C:\Program Files (x86)\Windows Kits\10\Source\10.0.22621.0\ucrt\lowio\open.cpp (method _sopen_nolock at line 833 for example)

[parbo]

Hmm, why didn't it work with fopen then? Maybe the longPathsAware manifest doesn't extend through to linked libraries/dll:s?

…

On Fri, Apr 21, 2023, 10:26 Victor Derks ***@***.***> wrote: Hi @parbo <https://github.com/parbo> FYI: It seems that the Windows Universal C CRT only calls CreateFileW. It performs internally a codepage\utf-8 conversion to a wchar_t before calling CreateFileW: C:\Program Files (x86)\Windows Kits\10\Source\10.0.22621.0\ucrt\lowio\open.cpp (method _sopen_nolock at line 833 for example) — Reply to this email directly, view it on GitHub <#1900 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA52CQ7SL6436E4ZQDQI67LXCJACXANCNFSM4VDY46VA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vbaderks]

@parbo. I am assuming that the longPathsAware manifest option applies to the .exe and is enabled process wide at startup or not. I didn't do a debug session in Ninja, just a debug-trace of a local app that uses fopen on Windows 11.

[parbo]

Maybe only some of the `*A` functions needed replacing, and not the `fopen` et al.

…

On Fri, Apr 21, 2023, 16:16 Victor Derks ***@***.***> wrote: @parbo <https://github.com/parbo>. I am assuming that the longPathsAware manifest option applies to the .exe and is enabled process wide at startup or not. I didn't do a debug session in Ninja, just a debug-trace of a local app that uses fopen on Windows 11. — Reply to this email directly, view it on GitHub <#1900 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA52CQ36XK7XM3Q4M5GONNDXCKJE7ANCNFSM4VDY46VA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Sculas]

I can confirm the above comment as well. Even though I configured Windows to allow long paths, Ninja is failing to build a React Native/Expo app with the New Architecture.

[jhasse]

How is this still an issue?

Because Windows still sucks.

[emmenlau]

How is this still an issue?

Because Windows still sucks.

True. But isn't it time to move on, and merge a solution?

[Sculas]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately.
Here's the workaround I used to compile a RN New Architecture app on Windows:

• Enable Long Paths first! Instructions to do that here.
• Download the latest build artifact of Ninja here and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed:
Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

edit 21/03/2024: replaced expired artifact link with a link to the latest workflow runs

[Jadeon-FPM]

@Sculas
I can confirm this works! Thank you!

I reinstalled Windows before trying this and assumed Windows 11 had long paths enabled by default. It doesn't, which is unfortunate. For anyone coming here due to this issue, make sure to do that first.

[sundayhd]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately. Here's the workaround I used to compile a RN New Architecture app on Windows:

• Download the latest build artifact of Ninja (I used this one) and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed: Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

Thanks !!!
I downloaded ninja from this path:
https://github.com/ninja-build/ninja/suites/18304129173/artifacts/1058811346
I extracted ninja-win.zip to C:\ninja\ (moved ninja.exe to C:\ninja).
I added this one on android/app/build.gradle

android {
    ....
    defaultConfig {
        .....

        // https://github.com/ninja-build/ninja/issues/1900
        externalNativeBuild {
            cmake {
                arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
            }
        }

Works great !! Thanks :-)

[CoorFun]

So, what's the plan for migrating Windows API calls from ANSI to Unicode style?

[XantreDev]

Why did latest release of ninja according to github was in 2022?

[jhasse]

@XantreGodlike Because the last release of ninja was in 2022.

[XantreDev]

Do new releases be planned?

[smalik007]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately. Here's the workaround I used to compile a RN New Architecture app on Windows:

• Enable Long Paths first! Instructions to do that here.
• Download the latest build artifact of Ninja (I used this one) and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed: Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

I tried with extra cmake flags but the problem still persist for me on latest master. The registry flag is enabled for long path , I tried to add some debugging prints to see what's happening and found the flag is not getting set, the ntdll.dll library call for registry flag of long path returns false. Also there are some build warnings about incompatible casting of function types, if that could be any of concern?

[jhasse]

You're using MinGW, @Sculas was referring to the artifact from GitHub Actions which builds with MSVC. Can you try that instead? Maybe it makes a difference because of the UCRT.

[smalik007]

The above built artifact link has expired. But I tried the latest built from the master (ninja-build-artifact), I no longer see the ninja error due long path however my build still fails, Now this could be the MinGW that might be causing it to fail now.
Thanks for the quick response :)

[Rc85]

With #2321 and other PRs now merged, it seems to be fixed! We're just missing an official release, unfortunately. Here's the workaround I used to compile a RN New Architecture app on Windows:

• Enable Long Paths first! Instructions to do that here.
• Download the latest build artifact of Ninja here and put it in C:\ninja\ninja.exe.
• Go to android/app/build.gradle inside of your RN project and add this bit of code inside of android.defaultConfig:

externalNativeBuild {
    cmake {
        arguments "-DCMAKE_MAKE_PROGRAM=C:\\ninja\\ninja.exe", "-DCMAKE_OBJECT_PATH_MAX=1024"
    }
}

To explain why this is needed: Even if you've added C:\ninja to your path, Gradle still uses ninja from the built-in CMake from the Android SDK. Of course, it doesn't strictly need to be in C:\ninja, but chances are that most people have it there already. With just the first argument, it should probably compile already. However, CMake is still warning us about potentially broken builds because the path is too long, so we set it to 1024 to fix that with the second argument.

Lastly, if you want to make this more "contributor-friendly", you can run something like where ninja in your build.gradle, instead of having a hardcoded path like this. Just make sure to check the version beforehand :)

edit 21/03/2024: replaced expired artifact link with a link to the latest workflow runs

The artifact link doesn't link to anywhere to download anything, so I assume you mean the latest version 1.11.1?

I have an RN project and a library (react-native-vision-camera) is giving me this error. I enabled long paths and tried your method by adding the code to my build.gradle and also to the camera's build.gradle, none worked. So maybe it's the ninja.exe that I was using?

The only thing that worked was rename my project path from F:/Projects/<project-name>/<dir1>/<dir2>/<dir3> to F:/p/r/m/a/c, which is ridiculous. It's so sad that in 2024, we still can't deal with long paths.

[jhasse]

There seem to be some issues left (only MinGW build? Without setting the registry value it should still not work?), but some are fixed. I will close this PR, please create new issues with specific failures that you see with 1.12.0 and in which circumstances they occur.

The artifact link doesn't link to anywhere to download anything, so I assume you mean the latest version 1.11.1?

No, it was a master build with some changes not in 1.11.1. Please try with 1.12.0 again.