Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use some kind of synchronization primitive when copying files for prebuilt detection #4234

Open
mmitche opened this issue Mar 15, 2024 · 3 comments
Labels
area-infra Source-build infrastructure and reporting

Comments

@mmitche
Copy link
Member

mmitche commented Mar 15, 2024

https://github.com/dotnet/installer/blob/4106448ffae0ac01c5cdec31608b7904aaf1ef05/src/SourceBuild/content/repo-projects/Directory.Build.targets#L495-L511 copies restored packages from the inner build to a global cache. Because many builds restore the same packages, it's very possible to race and cause locking issues.

Retries aren't a great solution here because they cause warnings, which can then get picked up as errors. Instead, if a global shared folder is desirable, a custom copy task should be used that handles synchronization between parallel build processes.

@mmitche mmitche converted this from a draft issue Mar 15, 2024
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@mthalman
Copy link
Member

Rather than doing any fancy synchronization it might be easiest for each repo to output to it's own staging directory which are then consolidated after all repos are done building.

@MichaelSimons MichaelSimons added area-infra Source-build infrastructure and reporting and removed untriaged labels Mar 21, 2024
@MichaelSimons MichaelSimons moved this from Backlog to 9.0 Preview 4 in .NET Source Build Mar 21, 2024
@MichaelSimons MichaelSimons moved this from 9.0 Preview 4 to 9.0 in .NET Source Build Mar 21, 2024
@mthalman
Copy link
Member

Rather than doing any fancy synchronization it might be easiest for each repo to output to it's own staging directory which are then consolidated after all repos are done building.

I guess this wouldn't work if using the clean-while-building option since the files would get deleted before it got to the end of the build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infra Source-build infrastructure and reporting
Projects
Status: 10.0
Development

No branches or pull requests

3 participants