Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add portable linux source build leg #75546

Merged
merged 15 commits into from
Oct 6, 2022
Merged

Conversation

mmitche
Copy link
Member

@mmitche mmitche commented Sep 13, 2022

Adds a portable linux source build leg to the official build. The idea is that these packages produced by this build should be relied upon in downstream PR validation, rather than the RID specific assets. This should allow for cleaner SB logic in downstream repos.

While doing this, I cleaned up a couple parameters to make it clearer what they were doing

Adds a portable linux source build leg to the official build. The idea is that these packages produced by this build should be relied upon in downstream PR validation, rather than the RID specific assets. This should allow for cleaner SB logic in downstream repos.

While doing this, I cleaned up a couple parameters to make it clearer what they were doing
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@mmitche
Copy link
Member Author

mmitche commented Sep 13, 2022

@mmitche
Copy link
Member Author

mmitche commented Sep 13, 2022

Artifacts looked good except for the logs. Those had overlapping artifact names. Fixed that.

https://dev.azure.com/dnceng/internal/_build/results?buildId=1991830&view=results

@mmitche
Copy link
Member Author

mmitche commented Sep 13, 2022

Getting closer here. Latest build looks correct except the publishing asset manifests need to have different names.

@mmitche mmitche closed this Sep 13, 2022
@mmitche mmitche reopened this Sep 13, 2022
Copy link
Member

@hoyosjs hoyosjs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM sans the manifest

eng/pipelines/common/platform-matrix.yml Show resolved Hide resolved
@mmitche
Copy link
Member Author

mmitche commented Sep 14, 2022

Depends on dotnet/arcade#10860

@mmitche
Copy link
Member Author

mmitche commented Sep 20, 2022

Depends on #75929 to add the right parameter to the common YAML step template.

@mmitche
Copy link
Member Author

mmitche commented Sep 20, 2022

Updated arcade to get ahead on validation, and this should be the final official build test: https://dev.azure.com/dnceng/internal/_build/results?buildId=1999350&view=results

@mmitche
Copy link
Member Author

mmitche commented Oct 3, 2022

@mmitche
Copy link
Member Author

mmitche commented Oct 3, 2022

@tmds @MichaelSimons I ended up reverting the change to use the banana RID here: bf026ca. I don't think this was correct, as it meant that runtime's official build was only producing source-build intermediates for the banana rid, which means that downstream repos could not actually consume these in their source-build legs.

I think the intention was to test a banana.blah RID (built on whatever container). if that's the case, maybe it makes sense to introduce an additional CI or PR (if it breaks enough) leg to validate these scenarios. But the official builds need to produce a real RID.

@tmds
Copy link
Member

tmds commented Oct 4, 2022

Yes, the goal is to verify you can build for an unknown rid.

This requires runtime to not use this rid while restoring artifacts, and to use it while naming output artifacts.

All other CI jobs use known rids, and in most cases the rid being restored is even the same as the one being built (like linux-x64 -> linux-x64). Then it goes unnoticed the builds are mixing up the output rid from the restore rid.

We should continue to have a CI job that builds an unknown rid, otherwise these mix-ups go undetected until they break source-build.

- template: /eng/common/templates/steps/source-build.yml
parameters:
platform:
buildScript: $(_sclEnableCommand) $(Build.SourcesDirectory)$(dir)build$(scriptExt)
nonPortable: true
# Use a custom RID that isn't in the RID graph here to validate we don't break the usage of custom rids that aren't in the graph.
targetRID: banana.24-x64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You remove the custom RID, but we want to keep it to make sure that we can build with arbitrary RIDs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we probably need another leg, or one that only runs in PRs? Preference?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ViktorHofer @tmds @MichaelSimons I made some tweaks:

  • Build only linux-x64 portable officially (nothing else flows)
  • Add missing PR legs for linux-x64 (to avoid official breaks)
  • Add rolling CI for Banana and centos7

@mmitche
Copy link
Member Author

mmitche commented Oct 4, 2022

@mmitche
Copy link
Member Author

mmitche commented Oct 5, 2022

Outputs looking good. Any additional comments?

eng/pipelines/global-build.yml Outdated Show resolved Hide resolved
@mmitche mmitche merged commit b1f5113 into dotnet:main Oct 6, 2022
@mmitche mmitche deleted the add-portable-linux-leg branch October 6, 2022 13:58
@ghost ghost locked as resolved and limited conversation to collaborators Nov 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants