Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow runtime packs to be resolved from the packs folder #14044

Closed
dsplaisted opened this issue Oct 8, 2020 · 7 comments · Fixed by #19596
Closed

Allow runtime packs to be resolved from the packs folder #14044

dsplaisted opened this issue Oct 8, 2020 · 7 comments · Fixed by #19596
Assignees
@dsplaisted
Labels
Area-Workloads
Milestone
6.0.1xx

Comments

@dsplaisted
Copy link
Member

For targeting packs, the SDK first checks if they are in the dotnet packs folder. If they are it uses them from there, otherwise it generates a PackageDownload item so that they will be downloaded to the packages folder as part of NuGet restore.

Currently, we don't check for runtime packs in the packs folder. This is because we haven't shipped any with the .NET SDK, as they are only used for self-contained apps and there are many different runtime packs for different RuntimeIdentifiers which represent different OS/architecture combinations.

For .NET 6, iOS and Android apps will need runtime packs, and the set of target RuntimeIdentifiers will be more limited. So they would like to deliver runtime packs as part of their optional SDK workloads.

This means we should update the SDK runtime pack resolution logic to first look in the dotnet packs folder and resolve from there if possible, before generating a PackageDownload item to download the runtime pack if necessary.
@dsplaisted dsplaisted added this to the 6.0.1xx milestone Oct 8, 2020
@dagood
Copy link
Member

dagood commented Oct 8, 2020

Sweet! Will this be implemented generally enough to let us start shipping source-built runtime packs? dotnet/source-build#1215

/cc @omajid @tmds

@dagood dagood mentioned this issue Oct 14, 2020
Merged
@pjcollins pjcollins mentioned this issue Oct 14, 2020
Merged
jonathanpeppers added a commit to jonathanpeppers/maui-workload that referenced this issue Feb 12, 2021
@jonathanpeppers
Revert "Download and include runtime packs"
69bc71f 
This reverts commit 6bd1b0e.

This idea doesn't fully work yet, see:

dotnet/sdk#14044

It partially works for Android, you can do:

    > .\bin\dotnet\dotnet build .\samples\HelloMaui\HelloMaui.Android\HelloMaui.Android.csproj

But it fails if you set `$(RuntimeIdentifier)`:

    > .\bin\dotnet\dotnet build .\samples\HelloMaui\HelloMaui.Android\HelloMaui.Android.csproj -r android.21-arm
    ...
    error NU1101: Unable to find package Microsoft.Android.Runtime.android.21-arm. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages, nuget.org

I think the first one only works because we set
`$(RuntimeIdentifiers)` *plural* by default--targeting two RIDs. There
is some special `restore` logic running when you define
`$(RuntimeIdentifier)` *singular*.
@jonathanpeppers jonathanpeppers mentioned this issue Feb 19, 2021
Merged
@marek-safar
Copy link
Contributor

/cc @pranavkm

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 23, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
ba840a8 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044
@jonathanpeppers jonathanpeppers mentioned this issue Jun 23, 2021
Merged
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 24, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
4929ed7 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 25, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
50494a5 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 28, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
c2ab215 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added an `android-aot` workload, in case you want to install this
  support by itself. This is useful, also as a way to include a
  user-friendly description.

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 29, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
8037ba7 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android-full

We can also create a smaller workload that excludes `bundletool` and
AOT support to get a smaller install footprint:

    dotnet workload install android

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added an `android-aot` workload, in case you want to install this
  support by itself. This is useful, also as a way to include a
  user-friendly description.

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-full
dd3936d 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-aot
13ba85b 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Jun 30, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-aot
6ec79a3 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

    dotnet workload install microsoft-android-sdk-full

Let's shorten this to be:

    dotnet workload install android

We can also create a new workload that extends `android`, and includes
AOT support:

    dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
to run `dotnet workload install` commands. This should give us more
confidence that our workload can be installed -- and our MSBuild tests
will consume the installed workload.

Other changes:

* Added our `Microsoft.Android.Runtime.*` packs to the
  `WorkloadManifest.json`. This was mostly omitted by mistake;
  however, runtime packs aren't resolved either due to:
  dotnet/sdk#14044

* Call `setup-test-environment.yaml` in `dotnet_create_msi` job

* Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
jonpryor pushed a commit to dotnet/android that referenced this issue Jul 6, 2021
@jonathanpeppers
[One .NET] dotnet workload install android or android-aot (#6045)
600e993 
Context: https://github.com/dotnet/maui/tree/ef4518bf502ffa6760625c4cb53fa0cb015bc6d1/src/Workload#net-maui-workload-ids

Right now to install the Android workload you would run:

	dotnet workload install microsoft-android-sdk-full

Shorten this to be:

	dotnet workload install android

We can also create a new workload that extends `android`, and
includes AOT support:

	dotnet workload install android-aot

To test this behavior on our CI, I reworked the `ExtractWorkloadPacks`
MSBuild target to run `dotnet workload install` commands.  This should
give us more confidence that our workload can be installed -- and that
our MSBuild tests will consume the installed workload.

Other changes:

  * Added our `Microsoft.Android.Runtime.*` packs to
    `WorkloadManifest.json`.  This was mostly omitted by mistake;
    however, runtime packs aren't resolved either due to:
    dotnet/sdk#14044
  
  * Call `setup-test-environment.yaml` in `dotnet_create_msi` job
  
  * Delete `dotnet/metadata` folder in `Step_InstallDotNetPreview`
  
  * Add `setup-test-environment.yaml` to another build phase
@steveisok
Copy link
Member

@dsplaisted Do you still anticipate being able to roll this out for 6?

@jonathanpeppers
Copy link
Member

I don't think iOS/Android/Maui workloads would be able to work offline without this?

@steveisok
Copy link
Member

I don't think iOS/Android/Maui workloads would be able to work offline without this?

From what I've seen, they would not.

@dsplaisted
Copy link
Member Author

Yes, we're still planning to do this for .NET 6.

@wli3
Copy link

wli3 commented Jul 8, 2021

note from workload meeting: This is a rather easy change. At the same time, not having the feature in not end of the world (use library pack with waste disk space). We should pick up this task after –architecture os work,

pjcollins added a commit to pjcollins/xamarin-macios that referenced this issue Jul 21, 2021
@pjcollins
[dotnet] Add runtime packs to workload manifests
4217ece 
Runtime pack entries should be included in all workload manifest files,
even though they will not yet be resolved from the packs folder due to
dotnet/sdk#14044.  These changes will also
allow us to start producing .msi files for the runtime packs for an
eventual inclusion in the Visual Studio installer.
@pjcollins pjcollins mentioned this issue Jul 21, 2021
Merged
rolfbjarne pushed a commit to xamarin/xamarin-macios that referenced this issue Jul 22, 2021
@pjcollins
[dotnet] Add runtime packs to workload manifests (#12168)
a150c1d 
Runtime pack entries should be included in all workload manifest files,
even though they will not yet be resolved from the packs folder due to
dotnet/sdk#14044.  These changes will also
allow us to start producing .msi files for the runtime packs for an
eventual inclusion in the Visual Studio installer.
@dsplaisted dsplaisted mentioned this issue Aug 9, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dsplaisted dsplaisted

Labels
Area-Workloads
Projects
None yet
Milestone
6.0.1xx
Development

Successfully merging a pull request may close this issue.

Runtime pack resolution dsplaisted/sdk
7 participants
@dsplaisted @marek-safar @steveisok @jonathanpeppers @wli3 @marcpopMSFT @dagood