Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDKMigration]Attempt to fix missing authentication. #10245

Merged
merged 16 commits into from
Nov 4, 2024

Conversation

ryuyu
Copy link
Contributor

@ryuyu ryuyu commented Oct 31, 2024

When a set of non null blob options are passed through to src/Catalog/Persistence/CloudBlobDirectoryWrapper.cs, the BlobServiceClient is recreated with the correct options to create the ContainerClient (containerClient inherits blob options from the creating service client).

Unfortunately, this is created without authentication information, and the authentication information is not available here.

This change attempts to "work around" this issue by instead passing through a serviceClientFactory so we can recreate with whatever options we need whenever we want.
Note that this adds a reference from NuGet.Service.Storage to Catalog project, which has its own Storage implementation. This requires us to disambiguate Storage (between NuGet.Services.Storage and catalog.persistence.Storage).

Things we tried that didn't work:

  1. Extract auth information from the BlobServiceClient. The authentication information is not available on the service client
  2. Pipe through additional parameters so we can pipe auth information through directly. This resulted in a changeset of roughly the same size. (Thanks @advay26 for exploring this option)
  3. Modify the blob options on the BlobServiceClient instead of creating a new one. This doesn't seem possible.

Related to https://github.com/NuGet/Engineering/issues/5584

advay26
advay26 previously approved these changes Oct 31, 2024
Copy link
Member

@advay26 advay26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some individual jobs will still need a few more DI changes to work, like search and some of the other V3 jobs, but those changes should be small

src/Ng/CommandHelpers.cs Outdated Show resolved Hide resolved
drewgillies
drewgillies previously approved these changes Oct 31, 2024
Copy link
Contributor

@drewgillies drewgillies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 Yes, there is namespace noise here, but we can circle back to this (let's track it). We need to review namespaces as we continue to work on streamlining the facade types. Otherwise, this looks very good.

@ryuyu ryuyu dismissed stale reviews from drewgillies and advay26 via e3149f3 October 31, 2024 23:22
@ryuyu ryuyu marked this pull request as ready for review October 31, 2024 23:31
@ryuyu ryuyu requested a review from a team as a code owner October 31, 2024 23:31
@ryuyu ryuyu changed the title [DRAFT][SDKMigration]Attempt to fix missing authentication. [SDKMigration]Attempt to fix missing authentication. Oct 31, 2024
advay26
advay26 previously approved these changes Nov 1, 2024
@erdembayar
Copy link
Contributor

Please add related workitem if you have one.

dannyjdev
dannyjdev previously approved these changes Nov 1, 2024
@@ -12,6 +12,8 @@
using NuGet.Services.Metadata.Catalog.Helpers;
using NuGet.Services.Metadata.Catalog.Persistence;

using Storage = NuGet.Services.Metadata.Catalog.Persistence.Storage;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems, you don't need this because you're using fully qualified namespace.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a quick look in the project and found that we actually already had to make this disambiguation before, so i'm going to update these to use the existing alias instead (CatalogStorage)

erdembayar
erdembayar previously approved these changes Nov 4, 2024
Copy link
Contributor

@erdembayar erdembayar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2-3 comments, not really blocking

@ryuyu ryuyu dismissed stale reviews from erdembayar, dannyjdev, and advay26 via f90a326 November 4, 2024 21:12
@@ -92,7 +94,7 @@ private static ICloudBlobDirectory GetCloudBlobDirectoryUri(Uri storageBaseUri)

var blobEndpoint = new Uri(storageBaseUri.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped));
// Create BlobServiceClient with anonymous credentials
var blobServiceClient = new BlobServiceClient(blobEndpoint);
var blobServiceClient = new BlobServiceClientFactory(blobEndpoint, new DefaultAzureCredential());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a usage of this method that requires the MSI client id? or did you just add the DefaultAzureCredential for testing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as we (@advay26 and I) could tell, this particular code path is only called in icons.
The original code doesn't pass any authentication information at all, so at this point, we are unsure exactly what it does/how it worked/works.
DefaultAzureCredential was put here as from what I could see, it is likely the "closest" alternative.
For whomever migrates icons later, assuming that this doesn't behave as expected, may need to enhance BlobServiceClientFactory to be able to return a no auth client.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually migrating icons, so I can verify when I start my tests for that job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand, DefaultAzureCredential picks up the MSI that is assigned to the VM, so it should hopefully just work for Icons too. I'm not sure what happens if we have a VM with multiple MSIs assigned to it

@ryuyu ryuyu merged commit a4de2bd into dev Nov 4, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants