Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: delegated identity support #244

Open
wants to merge 55 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
e979e99
ignore helm init and pod specs example
t-ysalazar Jun 7, 2022
bdf2cc1
Revert "ignore helm init and pod specs example"
t-ysalazar Jun 13, 2022
1cc644e
MI for image pull and mock test endpoint
fnuarnav Jun 20, 2022
6264659
add defualt image server when nothing is provided
fnuarnav Jun 20, 2022
a0530aa
added unit tests for MI
fnuarnav Jun 20, 2022
7216493
fix error; remove print statements
fnuarnav Jun 20, 2022
1a4ed8f
removed unused pod identity object
fnuarnav Jun 20, 2022
df701e1
use pointer variable for ContainerGroup.Identity
fnuarnav Jun 20, 2022
d846f64
added comments to describe types
fnuarnav Jun 21, 2022
ec26969
update comments
fnuarnav Jun 21, 2022
14c2f04
only add image reigstry creds for acr servernames
fnuarnav Jun 23, 2022
4f9b1e2
Merge branch 'virtual-kubelet:master' into master
t-ysalazar Jul 7, 2022
3c9d25a
Merge branch 'virtual-kubelet:master' into master
t-ysalazar Jul 11, 2022
b29a7c8
Merge branch 'fnuarnav/feature/mi-for-image-pull' into t-ysalazar/mi-…
t-ysalazar Jul 11, 2022
3b86b2f
rebase with master
t-ysalazar Jul 11, 2022
8adcb62
e2e 1, create cluster
t-ysalazar Jul 12, 2022
c9cda56
E2E delete cluster after test
t-ysalazar Jul 12, 2022
cc98532
e2e get client secret
t-ysalazar Jul 12, 2022
2039759
e2e add managed-identity
t-ysalazar Jul 13, 2022
48fa251
e2e get masterURI
t-ysalazar Jul 13, 2022
8189ffa
e2e helm
t-ysalazar Jul 13, 2022
e2aa4d1
e2e fix miURL
t-ysalazar Jul 14, 2022
e9eb315
e2e fix miURL
t-ysalazar Jul 14, 2022
e890b06
e2e mi pull pod
t-ysalazar Jul 14, 2022
03ca373
TestImagePullUsingKubeletIdentityAndSecrets
t-ysalazar Jul 14, 2022
fc2226e
e2e test TestImagePullUsingKubeletIdentityAndSecrets assign role MI
t-ysalazar Jul 15, 2022
4eabd8e
deployments_test const
t-ysalazar Jul 18, 2022
bfb5706
factorize TestImagePullUsingKubeletIdentity
t-ysalazar Jul 18, 2022
5f441ec
fix delete cluster
t-ysalazar Jul 18, 2022
d69ed08
e2e TestImagePull_KubeletIdentityInAKSCLuster/virtual_node_with_manag…
t-ysalazar Jul 19, 2022
ec1c281
e2e TestAKSDeployment_attachACR
t-ysalazar Jul 19, 2022
38e454f
TestImagePull_KubeletIdentityInAKSCLuster/virtual_node_with_no_secrets
t-ysalazar Jul 20, 2022
fe5cbd1
Merge branch 'virtual-kubelet:master' into master
t-ysalazar Jul 20, 2022
a823e5f
TestImagePull_KubeletIdentityInAKSCLuster
t-ysalazar Jul 20, 2022
884c479
fix merge conflict
t-ysalazar Jul 20, 2022
583d23d
Merge pull request #2 from t-ysalazar/t-ysalazar/mi-for-image-pull
suselva Jul 20, 2022
186a6fb
remove comment
suselva Jul 20, 2022
43df01a
Merge branch 'virtual-kubelet:master' into suselva/mi-image-pull
suselva Jul 21, 2022
d935e7c
update regex string escape
suselva Jul 21, 2022
093ca98
fix compatibility
t-ysalazar Jul 21, 2022
797c58e
parallelization
t-ysalazar Jul 21, 2022
a0ed92a
comments
t-ysalazar Jul 22, 2022
80e7dee
Merge branch 'virtual-kubelet:master' into t-ysalazar/mi-for-image-pull
t-ysalazar Jul 22, 2022
d6693be
Merge pull request #3 from t-ysalazar/t-ysalazar/mi-for-image-pull
t-ysalazar Jul 22, 2022
f9bfbec
Merge branch 't-ysalazar/mi-for-image-pull' of https://github.com/t-y…
t-ysalazar Jul 25, 2022
8545cad
e2e fix node assignation
t-ysalazar Jul 25, 2022
bed4401
Merge pull request #4 from t-ysalazar/t-ysalazar/mi-for-image-pull
t-ysalazar Jul 25, 2022
1ad2fdc
Merge branch 'master' into suselva/mi-image-pull
ryanzhang-oss Jul 27, 2022
c2d3e42
dynamic values in mi-image-pull podspecs
t-ysalazar Aug 4, 2022
f10e459
Merge pull request #5 from t-ysalazar/t-ysalazar/mi-for-image-pull
t-ysalazar Aug 4, 2022
b93be3d
add RG and location to helm; add azure dns ip to aci request
fnuarnav Aug 5, 2022
91812cf
supporting delegated identities passed as annotations
fnuarnav Aug 8, 2022
807a43f
use correct name DelegatedResources
fnuarnav Aug 10, 2022
c5772c3
read entire delegated resouces from base64 encoded annotation
fnuarnav Aug 11, 2022
181050b
acr names can have numbers
fnuarnav Aug 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions client/aci/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ import (
const (
defaultUserAgent = "virtual-kubelet/azure-arm-aci/2021-07-01"
apiVersion = "2021-07-01"
aksApiVersion = "2022-04-01"

containerGroupURLPath = "subscriptions/{{.subscriptionId}}/resourceGroups/{{.resourceGroup}}/providers/Microsoft.ContainerInstance/containerGroups/{{.containerGroupName}}"
containerGroupListURLPath = "subscriptions/{{.subscriptionId}}/providers/Microsoft.ContainerInstance/containerGroups"
containerGroupListByResourceGroupURLPath = "subscriptions/{{.subscriptionId}}/resourceGroups/{{.resourceGroup}}/providers/Microsoft.ContainerInstance/containerGroups"
containerLogsURLPath = containerGroupURLPath + "/containers/{{.containerName}}/logs"
containerExecURLPath = containerGroupURLPath + "/containers/{{.containerName}}/exec"
containerGroupMetricsURLPath = containerGroupURLPath + "/providers/microsoft.Insights/metrics"
aksClustersListURLPath = "subscriptions/{{.subscriptionId}}/resourceGroups/{{.resourceGroup}}/providers/Microsoft.ContainerService/managedClusters"
)

// Client is a client for interacting with Azure Container Instances.
Expand Down
67 changes: 66 additions & 1 deletion client/aci/list.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ func (c *Client) ListContainerGroups(ctx context.Context, resourceGroup string)
return nil, fmt.Errorf("Sending get container group list request failed: %v", err)
}
defer resp.Body.Close()

// 200 (OK) is a success response.
if err := api.CheckResponse(resp); err != nil {
return nil, err
Expand All @@ -69,3 +68,69 @@ func (c *Client) ListContainerGroups(ctx context.Context, resourceGroup string)

return &list, nil
}

func (c *Client) ListAKSClusters(ctx context.Context, resourceGroup string) (*AKSClusterListResult, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After migrating to use nodeUtils, you should find an easier way to get k8s configs using
k8sClient, err := nodeutil.ClientsetFromEnv(kubeConfigPath)


// make aksApiVersion a new constant
urlParams := url.Values{
"api-version": []string{aksApiVersion},
}

// Create the url.
uri := api.ResolveRelative(c.auth.ResourceManagerEndpoint, aksClustersListURLPath)
uri += "?" + url.Values(urlParams).Encode()

// Create the request.
req, err := http.NewRequest("GET", uri, nil)
if err != nil {
return nil, fmt.Errorf("Creating get AKS cluster list uri request failed: %v", err)
}
req = req.WithContext(ctx)

// Add the parameters to the url.
err = api.ExpandURL(req.URL, map[string]string{
"subscriptionId": c.auth.SubscriptionID,
"resourceGroup": resourceGroup,
})
if err != nil {
return nil, fmt.Errorf("Expanding URL with parameters failed: %v", err)
}

// Send the request.
resp, err := c.hc.Do(req)
if err != nil {
return nil, fmt.Errorf("Sending get clusters list request failed: %v", err)
}
defer resp.Body.Close()

// 200 (OK) is a success response.
if err := api.CheckResponse(resp); err != nil {
return nil, err
}

// Decode the body from the response.
if resp.Body == nil {
return nil, errors.New("List AKS clusters returned an empty body in the response")
}

var list AKSClusterListResult
if err := json.NewDecoder(resp.Body).Decode(&list); err != nil {
return nil, fmt.Errorf("Decoding get AKS clusters response body failed: %v", err)
}
return &list, nil

}

func (c *Client) GetAKSCluster(ctx context.Context, resourceGroup string, clusterFqdn string) (*AKSCluster, error) {
clusters, err := c.ListAKSClusters(ctx, resourceGroup)
if err != nil {
return nil, err
}

for _, cluster := range clusters.Value {
if cluster.Properties.Fqdn == clusterFqdn {
return &cluster, nil
}
}
return nil, fmt.Errorf("no cluster found with domain %s, in resource group %s", clusterFqdn, resourceGroup)
}
53 changes: 53 additions & 0 deletions client/aci/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,38 @@ type AzureFileVolume struct {
StorageAccountKey string `json:"storageAccountKey,omitempty"`
}

// AKSClusterListResult is the aks cluster list response that contains cluster properties
// ttps://management.azure.com/subscriptions/{subscription}/resourceGroups/{resouorce-groups}/providers/Microsoft.ContainerService/managedClusters/{clusterid}?api-version=2022-04-01
type AKSClusterListResult struct {
api.ResponseMetadata `json:"-"`
Value []AKSCluster `json:"value,omitempty"`
NextLink string `json:"nextLink,omitempty"`
}

// AKS cluster object along with some properties
type AKSCluster struct {
Id string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Properties AKSClusterPropertiesTruncated `json:"properties,omitempty"`
}

// truncated properties only include identity profile (kubelet identity)
type AKSClusterPropertiesTruncated struct {
Fqdn string `json:"fqdn,omitempty"`
IdentityProfile AKSIdentityProfile `json:"identityProfile,omitempty"`
}

// AKS Identity profile definition
type AKSIdentityProfile struct {
KubeletIdentity AzIdentity
}

// Azure managed identity definition
type AzIdentity struct {
ResourceId string
ClientId string
ObjectId string
}
// Container is a container instance.
type Container struct {
Name string `json:"name,omitempty"`
Expand All @@ -81,6 +113,7 @@ type ContainerGroup struct {
Location string `json:"location,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
ContainerGroupProperties `json:"properties,omitempty"`
Identity *ACIContainerGroupIdentity `json:"identity,omitempty"`
}

// ContainerGroupProperties is
Expand All @@ -99,6 +132,24 @@ type ContainerGroupProperties struct {
DNSConfig *DNSConfig `json:"dnsConfig,omitempty"`
}

// container group identity object
type ACIContainerGroupIdentity struct {
PrincipalId string `json:"principalid,omitempty"`
TenantId string `json:"tenantid,omitempty"`
Type string `json:"type,omitempty"`
UserAssignedIdentities map[string]map[string]string `json:"userassignedidentities,omitempty"`
DelegatedResources map[string]DelegatedIdentitySpec `json:"delegatedResources,omitempty"`
}

// delegated identity specification
type DelegatedIdentitySpec struct {
ResourceId string `json:"resourceId,omitempty"`
TenantId string `json:"tenantId,omitempty"`
ReferralResource string `json:"referralResource,omitempty"`
Location string `json:"location,omitempty"`

}

// ContainerGroupPropertiesInstanceView is the instance view of the container group. Only valid in response.
type ContainerGroupPropertiesInstanceView struct {
Events []Event `json:"events,omitempty"`
Expand Down Expand Up @@ -183,6 +234,8 @@ type ImageRegistryCredential struct {
Server string `json:"server,omitempty"`
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
IdentityURL string `json:"identityurl,omitempty"`
Identity string `json:"identity,omitempty"`
}

// IPAddress is IP address for the container group.
Expand Down
Loading