importing azurerm_pim_eligible_role_assignment fails with resource not existant #23111
Comments
|
@tim-krehan Looks you want to import the |
|
hmmm i had that issue with both active and eligible. Seems that it was a one-time thing? I think this issue can be closed - I will reference it, once I come upon that issue again. |
|
I am using the correct resource type and having the same issue. The I know imports have worked previously for me, not sure where the issue comes from. |
|
@audunsolemdal There appears to be a delay in the response of this resource.. The resource cannot deleted within the first 5 mins etc. Can you retry later? |
|
I believe this issue should be reopened. I can reproduce it right now… Then: I can play this chess game back and forward ♟️ In the UI, the assignment exists. To me it currently occurs that it is just orphaned completely. |
|
I agree that this is still a problem |
|
I could reproduce this issue as well. |
|
I'm having the same issue when using an index key, opened #23657 in case it's something specific with index key imports. One thing I also just noticed is that when destroying azurerm_pim_eligible_role_assignment resources, the assignment is never actually removed. If you're re-trying the creation after destruction, the "resource already exists" situation will recur and thus lead you to the import issue once more. The only sure fix has been clicking through the portal and removing the assignment. I just opened issue #23672 in hopes of getting it fixed. The azurerm_pim_eligible_role_assignment resource provider has been buggy since it launched, unfortunately this is just more of the same. |
|
It does not really help to resolve anyones issues but I'd like to also support the maintainers in sharing two discoveries I made over the recent weeks. Scheduling changesThe whole eligibility setup thing does not base on pure CRUD like objects it seems. It bases on "requests" (orders) you make and the system "processing" those. The resulting resources are read-only. It can be seen in the REST Docs that you always make a request of a kind and then you wait for a result that "pops up somewhere" else. E.g. revoking an eligibility needs a "request" of type An example for history: Afterwards, you can This brings me to point 2. New Azure UI
Since some days, you can do what I did via maddening REST calls via the UI! 🔔 It does not help much for the |
|
hi @tim-krehan good day. I'm not sure whether you are the one from hashcrop, we still have this kind of issue. however the object add definition is correct. We want to confirm any progress for this issue |
|
@Scarlettliuyc Sorry to disappoint you, I am just an user that is affected by this. |
|
@xuzhang3 FYI I just tried this on 3.85.0 release and it still fails. If I try to do the apply, it says it already exists. I add the import and it says it doesn't exist. |
|
@xuzhang3 I have also test version 3.85 and have observed same behaviour as @smokedlinq - apply still fails saying resources exists and import fails saying it doesn't. Can this issue please be re-opened? |
|
I was so sick of this that I "developed" this in the meantime: import { AuthorizationManagementClient } from "@azure/arm-authorization";
import { DefaultAzureCredential } from "@azure/identity";
import { v4 as uuidv4 } from 'uuid';
const tfOutputSanitized = `
/subscriptions/xxxxxxx-b530-xxxx-9b08-xxxxxxxxx|/subscriptions/xxxxxx-xxxx-4277-9b08-xxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/xxxxxx-89b4-xxxx-9698-xxxxxxxxx|xxxxxx-f49b-xxxx-bd59-xxxxxxxx
`;
/**
* This script is the automation of docs/docs/enterprise/single-tenant/troubleshooting.md#terraform-state-forgot-resources
* I now had to fix it the third time, so I decided to automate it.
* It would really be nice if either Microsoft or Hashicorp would fix this issue and until then we do this rain dance every 4-6 weeks.
* npm run repair:subscriptions
*
* The terraform apply output that is erroneous must be a bit revised to be used here, but it can nearly be pasted 1:1 and then the script will fix it.
*/
async function putRoleEligibilityScheduleRequests() {
const subscriptionsToFixToday: {subscription: string, roleDefinitionId: string, principalId: string}[] = [];
const lines = tfOutputSanitized.split("\n");
for (const line of lines) {
const parts = line.split("|");
if (parts.length === 3) {
const subscription = parts[0];
const roleDefinitionId = parts[1];
const principalId = parts[2];
subscriptionsToFixToday.push({subscription, roleDefinitionId, principalId});
}
}
console.log(`Parsed ${subscriptionsToFixToday.length} incidents to fix.`);
console.log(`Revoking eligible role assignments for ${subscriptionsToFixToday.length} subscriptions.`)
// Setup env and get a token for ARM
const subscriptionId = process.env["AUTHORIZATION_SUBSCRIPTION_ID"] || "00000000-0000-0000-0000-000000000000";
const credential = new DefaultAzureCredential();
const client = new AuthorizationManagementClient(credential, subscriptionId);
for (const assignment of subscriptionsToFixToday) {
const roleEligibilityScheduleRequestName = uuidv4();
const scope =`providers/Microsoft.Subscription${assignment.subscription}`;
const parameters = {
principalId: assignment.principalId,
requestType: "AdminRemove",
roleDefinitionId: assignment.roleDefinitionId,
};
const result = await client.roleEligibilityScheduleRequests.create(
scope,
roleEligibilityScheduleRequestName,
parameters
);
console.log(`Assignment [${result.status?.toUpperCase()}] for scope ${result.expandedProperties?.scope?.displayName} for role ${result.expandedProperties?.roleDefinition?.displayName} for principal ${result.expandedProperties?.principal?.displayName}.`);
}
}
putRoleEligibilityScheduleRequests();{
"name": "repair-kit",
...
"scripts": {
"repair:subscriptions": "npx ts-node src/subscriptions.ts"
},
...
"devDependencies": {
"@azure/arm-authorization": "^9.0.0",
"@azure/identity": "^4.0.0",
"@types/node": "^20.10.4",
"@types/uuid": "^9.0.7",
"uuid": "^9.0.1"
}
}Where Sorry but this is the best I got until now to get out of the misery. |
|
@smokedlinq can you share your TF script and the TRACE logs? I do not have the same env as you do so I can only generate a potentials scenario in my mind. One way you can do is check the role assignment in the PIM role assignment in the PIM role assignment page. The details I want to know is which scope you want to assign the role and the sub/upper resources role assign details. This role should be at least assigned to the different resource in different scope. |
He @xuzhang3, they are not. They are 100% the same as the last time we/I ran apply. The 🔑 is to wait until something @ARM/Entra changes, then the tf state breaks until fixes as above. Check also my message #23111 (comment) where I basically did what you wanted to see 💟 |
|
@unique-dominik @spectrum048k I cannot reproduce this error in #terraform {
# required_providers {
# azurerm = {
# source = "hashicorp/azurerm"
# version = "=3.85.0"
# }
# }
#}
provider "azurerm" {
features {
}
}
data "azurerm_client_config" "example" {}
resource "time_static" "example" {}
resource "azurerm_pim_eligible_role_assignment" "pim1" {
scope = "/subscriptions/<Sub ID>/resourceGroups/<Resource Group Name>"
role_definition_id = "<Role definition ID>"
principal_id = data.azurerm_client_config.example.object_id
justification = "test"
schedule {
start_date_time = time_static.example.rfc3339
expiration {
duration_days = 180
}
}
}
resource "azurerm_pim_eligible_role_assignment" "pim2" {
scope = "/subscriptions/Sub ID"
role_definition_id = "<Role definition ID>"
principal_id = data.azurerm_client_config.example.object_id
justification = "test"
schedule {
start_date_time = time_static.example.rfc3339
expiration {
duration_days = 180
}
}
}
|
|
Will try again next time yes. I can also not repro it right now, it needs some days time to break from the other end 😢 |
|
I too am still getting the OP error in 3.85. If the PIM eligibility exists in Azure, Terraform says it has to be imported. If I try to import it, I get the import error as well. If the eligibility doesn't exist in Azure, it creates without incident. If the eligibility is created by Terraform using the test that @xuzhang3 provided, it updates it, but it still doesn't provide for indefinite/permanent eligibility as described in #23366 and #22766 . It looks like the PR for #23295 has not been merged yet, which might help. |
|
@xuzhang3 In my testing I have noticed it takes circa 45 days for the pim assignment to 'fail' in the Azure backend before the issue presents itself in terraform. |
|
@spectrum048k Sounds about right, I guessed 4-6 weeks above 👍 |
|
Still experiencing this error for a |
|
@kristeey What is the status of the role assignment you are trying to import? Active or not active and does this role assignment created by current SPN? |
|
@xuzhang3 The status is |
|
Since this was a blocker I deleted the role assignment and reapplied it again. |
|
@spectrum048k the fix PR switch to new a endpoint and the new endpoint can get more role assignments which is not exist in the current state file. Run apply can pull the role assignments from remote and set to the state file. If possible I would suggest recreate the role assignment as a lot has changed with the new version |
|
@xuzhang3 Manual deleting the role assignment and re-apply it with terraform worked for me. |
Hi, We are also experiencing the same problem as @kristeey after using the latest version of the provider: Can you clarify this statement please: "Run apply can pull the role assignments from remote and set to the state file" - does this mean the root cause of the original issue of failing import of the resources was that terraform was not able to pull role assignments from the remote? |
|
@jakubslonxlab Yes, because they originally basically disappeared after 45 days 😢 |
|
Does anyone have a workaround for this until we have a permanent fix? |
|
The fixes have been merged. What do you want to work around? |
|
I'm trying to automate generating terraform from existing PIM assignments.
$management_groups = az account management-group list --no-register
$pim_assignments = $management_groups `
| ForEach-Object -ThrottleLimit 20 -Parallel {
$mg = $_
Write-Host "Fetching PIM assignments from $($mg.displayName)"
$url = "`"https://management.azure.com$($mg.id)/providers/Microsoft.Authorization/roleEligibilityScheduleInstances?api-version=2020-10-01`""
az rest --method get --url $url | ConvertFrom-Json | Select-Object -ExpandProperty value
}
$pim_assignments_to_import = $pim_assignments `
| ForEach-Object { $_.properties.expandedProperties } `
| Sort-Object -Property { $_ | ConvertTo-Json } -Unique
$entry_template = @"
import {
id = "%ID%"
to = azurerm_pim_eligible_role_assignment.%NAME%
}
"@
$content = ""
function sanitize($name) {
$name -replace "[^a-zA-Z0-9]", "_"
}
foreach ($v in $pim_assignments_to_import) {
$scope_id = $v.scope.id
$scope_name = sanitize($v.scope.displayName)
$role_definition_id = $v.roleDefinition.id
$role_definition_name = sanitize($v.roleDefinition.displayName)
$principal_id = $v.principal.id
$principal_name = sanitize($v.principal.displayName)
$ID = "$scope_id|$role_definition_id|$principal_id"
$NAME = "${scope_name}____${role_definition_name}____${principal_name}"
Add-Member -InputObject $v -MemberType NoteProperty -Name ResourceName -Value $NAME -Force
$content += $entry_template -replace "%ID%", $ID -replace "%NAME%", $NAME
}
Set-Content ".\ignore\pim_imports.tf" $content
Write-Host "Wrote $($content.Length) chars to pim_imports.tf"which yields import {
id = "/providers/Microsoft.Management/managementGroups/redacted|/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c|redacted-principal-id"
to = azurerm_pim_eligible_role_assignment.Tenant_Root_Group____Contributor____AAD_CloudOps_Tenant_Contributor
}
import {
id = "/providers/Microsoft.Management/managementGroups/redacted|/providers/Microsoft.Authorization/roleDefinitions/00482a5a-887f-4fb3-b363-3b7fe8e74483|redacted-principal-id"
to = azurerm_pim_eligible_role_assignment.Tenant_Root_Group____Key_Vault_Administrator____AAD_CloudOps_Tenant_KeyVaultAdmin
}
import {
id = "/providers/Microsoft.Management/managementGroups/redacted|/providers/Microsoft.Authorization/roleDefinitions/18d7d88d-d35e-4fb5-a5c3-7773c20a72d9|redacted-principal-id"
to = azurerm_pim_eligible_role_assignment.Tenant_Root_Group____User_Access_Administrator____AAD_CloudOps_Tenant_UserAccessAdmin
}then doing terraform plan -generate-config-out="generated.tf" 2>&1 > terraform_plan_generate_config_out.logyields So, only 1 out of 3 imports succeeded D: The PIM assignment that did succeed is newer than the others, and was also created using Terraform in a different state file. Running with So, Terraform is hitting using ❯ $requests = az rest --method get --url 'https://management.azure.com/providers/Microsoft.Management/managementGroups/redacted/providers/Microsoft.Authorization/roleEligibilityScheduleRequests?api-version=2020-10-01' | ConvertFrom-Json
❯ $requests.value.properties | % { $_.status + "`t" + $_.expandedProperties.roleDefinition.displayName }
Revoked Contributor
Revoked Contributor
Revoked Contributor
Revoked Contributor
Provisioned Key Vault AdministratorThis aligns with Key Vault Admin being the only PIM assignment of mine to successfully import Meanwhile, I used ❯ $instances = az rest --method get --url 'https://management.azure.com/providers/Microsoft.Management/managementGroups/redacted/providers/Microsoft.Authorization/roleEligibilityScheduleInstances?api-version=2020-10-01' | ConvertFrom-Json
❯ $instances.value.properties | % { $_.status + "`t" + $_.expandedProperties.roleDefinition.displayName }
Provisioned Contributor
Provisioned Key Vault Administrator
Provisioned User Access AdministratorOne issue I found: and the merged PR: which had this comment
I couldn't figure out if this was saying that the PR made this change, or if this was a good change to consider for the future. To me, it seems that |
|
Upon further review: supports justification = "Expiration Duration Set"
ticket {
number = "1"
system = "example ticket system"
}which is why Scroll horizontally to compare.
However, the azure portal interface doesn't use the
I tried creating a new PIM eligibility from the portal and it did create a new entry in roleEligibilityScheduleRequests, but it didn't include the It's likely that recreating the assignments I want to import would also create the entries that would let the import succeed... but that's a pain. Perhaps there's a middle ground? If the I believe this is the relevant source code I found this regex useful to censor info in vscode Find: Replace: |
I think @TeamDman is absolutely right. The justification and ticketInfo values doe not make any sense for a PIM eligibility role assignment wich is actually made by an admin. The justifcation and ticketinfo make only sense either for an active PIM assignment (which must created with an other resource: The issue that the import is not possible as, in my option, the same root cause as the suddenly recreation of the resource after approx. 45 days for which I just create a new issue because it is stillt not fixed: #25811. The root cause for the import problem and for #25811 seems to be in the section which @TeamDman also referenced above. See also for more context and explanation of my findings: #25811 (comment) |
|
@J0F3 A lot of larger organizations can and will separate privileges so that the same person or team cannot create and approve resources, IAM/RBAC/identity changes, etc. The justification and ticket info fields fall into that usage. For example, the person or process creating a PIM eligible role assignment might not be a full-on admin - they may be on a team that handles identity management or on a helpdesk that handles creating the requests, and approvals may be required at some point. Justification and ticket info would allow an auditor, infosec team member, SIEM, log review platform, automated process, etc. to note that a new PIM eligible role has been created and to check back against the ticketInfo field. It could then take the ticketInfo content and check against the ticketing system, and if there is a mismatch, fire an alert or send an automated API call to revoke the role. The same holds true for activating roles - "I need to activate Cloud App Admin in order to make Change X against Enterprise App Y as part of Change Control Z. I've gotten approval from the App Y team under ticket # 12345. Please approve." The ticketinfo and justification fields cover that for when the eligibility is created, and they cover the activation as well. It does show those properties in the API reference for PIM, so it seems they're part of the API itself - just not revealed in the Azure portal. My uninformed and unresearched opinion is that if the assignment is present in Azure, it doesn't write back its presence to the TF state in a way that the state can read or store. It's almost like there's a disconnect - Terraform sends a properly formatted API call but it mis-reads or drops the response, interpreting it as failed. I wish I had more knowledge to dig into this myself. @xuzhang3 @manicminer Is there any chance there's further PRs or work associated with fixing this? |
This documentation is for PIM for Entra Roles that is not the same API as it is for Azure Resources for wich
I know that and I am using PIM for Azure Resource on a daily basis in a larger organisation. Basically that is why the whole PIM system exists. Anyway, if the justification properties is part of the API and just not shown in the Portal the principle keeps the same. Any existing eligible role assignment must be get trough the |
|
@J0F3 You are correct - that original API reference was indeed for Entra roles since it references the directory as the scope, my mistake. However, the AzureRM PIM guide you linked mentions that we should use the Role Eligibility Schedule Request API, which in turn has properties.Justification and properties.ticketInfo as optional fields for the request body. The page section for "Grant eligible assignment" doesn't list those fields; they're mentioned within the API reference but not that page you linked. To me, those look like what the Azure resource PIM API would use from the justification and ticket arguments in TF, but I defer to the skills and knowledge of folks who operate at the API level more than me. |
|
Thanks @TeamDman, @J0F3 and @MohnJadden for the excellent analysis on the API interactions and discussion around required features. I agree with your conclusions and believe I have already resolved a number of bugs including the hard dependency on the These APIs are very counterintuitive and contradict each other in places, but progress is being made! If anyone has any insight into the relationship specifically between the |
tl;dr: If I haven't said it before, I'll say it now: I'm no API dev, but hopefully I can decipher the objects/properties of Azure such that actual devs can consume them. That said, here's my quick reading results:
Meanwhile, I'm guessing that calling the API shows all Azure resource PIM assignments in an entire tenant unless otherwise specified but I'm honestly not sure. I suck at query language, so I'm guessing the Additional Query Parameters section would be where you scope or specify what role assignment schedules you want by individual properties. |
|
Thanks @MohnJadden, that aligns with my understanding 👍 I don't believe there's any cause at this time to query the |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Is there an existing issue for this?
Community Note
Terraform fails while importing the azurerm_pim_eligible_role_assignment. The assignment is still present in Azure but not in the state. To manage it, I tried importing it but it cannot find the assignment in Azure.
The import fails with
Cannot import non-existent remote objectThe apply fails with
A resource with the ID "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-test-001|/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/00482a5a-887f-4fb3-b363-3b7fe8e74483|11111111-1111-1111-1111-111111111111" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_pim_eligible_role_assignment" for more information.Terraform Version
1.5.6
AzureRM Provider Version
3.71.0
Affected Resource(s)/Data Source(s)
azurerm_pim_eligible_role_assignment
Terraform Configuration Files
Debug Output/Panic Output
Expected Behaviour
The allready present assignment will get imported
Actual Behaviour
The Import throws the error, that the object in Azure is not existant, while it definitly is.
Even the apply fails to create it, stating that the object is allready present
Steps to Reproduce
Important Factoids
No response
References
No response
The text was updated successfully, but these errors were encountered: