-
Notifications
You must be signed in to change notification settings - Fork 16.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create VNET without destroying all subnets #2786
Comments
I guess you should use the incremental mode: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy#incremental-and-complete-deployments |
@MCKLMT I want to have a single shared VNET (template) that application teams cannot change, and I want to give each application their own subnet. Ideally my templates would be separated by who can change them (VNET template can be changed by one group, subnet templates each by different groups). The VNET template also has gateways, public IP addresses, etc... This causes me to have to manage the VNET without a template (because it will always blow away all the subnets when deployed, even in incremental mode). I understand this isn't currently supported, but why? It seems very inconsistent considering subnets are technically resources that stand on their own. |
A subnet takes part of a VNet so it's legitimate to found them in the same template than the VNet. I believe you should think about another way to achieve your goal. |
@MCKLMT can you think of any other set of resources with a child-parent relationship that work like that? There is no way to re-deploy an app service plan and destroy the web sites it contains (in incremental mode). There is no way to deploy an Azure SQL server and destroy the databases it contains (in incremental mode). Why are VNETs and subnets different? There is currently ARM template support for creating a subnet as a stand-alone resource (VNET already exists), so why does it seem acceptable that re-deploying the VNET destroys all subnets if the subnets are omitted? |
Because all the subnets should be defined in the template. You can't add subnets, one by one. |
@MCKLMT you are wrong, here is the example: |
You're right. But you can't define the subnets and the VNet in separate files in a deployment. |
@MCKLMT 👎 -- we can see that, but why the heck not? It doesn't matter if they're in the same file. If they're separate resources, it fails. Why is the child resource even exposed if the properties are going to remove it on the next run (incremental or not)? Pick your bug (and your container hierarchy). Either exposing subnets as standalone resources is a bug, or unspecified properties equating to null properties. Here's a lovely example: 301-subnet-driven-deployment. For all the BS workarounds it exemplifies for not having the deployment objects wipe the subnets created by the previous copy (a parametrized number of times), there's no way to reference a potentially existing resource, is there? So, there's no way to rerun it, for all its overengineered glory. Trivial constructive repro here: #2897 |
Subnets sure seem like stand-alone resources to me because (a) you can assign RBAC to them and (b) you can create them separately from the VNET. If that is true, to be consistent (and sane) deploying a parent resource by itself should not destroy child resources (just like Azure SQL server/database). I think a mistake was made when subnets were added as a property of the VNET (it should exclusively be a child resource). |
Did you ever get a resolution to this? I have come across the same issue and been referenced this link. Annoyingly I have the vNet and some Core infrastructure setup as part of a "Infrastructure Architecture" project. I then use a separate "Deploy a Subnet" step which creates a subnet as part of a Solution Project. However its frustrating that it deletes all the subnets (and associated settings, VPN connections, express route, NSG's etc... ) each time causing no end of issues. Is there a way to link steps between projects at all ? |
This is ABSOLUTELY an issue. Can Microsoft please address? I am running my templates in incremental mode which claims that resources not defined in the template will not be deleted. However, subnets missing in the original template WILL be deleted. Can someone from MS please comment? Thanks, |
@CtrlDot I will pass this on to the networking team but I would also suggest creating a support request for this instead so you can reach the networking team directly on the expected behavior. This seems like a Networking resource provider issue rather than a template issue. |
I want to highlight that our documentation (https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy) states the following: 'In incremental mode, Resource Manager leaves unchanged resources that exist in the resource group but are not specified in the template.' I understand that since subnets are a property of virtual networks, of course virtual network has to be specified when trying to update the subnet, Thus, the virtual network will not remain unchanged in incremental mode. A way work with this currently, if you would like to use templates and not PowerShell, CLI or Portal, would be to do a GET so you have most updated information about your virtual network without running the risk of resource deletion. Thank you for the feedback, we will look into it. |
Is there any news on this subject ? It is a major issue.. Same goes for NSGs, as this has to be attached to the subnet ( a property on the subnet, not the NSG ). We will need a property on the NSG to attach the Subnet, and will need Subnets to be fully acknowledable atm template objects, with a property to attach it to the Vnet. you Networks guys, turned this the wrong way around. |
+1 |
Any update on this issue. I have the same problem when using 2 seperate nested templates for creating Vnet and Creating subnets, every time you re-deploy it tries to delete the subnets |
+1 Having this issue when using a "virtualNetworks" resource without any subnet and a separate "virtualNetwork/subnets" resources to define all subnets (using a "copy" property to create all the subnets). |
+1 this issue. One of the comments above by @MCKLMT
This is not true, you can absolutely define the subnets in separate deployment files. Our scenario is that for some instances of our application we need a virtual network gateway for us to connect into (our production subscription), and in some scenarios we do not (our development & UAT subscriptions) -- We would like to achieve this by embedding a parameter in our ARM template that includes (or excludes) a virtual network gateway (and subnet!) based on this flag. When I try to re-deploy, even in incremental mode, the ARM deployment attempts to destroy the gateway subnet. Deploying from a clean slate to a new resource group works just fine though. My workaround is to include the gateway in the virtual network, regardless of if it is needed. While this is harmless, still feels frustrating that this isn't supported. -- Here's an example : azuredeploy.json
And a separate file for deploying a virtual network gateway - _vpn-gateway-yes.json
|
We have this issue too. We would love to manage each subnet separately. |
@mthoger you can use the copy attribute on properties now to work around this.
|
+1 |
* Due to a bug in the Microsoft Network template system, it is not possible to update an existing VNET DNS without deleting all existing subnets so the update-vnet-dns has been removed until the bug is fixed. RE: Azure/azure-quickstart-templates#2786
Hey found a way to achieve this, when deploying VNET i do a condition ( "condition": "[equals(parameters('rebuildVNET'),'Yes')]",) i then set it to default "no" in parameters, and have all subnet templates deployed as nested templates. This allows me to edit NSGs and add subnets in a running VNET with a connected VM, if needed i can rebuild using the condition but this would require subnets to be empty. Attached files as ZIP |
Please create a pull request creating a new template based on you discoveries. |
This is a big limitation for us. After our VNET is used, we are no longer able to deploy changes to it because it is attempting to delete our subnets which are created in a separate deployment. |
I thinking we may just have to give up on ARM Templates at this rate and look at Terraform. Why can't ARM Templates work like CloudFormation Stacks and only update the properties defined. |
To expand further on this issue, subnets in itself can also have some additional properties set like associated NSG, Routetable and Service Endpoints. Let's take this example: I have a vnet, in this vnet I have 1 subnet with a routetable configured. Then I decide that I want to add an additional subnet. I create my ARM template. In this template I'm specifying the vnet, the existing subnet and the new subnet. All good. But, I forgot to specify in the ARM template that the existing subnet already has a routetable configured. If I really wanted to un-configure those type of properties, then I should pass some kind of $null value. The main point here, once set, don't modify unless specified. PS: I see the same behavior when I'm using PowerShell, with Set-AzureRmVirtualNetworkSubnetConfig. It just removes properties set like routetable and service endpoints. |
@rdtechie I disagree about your $null idea, that’d be wildly inconsistent
with other Azure services and ARM templates. I think the ideal solution
here is to treat subjects like a sub resource and not a property of a vnet
resource
…On Tue, Aug 21, 2018 at 3:34 PM Richard Diphoorn ***@***.***> wrote:
To expand further on this issue, subnets in itself can also have some
additional properties set like associated NSG, Routetable and Service
Endpoints.
Let's take this example: I have a vnet, in this vnet I have 1 subnet with
a routetable configured. Then I decide that I want to add an additional
subnet. I create my ARM template. In this template I'm specifying the vnet,
the existing subnet and the new subnet.
All good.
But, I forgot to specify in the ARM template that the existing subnet
already has a routetable configured.
I deploy the template, and boom, gone is my configured routetable on the
existing subnet.
This is really an issue, because the increment mode in my opinion should
only update those things that you specify, and not delete/remove them.
If I really wanted to un-configure those type of properties, then I should
pass some kind of $null value.
The main point here, once set, don't modify unless specified.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2786 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAD89N2To_YtxBkmZHqkoFKi1--Y14VYks5uTG7JgaJpZM4K4sjJ>
.
|
Agree @kensykora , but that wasn't my main point. The issue that should be prevented is that sub-resources should not be modified unless specified. |
@ramandhillon84 do we have a rough ETA on when this functionality will be available in other regions? |
@scottmckendry This change is going in the next release of Network RP, so expecting it to be everywhere in about 4 weeks. |
@ramandhillon84 I just tested using API version 2023-11-01 (in westeurope), and the good news is that it is now possible to deploy a Vnet resource without destroying any existing subnets, so thanks for that! It also works from Bicep/ARM templates, but you do need to make sure that you leave out the |
@anthony-c-martin, @alex-frankel, @jeskew Is this correct behaviour? I remember docs stating that if null is passed to a parameter in ARM, it's not being passed to RP. Seems it's not like that. This brings back the discussion on conditional properties where we agreed that passing null is sufficient to not send property to RP. |
|
Can confirm it now works using 'Microsoft.Network/virtualNetworks@2023-11-01' in westeurope, by omitting the subnets property 💪 |
Sadly I can confirm that. In CARML (and now AVM) we have several cases where we have to create compliacted |
I tried using both API-Version @2023-11-01 & @2023-09-01. But I am still getting the below error when I try to re-deploy my bicep templates. Error: - "details":[{"code":"InUseSubnetCannotBeUpdated","message":"Subnet snet-cosno is in use and cannot be updated." Context: - Code:- targetScope = 'resourceGroup'
param name string
param region string
param addressPrefix string
param subnets array
param tags object
param ddosProtectionPlanId string
var subnets = [
{
name: 'snet-cosno'
properties: {
addressPrefix: '<REDACTED>'
privateLinkServiceNetworkPolicies: 'Enabled'
privateEndpointNetworkPolicies: 'Disabled'
networkSecurityGroup: {
id: resourceId('<REDACTED>')
}
}
}
{
name: 'snet-kv'
properties: {
addressPrefix: '<REDACTED>'
privateLinkServiceNetworkPolicies: 'Enabled'
privateEndpointNetworkPolicies: 'Disabled'
networkSecurityGroup: {
id: resourceId('<REDACTED>')
}
}
]
resource virtualNetwork 'Microsoft.Network/virtualNetworks@2023-11-01' = {
name: name
location: region
properties: {
addressSpace: {
addressPrefixes: [addressPrefix]
}
enableDdosProtection: true
ddosProtectionPlan: {
id: ddosProtectionPlanId
}
subnets: subnets
}
tags: tags
}
output name string = virtualNetwork.name Any help, why re-deployment isn't idempotent here? why |
Can you open a separate item for this one in Bicep Discussions? You have both a |
@alex-frankel, sorry, I didn't get you when you say Even when I changed the var naming from |
I think it was just a sidenote that you cannot have both a parameter and variable sharing the same name as he happened to notice it in your example. No guarantees that this interpretation is correct, but I tried 😉 |
@AlexanderSehr is right. Please open a net-new issue in the bicep repo so that we don't randomize this thread. This thread also should be closed soon - I believe the update is fully rolled out, but am waiting on @ramandhillon84 to confirm. |
Hi @ramandhillon84 & @AlexanderSehr , thanks for the update. Do we have an approx ETA? Ta |
This has been rolled out everywhere - closing!!!! |
@AlexanderSehr a question to deploying subnets - I'm testing out this feature and I get Conflict errors: |
Hey @miqm, |
We have actually been getting this since today. We had already been using |
We also did not for the longest time. It hit us a while ago out of nowhere and I can only assume this comes down to a race condition. That being said, it's just speculation and we'd need somebody from the Product Group to have a definitive answer. I can at least say that since adding the |
@AlexanderSehr are you able to get in touch internally with PG team to clarify this? |
Not in a straight forward way I'm afraid as I'm not part of Engineering. But I can try to find a way. |
This was my solution as well. My theory is that you can't update more than one subnet at a time as from the API's standpoint it's a change with the Virtual Network resource not a separate and distinct subnet resource. Adding batch() to my template resolved it by serializing the updates to the subnets in my template. |
Sure, except this worked before. So this is a change in the rp. |
Hey all - this closed issue is not the right place for this discussion. Has anyone opened a support case for this? I can also share this directly with the Networking team. |
We have not, we assumed this is intentional so we also applied the batch size limitation. For us this is not a problem, just wanted to share our experience. |
Hi folks, this does not seem like it is related to the subnet change we made here. However, if this is a problem for you, i'd suggest opening a support case and our support engineers or Product team would take a look. |
I want to create subnets dynamically (separately from my VNET). I see there is a way to create a subnet inside an existing VNET, indicating that a subnet is a child resource of a VNET.
https://github.com/Azure/azure-quickstart-templates/blob/master/101-subnet-add-vnet-existing/azuredeploy.json
However, I can't find a way to re-deploy the VNET template without blowing away all subnets, because if I leave off the subnets property it deletes all the VNETs. Is there a way to deploy a VNET without including (and destroying) the subnets?
Here is an example template deploying a VNET (which destroys all subnets)
The text was updated successfully, but these errors were encountered: