-
Notifications
You must be signed in to change notification settings - Fork 745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strong typing for parameters and outputs #4158
Comments
I would definitely go for proposal "Refer to types by name". The param type and resource type is similar and is IMHO easier to use. |
Here are some of my thoughts on this. All options could coexist, I don’t see any issues with ability to exactly type type or use typeof or auto. As for the problem with union types I’d limit it to objects and enums only and throw error if user assigns other type. second concern is what about arrays? I feel that lots of use cases would be to use for loop on parameter to create resources with different values of same property. I’d also leave object/array type and define what object type is being expected after or use decorator for it. Also, with option 1 I feel we might have a discussion on how to keep in sync or simplify writing resource types, but eventually if we implement some type aliases they could be used here as well. |
In our company we're use a web application gateway module that has parameter arrays for each specific property (httpListeners, gatewayIPConfigurations, ...) So I'm not quite sure how you could discover what type is inside the array. But definitely consider this one in your discussions |
In the Bicep community call there was an ask for example of how customers currently use For our use cases custom defined types or type inference based on With inference based on property usage I mean that one Example: // vnetInput
// {
// name string
// range string
// subnets array auto
// subnets
// {
// name string
// range string
// }
// }
param vnetInput auto
resource vnet 'Microsoft.Network/virtualNetworks@2020-11-01' {
name: name
location: resourceGroup().location
properties: {
addressSpace: {
addressPrefixes: [
range
]
}
subnets: [for subnet in subnets: {
name: subnet.name
properties: {
addressPrefix: subnet.range
}
}]
}
} |
I'm not sure if it's way out of scope, but could it be an answer to implement some kind of json-schema-like language as a part of the input validation for objects? Seeing as the actual output of bicep is json, and json already have pretty extensive tooling for definitions, then why not build upon that? examples: @schema({
properties:{
one:{
type:string
enum:[
'value1'
'value2'
]
}
two:{
type:integer
minimum:1
maximum:2
}
}
required:[
'one'
]
})
param first object = {
one:'value1'
two:9
} you could have it be able to reference exisiting external schemas: //defer to schema.json 'inputname' property
@schema('./schema.json#/inputname')
param otherfileJsonSchema object
//defer to json schema file, on the internet
@schema('http://../schema.json')
param remotefileSchema object
//defer to schema.bicep 'inputname' parameter validation.
@schema('./schema.bicep#/inputname')
param otherfileBicepSchema object or possibly for simplicity provide it with a example of input you want, and a schema+example would be generated: @example({
name:'someitem'
range:'somerange'
subnets:[
{
name:'somename'
range:'somerange'
}
]
})
param example object as for resource input? @resource('Microsoft.DocumentDB/databaseAccounts@2021-10-15')
param resource object i don't know if this is something you have already thought about and dismissed in another discussion, but if its not, please concider it. |
@WithHolm - this ask seems similar/the same as what is described in #3723 and is related to this issue as well. Generally speaking, we need to continue to expand ways of getting value out of the type system. As you mention, the foundation is already there to do all sorts of type validation, but we need to provide a way of describing your own. |
Adding that this should work for variables as well since they haven't been mentioned within this issue yet. I landed on this issue when I wasn't allowed to strongly type the Desired - type checking: var functionProperties SiteProperties = {
serverFarmId: appServicePlan.id
httpsOnly: true
siteConfig: {
webSocketsEnabled: false
use32BitWorkerProcess: false
}
}
resource functionApp 'Microsoft.Web/sites@2021-03-01' = {
name: functionAppName
location: targetResourceGroup
kind: 'functionapp'
identity: {
type: 'SystemAssigned'
}
properties: functionProperties
}
resource functionAppSlot 'Microsoft.Web/sites/slots@2021-03-01' = {
parent: functionApp
name: functionAppSlotName
location: targetResourceGroup
kind: 'functionapp'
identity: {
type: 'SystemAssigned'
}
properties: functionProperties
} Actual - no type checking 😢 var functionProperties = {
serverFarmId: appServicePlan.id
httpsOnly: true
siteConfig: {
webSocketsEnabled: false
use32BitWorkerProcess: false
}
}
resource functionApp 'Microsoft.Web/sites@2021-03-01' = {
name: functionAppName
location: targetResourceGroup
kind: 'functionapp'
identity: {
type: 'SystemAssigned'
}
properties: functionProperties
}
resource functionAppSlot 'Microsoft.Web/sites/slots@2021-03-01' = {
parent: functionApp
name: functionAppSlotName
location: targetResourceGroup
kind: 'functionapp'
identity: {
type: 'SystemAssigned'
}
properties: functionProperties
} |
Adding some notes from #6624 I like the simplicity of referring to the types using the symbolic name. I like typeof rather than 'type' or 'of type' and I like referencing the symbolic names of defined resources. This should allow types to propagate up from modules. In addition to the coding experience, where ever possible the bicep build command should implement what it can in the ARM template - description, validations, enums and ranges are what I would expect. param storageAcctName string
param azRegion string
param skuName string typeof objStorAcct.skuName.name = 'Standard_LRS'
param accessTier string typeof objStorAcct.properties.accessTier = 'Hot'
param kind string typeof objStorAcct.kind = 'StorageV2'
resource objStorAcct 'Microsoft.Storage/storageAccounts@2021-01-01' = {
name: storageAcctName
location: azRegion
properties: {
accessTier: accessTier
}
sku: {
name: skuName
}
kind: kind
} I'd like to throw in the possibility of using a type decorator, which might be easier to parse, although it makes the code larger @type(objStorAcct.skuName.name)
param skuName string = 'Standard_LRS'
@type(objStorAcct.properties.accessTier )
param accessTier string = 'Hot'
@type(objStorAcct.kind )
param kind string = 'StorageV2' Vars and objects are key to this - build-up of objects is getting pretty common in Bicep files, although its going to get complicated: param storageAcctName string
param azRegion string
param skuName string typeof objStorAcct.skuName.name = 'Standard_LRS'
param accessTier string typeof objStorAcct.properties.accessTier = 'Hot'
param kind string typeof objStorAcct.kind = 'StorageV2'
//var with nested types. The accessTier property would have to validate the accessTier param
var props typeof objStorAcct.properties = {
accessTier: accessTier
}
resource objStorAcct 'Microsoft.Storage/storageAccounts@2021-01-01' = {
name: storageAcctName
location: azRegion
properties: props
sku: {
name: skuName
}
kind: kind
} I think global types would be helpful: param azRegion string typeof azure.region Finally, I think |
I like the idea of The only case would be the use of typeof which can be a build-in function to resolve the type from string or literal specification of the type:
|
Hi, is there any progress or decisions/design regarding this topic. It would be a gamechanger to have this implemented. |
We are definitely going to implement something to enable strong typing -- both completely custom and with the ability to reference subschemas from resource types (or possibly elsewhere) through a Don't have an ETA as we are still not closed on design, but we recognize that this is a serious limitation in the consumption of modules -- particularly from an external registry! |
This looks excellent, I am super excited! |
@jeskew - one more thing to consider while improving strong typing - would be nice to have ability to define a dictionary type, where key is string and the value is a custom type. then we can use |
I am trying to define a type in one module bicep file, that references a type declared in another module bicep file. VS isn't recognising it: /modules/foo.bicep type Foo = {
name: string
} /recipes/bar.bicep type Bar = {
first: Foo
another: string
} In the above example the |
@dazinator Types have to be defined in the template where they're used, but work on type sharing is tracked in #9311 |
After enabling custom types and getting everything to compile, when actually deploying to azure, I get these new errors:
Undoing my refactoring of custom types resolves this issue again. |
Is there an update on this? The user-defined types are very nice for params, but sometimes I'd be happy to leverage the existing resource type definitions. It seems sub-optimal to redefine custom types for those properties (even if/when we're able to import/export them): for example, the bicep compiler knows that param gatewayName string
@minLength(1)
param frontendHttpListeners array
@minLength(1)
param frontendIPConfigurations array
// ideally
// param frontendIPConfigurations ApplicationGatewayFrontendIPConfiguration[]
resource appGateway 'Microsoft.Network/applicationGateways@2022-09-01' = {
name: gatewayName
location: resourceGroup().location
properties: {
frontendIPConfigurations: frontendIPConfigurations
httpListeners: frontendHttpListeners
// ... etc
}
} |
@Marchelune That might be more of a separate issue than one strictly having to do with custom types. Correct me if I'm wrong, but I seem to recall that Application Gateway is much like a Virtual Network in that all the child resources must be present when the parent is created (necessitating that single large module) since an update to the parent from multiple files (modules) isn't feasible. While importing/exporting custom types to shuffle data around the requisite modules would be helpful, I'm pretty sure that effort would be blocked more by the "cannot update some types, must do full deployments at once" issue. |
@WhitWaldo you are pretty bang on, AGW is not put together in a modular way, so if you need to update one part of it, you need to do a full "replace" (ie PUT complete object) to do any updates. //nothing here is actual, just a example of how you could reference a 'built in' type
type AgwFrontend ref('Microsoft.Network/applicationGateways@2022-11-01',ApplicationGatewayFrontendIPConfiguration)
param frontend_clean AgwFrontend[]
param frontend_dirty {
id: 'string'
name: 'string'
properties: {
privateIPAddress: 'string'
privateIPAllocationMethod: 'string'
privateLinkConfiguration: {
id: 'string'
}
publicIPAddress: {
id: 'string'
}
subnet: {
id: 'string'
}
}
}[] |
@WhitWaldo I agree that it is not really custom types, given that those type already exist. //======
// application-gateway.bicep
param gatewayName string
@minLength(1)
param frontendHttpListeners array
@minLength(1)
param frontendIPConfigurations array
resource appGateway 'Microsoft.Network/applicationGateways@2022-09-01' = {
name: gatewayName
location: resourceGroup().location
properties: {
frontendIPConfigurations: frontendIPConfigurations
httpListeners: frontendHttpListeners
// ... etc
}
}
//======
// application-gateway.main.bicep
module routingRules './helpers/gateway-frontend-routing-rules-config.bicep' = {
name: '${gatewayName}-routing-rules'
params: {
foo: 'whatever params'
// etc...
}
}
module gatewayModule './application-gateway.bicep' = {
name: 'app-gateway-${deploymentSuffix}'
params: {
gatewayName: gatewayName
requestRoutingRules: routingRules.outputs.appGatewayRoutingRules
rewriteRuleSets: routingRules.outputs.appGatewayRewriteRuleSets
// etc....
}
}
//======
// helpers/gateway-frontend-routing-rules-config.bicep
param foo string
output appGatewayRewriteRuleSets array = []
output appGatewayRoutingRules array = [for i in range(1,10): {
id: 'rule-${foo}-${i}'
// etc, you can have whatever complexity here
}] The "helper" module has no actual resources, it's really just a function that returns arrays/objects. You are right that it is deployed all at once - but that would still be the case here, whilst allowing cleaner code. At least on my side, with 6/7 listener/rule/backend combinations that must be parametrised per environment, the app gateway configuration via bicep becomes a barely maintainable 600 lines file. Extracting everything to a helpers makes it much clearer, but without strong types in those helpers, I am at risk of typos/missing properties (a risk that already exists when using intermediary |
When I spin up an App Gateway today, I'm mostly using custom types as a stand-in so I can get validation that I'm at least typing the names of the parameter values correctly, verify on the caller side that I'm passing the right stuff in where I intend and get strong typing for use in lambda and other functions. For example, we lack an import/export at the moment, so I've got these types at the start of my AppGw module: type ipAddressProperties = {
@description('The zones in which the IP address should be deployed')
ipZones: string[]
@description('Whether or not the IP address can be static (true) or not (false)')
isStaticIp: bool
@description('Whether the IP address is a standard (true) or basic (false) SKU')
isStandardIpSku: bool
@description('Whether or not the IP address is public (true) or not (false)')
isPublicIp: bool
}
@description('Used to identify the details of a specific certificate')
type certificateMapping = {
@description('The subject value of the certificate')
subject: string
@description('The name of the secret in the Key Vault')
secretName: string
@description('The thumbprint of the certificate')
thumbprint: string
@description('The identifier of the Key Vault instance')
keyVault: keyVaultIdentifier
}
@description('Used to identify a Key Vault and where it\'s deployed to')
type keyVaultIdentifier = {
@description('The name of the Key Vault')
name: string
@description('The ID of the subscription the Key Vault is associated with')
subscriptionId: string
@description('The name of the resource group the Key Vault is associated with')
resourceGroupName: string
}
type uaiDetail = {
@description('The name of the user-assigned identity that will execute the deployment script')
name: string
@description('The ID of the subscription that the user-assigned identity is associated with that will execute the deployment script')
subscriptionId: string
@description('The name of the resource group that the user-assigned identity is associated with that will execute the deployment script')
resourceGroupName: string
} And those are then followed by a wall of parameters that input those and still other parameters (including, note the use of the keyVaultIdentifier type in the certificateMapping so I can identify the key vault I'm pulling each from in case they differ: @description('The IP address aspects to assign to the Application Gateway')
param IpAddressProperties ipAddressProperties
@description('The various subjects for which certificates should be secured for both Front Door, Application Gateway and for deployment to the SF cluster')
param CertificateSubjects certificateMapping[]
@description('The details of the user-assigned identity that will execute the deployment script')
param UaiDetails uaiDetail And later on, continued use of the values out of these custom types: //Maintains a reference to the certificate in the Key Vault for this subject
resource Secret 'Microsoft.KeyVault/vaults/secrets@2022-07-01' existing = [for (c, i) in CertificateSubjects: {
name: '${c.keyVault.name}/${c.secretName}'
scope: resourceGroup(c.keyVault.subscriptionId, c.keyVault.resourceGroupName)
}]
var endpointSecretName = [for (endpoint, i) in AllEndpoints: {
secretName: first(map(filter(CertificateSubjects, cert => startsWith(cert.secretName, '*.') ? endsWith(endpoint.targetDomain, replace(cert.secretName, '*.', '')) : endpoint == cert.subject), c => c.secretName))
}] Then, later on in the block of AppGw resource logic, I can trivially reference: //...
sslCertificates: [for (c, i) in CertificateSubjects: {
name: c.secretName
properties: {
keyVaultSecretId: Secret[i].id
}
}] ...among all the other types. I agree - it'd be great if I could use a parent/child syntax of some sort to at least artificially move up and down the resource dependency tree (asked about this in a similar vein for load balancer a while ago in #724 ) with some way of having Bicep work out how to chain it all together for a single PUT request at deployment, but simplify the creation of these elaborate resources, but again.. I think that's a separate issue to this one. |
@Marchelune It occurs to me based on re-reading your last comment that you might not be aware of the preview custom types feature that I use above: https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/user-defined-data-types There's no import/export support yet (back and forth about it in #9311 ) but based on the last community call, that'll likely be showing up soon. |
@WhitWaldo thanks for your extensive feedback! I think we are very much aligned on that, in fact I did try user-defined types, but faced 2 issues:
That said, I do agree that user-defined types are a good intermediary solution, although I'd still personally prefer, as suggested in this proposal from @rynowak, something like param frontendIPConfigurations type 'Microsoft.Network/applicationGateways@2022-09-01#ApplicationGatewayFrontendIPConfiguration'[]
|
@Marchelune -- I believe we will get to what you are looking for with the planned |
Resolves #4158 This PR also updates the TemplateWriter to target a non-experimental ARM languageVersion that allows type definitions (2.0). Templates using experimental ARM features (e.g., extensibility and asserts) now target the 2.1-experimental language version. The languageVersion change entailed a large number of baseline changes. These are included in this PR in a separate commit, and I'll go through and flag any unexpected baseline change. ###### Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/Azure/bicep/pull/11461&drop=dogfoodAlpha
Resolves #4158 This PR also updates the TemplateWriter to target a non-experimental ARM languageVersion that allows type definitions (2.0). Templates using experimental ARM features (e.g., extensibility and asserts) now target the 2.1-experimental language version. The languageVersion change entailed a large number of baseline changes. These are included in this PR in a separate commit, and I'll go through and flag any unexpected baseline change. codeflow:open?pullrequest=https://github.com/Azure/bicep/pull/11461&drop=dogfoodAlpha
Resolves #4158 This PR also updates the TemplateWriter to target a non-experimental ARM languageVersion that allows type definitions (2.0). Templates using experimental ARM features (e.g., extensibility and asserts) now target the 2.1-experimental language version. The languageVersion change entailed a large number of baseline changes. These are included in this PR in a separate commit, and I'll go through and flag any unexpected baseline change. codeflow:open?pullrequest=https://github.com/Azure/bicep/pull/11461&drop=dogfoodAlpha
Is your feature request related to a problem? Please describe.
For parameters and outputs the set of possible types that code can declare is very limited (primitives, array, and object). When the type of a module parameter is
object
you don't get the same type safety guarantees that are normally possible in bicep. Having the ability to specify a more specific type thanobject
would add type-safety for complex parameters and outputs in modules.Here's a motivating example:
Load balancers can be large and complex. If you want to parameterize a load balancer with modules, you have the choice to either forego type checking, or write a flat list of parameters that get combined into an object.
note: There is already a proposal for strongly-typed resources as parameters/outputs. This is about objects that are not resources.
Describe the solution you'd like
I'd like the ability to specify or infer a more specific type so that the parameters and outputs can be type-checked. I've discussed a few different options with the team, and want to gather more feedback. It's possible that more than one of these solutions are desirable given that they optimize for different scenarios.
Proposal: Refer to types by name
The simplest idea to understand is being able to specify the type using the named definitions that the compiler already knows. These are based on the OpenAPI descriptions used to generate Bicep's type definitions. In this example the type of
rule
is determined by looking up the provided type string against the resource type definitions.The part after the
#
is the type name, which is looked up in the context of the provided resource type. Failure to find the specified type would be a warning the failure to find a declared resource type.To make this work we'd want to provide completion for the part after
#
. So the user can get completion for the whole string in 3 parts.note: the syntax shown here is chosen to be similar/consistent with this proposal for type-specifiers for resources.
Proposal: Add a typeof specifier
This is similar to
typeof
type operator in TypeScript. In this example that type of therule
parameter is specify by type checking the provided expression.The expression passed to
typeof
could be any expression (in theory) and could be combined with other type-specifier constructs as necessary. The expression is not evaluated, it is only type-checked.note: the TypeScript
typeof
type operator is limited to property access and indexing. We probably need a way to specify that we want the element type of an array. Typescript usestypeof MyArray[number]
, which would seem foreign in bicep. In this case I filled that in as[*]
but it needs more thought.typeof
a resource created in a loop might be confusing compared to other options.string | int
. There's no way to encode this in ARM-JSON today.Proposal: Type inference via target-typing
This is similar to type inference in where it applies in TypeScript or some functional languages. Similar to (but more complex) target-typed
new()
in C#. In this example the typerule
has its type inferred based on where it appears.note: This really optimizes for the use-case where a parameter is used once, or used in the same context. Lots of complex scenarios are possible when type inference gets involved.
any
in the worse case.string | int
. There's no way to encode this in ARM-JSON today.The text was updated successfully, but these errors were encountered: